2024-09-14T22:22:24.169 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-14T22:22:24.185 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-14T22:22:24.269 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904719 branch: squid description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writearound} email: ceph-qa@ceph.com first_in_suite: false job_id: '7904719' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: squid ceph: conf: client: rbd cache: true rbd cache policy: writearound global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: zstd bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: ecab7be58963a5bd03910cd73eef66405d7fdd50 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: ecab7be58963a5bd03910cd73eef66405d7fdd50 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: ecab7be58963a5bd03910cd73eef66405d7fdd50 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 4271 sha1: ecab7be58963a5bd03910cd73eef66405d7fdd50 sleep_before_teardown: 0 subset: 97/128 suite: rbd suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_ecab7be58963a5bd03910cd73eef66405d7fdd50/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: ecab7be58963a5bd03910cd73eef66405d7fdd50 targets: smithi001.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCR0nib8SBVTiYN6fbNaFSeWtuJX68mc9fQqPDMtsUK3EZnmiq8jqxLeiJVDpCo3VXVQRWgNkwq3386ra8fU4zg= smithi176.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKgtHjyzrLcUMjK57Ts2qKmxNrHdr33LIQFAb0TmaqkgGVm1Wt69/45cHSFBegPAq1DSuamUVF/g7grXSFST+kM= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-14_21:16:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-14T22:22:24.269 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_ecab7be58963a5bd03910cd73eef66405d7fdd50/qa; will attempt to use it 2024-09-14T22:22:24.270 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_ecab7be58963a5bd03910cd73eef66405d7fdd50/qa/tasks 2024-09-14T22:22:24.270 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-14T22:22:24.272 INFO:teuthology.task.internal:Checking packages... 2024-09-14T22:22:24.292 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'ecab7be58963a5bd03910cd73eef66405d7fdd50' 2024-09-14T22:22:24.292 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-14T22:22:24.292 INFO:teuthology.packaging:ref: None 2024-09-14T22:22:24.292 INFO:teuthology.packaging:tag: None 2024-09-14T22:22:24.292 INFO:teuthology.packaging:branch: squid 2024-09-14T22:22:24.292 INFO:teuthology.packaging:sha1: ecab7be58963a5bd03910cd73eef66405d7fdd50 2024-09-14T22:22:24.292 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=squid 2024-09-14T22:22:24.484 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-324-gecab7be5-1jammy 2024-09-14T22:22:24.486 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-14T22:22:24.492 INFO:teuthology.task.internal:no buildpackages task found 2024-09-14T22:22:24.492 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-14T22:22:24.506 INFO:teuthology.task.internal:Saving configuration 2024-09-14T22:22:24.517 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-14T22:22:24.523 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-14T22:22:24.548 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi001.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904719', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-14 22:16:55.680504', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINRrNcrx/ECsQB1oB/Kp9wYxuVY5e+WUdgr+psIomDRA'} 2024-09-14T22:22:24.567 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi176.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904719', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-14 22:16:55.682242', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBEeupxg9g9jS4+s33d4/qRBcw2r6D+B5j+Zl3AI/z3T'} 2024-09-14T22:22:24.568 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-14T22:22:24.573 INFO:teuthology.task.internal:roles: ubuntu@smithi001.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-14T22:22:24.573 INFO:teuthology.task.internal:roles: ubuntu@smithi176.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-14T22:22:24.573 INFO:teuthology.run_tasks:Running task console_log... 2024-09-14T22:22:24.631 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd61d76b370>, signals=[15]) 2024-09-14T22:22:24.631 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-14T22:22:24.636 INFO:teuthology.task.internal:Opening connections... 2024-09-14T22:22:24.637 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi001.front.sepia.ceph.com 2024-09-14T22:22:24.638 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T22:22:24.720 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi176.front.sepia.ceph.com 2024-09-14T22:22:24.721 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T22:22:24.799 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-14T22:22:24.804 DEBUG:teuthology.orchestra.run.smithi001:> uname -m 2024-09-14T22:22:24.809 INFO:teuthology.orchestra.run.smithi001.stdout:x86_64 2024-09-14T22:22:24.810 DEBUG:teuthology.orchestra.run.smithi001:> cat /etc/os-release 2024-09-14T22:22:24.856 INFO:teuthology.orchestra.run.smithi001.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-14T22:22:24.856 INFO:teuthology.orchestra.run.smithi001.stdout:NAME="Ubuntu" 2024-09-14T22:22:24.856 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION_ID="22.04" 2024-09-14T22:22:24.856 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-14T22:22:24.856 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION_CODENAME=jammy 2024-09-14T22:22:24.856 INFO:teuthology.orchestra.run.smithi001.stdout:ID=ubuntu 2024-09-14T22:22:24.856 INFO:teuthology.orchestra.run.smithi001.stdout:ID_LIKE=debian 2024-09-14T22:22:24.856 INFO:teuthology.orchestra.run.smithi001.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-14T22:22:24.856 INFO:teuthology.orchestra.run.smithi001.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-14T22:22:24.857 INFO:teuthology.orchestra.run.smithi001.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-14T22:22:24.857 INFO:teuthology.orchestra.run.smithi001.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-14T22:22:24.857 INFO:teuthology.orchestra.run.smithi001.stdout:UBUNTU_CODENAME=jammy 2024-09-14T22:22:24.857 INFO:teuthology.lock.ops:Updating smithi001.front.sepia.ceph.com on lock server 2024-09-14T22:22:24.880 DEBUG:teuthology.orchestra.run.smithi176:> uname -m 2024-09-14T22:22:24.885 INFO:teuthology.orchestra.run.smithi176.stdout:x86_64 2024-09-14T22:22:24.885 DEBUG:teuthology.orchestra.run.smithi176:> cat /etc/os-release 2024-09-14T22:22:24.930 INFO:teuthology.orchestra.run.smithi176.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-14T22:22:24.930 INFO:teuthology.orchestra.run.smithi176.stdout:NAME="Ubuntu" 2024-09-14T22:22:24.930 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION_ID="22.04" 2024-09-14T22:22:24.930 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-14T22:22:24.930 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION_CODENAME=jammy 2024-09-14T22:22:24.930 INFO:teuthology.orchestra.run.smithi176.stdout:ID=ubuntu 2024-09-14T22:22:24.930 INFO:teuthology.orchestra.run.smithi176.stdout:ID_LIKE=debian 2024-09-14T22:22:24.930 INFO:teuthology.orchestra.run.smithi176.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-14T22:22:24.930 INFO:teuthology.orchestra.run.smithi176.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-14T22:22:24.930 INFO:teuthology.orchestra.run.smithi176.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-14T22:22:24.930 INFO:teuthology.orchestra.run.smithi176.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-14T22:22:24.930 INFO:teuthology.orchestra.run.smithi176.stdout:UBUNTU_CODENAME=jammy 2024-09-14T22:22:24.931 INFO:teuthology.lock.ops:Updating smithi176.front.sepia.ceph.com on lock server 2024-09-14T22:22:24.963 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-14T22:22:24.971 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-14T22:22:24.978 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-14T22:22:24.979 DEBUG:teuthology.orchestra.run.smithi001:> test '!' -e /home/ubuntu/cephtest 2024-09-14T22:22:24.981 DEBUG:teuthology.orchestra.run.smithi176:> test '!' -e /home/ubuntu/cephtest 2024-09-14T22:22:24.985 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-14T22:22:24.989 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-14T22:22:24.990 DEBUG:teuthology.orchestra.run.smithi001:> test -z $(ls -A /var/lib/ceph) 2024-09-14T22:22:25.030 DEBUG:teuthology.orchestra.run.smithi176:> test -z $(ls -A /var/lib/ceph) 2024-09-14T22:22:25.072 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-14T22:22:25.121 INFO:teuthology.run_tasks:Running task kernel... 2024-09-14T22:22:25.134 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-14T22:22:25.134 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-14T22:22:25.134 DEBUG:teuthology.orchestra.run.smithi001:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-14T22:22:25.135 DEBUG:teuthology.orchestra.run.smithi176:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-14T22:22:25.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T22:22:25.140 DEBUG:teuthology.orchestra.run.smithi001:> uname -r 2024-09-14T22:22:25.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T22:22:25.141 DEBUG:teuthology.orchestra.run.smithi176:> uname -r 2024-09-14T22:22:25.185 INFO:teuthology.orchestra.run.smithi001.stdout:5.15.0-56-generic 2024-09-14T22:22:25.185 INFO:teuthology.task.kernel:Running kernel on smithi001: 5.15.0-56-generic 2024-09-14T22:22:25.185 DEBUG:teuthology.orchestra.run.smithi001:> sudo apt-get clean 2024-09-14T22:22:25.187 INFO:teuthology.orchestra.run.smithi176.stdout:5.15.0-56-generic 2024-09-14T22:22:25.187 INFO:teuthology.task.kernel:Running kernel on smithi176: 5.15.0-56-generic 2024-09-14T22:22:25.187 DEBUG:teuthology.orchestra.run.smithi176:> sudo apt-get clean 2024-09-14T22:22:25.306 DEBUG:teuthology.orchestra.run.smithi001:> sudo apt-get update 2024-09-14T22:22:25.328 DEBUG:teuthology.orchestra.run.smithi176:> sudo apt-get update 2024-09-14T22:22:25.456 INFO:teuthology.orchestra.run.smithi176.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-14T22:22:25.464 INFO:teuthology.orchestra.run.smithi001.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-14T22:22:25.464 INFO:teuthology.orchestra.run.smithi001.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-14T22:22:25.464 INFO:teuthology.orchestra.run.smithi001.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-14T22:22:25.466 INFO:teuthology.orchestra.run.smithi176.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-14T22:22:25.529 INFO:teuthology.orchestra.run.smithi001.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-14T22:22:25.533 INFO:teuthology.orchestra.run.smithi176.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-14T22:22:25.568 INFO:teuthology.orchestra.run.smithi176.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-14T22:22:25.671 INFO:teuthology.orchestra.run.smithi176.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,023 kB] 2024-09-14T22:22:25.717 INFO:teuthology.orchestra.run.smithi001.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-14T22:22:25.770 INFO:teuthology.orchestra.run.smithi176.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-14T22:22:25.775 INFO:teuthology.orchestra.run.smithi176.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-14T22:22:25.778 INFO:teuthology.orchestra.run.smithi176.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-14T22:22:25.778 INFO:teuthology.orchestra.run.smithi176.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-14T22:22:25.778 INFO:teuthology.orchestra.run.smithi176.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-14T22:22:25.796 INFO:teuthology.orchestra.run.smithi176.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-14T22:22:25.799 INFO:teuthology.orchestra.run.smithi001.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,023 kB] 2024-09-14T22:22:25.818 INFO:teuthology.orchestra.run.smithi001.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-14T22:22:25.818 INFO:teuthology.orchestra.run.smithi176.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-14T22:22:25.818 INFO:teuthology.orchestra.run.smithi176.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-14T22:22:25.823 INFO:teuthology.orchestra.run.smithi176.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-14T22:22:25.830 INFO:teuthology.orchestra.run.smithi176.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-14T22:22:25.832 INFO:teuthology.orchestra.run.smithi176.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-14T22:22:25.832 INFO:teuthology.orchestra.run.smithi176.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-14T22:22:25.833 INFO:teuthology.orchestra.run.smithi176.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-14T22:22:25.833 INFO:teuthology.orchestra.run.smithi176.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-14T22:22:25.838 INFO:teuthology.orchestra.run.smithi176.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-14T22:22:25.860 INFO:teuthology.orchestra.run.smithi176.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-14T22:22:25.861 INFO:teuthology.orchestra.run.smithi176.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-14T22:22:25.862 INFO:teuthology.orchestra.run.smithi001.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-14T22:22:25.862 INFO:teuthology.orchestra.run.smithi176.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-14T22:22:25.862 INFO:teuthology.orchestra.run.smithi176.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-14T22:22:25.862 INFO:teuthology.orchestra.run.smithi176.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-14T22:22:25.863 INFO:teuthology.orchestra.run.smithi176.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-14T22:22:25.863 INFO:teuthology.orchestra.run.smithi176.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-14T22:22:25.863 INFO:teuthology.orchestra.run.smithi176.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-14T22:22:25.864 INFO:teuthology.orchestra.run.smithi001.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-14T22:22:25.865 INFO:teuthology.orchestra.run.smithi001.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-14T22:22:25.865 INFO:teuthology.orchestra.run.smithi001.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-14T22:22:25.883 INFO:teuthology.orchestra.run.smithi001.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-14T22:22:25.884 INFO:teuthology.orchestra.run.smithi001.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-14T22:22:25.887 INFO:teuthology.orchestra.run.smithi001.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-14T22:22:25.887 INFO:teuthology.orchestra.run.smithi001.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-14T22:22:25.889 INFO:teuthology.orchestra.run.smithi001.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-14T22:22:25.892 INFO:teuthology.orchestra.run.smithi001.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-14T22:22:25.892 INFO:teuthology.orchestra.run.smithi001.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-14T22:22:25.913 INFO:teuthology.orchestra.run.smithi001.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-14T22:22:25.914 INFO:teuthology.orchestra.run.smithi001.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-14T22:22:25.914 INFO:teuthology.orchestra.run.smithi001.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-14T22:22:25.918 INFO:teuthology.orchestra.run.smithi001.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-14T22:22:25.918 INFO:teuthology.orchestra.run.smithi001.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-14T22:22:25.918 INFO:teuthology.orchestra.run.smithi001.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-14T22:22:25.920 INFO:teuthology.orchestra.run.smithi001.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-14T22:22:25.920 INFO:teuthology.orchestra.run.smithi001.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-14T22:22:25.934 INFO:teuthology.orchestra.run.smithi001.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-14T22:22:25.934 INFO:teuthology.orchestra.run.smithi001.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-14T22:22:25.934 INFO:teuthology.orchestra.run.smithi001.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-14T22:22:25.934 INFO:teuthology.orchestra.run.smithi001.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-14T22:22:25.939 INFO:teuthology.orchestra.run.smithi001.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-14T22:22:25.941 INFO:teuthology.orchestra.run.smithi001.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-14T22:22:25.941 INFO:teuthology.orchestra.run.smithi001.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-14T22:22:25.942 INFO:teuthology.orchestra.run.smithi001.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-14T22:22:25.942 INFO:teuthology.orchestra.run.smithi001.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-14T22:22:25.943 INFO:teuthology.orchestra.run.smithi001.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-14T22:22:25.943 INFO:teuthology.orchestra.run.smithi001.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-14T22:22:25.944 INFO:teuthology.orchestra.run.smithi001.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-14T22:22:25.959 INFO:teuthology.orchestra.run.smithi001.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-14T22:22:25.959 INFO:teuthology.orchestra.run.smithi001.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-14T22:22:25.960 INFO:teuthology.orchestra.run.smithi001.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-14T22:22:25.960 INFO:teuthology.orchestra.run.smithi001.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-14T22:22:25.960 INFO:teuthology.orchestra.run.smithi001.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-14T22:22:25.961 INFO:teuthology.orchestra.run.smithi001.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-14T22:22:26.008 INFO:teuthology.orchestra.run.smithi176.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-14T22:22:26.431 INFO:teuthology.orchestra.run.smithi176.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-14T22:22:26.447 INFO:teuthology.orchestra.run.smithi176.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-14T22:22:26.456 INFO:teuthology.orchestra.run.smithi176.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-14T22:22:26.456 INFO:teuthology.orchestra.run.smithi176.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-14T22:22:26.553 INFO:teuthology.orchestra.run.smithi176.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-14T22:22:26.554 INFO:teuthology.orchestra.run.smithi176.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-14T22:22:26.565 INFO:teuthology.orchestra.run.smithi176.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-14T22:22:26.565 INFO:teuthology.orchestra.run.smithi176.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-14T22:22:26.588 INFO:teuthology.orchestra.run.smithi176.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-14T22:22:26.610 INFO:teuthology.orchestra.run.smithi176.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-14T22:22:26.614 INFO:teuthology.orchestra.run.smithi176.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-14T22:22:26.617 INFO:teuthology.orchestra.run.smithi176.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-14T22:22:26.617 INFO:teuthology.orchestra.run.smithi176.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-14T22:22:26.693 INFO:teuthology.orchestra.run.smithi176.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-14T22:22:26.693 INFO:teuthology.orchestra.run.smithi176.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-14T22:22:32.077 INFO:teuthology.orchestra.run.smithi176.stdout:Fetched 16.0 MB in 2s (7,438 kB/s) 2024-09-14T22:22:32.522 INFO:teuthology.orchestra.run.smithi001.stdout:Fetched 16.0 MB in 2s (7,008 kB/s) 2024-09-14T22:22:33.471 INFO:teuthology.orchestra.run.smithi176.stdout:Reading package lists... 2024-09-14T22:22:33.489 DEBUG:teuthology.orchestra.run.smithi176:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-14T22:22:33.552 INFO:teuthology.orchestra.run.smithi176.stdout:Reading package lists... 2024-09-14T22:22:33.761 INFO:teuthology.orchestra.run.smithi176.stdout:Building dependency tree... 2024-09-14T22:22:33.762 INFO:teuthology.orchestra.run.smithi176.stdout:Reading state information... 2024-09-14T22:22:33.939 INFO:teuthology.orchestra.run.smithi176.stdout:The following packages were automatically installed and are no longer required: 2024-09-14T22:22:33.939 INFO:teuthology.orchestra.run.smithi176.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-14T22:22:33.939 INFO:teuthology.orchestra.run.smithi176.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-14T22:22:33.939 INFO:teuthology.orchestra.run.smithi176.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-14T22:22:33.940 INFO:teuthology.orchestra.run.smithi176.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-14T22:22:33.940 INFO:teuthology.orchestra.run.smithi176.stdout: python2.7-minimal 2024-09-14T22:22:33.940 INFO:teuthology.orchestra.run.smithi176.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-14T22:22:33.941 INFO:teuthology.orchestra.run.smithi176.stdout:The following additional packages will be installed: 2024-09-14T22:22:33.941 INFO:teuthology.orchestra.run.smithi176.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-14T22:22:33.941 INFO:teuthology.orchestra.run.smithi176.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-14T22:22:33.942 INFO:teuthology.orchestra.run.smithi176.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-14T22:22:33.943 INFO:teuthology.orchestra.run.smithi176.stdout:Suggested packages: 2024-09-14T22:22:33.943 INFO:teuthology.orchestra.run.smithi176.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-14T22:22:33.943 INFO:teuthology.orchestra.run.smithi176.stdout:Recommended packages: 2024-09-14T22:22:33.943 INFO:teuthology.orchestra.run.smithi176.stdout: thermald 2024-09-14T22:22:33.964 INFO:teuthology.orchestra.run.smithi001.stdout:Reading package lists... 2024-09-14T22:22:33.982 DEBUG:teuthology.orchestra.run.smithi001:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-14T22:22:33.995 INFO:teuthology.orchestra.run.smithi176.stdout:The following NEW packages will be installed: 2024-09-14T22:22:33.995 INFO:teuthology.orchestra.run.smithi176.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-14T22:22:33.995 INFO:teuthology.orchestra.run.smithi176.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-14T22:22:33.995 INFO:teuthology.orchestra.run.smithi176.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-14T22:22:33.997 INFO:teuthology.orchestra.run.smithi176.stdout:The following packages will be upgraded: 2024-09-14T22:22:33.997 INFO:teuthology.orchestra.run.smithi176.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-14T22:22:34.036 INFO:teuthology.orchestra.run.smithi001.stdout:Reading package lists... 2024-09-14T22:22:34.175 INFO:teuthology.orchestra.run.smithi176.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-14T22:22:34.175 INFO:teuthology.orchestra.run.smithi176.stdout:Need to get 113 MB of archives. 2024-09-14T22:22:34.175 INFO:teuthology.orchestra.run.smithi176.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-14T22:22:34.175 INFO:teuthology.orchestra.run.smithi176.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-14T22:22:34.225 INFO:teuthology.orchestra.run.smithi001.stdout:Building dependency tree... 2024-09-14T22:22:34.226 INFO:teuthology.orchestra.run.smithi001.stdout:Reading state information... 2024-09-14T22:22:34.397 INFO:teuthology.orchestra.run.smithi001.stdout:The following packages were automatically installed and are no longer required: 2024-09-14T22:22:34.397 INFO:teuthology.orchestra.run.smithi001.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-14T22:22:34.397 INFO:teuthology.orchestra.run.smithi001.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-14T22:22:34.398 INFO:teuthology.orchestra.run.smithi001.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-14T22:22:34.398 INFO:teuthology.orchestra.run.smithi001.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-14T22:22:34.398 INFO:teuthology.orchestra.run.smithi001.stdout: python2.7-minimal 2024-09-14T22:22:34.398 INFO:teuthology.orchestra.run.smithi001.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-14T22:22:34.399 INFO:teuthology.orchestra.run.smithi001.stdout:The following additional packages will be installed: 2024-09-14T22:22:34.399 INFO:teuthology.orchestra.run.smithi001.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-14T22:22:34.399 INFO:teuthology.orchestra.run.smithi001.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-14T22:22:34.400 INFO:teuthology.orchestra.run.smithi001.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-14T22:22:34.401 INFO:teuthology.orchestra.run.smithi001.stdout:Suggested packages: 2024-09-14T22:22:34.401 INFO:teuthology.orchestra.run.smithi001.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-14T22:22:34.401 INFO:teuthology.orchestra.run.smithi001.stdout:Recommended packages: 2024-09-14T22:22:34.401 INFO:teuthology.orchestra.run.smithi001.stdout: thermald 2024-09-14T22:22:34.447 INFO:teuthology.orchestra.run.smithi001.stdout:The following NEW packages will be installed: 2024-09-14T22:22:34.447 INFO:teuthology.orchestra.run.smithi001.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-14T22:22:34.447 INFO:teuthology.orchestra.run.smithi001.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-14T22:22:34.447 INFO:teuthology.orchestra.run.smithi001.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-14T22:22:34.448 INFO:teuthology.orchestra.run.smithi001.stdout:The following packages will be upgraded: 2024-09-14T22:22:34.449 INFO:teuthology.orchestra.run.smithi001.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-14T22:22:34.499 INFO:teuthology.orchestra.run.smithi001.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-14T22:22:34.500 INFO:teuthology.orchestra.run.smithi001.stdout:Need to get 113 MB of archives. 2024-09-14T22:22:34.500 INFO:teuthology.orchestra.run.smithi001.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-14T22:22:34.500 INFO:teuthology.orchestra.run.smithi001.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-14T22:22:34.893 INFO:teuthology.orchestra.run.smithi001.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-14T22:22:34.989 INFO:teuthology.orchestra.run.smithi001.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-14T22:22:35.570 INFO:teuthology.orchestra.run.smithi176.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-14T22:22:35.834 INFO:teuthology.orchestra.run.smithi001.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-14T22:22:35.834 INFO:teuthology.orchestra.run.smithi001.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-14T22:22:35.834 INFO:teuthology.orchestra.run.smithi001.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-14T22:22:35.966 INFO:teuthology.orchestra.run.smithi001.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-14T22:22:35.978 INFO:teuthology.orchestra.run.smithi176.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-14T22:22:35.988 INFO:teuthology.orchestra.run.smithi001.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-14T22:22:36.390 INFO:teuthology.orchestra.run.smithi001.stdout:Fetched 113 MB in 2s (73.9 MB/s) 2024-09-14T22:22:36.546 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-14T22:22:38.123 INFO:teuthology.orchestra.run.smithi176.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-14T22:22:38.123 INFO:teuthology.orchestra.run.smithi176.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-14T22:22:38.124 INFO:teuthology.orchestra.run.smithi176.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-14T22:22:38.550 INFO:teuthology.orchestra.run.smithi001.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-14T22:22:38.554 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T22:22:38.634 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T22:22:38.647 INFO:teuthology.orchestra.run.smithi176.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-14T22:22:38.720 INFO:teuthology.orchestra.run.smithi176.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-14T22:22:39.116 INFO:teuthology.orchestra.run.smithi176.stdout:Fetched 113 MB in 5s (24.0 MB/s) 2024-09-14T22:22:39.284 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-14T22:22:41.204 INFO:teuthology.orchestra.run.smithi176.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-14T22:22:41.209 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T22:22:41.272 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T22:22:41.277 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-14T22:22:41.299 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T22:22:41.398 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T22:22:41.872 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-14T22:22:41.893 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T22:22:41.922 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T22:22:43.789 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-14T22:22:43.810 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T22:22:43.900 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T22:22:44.400 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-14T22:22:44.421 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T22:22:44.458 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T22:22:48.290 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-14T22:22:48.398 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-14T22:22:48.715 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-14T22:22:48.843 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-14T22:22:49.179 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-14T22:22:49.202 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-14T22:22:49.246 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-14T22:22:50.688 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-14T22:22:50.790 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-14T22:22:51.123 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-14T22:22:51.252 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-14T22:22:51.571 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-14T22:22:51.593 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-14T22:22:51.663 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-14T22:22:54.836 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-14T22:22:54.873 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T22:22:54.911 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T22:22:57.257 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-14T22:22:57.295 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T22:22:57.331 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T22:22:57.863 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-14T22:22:57.988 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-14T22:22:58.330 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-14T22:22:58.435 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T22:22:58.580 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-14T22:22:58.705 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T22:22:59.936 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-14T22:23:00.055 INFO:teuthology.orchestra.run.smithi001.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-14T22:23:00.055 INFO:teuthology.orchestra.run.smithi001.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-14T22:23:00.055 INFO:teuthology.orchestra.run.smithi001.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-14T22:23:00.055 INFO:teuthology.orchestra.run.smithi001.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-14T22:23:00.059 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-14T22:23:00.216 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T22:23:00.383 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-14T22:23:00.505 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T22:23:00.631 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-14T22:23:00.758 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T22:23:02.167 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T22:23:02.345 INFO:teuthology.orchestra.run.smithi176.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-14T22:23:02.346 INFO:teuthology.orchestra.run.smithi176.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-14T22:23:02.346 INFO:teuthology.orchestra.run.smithi176.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-14T22:23:02.346 INFO:teuthology.orchestra.run.smithi176.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-14T22:23:02.494 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T22:23:03.594 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-14T22:23:03.728 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-14T22:23:03.871 INFO:teuthology.orchestra.run.smithi001.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T22:23:03.895 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T22:23:03.938 INFO:teuthology.orchestra.run.smithi001.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-14T22:23:03.938 INFO:teuthology.orchestra.run.smithi001.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-14T22:23:05.388 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-14T22:23:05.497 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-14T22:23:05.599 INFO:teuthology.orchestra.run.smithi176.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T22:23:05.655 INFO:teuthology.orchestra.run.smithi176.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-14T22:23:05.655 INFO:teuthology.orchestra.run.smithi176.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-14T22:23:18.402 INFO:teuthology.orchestra.run.smithi001.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-14T22:23:18.402 INFO:teuthology.orchestra.run.smithi001.stdout:Sourcing file `/etc/default/grub' 2024-09-14T22:23:18.407 INFO:teuthology.orchestra.run.smithi001.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-14T22:23:18.422 INFO:teuthology.orchestra.run.smithi001.stdout:Generating grub configuration file ... 2024-09-14T22:23:18.705 INFO:teuthology.orchestra.run.smithi001.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-14T22:23:18.720 INFO:teuthology.orchestra.run.smithi001.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-14T22:23:19.024 INFO:teuthology.orchestra.run.smithi001.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-14T22:23:19.031 INFO:teuthology.orchestra.run.smithi001.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-14T22:23:19.110 INFO:teuthology.orchestra.run.smithi001.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-14T22:23:19.164 INFO:teuthology.orchestra.run.smithi001.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-14T22:23:19.279 INFO:teuthology.orchestra.run.smithi001.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-14T22:23:19.280 INFO:teuthology.orchestra.run.smithi001.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-14T22:23:19.280 INFO:teuthology.orchestra.run.smithi001.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-14T22:23:19.302 INFO:teuthology.orchestra.run.smithi176.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-14T22:23:19.302 INFO:teuthology.orchestra.run.smithi176.stdout:Sourcing file `/etc/default/grub' 2024-09-14T22:23:19.318 INFO:teuthology.orchestra.run.smithi176.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-14T22:23:19.330 INFO:teuthology.orchestra.run.smithi001.stdout:done 2024-09-14T22:23:19.332 INFO:teuthology.orchestra.run.smithi176.stdout:Generating grub configuration file ... 2024-09-14T22:23:19.613 INFO:teuthology.orchestra.run.smithi176.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-14T22:23:19.635 INFO:teuthology.orchestra.run.smithi176.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-14T22:23:19.852 DEBUG:teuthology.orchestra.run.smithi001:> dpkg -s linux-image-generic 2024-09-14T22:23:19.912 INFO:teuthology.orchestra.run.smithi001.stdout:Package: linux-image-generic 2024-09-14T22:23:19.912 INFO:teuthology.orchestra.run.smithi001.stdout:Status: install ok installed 2024-09-14T22:23:19.912 INFO:teuthology.orchestra.run.smithi001.stdout:Priority: optional 2024-09-14T22:23:19.912 INFO:teuthology.orchestra.run.smithi001.stdout:Section: kernel 2024-09-14T22:23:19.912 INFO:teuthology.orchestra.run.smithi001.stdout:Installed-Size: 21 2024-09-14T22:23:19.912 INFO:teuthology.orchestra.run.smithi001.stdout:Maintainer: Ubuntu Kernel Team 2024-09-14T22:23:19.912 INFO:teuthology.orchestra.run.smithi001.stdout:Architecture: amd64 2024-09-14T22:23:19.912 INFO:teuthology.orchestra.run.smithi001.stdout:Source: linux-meta 2024-09-14T22:23:19.912 INFO:teuthology.orchestra.run.smithi001.stdout:Version: 5.15.0.121.121 2024-09-14T22:23:19.912 INFO:teuthology.orchestra.run.smithi001.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-14T22:23:19.913 INFO:teuthology.orchestra.run.smithi001.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-14T22:23:19.913 INFO:teuthology.orchestra.run.smithi001.stdout:Recommends: thermald 2024-09-14T22:23:19.913 INFO:teuthology.orchestra.run.smithi001.stdout:Description: Generic Linux kernel image 2024-09-14T22:23:19.913 INFO:teuthology.orchestra.run.smithi001.stdout: This package will always depend on the latest generic kernel image 2024-09-14T22:23:19.913 INFO:teuthology.orchestra.run.smithi001.stdout: available. 2024-09-14T22:23:19.914 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-14T22:23:19.914 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-14T22:23:19.914 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-14T22:23:19.914 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-14T22:23:19.914 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi001.front.sepia.ceph.com, path=None, version=distro) 2024-09-14T22:23:19.914 DEBUG:teuthology.orchestra.run.smithi001:> sudo apt-get clean 2024-09-14T22:23:19.953 INFO:teuthology.orchestra.run.smithi176.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-14T22:23:19.959 INFO:teuthology.orchestra.run.smithi176.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-14T22:23:20.025 INFO:teuthology.orchestra.run.smithi176.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-14T22:23:20.031 INFO:teuthology.orchestra.run.smithi176.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-14T22:23:20.074 DEBUG:teuthology.orchestra.run.smithi001:> sudo apt-get update 2024-09-14T22:23:20.180 INFO:teuthology.orchestra.run.smithi176.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-14T22:23:20.180 INFO:teuthology.orchestra.run.smithi176.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-14T22:23:20.180 INFO:teuthology.orchestra.run.smithi176.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-14T22:23:20.189 INFO:teuthology.orchestra.run.smithi001.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-14T22:23:20.189 INFO:teuthology.orchestra.run.smithi001.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-14T22:23:20.198 INFO:teuthology.orchestra.run.smithi001.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-14T22:23:20.216 INFO:teuthology.orchestra.run.smithi001.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-14T22:23:20.221 INFO:teuthology.orchestra.run.smithi176.stdout:done 2024-09-14T22:23:20.714 DEBUG:teuthology.orchestra.run.smithi176:> dpkg -s linux-image-generic 2024-09-14T22:23:20.783 INFO:teuthology.orchestra.run.smithi176.stdout:Package: linux-image-generic 2024-09-14T22:23:20.783 INFO:teuthology.orchestra.run.smithi176.stdout:Status: install ok installed 2024-09-14T22:23:20.783 INFO:teuthology.orchestra.run.smithi176.stdout:Priority: optional 2024-09-14T22:23:20.783 INFO:teuthology.orchestra.run.smithi176.stdout:Section: kernel 2024-09-14T22:23:20.784 INFO:teuthology.orchestra.run.smithi176.stdout:Installed-Size: 21 2024-09-14T22:23:20.784 INFO:teuthology.orchestra.run.smithi176.stdout:Maintainer: Ubuntu Kernel Team 2024-09-14T22:23:20.784 INFO:teuthology.orchestra.run.smithi176.stdout:Architecture: amd64 2024-09-14T22:23:20.784 INFO:teuthology.orchestra.run.smithi176.stdout:Source: linux-meta 2024-09-14T22:23:20.784 INFO:teuthology.orchestra.run.smithi176.stdout:Version: 5.15.0.121.121 2024-09-14T22:23:20.784 INFO:teuthology.orchestra.run.smithi176.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-14T22:23:20.784 INFO:teuthology.orchestra.run.smithi176.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-14T22:23:20.784 INFO:teuthology.orchestra.run.smithi176.stdout:Recommends: thermald 2024-09-14T22:23:20.784 INFO:teuthology.orchestra.run.smithi176.stdout:Description: Generic Linux kernel image 2024-09-14T22:23:20.784 INFO:teuthology.orchestra.run.smithi176.stdout: This package will always depend on the latest generic kernel image 2024-09-14T22:23:20.784 INFO:teuthology.orchestra.run.smithi176.stdout: available. 2024-09-14T22:23:20.785 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-14T22:23:20.785 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-14T22:23:20.785 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-14T22:23:20.785 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-14T22:23:20.785 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi176.front.sepia.ceph.com, path=None, version=distro) 2024-09-14T22:23:20.785 DEBUG:teuthology.orchestra.run.smithi176:> sudo apt-get clean 2024-09-14T22:23:20.983 DEBUG:teuthology.orchestra.run.smithi176:> sudo apt-get update 2024-09-14T22:23:21.106 INFO:teuthology.orchestra.run.smithi176.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-14T22:23:21.106 INFO:teuthology.orchestra.run.smithi176.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-14T22:23:21.116 INFO:teuthology.orchestra.run.smithi176.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-14T22:23:21.133 INFO:teuthology.orchestra.run.smithi176.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-14T22:23:21.658 INFO:teuthology.orchestra.run.smithi001.stdout:Reading package lists... 2024-09-14T22:23:21.676 DEBUG:teuthology.orchestra.run.smithi001:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-14T22:23:21.730 INFO:teuthology.orchestra.run.smithi001.stdout:Reading package lists... 2024-09-14T22:23:21.921 INFO:teuthology.orchestra.run.smithi001.stdout:Building dependency tree... 2024-09-14T22:23:21.922 INFO:teuthology.orchestra.run.smithi001.stdout:Reading state information... 2024-09-14T22:23:22.095 INFO:teuthology.orchestra.run.smithi001.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-14T22:23:22.095 INFO:teuthology.orchestra.run.smithi001.stdout:The following packages were automatically installed and are no longer required: 2024-09-14T22:23:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-14T22:23:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-14T22:23:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-14T22:23:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-14T22:23:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-14T22:23:22.096 INFO:teuthology.orchestra.run.smithi001.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-14T22:23:22.138 INFO:teuthology.orchestra.run.smithi001.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-14T22:23:22.140 DEBUG:teuthology.orchestra.run.smithi001:> dpkg -s linux-image-generic 2024-09-14T22:23:22.199 INFO:teuthology.orchestra.run.smithi001.stdout:Package: linux-image-generic 2024-09-14T22:23:22.199 INFO:teuthology.orchestra.run.smithi001.stdout:Status: install ok installed 2024-09-14T22:23:22.199 INFO:teuthology.orchestra.run.smithi001.stdout:Priority: optional 2024-09-14T22:23:22.199 INFO:teuthology.orchestra.run.smithi001.stdout:Section: kernel 2024-09-14T22:23:22.199 INFO:teuthology.orchestra.run.smithi001.stdout:Installed-Size: 21 2024-09-14T22:23:22.199 INFO:teuthology.orchestra.run.smithi001.stdout:Maintainer: Ubuntu Kernel Team 2024-09-14T22:23:22.199 INFO:teuthology.orchestra.run.smithi001.stdout:Architecture: amd64 2024-09-14T22:23:22.200 INFO:teuthology.orchestra.run.smithi001.stdout:Source: linux-meta 2024-09-14T22:23:22.200 INFO:teuthology.orchestra.run.smithi001.stdout:Version: 5.15.0.121.121 2024-09-14T22:23:22.200 INFO:teuthology.orchestra.run.smithi001.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-14T22:23:22.200 INFO:teuthology.orchestra.run.smithi001.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-14T22:23:22.200 INFO:teuthology.orchestra.run.smithi001.stdout:Recommends: thermald 2024-09-14T22:23:22.200 INFO:teuthology.orchestra.run.smithi001.stdout:Description: Generic Linux kernel image 2024-09-14T22:23:22.200 INFO:teuthology.orchestra.run.smithi001.stdout: This package will always depend on the latest generic kernel image 2024-09-14T22:23:22.200 INFO:teuthology.orchestra.run.smithi001.stdout: available. 2024-09-14T22:23:22.201 DEBUG:teuthology.orchestra.run.smithi001:> mktemp 2024-09-14T22:23:22.249 INFO:teuthology.orchestra.run.smithi001.stdout:/tmp/tmp.uLRTNm8ZoZ 2024-09-14T22:23:22.249 DEBUG:teuthology.orchestra.run.smithi001:> sudo cp /boot/grub/grub.cfg /tmp/tmp.uLRTNm8ZoZ 2024-09-14T22:23:22.305 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 0666 /tmp/tmp.uLRTNm8ZoZ 2024-09-14T22:23:22.421 DEBUG:teuthology.orchestra.remote:smithi001:/tmp/tmp.uLRTNm8ZoZ is 10KB 2024-09-14T22:23:22.434 DEBUG:teuthology.orchestra.run.smithi001:> rm -fr /tmp/tmp.uLRTNm8ZoZ 2024-09-14T22:23:22.439 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-14T22:23:22.492 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-14T22:23:22.492 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-14T22:23:22.492 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-14T22:23:22.560 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-14T22:23:22.560 DEBUG:teuthology.orchestra.run.smithi001:> sudo update-grub 2024-09-14T22:23:22.561 INFO:teuthology.orchestra.run.smithi176.stdout:Reading package lists... 2024-09-14T22:23:22.579 DEBUG:teuthology.orchestra.run.smithi176:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-14T22:23:22.639 INFO:teuthology.orchestra.run.smithi176.stdout:Reading package lists... 2024-09-14T22:23:22.846 INFO:teuthology.orchestra.run.smithi176.stdout:Building dependency tree... 2024-09-14T22:23:22.846 INFO:teuthology.orchestra.run.smithi176.stdout:Reading state information... 2024-09-14T22:23:23.027 INFO:teuthology.orchestra.run.smithi176.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-14T22:23:23.027 INFO:teuthology.orchestra.run.smithi176.stdout:The following packages were automatically installed and are no longer required: 2024-09-14T22:23:23.027 INFO:teuthology.orchestra.run.smithi176.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-14T22:23:23.027 INFO:teuthology.orchestra.run.smithi176.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-14T22:23:23.027 INFO:teuthology.orchestra.run.smithi176.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-14T22:23:23.028 INFO:teuthology.orchestra.run.smithi176.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-14T22:23:23.028 INFO:teuthology.orchestra.run.smithi176.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-14T22:23:23.028 INFO:teuthology.orchestra.run.smithi176.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-14T22:23:23.068 INFO:teuthology.orchestra.run.smithi176.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-14T22:23:23.070 DEBUG:teuthology.orchestra.run.smithi176:> dpkg -s linux-image-generic 2024-09-14T22:23:23.085 INFO:teuthology.orchestra.run.smithi176.stdout:Package: linux-image-generic 2024-09-14T22:23:23.085 INFO:teuthology.orchestra.run.smithi176.stdout:Status: install ok installed 2024-09-14T22:23:23.086 INFO:teuthology.orchestra.run.smithi176.stdout:Priority: optional 2024-09-14T22:23:23.086 INFO:teuthology.orchestra.run.smithi176.stdout:Section: kernel 2024-09-14T22:23:23.086 INFO:teuthology.orchestra.run.smithi176.stdout:Installed-Size: 21 2024-09-14T22:23:23.086 INFO:teuthology.orchestra.run.smithi176.stdout:Maintainer: Ubuntu Kernel Team 2024-09-14T22:23:23.086 INFO:teuthology.orchestra.run.smithi176.stdout:Architecture: amd64 2024-09-14T22:23:23.086 INFO:teuthology.orchestra.run.smithi176.stdout:Source: linux-meta 2024-09-14T22:23:23.086 INFO:teuthology.orchestra.run.smithi176.stdout:Version: 5.15.0.121.121 2024-09-14T22:23:23.086 INFO:teuthology.orchestra.run.smithi176.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-14T22:23:23.086 INFO:teuthology.orchestra.run.smithi176.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-14T22:23:23.086 INFO:teuthology.orchestra.run.smithi176.stdout:Recommends: thermald 2024-09-14T22:23:23.086 INFO:teuthology.orchestra.run.smithi176.stdout:Description: Generic Linux kernel image 2024-09-14T22:23:23.086 INFO:teuthology.orchestra.run.smithi176.stdout: This package will always depend on the latest generic kernel image 2024-09-14T22:23:23.086 INFO:teuthology.orchestra.run.smithi176.stdout: available. 2024-09-14T22:23:23.087 DEBUG:teuthology.orchestra.run.smithi176:> mktemp 2024-09-14T22:23:23.134 INFO:teuthology.orchestra.run.smithi176.stdout:/tmp/tmp.2KDdUztWUv 2024-09-14T22:23:23.135 DEBUG:teuthology.orchestra.run.smithi176:> sudo cp /boot/grub/grub.cfg /tmp/tmp.2KDdUztWUv 2024-09-14T22:23:23.190 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod 0666 /tmp/tmp.2KDdUztWUv 2024-09-14T22:23:23.315 DEBUG:teuthology.orchestra.remote:smithi176:/tmp/tmp.2KDdUztWUv is 10KB 2024-09-14T22:23:23.328 DEBUG:teuthology.orchestra.run.smithi176:> rm -fr /tmp/tmp.2KDdUztWUv 2024-09-14T22:23:23.333 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-14T22:23:23.385 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T22:23:23.385 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-14T22:23:23.385 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-14T22:23:23.452 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-14T22:23:23.452 DEBUG:teuthology.orchestra.run.smithi176:> sudo update-grub 2024-09-14T22:23:23.750 INFO:teuthology.orchestra.run.smithi001.stderr:Sourcing file `/etc/default/grub' 2024-09-14T22:23:23.751 INFO:teuthology.orchestra.run.smithi001.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-14T22:23:23.753 INFO:teuthology.orchestra.run.smithi001.stderr:Generating grub configuration file ... 2024-09-14T22:23:23.963 INFO:teuthology.orchestra.run.smithi001.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-14T22:23:23.969 INFO:teuthology.orchestra.run.smithi001.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-14T22:23:24.209 INFO:teuthology.orchestra.run.smithi001.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-14T22:23:24.215 INFO:teuthology.orchestra.run.smithi001.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-14T22:23:24.258 INFO:teuthology.orchestra.run.smithi001.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-14T22:23:24.263 INFO:teuthology.orchestra.run.smithi001.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-14T22:23:24.383 INFO:teuthology.orchestra.run.smithi001.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-14T22:23:24.383 INFO:teuthology.orchestra.run.smithi001.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-14T22:23:24.383 INFO:teuthology.orchestra.run.smithi001.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-14T22:23:24.400 INFO:teuthology.orchestra.run.smithi001.stderr:done 2024-09-14T22:23:24.409 DEBUG:teuthology.orchestra.run.smithi001:> sudo shutdown -r now 2024-09-14T22:23:24.629 INFO:teuthology.orchestra.run.smithi176.stderr:Sourcing file `/etc/default/grub' 2024-09-14T22:23:24.630 INFO:teuthology.orchestra.run.smithi176.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-14T22:23:24.633 INFO:teuthology.orchestra.run.smithi176.stderr:Generating grub configuration file ... 2024-09-14T22:23:24.829 INFO:teuthology.orchestra.run.smithi176.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-14T22:23:24.835 INFO:teuthology.orchestra.run.smithi176.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-14T22:23:25.124 INFO:teuthology.orchestra.run.smithi176.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-14T22:23:25.129 INFO:teuthology.orchestra.run.smithi176.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-14T22:23:25.169 INFO:teuthology.orchestra.run.smithi176.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-14T22:23:25.174 INFO:teuthology.orchestra.run.smithi176.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-14T22:23:25.312 INFO:teuthology.orchestra.run.smithi176.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-14T22:23:25.312 INFO:teuthology.orchestra.run.smithi176.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-14T22:23:25.313 INFO:teuthology.orchestra.run.smithi176.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-14T22:23:25.327 INFO:teuthology.orchestra.run.smithi176.stderr:done 2024-09-14T22:23:25.336 DEBUG:teuthology.orchestra.run.smithi176:> sudo shutdown -r now 2024-09-14T22:23:54.441 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-14T22:23:54.442 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-09-14T22:23:54.442 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T22:23:55.339 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-14T22:23:55.340 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-09-14T22:23:55.340 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T22:24:12.861 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.1 2024-09-14T22:24:13.885 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.176 2024-09-14T22:24:21.868 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-09-14T22:24:21.869 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T22:24:22.887 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-09-14T22:24:22.888 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T22:24:25.949 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.176 2024-09-14T22:24:37.960 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-09-14T22:24:37.961 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T22:24:41.021 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.176 2024-09-14T22:24:56.033 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-09-14T22:24:56.034 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T22:25:14.557 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.176 2024-09-14T22:25:21.871 DEBUG:teuthology.orchestra.remote:timed out 2024-09-14T22:25:32.565 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-09-14T22:25:32.566 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T22:25:32.943 DEBUG:teuthology.orchestra.run.smithi176:> true 2024-09-14T22:25:33.873 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-09-14T22:25:33.873 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T22:25:34.078 DEBUG:teuthology.orchestra.run.smithi001:> true 2024-09-14T22:25:34.148 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-09-14T22:25:34.148 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-121-generic"... 2024-09-14T22:25:34.149 DEBUG:teuthology.orchestra.run.smithi176:> uname -r 2024-09-14T22:25:34.197 INFO:teuthology.orchestra.run.smithi176.stdout:5.15.0-121-generic 2024-09-14T22:25:34.197 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-14T22:25:34.197 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-14T22:25:34.197 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-14T22:25:34.627 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-09-14T22:25:34.627 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-121-generic"... 2024-09-14T22:25:34.627 DEBUG:teuthology.orchestra.run.smithi001:> uname -r 2024-09-14T22:25:34.676 INFO:teuthology.orchestra.run.smithi001.stdout:5.15.0-121-generic 2024-09-14T22:25:34.676 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-14T22:25:34.676 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-14T22:25:34.676 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-14T22:25:35.198 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-14T22:25:35.198 DEBUG:teuthology.orchestra.run.smithi176:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-14T22:25:35.485 INFO:teuthology.orchestra.run.smithi176.stdout:ttyS1 2024-09-14T22:25:35.502 DEBUG:teuthology.parallel:result is None 2024-09-14T22:25:35.677 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-14T22:25:35.677 DEBUG:teuthology.orchestra.run.smithi001:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-14T22:25:35.745 INFO:teuthology.orchestra.run.smithi001.stdout:ttyS1 2024-09-14T22:25:35.761 DEBUG:teuthology.parallel:result is None 2024-09-14T22:25:35.762 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-14T22:25:35.768 INFO:teuthology.task.internal:Creating test directory... 2024-09-14T22:25:35.768 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-14T22:25:35.770 DEBUG:teuthology.orchestra.run.smithi176:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-14T22:25:35.776 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-14T22:25:35.787 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-14T22:25:35.799 INFO:teuthology.task.internal:Creating archive directory... 2024-09-14T22:25:35.799 DEBUG:teuthology.orchestra.run.smithi001:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-14T22:25:35.817 DEBUG:teuthology.orchestra.run.smithi176:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-14T22:25:35.842 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-14T22:25:35.851 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-14T22:25:35.851 DEBUG:teuthology.orchestra.run.smithi001:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-14T22:25:35.865 DEBUG:teuthology.orchestra.run.smithi176:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-14T22:25:35.880 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T22:25:35.888 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T22:25:35.900 INFO:teuthology.orchestra.run.smithi176.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T22:25:35.910 INFO:teuthology.orchestra.run.smithi176.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T22:25:35.912 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-14T22:25:35.920 INFO:teuthology.task.internal:Configuring sudo... 2024-09-14T22:25:35.920 DEBUG:teuthology.orchestra.run.smithi001:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-14T22:25:35.933 DEBUG:teuthology.orchestra.run.smithi176:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-14T22:25:35.968 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-14T22:25:35.977 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-14T22:25:35.977 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-14T22:25:35.989 DEBUG:teuthology.orchestra.run.smithi176:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-14T22:25:36.013 DEBUG:teuthology.orchestra.run.smithi001:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-14T22:25:36.040 DEBUG:teuthology.orchestra.run.smithi001:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-14T22:25:36.088 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-14T22:25:36.089 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-14T22:25:36.144 DEBUG:teuthology.orchestra.run.smithi176:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-14T22:25:36.150 DEBUG:teuthology.orchestra.run.smithi176:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-14T22:25:36.197 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T22:25:36.197 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-14T22:25:36.262 DEBUG:teuthology.orchestra.run.smithi001:> sudo service rsyslog restart 2024-09-14T22:25:36.264 DEBUG:teuthology.orchestra.run.smithi176:> sudo service rsyslog restart 2024-09-14T22:25:36.335 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-14T22:25:36.343 INFO:teuthology.task.internal:Starting timer... 2024-09-14T22:25:36.343 INFO:teuthology.run_tasks:Running task pcp... 2024-09-14T22:25:36.404 INFO:teuthology.run_tasks:Running task selinux... 2024-09-14T22:25:36.415 DEBUG:teuthology.task.selinux:Excluding smithi001: OS 'ubuntu' does not support SELinux 2024-09-14T22:25:36.415 DEBUG:teuthology.task.selinux:Excluding smithi176: OS 'ubuntu' does not support SELinux 2024-09-14T22:25:36.415 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-14T22:25:36.415 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-14T22:25:36.415 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-14T22:25:36.415 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-14T22:25:36.425 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-14T22:25:36.437 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-14T22:25:36.438 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi001.front.sepia.ceph.com,smithi176.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-14T22:33:11.844 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi001.front.sepia.ceph.com'), Remote(name='ubuntu@smithi176.front.sepia.ceph.com')] 2024-09-14T22:33:11.845 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-09-14T22:33:11.846 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T22:33:11.930 DEBUG:teuthology.orchestra.run.smithi001:> true 2024-09-14T22:33:12.013 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-09-14T22:33:12.013 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-09-14T22:33:12.014 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T22:33:12.095 DEBUG:teuthology.orchestra.run.smithi176:> true 2024-09-14T22:33:12.173 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-09-14T22:33:12.173 INFO:teuthology.run_tasks:Running task clock... 2024-09-14T22:33:12.183 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-14T22:33:12.183 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-14T22:33:12.183 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T22:33:12.185 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-14T22:33:12.186 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T22:33:12.213 INFO:teuthology.orchestra.run.smithi001.stdout:14 Sep 22:33:12 ntpd[18379]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-14T22:33:12.214 INFO:teuthology.orchestra.run.smithi001.stdout:14 Sep 22:33:12 ntpd[18379]: Command line: ntpd -gq 2024-09-14T22:33:12.214 INFO:teuthology.orchestra.run.smithi001.stdout:14 Sep 22:33:12 ntpd[18379]: ---------------------------------------------------- 2024-09-14T22:33:12.214 INFO:teuthology.orchestra.run.smithi001.stdout:14 Sep 22:33:12 ntpd[18379]: ntp-4 is maintained by Network Time Foundation, 2024-09-14T22:33:12.214 INFO:teuthology.orchestra.run.smithi001.stdout:14 Sep 22:33:12 ntpd[18379]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-14T22:33:12.214 INFO:teuthology.orchestra.run.smithi001.stdout:14 Sep 22:33:12 ntpd[18379]: corporation. Support and training for ntp-4 are 2024-09-14T22:33:12.214 INFO:teuthology.orchestra.run.smithi001.stdout:14 Sep 22:33:12 ntpd[18379]: available at https://www.nwtime.org/support 2024-09-14T22:33:12.214 INFO:teuthology.orchestra.run.smithi001.stdout:14 Sep 22:33:12 ntpd[18379]: ---------------------------------------------------- 2024-09-14T22:33:12.215 INFO:teuthology.orchestra.run.smithi001.stdout:14 Sep 22:33:12 ntpd[18379]: proto: precision = 0.061 usec (-24) 2024-09-14T22:33:12.215 INFO:teuthology.orchestra.run.smithi001.stdout:14 Sep 22:33:12 ntpd[18379]: basedate set to 2022-02-04 2024-09-14T22:33:12.215 INFO:teuthology.orchestra.run.smithi001.stdout:14 Sep 22:33:12 ntpd[18379]: gps base set to 2022-02-06 (week 2196) 2024-09-14T22:33:12.215 INFO:teuthology.orchestra.run.smithi001.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-14T22:33:12.215 INFO:teuthology.orchestra.run.smithi001.stdout:14 Sep 22:33:12 ntpd[18379]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-14T22:33:12.216 INFO:teuthology.orchestra.run.smithi001.stdout:14 Sep 22:33:12 ntpd[18379]: restrict ::: KOD does nothing without LIMITED. 2024-09-14T22:33:12.216 INFO:teuthology.orchestra.run.smithi001.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-14T22:33:12.217 INFO:teuthology.orchestra.run.smithi001.stdout:14 Sep 22:33:12 ntpd[18379]: Listen and drop on 0 v6wildcard [::]:123 2024-09-14T22:33:12.217 INFO:teuthology.orchestra.run.smithi001.stdout:14 Sep 22:33:12 ntpd[18379]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-14T22:33:12.217 INFO:teuthology.orchestra.run.smithi001.stdout:14 Sep 22:33:12 ntpd[18379]: Listen normally on 2 lo 127.0.0.1:123 2024-09-14T22:33:12.217 INFO:teuthology.orchestra.run.smithi001.stdout:14 Sep 22:33:12 ntpd[18379]: Listen normally on 3 ens1f0 172.21.15.1:123 2024-09-14T22:33:12.217 INFO:teuthology.orchestra.run.smithi001.stdout:14 Sep 22:33:12 ntpd[18379]: Listen normally on 4 lo [::1]:123 2024-09-14T22:33:12.217 INFO:teuthology.orchestra.run.smithi001.stdout:14 Sep 22:33:12 ntpd[18379]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:15e8%4]:123 2024-09-14T22:33:12.217 INFO:teuthology.orchestra.run.smithi001.stdout:14 Sep 22:33:12 ntpd[18379]: Listening on routing socket on fd #22 for interface updates 2024-09-14T22:33:12.244 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 22:33:12 ntpd[18454]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-14T22:33:12.244 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 22:33:12 ntpd[18454]: Command line: ntpd -gq 2024-09-14T22:33:12.244 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 22:33:12 ntpd[18454]: ---------------------------------------------------- 2024-09-14T22:33:12.244 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 22:33:12 ntpd[18454]: ntp-4 is maintained by Network Time Foundation, 2024-09-14T22:33:12.245 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 22:33:12 ntpd[18454]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-14T22:33:12.245 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 22:33:12 ntpd[18454]: corporation. Support and training for ntp-4 are 2024-09-14T22:33:12.245 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 22:33:12 ntpd[18454]: available at https://www.nwtime.org/support 2024-09-14T22:33:12.245 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 22:33:12 ntpd[18454]: ---------------------------------------------------- 2024-09-14T22:33:12.245 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 22:33:12 ntpd[18454]: proto: precision = 0.043 usec (-24) 2024-09-14T22:33:12.246 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 22:33:12 ntpd[18454]: basedate set to 2022-02-04 2024-09-14T22:33:12.246 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 22:33:12 ntpd[18454]: gps base set to 2022-02-06 (week 2196) 2024-09-14T22:33:12.246 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 22:33:12 ntpd[18454]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-14T22:33:12.246 INFO:teuthology.orchestra.run.smithi176.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-14T22:33:12.246 INFO:teuthology.orchestra.run.smithi176.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-14T22:33:12.247 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 22:33:12 ntpd[18454]: restrict ::: KOD does nothing without LIMITED. 2024-09-14T22:33:12.247 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 22:33:12 ntpd[18454]: Listen and drop on 0 v6wildcard [::]:123 2024-09-14T22:33:12.247 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 22:33:12 ntpd[18454]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-14T22:33:12.247 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 22:33:12 ntpd[18454]: Listen normally on 2 lo 127.0.0.1:123 2024-09-14T22:33:12.247 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 22:33:12 ntpd[18454]: Listen normally on 3 enp3s0f1 172.21.15.176:123 2024-09-14T22:33:12.247 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 22:33:12 ntpd[18454]: Listen normally on 4 lo [::1]:123 2024-09-14T22:33:12.247 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 22:33:12 ntpd[18454]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6f61%5]:123 2024-09-14T22:33:12.247 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 22:33:12 ntpd[18454]: Listening on routing socket on fd #22 for interface updates 2024-09-14T22:33:13.215 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:13 ntpd[18379]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:13.215 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:13 ntpd[18379]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:13.215 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:13 ntpd[18379]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:13.215 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:13 ntpd[18379]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:13.215 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:13 ntpd[18379]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:13.215 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:13 ntpd[18379]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:13.215 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:13 ntpd[18379]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:13.216 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:13 ntpd[18379]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:13.216 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:13 ntpd[18379]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T22:33:13.216 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:13 ntpd[18379]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T22:33:13.216 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:13 ntpd[18379]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:13.216 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:13 ntpd[18379]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:13.216 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:13 ntpd[18379]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T22:33:13.216 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:13 ntpd[18379]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T22:33:14.245 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:14 ntpd[18454]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:14.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:14 ntpd[18454]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:14.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:14 ntpd[18454]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:14.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:14 ntpd[18454]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:14.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:14 ntpd[18454]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:14.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:14 ntpd[18454]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:14.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:14 ntpd[18454]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T22:33:14.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:14 ntpd[18454]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T22:33:15.215 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:15 ntpd[18379]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:15.215 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:15 ntpd[18379]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:15.215 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:15 ntpd[18379]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:15.215 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:15 ntpd[18379]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:15.215 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:15 ntpd[18379]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:15.215 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:15 ntpd[18379]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:15.215 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:15 ntpd[18379]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T22:33:15.216 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:15 ntpd[18379]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T22:33:15.216 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:15 ntpd[18379]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:15.216 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:15 ntpd[18379]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:15.216 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:15 ntpd[18379]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T22:33:15.216 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:15 ntpd[18379]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T22:33:16.214 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:16 ntpd[18379]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:16.214 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:16 ntpd[18379]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:16.215 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:16 ntpd[18379]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:16.215 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:16 ntpd[18379]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:16.215 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:16 ntpd[18379]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T22:33:16.215 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:16 ntpd[18379]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T22:33:16.245 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:16 ntpd[18454]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:16.245 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:16 ntpd[18454]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:16.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:16 ntpd[18454]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:16.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:16 ntpd[18454]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:16.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:16 ntpd[18454]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:16.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:16 ntpd[18454]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:16.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:16 ntpd[18454]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T22:33:16.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:16 ntpd[18454]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T22:33:16.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:16 ntpd[18454]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:16.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:16 ntpd[18454]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:16.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:16 ntpd[18454]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T22:33:16.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:16 ntpd[18454]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T22:33:17.215 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:17 ntpd[18379]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:17.215 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:17 ntpd[18379]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:17.215 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:17 ntpd[18379]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:17.215 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:17 ntpd[18379]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:17.215 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:17 ntpd[18379]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:17.215 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:17 ntpd[18379]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:17.215 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:17 ntpd[18379]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T22:33:17.215 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:17 ntpd[18379]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T22:33:17.216 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:17 ntpd[18379]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:17.216 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:17 ntpd[18379]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:17.216 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:17 ntpd[18379]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T22:33:17.216 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:17 ntpd[18379]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T22:33:17.245 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:17 ntpd[18454]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:17.245 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:17 ntpd[18454]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:17.245 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:17 ntpd[18454]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:17.245 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:17 ntpd[18454]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:17.245 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:17 ntpd[18454]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T22:33:17.245 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:17 ntpd[18454]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T22:33:18.214 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:18 ntpd[18379]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:18.214 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:18 ntpd[18379]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:18.214 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:18 ntpd[18379]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:18.214 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:18 ntpd[18379]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:18.215 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:18 ntpd[18379]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T22:33:18.215 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:18 ntpd[18379]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T22:33:18.245 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:18 ntpd[18454]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:18.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:18 ntpd[18454]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:18.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:18 ntpd[18454]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:18.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:18 ntpd[18454]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:18.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:18 ntpd[18454]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:18.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:18 ntpd[18454]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:18.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:18 ntpd[18454]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T22:33:18.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:18 ntpd[18454]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T22:33:18.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:18 ntpd[18454]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:18.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:18 ntpd[18454]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:18.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:18 ntpd[18454]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T22:33:18.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:18 ntpd[18454]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T22:33:19.214 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:19 ntpd[18379]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:19.215 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:19 ntpd[18379]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:19.215 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:19 ntpd[18379]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:19.215 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:19 ntpd[18379]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:19.215 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:19 ntpd[18379]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:19.215 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:19 ntpd[18379]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:19.216 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:19 ntpd[18379]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T22:33:19.216 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:19 ntpd[18379]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T22:33:19.216 INFO:teuthology.orchestra.run.smithi001.stdout:14 Sep 22:33:19 ntpd[18379]: ntpd: time slew -0.001843 s 2024-09-14T22:33:19.216 INFO:teuthology.orchestra.run.smithi001.stdout:ntpd: time slew -0.001843s 2024-09-14T22:33:19.216 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:19 ntpd[18379]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-14T22:33:19.217 INFO:teuthology.orchestra.run.smithi001.stderr:14 Sep 22:33:19 ntpd[18379]: can't open /var/log/ntpstats/loopstats.20240914: Permission denied 2024-09-14T22:33:19.245 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:19 ntpd[18454]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:19.245 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:19 ntpd[18454]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:19.245 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:19 ntpd[18454]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:19.245 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:19 ntpd[18454]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:19.245 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:19 ntpd[18454]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T22:33:19.245 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:19 ntpd[18454]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T22:33:19.264 INFO:teuthology.orchestra.run.smithi001.stdout: remote refid st t when poll reach delay offset jitter 2024-09-14T22:33:19.264 INFO:teuthology.orchestra.run.smithi001.stdout:============================================================================== 2024-09-14T22:33:19.265 INFO:teuthology.orchestra.run.smithi001.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T22:33:19.265 INFO:teuthology.orchestra.run.smithi001.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T22:33:19.265 INFO:teuthology.orchestra.run.smithi001.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T22:33:19.265 INFO:teuthology.orchestra.run.smithi001.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T22:33:20.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:20 ntpd[18454]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:20.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:20 ntpd[18454]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:20.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:20 ntpd[18454]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:20.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:20 ntpd[18454]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:20.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:20 ntpd[18454]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:20.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:20 ntpd[18454]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:20.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:20 ntpd[18454]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T22:33:20.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:20 ntpd[18454]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T22:33:20.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:20 ntpd[18454]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T22:33:20.246 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:20 ntpd[18454]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T22:33:20.247 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:20 ntpd[18454]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T22:33:20.247 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:20 ntpd[18454]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T22:33:20.247 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 22:33:20 ntpd[18454]: ntpd: time slew -0.000193 s 2024-09-14T22:33:20.247 INFO:teuthology.orchestra.run.smithi176.stdout:ntpd: time slew -0.000193s 2024-09-14T22:33:20.247 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:20 ntpd[18454]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-14T22:33:20.247 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 22:33:20 ntpd[18454]: can't open /var/log/ntpstats/loopstats.20240914: Permission denied 2024-09-14T22:33:20.303 INFO:teuthology.orchestra.run.smithi176.stdout: remote refid st t when poll reach delay offset jitter 2024-09-14T22:33:20.303 INFO:teuthology.orchestra.run.smithi176.stdout:============================================================================== 2024-09-14T22:33:20.303 INFO:teuthology.orchestra.run.smithi176.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T22:33:20.303 INFO:teuthology.orchestra.run.smithi176.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T22:33:20.303 INFO:teuthology.orchestra.run.smithi176.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T22:33:20.303 INFO:teuthology.orchestra.run.smithi176.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T22:33:20.304 INFO:teuthology.run_tasks:Running task install... 2024-09-14T22:33:20.311 DEBUG:teuthology.task.install:project ceph 2024-09-14T22:33:20.311 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}} 2024-09-14T22:33:20.311 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'} 2024-09-14T22:33:20.311 INFO:teuthology.task.install:Using flavor: default 2024-09-14T22:33:20.316 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-14T22:33:20.316 INFO:teuthology.task.install:extra packages: [] 2024-09-14T22:33:20.317 DEBUG:teuthology.orchestra.run.smithi001:> sudo apt-key list | grep Ceph 2024-09-14T22:33:20.317 DEBUG:teuthology.orchestra.run.smithi176:> sudo apt-key list | grep Ceph 2024-09-14T22:33:20.389 INFO:teuthology.orchestra.run.smithi001.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-14T22:33:20.398 INFO:teuthology.orchestra.run.smithi176.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-14T22:33:20.422 INFO:teuthology.orchestra.run.smithi001.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-14T22:33:20.422 INFO:teuthology.orchestra.run.smithi001.stdout:uid [ unknown] Ceph.com (release key) 2024-09-14T22:33:20.423 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-14T22:33:20.423 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ecab7be58963a5bd03910cd73eef66405d7fdd50 2024-09-14T22:33:20.427 INFO:teuthology.orchestra.run.smithi176.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-14T22:33:20.428 INFO:teuthology.orchestra.run.smithi176.stdout:uid [ unknown] Ceph.com (release key) 2024-09-14T22:33:20.428 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-14T22:33:20.428 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ecab7be58963a5bd03910cd73eef66405d7fdd50 2024-09-14T22:33:20.568 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default/ 2024-09-14T22:33:20.569 INFO:teuthology.task.install.deb:Package version is 19.1.1-324-gecab7be5-1jammy 2024-09-14T22:33:20.572 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default/ 2024-09-14T22:33:20.572 INFO:teuthology.task.install.deb:Package version is 19.1.1-324-gecab7be5-1jammy 2024-09-14T22:33:20.678 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-14T22:33:20.678 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-14T22:33:20.683 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T22:33:20.683 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-14T22:33:20.694 DEBUG:teuthology.orchestra.run.smithi001:> sudo apt-get update 2024-09-14T22:33:20.696 DEBUG:teuthology.orchestra.run.smithi176:> sudo apt-get update 2024-09-14T22:33:20.865 INFO:teuthology.orchestra.run.smithi001.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-14T22:33:20.876 INFO:teuthology.orchestra.run.smithi001.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-14T22:33:20.876 INFO:teuthology.orchestra.run.smithi001.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-14T22:33:20.897 INFO:teuthology.orchestra.run.smithi001.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-14T22:33:20.919 INFO:teuthology.orchestra.run.smithi176.stdout:Ign:1 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy InRelease 2024-09-14T22:33:20.922 INFO:teuthology.orchestra.run.smithi001.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy InRelease 2024-09-14T22:33:20.941 INFO:teuthology.orchestra.run.smithi176.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-14T22:33:20.943 INFO:teuthology.orchestra.run.smithi001.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-14T22:33:20.964 INFO:teuthology.orchestra.run.smithi176.stdout:Ign:3 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-14T22:33:20.970 INFO:teuthology.orchestra.run.smithi001.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-14T22:33:20.991 INFO:teuthology.orchestra.run.smithi001.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,410 B] 2024-09-14T22:33:20.994 INFO:teuthology.orchestra.run.smithi176.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-14T22:33:20.994 INFO:teuthology.orchestra.run.smithi176.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,410 B] 2024-09-14T22:33:21.004 INFO:teuthology.orchestra.run.smithi176.stdout:Hit:6 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-14T22:33:21.018 INFO:teuthology.orchestra.run.smithi001.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-09-14T22:33:21.023 INFO:teuthology.orchestra.run.smithi176.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-09-14T22:33:21.067 INFO:teuthology.orchestra.run.smithi176.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-14T22:33:21.148 INFO:teuthology.orchestra.run.smithi176.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-14T22:33:21.239 INFO:teuthology.orchestra.run.smithi001.stdout:Fetched 29.0 kB in 0s (81.2 kB/s) 2024-09-14T22:33:21.370 INFO:teuthology.orchestra.run.smithi176.stdout:Fetched 29.0 kB in 0s (59.9 kB/s) 2024-09-14T22:33:22.285 INFO:teuthology.orchestra.run.smithi001.stdout:Reading package lists... 2024-09-14T22:33:22.302 DEBUG:teuthology.orchestra.run.smithi001:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.1.1-324-gecab7be5-1jammy cephadm=19.1.1-324-gecab7be5-1jammy ceph-mds=19.1.1-324-gecab7be5-1jammy ceph-mgr=19.1.1-324-gecab7be5-1jammy ceph-common=19.1.1-324-gecab7be5-1jammy ceph-fuse=19.1.1-324-gecab7be5-1jammy ceph-test=19.1.1-324-gecab7be5-1jammy ceph-volume=19.1.1-324-gecab7be5-1jammy radosgw=19.1.1-324-gecab7be5-1jammy python3-rados=19.1.1-324-gecab7be5-1jammy python3-rgw=19.1.1-324-gecab7be5-1jammy python3-cephfs=19.1.1-324-gecab7be5-1jammy python3-rbd=19.1.1-324-gecab7be5-1jammy libcephfs2=19.1.1-324-gecab7be5-1jammy libcephfs-dev=19.1.1-324-gecab7be5-1jammy librados2=19.1.1-324-gecab7be5-1jammy librbd1=19.1.1-324-gecab7be5-1jammy rbd-fuse=19.1.1-324-gecab7be5-1jammy 2024-09-14T22:33:22.367 INFO:teuthology.orchestra.run.smithi001.stdout:Reading package lists... 2024-09-14T22:33:22.392 INFO:teuthology.orchestra.run.smithi176.stdout:Reading package lists... 2024-09-14T22:33:22.409 DEBUG:teuthology.orchestra.run.smithi176:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.1.1-324-gecab7be5-1jammy cephadm=19.1.1-324-gecab7be5-1jammy ceph-mds=19.1.1-324-gecab7be5-1jammy ceph-mgr=19.1.1-324-gecab7be5-1jammy ceph-common=19.1.1-324-gecab7be5-1jammy ceph-fuse=19.1.1-324-gecab7be5-1jammy ceph-test=19.1.1-324-gecab7be5-1jammy ceph-volume=19.1.1-324-gecab7be5-1jammy radosgw=19.1.1-324-gecab7be5-1jammy python3-rados=19.1.1-324-gecab7be5-1jammy python3-rgw=19.1.1-324-gecab7be5-1jammy python3-cephfs=19.1.1-324-gecab7be5-1jammy python3-rbd=19.1.1-324-gecab7be5-1jammy libcephfs2=19.1.1-324-gecab7be5-1jammy libcephfs-dev=19.1.1-324-gecab7be5-1jammy librados2=19.1.1-324-gecab7be5-1jammy librbd1=19.1.1-324-gecab7be5-1jammy rbd-fuse=19.1.1-324-gecab7be5-1jammy 2024-09-14T22:33:22.471 INFO:teuthology.orchestra.run.smithi176.stdout:Reading package lists... 2024-09-14T22:33:22.581 INFO:teuthology.orchestra.run.smithi001.stdout:Building dependency tree... 2024-09-14T22:33:22.582 INFO:teuthology.orchestra.run.smithi001.stdout:Reading state information... 2024-09-14T22:33:22.680 INFO:teuthology.orchestra.run.smithi176.stdout:Building dependency tree... 2024-09-14T22:33:22.680 INFO:teuthology.orchestra.run.smithi176.stdout:Reading state information... 2024-09-14T22:33:22.804 INFO:teuthology.orchestra.run.smithi001.stdout:The following packages were automatically installed and are no longer required: 2024-09-14T22:33:22.804 INFO:teuthology.orchestra.run.smithi001.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-14T22:33:22.805 INFO:teuthology.orchestra.run.smithi001.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-14T22:33:22.805 INFO:teuthology.orchestra.run.smithi001.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-14T22:33:22.805 INFO:teuthology.orchestra.run.smithi001.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-14T22:33:22.805 INFO:teuthology.orchestra.run.smithi001.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-14T22:33:22.805 INFO:teuthology.orchestra.run.smithi001.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-14T22:33:22.806 INFO:teuthology.orchestra.run.smithi001.stdout:The following additional packages will be installed: 2024-09-14T22:33:22.806 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-14T22:33:22.808 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-14T22:33:22.808 INFO:teuthology.orchestra.run.smithi001.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-14T22:33:22.808 INFO:teuthology.orchestra.run.smithi001.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-14T22:33:22.808 INFO:teuthology.orchestra.run.smithi001.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-14T22:33:22.808 INFO:teuthology.orchestra.run.smithi001.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-14T22:33:22.808 INFO:teuthology.orchestra.run.smithi001.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-14T22:33:22.808 INFO:teuthology.orchestra.run.smithi001.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-14T22:33:22.808 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-14T22:33:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-14T22:33:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-14T22:33:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-14T22:33:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-14T22:33:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-14T22:33:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-14T22:33:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-14T22:33:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-14T22:33:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-14T22:33:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-14T22:33:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-14T22:33:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-14T22:33:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-14T22:33:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-14T22:33:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: xmlstarlet zip 2024-09-14T22:33:22.810 INFO:teuthology.orchestra.run.smithi001.stdout:Suggested packages: 2024-09-14T22:33:22.810 INFO:teuthology.orchestra.run.smithi001.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-14T22:33:22.810 INFO:teuthology.orchestra.run.smithi001.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-14T22:33:22.810 INFO:teuthology.orchestra.run.smithi001.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-14T22:33:22.810 INFO:teuthology.orchestra.run.smithi001.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-14T22:33:22.810 INFO:teuthology.orchestra.run.smithi001.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-14T22:33:22.810 INFO:teuthology.orchestra.run.smithi001.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-14T22:33:22.811 INFO:teuthology.orchestra.run.smithi001.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-14T22:33:22.811 INFO:teuthology.orchestra.run.smithi001.stdout:Recommended packages: 2024-09-14T22:33:22.811 INFO:teuthology.orchestra.run.smithi001.stdout: btrfs-tools 2024-09-14T22:33:22.878 INFO:teuthology.orchestra.run.smithi176.stdout:The following packages were automatically installed and are no longer required: 2024-09-14T22:33:22.878 INFO:teuthology.orchestra.run.smithi176.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-14T22:33:22.878 INFO:teuthology.orchestra.run.smithi176.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-14T22:33:22.879 INFO:teuthology.orchestra.run.smithi176.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-14T22:33:22.879 INFO:teuthology.orchestra.run.smithi176.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-14T22:33:22.879 INFO:teuthology.orchestra.run.smithi176.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-14T22:33:22.879 INFO:teuthology.orchestra.run.smithi176.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-14T22:33:22.880 INFO:teuthology.orchestra.run.smithi176.stdout:The following additional packages will be installed: 2024-09-14T22:33:22.880 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-14T22:33:22.880 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-14T22:33:22.880 INFO:teuthology.orchestra.run.smithi176.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-14T22:33:22.880 INFO:teuthology.orchestra.run.smithi176.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-14T22:33:22.881 INFO:teuthology.orchestra.run.smithi176.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-14T22:33:22.881 INFO:teuthology.orchestra.run.smithi176.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-14T22:33:22.882 INFO:teuthology.orchestra.run.smithi176.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-14T22:33:22.882 INFO:teuthology.orchestra.run.smithi176.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-14T22:33:22.882 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-14T22:33:22.882 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-14T22:33:22.882 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-14T22:33:22.882 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-14T22:33:22.882 INFO:teuthology.orchestra.run.smithi176.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-14T22:33:22.882 INFO:teuthology.orchestra.run.smithi176.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-14T22:33:22.882 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-14T22:33:22.882 INFO:teuthology.orchestra.run.smithi176.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-14T22:33:22.882 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-14T22:33:22.882 INFO:teuthology.orchestra.run.smithi176.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-14T22:33:22.883 INFO:teuthology.orchestra.run.smithi176.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-14T22:33:22.883 INFO:teuthology.orchestra.run.smithi176.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-14T22:33:22.883 INFO:teuthology.orchestra.run.smithi176.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-14T22:33:22.883 INFO:teuthology.orchestra.run.smithi176.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-14T22:33:22.883 INFO:teuthology.orchestra.run.smithi176.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-14T22:33:22.883 INFO:teuthology.orchestra.run.smithi176.stdout: xmlstarlet zip 2024-09-14T22:33:22.883 INFO:teuthology.orchestra.run.smithi176.stdout:Suggested packages: 2024-09-14T22:33:22.883 INFO:teuthology.orchestra.run.smithi176.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-14T22:33:22.883 INFO:teuthology.orchestra.run.smithi176.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-14T22:33:22.884 INFO:teuthology.orchestra.run.smithi176.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-14T22:33:22.884 INFO:teuthology.orchestra.run.smithi176.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-14T22:33:22.884 INFO:teuthology.orchestra.run.smithi176.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-14T22:33:22.884 INFO:teuthology.orchestra.run.smithi176.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-14T22:33:22.884 INFO:teuthology.orchestra.run.smithi176.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-14T22:33:22.884 INFO:teuthology.orchestra.run.smithi176.stdout:Recommended packages: 2024-09-14T22:33:22.884 INFO:teuthology.orchestra.run.smithi176.stdout: btrfs-tools 2024-09-14T22:33:22.917 INFO:teuthology.orchestra.run.smithi001.stdout:The following NEW packages will be installed: 2024-09-14T22:33:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-14T22:33:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-14T22:33:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-14T22:33:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-14T22:33:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-14T22:33:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-14T22:33:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-14T22:33:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-14T22:33:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-14T22:33:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-14T22:33:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-14T22:33:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-14T22:33:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-14T22:33:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-14T22:33:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-14T22:33:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-14T22:33:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-14T22:33:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-14T22:33:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-14T22:33:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-14T22:33:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-14T22:33:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-14T22:33:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-14T22:33:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-14T22:33:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-14T22:33:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-14T22:33:22.979 INFO:teuthology.orchestra.run.smithi001.stdout:0 upgraded, 118 newly installed, 0 to remove and 331 not upgraded. 2024-09-14T22:33:22.980 INFO:teuthology.orchestra.run.smithi001.stdout:Need to get 181 MB of archives. 2024-09-14T22:33:22.980 INFO:teuthology.orchestra.run.smithi001.stdout:After this operation, 829 MB of additional disk space will be used. 2024-09-14T22:33:22.980 INFO:teuthology.orchestra.run.smithi001.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-14T22:33:22.986 INFO:teuthology.orchestra.run.smithi176.stdout:The following NEW packages will be installed: 2024-09-14T22:33:22.986 INFO:teuthology.orchestra.run.smithi176.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-14T22:33:22.986 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-14T22:33:22.986 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-14T22:33:22.986 INFO:teuthology.orchestra.run.smithi176.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-14T22:33:22.986 INFO:teuthology.orchestra.run.smithi176.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-14T22:33:22.986 INFO:teuthology.orchestra.run.smithi176.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-14T22:33:22.987 INFO:teuthology.orchestra.run.smithi176.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-14T22:33:22.987 INFO:teuthology.orchestra.run.smithi176.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-14T22:33:22.988 INFO:teuthology.orchestra.run.smithi176.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-14T22:33:22.988 INFO:teuthology.orchestra.run.smithi176.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-14T22:33:22.988 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-14T22:33:22.988 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-14T22:33:22.988 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-14T22:33:22.988 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-14T22:33:22.988 INFO:teuthology.orchestra.run.smithi176.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-14T22:33:22.988 INFO:teuthology.orchestra.run.smithi176.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-14T22:33:22.988 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-14T22:33:22.988 INFO:teuthology.orchestra.run.smithi176.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-14T22:33:22.988 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-14T22:33:22.988 INFO:teuthology.orchestra.run.smithi176.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-14T22:33:22.989 INFO:teuthology.orchestra.run.smithi176.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-14T22:33:22.989 INFO:teuthology.orchestra.run.smithi176.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-14T22:33:22.989 INFO:teuthology.orchestra.run.smithi176.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-14T22:33:22.989 INFO:teuthology.orchestra.run.smithi176.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-14T22:33:22.989 INFO:teuthology.orchestra.run.smithi176.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-14T22:33:22.989 INFO:teuthology.orchestra.run.smithi176.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-14T22:33:23.046 INFO:teuthology.orchestra.run.smithi176.stdout:0 upgraded, 118 newly installed, 0 to remove and 331 not upgraded. 2024-09-14T22:33:23.046 INFO:teuthology.orchestra.run.smithi176.stdout:Need to get 181 MB of archives. 2024-09-14T22:33:23.046 INFO:teuthology.orchestra.run.smithi176.stdout:After this operation, 829 MB of additional disk space will be used. 2024-09-14T22:33:23.046 INFO:teuthology.orchestra.run.smithi176.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-14T22:33:23.060 INFO:teuthology.orchestra.run.smithi001.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.1.1-324-gecab7be5-1jammy [3,591 kB] 2024-09-14T22:33:23.068 INFO:teuthology.orchestra.run.smithi001.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-14T22:33:23.071 INFO:teuthology.orchestra.run.smithi001.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-14T22:33:23.088 INFO:teuthology.orchestra.run.smithi001.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-14T22:33:23.125 INFO:teuthology.orchestra.run.smithi176.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.1.1-324-gecab7be5-1jammy [3,591 kB] 2024-09-14T22:33:23.134 INFO:teuthology.orchestra.run.smithi176.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-14T22:33:23.136 INFO:teuthology.orchestra.run.smithi176.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-14T22:33:23.154 INFO:teuthology.orchestra.run.smithi176.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-14T22:33:23.158 INFO:teuthology.orchestra.run.smithi001.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-14T22:33:23.164 INFO:teuthology.orchestra.run.smithi001.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-14T22:33:23.178 INFO:teuthology.orchestra.run.smithi001.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-14T22:33:23.183 INFO:teuthology.orchestra.run.smithi001.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-14T22:33:23.183 INFO:teuthology.orchestra.run.smithi001.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-14T22:33:23.184 INFO:teuthology.orchestra.run.smithi001.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-14T22:33:23.193 INFO:teuthology.orchestra.run.smithi001.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-14T22:33:23.196 INFO:teuthology.orchestra.run.smithi001.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-14T22:33:23.198 INFO:teuthology.orchestra.run.smithi001.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-14T22:33:23.200 INFO:teuthology.orchestra.run.smithi001.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-14T22:33:23.200 INFO:teuthology.orchestra.run.smithi001.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-14T22:33:23.212 INFO:teuthology.orchestra.run.smithi176.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-14T22:33:23.215 INFO:teuthology.orchestra.run.smithi001.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-14T22:33:23.216 INFO:teuthology.orchestra.run.smithi176.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-14T22:33:23.217 INFO:teuthology.orchestra.run.smithi001.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-14T22:33:23.219 INFO:teuthology.orchestra.run.smithi001.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-14T22:33:23.220 INFO:teuthology.orchestra.run.smithi001.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-14T22:33:23.220 INFO:teuthology.orchestra.run.smithi001.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-14T22:33:23.228 INFO:teuthology.orchestra.run.smithi176.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-14T22:33:23.232 INFO:teuthology.orchestra.run.smithi001.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-14T22:33:23.232 INFO:teuthology.orchestra.run.smithi001.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-14T22:33:23.233 INFO:teuthology.orchestra.run.smithi176.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-14T22:33:23.233 INFO:teuthology.orchestra.run.smithi176.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-14T22:33:23.233 INFO:teuthology.orchestra.run.smithi001.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-14T22:33:23.233 INFO:teuthology.orchestra.run.smithi001.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-14T22:33:23.234 INFO:teuthology.orchestra.run.smithi176.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-14T22:33:23.234 INFO:teuthology.orchestra.run.smithi001.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-14T22:33:23.242 INFO:teuthology.orchestra.run.smithi176.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-14T22:33:23.244 INFO:teuthology.orchestra.run.smithi176.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-14T22:33:23.247 INFO:teuthology.orchestra.run.smithi176.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-14T22:33:23.248 INFO:teuthology.orchestra.run.smithi176.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-14T22:33:23.248 INFO:teuthology.orchestra.run.smithi001.stdout:Get:27 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.1.1-324-gecab7be5-1jammy [3,241 kB] 2024-09-14T22:33:23.249 INFO:teuthology.orchestra.run.smithi001.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-14T22:33:23.250 INFO:teuthology.orchestra.run.smithi001.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-14T22:33:23.251 INFO:teuthology.orchestra.run.smithi176.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-14T22:33:23.252 INFO:teuthology.orchestra.run.smithi001.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-14T22:33:23.253 INFO:teuthology.orchestra.run.smithi001.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-14T22:33:23.253 INFO:teuthology.orchestra.run.smithi176.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-14T22:33:23.254 INFO:teuthology.orchestra.run.smithi001.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-14T22:33:23.266 INFO:teuthology.orchestra.run.smithi176.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-14T22:33:23.267 INFO:teuthology.orchestra.run.smithi001.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-14T22:33:23.268 INFO:teuthology.orchestra.run.smithi176.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-14T22:33:23.268 INFO:teuthology.orchestra.run.smithi001.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-14T22:33:23.268 INFO:teuthology.orchestra.run.smithi001.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-14T22:33:23.269 INFO:teuthology.orchestra.run.smithi001.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-14T22:33:23.269 INFO:teuthology.orchestra.run.smithi176.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-14T22:33:23.269 INFO:teuthology.orchestra.run.smithi176.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-14T22:33:23.270 INFO:teuthology.orchestra.run.smithi176.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-14T22:33:23.270 INFO:teuthology.orchestra.run.smithi176.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-14T22:33:23.271 INFO:teuthology.orchestra.run.smithi001.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-14T22:33:23.271 INFO:teuthology.orchestra.run.smithi176.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-14T22:33:23.271 INFO:teuthology.orchestra.run.smithi176.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-14T22:33:23.284 INFO:teuthology.orchestra.run.smithi001.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-14T22:33:23.284 INFO:teuthology.orchestra.run.smithi001.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-14T22:33:23.285 INFO:teuthology.orchestra.run.smithi176.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-14T22:33:23.286 INFO:teuthology.orchestra.run.smithi176.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-14T22:33:23.290 INFO:teuthology.orchestra.run.smithi001.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-14T22:33:23.290 INFO:teuthology.orchestra.run.smithi001.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-14T22:33:23.290 INFO:teuthology.orchestra.run.smithi001.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-14T22:33:23.301 INFO:teuthology.orchestra.run.smithi001.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-14T22:33:23.302 INFO:teuthology.orchestra.run.smithi001.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-14T22:33:23.303 INFO:teuthology.orchestra.run.smithi176.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-14T22:33:23.304 INFO:teuthology.orchestra.run.smithi001.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-14T22:33:23.305 INFO:teuthology.orchestra.run.smithi001.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-14T22:33:23.306 INFO:teuthology.orchestra.run.smithi001.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-14T22:33:23.307 INFO:teuthology.orchestra.run.smithi176.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-14T22:33:23.308 INFO:teuthology.orchestra.run.smithi176.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-14T22:33:23.309 INFO:teuthology.orchestra.run.smithi176.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-14T22:33:23.310 INFO:teuthology.orchestra.run.smithi176.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-14T22:33:23.311 INFO:teuthology.orchestra.run.smithi176.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-14T22:33:23.312 INFO:teuthology.orchestra.run.smithi176.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-14T22:33:23.312 INFO:teuthology.orchestra.run.smithi176.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-14T22:33:23.317 INFO:teuthology.orchestra.run.smithi001.stdout:Get:48 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.1.1-324-gecab7be5-1jammy [718 kB] 2024-09-14T22:33:23.318 INFO:teuthology.orchestra.run.smithi001.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-14T22:33:23.320 INFO:teuthology.orchestra.run.smithi176.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-14T22:33:23.321 INFO:teuthology.orchestra.run.smithi001.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-14T22:33:23.321 INFO:teuthology.orchestra.run.smithi001.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-14T22:33:23.321 INFO:teuthology.orchestra.run.smithi176.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-14T22:33:23.322 INFO:teuthology.orchestra.run.smithi001.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-14T22:33:23.325 INFO:teuthology.orchestra.run.smithi001.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-14T22:33:23.330 INFO:teuthology.orchestra.run.smithi001.stdout:Get:54 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.1.1-324-gecab7be5-1jammy [356 kB] 2024-09-14T22:33:23.335 INFO:teuthology.orchestra.run.smithi001.stdout:Get:55 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.1.1-324-gecab7be5-1jammy [32.8 kB] 2024-09-14T22:33:23.336 INFO:teuthology.orchestra.run.smithi001.stdout:Get:56 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.1.1-324-gecab7be5-1jammy [180 kB] 2024-09-14T22:33:23.337 INFO:teuthology.orchestra.run.smithi176.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-14T22:33:23.339 INFO:teuthology.orchestra.run.smithi001.stdout:Get:57 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.1.1-324-gecab7be5-1jammy [67.8 kB] 2024-09-14T22:33:23.340 INFO:teuthology.orchestra.run.smithi001.stdout:Get:58 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.1.1-324-gecab7be5-1jammy [333 kB] 2024-09-14T22:33:23.346 INFO:teuthology.orchestra.run.smithi176.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-14T22:33:23.346 INFO:teuthology.orchestra.run.smithi176.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-14T22:33:23.348 INFO:teuthology.orchestra.run.smithi176.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-14T22:33:23.348 INFO:teuthology.orchestra.run.smithi176.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-14T22:33:23.351 INFO:teuthology.orchestra.run.smithi176.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-14T22:33:23.354 INFO:teuthology.orchestra.run.smithi176.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-14T22:33:23.355 INFO:teuthology.orchestra.run.smithi176.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-14T22:33:23.357 INFO:teuthology.orchestra.run.smithi176.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-14T22:33:23.358 INFO:teuthology.orchestra.run.smithi001.stdout:Get:59 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.1.1-324-gecab7be5-1jammy [6,875 kB] 2024-09-14T22:33:23.359 INFO:teuthology.orchestra.run.smithi176.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-14T22:33:23.372 INFO:teuthology.orchestra.run.smithi176.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-14T22:33:23.373 INFO:teuthology.orchestra.run.smithi176.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-14T22:33:23.374 INFO:teuthology.orchestra.run.smithi176.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-14T22:33:23.378 INFO:teuthology.orchestra.run.smithi176.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-14T22:33:23.380 INFO:teuthology.orchestra.run.smithi001.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-14T22:33:23.381 INFO:teuthology.orchestra.run.smithi001.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-14T22:33:23.382 INFO:teuthology.orchestra.run.smithi001.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-14T22:33:23.382 INFO:teuthology.orchestra.run.smithi001.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-14T22:33:23.383 INFO:teuthology.orchestra.run.smithi001.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-14T22:33:23.401 INFO:teuthology.orchestra.run.smithi001.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-14T22:33:23.403 INFO:teuthology.orchestra.run.smithi001.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-14T22:33:23.403 INFO:teuthology.orchestra.run.smithi001.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-14T22:33:23.405 INFO:teuthology.orchestra.run.smithi176.stdout:Get:52 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.1.1-324-gecab7be5-1jammy [3,241 kB] 2024-09-14T22:33:23.418 INFO:teuthology.orchestra.run.smithi001.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-14T22:33:23.418 INFO:teuthology.orchestra.run.smithi001.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-14T22:33:23.419 INFO:teuthology.orchestra.run.smithi001.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-14T22:33:23.419 INFO:teuthology.orchestra.run.smithi001.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-14T22:33:23.419 INFO:teuthology.orchestra.run.smithi001.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-14T22:33:23.420 INFO:teuthology.orchestra.run.smithi001.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-14T22:33:23.433 INFO:teuthology.orchestra.run.smithi001.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-14T22:33:23.434 INFO:teuthology.orchestra.run.smithi001.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-14T22:33:23.436 INFO:teuthology.orchestra.run.smithi001.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-14T22:33:23.443 INFO:teuthology.orchestra.run.smithi176.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-14T22:33:23.444 INFO:teuthology.orchestra.run.smithi176.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-14T22:33:23.446 INFO:teuthology.orchestra.run.smithi176.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-14T22:33:23.446 INFO:teuthology.orchestra.run.smithi176.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-14T22:33:23.447 INFO:teuthology.orchestra.run.smithi176.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-14T22:33:23.450 INFO:teuthology.orchestra.run.smithi001.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-14T22:33:23.451 INFO:teuthology.orchestra.run.smithi001.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-14T22:33:23.452 INFO:teuthology.orchestra.run.smithi001.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-14T22:33:23.455 INFO:teuthology.orchestra.run.smithi001.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-14T22:33:23.457 INFO:teuthology.orchestra.run.smithi001.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-14T22:33:23.458 INFO:teuthology.orchestra.run.smithi001.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-14T22:33:23.458 INFO:teuthology.orchestra.run.smithi001.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-14T22:33:23.464 INFO:teuthology.orchestra.run.smithi176.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-14T22:33:23.466 INFO:teuthology.orchestra.run.smithi176.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-14T22:33:23.466 INFO:teuthology.orchestra.run.smithi176.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-14T22:33:23.467 INFO:teuthology.orchestra.run.smithi001.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-14T22:33:23.468 INFO:teuthology.orchestra.run.smithi001.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-14T22:33:23.470 INFO:teuthology.orchestra.run.smithi001.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-14T22:33:23.481 INFO:teuthology.orchestra.run.smithi176.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-14T22:33:23.482 INFO:teuthology.orchestra.run.smithi176.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-14T22:33:23.482 INFO:teuthology.orchestra.run.smithi176.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-14T22:33:23.482 INFO:teuthology.orchestra.run.smithi176.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-14T22:33:23.483 INFO:teuthology.orchestra.run.smithi176.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-14T22:33:23.483 INFO:teuthology.orchestra.run.smithi176.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-14T22:33:23.484 INFO:teuthology.orchestra.run.smithi001.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-14T22:33:23.485 INFO:teuthology.orchestra.run.smithi001.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-14T22:33:23.486 INFO:teuthology.orchestra.run.smithi001.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-14T22:33:23.487 INFO:teuthology.orchestra.run.smithi001.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-14T22:33:23.487 INFO:teuthology.orchestra.run.smithi001.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-14T22:33:23.493 INFO:teuthology.orchestra.run.smithi001.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-14T22:33:23.494 INFO:teuthology.orchestra.run.smithi001.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-14T22:33:23.498 INFO:teuthology.orchestra.run.smithi176.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-14T22:33:23.498 INFO:teuthology.orchestra.run.smithi176.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-14T22:33:23.502 INFO:teuthology.orchestra.run.smithi001.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-14T22:33:23.502 INFO:teuthology.orchestra.run.smithi176.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-14T22:33:23.503 INFO:teuthology.orchestra.run.smithi001.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-14T22:33:23.504 INFO:teuthology.orchestra.run.smithi001.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-14T22:33:23.515 INFO:teuthology.orchestra.run.smithi176.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-14T22:33:23.516 INFO:teuthology.orchestra.run.smithi176.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-14T22:33:23.517 INFO:teuthology.orchestra.run.smithi176.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-14T22:33:23.520 INFO:teuthology.orchestra.run.smithi176.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-14T22:33:23.522 INFO:teuthology.orchestra.run.smithi176.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-14T22:33:23.523 INFO:teuthology.orchestra.run.smithi176.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-14T22:33:23.524 INFO:teuthology.orchestra.run.smithi176.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-14T22:33:23.528 INFO:teuthology.orchestra.run.smithi001.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.1.1-324-gecab7be5-1jammy [112 kB] 2024-09-14T22:33:23.531 INFO:teuthology.orchestra.run.smithi001.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.1.1-324-gecab7be5-1jammy [472 kB] 2024-09-14T22:33:23.532 INFO:teuthology.orchestra.run.smithi176.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-14T22:33:23.533 INFO:teuthology.orchestra.run.smithi176.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-14T22:33:23.536 INFO:teuthology.orchestra.run.smithi176.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-14T22:33:23.540 INFO:teuthology.orchestra.run.smithi001.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.1.1-324-gecab7be5-1jammy [25.9 MB] 2024-09-14T22:33:23.549 INFO:teuthology.orchestra.run.smithi176.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-14T22:33:23.552 INFO:teuthology.orchestra.run.smithi176.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-14T22:33:23.553 INFO:teuthology.orchestra.run.smithi176.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-14T22:33:23.553 INFO:teuthology.orchestra.run.smithi176.stdout:Get:83 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.1.1-324-gecab7be5-1jammy [718 kB] 2024-09-14T22:33:23.553 INFO:teuthology.orchestra.run.smithi176.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-14T22:33:23.553 INFO:teuthology.orchestra.run.smithi176.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-14T22:33:23.566 INFO:teuthology.orchestra.run.smithi176.stdout:Get:86 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.1.1-324-gecab7be5-1jammy [356 kB] 2024-09-14T22:33:23.567 INFO:teuthology.orchestra.run.smithi176.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-14T22:33:23.567 INFO:teuthology.orchestra.run.smithi176.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-14T22:33:23.568 INFO:teuthology.orchestra.run.smithi176.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-14T22:33:23.570 INFO:teuthology.orchestra.run.smithi176.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-14T22:33:23.570 INFO:teuthology.orchestra.run.smithi176.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-14T22:33:23.574 INFO:teuthology.orchestra.run.smithi176.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.1.1-324-gecab7be5-1jammy [32.8 kB] 2024-09-14T22:33:23.575 INFO:teuthology.orchestra.run.smithi176.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.1.1-324-gecab7be5-1jammy [180 kB] 2024-09-14T22:33:23.579 INFO:teuthology.orchestra.run.smithi176.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.1.1-324-gecab7be5-1jammy [67.8 kB] 2024-09-14T22:33:23.581 INFO:teuthology.orchestra.run.smithi176.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.1.1-324-gecab7be5-1jammy [333 kB] 2024-09-14T22:33:23.623 INFO:teuthology.orchestra.run.smithi176.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.1.1-324-gecab7be5-1jammy [6,875 kB] 2024-09-14T22:33:23.875 INFO:teuthology.orchestra.run.smithi176.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.1.1-324-gecab7be5-1jammy [112 kB] 2024-09-14T22:33:23.877 INFO:teuthology.orchestra.run.smithi176.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.1.1-324-gecab7be5-1jammy [472 kB] 2024-09-14T22:33:23.884 INFO:teuthology.orchestra.run.smithi176.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.1.1-324-gecab7be5-1jammy [25.9 MB] 2024-09-14T22:33:24.397 INFO:teuthology.orchestra.run.smithi001.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.1.1-324-gecab7be5-1jammy [5,145 kB] 2024-09-14T22:33:24.483 INFO:teuthology.orchestra.run.smithi001.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.1.1-324-gecab7be5-1jammy [245 kB] 2024-09-14T22:33:24.488 INFO:teuthology.orchestra.run.smithi001.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.1.1-324-gecab7be5-1jammy [124 kB] 2024-09-14T22:33:24.490 INFO:teuthology.orchestra.run.smithi001.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.1.1-324-gecab7be5-1jammy [1,555 kB] 2024-09-14T22:33:24.542 INFO:teuthology.orchestra.run.smithi001.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.1.1-324-gecab7be5-1jammy [6,143 kB] 2024-09-14T22:33:24.736 INFO:teuthology.orchestra.run.smithi001.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.1.1-324-gecab7be5-1jammy [22.8 MB] 2024-09-14T22:33:24.884 INFO:teuthology.orchestra.run.smithi176.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.1.1-324-gecab7be5-1jammy [5,145 kB] 2024-09-14T22:33:25.009 INFO:teuthology.orchestra.run.smithi176.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.1.1-324-gecab7be5-1jammy [245 kB] 2024-09-14T22:33:25.015 INFO:teuthology.orchestra.run.smithi176.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.1.1-324-gecab7be5-1jammy [124 kB] 2024-09-14T22:33:25.018 INFO:teuthology.orchestra.run.smithi176.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.1.1-324-gecab7be5-1jammy [1,555 kB] 2024-09-14T22:33:25.074 INFO:teuthology.orchestra.run.smithi176.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.1.1-324-gecab7be5-1jammy [6,143 kB] 2024-09-14T22:33:25.308 INFO:teuthology.orchestra.run.smithi176.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.1.1-324-gecab7be5-1jammy [22.8 MB] 2024-09-14T22:33:25.470 INFO:teuthology.orchestra.run.smithi001.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.1.1-324-gecab7be5-1jammy [14.1 kB] 2024-09-14T22:33:25.470 INFO:teuthology.orchestra.run.smithi001.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.1.1-324-gecab7be5-1jammy [877 kB] 2024-09-14T22:33:25.489 INFO:teuthology.orchestra.run.smithi001.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.1.1-324-gecab7be5-1jammy [2,462 kB] 2024-09-14T22:33:25.531 INFO:teuthology.orchestra.run.smithi001.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.1.1-324-gecab7be5-1jammy [794 kB] 2024-09-14T22:33:25.543 INFO:teuthology.orchestra.run.smithi001.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.1.1-324-gecab7be5-1jammy [152 kB] 2024-09-14T22:33:25.546 INFO:teuthology.orchestra.run.smithi001.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.1.1-324-gecab7be5-1jammy [2,275 kB] 2024-09-14T22:33:25.711 INFO:teuthology.orchestra.run.smithi001.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.1.1-324-gecab7be5-1jammy [8,625 kB] 2024-09-14T22:33:26.011 INFO:teuthology.orchestra.run.smithi001.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.1.1-324-gecab7be5-1jammy [14.2 kB] 2024-09-14T22:33:26.061 INFO:teuthology.orchestra.run.smithi001.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.1.1-324-gecab7be5-1jammy [51.4 MB] 2024-09-14T22:33:26.576 INFO:teuthology.orchestra.run.smithi176.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.1.1-324-gecab7be5-1jammy [14.1 kB] 2024-09-14T22:33:26.576 INFO:teuthology.orchestra.run.smithi176.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.1.1-324-gecab7be5-1jammy [877 kB] 2024-09-14T22:33:26.594 INFO:teuthology.orchestra.run.smithi176.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.1.1-324-gecab7be5-1jammy [2,462 kB] 2024-09-14T22:33:26.685 INFO:teuthology.orchestra.run.smithi176.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.1.1-324-gecab7be5-1jammy [794 kB] 2024-09-14T22:33:26.712 INFO:teuthology.orchestra.run.smithi176.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.1.1-324-gecab7be5-1jammy [152 kB] 2024-09-14T22:33:26.716 INFO:teuthology.orchestra.run.smithi176.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.1.1-324-gecab7be5-1jammy [2,275 kB] 2024-09-14T22:33:26.839 INFO:teuthology.orchestra.run.smithi176.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.1.1-324-gecab7be5-1jammy [8,625 kB] 2024-09-14T22:33:27.267 INFO:teuthology.orchestra.run.smithi176.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.1.1-324-gecab7be5-1jammy [14.2 kB] 2024-09-14T22:33:27.268 INFO:teuthology.orchestra.run.smithi176.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.1.1-324-gecab7be5-1jammy [51.4 MB] 2024-09-14T22:33:28.322 INFO:teuthology.orchestra.run.smithi001.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.1.1-324-gecab7be5-1jammy [130 kB] 2024-09-14T22:33:28.325 INFO:teuthology.orchestra.run.smithi001.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.1.1-324-gecab7be5-1jammy [39.0 kB] 2024-09-14T22:33:28.326 INFO:teuthology.orchestra.run.smithi001.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.1.1-324-gecab7be5-1jammy [13.6 MB] 2024-09-14T22:33:28.641 INFO:teuthology.orchestra.run.smithi001.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.1.1-324-gecab7be5-1jammy [91.7 kB] 2024-09-14T22:33:29.170 INFO:teuthology.orchestra.run.smithi176.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.1.1-324-gecab7be5-1jammy [130 kB] 2024-09-14T22:33:29.173 INFO:teuthology.orchestra.run.smithi176.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.1.1-324-gecab7be5-1jammy [39.0 kB] 2024-09-14T22:33:29.174 INFO:teuthology.orchestra.run.smithi176.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.1.1-324-gecab7be5-1jammy [13.6 MB] 2024-09-14T22:33:29.235 INFO:teuthology.orchestra.run.smithi001.stdout:Fetched 181 MB in 6s (31.7 MB/s) 2024-09-14T22:33:29.368 INFO:teuthology.orchestra.run.smithi176.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.1.1-324-gecab7be5-1jammy [91.7 kB] 2024-09-14T22:33:29.404 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-14T22:33:29.507 INFO:teuthology.orchestra.run.smithi001.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-14T22:33:29.513 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-14T22:33:29.594 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-14T22:33:29.934 INFO:teuthology.orchestra.run.smithi176.stdout:Fetched 181 MB in 6s (28.4 MB/s) 2024-09-14T22:33:29.948 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-14T22:33:29.974 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-14T22:33:30.014 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-14T22:33:30.097 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-14T22:33:30.197 INFO:teuthology.orchestra.run.smithi176.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-14T22:33:30.203 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-14T22:33:30.292 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-14T22:33:30.295 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-14T22:33:30.319 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-14T22:33:30.350 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-14T22:33:30.640 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-14T22:33:30.667 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-14T22:33:30.707 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-14T22:33:30.912 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-14T22:33:30.939 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-14T22:33:30.987 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T22:33:31.043 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-14T22:33:31.069 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-14T22:33:31.109 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-14T22:33:31.399 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-14T22:33:31.425 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-14T22:33:31.465 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T22:33:31.563 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-14T22:33:31.590 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-14T22:33:31.638 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T22:33:31.785 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-14T22:33:31.811 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-14T22:33:31.843 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T22:33:32.033 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-14T22:33:32.059 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-14T22:33:32.099 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T22:33:32.246 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-14T22:33:32.273 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-14T22:33:32.313 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-14T22:33:32.452 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-14T22:33:32.478 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-14T22:33:32.518 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T22:33:32.641 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package librados2. 2024-09-14T22:33:32.667 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../007-librados2_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:32.707 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking librados2 (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:32.847 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-14T22:33:32.873 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-14T22:33:32.914 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-14T22:33:33.143 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package librbd1. 2024-09-14T22:33:33.170 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../008-librbd1_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:33.210 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking librbd1 (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:33.233 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package librados2. 2024-09-14T22:33:33.259 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../007-librados2_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:33.299 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking librados2 (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:33.638 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package libcephfs2. 2024-09-14T22:33:33.664 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../009-libcephfs2_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:33.704 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking libcephfs2 (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:33.760 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package librbd1. 2024-09-14T22:33:33.787 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../008-librbd1_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:33.827 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking librbd1 (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:33.998 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-rados. 2024-09-14T22:33:34.014 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../010-python3-rados_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:34.056 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-rados (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:34.272 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libcephfs2. 2024-09-14T22:33:34.298 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../009-libcephfs2_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:34.338 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libcephfs2 (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:34.384 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-14T22:33:34.411 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.1.1-324-gecab7be5-1jammy_all.deb ... 2024-09-14T22:33:34.451 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-ceph-argparse (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:34.632 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-rados. 2024-09-14T22:33:34.647 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../010-python3-rados_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:34.678 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-cephfs. 2024-09-14T22:33:34.682 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-rados (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:34.694 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../012-python3-cephfs_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:34.728 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-cephfs (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:35.010 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-14T22:33:35.014 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-14T22:33:35.036 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.1.1-324-gecab7be5-1jammy_all.deb ... 2024-09-14T22:33:35.041 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../013-python3-ceph-common_19.1.1-324-gecab7be5-1jammy_all.deb ... 2024-09-14T22:33:35.076 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-ceph-argparse (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:35.081 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-ceph-common (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:35.329 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-cephfs. 2024-09-14T22:33:35.355 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../012-python3-cephfs_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:35.396 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-cephfs (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:35.609 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-14T22:33:35.635 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-14T22:33:35.675 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-14T22:33:35.707 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-14T22:33:35.733 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../013-python3-ceph-common_19.1.1-324-gecab7be5-1jammy_all.deb ... 2024-09-14T22:33:35.773 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-ceph-common (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:35.995 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-prettytable. 2024-09-14T22:33:36.021 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-09-14T22:33:36.078 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-14T22:33:36.093 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-14T22:33:36.119 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-14T22:33:36.159 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-14T22:33:36.339 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-rbd. 2024-09-14T22:33:36.355 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../016-python3-rbd_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:36.397 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-rbd (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:36.437 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-prettytable. 2024-09-14T22:33:36.464 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-09-14T22:33:36.504 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-14T22:33:36.750 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-14T22:33:36.773 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-rbd. 2024-09-14T22:33:36.776 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-14T22:33:36.788 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../016-python3-rbd_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:36.816 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-14T22:33:36.831 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-rbd (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:37.094 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-14T22:33:37.121 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-14T22:33:37.161 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-14T22:33:37.185 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-14T22:33:37.210 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-14T22:33:37.251 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-14T22:33:37.503 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-14T22:33:37.507 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-14T22:33:37.529 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-14T22:33:37.534 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-14T22:33:37.561 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-14T22:33:37.582 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-14T22:33:37.823 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-14T22:33:37.849 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-14T22:33:37.889 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-14T22:33:37.919 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package lua5.1. 2024-09-14T22:33:37.945 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-14T22:33:37.985 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-14T22:33:38.243 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package lua5.1. 2024-09-14T22:33:38.269 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-14T22:33:38.309 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-14T22:33:38.313 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package lua-any. 2024-09-14T22:33:38.329 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-09-14T22:33:38.380 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-14T22:33:38.629 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package lua-any. 2024-09-14T22:33:38.646 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-09-14T22:33:38.683 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package zip. 2024-09-14T22:33:38.687 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-14T22:33:38.709 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-09-14T22:33:38.758 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking zip (3.0-12build2) ... 2024-09-14T22:33:38.899 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package zip. 2024-09-14T22:33:38.915 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-09-14T22:33:38.958 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking zip (3.0-12build2) ... 2024-09-14T22:33:39.069 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package unzip. 2024-09-14T22:33:39.096 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-14T22:33:39.136 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-14T22:33:39.539 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package luarocks. 2024-09-14T22:33:39.556 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package unzip. 2024-09-14T22:33:39.566 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-14T22:33:39.582 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-14T22:33:39.606 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-14T22:33:39.786 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-14T22:33:40.030 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package librgw2. 2024-09-14T22:33:40.057 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../025-librgw2_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:40.105 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking librgw2 (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:40.156 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package luarocks. 2024-09-14T22:33:40.183 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-14T22:33:40.223 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-14T22:33:40.547 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package librgw2. 2024-09-14T22:33:40.573 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../025-librgw2_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:40.605 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking librgw2 (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:40.683 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-rgw. 2024-09-14T22:33:40.699 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../026-python3-rgw_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:40.741 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-rgw (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:41.094 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-14T22:33:41.121 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-14T22:33:41.141 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-rgw. 2024-09-14T22:33:41.161 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-14T22:33:41.167 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../026-python3-rgw_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:41.207 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-rgw (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:41.422 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package libradosstriper1. 2024-09-14T22:33:41.437 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../028-libradosstriper1_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:41.480 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking libradosstriper1 (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:41.552 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-14T22:33:41.578 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-14T22:33:41.618 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-14T22:33:41.913 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libradosstriper1. 2024-09-14T22:33:41.926 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package ceph-common. 2024-09-14T22:33:41.939 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../028-libradosstriper1_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:41.952 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../029-ceph-common_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:41.979 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libradosstriper1 (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:41.992 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking ceph-common (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:42.416 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-common. 2024-09-14T22:33:42.442 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../029-ceph-common_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:42.483 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-common (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:43.530 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package ceph-base. 2024-09-14T22:33:43.557 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../030-ceph-base_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:43.606 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking ceph-base (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:43.813 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-base. 2024-09-14T22:33:43.839 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../030-ceph-base_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:43.888 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-base (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:44.160 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-14T22:33:44.186 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-14T22:33:44.218 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-14T22:33:44.383 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-14T22:33:44.399 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-14T22:33:44.442 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-14T22:33:44.521 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-cheroot. 2024-09-14T22:33:44.547 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-14T22:33:44.587 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-14T22:33:44.744 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-cheroot. 2024-09-14T22:33:44.771 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-14T22:33:44.811 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-14T22:33:44.907 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-14T22:33:44.933 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-14T22:33:44.973 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-14T22:33:45.147 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-14T22:33:45.174 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-14T22:33:45.214 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-14T22:33:45.276 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-14T22:33:45.303 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-14T22:33:45.343 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-14T22:33:45.517 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-14T22:33:45.544 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-14T22:33:45.583 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-14T22:33:45.646 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-14T22:33:45.672 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-14T22:33:45.712 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-14T22:33:45.886 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-14T22:33:45.912 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-14T22:33:45.952 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-14T22:33:45.965 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-tempora. 2024-09-14T22:33:45.992 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-09-14T22:33:46.023 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-14T22:33:46.222 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-tempora. 2024-09-14T22:33:46.237 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-09-14T22:33:46.271 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-14T22:33:46.309 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-portend. 2024-09-14T22:33:46.336 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-09-14T22:33:46.376 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-14T22:33:46.550 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-portend. 2024-09-14T22:33:46.576 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-09-14T22:33:46.616 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-14T22:33:46.662 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-14T22:33:46.678 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-14T22:33:46.712 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-14T22:33:46.919 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-14T22:33:46.946 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-14T22:33:46.982 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-14T22:33:46.985 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-14T22:33:47.008 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-14T22:33:47.048 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-14T22:33:47.288 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-14T22:33:47.315 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-14T22:33:47.355 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-14T22:33:47.385 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-natsort. 2024-09-14T22:33:47.412 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-09-14T22:33:47.452 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-14T22:33:47.700 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-natsort. 2024-09-14T22:33:47.727 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-09-14T22:33:47.729 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-logutils. 2024-09-14T22:33:47.756 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-09-14T22:33:47.766 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-14T22:33:47.788 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-14T22:33:48.074 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-14T22:33:48.086 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-logutils. 2024-09-14T22:33:48.101 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-14T22:33:48.113 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-09-14T22:33:48.140 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-14T22:33:48.173 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-14T22:33:48.456 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-14T22:33:48.468 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-mako. 2024-09-14T22:33:48.482 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-14T22:33:48.495 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-14T22:33:48.522 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-14T22:33:48.535 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-14T22:33:48.783 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-mako. 2024-09-14T22:33:48.799 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-14T22:33:48.838 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-14T22:33:48.841 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-14T22:33:48.865 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-14T22:33:48.905 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-14T22:33:49.161 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-14T22:33:49.189 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-14T22:33:49.191 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-14T22:33:49.218 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-14T22:33:49.227 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-14T22:33:49.266 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-14T22:33:49.530 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-14T22:33:49.552 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-webob. 2024-09-14T22:33:49.557 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-14T22:33:49.579 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-14T22:33:49.597 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-14T22:33:49.618 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-14T22:33:49.891 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-webob. 2024-09-14T22:33:49.918 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-14T22:33:49.930 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-waitress. 2024-09-14T22:33:49.957 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-14T22:33:49.999 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-14T22:33:50.083 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-14T22:33:50.308 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-tempita. 2024-09-14T22:33:50.335 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-14T22:33:50.374 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-14T22:33:50.503 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-waitress. 2024-09-14T22:33:50.529 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-14T22:33:50.572 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-14T22:33:50.635 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-paste. 2024-09-14T22:33:50.651 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-14T22:33:50.685 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-14T22:33:50.889 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-tempita. 2024-09-14T22:33:50.915 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-14T22:33:50.983 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-14T22:33:51.233 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-paste. 2024-09-14T22:33:51.238 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-14T22:33:51.255 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-14T22:33:51.259 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-14T22:33:51.291 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-14T22:33:51.297 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-14T22:33:51.575 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-14T22:33:51.601 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-14T22:33:51.619 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-14T22:33:51.635 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-14T22:33:51.641 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-14T22:33:51.677 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-14T22:33:51.936 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-webtest. 2024-09-14T22:33:51.963 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-09-14T22:33:51.980 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-14T22:33:52.002 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-14T22:33:52.006 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-14T22:33:52.046 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-14T22:33:52.297 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-pecan. 2024-09-14T22:33:52.324 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-14T22:33:52.358 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-webtest. 2024-09-14T22:33:52.363 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-14T22:33:52.385 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-09-14T22:33:52.424 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-14T22:33:52.702 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-pecan. 2024-09-14T22:33:52.718 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-14T22:33:52.752 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-certifi. 2024-09-14T22:33:52.752 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-14T22:33:52.779 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-09-14T22:33:52.818 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-14T22:33:53.082 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-certifi. 2024-09-14T22:33:53.087 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-idna. 2024-09-14T22:33:53.105 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-09-14T22:33:53.114 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-14T22:33:53.140 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-14T22:33:53.154 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-14T22:33:53.443 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-idna. 2024-09-14T22:33:53.457 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-urllib3. 2024-09-14T22:33:53.469 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-14T22:33:53.484 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-14T22:33:53.509 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-14T22:33:53.523 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-14T22:33:53.804 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-urllib3. 2024-09-14T22:33:53.830 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-14T22:33:53.860 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-requests. 2024-09-14T22:33:53.870 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-14T22:33:53.887 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-14T22:33:53.926 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-14T22:33:54.190 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-requests. 2024-09-14T22:33:54.210 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-14T22:33:54.238 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-14T22:33:54.248 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-14T22:33:54.265 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-14T22:33:54.304 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-14T22:33:54.560 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-14T22:33:54.582 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-14T22:33:54.587 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-14T22:33:54.598 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.1.1-324-gecab7be5-1jammy_all.deb ... 2024-09-14T22:33:54.626 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-14T22:33:54.632 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking ceph-mgr-modules-core (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:54.937 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-14T22:33:54.964 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.1.1-324-gecab7be5-1jammy_all.deb ... 2024-09-14T22:33:55.004 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-mgr-modules-core (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:55.029 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-14T22:33:55.056 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:55.095 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking libsqlite3-mod-ceph (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:55.367 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-14T22:33:55.394 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:55.433 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libsqlite3-mod-ceph (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:55.753 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-14T22:33:55.779 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-14T22:33:55.819 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-14T22:33:56.182 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-14T22:33:56.210 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-14T22:33:56.249 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-14T22:33:56.414 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-openssl. 2024-09-14T22:33:56.441 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-09-14T22:33:56.548 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-openssl. 2024-09-14T22:33:56.576 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-09-14T22:33:56.618 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-14T22:33:56.797 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-14T22:33:56.946 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package ceph-mgr. 2024-09-14T22:33:56.973 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../063-ceph-mgr_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:57.012 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking ceph-mgr (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:57.284 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-mgr. 2024-09-14T22:33:57.299 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../063-ceph-mgr_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:57.334 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-mgr (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:57.424 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package ceph-mon. 2024-09-14T22:33:57.451 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../064-ceph-mon_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:57.490 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking ceph-mon (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:57.729 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-mon. 2024-09-14T22:33:57.755 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../064-ceph-mon_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:57.795 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-mon (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:58.044 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package ceph-osd. 2024-09-14T22:33:58.071 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../065-ceph-osd_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:58.102 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking ceph-osd (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:58.440 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-osd. 2024-09-14T22:33:58.467 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../065-ceph-osd_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:58.515 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-osd (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:59.181 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package ceph. 2024-09-14T22:33:59.208 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../066-ceph_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:59.247 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking ceph (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:59.566 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package ceph-fuse. 2024-09-14T22:33:59.576 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph. 2024-09-14T22:33:59.592 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../066-ceph_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:59.593 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../067-ceph-fuse_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:59.633 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking ceph-fuse (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:59.635 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:33:59.945 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package ceph-mds. 2024-09-14T22:33:59.971 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-fuse. 2024-09-14T22:33:59.971 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../068-ceph-mds_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:33:59.997 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../067-ceph-fuse_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:34:00.002 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking ceph-mds (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:00.037 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-fuse (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:00.372 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package cephadm. 2024-09-14T22:34:00.390 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-mds. 2024-09-14T22:34:00.399 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../069-cephadm_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:34:00.417 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../068-ceph-mds_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:34:00.438 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking cephadm (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:00.457 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-mds (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:00.752 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-14T22:34:00.777 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-14T22:34:00.801 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package cephadm. 2024-09-14T22:34:00.816 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-14T22:34:00.816 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../069-cephadm_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:34:00.851 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking cephadm (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:01.154 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-14T22:34:01.156 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-14T22:34:01.181 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-14T22:34:01.183 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-14T22:34:01.219 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-14T22:34:01.220 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-14T22:34:01.565 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-14T22:34:01.592 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-14T22:34:01.631 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-14T22:34:02.399 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-babel. 2024-09-14T22:34:02.426 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-14T22:34:02.465 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-14T22:34:02.644 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-babel. 2024-09-14T22:34:02.661 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-14T22:34:02.694 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-14T22:34:02.768 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-jinja2. 2024-09-14T22:34:02.796 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-14T22:34:02.835 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-14T22:34:02.972 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-jinja2. 2024-09-14T22:34:02.999 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-14T22:34:03.030 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-14T22:34:03.180 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-14T22:34:03.207 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.1.1-324-gecab7be5-1jammy_all.deb ... 2024-09-14T22:34:03.246 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking ceph-mgr-cephadm (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:03.334 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-14T22:34:03.360 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.1.1-324-gecab7be5-1jammy_all.deb ... 2024-09-14T22:34:03.391 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-mgr-cephadm (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:03.574 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-14T22:34:03.601 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-09-14T22:34:03.641 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-14T22:34:03.678 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-14T22:34:03.705 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-09-14T22:34:03.736 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-14T22:34:03.944 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-routes. 2024-09-14T22:34:03.971 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-14T22:34:04.010 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-14T22:34:04.031 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-routes. 2024-09-14T22:34:04.058 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-14T22:34:04.097 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-14T22:34:04.355 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-14T22:34:04.382 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.1.1-324-gecab7be5-1jammy_all.deb ... 2024-09-14T22:34:04.421 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking ceph-mgr-dashboard (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:04.450 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-14T22:34:04.477 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.1.1-324-gecab7be5-1jammy_all.deb ... 2024-09-14T22:34:04.517 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-mgr-dashboard (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:05.685 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-14T22:34:05.701 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-14T22:34:05.736 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-14T22:34:06.038 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-14T22:34:06.054 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-14T22:34:06.088 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-14T22:34:06.213 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-joblib. 2024-09-14T22:34:06.241 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-14T22:34:06.280 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-14T22:34:06.558 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-joblib. 2024-09-14T22:34:06.574 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-14T22:34:06.616 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-14T22:34:06.650 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-14T22:34:06.677 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-14T22:34:06.716 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-14T22:34:06.961 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-14T22:34:06.988 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-14T22:34:07.002 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-sklearn. 2024-09-14T22:34:07.028 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-14T22:34:07.030 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-14T22:34:07.060 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-14T22:34:07.314 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-sklearn. 2024-09-14T22:34:07.341 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-14T22:34:07.380 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-14T22:34:07.827 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-14T22:34:07.855 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.1.1-324-gecab7be5-1jammy_all.deb ... 2024-09-14T22:34:07.987 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking ceph-mgr-diskprediction-local (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:08.089 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-14T22:34:08.117 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.1.1-324-gecab7be5-1jammy_all.deb ... 2024-09-14T22:34:08.266 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-mgr-diskprediction-local (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:08.941 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-cachetools. 2024-09-14T22:34:08.969 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-09-14T22:34:09.008 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-14T22:34:09.144 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-cachetools. 2024-09-14T22:34:09.159 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-09-14T22:34:09.202 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-14T22:34:09.319 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-14T22:34:09.346 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-14T22:34:09.385 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-14T22:34:09.505 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-14T22:34:09.532 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-14T22:34:09.571 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-14T22:34:09.739 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-14T22:34:09.766 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-14T22:34:09.805 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-14T22:34:09.933 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-14T22:34:09.960 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-14T22:34:09.999 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-14T22:34:10.116 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-rsa. 2024-09-14T22:34:10.144 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-09-14T22:34:10.183 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-14T22:34:10.294 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-rsa. 2024-09-14T22:34:10.321 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-09-14T22:34:10.360 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-14T22:34:10.470 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-google-auth. 2024-09-14T22:34:10.498 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-09-14T22:34:10.528 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-14T22:34:10.681 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-google-auth. 2024-09-14T22:34:10.708 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-09-14T22:34:10.747 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-14T22:34:10.823 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-14T22:34:10.851 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-14T22:34:10.889 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-14T22:34:11.076 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-14T22:34:11.103 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-14T22:34:11.142 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-14T22:34:11.201 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-websocket. 2024-09-14T22:34:11.228 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-09-14T22:34:11.267 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-14T22:34:11.428 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-websocket. 2024-09-14T22:34:11.456 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-09-14T22:34:11.495 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-14T22:34:11.579 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-14T22:34:11.607 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-14T22:34:11.645 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-14T22:34:11.806 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-14T22:34:11.833 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-14T22:34:11.873 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-14T22:34:12.509 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-14T22:34:12.526 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.1.1-324-gecab7be5-1jammy_all.deb ... 2024-09-14T22:34:12.545 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-14T22:34:12.559 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking ceph-mgr-k8sevents (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:12.561 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.1.1-324-gecab7be5-1jammy_all.deb ... 2024-09-14T22:34:12.603 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-mgr-k8sevents (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:12.895 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-14T22:34:12.923 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-14T22:34:12.931 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-14T22:34:12.958 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-14T22:34:12.970 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-14T22:34:12.997 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-14T22:34:13.292 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-14T22:34:13.307 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-14T22:34:13.319 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-14T22:34:13.334 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-14T22:34:13.358 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-14T22:34:13.373 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-14T22:34:13.619 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package jq. 2024-09-14T22:34:13.642 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package jq. 2024-09-14T22:34:13.646 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-14T22:34:13.670 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-14T22:34:13.686 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-14T22:34:13.709 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-14T22:34:13.938 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package socat. 2024-09-14T22:34:13.954 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-14T22:34:13.961 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package socat. 2024-09-14T22:34:13.989 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-14T22:34:14.005 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-14T22:34:14.028 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-14T22:34:14.306 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package xmlstarlet. 2024-09-14T22:34:14.323 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-14T22:34:14.356 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-14T22:34:14.358 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package xmlstarlet. 2024-09-14T22:34:14.386 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-14T22:34:14.433 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-14T22:34:14.704 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package ceph-test. 2024-09-14T22:34:14.732 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../097-ceph-test_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:34:14.770 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking ceph-test (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:14.804 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-test. 2024-09-14T22:34:14.831 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../097-ceph-test_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:34:14.870 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-test (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:17.200 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-volume. 2024-09-14T22:34:17.227 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../098-ceph-volume_19.1.1-324-gecab7be5-1jammy_all.deb ... 2024-09-14T22:34:17.258 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-volume (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:17.391 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package ceph-volume. 2024-09-14T22:34:17.419 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../098-ceph-volume_19.1.1-324-gecab7be5-1jammy_all.deb ... 2024-09-14T22:34:17.458 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking ceph-volume (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:17.536 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-14T22:34:17.552 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../099-libcephfs-dev_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:34:17.586 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libcephfs-dev (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:17.779 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-14T22:34:17.807 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../099-libcephfs-dev_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:34:17.845 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking libcephfs-dev (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:17.847 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-14T22:34:17.875 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-14T22:34:17.913 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-14T22:34:18.123 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-14T22:34:18.139 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-14T22:34:18.172 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-14T22:34:18.293 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-14T22:34:18.321 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-09-14T22:34:18.360 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-14T22:34:18.460 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-14T22:34:18.488 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-09-14T22:34:18.527 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-14T22:34:18.738 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package nvme-cli. 2024-09-14T22:34:18.766 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-14T22:34:18.805 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-14T22:34:18.898 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package nvme-cli. 2024-09-14T22:34:18.926 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-14T22:34:18.964 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-14T22:34:19.229 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package pkg-config. 2024-09-14T22:34:19.256 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-14T22:34:19.295 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-14T22:34:19.394 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package pkg-config. 2024-09-14T22:34:19.422 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-14T22:34:19.460 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-14T22:34:19.547 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-14T22:34:19.563 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-14T22:34:19.606 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-14T22:34:19.713 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-14T22:34:19.730 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-14T22:34:19.763 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-14T22:34:20.051 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-14T22:34:20.078 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-14T22:34:20.117 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-14T22:34:20.183 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-14T22:34:20.211 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-14T22:34:20.250 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-14T22:34:20.403 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-pastescript. 2024-09-14T22:34:20.431 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-09-14T22:34:20.469 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-14T22:34:20.494 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-pastescript. 2024-09-14T22:34:20.510 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-09-14T22:34:20.544 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-14T22:34:20.789 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-pluggy. 2024-09-14T22:34:20.817 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-14T22:34:20.856 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-pluggy. 2024-09-14T22:34:20.856 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-14T22:34:20.884 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-14T22:34:20.922 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-14T22:34:21.142 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-psutil. 2024-09-14T22:34:21.170 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-14T22:34:21.259 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-psutil. 2024-09-14T22:34:21.286 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-14T22:34:21.325 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-14T22:34:21.432 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-14T22:34:21.653 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-py. 2024-09-14T22:34:21.681 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-09-14T22:34:21.719 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-14T22:34:21.845 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-py. 2024-09-14T22:34:21.872 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-09-14T22:34:21.911 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-14T22:34:22.048 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-pygments. 2024-09-14T22:34:22.076 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-14T22:34:22.114 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-14T22:34:22.231 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-pygments. 2024-09-14T22:34:22.259 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-14T22:34:22.297 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-14T22:34:22.585 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-14T22:34:22.613 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-14T22:34:22.777 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-14T22:34:22.787 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-14T22:34:22.805 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-14T22:34:22.843 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-14T22:34:23.146 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-toml. 2024-09-14T22:34:23.173 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-09-14T22:34:23.179 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-toml. 2024-09-14T22:34:23.208 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-09-14T22:34:23.212 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-14T22:34:23.245 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-14T22:34:23.516 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-pytest. 2024-09-14T22:34:23.543 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-14T22:34:23.548 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-pytest. 2024-09-14T22:34:23.577 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-14T22:34:23.581 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-14T22:34:23.615 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-14T22:34:23.918 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-simplejson. 2024-09-14T22:34:23.935 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-simplejson. 2024-09-14T22:34:23.935 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-14T22:34:23.963 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-14T22:34:23.976 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-14T22:34:24.001 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-14T22:34:24.229 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-14T22:34:24.246 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-14T22:34:24.270 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-14T22:34:24.287 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-14T22:34:24.287 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-14T22:34:24.328 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-14T22:34:25.025 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package radosgw. 2024-09-14T22:34:25.054 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../116-radosgw_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:34:25.091 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking radosgw (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:25.116 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package radosgw. 2024-09-14T22:34:25.144 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../116-radosgw_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:34:25.182 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking radosgw (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:25.968 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package rbd-fuse. 2024-09-14T22:34:25.985 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../117-rbd-fuse_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:34:25.986 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package rbd-fuse. 2024-09-14T22:34:26.014 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../117-rbd-fuse_19.1.1-324-gecab7be5-1jammy_amd64.deb ... 2024-09-14T22:34:26.018 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking rbd-fuse (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:26.052 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking rbd-fuse (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:26.331 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-14T22:34:26.345 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-14T22:34:26.598 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-14T22:34:26.599 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-14T22:34:26.699 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-14T22:34:26.725 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-14T22:34:26.944 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-14T22:34:26.994 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-14T22:34:27.273 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-14T22:34:27.313 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-14T22:34:27.682 INFO:teuthology.orchestra.run.smithi176.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-14T22:34:27.715 INFO:teuthology.orchestra.run.smithi001.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-14T22:34:27.758 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up cephadm (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:27.785 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up cephadm (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:28.319 INFO:teuthology.orchestra.run.smithi001.stdout:Adding system user cephadm....done 2024-09-14T22:34:28.325 INFO:teuthology.orchestra.run.smithi176.stdout:Adding system user cephadm....done 2024-09-14T22:34:28.473 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-14T22:34:28.474 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-14T22:34:28.733 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-14T22:34:28.826 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-14T22:34:28.984 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-14T22:34:29.085 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-14T22:34:29.093 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-14T22:34:29.211 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-14T22:34:29.327 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-14T22:34:29.488 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-14T22:34:29.587 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-14T22:34:29.713 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-14T22:34:29.722 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-14T22:34:29.823 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-14T22:34:30.014 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-14T22:34:30.125 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-14T22:34:30.367 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-14T22:34:30.518 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-14T22:34:30.626 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-14T22:34:30.786 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-14T22:34:30.902 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-14T22:34:31.062 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-14T22:34:31.145 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-14T22:34:31.297 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-14T22:34:31.396 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-14T22:34:31.548 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-14T22:34:31.673 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-14T22:34:31.807 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-14T22:34:31.923 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-ceph-argparse (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:32.067 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-ceph-argparse (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:32.166 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-14T22:34:32.292 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-14T22:34:32.410 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-14T22:34:32.468 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-14T22:34:32.519 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-14T22:34:32.594 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-14T22:34:32.720 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-14T22:34:32.778 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-14T22:34:32.854 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-14T22:34:33.089 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-14T22:34:33.187 INFO:teuthology.orchestra.run.smithi176.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-14T22:34:33.217 INFO:teuthology.orchestra.run.smithi176.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-14T22:34:33.273 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-14T22:34:33.374 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-14T22:34:33.716 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-14T22:34:33.731 INFO:teuthology.orchestra.run.smithi001.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-14T22:34:33.787 INFO:teuthology.orchestra.run.smithi001.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-14T22:34:33.850 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-14T22:34:33.851 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-14T22:34:33.976 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-14T22:34:34.102 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-14T22:34:34.344 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-14T22:34:34.369 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-14T22:34:34.470 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-14T22:34:34.671 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up zip (3.0-12build2) ... 2024-09-14T22:34:34.755 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-14T22:34:34.821 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-14T22:34:35.006 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-14T22:34:35.324 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up zip (3.0-12build2) ... 2024-09-14T22:34:35.449 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-14T22:34:35.515 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-14T22:34:35.766 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-14T22:34:35.892 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-14T22:34:36.026 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-14T22:34:36.168 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-14T22:34:36.327 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-14T22:34:36.427 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-14T22:34:36.545 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-14T22:34:36.654 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-14T22:34:36.687 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-14T22:34:36.947 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-14T22:34:37.063 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-14T22:34:37.357 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-14T22:34:37.447 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-14T22:34:37.624 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-14T22:34:37.724 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-14T22:34:38.009 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-14T22:34:38.067 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-14T22:34:38.302 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-14T22:34:38.335 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-14T22:34:38.435 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-ceph-common (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:38.653 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-14T22:34:38.728 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-14T22:34:38.913 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-14T22:34:39.022 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-ceph-common (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:39.046 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-14T22:34:39.323 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-14T22:34:39.650 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-14T22:34:40.151 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-14T22:34:40.386 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T22:34:40.521 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-14T22:34:40.788 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-14T22:34:40.880 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-14T22:34:40.897 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-14T22:34:41.007 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-14T22:34:41.107 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T22:34:41.241 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-14T22:34:41.283 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-14T22:34:41.509 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-14T22:34:41.526 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-14T22:34:41.660 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-14T22:34:41.785 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T22:34:41.786 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-14T22:34:41.895 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-14T22:34:42.110 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-14T22:34:42.321 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-14T22:34:42.355 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-14T22:34:42.597 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-14T22:34:42.598 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T22:34:42.715 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-14T22:34:42.848 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-14T22:34:43.042 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-14T22:34:43.099 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-14T22:34:43.301 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-14T22:34:43.341 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-14T22:34:43.577 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-14T22:34:43.618 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-14T22:34:43.744 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-14T22:34:43.845 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-14T22:34:44.028 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-14T22:34:44.088 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-14T22:34:44.154 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-14T22:34:44.347 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-14T22:34:44.455 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-14T22:34:44.473 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-14T22:34:44.773 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-14T22:34:44.824 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-14T22:34:44.950 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-14T22:34:45.015 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-14T22:34:45.116 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-14T22:34:45.226 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-14T22:34:45.251 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-14T22:34:45.316 INFO:teuthology.orchestra.run.smithi176.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-14T22:34:45.586 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-14T22:34:45.586 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-14T22:34:45.711 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-14T22:34:45.845 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-14T22:34:45.963 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-14T22:34:46.072 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-14T22:34:46.105 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-14T22:34:46.161 INFO:teuthology.orchestra.run.smithi001.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-14T22:34:46.355 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-14T22:34:46.399 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-14T22:34:46.481 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-14T22:34:46.525 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-14T22:34:46.724 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T22:34:46.850 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-14T22:34:46.935 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-14T22:34:47.172 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-14T22:34:47.203 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-14T22:34:47.329 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-14T22:34:47.469 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-14T22:34:47.580 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T22:34:47.681 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-14T22:34:47.745 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-14T22:34:47.871 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-14T22:34:47.990 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-14T22:34:48.139 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-14T22:34:48.299 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-14T22:34:48.406 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-14T22:34:48.584 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-14T22:34:48.710 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-14T22:34:48.751 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-14T22:34:48.994 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-14T22:34:49.128 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-14T22:34:49.254 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-14T22:34:49.412 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-14T22:34:49.590 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-14T22:34:49.748 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-14T22:34:49.857 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up librados2 (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:49.958 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libsqlite3-mod-ceph (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:49.992 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-14T22:34:50.076 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-14T22:34:50.285 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-14T22:34:50.412 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libcephfs2 (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:50.530 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libradosstriper1 (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:50.630 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-14T22:34:50.631 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-14T22:34:50.739 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up librados2 (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:50.865 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up libsqlite3-mod-ceph (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:50.899 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up librbd1 (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:50.991 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-14T22:34:51.000 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-mgr-modules-core (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:51.109 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-fuse (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:51.324 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-14T22:34:51.325 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-14T22:34:51.327 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up libcephfs2 (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:51.453 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up libradosstriper1 (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:51.562 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-14T22:34:51.815 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up librbd1 (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:51.877 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libcephfs-dev (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:51.923 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up ceph-mgr-modules-core (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:52.037 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-rados (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:52.049 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up ceph-fuse (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:52.163 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up librgw2 (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:52.264 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-14T22:34:52.270 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-14T22:34:52.270 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-14T22:34:52.759 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up libcephfs-dev (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:52.885 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-rados (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:53.011 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up librgw2 (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:53.153 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-14T22:34:53.435 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-rbd (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:53.569 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up rbd-fuse (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:53.695 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-rgw (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:53.821 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-cephfs (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:53.930 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-common (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:54.439 INFO:teuthology.orchestra.run.smithi176.stdout:Adding group ceph....done 2024-09-14T22:34:54.441 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-rbd (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:54.567 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up rbd-fuse (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:54.693 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-rgw (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:54.818 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-cephfs (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:54.841 INFO:teuthology.orchestra.run.smithi176.stdout:Adding system user ceph....done 2024-09-14T22:34:54.944 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up ceph-common (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:54.958 INFO:teuthology.orchestra.run.smithi176.stdout:Setting system user ceph properties....done 2024-09-14T22:34:54.974 INFO:teuthology.orchestra.run.smithi176.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-14T22:34:55.107 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-14T22:34:55.424 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-14T22:34:55.454 INFO:teuthology.orchestra.run.smithi001.stdout:Adding group ceph....done 2024-09-14T22:34:55.790 INFO:teuthology.orchestra.run.smithi001.stdout:Adding system user ceph....done 2024-09-14T22:34:55.872 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-test (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:55.957 INFO:teuthology.orchestra.run.smithi001.stdout:Setting system user ceph properties....done 2024-09-14T22:34:55.974 INFO:teuthology.orchestra.run.smithi001.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-14T22:34:55.981 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up radosgw (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:56.117 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-14T22:34:56.401 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-14T22:34:56.401 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-14T22:34:56.466 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-14T22:34:56.875 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-base (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:56.921 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up ceph-test (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:57.038 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up radosgw (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:57.220 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-14T22:34:57.485 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-14T22:34:57.486 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-14T22:34:57.765 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-mds (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:57.898 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up ceph-base (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:57.983 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-14T22:34:57.983 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-14T22:34:58.273 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-14T22:34:58.457 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-mgr (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:58.654 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-14T22:34:58.654 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-14T22:34:58.734 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up ceph-mds (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:58.985 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-14T22:34:58.985 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-14T22:34:59.117 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-osd (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:59.389 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-14T22:34:59.389 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-14T22:34:59.670 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up ceph-mgr (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:59.868 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-mgr-k8sevents (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:34:59.885 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-14T22:34:59.885 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-14T22:34:59.994 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-mgr-diskprediction-local (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:35:00.145 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-mon (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:35:00.330 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up ceph-osd (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:35:00.356 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-14T22:35:00.357 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-14T22:35:00.624 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-14T22:35:00.624 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-14T22:35:00.821 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-mgr-cephadm (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:35:00.939 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:35:01.040 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-mgr-dashboard (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:35:01.115 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up ceph-mgr-k8sevents (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:35:01.183 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-volume (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:35:01.241 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up ceph-mgr-diskprediction-local (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:35:01.392 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up ceph-mon (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:35:01.544 INFO:teuthology.orchestra.run.smithi176.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-14T22:35:01.613 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-14T22:35:01.613 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-14T22:35:01.653 INFO:teuthology.orchestra.run.smithi176.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-14T22:35:01.805 INFO:teuthology.orchestra.run.smithi176.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-14T22:35:02.110 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up ceph-mgr-cephadm (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:35:02.286 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up ceph (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:35:02.387 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up ceph-mgr-dashboard (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:35:02.521 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up ceph-volume (19.1.1-324-gecab7be5-1jammy) ... 2024-09-14T22:35:02.966 INFO:teuthology.orchestra.run.smithi001.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-14T22:35:03.115 INFO:teuthology.orchestra.run.smithi176.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-14T22:35:03.115 INFO:teuthology.orchestra.run.smithi001.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-14T22:35:03.202 INFO:teuthology.orchestra.run.smithi001.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-14T22:35:03.720 INFO:teuthology.orchestra.run.smithi176.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-14T22:35:03.725 DEBUG:teuthology.parallel:result is None 2024-09-14T22:35:04.261 INFO:teuthology.orchestra.run.smithi001.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-14T22:35:05.225 INFO:teuthology.orchestra.run.smithi001.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-14T22:35:05.230 DEBUG:teuthology.parallel:result is None 2024-09-14T22:35:05.231 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ecab7be58963a5bd03910cd73eef66405d7fdd50 2024-09-14T22:35:05.380 DEBUG:teuthology.orchestra.run.smithi001:> dpkg-query -W -f '${Version}' ceph 2024-09-14T22:35:05.399 INFO:teuthology.orchestra.run.smithi001.stdout:19.1.1-324-gecab7be5-1jammy 2024-09-14T22:35:05.400 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-324-gecab7be5-1jammy 2024-09-14T22:35:05.400 INFO:teuthology.task.install:The correct ceph version 19.1.1-324-gecab7be5-1jammy is installed. 2024-09-14T22:35:05.402 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ecab7be58963a5bd03910cd73eef66405d7fdd50 2024-09-14T22:35:05.542 DEBUG:teuthology.orchestra.run.smithi176:> dpkg-query -W -f '${Version}' ceph 2024-09-14T22:35:05.561 INFO:teuthology.orchestra.run.smithi176.stdout:19.1.1-324-gecab7be5-1jammy 2024-09-14T22:35:05.561 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-324-gecab7be5-1jammy 2024-09-14T22:35:05.562 INFO:teuthology.task.install:The correct ceph version 19.1.1-324-gecab7be5-1jammy is installed. 2024-09-14T22:35:05.563 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-14T22:35:05.563 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-14T22:35:05.564 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-14T22:35:05.578 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T22:35:05.578 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-14T22:35:05.616 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-14T22:35:05.616 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-14T22:35:05.616 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/daemon-helper 2024-09-14T22:35:05.635 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-14T22:35:05.691 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T22:35:05.691 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/usr/bin/daemon-helper 2024-09-14T22:35:05.705 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-14T22:35:05.759 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-14T22:35:05.759 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-14T22:35:05.759 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-14T22:35:05.773 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-14T22:35:05.827 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T22:35:05.827 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-14T22:35:05.840 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-14T22:35:05.894 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-14T22:35:05.894 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-14T22:35:05.894 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/stdin-killer 2024-09-14T22:35:05.908 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-14T22:35:05.963 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T22:35:05.963 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/usr/bin/stdin-killer 2024-09-14T22:35:05.977 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-14T22:35:06.030 INFO:teuthology.run_tasks:Running task ceph... 2024-09-14T22:35:06.108 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-14T22:35:06.108 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 777 /var/log/ceph 2024-09-14T22:35:06.111 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod 777 /var/log/ceph 2024-09-14T22:35:06.124 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-14T22:35:06.124 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-14T22:35:06.164 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-14T22:35:06.178 INFO:tasks.ceph:Creating extra log directories... 2024-09-14T22:35:06.178 DEBUG:teuthology.orchestra.run.smithi001:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-14T22:35:06.220 DEBUG:teuthology.orchestra.run.smithi176:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-14T22:35:06.242 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-14T22:35:06.242 INFO:tasks.ceph:config {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writearound'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-14T22:35:06.242 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904719', 'branch': 'squid', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writearound}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7904719', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writearound'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 4271, 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50', 'sleep_before_teardown': 0, 'subset': '97/128', 'suite': 'rbd', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_ecab7be58963a5bd03910cd73eef66405d7fdd50/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50', 'targets': {'smithi001.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCR0nib8SBVTiYN6fbNaFSeWtuJX68mc9fQqPDMtsUK3EZnmiq8jqxLeiJVDpCo3VXVQRWgNkwq3386ra8fU4zg=', 'smithi176.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKgtHjyzrLcUMjK57Ts2qKmxNrHdr33LIQFAb0TmaqkgGVm1Wt69/45cHSFBegPAq1DSuamUVF/g7grXSFST+kM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-14_21:16:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-14T22:35:06.242 DEBUG:teuthology.orchestra.run.smithi001:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-14T22:35:06.276 DEBUG:teuthology.orchestra.run.smithi176:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-14T22:35:06.281 DEBUG:teuthology.orchestra.run.smithi001:> sudo install -d -m0777 -- /var/run/ceph 2024-09-14T22:35:06.324 DEBUG:teuthology.orchestra.run.smithi176:> sudo install -d -m0777 -- /var/run/ceph 2024-09-14T22:35:06.337 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-14T22:35:06.337 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/scratch_devs of=/dev/stdout 2024-09-14T22:35:06.383 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-14T22:35:06.383 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_1 2024-09-14T22:35:06.431 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-14T22:35:06.432 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T22:35:06.432 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 5h/5d Inode: 871 Links: 1 2024-09-14T22:35:06.432 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T22:35:06.432 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-09-14 22:29:38.047244316 +0000 2024-09-14T22:35:06.432 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-09-14 22:29:37.755246925 +0000 2024-09-14T22:35:06.432 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-09-14 22:29:37.755246925 +0000 2024-09-14T22:35:06.432 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: - 2024-09-14T22:35:06.432 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-14T22:35:06.486 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-09-14T22:35:06.486 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-09-14T22:35:06.486 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000317941 s, 1.6 MB/s 2024-09-14T22:35:06.488 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-14T22:35:06.537 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_2 2024-09-14T22:35:06.583 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-14T22:35:06.583 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T22:35:06.584 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 5h/5d Inode: 891 Links: 1 2024-09-14T22:35:06.584 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T22:35:06.584 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-09-14 22:29:38.307241995 +0000 2024-09-14T22:35:06.584 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-09-14 22:29:38.303242031 +0000 2024-09-14T22:35:06.584 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-09-14 22:29:38.303242031 +0000 2024-09-14T22:35:06.584 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: - 2024-09-14T22:35:06.584 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-14T22:35:06.637 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-09-14T22:35:06.638 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-09-14T22:35:06.638 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000303022 s, 1.7 MB/s 2024-09-14T22:35:06.639 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-14T22:35:06.688 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_3 2024-09-14T22:35:06.735 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-14T22:35:06.736 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T22:35:06.736 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 5h/5d Inode: 906 Links: 1 2024-09-14T22:35:06.736 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T22:35:06.736 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-09-14 22:29:39.039235456 +0000 2024-09-14T22:35:06.736 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-09-14 22:29:38.779237778 +0000 2024-09-14T22:35:06.736 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-09-14 22:29:38.779237778 +0000 2024-09-14T22:35:06.736 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: - 2024-09-14T22:35:06.736 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-14T22:35:06.790 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-09-14T22:35:06.790 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-09-14T22:35:06.790 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000302398 s, 1.7 MB/s 2024-09-14T22:35:06.792 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-14T22:35:06.840 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_4 2024-09-14T22:35:06.888 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-14T22:35:06.888 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T22:35:06.888 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 5h/5d Inode: 928 Links: 1 2024-09-14T22:35:06.888 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T22:35:06.888 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-09-14 22:29:39.531231062 +0000 2024-09-14T22:35:06.888 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-09-14 22:29:39.275233348 +0000 2024-09-14T22:35:06.890 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-09-14 22:29:39.275233348 +0000 2024-09-14T22:35:06.890 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: - 2024-09-14T22:35:06.890 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-14T22:35:06.942 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-09-14T22:35:06.942 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-09-14T22:35:06.942 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000306126 s, 1.7 MB/s 2024-09-14T22:35:06.943 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-14T22:35:06.992 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T22:35:06.992 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T22:35:06.993 DEBUG:teuthology.orchestra.run.smithi176:> dd if=/scratch_devs of=/dev/stdout 2024-09-14T22:35:06.998 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-14T22:35:06.998 DEBUG:teuthology.orchestra.run.smithi176:> stat /dev/vg_nvme/lv_1 2024-09-14T22:35:07.044 INFO:teuthology.orchestra.run.smithi176.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-14T22:35:07.044 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T22:35:07.045 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 5h/5d Inode: 878 Links: 1 2024-09-14T22:35:07.045 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T22:35:07.045 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 2024-09-14 22:29:34.094313735 +0000 2024-09-14T22:35:07.045 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2024-09-14 22:29:33.850315913 +0000 2024-09-14T22:35:07.045 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2024-09-14 22:29:33.850315913 +0000 2024-09-14T22:35:07.045 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: - 2024-09-14T22:35:07.045 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-14T22:35:07.098 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records in 2024-09-14T22:35:07.098 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records out 2024-09-14T22:35:07.098 INFO:teuthology.orchestra.run.smithi176.stderr:512 bytes copied, 0.000386956 s, 1.3 MB/s 2024-09-14T22:35:07.099 DEBUG:teuthology.orchestra.run.smithi176:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-14T22:35:07.149 DEBUG:teuthology.orchestra.run.smithi176:> stat /dev/vg_nvme/lv_2 2024-09-14T22:35:07.195 INFO:teuthology.orchestra.run.smithi176.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-14T22:35:07.195 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T22:35:07.195 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 5h/5d Inode: 898 Links: 1 2024-09-14T22:35:07.196 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T22:35:07.196 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 2024-09-14 22:29:34.594309268 +0000 2024-09-14T22:35:07.196 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2024-09-14 22:29:34.350311447 +0000 2024-09-14T22:35:07.196 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2024-09-14 22:29:34.350311447 +0000 2024-09-14T22:35:07.196 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: - 2024-09-14T22:35:07.196 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-14T22:35:07.250 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records in 2024-09-14T22:35:07.251 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records out 2024-09-14T22:35:07.251 INFO:teuthology.orchestra.run.smithi176.stderr:512 bytes copied, 0.000444992 s, 1.2 MB/s 2024-09-14T22:35:07.252 DEBUG:teuthology.orchestra.run.smithi176:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-14T22:35:07.301 DEBUG:teuthology.orchestra.run.smithi176:> stat /dev/vg_nvme/lv_3 2024-09-14T22:35:07.348 INFO:teuthology.orchestra.run.smithi176.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-14T22:35:07.348 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T22:35:07.348 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 5h/5d Inode: 908 Links: 1 2024-09-14T22:35:07.349 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T22:35:07.349 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 2024-09-14 22:29:35.070305017 +0000 2024-09-14T22:35:07.349 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2024-09-14 22:29:34.834307125 +0000 2024-09-14T22:35:07.349 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2024-09-14 22:29:34.834307125 +0000 2024-09-14T22:35:07.349 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: - 2024-09-14T22:35:07.349 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-14T22:35:07.402 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records in 2024-09-14T22:35:07.402 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records out 2024-09-14T22:35:07.402 INFO:teuthology.orchestra.run.smithi176.stderr:512 bytes copied, 0.000421417 s, 1.2 MB/s 2024-09-14T22:35:07.403 DEBUG:teuthology.orchestra.run.smithi176:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-14T22:35:07.453 DEBUG:teuthology.orchestra.run.smithi176:> stat /dev/vg_nvme/lv_4 2024-09-14T22:35:07.499 INFO:teuthology.orchestra.run.smithi176.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-14T22:35:07.499 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T22:35:07.500 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 5h/5d Inode: 931 Links: 1 2024-09-14T22:35:07.500 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T22:35:07.500 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 2024-09-14 22:29:35.602300264 +0000 2024-09-14T22:35:07.500 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2024-09-14 22:29:35.326302729 +0000 2024-09-14T22:35:07.500 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2024-09-14 22:29:35.326302729 +0000 2024-09-14T22:35:07.500 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: - 2024-09-14T22:35:07.500 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-14T22:35:07.554 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records in 2024-09-14T22:35:07.554 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records out 2024-09-14T22:35:07.554 INFO:teuthology.orchestra.run.smithi176.stderr:512 bytes copied, 0.000327012 s, 1.6 MB/s 2024-09-14T22:35:07.555 DEBUG:teuthology.orchestra.run.smithi176:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-14T22:35:07.605 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T22:35:07.605 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi001.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi176.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-14T22:35:07.605 INFO:tasks.ceph:Generating config... 2024-09-14T22:35:07.606 INFO:tasks.ceph:[client] rbd cache = True 2024-09-14T22:35:07.606 INFO:tasks.ceph:[client] rbd cache policy = writearound 2024-09-14T22:35:07.606 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-14T22:35:07.606 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-14T22:35:07.607 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-14T22:35:07.607 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-14T22:35:07.607 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-14T22:35:07.607 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-14T22:35:07.607 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-14T22:35:07.607 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-14T22:35:07.607 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-14T22:35:07.607 INFO:tasks.ceph:[osd] bluestore compression algorithm = zstd 2024-09-14T22:35:07.607 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-14T22:35:07.607 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-14T22:35:07.607 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-14T22:35:07.607 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-14T22:35:07.607 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-14T22:35:07.607 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-14T22:35:07.608 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-14T22:35:07.608 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-14T22:35:07.608 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-14T22:35:07.608 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-14T22:35:07.608 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-14T22:35:07.608 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-14T22:35:07.608 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-14T22:35:07.608 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-14T22:35:07.608 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-14T22:35:07.608 INFO:tasks.ceph:Setting up mon.a... 2024-09-14T22:35:07.608 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-14T22:35:07.643 INFO:teuthology.orchestra.run.smithi001.stdout:creating /etc/ceph/ceph.keyring 2024-09-14T22:35:07.659 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-14T22:35:07.741 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-14T22:35:07.794 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.1'), ('mon.c', '[v2:172.21.15.1:3301,v1:172.21.15.1:6790]'), ('mon.b', '172.21.15.176')] 2024-09-14T22:35:07.794 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.1,[v2:172.21.15.1:3301,v1:172.21.15.1:6790],172.21.15.176', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd cache': True, 'rbd cache policy': 'writearound'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-14T22:35:07.795 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-14T22:35:07.795 DEBUG:teuthology.orchestra.run.smithi001:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-14T22:35:07.840 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.1 --addv c '[v2:172.21.15.1:3301,v1:172.21.15.1:6790]' --add b 172.21.15.176 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-14T22:35:07.925 INFO:teuthology.orchestra.run.smithi001.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-14T22:35:07.925 INFO:teuthology.orchestra.run.smithi001.stdout:monmaptool: generated fsid f07f407a-b17d-42c4-bf73-92dd80114a4f 2024-09-14T22:35:07.925 INFO:teuthology.orchestra.run.smithi001.stdout:setting min_mon_release = quincy 2024-09-14T22:35:07.925 INFO:teuthology.orchestra.run.smithi001.stdout:epoch 0 2024-09-14T22:35:07.925 INFO:teuthology.orchestra.run.smithi001.stdout:fsid f07f407a-b17d-42c4-bf73-92dd80114a4f 2024-09-14T22:35:07.925 INFO:teuthology.orchestra.run.smithi001.stdout:last_changed 2024-09-14T22:35:07.923922+0000 2024-09-14T22:35:07.925 INFO:teuthology.orchestra.run.smithi001.stdout:created 2024-09-14T22:35:07.923922+0000 2024-09-14T22:35:07.925 INFO:teuthology.orchestra.run.smithi001.stdout:min_mon_release 17 (quincy) 2024-09-14T22:35:07.925 INFO:teuthology.orchestra.run.smithi001.stdout:election_strategy: 1 2024-09-14T22:35:07.925 INFO:teuthology.orchestra.run.smithi001.stdout:0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-09-14T22:35:07.925 INFO:teuthology.orchestra.run.smithi001.stdout:1: [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] mon.b 2024-09-14T22:35:07.926 INFO:teuthology.orchestra.run.smithi001.stdout:2: [v2:172.21.15.1:3301/0,v1:172.21.15.1:6790/0] mon.c 2024-09-14T22:35:07.926 INFO:teuthology.orchestra.run.smithi001.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-14T22:35:07.927 DEBUG:teuthology.orchestra.run.smithi001:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-14T22:35:07.976 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID f07f407a-b17d-42c4-bf73-92dd80114a4f... 2024-09-14T22:35:07.977 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-14T22:35:08.020 DEBUG:teuthology.orchestra.run.smithi176:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-14T22:35:08.050 INFO:teuthology.orchestra.run.smithi001.stdout:[global] 2024-09-14T22:35:08.050 INFO:teuthology.orchestra.run.smithi001.stdout: chdir = "" 2024-09-14T22:35:08.050 INFO:teuthology.orchestra.run.smithi001.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-14T22:35:08.050 INFO:teuthology.orchestra.run.smithi001.stdout: auth supported = cephx 2024-09-14T22:35:08.050 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:35:08.050 INFO:teuthology.orchestra.run.smithi001.stdout: filestore xattr use omap = true 2024-09-14T22:35:08.050 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:35:08.050 INFO:teuthology.orchestra.run.smithi001.stdout: mon clock drift allowed = 1.000 2024-09-14T22:35:08.051 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:35:08.051 INFO:teuthology.orchestra.run.smithi001.stdout: osd crush chooseleaf type = 0 2024-09-14T22:35:08.051 INFO:teuthology.orchestra.run.smithi001.stdout: auth debug = true 2024-09-14T22:35:08.051 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:35:08.051 INFO:teuthology.orchestra.run.smithi001.stdout: ms die on old message = true 2024-09-14T22:35:08.051 INFO:teuthology.orchestra.run.smithi001.stdout: ms die on bug = true 2024-09-14T22:35:08.051 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:35:08.051 INFO:teuthology.orchestra.run.smithi001.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-14T22:35:08.051 INFO:teuthology.orchestra.run.smithi001.stdout: mon pg warn max object skew = 0 2024-09-14T22:35:08.051 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:35:08.052 INFO:teuthology.orchestra.run.smithi001.stdout: # disable pg_autoscaler by default for new pools 2024-09-14T22:35:08.052 INFO:teuthology.orchestra.run.smithi001.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-14T22:35:08.052 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:35:08.052 INFO:teuthology.orchestra.run.smithi001.stdout: osd pool default size = 2 2024-09-14T22:35:08.052 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:35:08.052 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd allow primary affinity = true 2024-09-14T22:35:08.052 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd allow pg remap = true 2024-09-14T22:35:08.052 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on legacy crush tunables = false 2024-09-14T22:35:08.052 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on crush straw calc version zero = false 2024-09-14T22:35:08.053 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on no sortbitwise = false 2024-09-14T22:35:08.053 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on osd down out interval zero = false 2024-09-14T22:35:08.053 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on too few osds = false 2024-09-14T22:35:08.053 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-14T22:35:08.053 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-14T22:35:08.053 INFO:teuthology.orchestra.run.smithi001.stdout: mon_allow_pool_size_one = true 2024-09-14T22:35:08.053 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:35:08.053 INFO:teuthology.orchestra.run.smithi001.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-14T22:35:08.053 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:35:08.053 INFO:teuthology.orchestra.run.smithi001.stdout: osd default data pool replay window = 5 2024-09-14T22:35:08.053 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:35:08.054 INFO:teuthology.orchestra.run.smithi001.stdout: mon allow pool delete = true 2024-09-14T22:35:08.054 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:35:08.054 INFO:teuthology.orchestra.run.smithi001.stdout: mon cluster log file level = debug 2024-09-14T22:35:08.054 INFO:teuthology.orchestra.run.smithi001.stdout: debug asserts on shutdown = true 2024-09-14T22:35:08.054 INFO:teuthology.orchestra.run.smithi001.stdout: mon health detail to clog = false 2024-09-14T22:35:08.054 INFO:teuthology.orchestra.run.smithi001.stdout: mon host = "172.21.15.1,[v2:172.21.15.1:3301,v1:172.21.15.1:6790],172.21.15.176" 2024-09-14T22:35:08.054 INFO:teuthology.orchestra.run.smithi001.stdout: mon client directed command retry = 5 2024-09-14T22:35:08.054 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on pool no app = False 2024-09-14T22:35:08.054 INFO:teuthology.orchestra.run.smithi001.stdout: ms inject socket failures = 5000 2024-09-14T22:35:08.054 INFO:teuthology.orchestra.run.smithi001.stdout: fsid = f07f407a-b17d-42c4-bf73-92dd80114a4f 2024-09-14T22:35:08.054 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:35:08.054 INFO:teuthology.orchestra.run.smithi001.stdout:[osd] 2024-09-14T22:35:08.054 INFO:teuthology.orchestra.run.smithi001.stdout: osd journal size = 100 2024-09-14T22:35:08.055 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:35:08.055 INFO:teuthology.orchestra.run.smithi001.stdout: osd scrub load threshold = 5.0 2024-09-14T22:35:08.055 INFO:teuthology.orchestra.run.smithi001.stdout: osd scrub max interval = 600 2024-09-14T22:35:08.055 INFO:teuthology.orchestra.run.smithi001.stdout: osd mclock profile = high_recovery_ops 2024-09-14T22:35:08.055 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:35:08.055 INFO:teuthology.orchestra.run.smithi001.stdout: osd recover clone overlap = true 2024-09-14T22:35:08.055 INFO:teuthology.orchestra.run.smithi001.stdout: osd recovery max chunk = 1048576 2024-09-14T22:35:08.055 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:35:08.055 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug shutdown = true 2024-09-14T22:35:08.055 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug op order = true 2024-09-14T22:35:08.055 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug verify stray on activate = true 2024-09-14T22:35:08.055 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug trim objects = true 2024-09-14T22:35:08.055 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:35:08.056 INFO:teuthology.orchestra.run.smithi001.stdout: osd open classes on start = true 2024-09-14T22:35:08.056 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug pg log writeout = true 2024-09-14T22:35:08.056 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:35:08.056 INFO:teuthology.orchestra.run.smithi001.stdout: osd deep scrub update digest min age = 30 2024-09-14T22:35:08.056 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:35:08.056 INFO:teuthology.orchestra.run.smithi001.stdout: osd map max advance = 10 2024-09-14T22:35:08.056 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:35:08.056 INFO:teuthology.orchestra.run.smithi001.stdout: journal zero on create = true 2024-09-14T22:35:08.056 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:35:08.056 INFO:teuthology.orchestra.run.smithi001.stdout: filestore ondisk finisher threads = 3 2024-09-14T22:35:08.056 INFO:teuthology.orchestra.run.smithi001.stdout: filestore apply finisher threads = 3 2024-09-14T22:35:08.056 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:35:08.056 INFO:teuthology.orchestra.run.smithi001.stdout: bdev debug aio = true 2024-09-14T22:35:08.057 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug misdirected ops = true 2024-09-14T22:35:08.057 INFO:teuthology.orchestra.run.smithi001.stdout: bluestore block size = 96636764160 2024-09-14T22:35:08.057 INFO:teuthology.orchestra.run.smithi001.stdout: bluestore compression algorithm = zstd 2024-09-14T22:35:08.057 INFO:teuthology.orchestra.run.smithi001.stdout: bluestore compression mode = aggressive 2024-09-14T22:35:08.057 INFO:teuthology.orchestra.run.smithi001.stdout: bluestore fsck on mount = True 2024-09-14T22:35:08.057 INFO:teuthology.orchestra.run.smithi001.stdout: debug bluefs = 1/20 2024-09-14T22:35:08.057 INFO:teuthology.orchestra.run.smithi001.stdout: debug bluestore = 1/20 2024-09-14T22:35:08.057 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-09-14T22:35:08.057 INFO:teuthology.orchestra.run.smithi001.stdout: debug osd = 20 2024-09-14T22:35:08.057 INFO:teuthology.orchestra.run.smithi001.stdout: debug rocksdb = 4/10 2024-09-14T22:35:08.057 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd backfillfull_ratio = 0.85 2024-09-14T22:35:08.057 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd full ratio = 0.9 2024-09-14T22:35:08.057 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd nearfull ratio = 0.8 2024-09-14T22:35:08.058 INFO:teuthology.orchestra.run.smithi001.stdout: osd blocked scrub grace period = 3600 2024-09-14T22:35:08.058 INFO:teuthology.orchestra.run.smithi001.stdout: osd failsafe full ratio = 0.95 2024-09-14T22:35:08.058 INFO:teuthology.orchestra.run.smithi001.stdout: osd max markdown count = 1000 2024-09-14T22:35:08.058 INFO:teuthology.orchestra.run.smithi001.stdout: osd objectstore = bluestore 2024-09-14T22:35:08.058 INFO:teuthology.orchestra.run.smithi001.stdout: osd shutdown pgref assert = True 2024-09-14T22:35:08.058 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:35:08.058 INFO:teuthology.orchestra.run.smithi001.stdout:[mgr] 2024-09-14T22:35:08.058 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-09-14T22:35:08.058 INFO:teuthology.orchestra.run.smithi001.stdout: debug mgr = 20 2024-09-14T22:35:08.058 INFO:teuthology.orchestra.run.smithi001.stdout: debug mon = 20 2024-09-14T22:35:08.058 INFO:teuthology.orchestra.run.smithi001.stdout: debug auth = 20 2024-09-14T22:35:08.058 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min pgs per osd = 4 2024-09-14T22:35:08.058 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min bytes per osd = 10 2024-09-14T22:35:08.059 INFO:teuthology.orchestra.run.smithi001.stdout: mgr/telemetry/nag = false 2024-09-14T22:35:08.059 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:35:08.059 INFO:teuthology.orchestra.run.smithi001.stdout:[mon] 2024-09-14T22:35:08.059 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-09-14T22:35:08.059 INFO:teuthology.orchestra.run.smithi001.stdout: debug mon = 20 2024-09-14T22:35:08.059 INFO:teuthology.orchestra.run.smithi001.stdout: debug paxos = 20 2024-09-14T22:35:08.059 INFO:teuthology.orchestra.run.smithi001.stdout: debug auth = 20 2024-09-14T22:35:08.059 INFO:teuthology.orchestra.run.smithi001.stdout: mon data avail warn = 5 2024-09-14T22:35:08.059 INFO:teuthology.orchestra.run.smithi001.stdout: mon mgr mkfs grace = 240 2024-09-14T22:35:08.059 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min pgs per osd = 4 2024-09-14T22:35:08.059 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd reporter subtree level = osd 2024-09-14T22:35:08.059 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd prime pg temp = true 2024-09-14T22:35:08.059 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min bytes per osd = 10 2024-09-14T22:35:08.059 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:35:08.060 INFO:teuthology.orchestra.run.smithi001.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-14T22:35:08.060 INFO:teuthology.orchestra.run.smithi001.stdout: auth mon ticket ttl = 660 # 11m 2024-09-14T22:35:08.060 INFO:teuthology.orchestra.run.smithi001.stdout: auth service ticket ttl = 240 # 4m 2024-09-14T22:35:08.060 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:35:08.060 INFO:teuthology.orchestra.run.smithi001.stdout: # don't complain about insecure global_id in the test suite 2024-09-14T22:35:08.060 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-14T22:35:08.060 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-14T22:35:08.060 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:35:08.060 INFO:teuthology.orchestra.run.smithi001.stdout: # 1m isn't quite enough 2024-09-14T22:35:08.060 INFO:teuthology.orchestra.run.smithi001.stdout: mon_down_mkfs_grace = 2m 2024-09-14T22:35:08.060 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:35:08.060 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_filestore_osds = false 2024-09-14T22:35:08.060 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:35:08.061 INFO:teuthology.orchestra.run.smithi001.stdout:[client] 2024-09-14T22:35:08.061 INFO:teuthology.orchestra.run.smithi001.stdout: rgw cache enabled = true 2024-09-14T22:35:08.061 INFO:teuthology.orchestra.run.smithi001.stdout: rgw enable ops log = true 2024-09-14T22:35:08.061 INFO:teuthology.orchestra.run.smithi001.stdout: rgw enable usage log = true 2024-09-14T22:35:08.061 INFO:teuthology.orchestra.run.smithi001.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-14T22:35:08.061 INFO:teuthology.orchestra.run.smithi001.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-14T22:35:08.061 INFO:teuthology.orchestra.run.smithi001.stdout: rbd cache = True 2024-09-14T22:35:08.061 INFO:teuthology.orchestra.run.smithi001.stdout: rbd cache policy = writearound 2024-09-14T22:35:08.061 INFO:teuthology.orchestra.run.smithi001.stdout:[mon.a] 2024-09-14T22:35:08.061 INFO:teuthology.orchestra.run.smithi001.stdout:[mon.c] 2024-09-14T22:35:08.061 INFO:teuthology.orchestra.run.smithi001.stdout:[mon.b] 2024-09-14T22:35:08.063 INFO:teuthology.orchestra.run.smithi176.stdout:[global] 2024-09-14T22:35:08.063 INFO:teuthology.orchestra.run.smithi176.stdout: chdir = "" 2024-09-14T22:35:08.063 INFO:teuthology.orchestra.run.smithi176.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-14T22:35:08.063 INFO:teuthology.orchestra.run.smithi176.stdout: auth supported = cephx 2024-09-14T22:35:08.063 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T22:35:08.063 INFO:teuthology.orchestra.run.smithi176.stdout: filestore xattr use omap = true 2024-09-14T22:35:08.063 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T22:35:08.063 INFO:teuthology.orchestra.run.smithi176.stdout: mon clock drift allowed = 1.000 2024-09-14T22:35:08.063 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T22:35:08.063 INFO:teuthology.orchestra.run.smithi176.stdout: osd crush chooseleaf type = 0 2024-09-14T22:35:08.063 INFO:teuthology.orchestra.run.smithi176.stdout: auth debug = true 2024-09-14T22:35:08.064 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T22:35:08.064 INFO:teuthology.orchestra.run.smithi176.stdout: ms die on old message = true 2024-09-14T22:35:08.064 INFO:teuthology.orchestra.run.smithi176.stdout: ms die on bug = true 2024-09-14T22:35:08.064 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T22:35:08.064 INFO:teuthology.orchestra.run.smithi176.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-14T22:35:08.064 INFO:teuthology.orchestra.run.smithi176.stdout: mon pg warn max object skew = 0 2024-09-14T22:35:08.064 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T22:35:08.064 INFO:teuthology.orchestra.run.smithi176.stdout: # disable pg_autoscaler by default for new pools 2024-09-14T22:35:08.064 INFO:teuthology.orchestra.run.smithi176.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-14T22:35:08.064 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T22:35:08.064 INFO:teuthology.orchestra.run.smithi176.stdout: osd pool default size = 2 2024-09-14T22:35:08.064 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T22:35:08.064 INFO:teuthology.orchestra.run.smithi176.stdout: mon osd allow primary affinity = true 2024-09-14T22:35:08.065 INFO:teuthology.orchestra.run.smithi176.stdout: mon osd allow pg remap = true 2024-09-14T22:35:08.065 INFO:teuthology.orchestra.run.smithi176.stdout: mon warn on legacy crush tunables = false 2024-09-14T22:35:08.065 INFO:teuthology.orchestra.run.smithi176.stdout: mon warn on crush straw calc version zero = false 2024-09-14T22:35:08.065 INFO:teuthology.orchestra.run.smithi176.stdout: mon warn on no sortbitwise = false 2024-09-14T22:35:08.065 INFO:teuthology.orchestra.run.smithi176.stdout: mon warn on osd down out interval zero = false 2024-09-14T22:35:08.065 INFO:teuthology.orchestra.run.smithi176.stdout: mon warn on too few osds = false 2024-09-14T22:35:08.065 INFO:teuthology.orchestra.run.smithi176.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-14T22:35:08.065 INFO:teuthology.orchestra.run.smithi176.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-14T22:35:08.065 INFO:teuthology.orchestra.run.smithi176.stdout: mon_allow_pool_size_one = true 2024-09-14T22:35:08.065 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T22:35:08.065 INFO:teuthology.orchestra.run.smithi176.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-14T22:35:08.065 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T22:35:08.065 INFO:teuthology.orchestra.run.smithi176.stdout: osd default data pool replay window = 5 2024-09-14T22:35:08.065 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T22:35:08.066 INFO:teuthology.orchestra.run.smithi176.stdout: mon allow pool delete = true 2024-09-14T22:35:08.066 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T22:35:08.066 INFO:teuthology.orchestra.run.smithi176.stdout: mon cluster log file level = debug 2024-09-14T22:35:08.066 INFO:teuthology.orchestra.run.smithi176.stdout: debug asserts on shutdown = true 2024-09-14T22:35:08.066 INFO:teuthology.orchestra.run.smithi176.stdout: mon health detail to clog = false 2024-09-14T22:35:08.066 INFO:teuthology.orchestra.run.smithi176.stdout: mon host = "172.21.15.1,[v2:172.21.15.1:3301,v1:172.21.15.1:6790],172.21.15.176" 2024-09-14T22:35:08.066 INFO:teuthology.orchestra.run.smithi176.stdout: mon client directed command retry = 5 2024-09-14T22:35:08.066 INFO:teuthology.orchestra.run.smithi176.stdout: mon warn on pool no app = False 2024-09-14T22:35:08.066 INFO:teuthology.orchestra.run.smithi176.stdout: ms inject socket failures = 5000 2024-09-14T22:35:08.066 INFO:teuthology.orchestra.run.smithi176.stdout: fsid = f07f407a-b17d-42c4-bf73-92dd80114a4f 2024-09-14T22:35:08.066 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T22:35:08.066 INFO:teuthology.orchestra.run.smithi176.stdout:[osd] 2024-09-14T22:35:08.066 INFO:teuthology.orchestra.run.smithi176.stdout: osd journal size = 100 2024-09-14T22:35:08.066 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T22:35:08.067 INFO:teuthology.orchestra.run.smithi176.stdout: osd scrub load threshold = 5.0 2024-09-14T22:35:08.067 INFO:teuthology.orchestra.run.smithi176.stdout: osd scrub max interval = 600 2024-09-14T22:35:08.067 INFO:teuthology.orchestra.run.smithi176.stdout: osd mclock profile = high_recovery_ops 2024-09-14T22:35:08.067 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T22:35:08.067 INFO:teuthology.orchestra.run.smithi176.stdout: osd recover clone overlap = true 2024-09-14T22:35:08.067 INFO:teuthology.orchestra.run.smithi176.stdout: osd recovery max chunk = 1048576 2024-09-14T22:35:08.067 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T22:35:08.067 INFO:teuthology.orchestra.run.smithi176.stdout: osd debug shutdown = true 2024-09-14T22:35:08.067 INFO:teuthology.orchestra.run.smithi176.stdout: osd debug op order = true 2024-09-14T22:35:08.067 INFO:teuthology.orchestra.run.smithi176.stdout: osd debug verify stray on activate = true 2024-09-14T22:35:08.067 INFO:teuthology.orchestra.run.smithi176.stdout: osd debug trim objects = true 2024-09-14T22:35:08.067 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T22:35:08.067 INFO:teuthology.orchestra.run.smithi176.stdout: osd open classes on start = true 2024-09-14T22:35:08.067 INFO:teuthology.orchestra.run.smithi176.stdout: osd debug pg log writeout = true 2024-09-14T22:35:08.067 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T22:35:08.068 INFO:teuthology.orchestra.run.smithi176.stdout: osd deep scrub update digest min age = 30 2024-09-14T22:35:08.068 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T22:35:08.068 INFO:teuthology.orchestra.run.smithi176.stdout: osd map max advance = 10 2024-09-14T22:35:08.068 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T22:35:08.068 INFO:teuthology.orchestra.run.smithi176.stdout: journal zero on create = true 2024-09-14T22:35:08.068 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T22:35:08.068 INFO:teuthology.orchestra.run.smithi176.stdout: filestore ondisk finisher threads = 3 2024-09-14T22:35:08.068 INFO:teuthology.orchestra.run.smithi176.stdout: filestore apply finisher threads = 3 2024-09-14T22:35:08.068 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T22:35:08.068 INFO:teuthology.orchestra.run.smithi176.stdout: bdev debug aio = true 2024-09-14T22:35:08.068 INFO:teuthology.orchestra.run.smithi176.stdout: osd debug misdirected ops = true 2024-09-14T22:35:08.068 INFO:teuthology.orchestra.run.smithi176.stdout: bluestore block size = 96636764160 2024-09-14T22:35:08.068 INFO:teuthology.orchestra.run.smithi176.stdout: bluestore compression algorithm = zstd 2024-09-14T22:35:08.068 INFO:teuthology.orchestra.run.smithi176.stdout: bluestore compression mode = aggressive 2024-09-14T22:35:08.069 INFO:teuthology.orchestra.run.smithi176.stdout: bluestore fsck on mount = True 2024-09-14T22:35:08.069 INFO:teuthology.orchestra.run.smithi176.stdout: debug bluefs = 1/20 2024-09-14T22:35:08.069 INFO:teuthology.orchestra.run.smithi176.stdout: debug bluestore = 1/20 2024-09-14T22:35:08.069 INFO:teuthology.orchestra.run.smithi176.stdout: debug ms = 1 2024-09-14T22:35:08.069 INFO:teuthology.orchestra.run.smithi176.stdout: debug osd = 20 2024-09-14T22:35:08.069 INFO:teuthology.orchestra.run.smithi176.stdout: debug rocksdb = 4/10 2024-09-14T22:35:08.069 INFO:teuthology.orchestra.run.smithi176.stdout: mon osd backfillfull_ratio = 0.85 2024-09-14T22:35:08.069 INFO:teuthology.orchestra.run.smithi176.stdout: mon osd full ratio = 0.9 2024-09-14T22:35:08.069 INFO:teuthology.orchestra.run.smithi176.stdout: mon osd nearfull ratio = 0.8 2024-09-14T22:35:08.069 INFO:teuthology.orchestra.run.smithi176.stdout: osd blocked scrub grace period = 3600 2024-09-14T22:35:08.069 INFO:teuthology.orchestra.run.smithi176.stdout: osd failsafe full ratio = 0.95 2024-09-14T22:35:08.069 INFO:teuthology.orchestra.run.smithi176.stdout: osd max markdown count = 1000 2024-09-14T22:35:08.069 INFO:teuthology.orchestra.run.smithi176.stdout: osd objectstore = bluestore 2024-09-14T22:35:08.069 INFO:teuthology.orchestra.run.smithi176.stdout: osd shutdown pgref assert = True 2024-09-14T22:35:08.070 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T22:35:08.070 INFO:teuthology.orchestra.run.smithi176.stdout:[mgr] 2024-09-14T22:35:08.070 INFO:teuthology.orchestra.run.smithi176.stdout: debug ms = 1 2024-09-14T22:35:08.070 INFO:teuthology.orchestra.run.smithi176.stdout: debug mgr = 20 2024-09-14T22:35:08.070 INFO:teuthology.orchestra.run.smithi176.stdout: debug mon = 20 2024-09-14T22:35:08.070 INFO:teuthology.orchestra.run.smithi176.stdout: debug auth = 20 2024-09-14T22:35:08.070 INFO:teuthology.orchestra.run.smithi176.stdout: mon reweight min pgs per osd = 4 2024-09-14T22:35:08.070 INFO:teuthology.orchestra.run.smithi176.stdout: mon reweight min bytes per osd = 10 2024-09-14T22:35:08.070 INFO:teuthology.orchestra.run.smithi176.stdout: mgr/telemetry/nag = false 2024-09-14T22:35:08.070 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T22:35:08.070 INFO:teuthology.orchestra.run.smithi176.stdout:[mon] 2024-09-14T22:35:08.070 INFO:teuthology.orchestra.run.smithi176.stdout: debug ms = 1 2024-09-14T22:35:08.070 INFO:teuthology.orchestra.run.smithi176.stdout: debug mon = 20 2024-09-14T22:35:08.071 INFO:teuthology.orchestra.run.smithi176.stdout: debug paxos = 20 2024-09-14T22:35:08.071 INFO:teuthology.orchestra.run.smithi176.stdout: debug auth = 20 2024-09-14T22:35:08.071 INFO:teuthology.orchestra.run.smithi176.stdout: mon data avail warn = 5 2024-09-14T22:35:08.071 INFO:teuthology.orchestra.run.smithi176.stdout: mon mgr mkfs grace = 240 2024-09-14T22:35:08.071 INFO:teuthology.orchestra.run.smithi176.stdout: mon reweight min pgs per osd = 4 2024-09-14T22:35:08.071 INFO:teuthology.orchestra.run.smithi176.stdout: mon osd reporter subtree level = osd 2024-09-14T22:35:08.071 INFO:teuthology.orchestra.run.smithi176.stdout: mon osd prime pg temp = true 2024-09-14T22:35:08.071 INFO:teuthology.orchestra.run.smithi176.stdout: mon reweight min bytes per osd = 10 2024-09-14T22:35:08.071 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T22:35:08.071 INFO:teuthology.orchestra.run.smithi176.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-14T22:35:08.071 INFO:teuthology.orchestra.run.smithi176.stdout: auth mon ticket ttl = 660 # 11m 2024-09-14T22:35:08.071 INFO:teuthology.orchestra.run.smithi176.stdout: auth service ticket ttl = 240 # 4m 2024-09-14T22:35:08.071 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T22:35:08.071 INFO:teuthology.orchestra.run.smithi176.stdout: # don't complain about insecure global_id in the test suite 2024-09-14T22:35:08.072 INFO:teuthology.orchestra.run.smithi176.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-14T22:35:08.072 INFO:teuthology.orchestra.run.smithi176.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-14T22:35:08.072 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T22:35:08.072 INFO:teuthology.orchestra.run.smithi176.stdout: # 1m isn't quite enough 2024-09-14T22:35:08.072 INFO:teuthology.orchestra.run.smithi176.stdout: mon_down_mkfs_grace = 2m 2024-09-14T22:35:08.072 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T22:35:08.072 INFO:teuthology.orchestra.run.smithi176.stdout: mon_warn_on_filestore_osds = false 2024-09-14T22:35:08.072 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T22:35:08.072 INFO:teuthology.orchestra.run.smithi176.stdout:[client] 2024-09-14T22:35:08.072 INFO:teuthology.orchestra.run.smithi176.stdout: rgw cache enabled = true 2024-09-14T22:35:08.072 INFO:teuthology.orchestra.run.smithi176.stdout: rgw enable ops log = true 2024-09-14T22:35:08.072 INFO:teuthology.orchestra.run.smithi176.stdout: rgw enable usage log = true 2024-09-14T22:35:08.072 INFO:teuthology.orchestra.run.smithi176.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-14T22:35:08.073 INFO:teuthology.orchestra.run.smithi176.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-14T22:35:08.073 INFO:teuthology.orchestra.run.smithi176.stdout: rbd cache = True 2024-09-14T22:35:08.073 INFO:teuthology.orchestra.run.smithi176.stdout: rbd cache policy = writearound 2024-09-14T22:35:08.073 INFO:teuthology.orchestra.run.smithi176.stdout:[mon.a] 2024-09-14T22:35:08.073 INFO:teuthology.orchestra.run.smithi176.stdout:[mon.c] 2024-09-14T22:35:08.073 INFO:teuthology.orchestra.run.smithi176.stdout:[mon.b] 2024-09-14T22:35:08.073 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-14T22:35:08.073 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-14T22:35:08.158 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-14T22:35:08.158 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-14T22:35:08.159 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-14T22:35:08.164 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-14T22:35:08.164 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-14T22:35:08.210 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi001.front.sepia.ceph.com 2024-09-14T22:35:08.211 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-14T22:35:08.211 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-14T22:35:08.211 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-14T22:35:08.276 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-14T22:35:08.276 DEBUG:teuthology.orchestra.run.smithi001:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-14T22:35:08.324 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi176.front.sepia.ceph.com 2024-09-14T22:35:08.324 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T22:35:08.324 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-14T22:35:08.324 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-14T22:35:08.348 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T22:35:08.348 DEBUG:teuthology.orchestra.run.smithi176:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-14T22:35:08.396 INFO:tasks.ceph:Setting up mon nodes... 2024-09-14T22:35:08.396 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-14T22:35:08.396 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-14T22:35:08.442 INFO:teuthology.orchestra.run.smithi001.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-14T22:35:08.445 DEBUG:teuthology.orchestra.run.smithi176:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-14T22:35:08.489 INFO:teuthology.orchestra.run.smithi176.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-14T22:35:08.503 INFO:tasks.ceph:Setting up mds nodes... 2024-09-14T22:35:08.503 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-14T22:35:08.503 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-14T22:35:08.540 INFO:teuthology.orchestra.run.smithi001.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-14T22:35:08.551 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-14T22:35:08.587 INFO:teuthology.orchestra.run.smithi176.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-14T22:35:08.599 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-14T22:35:08.599 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi001.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi176.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-14T22:35:08.600 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-14T22:35:08.612 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T22:35:08.612 INFO:tasks.ceph:role: osd.0 2024-09-14T22:35:08.612 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi001.front.sepia.ceph.com 2024-09-14T22:35:08.612 DEBUG:teuthology.orchestra.run.smithi001:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-14T22:35:08.667 INFO:teuthology.orchestra.run.smithi001.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T22:35:08.667 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T22:35:08.667 INFO:teuthology.orchestra.run.smithi001.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T22:35:08.667 INFO:teuthology.orchestra.run.smithi001.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-14T22:35:08.667 INFO:teuthology.orchestra.run.smithi001.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T22:35:08.667 INFO:teuthology.orchestra.run.smithi001.stdout: = sunit=0 swidth=0 blks 2024-09-14T22:35:08.667 INFO:teuthology.orchestra.run.smithi001.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T22:35:08.667 INFO:teuthology.orchestra.run.smithi001.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-14T22:35:08.667 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T22:35:08.668 INFO:teuthology.orchestra.run.smithi001.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T22:35:08.691 INFO:teuthology.orchestra.run.smithi001.stdout:Discarding blocks...Done. 2024-09-14T22:35:08.692 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi001.front.sepia.ceph.com -o noatime 2024-09-14T22:35:08.693 DEBUG:teuthology.orchestra.run.smithi001:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-14T22:35:08.750 DEBUG:teuthology.orchestra.run.smithi001:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-14T22:35:08.801 INFO:teuthology.orchestra.run.smithi001.stderr:sudo: /sbin/restorecon: command not found 2024-09-14T22:35:08.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T22:35:08.802 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-14T22:35:08.859 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T22:35:08.859 INFO:tasks.ceph:role: osd.1 2024-09-14T22:35:08.859 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi001.front.sepia.ceph.com 2024-09-14T22:35:08.859 DEBUG:teuthology.orchestra.run.smithi001:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-14T22:35:08.914 INFO:teuthology.orchestra.run.smithi001.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T22:35:08.915 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T22:35:08.915 INFO:teuthology.orchestra.run.smithi001.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T22:35:08.915 INFO:teuthology.orchestra.run.smithi001.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-14T22:35:08.915 INFO:teuthology.orchestra.run.smithi001.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T22:35:08.915 INFO:teuthology.orchestra.run.smithi001.stdout: = sunit=0 swidth=0 blks 2024-09-14T22:35:08.915 INFO:teuthology.orchestra.run.smithi001.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T22:35:08.915 INFO:teuthology.orchestra.run.smithi001.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-14T22:35:08.915 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T22:35:08.915 INFO:teuthology.orchestra.run.smithi001.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T22:35:08.939 INFO:teuthology.orchestra.run.smithi001.stdout:Discarding blocks...Done. 2024-09-14T22:35:08.940 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi001.front.sepia.ceph.com -o noatime 2024-09-14T22:35:08.941 DEBUG:teuthology.orchestra.run.smithi001:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-14T22:35:09.000 DEBUG:teuthology.orchestra.run.smithi001:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-14T22:35:09.051 INFO:teuthology.orchestra.run.smithi001.stderr:sudo: /sbin/restorecon: command not found 2024-09-14T22:35:09.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T22:35:09.052 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-14T22:35:09.105 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T22:35:09.106 INFO:tasks.ceph:role: osd.2 2024-09-14T22:35:09.106 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi001.front.sepia.ceph.com 2024-09-14T22:35:09.106 DEBUG:teuthology.orchestra.run.smithi001:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-14T22:35:09.160 INFO:teuthology.orchestra.run.smithi001.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T22:35:09.161 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T22:35:09.161 INFO:teuthology.orchestra.run.smithi001.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T22:35:09.161 INFO:teuthology.orchestra.run.smithi001.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-14T22:35:09.161 INFO:teuthology.orchestra.run.smithi001.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T22:35:09.161 INFO:teuthology.orchestra.run.smithi001.stdout: = sunit=0 swidth=0 blks 2024-09-14T22:35:09.161 INFO:teuthology.orchestra.run.smithi001.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T22:35:09.161 INFO:teuthology.orchestra.run.smithi001.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-14T22:35:09.161 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T22:35:09.161 INFO:teuthology.orchestra.run.smithi001.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T22:35:09.184 INFO:teuthology.orchestra.run.smithi001.stdout:Discarding blocks...Done. 2024-09-14T22:35:09.186 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi001.front.sepia.ceph.com -o noatime 2024-09-14T22:35:09.186 DEBUG:teuthology.orchestra.run.smithi001:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-14T22:35:09.244 DEBUG:teuthology.orchestra.run.smithi001:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-14T22:35:09.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T22:35:09.296 INFO:teuthology.orchestra.run.smithi001.stderr:sudo: /sbin/restorecon: command not found 2024-09-14T22:35:09.296 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-14T22:35:09.352 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T22:35:09.352 INFO:tasks.ceph:role: osd.3 2024-09-14T22:35:09.352 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi001.front.sepia.ceph.com 2024-09-14T22:35:09.353 DEBUG:teuthology.orchestra.run.smithi001:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-14T22:35:09.410 INFO:teuthology.orchestra.run.smithi001.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T22:35:09.410 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T22:35:09.410 INFO:teuthology.orchestra.run.smithi001.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T22:35:09.410 INFO:teuthology.orchestra.run.smithi001.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-14T22:35:09.410 INFO:teuthology.orchestra.run.smithi001.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T22:35:09.410 INFO:teuthology.orchestra.run.smithi001.stdout: = sunit=0 swidth=0 blks 2024-09-14T22:35:09.410 INFO:teuthology.orchestra.run.smithi001.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T22:35:09.410 INFO:teuthology.orchestra.run.smithi001.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-14T22:35:09.410 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T22:35:09.410 INFO:teuthology.orchestra.run.smithi001.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T22:35:09.436 INFO:teuthology.orchestra.run.smithi001.stdout:Discarding blocks...Done. 2024-09-14T22:35:09.438 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi001.front.sepia.ceph.com -o noatime 2024-09-14T22:35:09.438 DEBUG:teuthology.orchestra.run.smithi001:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-14T22:35:09.501 DEBUG:teuthology.orchestra.run.smithi001:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-14T22:35:09.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T22:35:09.554 INFO:teuthology.orchestra.run.smithi001.stderr:sudo: /sbin/restorecon: command not found 2024-09-14T22:35:09.554 DEBUG:teuthology.orchestra.run.smithi001:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T22:35:09.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-14T22:35:09.663+0000 7fd1874ec7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-14T22:35:09.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-14T22:35:09.663+0000 7fd1874ec7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-14T22:35:09.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-14T22:35:09.663+0000 7fd1874ec7c0 -1 bdev(0x55ce4feab800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-14T22:35:09.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-14T22:35:09.663+0000 7fd1874ec7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-14T22:35:12.807 DEBUG:teuthology.orchestra.run.smithi001:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-14T22:35:12.821 DEBUG:teuthology.orchestra.run.smithi001:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T22:35:12.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-14T22:35:12.903+0000 7f6a80afb7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-14T22:35:12.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-14T22:35:12.903+0000 7f6a80afb7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-14T22:35:12.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-14T22:35:12.903+0000 7f6a80afb7c0 -1 bdev(0x559b6f53d800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-14T22:35:12.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-14T22:35:12.903+0000 7f6a80afb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-14T22:35:16.055 DEBUG:teuthology.orchestra.run.smithi001:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-14T22:35:16.112 DEBUG:teuthology.orchestra.run.smithi001:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T22:35:16.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-14T22:35:16.191+0000 7fdca255f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-14T22:35:16.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-14T22:35:16.191+0000 7fdca255f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-14T22:35:16.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-14T22:35:16.191+0000 7fdca255f7c0 -1 bdev(0x55bb4e8a5800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-14T22:35:16.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-14T22:35:16.191+0000 7fdca255f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-14T22:35:19.350 DEBUG:teuthology.orchestra.run.smithi001:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-14T22:35:19.409 DEBUG:teuthology.orchestra.run.smithi001:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T22:35:19.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-14T22:35:19.487+0000 7f5e418507c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-14T22:35:19.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-14T22:35:19.487+0000 7f5e418507c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-14T22:35:19.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-14T22:35:19.487+0000 7f5e418507c0 -1 bdev(0x5562e412f800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-14T22:35:19.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-14T22:35:19.487+0000 7f5e418507c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-14T22:35:22.634 DEBUG:teuthology.orchestra.run.smithi001:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-14T22:35:22.696 DEBUG:teuthology.orchestra.run.smithi176:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-14T22:35:22.710 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T22:35:22.710 INFO:tasks.ceph:role: osd.4 2024-09-14T22:35:22.710 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi176.front.sepia.ceph.com 2024-09-14T22:35:22.711 DEBUG:teuthology.orchestra.run.smithi176:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-14T22:35:22.771 INFO:teuthology.orchestra.run.smithi176.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T22:35:22.772 INFO:teuthology.orchestra.run.smithi176.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T22:35:22.772 INFO:teuthology.orchestra.run.smithi176.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T22:35:22.772 INFO:teuthology.orchestra.run.smithi176.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-14T22:35:22.772 INFO:teuthology.orchestra.run.smithi176.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T22:35:22.772 INFO:teuthology.orchestra.run.smithi176.stdout: = sunit=0 swidth=0 blks 2024-09-14T22:35:22.772 INFO:teuthology.orchestra.run.smithi176.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T22:35:22.772 INFO:teuthology.orchestra.run.smithi176.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-14T22:35:22.772 INFO:teuthology.orchestra.run.smithi176.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T22:35:22.772 INFO:teuthology.orchestra.run.smithi176.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T22:35:23.222 INFO:teuthology.orchestra.run.smithi176.stdout:Discarding blocks...Done. 2024-09-14T22:35:23.223 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi176.front.sepia.ceph.com -o noatime 2024-09-14T22:35:23.224 DEBUG:teuthology.orchestra.run.smithi176:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-14T22:35:23.321 DEBUG:teuthology.orchestra.run.smithi176:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-14T22:35:23.331 INFO:teuthology.orchestra.run.smithi176.stderr:sudo: /sbin/restorecon: command not found 2024-09-14T22:35:23.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T22:35:23.332 DEBUG:teuthology.orchestra.run.smithi176:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-14T22:35:23.387 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T22:35:23.388 INFO:tasks.ceph:role: osd.5 2024-09-14T22:35:23.388 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi176.front.sepia.ceph.com 2024-09-14T22:35:23.388 DEBUG:teuthology.orchestra.run.smithi176:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-14T22:35:23.458 INFO:teuthology.orchestra.run.smithi176.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T22:35:23.458 INFO:teuthology.orchestra.run.smithi176.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T22:35:23.458 INFO:teuthology.orchestra.run.smithi176.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T22:35:23.458 INFO:teuthology.orchestra.run.smithi176.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-14T22:35:23.458 INFO:teuthology.orchestra.run.smithi176.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T22:35:23.458 INFO:teuthology.orchestra.run.smithi176.stdout: = sunit=0 swidth=0 blks 2024-09-14T22:35:23.458 INFO:teuthology.orchestra.run.smithi176.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T22:35:23.459 INFO:teuthology.orchestra.run.smithi176.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-14T22:35:23.459 INFO:teuthology.orchestra.run.smithi176.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T22:35:23.459 INFO:teuthology.orchestra.run.smithi176.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T22:35:23.928 INFO:teuthology.orchestra.run.smithi176.stdout:Discarding blocks...Done. 2024-09-14T22:35:23.930 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi176.front.sepia.ceph.com -o noatime 2024-09-14T22:35:23.930 DEBUG:teuthology.orchestra.run.smithi176:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-14T22:35:24.069 DEBUG:teuthology.orchestra.run.smithi176:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-14T22:35:24.079 INFO:teuthology.orchestra.run.smithi176.stderr:sudo: /sbin/restorecon: command not found 2024-09-14T22:35:24.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T22:35:24.080 DEBUG:teuthology.orchestra.run.smithi176:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-14T22:35:24.139 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T22:35:24.139 INFO:tasks.ceph:role: osd.6 2024-09-14T22:35:24.139 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi176.front.sepia.ceph.com 2024-09-14T22:35:24.140 DEBUG:teuthology.orchestra.run.smithi176:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-14T22:35:24.203 INFO:teuthology.orchestra.run.smithi176.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T22:35:24.203 INFO:teuthology.orchestra.run.smithi176.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T22:35:24.203 INFO:teuthology.orchestra.run.smithi176.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T22:35:24.203 INFO:teuthology.orchestra.run.smithi176.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-14T22:35:24.203 INFO:teuthology.orchestra.run.smithi176.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T22:35:24.203 INFO:teuthology.orchestra.run.smithi176.stdout: = sunit=0 swidth=0 blks 2024-09-14T22:35:24.203 INFO:teuthology.orchestra.run.smithi176.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T22:35:24.203 INFO:teuthology.orchestra.run.smithi176.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-14T22:35:24.203 INFO:teuthology.orchestra.run.smithi176.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T22:35:24.203 INFO:teuthology.orchestra.run.smithi176.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T22:35:24.682 INFO:teuthology.orchestra.run.smithi176.stdout:Discarding blocks...Done. 2024-09-14T22:35:24.684 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi176.front.sepia.ceph.com -o noatime 2024-09-14T22:35:24.684 DEBUG:teuthology.orchestra.run.smithi176:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-14T22:35:24.821 DEBUG:teuthology.orchestra.run.smithi176:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-14T22:35:24.832 INFO:teuthology.orchestra.run.smithi176.stderr:sudo: /sbin/restorecon: command not found 2024-09-14T22:35:24.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T22:35:24.833 DEBUG:teuthology.orchestra.run.smithi176:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-14T22:35:24.887 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T22:35:24.888 INFO:tasks.ceph:role: osd.7 2024-09-14T22:35:24.888 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi176.front.sepia.ceph.com 2024-09-14T22:35:24.888 DEBUG:teuthology.orchestra.run.smithi176:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-14T22:35:24.951 INFO:teuthology.orchestra.run.smithi176.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T22:35:24.951 INFO:teuthology.orchestra.run.smithi176.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T22:35:24.951 INFO:teuthology.orchestra.run.smithi176.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T22:35:24.951 INFO:teuthology.orchestra.run.smithi176.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-14T22:35:24.951 INFO:teuthology.orchestra.run.smithi176.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T22:35:24.951 INFO:teuthology.orchestra.run.smithi176.stdout: = sunit=0 swidth=0 blks 2024-09-14T22:35:24.951 INFO:teuthology.orchestra.run.smithi176.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T22:35:24.952 INFO:teuthology.orchestra.run.smithi176.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-14T22:35:24.952 INFO:teuthology.orchestra.run.smithi176.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T22:35:24.952 INFO:teuthology.orchestra.run.smithi176.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T22:35:25.373 INFO:teuthology.orchestra.run.smithi176.stdout:Discarding blocks...Done. 2024-09-14T22:35:25.375 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi176.front.sepia.ceph.com -o noatime 2024-09-14T22:35:25.376 DEBUG:teuthology.orchestra.run.smithi176:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-14T22:35:25.467 DEBUG:teuthology.orchestra.run.smithi176:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-14T22:35:25.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T22:35:25.478 INFO:teuthology.orchestra.run.smithi176.stderr:sudo: /sbin/restorecon: command not found 2024-09-14T22:35:25.478 DEBUG:teuthology.orchestra.run.smithi176:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T22:35:25.595 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-14T22:35:25.592+0000 7f0bdde587c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-14T22:35:25.599 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-14T22:35:25.592+0000 7f0bdde587c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-14T22:35:25.599 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-14T22:35:25.592+0000 7f0bdde587c0 -1 bdev(0x555ed6807800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-14T22:35:25.599 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-14T22:35:25.592+0000 7f0bdde587c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-14T22:35:28.743 DEBUG:teuthology.orchestra.run.smithi176:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-14T22:35:28.758 DEBUG:teuthology.orchestra.run.smithi176:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T22:35:28.837 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-14T22:35:28.832+0000 7f73d3d087c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-14T22:35:28.838 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-14T22:35:28.832+0000 7f73d3d087c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-14T22:35:28.838 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-14T22:35:28.832+0000 7f73d3d087c0 -1 bdev(0x55bf109db800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-14T22:35:28.838 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-14T22:35:28.832+0000 7f73d3d087c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-14T22:35:31.942 DEBUG:teuthology.orchestra.run.smithi176:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-14T22:35:32.000 DEBUG:teuthology.orchestra.run.smithi176:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T22:35:32.082 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-14T22:35:32.076+0000 7f82034477c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-14T22:35:32.083 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-14T22:35:32.080+0000 7f82034477c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-14T22:35:32.083 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-14T22:35:32.080+0000 7f82034477c0 -1 bdev(0x55d8bda8d800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-14T22:35:32.083 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-14T22:35:32.080+0000 7f82034477c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-14T22:35:35.221 DEBUG:teuthology.orchestra.run.smithi176:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-14T22:35:35.280 DEBUG:teuthology.orchestra.run.smithi176:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T22:35:35.362 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-14T22:35:35.356+0000 7f21c9e4d7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-14T22:35:35.363 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-14T22:35:35.360+0000 7f21c9e4d7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-14T22:35:35.363 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-14T22:35:35.360+0000 7f21c9e4d7c0 -1 bdev(0x557d82f87800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-14T22:35:35.363 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-14T22:35:35.360+0000 7f21c9e4d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-14T22:35:38.487 DEBUG:teuthology.orchestra.run.smithi176:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-14T22:35:38.544 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-14T22:35:38.544 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-14T22:35:38.544 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-14T22:35:38.560 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-14T22:35:38.560 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-14T22:35:38.615 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-14T22:35:38.615 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-14T22:35:38.671 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-14T22:35:38.671 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-14T22:35:38.726 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-14T22:35:38.727 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-14T22:35:38.786 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T22:35:38.786 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-14T22:35:38.800 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T22:35:38.801 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-14T22:35:38.876 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T22:35:38.877 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-14T22:35:38.945 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T22:35:38.945 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-14T22:35:39.003 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T22:35:39.003 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-14T22:35:39.061 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-14T22:35:39.061 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-14T22:35:39.068 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T22:35:39.069 DEBUG:teuthology.orchestra.run.smithi176:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-14T22:35:39.108 INFO:tasks.ceph:Adding keys to all mons... 2024-09-14T22:35:39.109 DEBUG:teuthology.orchestra.run.smithi001:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-14T22:35:39.113 DEBUG:teuthology.orchestra.run.smithi176:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-14T22:35:39.154 INFO:teuthology.orchestra.run.smithi001.stdout:[mgr.y] 2024-09-14T22:35:39.154 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCcD+ZmFglOGhAAya5QxdP0SO6W3z14Hz1+HA== 2024-09-14T22:35:39.154 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.0] 2024-09-14T22:35:39.154 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCdD+ZmHNHdJxAALYh3TzHrtrjrXoA+47iyMQ== 2024-09-14T22:35:39.154 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.1] 2024-09-14T22:35:39.154 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCgD+ZmMFUHNhAAxDZ5TkFq65baS7ydJfF1zQ== 2024-09-14T22:35:39.154 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.2] 2024-09-14T22:35:39.155 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCkD+ZmfuufCxAAILUG86M//yulFNVm20LzTQ== 2024-09-14T22:35:39.155 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.3] 2024-09-14T22:35:39.155 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCnD+Zmwts4HRAAU+KgR6hjDSQrM0LprZHmFg== 2024-09-14T22:35:39.155 INFO:teuthology.orchestra.run.smithi001.stdout:[mgr.x] 2024-09-14T22:35:39.155 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCcD+ZmNMEZHRAAvN02yLELEAtNRm7QieC/hA== 2024-09-14T22:35:39.155 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.4] 2024-09-14T22:35:39.155 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCtD+ZmYbhwIxAAMPDlMSUxnYiVY2OCBbT1rQ== 2024-09-14T22:35:39.155 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.5] 2024-09-14T22:35:39.155 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCwD+ZmcCnXMRAAs874srJ8M4CxPC0aOezzig== 2024-09-14T22:35:39.155 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.6] 2024-09-14T22:35:39.155 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQC0D+ZmZEnfBBAA8Exl1pSW8c0YJdlxhYAMYw== 2024-09-14T22:35:39.155 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.7] 2024-09-14T22:35:39.156 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQC3D+ZmveuPFRAAxadLCZKJgbDNfHmZ30l67Q== 2024-09-14T22:35:39.156 INFO:teuthology.orchestra.run.smithi001.stdout:[client.0] 2024-09-14T22:35:39.156 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCcD+ZmYX4lIBAAFEVZr7oJpk7OfL6E8KtKAg== 2024-09-14T22:35:39.156 INFO:teuthology.orchestra.run.smithi001.stdout:[client.1] 2024-09-14T22:35:39.156 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCcD+ZmAtvuIhAAB3eDD1Sz7gNle9YQsazy1A== 2024-09-14T22:35:39.161 INFO:teuthology.orchestra.run.smithi176.stdout:[mgr.y] 2024-09-14T22:35:39.161 INFO:teuthology.orchestra.run.smithi176.stdout: key = AQCcD+ZmFglOGhAAya5QxdP0SO6W3z14Hz1+HA== 2024-09-14T22:35:39.161 INFO:teuthology.orchestra.run.smithi176.stdout:[osd.0] 2024-09-14T22:35:39.161 INFO:teuthology.orchestra.run.smithi176.stdout: key = AQCdD+ZmHNHdJxAALYh3TzHrtrjrXoA+47iyMQ== 2024-09-14T22:35:39.161 INFO:teuthology.orchestra.run.smithi176.stdout:[osd.1] 2024-09-14T22:35:39.161 INFO:teuthology.orchestra.run.smithi176.stdout: key = AQCgD+ZmMFUHNhAAxDZ5TkFq65baS7ydJfF1zQ== 2024-09-14T22:35:39.161 INFO:teuthology.orchestra.run.smithi176.stdout:[osd.2] 2024-09-14T22:35:39.161 INFO:teuthology.orchestra.run.smithi176.stdout: key = AQCkD+ZmfuufCxAAILUG86M//yulFNVm20LzTQ== 2024-09-14T22:35:39.162 INFO:teuthology.orchestra.run.smithi176.stdout:[osd.3] 2024-09-14T22:35:39.162 INFO:teuthology.orchestra.run.smithi176.stdout: key = AQCnD+Zmwts4HRAAU+KgR6hjDSQrM0LprZHmFg== 2024-09-14T22:35:39.162 INFO:teuthology.orchestra.run.smithi176.stdout:[mgr.x] 2024-09-14T22:35:39.162 INFO:teuthology.orchestra.run.smithi176.stdout: key = AQCcD+ZmNMEZHRAAvN02yLELEAtNRm7QieC/hA== 2024-09-14T22:35:39.162 INFO:teuthology.orchestra.run.smithi176.stdout:[osd.4] 2024-09-14T22:35:39.162 INFO:teuthology.orchestra.run.smithi176.stdout: key = AQCtD+ZmYbhwIxAAMPDlMSUxnYiVY2OCBbT1rQ== 2024-09-14T22:35:39.162 INFO:teuthology.orchestra.run.smithi176.stdout:[osd.5] 2024-09-14T22:35:39.162 INFO:teuthology.orchestra.run.smithi176.stdout: key = AQCwD+ZmcCnXMRAAs874srJ8M4CxPC0aOezzig== 2024-09-14T22:35:39.162 INFO:teuthology.orchestra.run.smithi176.stdout:[osd.6] 2024-09-14T22:35:39.162 INFO:teuthology.orchestra.run.smithi176.stdout: key = AQC0D+ZmZEnfBBAA8Exl1pSW8c0YJdlxhYAMYw== 2024-09-14T22:35:39.162 INFO:teuthology.orchestra.run.smithi176.stdout:[osd.7] 2024-09-14T22:35:39.162 INFO:teuthology.orchestra.run.smithi176.stdout: key = AQC3D+ZmveuPFRAAxadLCZKJgbDNfHmZ30l67Q== 2024-09-14T22:35:39.162 INFO:teuthology.orchestra.run.smithi176.stdout:[client.0] 2024-09-14T22:35:39.163 INFO:teuthology.orchestra.run.smithi176.stdout: key = AQCcD+ZmYX4lIBAAFEVZr7oJpk7OfL6E8KtKAg== 2024-09-14T22:35:39.163 INFO:teuthology.orchestra.run.smithi176.stdout:[client.1] 2024-09-14T22:35:39.163 INFO:teuthology.orchestra.run.smithi176.stdout: key = AQCcD+ZmAtvuIhAAB3eDD1Sz7gNle9YQsazy1A== 2024-09-14T22:35:39.164 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T22:35:39.200 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T22:35:39.252 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T22:35:39.254 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T22:35:39.295 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T22:35:39.341 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T22:35:39.381 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T22:35:39.424 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T22:35:39.464 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T22:35:39.508 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T22:35:39.550 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T22:35:39.596 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T22:35:39.636 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T22:35:39.680 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T22:35:39.721 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T22:35:39.764 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T22:35:39.810 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T22:35:39.812 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T22:35:39.895 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T22:35:39.897 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T22:35:39.981 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T22:35:39.984 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T22:35:40.066 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T22:35:40.068 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T22:35:40.147 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-14T22:35:40.148 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-14T22:35:40.160 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-14T22:35:40.257 DEBUG:teuthology.orchestra.run.smithi001:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-14T22:35:40.270 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-14T22:35:40.321 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-14T22:35:40.414 DEBUG:teuthology.orchestra.run.smithi001:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-14T22:35:40.425 DEBUG:teuthology.orchestra.run.smithi176:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-14T22:35:40.439 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-14T22:35:40.536 DEBUG:teuthology.orchestra.run.smithi176:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-14T22:35:40.548 DEBUG:teuthology.orchestra.run.smithi001:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-14T22:35:40.550 DEBUG:teuthology.orchestra.run.smithi176:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-14T22:35:40.595 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-14T22:35:40.595 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-14T22:35:40.595 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-14T22:35:40.597 INFO:tasks.ceph.mon.a:Started 2024-09-14T22:35:40.598 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-14T22:35:40.598 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-14T22:35:40.600 INFO:tasks.ceph.mon.c:Started 2024-09-14T22:35:40.600 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-14T22:35:40.600 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-14T22:35:40.640 INFO:tasks.ceph.mon.b:Started 2024-09-14T22:35:40.641 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-14T22:35:40.641 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-14T22:35:40.641 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-14T22:35:40.645 INFO:tasks.ceph.mgr.y:Started 2024-09-14T22:35:40.645 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-14T22:35:40.646 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-14T22:35:40.648 INFO:tasks.ceph.mgr.x:Started 2024-09-14T22:35:40.648 INFO:tasks.ceph:Setting crush tunables to default 2024-09-14T22:35:40.648 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --cluster ceph osd crush tunables default 2024-09-14T22:35:46.110 INFO:teuthology.orchestra.run.smithi001.stderr:adjusted tunables profile to default 2024-09-14T22:35:46.130 INFO:tasks.ceph:check_enable_crimson: False 2024-09-14T22:35:46.130 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-14T22:35:46.130 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-14T22:35:46.130 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-14T22:35:46.142 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-14T22:35:46.142 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-14T22:35:46.192 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-14T22:35:46.193 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-14T22:35:46.245 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-14T22:35:46.245 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-14T22:35:46.294 INFO:tasks.ceph.mgr.x.smithi176.stderr:2024-09-14T22:35:46.288+0000 7f10fd0a7180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-14T22:35:46.299 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T22:35:46.299 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-14T22:35:46.312 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T22:35:46.312 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-14T22:35:46.366 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T22:35:46.366 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-14T22:35:46.417 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T22:35:46.417 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-14T22:35:46.432 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-14T22:35:46.426+0000 7f255868d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-14T22:35:46.469 DEBUG:teuthology.orchestra.run.smithi176:> sudo ceph --cluster ceph osd new 9831fcc5-9040-4a80-99d4-838225e65139 0 2024-09-14T22:35:46.721 INFO:teuthology.orchestra.run.smithi176.stdout:0 2024-09-14T22:35:46.740 DEBUG:teuthology.orchestra.run.smithi176:> sudo ceph --cluster ceph osd new 7f79abe8-115d-46b2-8677-72fcb0c8ab7d 1 2024-09-14T22:35:46.814 INFO:tasks.ceph.mgr.x.smithi176.stderr:2024-09-14T22:35:46.808+0000 7f10fd0a7180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-14T22:35:46.919 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-14T22:35:46.914+0000 7f255868d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-14T22:35:46.942 INFO:teuthology.orchestra.run.smithi176.stdout:1 2024-09-14T22:35:46.960 DEBUG:teuthology.orchestra.run.smithi176:> sudo ceph --cluster ceph osd new 1f5768b2-4db4-4b2e-8d5e-5206f8bde28d 2 2024-09-14T22:35:47.027 INFO:tasks.ceph.mgr.x.smithi176.stderr:2024-09-14T22:35:47.024+0000 7f10fd0a7180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-14T22:35:47.117 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-14T22:35:47.110+0000 7f255868d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-14T22:35:47.155 INFO:teuthology.orchestra.run.smithi176.stdout:2 2024-09-14T22:35:47.170 DEBUG:teuthology.orchestra.run.smithi176:> sudo ceph --cluster ceph osd new 8103f3c7-7899-4fc3-bb02-dd2273120138 3 2024-09-14T22:35:47.371 INFO:teuthology.orchestra.run.smithi176.stdout:3 2024-09-14T22:35:47.387 DEBUG:teuthology.orchestra.run.smithi176:> sudo ceph --cluster ceph osd new 23cce5cb-726e-44c2-8edb-c2567a89dfbc 4 2024-09-14T22:35:47.588 INFO:teuthology.orchestra.run.smithi176.stdout:4 2024-09-14T22:35:47.601 INFO:tasks.ceph.mgr.x.smithi176.stderr:2024-09-14T22:35:47.596+0000 7f10fd0a7180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-14T22:35:47.605 DEBUG:teuthology.orchestra.run.smithi176:> sudo ceph --cluster ceph osd new eb2f187b-d2f1-49ab-ad5e-ea6dc441a28e 5 2024-09-14T22:35:47.657 INFO:tasks.ceph.mgr.x.smithi176.stderr:2024-09-14T22:35:47.652+0000 7f10fd0a7180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-14T22:35:47.696 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-14T22:35:47.690+0000 7f255868d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-14T22:35:47.711 INFO:tasks.ceph.mgr.x.smithi176.stderr:2024-09-14T22:35:47.704+0000 7f10fd0a7180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-14T22:35:47.752 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-14T22:35:47.746+0000 7f255868d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-14T22:35:47.790 INFO:tasks.ceph.mgr.x.smithi176.stderr:2024-09-14T22:35:47.784+0000 7f10fd0a7180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-14T22:35:47.807 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-14T22:35:47.802+0000 7f255868d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-14T22:35:47.810 INFO:teuthology.orchestra.run.smithi176.stdout:5 2024-09-14T22:35:47.826 DEBUG:teuthology.orchestra.run.smithi176:> sudo ceph --cluster ceph osd new d371aed8-2ad2-49a5-8ece-4b7fd37b4822 6 2024-09-14T22:35:47.846 INFO:tasks.ceph.mgr.x.smithi176.stderr:2024-09-14T22:35:47.840+0000 7f10fd0a7180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-14T22:35:47.888 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-14T22:35:47.882+0000 7f255868d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-14T22:35:47.925 INFO:tasks.ceph.mgr.x.smithi176.stderr:2024-09-14T22:35:47.920+0000 7f10fd0a7180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-14T22:35:47.943 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-14T22:35:47.938+0000 7f255868d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-14T22:35:47.980 INFO:tasks.ceph.mgr.x.smithi176.stderr:2024-09-14T22:35:47.976+0000 7f10fd0a7180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-14T22:35:48.023 INFO:teuthology.orchestra.run.smithi176.stdout:6 2024-09-14T22:35:48.026 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-14T22:35:48.018+0000 7f255868d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-14T22:35:48.040 DEBUG:teuthology.orchestra.run.smithi176:> sudo ceph --cluster ceph osd new a012c819-8d49-4310-a840-c2752d9314ab 7 2024-09-14T22:35:48.082 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-14T22:35:48.074+0000 7f255868d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-14T22:35:48.137 INFO:tasks.ceph.mgr.x.smithi176.stderr:2024-09-14T22:35:48.132+0000 7f10fd0a7180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-14T22:35:48.200 INFO:tasks.ceph.mgr.x.smithi176.stderr:2024-09-14T22:35:48.196+0000 7f10fd0a7180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-14T22:35:48.239 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-14T22:35:48.234+0000 7f255868d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-14T22:35:48.288 INFO:teuthology.orchestra.run.smithi176.stdout:7 2024-09-14T22:35:48.304 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-14T22:35:48.304 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-14T22:35:48.305 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-14T22:35:48.298+0000 7f255868d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-14T22:35:48.307 INFO:tasks.ceph.osd.0:Started 2024-09-14T22:35:48.307 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-14T22:35:48.307 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-14T22:35:48.310 INFO:tasks.ceph.osd.1:Started 2024-09-14T22:35:48.310 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-14T22:35:48.310 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-14T22:35:48.312 INFO:tasks.ceph.osd.2:Started 2024-09-14T22:35:48.313 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-14T22:35:48.313 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-14T22:35:48.315 INFO:tasks.ceph.osd.3:Started 2024-09-14T22:35:48.315 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-14T22:35:48.315 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-14T22:35:48.318 INFO:tasks.ceph.osd.4:Started 2024-09-14T22:35:48.318 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-14T22:35:48.318 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-14T22:35:48.320 INFO:tasks.ceph.osd.5:Started 2024-09-14T22:35:48.320 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-14T22:35:48.320 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-14T22:35:48.322 INFO:tasks.ceph.osd.6:Started 2024-09-14T22:35:48.322 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-14T22:35:48.322 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-14T22:35:48.324 INFO:tasks.ceph.osd.7:Started 2024-09-14T22:35:48.325 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T22:35:48.523 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:35:48.523 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":10,"fsid":"f07f407a-b17d-42c4-bf73-92dd80114a4f","created":"2024-09-14T22:35:45.983055+0000","modified":"2024-09-14T22:35:48.283160+0000","last_up_change":"0.000000","last_in_change":"2024-09-14T22:35:48.283160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"9831fcc5-9040-4a80-99d4-838225e65139","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7f79abe8-115d-46b2-8677-72fcb0c8ab7d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1f5768b2-4db4-4b2e-8d5e-5206f8bde28d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"8103f3c7-7899-4fc3-bb02-dd2273120138","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"23cce5cb-726e-44c2-8edb-c2567a89dfbc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"eb2f187b-d2f1-49ab-ad5e-ea6dc441a28e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d371aed8-2ad2-49a5-8ece-4b7fd37b4822","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a012c819-8d49-4310-a840-c2752d9314ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T22:35:48.538 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-14T22:35:48.538 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-14T22:35:48.870 INFO:tasks.ceph.mgr.x.smithi176.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-14T22:35:48.870 INFO:tasks.ceph.mgr.x.smithi176.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-14T22:35:48.870 INFO:tasks.ceph.mgr.x.smithi176.stderr: from numpy import show_config as show_numpy_config 2024-09-14T22:35:48.941 INFO:tasks.ceph.mgr.x.smithi176.stderr:2024-09-14T22:35:48.936+0000 7f10fd0a7180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-14T22:35:48.972 INFO:tasks.ceph.mgr.y.smithi001.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-14T22:35:48.972 INFO:tasks.ceph.mgr.y.smithi001.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-14T22:35:48.972 INFO:tasks.ceph.mgr.y.smithi001.stderr: from numpy import show_config as show_numpy_config 2024-09-14T22:35:49.036 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-14T22:35:49.030+0000 7f255868d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-14T22:35:49.115 INFO:tasks.ceph.mgr.x.smithi176.stderr:2024-09-14T22:35:49.108+0000 7f10fd0a7180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-14T22:35:49.222 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-14T22:35:49.214+0000 7f255868d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-14T22:35:49.291 INFO:tasks.ceph.mgr.x.smithi176.stderr:2024-09-14T22:35:49.284+0000 7f10fd0a7180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-14T22:35:49.343 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-14T22:35:49.361 INFO:tasks.ceph.mgr.x.smithi176.stderr:2024-09-14T22:35:49.356+0000 7f10fd0a7180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-14T22:35:49.392 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-14T22:35:49.386+0000 7f255868d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-14T22:35:49.457 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:35:49.452+0000 7f00407fe7c0 -1 Falling back to public interface 2024-09-14T22:35:49.457 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:35:49.452+0000 7f17e6afb7c0 -1 Falling back to public interface 2024-09-14T22:35:49.459 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:35:49.454+0000 7feb6a1687c0 -1 Falling back to public interface 2024-09-14T22:35:49.465 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-14T22:35:49.458+0000 7f255868d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-14T22:35:49.476 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:35:49.472+0000 7f363a2377c0 -1 Falling back to public interface 2024-09-14T22:35:49.509 INFO:tasks.ceph.mgr.x.smithi176.stderr:2024-09-14T22:35:49.504+0000 7f10fd0a7180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-14T22:35:49.535 INFO:teuthology.misc.health.smithi001.stdout: 2024-09-14T22:35:49.536 INFO:teuthology.misc.health.smithi001.stdout:{"epoch":10,"fsid":"f07f407a-b17d-42c4-bf73-92dd80114a4f","created":"2024-09-14T22:35:45.983055+0000","modified":"2024-09-14T22:35:48.283160+0000","last_up_change":"0.000000","last_in_change":"2024-09-14T22:35:48.283160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"9831fcc5-9040-4a80-99d4-838225e65139","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7f79abe8-115d-46b2-8677-72fcb0c8ab7d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1f5768b2-4db4-4b2e-8d5e-5206f8bde28d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"8103f3c7-7899-4fc3-bb02-dd2273120138","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"23cce5cb-726e-44c2-8edb-c2567a89dfbc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"eb2f187b-d2f1-49ab-ad5e-ea6dc441a28e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d371aed8-2ad2-49a5-8ece-4b7fd37b4822","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a012c819-8d49-4310-a840-c2752d9314ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T22:35:49.552 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-14T22:35:49.621 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-14T22:35:49.614+0000 7f255868d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-14T22:35:49.693 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:35:49.688+0000 7f8b2c4ee7c0 -1 Falling back to public interface 2024-09-14T22:35:49.716 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:35:49.710+0000 7fead0dd07c0 -1 Falling back to public interface 2024-09-14T22:35:49.739 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:35:49.734+0000 7f82f42107c0 -1 Falling back to public interface 2024-09-14T22:35:50.143 INFO:tasks.ceph.mgr.x.smithi176.stderr:2024-09-14T22:35:50.136+0000 7f10fd0a7180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-14T22:35:50.217 INFO:tasks.ceph.mgr.x.smithi176.stderr:2024-09-14T22:35:50.212+0000 7f10fd0a7180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-14T22:35:50.246 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-14T22:35:50.238+0000 7f255868d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-14T22:35:50.268 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:35:50.262+0000 7fabec31c7c0 -1 Falling back to public interface 2024-09-14T22:35:50.325 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-14T22:35:50.318+0000 7f255868d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-14T22:35:50.399 INFO:tasks.ceph.mgr.x.smithi176.stderr:2024-09-14T22:35:50.392+0000 7f10fd0a7180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-14T22:35:50.472 INFO:tasks.ceph.mgr.x.smithi176.stderr:2024-09-14T22:35:50.468+0000 7f10fd0a7180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-14T22:35:50.515 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-14T22:35:50.510+0000 7f255868d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-14T22:35:50.592 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-14T22:35:50.586+0000 7f255868d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-14T22:35:50.762 INFO:tasks.ceph.mgr.x.smithi176.stderr:2024-09-14T22:35:50.756+0000 7f10fd0a7180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-14T22:35:50.823 INFO:tasks.ceph.mgr.x.smithi176.stderr:2024-09-14T22:35:50.816+0000 7f10fd0a7180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-14T22:35:50.902 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-14T22:35:50.894+0000 7f255868d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-14T22:35:50.969 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-14T22:35:50.962+0000 7f255868d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-14T22:35:51.873 INFO:tasks.ceph.mgr.x.smithi176.stderr:2024-09-14T22:35:51.867+0000 7f10a3ce0640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-14T22:35:51.874 INFO:tasks.ceph.mgr.x.smithi176.stderr:2024-09-14T22:35:51.867+0000 7f10a3ce0640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-14T22:35:53.531 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:35:53.526+0000 7feb6a1687c0 -1 osd.3 0 log_to_monitors true 2024-09-14T22:35:53.557 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:35:53.551+0000 7f17e6afb7c0 -1 osd.5 0 log_to_monitors true 2024-09-14T22:35:53.588 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:35:53.583+0000 7f363a2377c0 -1 osd.4 0 log_to_monitors true 2024-09-14T22:35:53.712 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:35:53.707+0000 7f00407fe7c0 -1 osd.6 0 log_to_monitors true 2024-09-14T22:35:53.736 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:35:53.730+0000 7fead0dd07c0 -1 osd.1 0 log_to_monitors true 2024-09-14T22:35:53.886 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:35:53.878+0000 7f82f42107c0 -1 osd.2 0 log_to_monitors true 2024-09-14T22:35:53.992 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:35:53.987+0000 7f8b2c4ee7c0 -1 osd.7 0 log_to_monitors true 2024-09-14T22:35:54.468 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:35:54.462+0000 7fabec31c7c0 -1 osd.0 0 log_to_monitors true 2024-09-14T22:35:55.799 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:35:55.794+0000 7feacd58b640 -1 osd.1 0 waiting for initial osdmap 2024-09-14T22:35:55.819 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:35:55.814+0000 7feac4bae640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T22:35:55.852 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:35:55.846+0000 7feb66111640 -1 osd.3 0 waiting for initial osdmap 2024-09-14T22:35:55.884 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:35:55.878+0000 7feb5df46640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T22:35:56.036 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:35:56.031+0000 7f36361e0640 -1 osd.4 0 waiting for initial osdmap 2024-09-14T22:35:56.070 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:35:56.063+0000 7f362e015640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T22:35:56.071 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:35:56.063+0000 7f003c7a7640 -1 osd.6 0 waiting for initial osdmap 2024-09-14T22:35:56.084 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:35:56.079+0000 7f17e2aa4640 -1 osd.5 0 waiting for initial osdmap 2024-09-14T22:35:56.104 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:35:56.099+0000 7f00345dc640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T22:35:56.121 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:35:56.115+0000 7f17da8d9640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T22:35:56.357 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-14T22:35:56.727 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:35:56.719+0000 7f8b28ca9640 -1 osd.7 0 waiting for initial osdmap 2024-09-14T22:35:56.748 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:35:56.743+0000 7f8b202cc640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T22:35:56.801 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:35:56.794+0000 7fabe82c5640 -1 osd.0 0 waiting for initial osdmap 2024-09-14T22:35:56.808 INFO:teuthology.misc.health.smithi001.stdout: 2024-09-14T22:35:56.809 INFO:teuthology.misc.health.smithi001.stdout:{"epoch":13,"fsid":"f07f407a-b17d-42c4-bf73-92dd80114a4f","created":"2024-09-14T22:35:45.983055+0000","modified":"2024-09-14T22:35:55.879280+0000","last_up_change":"2024-09-14T22:35:55.879280+0000","last_in_change":"2024-09-14T22:35:48.283160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"9831fcc5-9040-4a80-99d4-838225e65139","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7f79abe8-115d-46b2-8677-72fcb0c8ab7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":719990653},{"type":"v1","addr":"172.21.15.1:6809","nonce":719990653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":719990653},{"type":"v1","addr":"172.21.15.1:6811","nonce":719990653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":719990653},{"type":"v1","addr":"172.21.15.1:6815","nonce":719990653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":719990653},{"type":"v1","addr":"172.21.15.1:6813","nonce":719990653}]},"public_addr":"172.21.15.1:6809/719990653","cluster_addr":"172.21.15.1:6811/719990653","heartbeat_back_addr":"172.21.15.1:6815/719990653","heartbeat_front_addr":"172.21.15.1:6813/719990653","state":["exists","up"]},{"osd":2,"uuid":"1f5768b2-4db4-4b2e-8d5e-5206f8bde28d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"8103f3c7-7899-4fc3-bb02-dd2273120138","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"23cce5cb-726e-44c2-8edb-c2567a89dfbc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"eb2f187b-d2f1-49ab-ad5e-ea6dc441a28e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d371aed8-2ad2-49a5-8ece-4b7fd37b4822","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a012c819-8d49-4310-a840-c2752d9314ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T22:35:56.816 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:35:56.810+0000 7f82f01b9640 -1 osd.2 0 waiting for initial osdmap 2024-09-14T22:35:56.827 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-09-14T22:35:56.831 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:35:56.826+0000 7fabe00fa640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T22:35:56.855 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:35:56.850+0000 7f82e7fee640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T22:36:03.633 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-14T22:36:03.940 INFO:teuthology.misc.health.smithi001.stdout: 2024-09-14T22:36:03.940 INFO:teuthology.misc.health.smithi001.stdout:{"epoch":18,"fsid":"f07f407a-b17d-42c4-bf73-92dd80114a4f","created":"2024-09-14T22:35:45.983055+0000","modified":"2024-09-14T22:36:00.993882+0000","last_up_change":"2024-09-14T22:35:56.891281+0000","last_in_change":"2024-09-14T22:35:48.283160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T22:35:58.938784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9831fcc5-9040-4a80-99d4-838225e65139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1702929600},{"type":"v1","addr":"172.21.15.1:6825","nonce":1702929600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1702929600},{"type":"v1","addr":"172.21.15.1:6827","nonce":1702929600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":1702929600},{"type":"v1","addr":"172.21.15.1:6831","nonce":1702929600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":1702929600},{"type":"v1","addr":"172.21.15.1:6829","nonce":1702929600}]},"public_addr":"172.21.15.1:6825/1702929600","cluster_addr":"172.21.15.1:6827/1702929600","heartbeat_back_addr":"172.21.15.1:6831/1702929600","heartbeat_front_addr":"172.21.15.1:6829/1702929600","state":["exists","up"]},{"osd":1,"uuid":"7f79abe8-115d-46b2-8677-72fcb0c8ab7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":719990653},{"type":"v1","addr":"172.21.15.1:6809","nonce":719990653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":719990653},{"type":"v1","addr":"172.21.15.1:6811","nonce":719990653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":719990653},{"type":"v1","addr":"172.21.15.1:6815","nonce":719990653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":719990653},{"type":"v1","addr":"172.21.15.1:6813","nonce":719990653}]},"public_addr":"172.21.15.1:6809/719990653","cluster_addr":"172.21.15.1:6811/719990653","heartbeat_back_addr":"172.21.15.1:6815/719990653","heartbeat_front_addr":"172.21.15.1:6813/719990653","state":["exists","up"]},{"osd":2,"uuid":"1f5768b2-4db4-4b2e-8d5e-5206f8bde28d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2536940607},{"type":"v1","addr":"172.21.15.1:6817","nonce":2536940607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2536940607},{"type":"v1","addr":"172.21.15.1:6819","nonce":2536940607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2536940607},{"type":"v1","addr":"172.21.15.1:6823","nonce":2536940607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2536940607},{"type":"v1","addr":"172.21.15.1:6821","nonce":2536940607}]},"public_addr":"172.21.15.1:6817/2536940607","cluster_addr":"172.21.15.1:6819/2536940607","heartbeat_back_addr":"172.21.15.1:6823/2536940607","heartbeat_front_addr":"172.21.15.1:6821/2536940607","state":["exists","up"]},{"osd":3,"uuid":"8103f3c7-7899-4fc3-bb02-dd2273120138","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":750403671},{"type":"v1","addr":"172.21.15.1:6801","nonce":750403671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":750403671},{"type":"v1","addr":"172.21.15.1:6803","nonce":750403671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":750403671},{"type":"v1","addr":"172.21.15.1:6807","nonce":750403671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":750403671},{"type":"v1","addr":"172.21.15.1:6805","nonce":750403671}]},"public_addr":"172.21.15.1:6801/750403671","cluster_addr":"172.21.15.1:6803/750403671","heartbeat_back_addr":"172.21.15.1:6807/750403671","heartbeat_front_addr":"172.21.15.1:6805/750403671","state":["exists","up"]},{"osd":4,"uuid":"23cce5cb-726e-44c2-8edb-c2567a89dfbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":1408471829},{"type":"v1","addr":"172.21.15.176:6817","nonce":1408471829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":1408471829},{"type":"v1","addr":"172.21.15.176:6819","nonce":1408471829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":1408471829},{"type":"v1","addr":"172.21.15.176:6823","nonce":1408471829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":1408471829},{"type":"v1","addr":"172.21.15.176:6821","nonce":1408471829}]},"public_addr":"172.21.15.176:6817/1408471829","cluster_addr":"172.21.15.176:6819/1408471829","heartbeat_back_addr":"172.21.15.176:6823/1408471829","heartbeat_front_addr":"172.21.15.176:6821/1408471829","state":["exists","up"]},{"osd":5,"uuid":"eb2f187b-d2f1-49ab-ad5e-ea6dc441a28e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6805","nonce":1945902326},{"type":"v1","addr":"172.21.15.176:6808","nonce":1945902326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":1945902326},{"type":"v1","addr":"172.21.15.176:6811","nonce":1945902326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":1945902326},{"type":"v1","addr":"172.21.15.176:6815","nonce":1945902326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":1945902326},{"type":"v1","addr":"172.21.15.176:6813","nonce":1945902326}]},"public_addr":"172.21.15.176:6808/1945902326","cluster_addr":"172.21.15.176:6811/1945902326","heartbeat_back_addr":"172.21.15.176:6815/1945902326","heartbeat_front_addr":"172.21.15.176:6813/1945902326","state":["exists","up"]},{"osd":6,"uuid":"d371aed8-2ad2-49a5-8ece-4b7fd37b4822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":4116085586},{"type":"v1","addr":"172.21.15.176:6801","nonce":4116085586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":4116085586},{"type":"v1","addr":"172.21.15.176:6803","nonce":4116085586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6807","nonce":4116085586},{"type":"v1","addr":"172.21.15.176:6809","nonce":4116085586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":4116085586},{"type":"v1","addr":"172.21.15.176:6806","nonce":4116085586}]},"public_addr":"172.21.15.176:6801/4116085586","cluster_addr":"172.21.15.176:6803/4116085586","heartbeat_back_addr":"172.21.15.176:6809/4116085586","heartbeat_front_addr":"172.21.15.176:6806/4116085586","state":["exists","up"]},{"osd":7,"uuid":"a012c819-8d49-4310-a840-c2752d9314ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":877760244},{"type":"v1","addr":"172.21.15.176:6825","nonce":877760244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":877760244},{"type":"v1","addr":"172.21.15.176:6827","nonce":877760244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":877760244},{"type":"v1","addr":"172.21.15.176:6831","nonce":877760244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":877760244},{"type":"v1","addr":"172.21.15.176:6829","nonce":877760244}]},"public_addr":"172.21.15.176:6825/877760244","cluster_addr":"172.21.15.176:6827/877760244","heartbeat_back_addr":"172.21.15.176:6831/877760244","heartbeat_front_addr":"172.21.15.176:6829/877760244","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:55.511688+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.742414+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.871355+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.504553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.589640+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.518845+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.731632+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.952634+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T22:36:03.958 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-14T22:36:03.958 INFO:tasks.ceph:Creating RBD pool 2024-09-14T22:36:03.958 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-14T22:36:05.019 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'rbd' created 2024-09-14T22:36:05.044 DEBUG:teuthology.orchestra.run.smithi001:> rbd --cluster ceph pool init rbd 2024-09-14T22:36:08.073 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-14T22:36:08.074 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-14T22:36:08.074 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-14T22:36:08.074 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-14T22:36:08.075 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-14T22:36:08.430 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:36:08.449 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":446003092},{"type":"v1","addr":"172.21.15.176:6833","nonce":446003092}]},"active_addr":"172.21.15.176:6833/446003092","active_change":"2024-09-14T22:35:50.844374+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.176:0","nonce":3972972401}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.176:0","nonce":1473633451}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.176:0","nonce":2319098}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.176:0","nonce":3645090291}]}]} 2024-09-14T22:36:08.453 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-14T22:36:08.453 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-14T22:36:08.453 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T22:36:08.751 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:36:08.752 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":22,"fsid":"f07f407a-b17d-42c4-bf73-92dd80114a4f","created":"2024-09-14T22:35:45.983055+0000","modified":"2024-09-14T22:36:08.035426+0000","last_up_change":"2024-09-14T22:35:56.891281+0000","last_in_change":"2024-09-14T22:35:48.283160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T22:35:58.938784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T22:36:04.304242+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9831fcc5-9040-4a80-99d4-838225e65139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1702929600},{"type":"v1","addr":"172.21.15.1:6825","nonce":1702929600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1702929600},{"type":"v1","addr":"172.21.15.1:6827","nonce":1702929600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":1702929600},{"type":"v1","addr":"172.21.15.1:6831","nonce":1702929600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":1702929600},{"type":"v1","addr":"172.21.15.1:6829","nonce":1702929600}]},"public_addr":"172.21.15.1:6825/1702929600","cluster_addr":"172.21.15.1:6827/1702929600","heartbeat_back_addr":"172.21.15.1:6831/1702929600","heartbeat_front_addr":"172.21.15.1:6829/1702929600","state":["exists","up"]},{"osd":1,"uuid":"7f79abe8-115d-46b2-8677-72fcb0c8ab7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":719990653},{"type":"v1","addr":"172.21.15.1:6809","nonce":719990653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":719990653},{"type":"v1","addr":"172.21.15.1:6811","nonce":719990653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":719990653},{"type":"v1","addr":"172.21.15.1:6815","nonce":719990653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":719990653},{"type":"v1","addr":"172.21.15.1:6813","nonce":719990653}]},"public_addr":"172.21.15.1:6809/719990653","cluster_addr":"172.21.15.1:6811/719990653","heartbeat_back_addr":"172.21.15.1:6815/719990653","heartbeat_front_addr":"172.21.15.1:6813/719990653","state":["exists","up"]},{"osd":2,"uuid":"1f5768b2-4db4-4b2e-8d5e-5206f8bde28d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2536940607},{"type":"v1","addr":"172.21.15.1:6817","nonce":2536940607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2536940607},{"type":"v1","addr":"172.21.15.1:6819","nonce":2536940607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2536940607},{"type":"v1","addr":"172.21.15.1:6823","nonce":2536940607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2536940607},{"type":"v1","addr":"172.21.15.1:6821","nonce":2536940607}]},"public_addr":"172.21.15.1:6817/2536940607","cluster_addr":"172.21.15.1:6819/2536940607","heartbeat_back_addr":"172.21.15.1:6823/2536940607","heartbeat_front_addr":"172.21.15.1:6821/2536940607","state":["exists","up"]},{"osd":3,"uuid":"8103f3c7-7899-4fc3-bb02-dd2273120138","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":750403671},{"type":"v1","addr":"172.21.15.1:6801","nonce":750403671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":750403671},{"type":"v1","addr":"172.21.15.1:6803","nonce":750403671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":750403671},{"type":"v1","addr":"172.21.15.1:6807","nonce":750403671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":750403671},{"type":"v1","addr":"172.21.15.1:6805","nonce":750403671}]},"public_addr":"172.21.15.1:6801/750403671","cluster_addr":"172.21.15.1:6803/750403671","heartbeat_back_addr":"172.21.15.1:6807/750403671","heartbeat_front_addr":"172.21.15.1:6805/750403671","state":["exists","up"]},{"osd":4,"uuid":"23cce5cb-726e-44c2-8edb-c2567a89dfbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":1408471829},{"type":"v1","addr":"172.21.15.176:6817","nonce":1408471829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":1408471829},{"type":"v1","addr":"172.21.15.176:6819","nonce":1408471829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":1408471829},{"type":"v1","addr":"172.21.15.176:6823","nonce":1408471829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":1408471829},{"type":"v1","addr":"172.21.15.176:6821","nonce":1408471829}]},"public_addr":"172.21.15.176:6817/1408471829","cluster_addr":"172.21.15.176:6819/1408471829","heartbeat_back_addr":"172.21.15.176:6823/1408471829","heartbeat_front_addr":"172.21.15.176:6821/1408471829","state":["exists","up"]},{"osd":5,"uuid":"eb2f187b-d2f1-49ab-ad5e-ea6dc441a28e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6805","nonce":1945902326},{"type":"v1","addr":"172.21.15.176:6808","nonce":1945902326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":1945902326},{"type":"v1","addr":"172.21.15.176:6811","nonce":1945902326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":1945902326},{"type":"v1","addr":"172.21.15.176:6815","nonce":1945902326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":1945902326},{"type":"v1","addr":"172.21.15.176:6813","nonce":1945902326}]},"public_addr":"172.21.15.176:6808/1945902326","cluster_addr":"172.21.15.176:6811/1945902326","heartbeat_back_addr":"172.21.15.176:6815/1945902326","heartbeat_front_addr":"172.21.15.176:6813/1945902326","state":["exists","up"]},{"osd":6,"uuid":"d371aed8-2ad2-49a5-8ece-4b7fd37b4822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":4116085586},{"type":"v1","addr":"172.21.15.176:6801","nonce":4116085586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":4116085586},{"type":"v1","addr":"172.21.15.176:6803","nonce":4116085586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6807","nonce":4116085586},{"type":"v1","addr":"172.21.15.176:6809","nonce":4116085586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":4116085586},{"type":"v1","addr":"172.21.15.176:6806","nonce":4116085586}]},"public_addr":"172.21.15.176:6801/4116085586","cluster_addr":"172.21.15.176:6803/4116085586","heartbeat_back_addr":"172.21.15.176:6809/4116085586","heartbeat_front_addr":"172.21.15.176:6806/4116085586","state":["exists","up"]},{"osd":7,"uuid":"a012c819-8d49-4310-a840-c2752d9314ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":877760244},{"type":"v1","addr":"172.21.15.176:6825","nonce":877760244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":877760244},{"type":"v1","addr":"172.21.15.176:6827","nonce":877760244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":877760244},{"type":"v1","addr":"172.21.15.176:6831","nonce":877760244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":877760244},{"type":"v1","addr":"172.21.15.176:6829","nonce":877760244}]},"public_addr":"172.21.15.176:6825/877760244","cluster_addr":"172.21.15.176:6827/877760244","heartbeat_back_addr":"172.21.15.176:6831/877760244","heartbeat_front_addr":"172.21.15.176:6829/877760244","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:55.511688+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.742414+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.871355+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.504553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.589640+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.518845+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.731632+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.952634+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T22:36:08.771 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-14T22:36:08.771 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T22:36:09.081 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:36:09.081 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":22,"fsid":"f07f407a-b17d-42c4-bf73-92dd80114a4f","created":"2024-09-14T22:35:45.983055+0000","modified":"2024-09-14T22:36:08.035426+0000","last_up_change":"2024-09-14T22:35:56.891281+0000","last_in_change":"2024-09-14T22:35:48.283160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T22:35:58.938784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T22:36:04.304242+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9831fcc5-9040-4a80-99d4-838225e65139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1702929600},{"type":"v1","addr":"172.21.15.1:6825","nonce":1702929600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1702929600},{"type":"v1","addr":"172.21.15.1:6827","nonce":1702929600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":1702929600},{"type":"v1","addr":"172.21.15.1:6831","nonce":1702929600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":1702929600},{"type":"v1","addr":"172.21.15.1:6829","nonce":1702929600}]},"public_addr":"172.21.15.1:6825/1702929600","cluster_addr":"172.21.15.1:6827/1702929600","heartbeat_back_addr":"172.21.15.1:6831/1702929600","heartbeat_front_addr":"172.21.15.1:6829/1702929600","state":["exists","up"]},{"osd":1,"uuid":"7f79abe8-115d-46b2-8677-72fcb0c8ab7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":719990653},{"type":"v1","addr":"172.21.15.1:6809","nonce":719990653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":719990653},{"type":"v1","addr":"172.21.15.1:6811","nonce":719990653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":719990653},{"type":"v1","addr":"172.21.15.1:6815","nonce":719990653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":719990653},{"type":"v1","addr":"172.21.15.1:6813","nonce":719990653}]},"public_addr":"172.21.15.1:6809/719990653","cluster_addr":"172.21.15.1:6811/719990653","heartbeat_back_addr":"172.21.15.1:6815/719990653","heartbeat_front_addr":"172.21.15.1:6813/719990653","state":["exists","up"]},{"osd":2,"uuid":"1f5768b2-4db4-4b2e-8d5e-5206f8bde28d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2536940607},{"type":"v1","addr":"172.21.15.1:6817","nonce":2536940607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2536940607},{"type":"v1","addr":"172.21.15.1:6819","nonce":2536940607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2536940607},{"type":"v1","addr":"172.21.15.1:6823","nonce":2536940607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2536940607},{"type":"v1","addr":"172.21.15.1:6821","nonce":2536940607}]},"public_addr":"172.21.15.1:6817/2536940607","cluster_addr":"172.21.15.1:6819/2536940607","heartbeat_back_addr":"172.21.15.1:6823/2536940607","heartbeat_front_addr":"172.21.15.1:6821/2536940607","state":["exists","up"]},{"osd":3,"uuid":"8103f3c7-7899-4fc3-bb02-dd2273120138","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":750403671},{"type":"v1","addr":"172.21.15.1:6801","nonce":750403671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":750403671},{"type":"v1","addr":"172.21.15.1:6803","nonce":750403671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":750403671},{"type":"v1","addr":"172.21.15.1:6807","nonce":750403671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":750403671},{"type":"v1","addr":"172.21.15.1:6805","nonce":750403671}]},"public_addr":"172.21.15.1:6801/750403671","cluster_addr":"172.21.15.1:6803/750403671","heartbeat_back_addr":"172.21.15.1:6807/750403671","heartbeat_front_addr":"172.21.15.1:6805/750403671","state":["exists","up"]},{"osd":4,"uuid":"23cce5cb-726e-44c2-8edb-c2567a89dfbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":1408471829},{"type":"v1","addr":"172.21.15.176:6817","nonce":1408471829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":1408471829},{"type":"v1","addr":"172.21.15.176:6819","nonce":1408471829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":1408471829},{"type":"v1","addr":"172.21.15.176:6823","nonce":1408471829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":1408471829},{"type":"v1","addr":"172.21.15.176:6821","nonce":1408471829}]},"public_addr":"172.21.15.176:6817/1408471829","cluster_addr":"172.21.15.176:6819/1408471829","heartbeat_back_addr":"172.21.15.176:6823/1408471829","heartbeat_front_addr":"172.21.15.176:6821/1408471829","state":["exists","up"]},{"osd":5,"uuid":"eb2f187b-d2f1-49ab-ad5e-ea6dc441a28e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6805","nonce":1945902326},{"type":"v1","addr":"172.21.15.176:6808","nonce":1945902326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":1945902326},{"type":"v1","addr":"172.21.15.176:6811","nonce":1945902326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":1945902326},{"type":"v1","addr":"172.21.15.176:6815","nonce":1945902326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":1945902326},{"type":"v1","addr":"172.21.15.176:6813","nonce":1945902326}]},"public_addr":"172.21.15.176:6808/1945902326","cluster_addr":"172.21.15.176:6811/1945902326","heartbeat_back_addr":"172.21.15.176:6815/1945902326","heartbeat_front_addr":"172.21.15.176:6813/1945902326","state":["exists","up"]},{"osd":6,"uuid":"d371aed8-2ad2-49a5-8ece-4b7fd37b4822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":4116085586},{"type":"v1","addr":"172.21.15.176:6801","nonce":4116085586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":4116085586},{"type":"v1","addr":"172.21.15.176:6803","nonce":4116085586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6807","nonce":4116085586},{"type":"v1","addr":"172.21.15.176:6809","nonce":4116085586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":4116085586},{"type":"v1","addr":"172.21.15.176:6806","nonce":4116085586}]},"public_addr":"172.21.15.176:6801/4116085586","cluster_addr":"172.21.15.176:6803/4116085586","heartbeat_back_addr":"172.21.15.176:6809/4116085586","heartbeat_front_addr":"172.21.15.176:6806/4116085586","state":["exists","up"]},{"osd":7,"uuid":"a012c819-8d49-4310-a840-c2752d9314ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":877760244},{"type":"v1","addr":"172.21.15.176:6825","nonce":877760244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":877760244},{"type":"v1","addr":"172.21.15.176:6827","nonce":877760244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":877760244},{"type":"v1","addr":"172.21.15.176:6831","nonce":877760244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":877760244},{"type":"v1","addr":"172.21.15.176:6829","nonce":877760244}]},"public_addr":"172.21.15.176:6825/877760244","cluster_addr":"172.21.15.176:6827/877760244","heartbeat_back_addr":"172.21.15.176:6831/877760244","heartbeat_front_addr":"172.21.15.176:6829/877760244","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:55.511688+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.742414+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.871355+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.504553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.589640+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.518845+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.731632+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.952634+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T22:36:09.100 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-14T22:36:09.101 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-14T22:36:09.101 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-14T22:36:09.102 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-14T22:36:09.102 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-14T22:36:09.102 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-14T22:36:09.103 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-14T22:36:09.103 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-14T22:36:09.354 INFO:teuthology.orchestra.run.smithi001.stdout:60129542150 2024-09-14T22:36:09.354 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T22:36:09.366 INFO:teuthology.orchestra.run.smithi001.stdout:60129542148 2024-09-14T22:36:09.366 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T22:36:09.368 INFO:teuthology.orchestra.run.smithi001.stdout:60129542149 2024-09-14T22:36:09.368 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T22:36:09.375 INFO:teuthology.orchestra.run.smithi001.stdout:60129542149 2024-09-14T22:36:09.376 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T22:36:09.377 INFO:teuthology.orchestra.run.smithi001.stdout:55834574854 2024-09-14T22:36:09.377 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T22:36:09.382 INFO:teuthology.orchestra.run.smithi001.stdout:60129542150 2024-09-14T22:36:09.382 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-14T22:36:09.386 INFO:teuthology.orchestra.run.smithi001.stdout:60129542149 2024-09-14T22:36:09.386 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-14T22:36:09.388 INFO:teuthology.orchestra.run.smithi001.stdout:60129542150 2024-09-14T22:36:09.389 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T22:36:09.859 INFO:teuthology.orchestra.run.smithi001.stdout:55834574853 2024-09-14T22:36:09.862 INFO:teuthology.orchestra.run.smithi001.stdout:60129542149 2024-09-14T22:36:09.872 INFO:teuthology.orchestra.run.smithi001.stdout:60129542148 2024-09-14T22:36:09.888 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2024-09-14T22:36:09.898 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.3 2024-09-14T22:36:09.903 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.0 2024-09-14T22:36:09.908 INFO:teuthology.orchestra.run.smithi001.stdout:60129542147 2024-09-14T22:36:09.909 INFO:teuthology.orchestra.run.smithi001.stdout:60129542149 2024-09-14T22:36:09.920 INFO:teuthology.orchestra.run.smithi001.stdout:60129542148 2024-09-14T22:36:09.935 INFO:teuthology.orchestra.run.smithi001.stdout:60129542149 2024-09-14T22:36:09.938 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.6 2024-09-14T22:36:09.938 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.2 2024-09-14T22:36:09.941 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.4 2024-09-14T22:36:09.953 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.5 2024-09-14T22:36:09.965 INFO:teuthology.orchestra.run.smithi001.stdout:60129542148 2024-09-14T22:36:09.982 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.7 2024-09-14T22:36:10.889 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T22:36:10.899 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T22:36:10.903 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T22:36:10.939 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-14T22:36:10.939 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T22:36:10.942 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T22:36:10.954 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T22:36:10.983 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-14T22:36:11.397 INFO:teuthology.orchestra.run.smithi001.stdout:55834574854 2024-09-14T22:36:11.429 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2024-09-14T22:36:11.429 DEBUG:teuthology.parallel:result is None 2024-09-14T22:36:11.454 INFO:teuthology.orchestra.run.smithi001.stdout:60129542150 2024-09-14T22:36:11.474 INFO:teuthology.orchestra.run.smithi001.stdout:60129542150 2024-09-14T22:36:11.480 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.3 2024-09-14T22:36:11.480 DEBUG:teuthology.parallel:result is None 2024-09-14T22:36:11.486 INFO:teuthology.orchestra.run.smithi001.stdout:60129542149 2024-09-14T22:36:11.493 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.5 2024-09-14T22:36:11.494 DEBUG:teuthology.parallel:result is None 2024-09-14T22:36:11.510 INFO:teuthology.orchestra.run.smithi001.stdout:60129542148 2024-09-14T22:36:11.511 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.0 2024-09-14T22:36:11.511 DEBUG:teuthology.parallel:result is None 2024-09-14T22:36:11.515 INFO:teuthology.orchestra.run.smithi001.stdout:60129542150 2024-09-14T22:36:11.530 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.2 2024-09-14T22:36:11.530 DEBUG:teuthology.parallel:result is None 2024-09-14T22:36:11.530 INFO:teuthology.orchestra.run.smithi001.stdout:60129542149 2024-09-14T22:36:11.534 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.6 2024-09-14T22:36:11.534 DEBUG:teuthology.parallel:result is None 2024-09-14T22:36:11.548 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.4 2024-09-14T22:36:11.549 DEBUG:teuthology.parallel:result is None 2024-09-14T22:36:11.559 INFO:teuthology.orchestra.run.smithi001.stdout:60129542149 2024-09-14T22:36:11.575 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.7 2024-09-14T22:36:11.576 DEBUG:teuthology.parallel:result is None 2024-09-14T22:36:11.576 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-14T22:36:11.576 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:36:11.896 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:36:11.897 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-14T22:36:11.913 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-14T22:36:10.859994+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":234760,"kb_used_data":1880,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739960,"statfs":{"total":773094113280,"available":772853719040,"internally_reserved":0,"allocated":1925120,"data_stored":1796790,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.841221"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.042227+0000","last_change":"2024-09-14T22:36:08.042860+0000","last_active":"2024-09-14T22:36:08.042227+0000","last_peered":"2024-09-14T22:36:08.042227+0000","last_clean":"2024-09-14T22:36:08.042227+0000","last_became_active":"2024-09-14T22:36:06.040961+0000","last_became_peered":"2024-09-14T22:36:06.040961+0000","last_unstale":"2024-09-14T22:36:08.042227+0000","last_undegraded":"2024-09-14T22:36:08.042227+0000","last_fullsized":"2024-09-14T22:36:08.042227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:03:48.547834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010780950000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.042735+0000","last_change":"2024-09-14T22:36:08.043219+0000","last_active":"2024-09-14T22:36:08.042735+0000","last_peered":"2024-09-14T22:36:08.042735+0000","last_clean":"2024-09-14T22:36:08.042735+0000","last_became_active":"2024-09-14T22:36:06.039140+0000","last_became_peered":"2024-09-14T22:36:06.039140+0000","last_unstale":"2024-09-14T22:36:08.042735+0000","last_undegraded":"2024-09-14T22:36:08.042735+0000","last_fullsized":"2024-09-14T22:36:08.042735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:59:25.513497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086089100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.054127+0000","last_change":"2024-09-14T22:36:08.054606+0000","last_active":"2024-09-14T22:36:08.054127+0000","last_peered":"2024-09-14T22:36:08.054127+0000","last_clean":"2024-09-14T22:36:08.054127+0000","last_became_active":"2024-09-14T22:36:06.266908+0000","last_became_peered":"2024-09-14T22:36:06.266908+0000","last_unstale":"2024-09-14T22:36:08.054127+0000","last_undegraded":"2024-09-14T22:36:08.054127+0000","last_fullsized":"2024-09-14T22:36:08.054127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:36.738643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098249300000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.042581+0000","last_change":"2024-09-14T22:36:08.043286+0000","last_active":"2024-09-14T22:36:08.042581+0000","last_peered":"2024-09-14T22:36:08.042581+0000","last_clean":"2024-09-14T22:36:08.042581+0000","last_became_active":"2024-09-14T22:36:06.268228+0000","last_became_peered":"2024-09-14T22:36:06.268228+0000","last_unstale":"2024-09-14T22:36:08.042581+0000","last_undegraded":"2024-09-14T22:36:08.042581+0000","last_fullsized":"2024-09-14T22:36:08.042581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:46:31.845583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010230490000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.055938+0000","last_change":"2024-09-14T22:36:08.055938+0000","last_active":"2024-09-14T22:36:08.055938+0000","last_peered":"2024-09-14T22:36:08.055938+0000","last_clean":"2024-09-14T22:36:08.055938+0000","last_became_active":"2024-09-14T22:36:06.039600+0000","last_became_peered":"2024-09-14T22:36:06.039600+0000","last_unstale":"2024-09-14T22:36:08.055938+0000","last_undegraded":"2024-09-14T22:36:08.055938+0000","last_fullsized":"2024-09-14T22:36:08.055938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:49:09.245563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021617870000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.043157+0000","last_change":"2024-09-14T22:36:08.043683+0000","last_active":"2024-09-14T22:36:08.043157+0000","last_peered":"2024-09-14T22:36:08.043157+0000","last_clean":"2024-09-14T22:36:08.043157+0000","last_became_active":"2024-09-14T22:36:06.268172+0000","last_became_peered":"2024-09-14T22:36:06.268172+0000","last_unstale":"2024-09-14T22:36:08.043157+0000","last_undegraded":"2024-09-14T22:36:08.043157+0000","last_fullsized":"2024-09-14T22:36:08.043157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:08:21.029422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010670619999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.054264+0000","last_change":"2024-09-14T22:36:08.054759+0000","last_active":"2024-09-14T22:36:08.054264+0000","last_peered":"2024-09-14T22:36:08.054264+0000","last_clean":"2024-09-14T22:36:08.054264+0000","last_became_active":"2024-09-14T22:36:06.039969+0000","last_became_peered":"2024-09-14T22:36:06.039969+0000","last_unstale":"2024-09-14T22:36:08.054264+0000","last_undegraded":"2024-09-14T22:36:08.054264+0000","last_fullsized":"2024-09-14T22:36:08.054264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:03:45.369532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010817520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.054346+0000","last_change":"2024-09-14T22:36:08.054925+0000","last_active":"2024-09-14T22:36:08.054346+0000","last_peered":"2024-09-14T22:36:08.054346+0000","last_clean":"2024-09-14T22:36:08.054346+0000","last_became_active":"2024-09-14T22:36:06.038679+0000","last_became_peered":"2024-09-14T22:36:06.038679+0000","last_unstale":"2024-09-14T22:36:08.054346+0000","last_undegraded":"2024-09-14T22:36:08.054346+0000","last_fullsized":"2024-09-14T22:36:08.054346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:34:54.969526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011401880000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.053387+0000","last_change":"2024-09-14T22:36:00.402862+0000","last_active":"2024-09-14T22:36:08.053387+0000","last_peered":"2024-09-14T22:36:08.053387+0000","last_clean":"2024-09-14T22:36:08.053387+0000","last_became_active":"2024-09-14T22:36:00.400529+0000","last_became_peered":"2024-09-14T22:36:00.400529+0000","last_unstale":"2024-09-14T22:36:08.053387+0000","last_undegraded":"2024-09-14T22:36:08.053387+0000","last_fullsized":"2024-09-14T22:36:08.053387+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_clean_scrub_stamp":"2024-09-14T22:35:58.989025+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:07:12.951728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":282624,"data_stored":667370,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":77002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":77021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":77002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":77002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":77002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":77021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31472,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340368,"statfs":{"total":96636764160,"available":96604536832,"internally_reserved":0,"allocated":282624,"data_stored":667370,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:36:11.914 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:36:12.200 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:36:12.200 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-14T22:36:12.218 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-14T22:36:10.859994+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":234760,"kb_used_data":1880,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739960,"statfs":{"total":773094113280,"available":772853719040,"internally_reserved":0,"allocated":1925120,"data_stored":1796790,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.841221"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.042227+0000","last_change":"2024-09-14T22:36:08.042860+0000","last_active":"2024-09-14T22:36:08.042227+0000","last_peered":"2024-09-14T22:36:08.042227+0000","last_clean":"2024-09-14T22:36:08.042227+0000","last_became_active":"2024-09-14T22:36:06.040961+0000","last_became_peered":"2024-09-14T22:36:06.040961+0000","last_unstale":"2024-09-14T22:36:08.042227+0000","last_undegraded":"2024-09-14T22:36:08.042227+0000","last_fullsized":"2024-09-14T22:36:08.042227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:03:48.547834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010780950000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.042735+0000","last_change":"2024-09-14T22:36:08.043219+0000","last_active":"2024-09-14T22:36:08.042735+0000","last_peered":"2024-09-14T22:36:08.042735+0000","last_clean":"2024-09-14T22:36:08.042735+0000","last_became_active":"2024-09-14T22:36:06.039140+0000","last_became_peered":"2024-09-14T22:36:06.039140+0000","last_unstale":"2024-09-14T22:36:08.042735+0000","last_undegraded":"2024-09-14T22:36:08.042735+0000","last_fullsized":"2024-09-14T22:36:08.042735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:59:25.513497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086089100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.054127+0000","last_change":"2024-09-14T22:36:08.054606+0000","last_active":"2024-09-14T22:36:08.054127+0000","last_peered":"2024-09-14T22:36:08.054127+0000","last_clean":"2024-09-14T22:36:08.054127+0000","last_became_active":"2024-09-14T22:36:06.266908+0000","last_became_peered":"2024-09-14T22:36:06.266908+0000","last_unstale":"2024-09-14T22:36:08.054127+0000","last_undegraded":"2024-09-14T22:36:08.054127+0000","last_fullsized":"2024-09-14T22:36:08.054127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:36.738643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098249300000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.042581+0000","last_change":"2024-09-14T22:36:08.043286+0000","last_active":"2024-09-14T22:36:08.042581+0000","last_peered":"2024-09-14T22:36:08.042581+0000","last_clean":"2024-09-14T22:36:08.042581+0000","last_became_active":"2024-09-14T22:36:06.268228+0000","last_became_peered":"2024-09-14T22:36:06.268228+0000","last_unstale":"2024-09-14T22:36:08.042581+0000","last_undegraded":"2024-09-14T22:36:08.042581+0000","last_fullsized":"2024-09-14T22:36:08.042581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:46:31.845583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010230490000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.055938+0000","last_change":"2024-09-14T22:36:08.055938+0000","last_active":"2024-09-14T22:36:08.055938+0000","last_peered":"2024-09-14T22:36:08.055938+0000","last_clean":"2024-09-14T22:36:08.055938+0000","last_became_active":"2024-09-14T22:36:06.039600+0000","last_became_peered":"2024-09-14T22:36:06.039600+0000","last_unstale":"2024-09-14T22:36:08.055938+0000","last_undegraded":"2024-09-14T22:36:08.055938+0000","last_fullsized":"2024-09-14T22:36:08.055938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:49:09.245563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021617870000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.043157+0000","last_change":"2024-09-14T22:36:08.043683+0000","last_active":"2024-09-14T22:36:08.043157+0000","last_peered":"2024-09-14T22:36:08.043157+0000","last_clean":"2024-09-14T22:36:08.043157+0000","last_became_active":"2024-09-14T22:36:06.268172+0000","last_became_peered":"2024-09-14T22:36:06.268172+0000","last_unstale":"2024-09-14T22:36:08.043157+0000","last_undegraded":"2024-09-14T22:36:08.043157+0000","last_fullsized":"2024-09-14T22:36:08.043157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:08:21.029422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010670619999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.054264+0000","last_change":"2024-09-14T22:36:08.054759+0000","last_active":"2024-09-14T22:36:08.054264+0000","last_peered":"2024-09-14T22:36:08.054264+0000","last_clean":"2024-09-14T22:36:08.054264+0000","last_became_active":"2024-09-14T22:36:06.039969+0000","last_became_peered":"2024-09-14T22:36:06.039969+0000","last_unstale":"2024-09-14T22:36:08.054264+0000","last_undegraded":"2024-09-14T22:36:08.054264+0000","last_fullsized":"2024-09-14T22:36:08.054264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:03:45.369532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010817520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.054346+0000","last_change":"2024-09-14T22:36:08.054925+0000","last_active":"2024-09-14T22:36:08.054346+0000","last_peered":"2024-09-14T22:36:08.054346+0000","last_clean":"2024-09-14T22:36:08.054346+0000","last_became_active":"2024-09-14T22:36:06.038679+0000","last_became_peered":"2024-09-14T22:36:06.038679+0000","last_unstale":"2024-09-14T22:36:08.054346+0000","last_undegraded":"2024-09-14T22:36:08.054346+0000","last_fullsized":"2024-09-14T22:36:08.054346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:34:54.969526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011401880000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.053387+0000","last_change":"2024-09-14T22:36:00.402862+0000","last_active":"2024-09-14T22:36:08.053387+0000","last_peered":"2024-09-14T22:36:08.053387+0000","last_clean":"2024-09-14T22:36:08.053387+0000","last_became_active":"2024-09-14T22:36:00.400529+0000","last_became_peered":"2024-09-14T22:36:00.400529+0000","last_unstale":"2024-09-14T22:36:08.053387+0000","last_undegraded":"2024-09-14T22:36:08.053387+0000","last_fullsized":"2024-09-14T22:36:08.053387+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_clean_scrub_stamp":"2024-09-14T22:35:58.989025+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:07:12.951728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":282624,"data_stored":667370,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":77002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":77021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":77002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":77002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":77002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":77021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31472,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340368,"statfs":{"total":96636764160,"available":96604536832,"internally_reserved":0,"allocated":282624,"data_stored":667370,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:36:12.219 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-14T22:36:12.219 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-14T22:36:12.219 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-14T22:36:12.219 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-14T22:36:12.533 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:36:12.534 INFO:teuthology.orchestra.run.smithi001.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-14T22:36:12.551 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-14T22:36:12.551 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-14T22:36:12.561 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-14T22:36:12.561 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writearound'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}} 2024-09-14T22:36:12.561 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-14T22:36:12.562 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-14T22:36:12.871 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-14T22:36:12.871 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-14T22:36:12.871 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:36:13.153 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:36:13.154 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-14T22:36:13.171 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-14T22:36:12.860661+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":234760,"kb_used_data":1880,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739960,"statfs":{"total":773094113280,"available":772853719040,"internally_reserved":0,"allocated":1925120,"data_stored":1796790,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.832200"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.042227+0000","last_change":"2024-09-14T22:36:08.042860+0000","last_active":"2024-09-14T22:36:08.042227+0000","last_peered":"2024-09-14T22:36:08.042227+0000","last_clean":"2024-09-14T22:36:08.042227+0000","last_became_active":"2024-09-14T22:36:06.040961+0000","last_became_peered":"2024-09-14T22:36:06.040961+0000","last_unstale":"2024-09-14T22:36:08.042227+0000","last_undegraded":"2024-09-14T22:36:08.042227+0000","last_fullsized":"2024-09-14T22:36:08.042227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:03:48.547834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010780950000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.042735+0000","last_change":"2024-09-14T22:36:08.043219+0000","last_active":"2024-09-14T22:36:08.042735+0000","last_peered":"2024-09-14T22:36:08.042735+0000","last_clean":"2024-09-14T22:36:08.042735+0000","last_became_active":"2024-09-14T22:36:06.039140+0000","last_became_peered":"2024-09-14T22:36:06.039140+0000","last_unstale":"2024-09-14T22:36:08.042735+0000","last_undegraded":"2024-09-14T22:36:08.042735+0000","last_fullsized":"2024-09-14T22:36:08.042735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:59:25.513497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086089100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.054127+0000","last_change":"2024-09-14T22:36:08.054606+0000","last_active":"2024-09-14T22:36:08.054127+0000","last_peered":"2024-09-14T22:36:08.054127+0000","last_clean":"2024-09-14T22:36:08.054127+0000","last_became_active":"2024-09-14T22:36:06.266908+0000","last_became_peered":"2024-09-14T22:36:06.266908+0000","last_unstale":"2024-09-14T22:36:08.054127+0000","last_undegraded":"2024-09-14T22:36:08.054127+0000","last_fullsized":"2024-09-14T22:36:08.054127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:36.738643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098249300000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.042581+0000","last_change":"2024-09-14T22:36:08.043286+0000","last_active":"2024-09-14T22:36:08.042581+0000","last_peered":"2024-09-14T22:36:08.042581+0000","last_clean":"2024-09-14T22:36:08.042581+0000","last_became_active":"2024-09-14T22:36:06.268228+0000","last_became_peered":"2024-09-14T22:36:06.268228+0000","last_unstale":"2024-09-14T22:36:08.042581+0000","last_undegraded":"2024-09-14T22:36:08.042581+0000","last_fullsized":"2024-09-14T22:36:08.042581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:46:31.845583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010230490000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.055938+0000","last_change":"2024-09-14T22:36:08.055938+0000","last_active":"2024-09-14T22:36:08.055938+0000","last_peered":"2024-09-14T22:36:08.055938+0000","last_clean":"2024-09-14T22:36:08.055938+0000","last_became_active":"2024-09-14T22:36:06.039600+0000","last_became_peered":"2024-09-14T22:36:06.039600+0000","last_unstale":"2024-09-14T22:36:08.055938+0000","last_undegraded":"2024-09-14T22:36:08.055938+0000","last_fullsized":"2024-09-14T22:36:08.055938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:49:09.245563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021617870000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.043157+0000","last_change":"2024-09-14T22:36:08.043683+0000","last_active":"2024-09-14T22:36:08.043157+0000","last_peered":"2024-09-14T22:36:08.043157+0000","last_clean":"2024-09-14T22:36:08.043157+0000","last_became_active":"2024-09-14T22:36:06.268172+0000","last_became_peered":"2024-09-14T22:36:06.268172+0000","last_unstale":"2024-09-14T22:36:08.043157+0000","last_undegraded":"2024-09-14T22:36:08.043157+0000","last_fullsized":"2024-09-14T22:36:08.043157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:08:21.029422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010670619999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.054264+0000","last_change":"2024-09-14T22:36:08.054759+0000","last_active":"2024-09-14T22:36:08.054264+0000","last_peered":"2024-09-14T22:36:08.054264+0000","last_clean":"2024-09-14T22:36:08.054264+0000","last_became_active":"2024-09-14T22:36:06.039969+0000","last_became_peered":"2024-09-14T22:36:06.039969+0000","last_unstale":"2024-09-14T22:36:08.054264+0000","last_undegraded":"2024-09-14T22:36:08.054264+0000","last_fullsized":"2024-09-14T22:36:08.054264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:03:45.369532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010817520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.054346+0000","last_change":"2024-09-14T22:36:08.054925+0000","last_active":"2024-09-14T22:36:08.054346+0000","last_peered":"2024-09-14T22:36:08.054346+0000","last_clean":"2024-09-14T22:36:08.054346+0000","last_became_active":"2024-09-14T22:36:06.038679+0000","last_became_peered":"2024-09-14T22:36:06.038679+0000","last_unstale":"2024-09-14T22:36:08.054346+0000","last_undegraded":"2024-09-14T22:36:08.054346+0000","last_fullsized":"2024-09-14T22:36:08.054346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:34:54.969526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011401880000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.053387+0000","last_change":"2024-09-14T22:36:00.402862+0000","last_active":"2024-09-14T22:36:08.053387+0000","last_peered":"2024-09-14T22:36:08.053387+0000","last_clean":"2024-09-14T22:36:08.053387+0000","last_became_active":"2024-09-14T22:36:00.400529+0000","last_became_peered":"2024-09-14T22:36:00.400529+0000","last_unstale":"2024-09-14T22:36:08.053387+0000","last_undegraded":"2024-09-14T22:36:08.053387+0000","last_fullsized":"2024-09-14T22:36:08.053387+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_clean_scrub_stamp":"2024-09-14T22:35:58.989025+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:07:12.951728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":282624,"data_stored":667370,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":77002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":77021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":77002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":77002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":77002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":77021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31472,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340368,"statfs":{"total":96636764160,"available":96604536832,"internally_reserved":0,"allocated":282624,"data_stored":667370,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:36:13.172 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:36:13.467 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:36:13.467 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-14T22:36:13.483 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-14T22:36:12.860661+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":234760,"kb_used_data":1880,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739960,"statfs":{"total":773094113280,"available":772853719040,"internally_reserved":0,"allocated":1925120,"data_stored":1796790,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.832200"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.042227+0000","last_change":"2024-09-14T22:36:08.042860+0000","last_active":"2024-09-14T22:36:08.042227+0000","last_peered":"2024-09-14T22:36:08.042227+0000","last_clean":"2024-09-14T22:36:08.042227+0000","last_became_active":"2024-09-14T22:36:06.040961+0000","last_became_peered":"2024-09-14T22:36:06.040961+0000","last_unstale":"2024-09-14T22:36:08.042227+0000","last_undegraded":"2024-09-14T22:36:08.042227+0000","last_fullsized":"2024-09-14T22:36:08.042227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:03:48.547834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010780950000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.042735+0000","last_change":"2024-09-14T22:36:08.043219+0000","last_active":"2024-09-14T22:36:08.042735+0000","last_peered":"2024-09-14T22:36:08.042735+0000","last_clean":"2024-09-14T22:36:08.042735+0000","last_became_active":"2024-09-14T22:36:06.039140+0000","last_became_peered":"2024-09-14T22:36:06.039140+0000","last_unstale":"2024-09-14T22:36:08.042735+0000","last_undegraded":"2024-09-14T22:36:08.042735+0000","last_fullsized":"2024-09-14T22:36:08.042735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:59:25.513497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086089100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.054127+0000","last_change":"2024-09-14T22:36:08.054606+0000","last_active":"2024-09-14T22:36:08.054127+0000","last_peered":"2024-09-14T22:36:08.054127+0000","last_clean":"2024-09-14T22:36:08.054127+0000","last_became_active":"2024-09-14T22:36:06.266908+0000","last_became_peered":"2024-09-14T22:36:06.266908+0000","last_unstale":"2024-09-14T22:36:08.054127+0000","last_undegraded":"2024-09-14T22:36:08.054127+0000","last_fullsized":"2024-09-14T22:36:08.054127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:36.738643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098249300000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.042581+0000","last_change":"2024-09-14T22:36:08.043286+0000","last_active":"2024-09-14T22:36:08.042581+0000","last_peered":"2024-09-14T22:36:08.042581+0000","last_clean":"2024-09-14T22:36:08.042581+0000","last_became_active":"2024-09-14T22:36:06.268228+0000","last_became_peered":"2024-09-14T22:36:06.268228+0000","last_unstale":"2024-09-14T22:36:08.042581+0000","last_undegraded":"2024-09-14T22:36:08.042581+0000","last_fullsized":"2024-09-14T22:36:08.042581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:46:31.845583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010230490000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.055938+0000","last_change":"2024-09-14T22:36:08.055938+0000","last_active":"2024-09-14T22:36:08.055938+0000","last_peered":"2024-09-14T22:36:08.055938+0000","last_clean":"2024-09-14T22:36:08.055938+0000","last_became_active":"2024-09-14T22:36:06.039600+0000","last_became_peered":"2024-09-14T22:36:06.039600+0000","last_unstale":"2024-09-14T22:36:08.055938+0000","last_undegraded":"2024-09-14T22:36:08.055938+0000","last_fullsized":"2024-09-14T22:36:08.055938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:49:09.245563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021617870000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.043157+0000","last_change":"2024-09-14T22:36:08.043683+0000","last_active":"2024-09-14T22:36:08.043157+0000","last_peered":"2024-09-14T22:36:08.043157+0000","last_clean":"2024-09-14T22:36:08.043157+0000","last_became_active":"2024-09-14T22:36:06.268172+0000","last_became_peered":"2024-09-14T22:36:06.268172+0000","last_unstale":"2024-09-14T22:36:08.043157+0000","last_undegraded":"2024-09-14T22:36:08.043157+0000","last_fullsized":"2024-09-14T22:36:08.043157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:08:21.029422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010670619999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.054264+0000","last_change":"2024-09-14T22:36:08.054759+0000","last_active":"2024-09-14T22:36:08.054264+0000","last_peered":"2024-09-14T22:36:08.054264+0000","last_clean":"2024-09-14T22:36:08.054264+0000","last_became_active":"2024-09-14T22:36:06.039969+0000","last_became_peered":"2024-09-14T22:36:06.039969+0000","last_unstale":"2024-09-14T22:36:08.054264+0000","last_undegraded":"2024-09-14T22:36:08.054264+0000","last_fullsized":"2024-09-14T22:36:08.054264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:03:45.369532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010817520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.054346+0000","last_change":"2024-09-14T22:36:08.054925+0000","last_active":"2024-09-14T22:36:08.054346+0000","last_peered":"2024-09-14T22:36:08.054346+0000","last_clean":"2024-09-14T22:36:08.054346+0000","last_became_active":"2024-09-14T22:36:06.038679+0000","last_became_peered":"2024-09-14T22:36:06.038679+0000","last_unstale":"2024-09-14T22:36:08.054346+0000","last_undegraded":"2024-09-14T22:36:08.054346+0000","last_fullsized":"2024-09-14T22:36:08.054346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:34:54.969526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011401880000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:36:08.053387+0000","last_change":"2024-09-14T22:36:00.402862+0000","last_active":"2024-09-14T22:36:08.053387+0000","last_peered":"2024-09-14T22:36:08.053387+0000","last_clean":"2024-09-14T22:36:08.053387+0000","last_became_active":"2024-09-14T22:36:00.400529+0000","last_became_peered":"2024-09-14T22:36:00.400529+0000","last_unstale":"2024-09-14T22:36:08.053387+0000","last_undegraded":"2024-09-14T22:36:08.053387+0000","last_fullsized":"2024-09-14T22:36:08.053387+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_clean_scrub_stamp":"2024-09-14T22:35:58.989025+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:07:12.951728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":282624,"data_stored":667370,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":77002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":77021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":77002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":77002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":77002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":77021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31472,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340368,"statfs":{"total":96636764160,"available":96604536832,"internally_reserved":0,"allocated":282624,"data_stored":667370,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:36:13.484 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-14T22:36:13.484 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-14T22:36:13.775 INFO:teuthology.orchestra.run.smithi001.stdout:epoch 22 2024-09-14T22:36:13.775 INFO:teuthology.orchestra.run.smithi001.stdout:fsid f07f407a-b17d-42c4-bf73-92dd80114a4f 2024-09-14T22:36:13.775 INFO:teuthology.orchestra.run.smithi001.stdout:created 2024-09-14T22:35:45.983055+0000 2024-09-14T22:36:13.775 INFO:teuthology.orchestra.run.smithi001.stdout:modified 2024-09-14T22:36:08.035426+0000 2024-09-14T22:36:13.776 INFO:teuthology.orchestra.run.smithi001.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-14T22:36:13.776 INFO:teuthology.orchestra.run.smithi001.stdout:crush_version 5 2024-09-14T22:36:13.776 INFO:teuthology.orchestra.run.smithi001.stdout:full_ratio 0.95 2024-09-14T22:36:13.776 INFO:teuthology.orchestra.run.smithi001.stdout:backfillfull_ratio 0.9 2024-09-14T22:36:13.776 INFO:teuthology.orchestra.run.smithi001.stdout:nearfull_ratio 0.85 2024-09-14T22:36:13.776 INFO:teuthology.orchestra.run.smithi001.stdout:require_min_compat_client luminous 2024-09-14T22:36:13.776 INFO:teuthology.orchestra.run.smithi001.stdout:min_compat_client jewel 2024-09-14T22:36:13.776 INFO:teuthology.orchestra.run.smithi001.stdout:require_osd_release squid 2024-09-14T22:36:13.776 INFO:teuthology.orchestra.run.smithi001.stdout:stretch_mode_enabled false 2024-09-14T22:36:13.776 INFO:teuthology.orchestra.run.smithi001.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 18 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-14T22:36:13.776 INFO:teuthology.orchestra.run.smithi001.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 22 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.99 2024-09-14T22:36:13.776 INFO:teuthology.orchestra.run.smithi001.stdout: removed_snaps_queue [2~1] 2024-09-14T22:36:13.776 INFO:teuthology.orchestra.run.smithi001.stdout:max_osd 8 2024-09-14T22:36:13.776 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6824/1702929600,v1:172.21.15.1:6825/1702929600] [v2:172.21.15.1:6826/1702929600,v1:172.21.15.1:6827/1702929600] exists,up 9831fcc5-9040-4a80-99d4-838225e65139 2024-09-14T22:36:13.777 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1 up in weight 1 up_from 13 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6808/719990653,v1:172.21.15.1:6809/719990653] [v2:172.21.15.1:6810/719990653,v1:172.21.15.1:6811/719990653] exists,up 7f79abe8-115d-46b2-8677-72fcb0c8ab7d 2024-09-14T22:36:13.777 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6816/2536940607,v1:172.21.15.1:6817/2536940607] [v2:172.21.15.1:6818/2536940607,v1:172.21.15.1:6819/2536940607] exists,up 1f5768b2-4db4-4b2e-8d5e-5206f8bde28d 2024-09-14T22:36:13.777 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6800/750403671,v1:172.21.15.1:6801/750403671] [v2:172.21.15.1:6802/750403671,v1:172.21.15.1:6803/750403671] exists,up 8103f3c7-7899-4fc3-bb02-dd2273120138 2024-09-14T22:36:13.777 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.176:6816/1408471829,v1:172.21.15.176:6817/1408471829] [v2:172.21.15.176:6818/1408471829,v1:172.21.15.176:6819/1408471829] exists,up 23cce5cb-726e-44c2-8edb-c2567a89dfbc 2024-09-14T22:36:13.777 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.176:6805/1945902326,v1:172.21.15.176:6808/1945902326] [v2:172.21.15.176:6810/1945902326,v1:172.21.15.176:6811/1945902326] exists,up eb2f187b-d2f1-49ab-ad5e-ea6dc441a28e 2024-09-14T22:36:13.777 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.176:6800/4116085586,v1:172.21.15.176:6801/4116085586] [v2:172.21.15.176:6802/4116085586,v1:172.21.15.176:6803/4116085586] exists,up d371aed8-2ad2-49a5-8ece-4b7fd37b4822 2024-09-14T22:36:13.777 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.176:6824/877760244,v1:172.21.15.176:6825/877760244] [v2:172.21.15.176:6826/877760244,v1:172.21.15.176:6827/877760244] exists,up a012c819-8d49-4310-a840-c2752d9314ab 2024-09-14T22:36:13.794 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6824/1702929600,v1:172.21.15.1:6825/1702929600] [v2:172.21.15.1:6826/1702929600,v1:172.21.15.1:6827/1702929600] exists,up 9831fcc5-9040-4a80-99d4-838225e65139', 'osd.1 up in weight 1 up_from 13 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6808/719990653,v1:172.21.15.1:6809/719990653] [v2:172.21.15.1:6810/719990653,v1:172.21.15.1:6811/719990653] exists,up 7f79abe8-115d-46b2-8677-72fcb0c8ab7d', 'osd.2 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6816/2536940607,v1:172.21.15.1:6817/2536940607] [v2:172.21.15.1:6818/2536940607,v1:172.21.15.1:6819/2536940607] exists,up 1f5768b2-4db4-4b2e-8d5e-5206f8bde28d', 'osd.3 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6800/750403671,v1:172.21.15.1:6801/750403671] [v2:172.21.15.1:6802/750403671,v1:172.21.15.1:6803/750403671] exists,up 8103f3c7-7899-4fc3-bb02-dd2273120138', 'osd.4 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.176:6816/1408471829,v1:172.21.15.176:6817/1408471829] [v2:172.21.15.176:6818/1408471829,v1:172.21.15.176:6819/1408471829] exists,up 23cce5cb-726e-44c2-8edb-c2567a89dfbc', 'osd.5 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.176:6805/1945902326,v1:172.21.15.176:6808/1945902326] [v2:172.21.15.176:6810/1945902326,v1:172.21.15.176:6811/1945902326] exists,up eb2f187b-d2f1-49ab-ad5e-ea6dc441a28e', 'osd.6 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.176:6800/4116085586,v1:172.21.15.176:6801/4116085586] [v2:172.21.15.176:6802/4116085586,v1:172.21.15.176:6803/4116085586] exists,up d371aed8-2ad2-49a5-8ece-4b7fd37b4822', 'osd.7 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.176:6824/877760244,v1:172.21.15.176:6825/877760244] [v2:172.21.15.176:6826/877760244,v1:172.21.15.176:6827/877760244] exists,up a012c819-8d49-4310-a840-c2752d9314ab'] 2024-09-14T22:36:13.794 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-14T22:36:13.903 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:13.903 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mon.a", 2024-09-14T22:36:13.903 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster": "ceph", 2024-09-14T22:36:13.903 INFO:teuthology.orchestra.run.smithi001.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-14T22:36:13.903 INFO:teuthology.orchestra.run.smithi001.stdout: "admin_socket_mode": "", 2024-09-14T22:36:13.903 INFO:teuthology.orchestra.run.smithi001.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-14T22:36:13.903 INFO:teuthology.orchestra.run.smithi001.stdout: "auth_client_required": "cephx, none", 2024-09-14T22:36:13.903 INFO:teuthology.orchestra.run.smithi001.stdout: "auth_cluster_required": "cephx", 2024-09-14T22:36:13.904 INFO:teuthology.orchestra.run.smithi001.stdout: "auth_debug": "true", 2024-09-14T22:36:13.904 INFO:teuthology.orchestra.run.smithi001.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-14T22:36:13.904 INFO:teuthology.orchestra.run.smithi001.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-14T22:36:13.904 INFO:teuthology.orchestra.run.smithi001.stdout: "auth_service_required": "cephx", 2024-09-14T22:36:13.904 INFO:teuthology.orchestra.run.smithi001.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-14T22:36:13.904 INFO:teuthology.orchestra.run.smithi001.stdout: "auth_supported": "cephx", 2024-09-14T22:36:13.904 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_aio": "true", 2024-09-14T22:36:13.904 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-14T22:36:13.904 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_aio_poll_ms": "250", 2024-09-14T22:36:13.904 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_aio_reap_max": "16", 2024-09-14T22:36:13.904 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_async_discard": "false", 2024-09-14T22:36:13.904 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_block_size": "4096", 2024-09-14T22:36:13.904 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_debug_aio": "false", 2024-09-14T22:36:13.905 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-14T22:36:13.905 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-14T22:36:13.905 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_debug_inflight_ios": "false", 2024-09-14T22:36:13.905 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_enable_discard": "false", 2024-09-14T22:36:13.905 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_flock_retry": "3", 2024-09-14T22:36:13.905 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-14T22:36:13.905 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_inject_crash": "0", 2024-09-14T22:36:13.905 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-14T22:36:13.905 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_ioring": "false", 2024-09-14T22:36:13.905 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_ioring_hipri": "false", 2024-09-14T22:36:13.905 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-14T22:36:13.905 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-14T22:36:13.905 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-14T22:36:13.906 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-14T22:36:13.906 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-14T22:36:13.906 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-14T22:36:13.906 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_type": "", 2024-09-14T22:36:13.906 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_alloc_size": "1048576", 2024-09-14T22:36:13.906 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_allocator": "hybrid", 2024-09-14T22:36:13.906 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_buffered_io": "true", 2024-09-14T22:36:13.906 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_check_for_zeros": "false", 2024-09-14T22:36:13.906 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-14T22:36:13.906 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-14T22:36:13.906 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_compact_log_sync": "false", 2024-09-14T22:36:13.906 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-14T22:36:13.906 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-14T22:36:13.907 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-14T22:36:13.907 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-14T22:36:13.907 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_max_log_runway": "4194304", 2024-09-14T22:36:13.907 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_max_prefetch": "1048576", 2024-09-14T22:36:13.907 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_min_flush_size": "524288", 2024-09-14T22:36:13.907 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_min_log_runway": "1048576", 2024-09-14T22:36:13.907 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_replay_recovery": "false", 2024-09-14T22:36:13.907 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-14T22:36:13.907 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-14T22:36:13.907 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_sync_write": "false", 2024-09-14T22:36:13.907 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-14T22:36:13.907 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-14T22:36:13.908 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-14T22:36:13.908 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_allocation_from_file": "true", 2024-09-14T22:36:13.908 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_allocator": "hybrid", 2024-09-14T22:36:13.908 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-14T22:36:13.908 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-14T22:36:13.908 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-14T22:36:13.908 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-14T22:36:13.908 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_bdev_label_multi": "true", 2024-09-14T22:36:13.908 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-14T22:36:13.908 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-14T22:36:13.908 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-14T22:36:13.908 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-14T22:36:13.908 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-14T22:36:13.908 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_block_create": "true", 2024-09-14T22:36:13.908 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_block_db_create": "false", 2024-09-14T22:36:13.909 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_block_db_path": "", 2024-09-14T22:36:13.909 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_block_db_size": "0", 2024-09-14T22:36:13.909 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_block_path": "", 2024-09-14T22:36:13.909 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_block_preallocate_file": "false", 2024-09-14T22:36:13.909 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_block_size": "107374182400", 2024-09-14T22:36:13.909 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_block_wal_create": "false", 2024-09-14T22:36:13.909 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_block_wal_path": "", 2024-09-14T22:36:13.909 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_block_wal_size": "100663296", 2024-09-14T22:36:13.909 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_bluefs": "true", 2024-09-14T22:36:13.909 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-14T22:36:13.909 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-14T22:36:13.909 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-14T22:36:13.909 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-14T22:36:13.909 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-14T22:36:13.910 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-14T22:36:13.910 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-14T22:36:13.910 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-14T22:36:13.910 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_autotune": "true", 2024-09-14T22:36:13.910 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-14T22:36:13.910 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-14T22:36:13.910 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-14T22:36:13.910 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-14T22:36:13.910 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_size": "0", 2024-09-14T22:36:13.910 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-14T22:36:13.910 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-14T22:36:13.910 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-14T22:36:13.910 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-14T22:36:13.910 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_type": "2q", 2024-09-14T22:36:13.910 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-14T22:36:13.911 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_clone_cow": "true", 2024-09-14T22:36:13.911 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-14T22:36:13.911 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-14T22:36:13.911 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-14T22:36:13.911 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-14T22:36:13.911 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-14T22:36:13.911 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-14T22:36:13.911 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-14T22:36:13.911 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_compression_mode": "none", 2024-09-14T22:36:13.911 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-14T22:36:13.911 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_csum_type": "crc32c", 2024-09-14T22:36:13.911 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-14T22:36:13.911 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_freelist": "false", 2024-09-14T22:36:13.911 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-14T22:36:13.911 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-14T22:36:13.912 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-14T22:36:13.912 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-14T22:36:13.912 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-14T22:36:13.912 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-14T22:36:13.912 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-14T22:36:13.912 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-14T22:36:13.912 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-14T22:36:13.912 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-14T22:36:13.912 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-14T22:36:13.912 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-14T22:36:13.912 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_small_allocations": "0", 2024-09-14T22:36:13.912 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-14T22:36:13.912 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_default_buffered_read": "true", 2024-09-14T22:36:13.912 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_default_buffered_write": "false", 2024-09-14T22:36:13.913 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-14T22:36:13.913 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-14T22:36:13.913 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-14T22:36:13.913 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-14T22:36:13.913 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-14T22:36:13.913 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-14T22:36:13.913 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-14T22:36:13.913 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-14T22:36:13.913 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-14T22:36:13.913 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fail_eio": "false", 2024-09-14T22:36:13.914 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-14T22:36:13.914 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-14T22:36:13.914 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-14T22:36:13.914 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-14T22:36:13.914 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-14T22:36:13.914 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-14T22:36:13.914 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_on_mount": "false", 2024-09-14T22:36:13.914 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-14T22:36:13.914 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_on_umount": "false", 2024-09-14T22:36:13.914 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-14T22:36:13.914 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-14T22:36:13.914 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-14T22:36:13.914 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-14T22:36:13.914 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-14T22:36:13.914 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-14T22:36:13.914 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-14T22:36:13.915 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-14T22:36:13.915 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_ignore_data_csum": "false", 2024-09-14T22:36:13.915 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-14T22:36:13.915 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-14T22:36:13.915 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-14T22:36:13.915 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-14T22:36:13.915 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_log_op_age": "5.000000", 2024-09-14T22:36:13.915 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_max_alloc_size": "0", 2024-09-14T22:36:13.915 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_max_blob_size": "0", 2024-09-14T22:36:13.915 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-14T22:36:13.915 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-14T22:36:13.915 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-14T22:36:13.915 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_max_deferred_txc": "32", 2024-09-14T22:36:13.916 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_min_alloc_size": "0", 2024-09-14T22:36:13.916 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-14T22:36:13.916 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-14T22:36:13.916 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_nid_prealloc": "1024", 2024-09-14T22:36:13.916 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-14T22:36:13.916 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-14T22:36:13.916 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-14T22:36:13.916 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-14T22:36:13.916 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_retry_disk_reads": "3", 2024-09-14T22:36:13.916 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_rocksdb_cf": "true", 2024-09-14T22:36:13.916 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-14T22:36:13.916 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-14T22:36:13.916 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-14T22:36:13.916 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-14T22:36:13.916 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-14T22:36:13.917 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-14T22:36:13.917 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-14T22:36:13.917 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-14T22:36:13.917 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_spdk_mem": "512", 2024-09-14T22:36:13.917 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-14T22:36:13.917 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-14T22:36:13.917 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-14T22:36:13.917 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-14T22:36:13.917 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-14T22:36:13.917 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-14T22:36:13.917 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-14T22:36:13.917 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_tracing": "false", 2024-09-14T22:36:13.918 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-14T22:36:13.918 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-14T22:36:13.918 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-14T22:36:13.918 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-14T22:36:13.918 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-14T22:36:13.918 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-14T22:36:13.918 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-14T22:36:13.918 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-14T22:36:13.918 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-14T22:36:13.918 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_zero_block_detection": "false", 2024-09-14T22:36:13.918 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-14T22:36:13.918 INFO:teuthology.orchestra.run.smithi001.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-14T22:36:13.918 INFO:teuthology.orchestra.run.smithi001.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-14T22:36:13.918 INFO:teuthology.orchestra.run.smithi001.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-14T22:36:13.918 INFO:teuthology.orchestra.run.smithi001.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-14T22:36:13.919 INFO:teuthology.orchestra.run.smithi001.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-14T22:36:13.919 INFO:teuthology.orchestra.run.smithi001.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-14T22:36:13.919 INFO:teuthology.orchestra.run.smithi001.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-14T22:36:13.919 INFO:teuthology.orchestra.run.smithi001.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-14T22:36:13.919 INFO:teuthology.orchestra.run.smithi001.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-14T22:36:13.919 INFO:teuthology.orchestra.run.smithi001.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-14T22:36:13.919 INFO:teuthology.orchestra.run.smithi001.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-14T22:36:13.919 INFO:teuthology.orchestra.run.smithi001.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-14T22:36:13.919 INFO:teuthology.orchestra.run.smithi001.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-14T22:36:13.919 INFO:teuthology.orchestra.run.smithi001.stdout: "cephx_cluster_require_signatures": "false", 2024-09-14T22:36:13.919 INFO:teuthology.orchestra.run.smithi001.stdout: "cephx_cluster_require_version": "2", 2024-09-14T22:36:13.919 INFO:teuthology.orchestra.run.smithi001.stdout: "cephx_require_signatures": "false", 2024-09-14T22:36:13.919 INFO:teuthology.orchestra.run.smithi001.stdout: "cephx_require_version": "2", 2024-09-14T22:36:13.920 INFO:teuthology.orchestra.run.smithi001.stdout: "cephx_service_require_signatures": "false", 2024-09-14T22:36:13.920 INFO:teuthology.orchestra.run.smithi001.stdout: "cephx_service_require_version": "2", 2024-09-14T22:36:13.920 INFO:teuthology.orchestra.run.smithi001.stdout: "cephx_sign_messages": "true", 2024-09-14T22:36:13.920 INFO:teuthology.orchestra.run.smithi001.stdout: "chdir": "", 2024-09-14T22:36:13.920 INFO:teuthology.orchestra.run.smithi001.stdout: "client_acl_type": "", 2024-09-14T22:36:13.920 INFO:teuthology.orchestra.run.smithi001.stdout: "client_asio_thread_count": "2", 2024-09-14T22:36:13.920 INFO:teuthology.orchestra.run.smithi001.stdout: "client_cache_mid": "0.750000", 2024-09-14T22:36:13.920 INFO:teuthology.orchestra.run.smithi001.stdout: "client_cache_size": "16384", 2024-09-14T22:36:13.920 INFO:teuthology.orchestra.run.smithi001.stdout: "client_caps_release_delay": "5", 2024-09-14T22:36:13.920 INFO:teuthology.orchestra.run.smithi001.stdout: "client_check_pool_perm": "true", 2024-09-14T22:36:13.920 INFO:teuthology.orchestra.run.smithi001.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-14T22:36:13.920 INFO:teuthology.orchestra.run.smithi001.stdout: "client_debug_force_sync_read": "false", 2024-09-14T22:36:13.920 INFO:teuthology.orchestra.run.smithi001.stdout: "client_debug_getattr_caps": "false", 2024-09-14T22:36:13.920 INFO:teuthology.orchestra.run.smithi001.stdout: "client_debug_inject_features": "", 2024-09-14T22:36:13.920 INFO:teuthology.orchestra.run.smithi001.stdout: "client_debug_inject_tick_delay": "0", 2024-09-14T22:36:13.921 INFO:teuthology.orchestra.run.smithi001.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-14T22:36:13.921 INFO:teuthology.orchestra.run.smithi001.stdout: "client_die_on_failed_remount": "false", 2024-09-14T22:36:13.921 INFO:teuthology.orchestra.run.smithi001.stdout: "client_dirsize_rbytes": "true", 2024-09-14T22:36:13.921 INFO:teuthology.orchestra.run.smithi001.stdout: "client_force_lazyio": "false", 2024-09-14T22:36:13.921 INFO:teuthology.orchestra.run.smithi001.stdout: "client_fs": "", 2024-09-14T22:36:13.921 INFO:teuthology.orchestra.run.smithi001.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-14T22:36:13.921 INFO:teuthology.orchestra.run.smithi001.stdout: "client_inject_release_failure": "false", 2024-09-14T22:36:13.921 INFO:teuthology.orchestra.run.smithi001.stdout: "client_max_inline_size": "4096", 2024-09-14T22:36:13.921 INFO:teuthology.orchestra.run.smithi001.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-14T22:36:13.921 INFO:teuthology.orchestra.run.smithi001.stdout: "client_mds_namespace": "", 2024-09-14T22:36:13.921 INFO:teuthology.orchestra.run.smithi001.stdout: "client_metadata": "", 2024-09-14T22:36:13.921 INFO:teuthology.orchestra.run.smithi001.stdout: "client_mount_gid": "-1", 2024-09-14T22:36:13.922 INFO:teuthology.orchestra.run.smithi001.stdout: "client_mount_timeout": "300", 2024-09-14T22:36:13.922 INFO:teuthology.orchestra.run.smithi001.stdout: "client_mount_uid": "-1", 2024-09-14T22:36:13.922 INFO:teuthology.orchestra.run.smithi001.stdout: "client_mountpoint": "/", 2024-09-14T22:36:13.922 INFO:teuthology.orchestra.run.smithi001.stdout: "client_notify_timeout": "10", 2024-09-14T22:36:13.922 INFO:teuthology.orchestra.run.smithi001.stdout: "client_oc": "true", 2024-09-14T22:36:13.922 INFO:teuthology.orchestra.run.smithi001.stdout: "client_oc_max_dirty": "104857600", 2024-09-14T22:36:13.922 INFO:teuthology.orchestra.run.smithi001.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-14T22:36:13.922 INFO:teuthology.orchestra.run.smithi001.stdout: "client_oc_max_objects": "1000", 2024-09-14T22:36:13.922 INFO:teuthology.orchestra.run.smithi001.stdout: "client_oc_size": "209715200", 2024-09-14T22:36:13.922 INFO:teuthology.orchestra.run.smithi001.stdout: "client_oc_target_dirty": "8388608", 2024-09-14T22:36:13.922 INFO:teuthology.orchestra.run.smithi001.stdout: "client_permissions": "true", 2024-09-14T22:36:13.922 INFO:teuthology.orchestra.run.smithi001.stdout: "client_quota": "true", 2024-09-14T22:36:13.922 INFO:teuthology.orchestra.run.smithi001.stdout: "client_quota_df": "true", 2024-09-14T22:36:13.922 INFO:teuthology.orchestra.run.smithi001.stdout: "client_readahead_max_bytes": "0", 2024-09-14T22:36:13.922 INFO:teuthology.orchestra.run.smithi001.stdout: "client_readahead_max_periods": "4", 2024-09-14T22:36:13.923 INFO:teuthology.orchestra.run.smithi001.stdout: "client_readahead_min": "131072", 2024-09-14T22:36:13.923 INFO:teuthology.orchestra.run.smithi001.stdout: "client_reconnect_stale": "false", 2024-09-14T22:36:13.923 INFO:teuthology.orchestra.run.smithi001.stdout: "client_shutdown_timeout": "30", 2024-09-14T22:36:13.923 INFO:teuthology.orchestra.run.smithi001.stdout: "client_snapdir": ".snap", 2024-09-14T22:36:13.923 INFO:teuthology.orchestra.run.smithi001.stdout: "client_tick_interval": "1", 2024-09-14T22:36:13.923 INFO:teuthology.orchestra.run.smithi001.stdout: "client_trace": "", 2024-09-14T22:36:13.923 INFO:teuthology.orchestra.run.smithi001.stdout: "client_try_dentry_invalidate": "false", 2024-09-14T22:36:13.923 INFO:teuthology.orchestra.run.smithi001.stdout: "client_use_faked_inos": "false", 2024-09-14T22:36:13.923 INFO:teuthology.orchestra.run.smithi001.stdout: "client_use_random_mds": "false", 2024-09-14T22:36:13.923 INFO:teuthology.orchestra.run.smithi001.stdout: "clog_to_graylog": "false", 2024-09-14T22:36:13.923 INFO:teuthology.orchestra.run.smithi001.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-14T22:36:13.923 INFO:teuthology.orchestra.run.smithi001.stdout: "clog_to_graylog_port": "12201", 2024-09-14T22:36:13.923 INFO:teuthology.orchestra.run.smithi001.stdout: "clog_to_monitors": "default=true", 2024-09-14T22:36:13.923 INFO:teuthology.orchestra.run.smithi001.stdout: "clog_to_syslog": "false", 2024-09-14T22:36:13.923 INFO:teuthology.orchestra.run.smithi001.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-14T22:36:13.924 INFO:teuthology.orchestra.run.smithi001.stdout: "clog_to_syslog_level": "info", 2024-09-14T22:36:13.924 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "-", 2024-09-14T22:36:13.924 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_network": "", 2024-09-14T22:36:13.924 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_network_interface": "", 2024-09-14T22:36:13.924 INFO:teuthology.orchestra.run.smithi001.stdout: "compressor_zlib_isal": "false", 2024-09-14T22:36:13.924 INFO:teuthology.orchestra.run.smithi001.stdout: "compressor_zlib_level": "5", 2024-09-14T22:36:13.924 INFO:teuthology.orchestra.run.smithi001.stdout: "compressor_zlib_winsize": "-15", 2024-09-14T22:36:13.924 INFO:teuthology.orchestra.run.smithi001.stdout: "compressor_zstd_level": "1", 2024-09-14T22:36:13.924 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-14T22:36:13.924 INFO:teuthology.orchestra.run.smithi001.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-14T22:36:13.924 INFO:teuthology.orchestra.run.smithi001.stdout: "crimson_alien_op_num_threads": "6", 2024-09-14T22:36:13.924 INFO:teuthology.orchestra.run.smithi001.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-14T22:36:13.924 INFO:teuthology.orchestra.run.smithi001.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-14T22:36:13.925 INFO:teuthology.orchestra.run.smithi001.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-14T22:36:13.925 INFO:teuthology.orchestra.run.smithi001.stdout: "crimson_osd_stat_interval": "0", 2024-09-14T22:36:13.925 INFO:teuthology.orchestra.run.smithi001.stdout: "crimson_seastar_cpu_cores": "", 2024-09-14T22:36:13.925 INFO:teuthology.orchestra.run.smithi001.stdout: "crimson_seastar_num_threads": "0", 2024-09-14T22:36:13.925 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "", 2024-09-14T22:36:13.925 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location_hook": "", 2024-09-14T22:36:13.925 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location_hook_timeout": "10", 2024-09-14T22:36:13.925 INFO:teuthology.orchestra.run.smithi001.stdout: "daemonize": "false", 2024-09-14T22:36:13.925 INFO:teuthology.orchestra.run.smithi001.stdout: "daos_pool": "tank", 2024-09-14T22:36:13.925 INFO:teuthology.orchestra.run.smithi001.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-14T22:36:13.925 INFO:teuthology.orchestra.run.smithi001.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-14T22:36:13.925 INFO:teuthology.orchestra.run.smithi001.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-14T22:36:13.925 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_alienstore": "0/5", 2024-09-14T22:36:13.925 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_allow_any_pool_priority": "false", 2024-09-14T22:36:13.926 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_asok": "1/5", 2024-09-14T22:36:13.926 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_asok_assert_abort": "false", 2024-09-14T22:36:13.926 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_asserts_on_shutdown": "true", 2024-09-14T22:36:13.926 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_auth": "20/20", 2024-09-14T22:36:13.926 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_bdev": "1/3", 2024-09-14T22:36:13.926 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_bluefs": "1/5", 2024-09-14T22:36:13.926 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_bluestore": "1/5", 2024-09-14T22:36:13.926 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_buffer": "0/1", 2024-09-14T22:36:13.926 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_ceph_exporter": "1/5", 2024-09-14T22:36:13.926 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_cephfs_mirror": "0/5", 2024-09-14T22:36:13.926 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_cephsqlite": "0/5", 2024-09-14T22:36:13.926 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_client": "0/5", 2024-09-14T22:36:13.926 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_compressor": "1/5", 2024-09-14T22:36:13.927 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_context": "0/1", 2024-09-14T22:36:13.927 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_crimson_interrupt": "0/5", 2024-09-14T22:36:13.927 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_crush": "1/1", 2024-09-14T22:36:13.927 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_crypto": "1/5", 2024-09-14T22:36:13.927 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_cyanstore": "0/5", 2024-09-14T22:36:13.927 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_deliberately_leak_memory": "false", 2024-09-14T22:36:13.927 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_disable_randomized_ping": "false", 2024-09-14T22:36:13.927 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_dpdk": "1/5", 2024-09-14T22:36:13.927 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_eventtrace": "1/5", 2024-09-14T22:36:13.927 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_filer": "0/1", 2024-09-14T22:36:13.927 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_filestore": "1/3", 2024-09-14T22:36:13.927 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_finisher": "1/1", 2024-09-14T22:36:13.927 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_fuse": "1/5", 2024-09-14T22:36:13.927 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_heartbeat_testing_span": "0", 2024-09-14T22:36:13.927 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_heartbeatmap": "1/5", 2024-09-14T22:36:13.928 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-14T22:36:13.928 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_javaclient": "1/5", 2024-09-14T22:36:13.928 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_journal": "1/3", 2024-09-14T22:36:13.928 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_journaler": "0/5", 2024-09-14T22:36:13.928 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_kstore": "1/5", 2024-09-14T22:36:13.928 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_lockdep": "0/1", 2024-09-14T22:36:13.928 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_mclock": "1/5", 2024-09-14T22:36:13.928 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_mds": "1/5", 2024-09-14T22:36:13.928 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_mds_balancer": "1/5", 2024-09-14T22:36:13.928 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_mds_locker": "1/5", 2024-09-14T22:36:13.928 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_mds_log": "1/5", 2024-09-14T22:36:13.928 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_mds_log_expire": "1/5", 2024-09-14T22:36:13.928 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_mds_migrator": "1/5", 2024-09-14T22:36:13.928 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_mds_quiesce": "3/5", 2024-09-14T22:36:13.929 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_memstore": "1/5", 2024-09-14T22:36:13.929 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_mgr": "2/5", 2024-09-14T22:36:13.929 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_mgrc": "1/5", 2024-09-14T22:36:13.929 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_mon": "20/20", 2024-09-14T22:36:13.929 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_monc": "0/10", 2024-09-14T22:36:13.929 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_ms": "1/1", 2024-09-14T22:36:13.929 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_none": "0/5", 2024-09-14T22:36:13.929 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_objclass": "0/5", 2024-09-14T22:36:13.929 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_objectcacher": "0/5", 2024-09-14T22:36:13.929 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_objecter": "0/1", 2024-09-14T22:36:13.929 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_optracker": "0/5", 2024-09-14T22:36:13.929 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_osd": "1/5", 2024-09-14T22:36:13.929 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_paxos": "20/20", 2024-09-14T22:36:13.930 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_perfcounter": "1/5", 2024-09-14T22:36:13.930 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_prioritycache": "1/5", 2024-09-14T22:36:13.930 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_rados": "0/5", 2024-09-14T22:36:13.930 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_rbd": "0/5", 2024-09-14T22:36:13.930 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_rbd_mirror": "0/5", 2024-09-14T22:36:13.930 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_rbd_pwl": "0/5", 2024-09-14T22:36:13.930 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_rbd_replay": "0/5", 2024-09-14T22:36:13.930 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_refs": "0/0", 2024-09-14T22:36:13.930 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_reserver": "1/1", 2024-09-14T22:36:13.930 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_rgw": "1/5", 2024-09-14T22:36:13.930 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_rgw_access": "1/5", 2024-09-14T22:36:13.930 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_rgw_datacache": "1/5", 2024-09-14T22:36:13.930 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_rgw_dbstore": "1/5", 2024-09-14T22:36:13.930 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_rgw_flight": "1/5", 2024-09-14T22:36:13.931 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-14T22:36:13.931 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_rgw_sync": "1/5", 2024-09-14T22:36:13.931 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_rocksdb": "4/5", 2024-09-14T22:36:13.931 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_seastore": "0/5", 2024-09-14T22:36:13.931 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_seastore_backref": "0/5", 2024-09-14T22:36:13.931 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_seastore_cache": "0/5", 2024-09-14T22:36:13.931 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_seastore_cleaner": "0/5", 2024-09-14T22:36:13.931 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_seastore_device": "0/5", 2024-09-14T22:36:13.931 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_seastore_epm": "0/5", 2024-09-14T22:36:13.931 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-14T22:36:13.931 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_seastore_journal": "0/5", 2024-09-14T22:36:13.931 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_seastore_lba": "0/5", 2024-09-14T22:36:13.931 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_seastore_odata": "0/5", 2024-09-14T22:36:13.931 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_seastore_omap": "0/5", 2024-09-14T22:36:13.932 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_seastore_onode": "0/5", 2024-09-14T22:36:13.932 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_seastore_t": "0/5", 2024-09-14T22:36:13.932 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_seastore_tm": "0/5", 2024-09-14T22:36:13.932 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_striper": "0/1", 2024-09-14T22:36:13.932 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_test": "0/5", 2024-09-14T22:36:13.932 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_throttle": "1/1", 2024-09-14T22:36:13.932 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_timer": "0/1", 2024-09-14T22:36:13.932 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_tp": "0/5", 2024-09-14T22:36:13.932 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_trace": "1/5", 2024-09-14T22:36:13.932 INFO:teuthology.orchestra.run.smithi001.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-14T22:36:13.932 INFO:teuthology.orchestra.run.smithi001.stdout: "device_failure_prediction_mode": "none", 2024-09-14T22:36:13.932 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-14T22:36:13.933 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-09-14T22:36:13.933 INFO:teuthology.orchestra.run.smithi001.stdout: "err_to_graylog": "false", 2024-09-14T22:36:13.933 INFO:teuthology.orchestra.run.smithi001.stdout: "err_to_journald": "false", 2024-09-14T22:36:13.933 INFO:teuthology.orchestra.run.smithi001.stdout: "err_to_stderr": "true", 2024-09-14T22:36:13.933 INFO:teuthology.orchestra.run.smithi001.stdout: "err_to_syslog": "false", 2024-09-14T22:36:13.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event_tracing": "false", 2024-09-14T22:36:13.933 INFO:teuthology.orchestra.run.smithi001.stdout: "exporter_addr": "0.0.0.0", 2024-09-14T22:36:13.933 INFO:teuthology.orchestra.run.smithi001.stdout: "exporter_http_port": "9926", 2024-09-14T22:36:13.933 INFO:teuthology.orchestra.run.smithi001.stdout: "exporter_prio_limit": "5", 2024-09-14T22:36:13.933 INFO:teuthology.orchestra.run.smithi001.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-14T22:36:13.933 INFO:teuthology.orchestra.run.smithi001.stdout: "exporter_sort_metrics": "true", 2024-09-14T22:36:13.933 INFO:teuthology.orchestra.run.smithi001.stdout: "exporter_stats_period": "5", 2024-09-14T22:36:13.933 INFO:teuthology.orchestra.run.smithi001.stdout: "fake_statfs_for_testing": "0", 2024-09-14T22:36:13.934 INFO:teuthology.orchestra.run.smithi001.stdout: "fatal_signal_handlers": "true", 2024-09-14T22:36:13.934 INFO:teuthology.orchestra.run.smithi001.stdout: "filer_max_purge_ops": "10", 2024-09-14T22:36:13.934 INFO:teuthology.orchestra.run.smithi001.stdout: "filer_max_truncate_ops": "128", 2024-09-14T22:36:13.934 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_apply_finisher_threads": "1", 2024-09-14T22:36:13.934 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_blackhole": "false", 2024-09-14T22:36:13.934 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_btrfs_clone_range": "true", 2024-09-14T22:36:13.934 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_btrfs_snap": "true", 2024-09-14T22:36:13.934 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_caller_concurrency": "10", 2024-09-14T22:36:13.934 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_collect_device_partition_information": "true", 2024-09-14T22:36:13.934 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_commit_timeout": "600.000000", 2024-09-14T22:36:13.934 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_debug_inject_read_err": "false", 2024-09-14T22:36:13.934 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_debug_omap_check": "false", 2024-09-14T22:36:13.934 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-14T22:36:13.935 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_debug_verify_split": "false", 2024-09-14T22:36:13.935 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_dump_file": "", 2024-09-14T22:36:13.935 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-14T22:36:13.935 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-14T22:36:13.935 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_fadvise": "true", 2024-09-14T22:36:13.935 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_fail_eio": "true", 2024-09-14T22:36:13.935 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_fd_cache_shards": "16", 2024-09-14T22:36:13.935 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_fd_cache_size": "128", 2024-09-14T22:36:13.935 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_fiemap": "false", 2024-09-14T22:36:13.935 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_fiemap_threshold": "4096", 2024-09-14T22:36:13.935 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-14T22:36:13.935 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-14T22:36:13.935 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_inject_stall": "0", 2024-09-14T22:36:13.935 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_journal_parallel": "false", 2024-09-14T22:36:13.936 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_journal_trailing": "false", 2024-09-14T22:36:13.936 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_journal_writeahead": "false", 2024-09-14T22:36:13.936 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_kill_at": "0", 2024-09-14T22:36:13.936 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-14T22:36:13.936 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-14T22:36:13.936 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-14T22:36:13.936 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-14T22:36:13.936 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-14T22:36:13.936 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_inline_xattrs": "0", 2024-09-14T22:36:13.936 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-14T22:36:13.936 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-14T22:36:13.936 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-14T22:36:13.936 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-14T22:36:13.936 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_xattr_value_size": "0", 2024-09-14T22:36:13.936 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-14T22:36:13.937 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-14T22:36:13.937 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-14T22:36:13.937 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_merge_threshold": "-10", 2024-09-14T22:36:13.937 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-14T22:36:13.937 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_odsync_write": "false", 2024-09-14T22:36:13.937 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_omap_backend": "rocksdb", 2024-09-14T22:36:13.937 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_omap_backend_path": "", 2024-09-14T22:36:13.937 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-14T22:36:13.937 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-14T22:36:13.937 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-14T22:36:13.937 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_op_thread_timeout": "60", 2024-09-14T22:36:13.937 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_op_threads": "2", 2024-09-14T22:36:13.937 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_punch_hole": "false", 2024-09-14T22:36:13.937 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-14T22:36:13.937 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-14T22:36:13.938 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-14T22:36:13.938 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-14T22:36:13.938 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-14T22:36:13.938 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-14T22:36:13.938 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-14T22:36:13.938 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-14T22:36:13.938 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-14T22:36:13.938 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_queue_max_ops": "50", 2024-09-14T22:36:13.938 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-14T22:36:13.938 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_seek_data_hole": "false", 2024-09-14T22:36:13.938 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_sloppy_crc": "false", 2024-09-14T22:36:13.938 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-14T22:36:13.938 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_splice": "false", 2024-09-14T22:36:13.939 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_split_multiple": "2", 2024-09-14T22:36:13.939 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_split_rand_factor": "20", 2024-09-14T22:36:13.939 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_update_to": "1000", 2024-09-14T22:36:13.939 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-14T22:36:13.939 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-14T22:36:13.939 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-14T22:36:13.939 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-14T22:36:13.939 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-14T22:36:13.939 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-14T22:36:13.939 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_enable": "true", 2024-09-14T22:36:13.939 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-14T22:36:13.939 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-14T22:36:13.939 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-14T22:36:13.940 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-14T22:36:13.940 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-14T22:36:13.940 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-14T22:36:13.940 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_xfs_extsize": "false", 2024-09-14T22:36:13.940 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_zfs_snap": "false", 2024-09-14T22:36:13.940 INFO:teuthology.orchestra.run.smithi001.stdout: "fio_dir": "/tmp/fio", 2024-09-14T22:36:13.940 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "f07f407a-b17d-42c4-bf73-92dd80114a4f", 2024-09-14T22:36:13.940 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_allow_other": "true", 2024-09-14T22:36:13.940 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_atomic_o_trunc": "true", 2024-09-14T22:36:13.940 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_big_writes": "true", 2024-09-14T22:36:13.940 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_debug": "false", 2024-09-14T22:36:13.940 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_default_permissions": "false", 2024-09-14T22:36:13.940 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_disable_pagecache": "false", 2024-09-14T22:36:13.940 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_max_write": "0", 2024-09-14T22:36:13.941 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_multithreaded": "true", 2024-09-14T22:36:13.941 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_require_active_mds": "true", 2024-09-14T22:36:13.941 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_set_user_groups": "true", 2024-09-14T22:36:13.941 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_splice_move": "true", 2024-09-14T22:36:13.941 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_splice_read": "true", 2024-09-14T22:36:13.941 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_splice_write": "true", 2024-09-14T22:36:13.941 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-14T22:36:13.941 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_use_invalidate_cb": "true", 2024-09-14T22:36:13.941 INFO:teuthology.orchestra.run.smithi001.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-14T22:36:13.941 INFO:teuthology.orchestra.run.smithi001.stdout: "gss_target_name": "ceph", 2024-09-14T22:36:13.941 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_file": "", 2024-09-14T22:36:13.941 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_inject_failure": "0", 2024-09-14T22:36:13.941 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_interval": "5", 2024-09-14T22:36:13.941 INFO:teuthology.orchestra.run.smithi001.stdout: "host": "smithi001", 2024-09-14T22:36:13.942 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-14T22:36:13.942 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-14T22:36:13.942 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-14T22:36:13.942 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-14T22:36:13.942 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-14T22:36:13.942 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-14T22:36:13.942 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-14T22:36:13.942 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-14T22:36:13.942 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-14T22:36:13.942 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-14T22:36:13.942 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-14T22:36:13.942 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-14T22:36:13.942 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-14T22:36:13.942 INFO:teuthology.orchestra.run.smithi001.stdout: "inject_early_sigterm": "false", 2024-09-14T22:36:13.942 INFO:teuthology.orchestra.run.smithi001.stdout: "jaeger_agent_port": "6799", 2024-09-14T22:36:13.943 INFO:teuthology.orchestra.run.smithi001.stdout: "jaeger_tracing_enable": "false", 2024-09-14T22:36:13.943 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_aio": "true", 2024-09-14T22:36:13.943 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_align_min_size": "65536", 2024-09-14T22:36:13.943 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_block_align": "true", 2024-09-14T22:36:13.943 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_block_size": "4096", 2024-09-14T22:36:13.943 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_dio": "true", 2024-09-14T22:36:13.943 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_discard": "false", 2024-09-14T22:36:13.943 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_force_aio": "false", 2024-09-14T22:36:13.943 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_ignore_corruption": "false", 2024-09-14T22:36:13.943 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_max_write_bytes": "10485760", 2024-09-14T22:36:13.943 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_max_write_entries": "100", 2024-09-14T22:36:13.943 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_replay_from": "0", 2024-09-14T22:36:13.943 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-14T22:36:13.943 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-14T22:36:13.944 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-14T22:36:13.944 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-14T22:36:13.944 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_write_header_frequency": "0", 2024-09-14T22:36:13.944 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_zero_on_create": "false", 2024-09-14T22:36:13.944 INFO:teuthology.orchestra.run.smithi001.stdout: "journaler_prefetch_periods": "10", 2024-09-14T22:36:13.944 INFO:teuthology.orchestra.run.smithi001.stdout: "journaler_prezero_periods": "5", 2024-09-14T22:36:13.944 INFO:teuthology.orchestra.run.smithi001.stdout: "journaler_write_head_interval": "15", 2024-09-14T22:36:13.944 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "", 2024-09-14T22:36:13.944 INFO:teuthology.orchestra.run.smithi001.stdout: "keyfile": "", 2024-09-14T22:36:13.944 INFO:teuthology.orchestra.run.smithi001.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-14T22:36:13.944 INFO:teuthology.orchestra.run.smithi001.stdout: "kstore_backend": "rocksdb", 2024-09-14T22:36:13.944 INFO:teuthology.orchestra.run.smithi001.stdout: "kstore_default_stripe_size": "65536", 2024-09-14T22:36:13.944 INFO:teuthology.orchestra.run.smithi001.stdout: "kstore_fsck_on_mount": "false", 2024-09-14T22:36:13.944 INFO:teuthology.orchestra.run.smithi001.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-14T22:36:13.945 INFO:teuthology.orchestra.run.smithi001.stdout: "kstore_max_bytes": "67108864", 2024-09-14T22:36:13.945 INFO:teuthology.orchestra.run.smithi001.stdout: "kstore_max_ops": "512", 2024-09-14T22:36:13.945 INFO:teuthology.orchestra.run.smithi001.stdout: "kstore_nid_prealloc": "1024", 2024-09-14T22:36:13.945 INFO:teuthology.orchestra.run.smithi001.stdout: "kstore_onode_map_size": "1024", 2024-09-14T22:36:13.945 INFO:teuthology.orchestra.run.smithi001.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-14T22:36:13.945 INFO:teuthology.orchestra.run.smithi001.stdout: "kstore_sync_submit_transaction": "false", 2024-09-14T22:36:13.945 INFO:teuthology.orchestra.run.smithi001.stdout: "kstore_sync_transaction": "false", 2024-09-14T22:36:13.945 INFO:teuthology.orchestra.run.smithi001.stdout: "librados_thread_count": "2", 2024-09-14T22:36:13.945 INFO:teuthology.orchestra.run.smithi001.stdout: "lockdep": "false", 2024-09-14T22:36:13.945 INFO:teuthology.orchestra.run.smithi001.stdout: "lockdep_force_backtrace": "false", 2024-09-14T22:36:13.945 INFO:teuthology.orchestra.run.smithi001.stdout: "log_coarse_timestamps": "true", 2024-09-14T22:36:13.945 INFO:teuthology.orchestra.run.smithi001.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-14T22:36:13.945 INFO:teuthology.orchestra.run.smithi001.stdout: "log_flush_on_exit": "false", 2024-09-14T22:36:13.946 INFO:teuthology.orchestra.run.smithi001.stdout: "log_graylog_host": "127.0.0.1", 2024-09-14T22:36:13.946 INFO:teuthology.orchestra.run.smithi001.stdout: "log_graylog_port": "12201", 2024-09-14T22:36:13.946 INFO:teuthology.orchestra.run.smithi001.stdout: "log_max_new": "1000", 2024-09-14T22:36:13.946 INFO:teuthology.orchestra.run.smithi001.stdout: "log_max_recent": "10000", 2024-09-14T22:36:13.946 INFO:teuthology.orchestra.run.smithi001.stdout: "log_stderr_prefix": "", 2024-09-14T22:36:13.946 INFO:teuthology.orchestra.run.smithi001.stdout: "log_stop_at_utilization": "0.970000", 2024-09-14T22:36:13.946 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": "true", 2024-09-14T22:36:13.946 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_graylog": "false", 2024-09-14T22:36:13.946 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_journald": "false", 2024-09-14T22:36:13.946 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_stderr": "false", 2024-09-14T22:36:13.946 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_syslog": "false", 2024-09-14T22:36:13.946 INFO:teuthology.orchestra.run.smithi001.stdout: "max_rotating_auth_attempts": "10", 2024-09-14T22:36:13.946 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-14T22:36:13.946 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_action_on_write_error": "1", 2024-09-14T22:36:13.947 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_alternate_name_max": "8192", 2024-09-14T22:36:13.947 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_asio_thread_count": "2", 2024-09-14T22:36:13.947 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_export_pin": "true", 2024-09-14T22:36:13.947 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_fragment_dirs": "true", 2024-09-14T22:36:13.947 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-14T22:36:13.947 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_fragment_interval": "5", 2024-09-14T22:36:13.947 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-14T22:36:13.947 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-14T22:36:13.947 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_interval": "10", 2024-09-14T22:36:13.947 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_max": "-1", 2024-09-14T22:36:13.947 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_max_until": "-1", 2024-09-14T22:36:13.947 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_merge_size": "50", 2024-09-14T22:36:13.947 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_midchunk": "0.300000", 2024-09-14T22:36:13.947 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-14T22:36:13.947 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_min_start": "0.200000", 2024-09-14T22:36:13.948 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_minchunk": "0.001000", 2024-09-14T22:36:13.948 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_mode": "0", 2024-09-14T22:36:13.948 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_need_max": "1.200000", 2024-09-14T22:36:13.948 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_need_min": "0.800000", 2024-09-14T22:36:13.948 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_overload_epochs": "2", 2024-09-14T22:36:13.948 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-14T22:36:13.948 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-14T22:36:13.948 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_split_bits": "3", 2024-09-14T22:36:13.948 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-14T22:36:13.948 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_split_size": "10000", 2024-09-14T22:36:13.948 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-14T22:36:13.948 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_target_decay": "10.000000", 2024-09-14T22:36:13.948 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-14T22:36:13.948 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_beacon_grace": "15.000000", 2024-09-14T22:36:13.948 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_beacon_interval": "4.000000", 2024-09-14T22:36:13.948 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-14T22:36:13.949 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-14T22:36:13.949 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cache_mid": "0.700000", 2024-09-14T22:36:13.949 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-14T22:36:13.949 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cache_quiesce_delay": "0", 2024-09-14T22:36:13.949 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-14T22:36:13.949 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-14T22:36:13.949 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-14T22:36:13.949 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cache_release_free_interval": "10", 2024-09-14T22:36:13.949 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cache_reservation": "0.050000", 2024-09-14T22:36:13.949 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-14T22:36:13.949 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cache_trim_interval": "1", 2024-09-14T22:36:13.949 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cache_trim_threshold": "262144", 2024-09-14T22:36:13.949 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-14T22:36:13.950 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-14T22:36:13.950 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-14T22:36:13.950 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_client_prealloc_inos": "1000", 2024-09-14T22:36:13.950 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-14T22:36:13.950 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_connect_bootstrapping": "false", 2024-09-14T22:36:13.950 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_damage_table_max_entries": "10000", 2024-09-14T22:36:13.950 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-14T22:36:13.950 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_debug_auth_pins": "false", 2024-09-14T22:36:13.950 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_debug_frag": "false", 2024-09-14T22:36:13.950 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_debug_scatterstat": "false", 2024-09-14T22:36:13.950 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_debug_subtrees": "false", 2024-09-14T22:36:13.950 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_decay_halflife": "5.000000", 2024-09-14T22:36:13.950 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_default_dir_hash": "2", 2024-09-14T22:36:13.950 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_defer_session_stale": "true", 2024-09-14T22:36:13.950 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_deny_all_reconnect": "false", 2024-09-14T22:36:13.951 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_dir_keys_per_op": "16384", 2024-09-14T22:36:13.951 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_dir_max_commit_size": "10", 2024-09-14T22:36:13.951 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_dir_max_entries": "0", 2024-09-14T22:36:13.951 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_dir_prefetch": "true", 2024-09-14T22:36:13.951 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-14T22:36:13.951 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-14T22:36:13.951 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_dump_cache_on_map": "false", 2024-09-14T22:36:13.951 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-14T22:36:13.951 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-14T22:36:13.951 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_early_reply": "true", 2024-09-14T22:36:13.951 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_enable_op_tracker": "true", 2024-09-14T22:36:13.951 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_enforce_unique_name": "true", 2024-09-14T22:36:13.951 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-14T22:36:13.951 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-14T22:36:13.952 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_export_ephemeral_random": "true", 2024-09-14T22:36:13.952 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-14T22:36:13.952 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-14T22:36:13.952 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-14T22:36:13.952 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-14T22:36:13.952 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-14T22:36:13.952 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-14T22:36:13.952 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-14T22:36:13.952 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-14T22:36:13.952 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_health_summarize_threshold": "10", 2024-09-14T22:36:13.952 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-14T22:36:13.952 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-14T22:36:13.952 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_inject_health_dummy": "false", 2024-09-14T22:36:13.952 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-14T22:36:13.953 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_inject_migrator_session_race": "false", 2024-09-14T22:36:13.953 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-14T22:36:13.953 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-14T22:36:13.953 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-14T22:36:13.953 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_join_fs": "", 2024-09-14T22:36:13.953 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_journal_format": "1", 2024-09-14T22:36:13.953 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-14T22:36:13.953 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_kill_create_at": "0", 2024-09-14T22:36:13.953 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_kill_export_at": "0", 2024-09-14T22:36:13.953 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_kill_import_at": "0", 2024-09-14T22:36:13.953 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_kill_journal_at": "0", 2024-09-14T22:36:13.953 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_kill_journal_expire_at": "0", 2024-09-14T22:36:13.953 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_kill_journal_replay_at": "0", 2024-09-14T22:36:13.954 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_kill_link_at": "0", 2024-09-14T22:36:13.954 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_kill_mdstable_at": "0", 2024-09-14T22:36:13.954 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_kill_openc_at": "0", 2024-09-14T22:36:13.954 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_kill_rename_at": "0", 2024-09-14T22:36:13.954 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_kill_shutdown_at": "0", 2024-09-14T22:36:13.954 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_log_event_large_threshold": "524288", 2024-09-14T22:36:13.954 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_log_events_per_segment": "1024", 2024-09-14T22:36:13.954 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-14T22:36:13.954 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_log_max_events": "-1", 2024-09-14T22:36:13.954 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_log_max_segments": "128", 2024-09-14T22:36:13.954 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_log_pause": "false", 2024-09-14T22:36:13.954 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_log_segment_size": "0", 2024-09-14T22:36:13.954 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-14T22:36:13.954 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-14T22:36:13.955 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-14T22:36:13.955 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_log_trim_threshold": "128", 2024-09-14T22:36:13.955 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-14T22:36:13.955 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_log_warn_factor": "2.000000", 2024-09-14T22:36:13.955 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_max_caps_per_client": "1048576", 2024-09-14T22:36:13.955 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_max_completed_flushes": "100000", 2024-09-14T22:36:13.955 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_max_completed_requests": "100000", 2024-09-14T22:36:13.955 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_max_export_size": "20971520", 2024-09-14T22:36:13.955 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_max_file_recover": "32", 2024-09-14T22:36:13.955 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_max_purge_files": "64", 2024-09-14T22:36:13.955 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_max_purge_ops": "8192", 2024-09-14T22:36:13.955 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-14T22:36:13.955 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-14T22:36:13.955 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_max_snaps_per_dir": "100", 2024-09-14T22:36:13.955 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_metrics_update_interval": "2", 2024-09-14T22:36:13.956 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_min_caps_per_client": "100", 2024-09-14T22:36:13.956 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_min_caps_working_set": "10000", 2024-09-14T22:36:13.956 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-14T22:36:13.956 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_numa_node": "-1", 2024-09-14T22:36:13.956 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-14T22:36:13.956 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_op_complaint_time": "30.000000", 2024-09-14T22:36:13.956 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_op_history_duration": "600", 2024-09-14T22:36:13.956 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_op_history_size": "20", 2024-09-14T22:36:13.956 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_op_history_slow_op_size": "20", 2024-09-14T22:36:13.956 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-14T22:36:13.956 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_op_log_threshold": "5", 2024-09-14T22:36:13.956 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_ping_grace": "15", 2024-09-14T22:36:13.956 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_ping_interval": "5", 2024-09-14T22:36:13.956 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-14T22:36:13.957 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-14T22:36:13.957 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_recall_max_caps": "30000", 2024-09-14T22:36:13.957 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-14T22:36:13.957 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-14T22:36:13.957 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-14T22:36:13.957 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_recall_warning_threshold": "262144", 2024-09-14T22:36:13.957 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-14T22:36:13.957 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_replay_interval": "1.000000", 2024-09-14T22:36:13.957 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-14T22:36:13.957 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-14T22:36:13.957 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_root_ino_gid": "0", 2024-09-14T22:36:13.957 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_root_ino_uid": "0", 2024-09-14T22:36:13.957 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-14T22:36:13.957 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-14T22:36:13.958 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-14T22:36:13.958 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-14T22:36:13.958 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-14T22:36:13.958 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-14T22:36:13.958 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-14T22:36:13.958 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-14T22:36:13.958 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-14T22:36:13.958 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-14T22:36:13.958 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_shutdown_check": "0", 2024-09-14T22:36:13.958 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_skip_ino": "0", 2024-09-14T22:36:13.958 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-14T22:36:13.958 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_snap_max_uid": "4294967294", 2024-09-14T22:36:13.958 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_snap_min_uid": "0", 2024-09-14T22:36:13.958 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_snap_rstat": "false", 2024-09-14T22:36:13.958 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_standby_replay_damaged": "false", 2024-09-14T22:36:13.959 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_symlink_recovery": "true", 2024-09-14T22:36:13.959 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-14T22:36:13.959 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_thrash_exports": "0", 2024-09-14T22:36:13.959 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_thrash_fragments": "0", 2024-09-14T22:36:13.959 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_tick_interval": "5.000000", 2024-09-14T22:36:13.959 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_valgrind_exit": "false", 2024-09-14T22:36:13.959 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_verify_backtrace": "1", 2024-09-14T22:36:13.959 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_verify_scatter": "false", 2024-09-14T22:36:13.959 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-14T22:36:13.959 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_wipe_sessions": "false", 2024-09-14T22:36:13.959 INFO:teuthology.orchestra.run.smithi001.stdout: "mempool_debug": "false", 2024-09-14T22:36:13.959 INFO:teuthology.orchestra.run.smithi001.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-14T22:36:13.959 INFO:teuthology.orchestra.run.smithi001.stdout: "memstore_device_bytes": "1073741824", 2024-09-14T22:36:13.959 INFO:teuthology.orchestra.run.smithi001.stdout: "memstore_page_set": "false", 2024-09-14T22:36:13.959 INFO:teuthology.orchestra.run.smithi001.stdout: "memstore_page_size": "65536", 2024-09-14T22:36:13.960 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_client_bytes": "134217728", 2024-09-14T22:36:13.960 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_client_messages": "512", 2024-09-14T22:36:13.960 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-14T22:36:13.960 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-14T22:36:13.960 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-14T22:36:13.960 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-14T22:36:13.960 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_disabled_modules": "", 2024-09-14T22:36:13.960 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_enable_op_tracker": "true", 2024-09-14T22:36:13.960 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-14T22:36:13.960 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_max_pg_num_change": "128", 2024-09-14T22:36:13.960 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_mds_bytes": "134217728", 2024-09-14T22:36:13.960 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_mds_messages": "128", 2024-09-14T22:36:13.960 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-14T22:36:13.961 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_mon_bytes": "134217728", 2024-09-14T22:36:13.961 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_mon_messages": "128", 2024-09-14T22:36:13.961 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-14T22:36:13.961 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-14T22:36:13.961 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_op_history_duration": "600", 2024-09-14T22:36:13.961 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_op_history_size": "20", 2024-09-14T22:36:13.961 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-14T22:36:13.961 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-14T22:36:13.961 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_op_log_threshold": "5", 2024-09-14T22:36:13.961 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_osd_bytes": "536870912", 2024-09-14T22:36:13.961 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_osd_messages": "8192", 2024-09-14T22:36:13.961 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_pool": "true", 2024-09-14T22:36:13.961 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-14T22:36:13.962 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_standby_modules": "true", 2024-09-14T22:36:13.962 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_stats_period": "5", 2024-09-14T22:36:13.962 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_stats_threshold": "5", 2024-09-14T22:36:13.962 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_tick_period": "2", 2024-09-14T22:36:13.962 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-14T22:36:13.962 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-14T22:36:13.962 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_allow_pool_delete": "true", 2024-09-14T22:36:13.962 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_allow_pool_size_one": "true", 2024-09-14T22:36:13.962 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_auth_validate_all_caps": "true", 2024-09-14T22:36:13.962 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-14T22:36:13.962 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-14T22:36:13.962 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_bytes": "104857600", 2024-09-14T22:36:13.962 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_directed_command_retry": "5", 2024-09-14T22:36:13.962 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-14T22:36:13.962 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-14T22:36:13.963 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-14T22:36:13.963 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-14T22:36:13.963 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_hunt_parallel": "3", 2024-09-14T22:36:13.963 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_log_interval": "1.000000", 2024-09-14T22:36:13.963 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-14T22:36:13.963 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_ping_interval": "10.000000", 2024-09-14T22:36:13.963 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-14T22:36:13.963 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_target_rank": "-1", 2024-09-14T22:36:13.963 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-14T22:36:13.963 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-14T22:36:13.963 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-14T22:36:13.963 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cluster_log_file_level": "debug", 2024-09-14T22:36:13.963 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cluster_log_to_file": "true", 2024-09-14T22:36:13.963 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-14T22:36:13.964 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-14T22:36:13.964 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-14T22:36:13.964 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cluster_log_to_journald": "false", 2024-09-14T22:36:13.964 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-14T22:36:13.964 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-14T22:36:13.964 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-14T22:36:13.964 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-14T22:36:13.964 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_compact_on_bootstrap": "false", 2024-09-14T22:36:13.964 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_compact_on_start": "false", 2024-09-14T22:36:13.964 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_compact_on_trim": "true", 2024-09-14T22:36:13.964 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-14T22:36:13.964 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-14T22:36:13.964 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-14T22:36:13.965 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cpu_threads": "4", 2024-09-14T22:36:13.965 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_crush_min_required_version": "hammer", 2024-09-14T22:36:13.965 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_daemon_bytes": "419430400", 2024-09-14T22:36:13.965 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-14T22:36:13.965 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_data_avail_crit": "5", 2024-09-14T22:36:13.965 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_data_avail_warn": "5", 2024-09-14T22:36:13.965 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_data_size_warn": "16106127360", 2024-09-14T22:36:13.965 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-14T22:36:13.965 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-14T22:36:13.965 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_debug_dump_json": "false", 2024-09-14T22:36:13.965 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-14T22:36:13.965 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_debug_dump_transactions": "false", 2024-09-14T22:36:13.965 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_debug_extra_checks": "false", 2024-09-14T22:36:13.965 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-14T22:36:13.966 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-14T22:36:13.966 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_debug_no_require_reef": "false", 2024-09-14T22:36:13.966 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_debug_no_require_squid": "false", 2024-09-14T22:36:13.966 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-14T22:36:13.966 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-14T22:36:13.966 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-14T22:36:13.966 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_down_mkfs_grace": "120", 2024-09-14T22:36:13.966 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_down_uptime_grace": "60", 2024-09-14T22:36:13.966 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_election_default_strategy": "1", 2024-09-14T22:36:13.966 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_election_timeout": "5.000000", 2024-09-14T22:36:13.966 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-14T22:36:13.966 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_elector_ping_divisor": "2", 2024-09-14T22:36:13.966 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-14T22:36:13.966 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_enable_op_tracker": "true", 2024-09-14T22:36:13.967 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_fake_pool_delete": "false", 2024-09-14T22:36:13.967 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_force_quorum_join": "false", 2024-09-14T22:36:13.967 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-14T22:36:13.967 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_globalid_prealloc": "10000", 2024-09-14T22:36:13.967 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_health_detail_to_clog": "false", 2024-09-14T22:36:13.967 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_health_log_update_period": "5", 2024-09-14T22:36:13.967 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_health_max_detail": "50", 2024-09-14T22:36:13.967 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_health_to_clog": "true", 2024-09-14T22:36:13.967 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_health_to_clog_interval": "600", 2024-09-14T22:36:13.967 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-14T22:36:13.967 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_host": "172.21.15.1,[v2:172.21.15.1:3301,v1:172.21.15.1:6790],172.21.15.176", 2024-09-14T22:36:13.967 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_host_override": "", 2024-09-14T22:36:13.967 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_initial_members": "", 2024-09-14T22:36:13.968 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-14T22:36:13.968 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-14T22:36:13.968 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-14T22:36:13.968 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-14T22:36:13.968 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-14T22:36:13.968 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_lease": "5.000000", 2024-09-14T22:36:13.968 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-14T22:36:13.968 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-14T22:36:13.968 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_log_full_interval": "50", 2024-09-14T22:36:13.968 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_log_max": "10000", 2024-09-14T22:36:13.968 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_log_max_summary": "50", 2024-09-14T22:36:13.968 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-14T22:36:13.968 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_max_log_epochs": "500", 2024-09-14T22:36:13.968 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-14T22:36:13.969 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-14T22:36:13.969 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_max_osd": "10000", 2024-09-14T22:36:13.969 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_max_pg_per_osd": "10000", 2024-09-14T22:36:13.969 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_max_pool_pg_num": "65536", 2024-09-14T22:36:13.969 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-14T22:36:13.969 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-14T22:36:13.969 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_mds_force_trim_to": "0", 2024-09-14T22:36:13.969 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_mds_skip_sanity": "false", 2024-09-14T22:36:13.969 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_memory_autotune": "true", 2024-09-14T22:36:13.969 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_memory_target": "2147483648", 2024-09-14T22:36:13.969 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_mgr_beacon_grace": "30", 2024-09-14T22:36:13.969 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-14T22:36:13.969 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_mgr_digest_period": "5", 2024-09-14T22:36:13.970 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_mgr_inactive_grace": "60", 2024-09-14T22:36:13.970 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-14T22:36:13.970 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-14T22:36:13.970 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_min_osdmap_epochs": "500", 2024-09-14T22:36:13.970 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_op_complaint_time": "30", 2024-09-14T22:36:13.970 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_op_history_duration": "600", 2024-09-14T22:36:13.970 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_op_history_size": "20", 2024-09-14T22:36:13.970 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_op_history_slow_op_size": "20", 2024-09-14T22:36:13.970 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-14T22:36:13.970 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_op_log_threshold": "5", 2024-09-14T22:36:13.970 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-14T22:36:13.970 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-14T22:36:13.970 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-14T22:36:13.970 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_auto_mark_in": "false", 2024-09-14T22:36:13.970 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-14T22:36:13.971 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-14T22:36:13.971 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-14T22:36:13.971 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_cache_size": "500", 2024-09-14T22:36:13.971 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-14T22:36:13.971 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-14T22:36:13.971 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-14T22:36:13.971 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_down_out_interval": "600", 2024-09-14T22:36:13.971 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-14T22:36:13.971 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-14T22:36:13.971 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_force_trim_to": "0", 2024-09-14T22:36:13.971 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-14T22:36:13.971 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-14T22:36:13.971 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-14T22:36:13.971 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-14T22:36:13.972 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-14T22:36:13.972 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-14T22:36:13.972 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-14T22:36:13.972 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-14T22:36:13.972 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_min_down_reporters": "2", 2024-09-14T22:36:13.972 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-14T22:36:13.972 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-14T22:36:13.972 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-14T22:36:13.972 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-14T22:36:13.972 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-14T22:36:13.972 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-14T22:36:13.972 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_report_timeout": "900", 2024-09-14T22:36:13.972 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-14T22:36:13.972 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-14T22:36:13.972 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-14T22:36:13.973 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-14T22:36:13.973 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-14T22:36:13.973 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-14T22:36:13.973 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-14T22:36:13.973 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-14T22:36:13.973 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-14T22:36:13.973 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_pg_stuck_threshold": "60", 2024-09-14T22:36:13.973 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-14T22:36:13.973 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-14T22:36:13.973 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-14T22:36:13.973 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-14T22:36:13.973 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-14T22:36:13.973 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-14T22:36:13.973 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_probe_timeout": "2.000000", 2024-09-14T22:36:13.974 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_reweight_max_change": "0.050000", 2024-09-14T22:36:13.974 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_reweight_max_osds": "4", 2024-09-14T22:36:13.974 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-14T22:36:13.974 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-14T22:36:13.974 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-14T22:36:13.974 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-14T22:36:13.974 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-14T22:36:13.974 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_scrub_interval": "86400", 2024-09-14T22:36:13.974 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_scrub_max_keys": "100", 2024-09-14T22:36:13.974 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_scrub_timeout": "300", 2024-09-14T22:36:13.974 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_session_timeout": "300", 2024-09-14T22:36:13.974 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_smart_report_timeout": "5", 2024-09-14T22:36:13.974 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_stat_smooth_intervals": "6", 2024-09-14T22:36:13.974 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-14T22:36:13.974 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_stretch_pool_min_size": "2", 2024-09-14T22:36:13.975 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_stretch_pool_size": "4", 2024-09-14T22:36:13.975 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-14T22:36:13.975 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-14T22:36:13.975 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_sync_debug": "false", 2024-09-14T22:36:13.975 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-14T22:36:13.975 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-14T22:36:13.975 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_sync_provider_kill_at": "0", 2024-09-14T22:36:13.975 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_sync_requester_kill_at": "0", 2024-09-14T22:36:13.975 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_sync_timeout": "60.000000", 2024-09-14T22:36:13.975 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_target_pg_per_osd": "100", 2024-09-14T22:36:13.975 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_tick_interval": "5", 2024-09-14T22:36:13.975 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_timecheck_interval": "300.000000", 2024-09-14T22:36:13.975 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-14T22:36:13.975 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_use_min_delay_socket": "false", 2024-09-14T22:36:13.976 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_older_version_delay": "604800", 2024-09-14T22:36:13.976 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-14T22:36:13.976 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-14T22:36:13.976 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-14T22:36:13.976 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-14T22:36:13.976 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-14T22:36:13.976 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-14T22:36:13.976 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-14T22:36:13.976 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_misplaced": "false", 2024-09-14T22:36:13.976 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-14T22:36:13.976 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_older_version": "true", 2024-09-14T22:36:13.976 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-14T22:36:13.976 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-14T22:36:13.976 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-14T22:36:13.977 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-14T22:36:13.977 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-14T22:36:13.977 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-14T22:36:13.977 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-14T22:36:13.977 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-14T22:36:13.977 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-14T22:36:13.977 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": "", 2024-09-14T22:36:13.977 INFO:teuthology.orchestra.run.smithi001.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-14T22:36:13.977 INFO:teuthology.orchestra.run.smithi001.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-14T22:36:13.977 INFO:teuthology.orchestra.run.smithi001.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-14T22:36:13.977 INFO:teuthology.orchestra.run.smithi001.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-14T22:36:13.977 INFO:teuthology.orchestra.run.smithi001.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-14T22:36:13.977 INFO:teuthology.orchestra.run.smithi001.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-14T22:36:13.977 INFO:teuthology.orchestra.run.smithi001.stdout: "motr_tracing_enabled": "false", 2024-09-14T22:36:13.977 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_op_threads": "3", 2024-09-14T22:36:13.978 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-14T22:36:13.978 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_cm": "false", 2024-09-14T22:36:13.978 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_device_name": "", 2024-09-14T22:36:13.978 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_dscp": "96", 2024-09-14T22:36:13.978 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-14T22:36:13.978 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-14T22:36:13.978 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_local_gid": "", 2024-09-14T22:36:13.978 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-14T22:36:13.978 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_port_num": "1", 2024-09-14T22:36:13.978 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-14T22:36:13.978 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-14T22:36:13.978 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-14T22:36:13.978 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-14T22:36:13.979 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_sl": "3", 2024-09-14T22:36:13.979 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_support_srq": "true", 2024-09-14T22:36:13.979 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_type": "ib", 2024-09-14T22:36:13.979 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_reap_threshold": "5", 2024-09-14T22:36:13.979 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_bind_before_connect": "false", 2024-09-14T22:36:13.979 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_bind_ipv4": "true", 2024-09-14T22:36:13.979 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_bind_ipv6": "false", 2024-09-14T22:36:13.979 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_bind_msgr1": "true", 2024-09-14T22:36:13.979 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_bind_msgr2": "true", 2024-09-14T22:36:13.979 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_bind_port_max": "7568", 2024-09-14T22:36:13.979 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_bind_port_min": "6800", 2024-09-14T22:36:13.979 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-14T22:36:13.979 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_bind_retry_count": "3", 2024-09-14T22:36:13.980 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_bind_retry_delay": "5", 2024-09-14T22:36:13.980 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_blackhole_client": "false", 2024-09-14T22:36:13.980 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_blackhole_mds": "false", 2024-09-14T22:36:13.980 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_blackhole_mgr": "false", 2024-09-14T22:36:13.980 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_blackhole_mon": "false", 2024-09-14T22:36:13.980 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_blackhole_osd": "false", 2024-09-14T22:36:13.980 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_client_mode": "crc secure", 2024-09-14T22:36:13.980 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_cluster_mode": "crc secure", 2024-09-14T22:36:13.980 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_cluster_type": "", 2024-09-14T22:36:13.980 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_compress_secure": "false", 2024-09-14T22:36:13.980 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_connection_idle_timeout": "900", 2024-09-14T22:36:13.980 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_connection_ready_timeout": "10", 2024-09-14T22:36:13.980 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_crc_data": "true", 2024-09-14T22:36:13.980 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_crc_header": "true", 2024-09-14T22:36:13.981 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_die_on_bad_msg": "false", 2024-09-14T22:36:13.981 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_die_on_bug": "true", 2024-09-14T22:36:13.981 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_die_on_old_message": "true", 2024-09-14T22:36:13.981 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_die_on_skipped_message": "false", 2024-09-14T22:36:13.981 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-14T22:36:13.981 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-14T22:36:13.981 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_coremask": "0xF", 2024-09-14T22:36:13.981 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-14T22:36:13.981 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-14T22:36:13.981 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_enable_tso": "true", 2024-09-14T22:36:13.981 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-14T22:36:13.981 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-14T22:36:13.981 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_hugepages": "", 2024-09-14T22:36:13.981 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-14T22:36:13.981 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-14T22:36:13.982 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_lro": "true", 2024-09-14T22:36:13.982 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_memory_channel": "4", 2024-09-14T22:36:13.982 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-14T22:36:13.982 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_pmd": "", 2024-09-14T22:36:13.982 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_port_id": "0", 2024-09-14T22:36:13.982 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-14T22:36:13.982 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-14T22:36:13.982 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dump_on_send": "false", 2024-09-14T22:36:13.982 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_initial_backoff": "0.200000", 2024-09-14T22:36:13.982 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_inject_delay_max": "1.000000", 2024-09-14T22:36:13.982 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-14T22:36:13.982 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_inject_delay_type": "", 2024-09-14T22:36:13.982 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-14T22:36:13.983 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_inject_network_congestion": "0", 2024-09-14T22:36:13.983 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_inject_socket_failures": "5000", 2024-09-14T22:36:13.983 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_learn_addr_from_peer": "true", 2024-09-14T22:36:13.983 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_max_accept_failures": "4", 2024-09-14T22:36:13.983 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_max_backoff": "15.000000", 2024-09-14T22:36:13.983 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_mon_client_mode": "secure crc", 2024-09-14T22:36:13.983 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-14T22:36:13.983 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_mon_service_mode": "secure crc", 2024-09-14T22:36:13.983 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_osd_compress_min_size": "1024", 2024-09-14T22:36:13.983 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_osd_compress_mode": "none", 2024-09-14T22:36:13.983 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-14T22:36:13.983 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-14T22:36:13.983 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_pq_min_cost": "65536", 2024-09-14T22:36:13.983 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_public_type": "", 2024-09-14T22:36:13.984 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_service_mode": "crc secure", 2024-09-14T22:36:13.984 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_tcp_listen_backlog": "512", 2024-09-14T22:36:13.984 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_tcp_nodelay": "true", 2024-09-14T22:36:13.984 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-14T22:36:13.984 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_tcp_rcvbuf": "0", 2024-09-14T22:36:13.984 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_type": "async+posix", 2024-09-14T22:36:13.984 INFO:teuthology.orchestra.run.smithi001.stdout: "no_config_file": "false", 2024-09-14T22:36:13.984 INFO:teuthology.orchestra.run.smithi001.stdout: "objecter_completion_locks_per_session": "32", 2024-09-14T22:36:13.984 INFO:teuthology.orchestra.run.smithi001.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-14T22:36:13.984 INFO:teuthology.orchestra.run.smithi001.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-14T22:36:13.984 INFO:teuthology.orchestra.run.smithi001.stdout: "objecter_inflight_ops": "1024", 2024-09-14T22:36:13.984 INFO:teuthology.orchestra.run.smithi001.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-14T22:36:13.984 INFO:teuthology.orchestra.run.smithi001.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-14T22:36:13.984 INFO:teuthology.orchestra.run.smithi001.stdout: "objecter_tick_interval": "5.000000", 2024-09-14T22:36:13.984 INFO:teuthology.orchestra.run.smithi001.stdout: "objecter_timeout": "10.000000", 2024-09-14T22:36:13.985 INFO:teuthology.orchestra.run.smithi001.stdout: "objectstore_blackhole": "false", 2024-09-14T22:36:13.985 INFO:teuthology.orchestra.run.smithi001.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-14T22:36:13.985 INFO:teuthology.orchestra.run.smithi001.stdout: "openssl_engine_opts": "", 2024-09-14T22:36:13.985 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_agent_delay_time": "5.000000", 2024-09-14T22:36:13.985 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_agent_hist_halflife": "1000", 2024-09-14T22:36:13.985 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_agent_max_low_ops": "2", 2024-09-14T22:36:13.985 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_agent_max_ops": "4", 2024-09-14T22:36:13.985 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-14T22:36:13.985 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-14T22:36:13.985 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_agent_slop": "0.020000", 2024-09-14T22:36:13.985 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-14T22:36:13.985 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-14T22:36:13.985 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_asio_thread_count": "2", 2024-09-14T22:36:13.985 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_async_recovery_min_cost": "100", 2024-09-14T22:36:13.986 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-14T22:36:13.986 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-14T22:36:13.986 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_backfill_scan_max": "512", 2024-09-14T22:36:13.986 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_backfill_scan_min": "64", 2024-09-14T22:36:13.986 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_backoff_on_degraded": "false", 2024-09-14T22:36:13.986 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_backoff_on_peering": "false", 2024-09-14T22:36:13.986 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_backoff_on_unfound": "true", 2024-09-14T22:36:13.986 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_beacon_report_interval": "300", 2024-09-14T22:36:13.986 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bench_duration": "30", 2024-09-14T22:36:13.986 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-14T22:36:13.986 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bench_max_block_size": "67108864", 2024-09-14T22:36:13.986 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-14T22:36:13.986 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_blkin_trace_all": "false", 2024-09-14T22:36:13.986 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-14T22:36:13.987 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-14T22:36:13.987 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-14T22:36:13.987 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-14T22:36:13.987 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_check_for_log_corruption": "false", 2024-09-14T22:36:13.987 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-14T22:36:13.987 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-14T22:36:13.987 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-09-14T22:36:13.987 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-14T22:36:13.987 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_class_update_on_start": "true", 2024-09-14T22:36:13.987 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_client_message_cap": "256", 2024-09-14T22:36:13.987 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_client_message_size_cap": "524288000", 2024-09-14T22:36:13.987 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_client_op_priority": "63", 2024-09-14T22:36:13.987 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_client_watch_timeout": "30", 2024-09-14T22:36:13.987 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_command_max_records": "256", 2024-09-14T22:36:13.988 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_compact_on_start": "false", 2024-09-14T22:36:13.988 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-14T22:36:13.988 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-14T22:36:13.988 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-14T22:36:13.988 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_crush_update_on_start": "true", 2024-09-14T22:36:13.988 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_crush_update_weight_set": "true", 2024-09-14T22:36:13.988 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-14T22:36:13.988 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-14T22:36:13.988 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-14T22:36:13.988 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-14T22:36:13.988 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-14T22:36:13.988 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_feed_pullee": "-1", 2024-09-14T22:36:13.988 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-14T22:36:13.989 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-14T22:36:13.989 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-14T22:36:13.989 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_misdirected_ops": "false", 2024-09-14T22:36:13.989 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_no_acting_change": "false", 2024-09-14T22:36:13.989 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_no_purge_strays": "false", 2024-09-14T22:36:13.989 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_op_order": "false", 2024-09-14T22:36:13.989 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-14T22:36:13.989 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-14T22:36:13.989 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-14T22:36:13.989 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-14T22:36:13.989 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_shutdown": "false", 2024-09-14T22:36:13.989 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-14T22:36:13.989 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-14T22:36:13.989 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_trim_objects": "false", 2024-09-14T22:36:13.989 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-14T22:36:13.989 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-14T22:36:13.990 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_verify_snaps": "false", 2024-09-14T22:36:13.990 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-14T22:36:13.990 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-14T22:36:13.990 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_deep_scrub_keys": "1024", 2024-09-14T22:36:13.990 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-14T22:36:13.990 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-14T22:36:13.990 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-14T22:36:13.990 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_deep_scrub_stride": "524288", 2024-09-14T22:36:13.990 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-14T22:36:13.990 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-14T22:36:13.990 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_default_notify_timeout": "30", 2024-09-14T22:36:13.990 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_delete_sleep": "0.000000", 2024-09-14T22:36:13.990 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-14T22:36:13.990 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-14T22:36:13.990 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-14T22:36:13.991 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_discard_disconnected_ops": "true", 2024-09-14T22:36:13.991 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_enable_op_tracker": "true", 2024-09-14T22:36:13.991 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-14T22:36:13.991 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-14T22:36:13.991 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-14T22:36:13.991 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-14T22:36:13.991 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_fast_info": "true", 2024-09-14T22:36:13.991 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_fast_shutdown": "true", 2024-09-14T22:36:13.991 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-14T22:36:13.991 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-14T22:36:13.991 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-14T22:36:13.991 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-14T22:36:13.991 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-14T22:36:13.992 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_function_tracing": "false", 2024-09-14T22:36:13.992 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_heartbeat_grace": "20", 2024-09-14T22:36:13.992 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_heartbeat_interval": "6", 2024-09-14T22:36:13.992 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-14T22:36:13.992 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_heartbeat_min_peers": "10", 2024-09-14T22:36:13.992 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_heartbeat_min_size": "2000", 2024-09-14T22:36:13.992 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_heartbeat_stale": "600", 2024-09-14T22:36:13.992 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-14T22:36:13.992 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_hit_set_max_size": "100000", 2024-09-14T22:36:13.992 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_hit_set_min_size": "1000", 2024-09-14T22:36:13.992 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-14T22:36:13.992 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-14T22:36:13.992 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-14T22:36:13.993 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-14T22:36:13.993 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-14T22:36:13.993 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-14T22:36:13.993 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_journal_size": "5120", 2024-09-14T22:36:13.993 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_kill_backfill_at": "0", 2024-09-14T22:36:13.993 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-14T22:36:13.993 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_map_cache_size": "50", 2024-09-14T22:36:13.993 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_map_dedup": "true", 2024-09-14T22:36:13.993 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_map_message_max": "40", 2024-09-14T22:36:13.993 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-14T22:36:13.993 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_map_share_max_epochs": "40", 2024-09-14T22:36:13.993 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_attr_name_len": "100", 2024-09-14T22:36:13.993 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_attr_size": "0", 2024-09-14T22:36:13.993 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_backfills": "1", 2024-09-14T22:36:13.993 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_markdown_count": "5", 2024-09-14T22:36:13.994 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_markdown_period": "600", 2024-09-14T22:36:13.994 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_object_name_len": "2048", 2024-09-14T22:36:13.994 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_object_namespace_len": "256", 2024-09-14T22:36:13.994 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_object_size": "134217728", 2024-09-14T22:36:13.994 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-14T22:36:13.994 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-14T22:36:13.994 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_pg_blocked_by": "16", 2024-09-14T22:36:13.994 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_pg_log_entries": "10000", 2024-09-14T22:36:13.994 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-14T22:36:13.994 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_pgls": "1024", 2024-09-14T22:36:13.994 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_push_cost": "8388608", 2024-09-14T22:36:13.994 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_push_objects": "10", 2024-09-14T22:36:13.994 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_scrubs": "3", 2024-09-14T22:36:13.994 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-14T22:36:13.995 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_trimming_pgs": "2", 2024-09-14T22:36:13.995 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_write_op_reply_len": "64", 2024-09-14T22:36:13.995 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_write_size": "90", 2024-09-14T22:36:13.995 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-14T22:36:13.995 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-14T22:36:13.995 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-14T22:36:13.995 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-14T22:36:13.995 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-14T22:36:13.995 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-14T22:36:13.995 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-14T22:36:13.995 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-14T22:36:13.995 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_profile": "balanced", 2024-09-14T22:36:13.995 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-14T22:36:13.995 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-14T22:36:13.996 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-14T22:36:13.996 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-14T22:36:13.996 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-14T22:36:13.996 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-14T22:36:13.996 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-14T22:36:13.996 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-14T22:36:13.996 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-14T22:36:13.996 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-14T22:36:13.996 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-14T22:36:13.996 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_memory_base": "805306368", 2024-09-14T22:36:13.996 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_memory_cache_min": "134217728", 2024-09-14T22:36:13.996 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-14T22:36:13.996 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-14T22:36:13.997 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_memory_target": "4294967296", 2024-09-14T22:36:13.997 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_memory_target_autotune": "false", 2024-09-14T22:36:13.997 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-14T22:36:13.997 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_min_pg_log_entries": "250", 2024-09-14T22:36:13.997 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_min_recovery_priority": "0", 2024-09-14T22:36:13.997 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-14T22:36:13.997 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-14T22:36:13.997 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mon_report_interval": "5", 2024-09-14T22:36:13.997 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-14T22:36:13.997 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-14T22:36:13.997 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_num_cache_shards": "32", 2024-09-14T22:36:13.997 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_num_op_tracker_shard": "32", 2024-09-14T22:36:13.997 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_numa_auto_affinity": "true", 2024-09-14T22:36:13.997 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_numa_node": "-1", 2024-09-14T22:36:13.997 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_numa_prefer_iface": "true", 2024-09-14T22:36:13.998 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-14T22:36:13.998 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_objecter_finishers": "1", 2024-09-14T22:36:13.998 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_objectstore": "bluestore", 2024-09-14T22:36:13.998 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_objectstore_fuse": "false", 2024-09-14T22:36:13.998 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_objectstore_tracing": "false", 2024-09-14T22:36:13.998 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_complaint_time": "30.000000", 2024-09-14T22:36:13.998 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_history_duration": "600", 2024-09-14T22:36:13.998 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_history_size": "20", 2024-09-14T22:36:13.998 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_history_slow_op_size": "20", 2024-09-14T22:36:13.998 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-14T22:36:13.998 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_log_threshold": "5", 2024-09-14T22:36:13.998 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_num_shards": "0", 2024-09-14T22:36:13.998 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_num_shards_hdd": "5", 2024-09-14T22:36:13.998 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_num_shards_ssd": "8", 2024-09-14T22:36:13.998 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-14T22:36:13.999 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-14T22:36:13.999 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-14T22:36:13.999 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-14T22:36:13.999 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_pq_min_cost": "65536", 2024-09-14T22:36:13.999 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-14T22:36:13.999 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_queue_cut_off": "high", 2024-09-14T22:36:13.999 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-14T22:36:13.999 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_thread_timeout": "15", 2024-09-14T22:36:13.999 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_open_classes_on_start": "true", 2024-09-14T22:36:13.999 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_os_flags": "0", 2024-09-14T22:36:13.999 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_peering_op_priority": "255", 2024-09-14T22:36:13.999 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pg_delete_cost": "1048576", 2024-09-14T22:36:13.999 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pg_delete_priority": "5", 2024-09-14T22:36:13.999 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-14T22:36:13.999 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-14T22:36:14.000 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-14T22:36:14.000 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pg_log_trim_max": "10000", 2024-09-14T22:36:14.000 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pg_log_trim_min": "100", 2024-09-14T22:36:14.000 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-14T22:36:14.000 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-14T22:36:14.000 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-14T22:36:14.000 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-14T22:36:14.000 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-14T22:36:14.000 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-14T22:36:14.000 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-14T22:36:14.000 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-14T22:36:14.000 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-14T22:36:14.000 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-14T22:36:14.000 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_crimson": "false", 2024-09-14T22:36:14.001 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-14T22:36:14.001 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-14T22:36:14.001 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-14T22:36:14.001 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-14T22:36:14.001 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-14T22:36:14.001 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-14T22:36:14.001 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-14T22:36:14.001 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-14T22:36:14.001 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_flags": "0", 2024-09-14T22:36:14.001 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-14T22:36:14.001 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_min_size": "0", 2024-09-14T22:36:14.001 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-14T22:36:14.001 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_pg_num": "32", 2024-09-14T22:36:14.001 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_pgp_num": "0", 2024-09-14T22:36:14.002 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-14T22:36:14.002 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_read_ratio": "70", 2024-09-14T22:36:14.002 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_size": "2", 2024-09-14T22:36:14.002 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_type": "replicated", 2024-09-14T22:36:14.002 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-14T22:36:14.002 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-14T22:36:14.002 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_push_per_object_cost": "1000", 2024-09-14T22:36:14.002 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-14T22:36:14.002 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recover_clone_overlap": "true", 2024-09-14T22:36:14.002 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-14T22:36:14.002 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_cost": "20971520", 2024-09-14T22:36:14.002 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-14T22:36:14.002 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_max_active": "0", 2024-09-14T22:36:14.003 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-14T22:36:14.003 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-14T22:36:14.003 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-14T22:36:14.003 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-14T22:36:14.003 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_max_single_start": "1", 2024-09-14T22:36:14.003 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_op_priority": "3", 2024-09-14T22:36:14.003 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-14T22:36:14.003 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_priority": "5", 2024-09-14T22:36:14.003 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-14T22:36:14.003 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_sleep": "0.000000", 2024-09-14T22:36:14.003 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-14T22:36:14.003 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-14T22:36:14.003 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-14T22:36:14.003 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_repair_during_recovery": "false", 2024-09-14T22:36:14.003 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_requested_scrub_priority": "120", 2024-09-14T22:36:14.003 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-14T22:36:14.004 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-14T22:36:14.004 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_auto_repair": "false", 2024-09-14T22:36:14.004 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-14T22:36:14.004 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-14T22:36:14.004 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_begin_hour": "0", 2024-09-14T22:36:14.004 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_begin_week_day": "0", 2024-09-14T22:36:14.004 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_chunk_max": "25", 2024-09-14T22:36:14.004 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_chunk_min": "5", 2024-09-14T22:36:14.004 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_cost": "52428800", 2024-09-14T22:36:14.004 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-14T22:36:14.004 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_during_recovery": "false", 2024-09-14T22:36:14.004 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_end_hour": "0", 2024-09-14T22:36:14.004 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_end_week_day": "0", 2024-09-14T22:36:14.004 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_event_cost": "4096", 2024-09-14T22:36:14.005 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-14T22:36:14.005 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-14T22:36:14.005 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_invalid_stats": "true", 2024-09-14T22:36:14.005 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-14T22:36:14.005 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-14T22:36:14.005 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_max_preemptions": "5", 2024-09-14T22:36:14.005 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-14T22:36:14.005 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_priority": "5", 2024-09-14T22:36:14.005 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_sleep": "0.000000", 2024-09-14T22:36:14.005 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-14T22:36:14.005 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-14T22:36:14.005 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-14T22:36:14.005 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_skip_data_digest": "false", 2024-09-14T22:36:14.005 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_smart_report_timeout": "5", 2024-09-14T22:36:14.005 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_snap_trim_cost": "1048576", 2024-09-14T22:36:14.006 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_snap_trim_priority": "5", 2024-09-14T22:36:14.006 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-14T22:36:14.006 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-14T22:36:14.006 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-14T22:36:14.006 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-14T22:36:14.006 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-14T22:36:14.006 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-14T22:36:14.006 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-14T22:36:14.006 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_target_transaction_size": "30", 2024-09-14T22:36:14.006 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-14T22:36:14.006 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-14T22:36:14.006 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-14T22:36:14.006 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-14T22:36:14.006 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-14T22:36:14.007 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-14T22:36:14.007 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-14T22:36:14.007 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-14T22:36:14.007 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-14T22:36:14.007 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-14T22:36:14.007 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_tracing": "false", 2024-09-14T22:36:14.007 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_use_stale_snap": "false", 2024-09-14T22:36:14.007 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-14T22:36:14.007 INFO:teuthology.orchestra.run.smithi001.stdout: "osdc_blkin_trace_all": "false", 2024-09-14T22:36:14.007 INFO:teuthology.orchestra.run.smithi001.stdout: "paxos_kill_at": "0", 2024-09-14T22:36:14.007 INFO:teuthology.orchestra.run.smithi001.stdout: "paxos_max_join_drift": "10", 2024-09-14T22:36:14.007 INFO:teuthology.orchestra.run.smithi001.stdout: "paxos_min": "500", 2024-09-14T22:36:14.007 INFO:teuthology.orchestra.run.smithi001.stdout: "paxos_min_wait": "0.050000", 2024-09-14T22:36:14.007 INFO:teuthology.orchestra.run.smithi001.stdout: "paxos_propose_interval": "1.000000", 2024-09-14T22:36:14.008 INFO:teuthology.orchestra.run.smithi001.stdout: "paxos_service_trim_max": "500", 2024-09-14T22:36:14.008 INFO:teuthology.orchestra.run.smithi001.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-14T22:36:14.008 INFO:teuthology.orchestra.run.smithi001.stdout: "paxos_service_trim_min": "250", 2024-09-14T22:36:14.008 INFO:teuthology.orchestra.run.smithi001.stdout: "paxos_stash_full_interval": "25", 2024-09-14T22:36:14.008 INFO:teuthology.orchestra.run.smithi001.stdout: "paxos_trim_max": "500", 2024-09-14T22:36:14.008 INFO:teuthology.orchestra.run.smithi001.stdout: "paxos_trim_min": "250", 2024-09-14T22:36:14.008 INFO:teuthology.orchestra.run.smithi001.stdout: "perf": "true", 2024-09-14T22:36:14.008 INFO:teuthology.orchestra.run.smithi001.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-14T22:36:14.008 INFO:teuthology.orchestra.run.smithi001.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-14T22:36:14.008 INFO:teuthology.orchestra.run.smithi001.stdout: "plugin_dir": "/usr/lib/ceph", 2024-09-14T22:36:14.008 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "-", 2024-09-14T22:36:14.008 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrv": "", 2024-09-14T22:36:14.008 INFO:teuthology.orchestra.run.smithi001.stdout: "public_bind_addr": "-", 2024-09-14T22:36:14.008 INFO:teuthology.orchestra.run.smithi001.stdout: "public_network": "", 2024-09-14T22:36:14.008 INFO:teuthology.orchestra.run.smithi001.stdout: "public_network_interface": "", 2024-09-14T22:36:14.008 INFO:teuthology.orchestra.run.smithi001.stdout: "qat_compressor_busy_polling": "false", 2024-09-14T22:36:14.009 INFO:teuthology.orchestra.run.smithi001.stdout: "qat_compressor_enabled": "false", 2024-09-14T22:36:14.009 INFO:teuthology.orchestra.run.smithi001.stdout: "qat_compressor_session_max_number": "256", 2024-09-14T22:36:14.009 INFO:teuthology.orchestra.run.smithi001.stdout: "rados_mon_op_timeout": "0", 2024-09-14T22:36:14.009 INFO:teuthology.orchestra.run.smithi001.stdout: "rados_osd_op_timeout": "0", 2024-09-14T22:36:14.009 INFO:teuthology.orchestra.run.smithi001.stdout: "rados_tracing": "false", 2024-09-14T22:36:14.009 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_atime_update_interval": "60", 2024-09-14T22:36:14.009 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-14T22:36:14.009 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_balance_parent_reads": "false", 2024-09-14T22:36:14.009 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_balance_snap_reads": "false", 2024-09-14T22:36:14.009 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_blkin_trace_all": "false", 2024-09-14T22:36:14.009 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-14T22:36:14.009 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-14T22:36:14.009 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_cache": "true", 2024-09-14T22:36:14.009 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-14T22:36:14.010 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-14T22:36:14.010 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-14T22:36:14.010 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-14T22:36:14.010 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_cache_policy": "writearound", 2024-09-14T22:36:14.010 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_cache_size": "33554432", 2024-09-14T22:36:14.010 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-14T22:36:14.010 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-14T22:36:14.010 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_clone_copy_on_read": "false", 2024-09-14T22:36:14.010 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_compression_hint": "none", 2024-09-14T22:36:14.010 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_concurrent_management_ops": "10", 2024-09-14T22:36:14.010 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-14T22:36:14.010 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_default_clone_format": "auto", 2024-09-14T22:36:14.010 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_default_data_pool": "", 2024-09-14T22:36:14.010 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_default_features": "61", 2024-09-14T22:36:14.010 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_default_format": "2", 2024-09-14T22:36:14.010 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_default_map_options": "", 2024-09-14T22:36:14.011 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_default_order": "22", 2024-09-14T22:36:14.011 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_default_pool": "rbd", 2024-09-14T22:36:14.011 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-14T22:36:14.011 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_default_stripe_count": "0", 2024-09-14T22:36:14.011 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_default_stripe_unit": "0", 2024-09-14T22:36:14.011 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-14T22:36:14.011 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-14T22:36:14.011 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-14T22:36:14.011 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_enable_alloc_hint": "true", 2024-09-14T22:36:14.011 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-14T22:36:14.011 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_io_scheduler": "simple", 2024-09-14T22:36:14.011 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-14T22:36:14.011 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-14T22:36:14.011 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-14T22:36:14.011 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-14T22:36:14.012 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-14T22:36:14.012 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-14T22:36:14.012 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-14T22:36:14.012 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-14T22:36:14.012 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-14T22:36:14.012 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_journal_order": "24", 2024-09-14T22:36:14.012 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_journal_pool": "", 2024-09-14T22:36:14.012 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_journal_splay_width": "4", 2024-09-14T22:36:14.012 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_localize_parent_reads": "false", 2024-09-14T22:36:14.012 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_localize_snap_reads": "false", 2024-09-14T22:36:14.012 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-14T22:36:14.012 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-14T22:36:14.012 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-14T22:36:14.012 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-14T22:36:14.013 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-14T22:36:14.013 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-14T22:36:14.013 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-14T22:36:14.013 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-14T22:36:14.013 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-14T22:36:14.013 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-14T22:36:14.013 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-14T22:36:14.013 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-14T22:36:14.013 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-14T22:36:14.013 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-14T22:36:14.013 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-14T22:36:14.013 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-14T22:36:14.013 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-14T22:36:14.013 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-14T22:36:14.014 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-14T22:36:14.014 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-14T22:36:14.014 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-14T22:36:14.014 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-14T22:36:14.014 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-14T22:36:14.014 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-14T22:36:14.014 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-14T22:36:14.014 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-14T22:36:14.014 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-14T22:36:14.014 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-14T22:36:14.014 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-14T22:36:14.014 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-14T22:36:14.014 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-14T22:36:14.014 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mtime_update_interval": "60", 2024-09-14T22:36:14.015 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_non_blocking_aio": "true", 2024-09-14T22:36:14.015 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_op_thread_timeout": "60", 2024-09-14T22:36:14.015 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_op_threads": "1", 2024-09-14T22:36:14.015 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_parent_cache_enabled": "false", 2024-09-14T22:36:14.015 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-14T22:36:14.015 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-14T22:36:14.015 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-14T22:36:14.015 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_plugins": "", 2024-09-14T22:36:14.015 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_bps_burst": "0", 2024-09-14T22:36:14.015 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-14T22:36:14.015 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_bps_limit": "0", 2024-09-14T22:36:14.015 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_exclude_ops": "0", 2024-09-14T22:36:14.015 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_iops_burst": "0", 2024-09-14T22:36:14.016 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-14T22:36:14.016 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_iops_limit": "0", 2024-09-14T22:36:14.016 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-14T22:36:14.016 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-14T22:36:14.016 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-14T22:36:14.016 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-14T22:36:14.016 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-14T22:36:14.016 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-14T22:36:14.016 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-14T22:36:14.016 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-14T22:36:14.016 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-14T22:36:14.016 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-14T22:36:14.016 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-14T22:36:14.017 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-14T22:36:14.017 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-14T22:36:14.017 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-14T22:36:14.017 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_read_from_replica_policy": "default", 2024-09-14T22:36:14.017 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-14T22:36:14.017 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-14T22:36:14.017 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-14T22:36:14.017 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-14T22:36:14.017 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_skip_partial_discard": "true", 2024-09-14T22:36:14.017 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-14T22:36:14.017 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_tracing": "false", 2024-09-14T22:36:14.017 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_validate_names": "true", 2024-09-14T22:36:14.017 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_validate_pool": "true", 2024-09-14T22:36:14.017 INFO:teuthology.orchestra.run.smithi001.stdout: "restapi_base_url": "", 2024-09-14T22:36:14.018 INFO:teuthology.orchestra.run.smithi001.stdout: "restapi_log_level": "", 2024-09-14T22:36:14.018 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-14T22:36:14.018 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-14T22:36:14.018 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_acl_grants_max_num": "100", 2024-09-14T22:36:14.018 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_admin_entry": "admin", 2024-09-14T22:36:14.018 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-14T22:36:14.018 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_backend_store": "rados", 2024-09-14T22:36:14.018 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_barbican_url": "", 2024-09-14T22:36:14.018 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_beast_enable_async": "true", 2024-09-14T22:36:14.018 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_bucket_counters_cache": "false", 2024-09-14T22:36:14.018 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-14T22:36:14.018 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-14T22:36:14.018 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-14T22:36:14.018 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-14T22:36:14.019 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-14T22:36:14.019 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-14T22:36:14.019 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-14T22:36:14.019 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-14T22:36:14.019 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_cache_enabled": "true", 2024-09-14T22:36:14.019 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_cache_expiry_interval": "900", 2024-09-14T22:36:14.019 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_cache_lru_size": "25000", 2024-09-14T22:36:14.019 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_config_store": "rados", 2024-09-14T22:36:14.019 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_content_length_compat": "false", 2024-09-14T22:36:14.019 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_copy_obj_progress": "true", 2024-09-14T22:36:14.019 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-14T22:36:14.019 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_cors_rules_max_num": "100", 2024-09-14T22:36:14.019 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_cross_domain_policy": "", 2024-09-14T22:36:14.019 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-14T22:36:14.020 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_kmip_addr": "", 2024-09-14T22:36:14.020 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-14T22:36:14.020 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-14T22:36:14.020 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-14T22:36:14.020 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-14T22:36:14.020 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_kmip_password": "", 2024-09-14T22:36:14.020 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-14T22:36:14.020 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_kmip_username": "", 2024-09-14T22:36:14.020 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_require_ssl": "true", 2024-09-14T22:36:14.020 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-14T22:36:14.020 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-14T22:36:14.020 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-14T22:36:14.020 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-14T22:36:14.020 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-14T22:36:14.020 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-14T22:36:14.021 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-14T22:36:14.021 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-14T22:36:14.021 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-14T22:36:14.021 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-14T22:36:14.021 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-14T22:36:14.021 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-14T22:36:14.021 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-14T22:36:14.021 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-14T22:36:14.021 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-14T22:36:14.021 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_vault_addr": "", 2024-09-14T22:36:14.021 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_vault_auth": "token", 2024-09-14T22:36:14.021 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_vault_namespace": "", 2024-09-14T22:36:14.021 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_vault_prefix": "", 2024-09-14T22:36:14.021 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-14T22:36:14.022 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-14T22:36:14.022 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-14T22:36:14.022 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-14T22:36:14.022 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_vault_token_file": "", 2024-09-14T22:36:14.022 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-14T22:36:14.022 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_curl_buffersize": "524288", 2024-09-14T22:36:14.022 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-14T22:36:14.022 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_curl_low_speed_time": "300", 2024-09-14T22:36:14.022 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-14T22:36:14.022 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-14T22:36:14.022 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-14T22:36:14.022 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-14T22:36:14.022 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-14T22:36:14.022 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-14T22:36:14.023 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-14T22:36:14.023 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-14T22:36:14.023 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-14T22:36:14.023 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-14T22:36:14.023 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_d4n_host": "127.0.0.1", 2024-09-14T22:36:14.023 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_d4n_port": "6379", 2024-09-14T22:36:14.023 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-14T22:36:14.023 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_data_log_changes_size": "1000", 2024-09-14T22:36:14.023 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_data_log_num_shards": "128", 2024-09-14T22:36:14.023 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-14T22:36:14.023 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_data_log_window": "30", 2024-09-14T22:36:14.023 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-14T22:36:14.023 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-14T22:36:14.023 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-14T22:36:14.023 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-14T22:36:14.024 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-14T22:36:14.024 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-14T22:36:14.024 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-14T22:36:14.024 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-14T22:36:14.024 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-14T22:36:14.024 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-14T22:36:14.024 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-14T22:36:14.024 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-14T22:36:14.024 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-14T22:36:14.024 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-14T22:36:14.024 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-14T22:36:14.024 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-14T22:36:14.024 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-14T22:36:14.025 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-14T22:36:14.025 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-14T22:36:14.025 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-14T22:36:14.025 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-14T22:36:14.025 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-14T22:36:14.025 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-14T22:36:14.025 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-14T22:36:14.025 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-14T22:36:14.025 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dns_name": "", 2024-09-14T22:36:14.025 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dns_s3website_name": "", 2024-09-14T22:36:14.025 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dynamic_resharding": "true", 2024-09-14T22:36:14.025 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-14T22:36:14.025 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_enable_gc_threads": "true", 2024-09-14T22:36:14.025 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_enable_lc_threads": "true", 2024-09-14T22:36:14.026 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_enable_ops_log": "false", 2024-09-14T22:36:14.026 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_enable_quota_threads": "true", 2024-09-14T22:36:14.026 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_enable_static_website": "false", 2024-09-14T22:36:14.026 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_enable_usage_log": "false", 2024-09-14T22:36:14.026 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_enforce_swift_acls": "true", 2024-09-14T22:36:14.026 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_exit_timeout_secs": "120", 2024-09-14T22:36:14.026 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_expose_bucket": "false", 2024-09-14T22:36:14.026 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_extended_http_attrs": "", 2024-09-14T22:36:14.026 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_filter": "none", 2024-09-14T22:36:14.026 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-14T22:36:14.026 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_frontends": "beast port=7480", 2024-09-14T22:36:14.026 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-14T22:36:14.026 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_gc_max_deferred": "50", 2024-09-14T22:36:14.026 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-14T22:36:14.027 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_gc_max_objs": "32", 2024-09-14T22:36:14.027 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-14T22:36:14.027 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-14T22:36:14.027 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-14T22:36:14.027 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-14T22:36:14.027 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_gc_processor_period": "3600", 2024-09-14T22:36:14.027 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-14T22:36:14.027 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-14T22:36:14.027 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-14T22:36:14.027 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-14T22:36:14.027 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_init_timeout": "300", 2024-09-14T22:36:14.027 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-14T22:36:14.027 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-14T22:36:14.027 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_kafka_connection_idle": "300", 2024-09-14T22:36:14.028 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-14T22:36:14.028 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-14T22:36:14.028 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-14T22:36:14.028 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-14T22:36:14.028 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-14T22:36:14.028 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_admin_domain": "", 2024-09-14T22:36:14.028 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_admin_password": "", 2024-09-14T22:36:14.028 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_admin_password_path": "", 2024-09-14T22:36:14.028 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_admin_project": "", 2024-09-14T22:36:14.028 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_admin_tenant": "", 2024-09-14T22:36:14.028 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_admin_token": "", 2024-09-14T22:36:14.028 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_admin_token_path": "", 2024-09-14T22:36:14.028 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_admin_user": "", 2024-09-14T22:36:14.028 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_api_version": "2", 2024-09-14T22:36:14.029 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_barbican_domain": "", 2024-09-14T22:36:14.029 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_barbican_password": "", 2024-09-14T22:36:14.029 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_barbican_project": "", 2024-09-14T22:36:14.029 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-14T22:36:14.029 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_barbican_user": "", 2024-09-14T22:36:14.029 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-14T22:36:14.029 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-14T22:36:14.029 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-14T22:36:14.029 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-14T22:36:14.029 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-14T22:36:14.029 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_url": "", 2024-09-14T22:36:14.029 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-14T22:36:14.029 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_lc_debug_interval": "-1", 2024-09-14T22:36:14.030 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_lc_lock_max_time": "90", 2024-09-14T22:36:14.030 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_lc_max_objs": "32", 2024-09-14T22:36:14.030 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_lc_max_rules": "1000", 2024-09-14T22:36:14.030 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_lc_max_worker": "3", 2024-09-14T22:36:14.030 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-14T22:36:14.030 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_lc_thread_delay": "0", 2024-09-14T22:36:14.030 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-14T22:36:14.030 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_ldap_dnattr": "uid", 2024-09-14T22:36:14.030 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-14T22:36:14.030 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_ldap_searchfilter": "", 2024-09-14T22:36:14.030 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-14T22:36:14.030 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_ldap_uri": "", 2024-09-14T22:36:14.030 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-14T22:36:14.030 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-14T22:36:14.030 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-14T22:36:14.031 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_log_http_headers": "", 2024-09-14T22:36:14.031 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-14T22:36:14.031 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-14T22:36:14.031 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_log_object_name_utc": "false", 2024-09-14T22:36:14.031 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-14T22:36:14.031 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-14T22:36:14.031 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_attr_name_len": "0", 2024-09-14T22:36:14.031 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_attr_size": "0", 2024-09-14T22:36:14.031 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-14T22:36:14.031 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_chunk_size": "4194304", 2024-09-14T22:36:14.031 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-14T22:36:14.031 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-14T22:36:14.031 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-14T22:36:14.032 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_listing_results": "1000", 2024-09-14T22:36:14.032 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_notify_retries": "10", 2024-09-14T22:36:14.032 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-14T22:36:14.032 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_put_param_size": "1048576", 2024-09-14T22:36:14.032 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_put_size": "5368709120", 2024-09-14T22:36:14.032 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_slo_entries": "1000", 2024-09-14T22:36:14.032 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_md_log_max_shards": "64", 2024-09-14T22:36:14.032 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-14T22:36:14.032 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-14T22:36:14.032 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-14T22:36:14.032 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-14T22:36:14.032 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_mp_lock_max_time": "600", 2024-09-14T22:36:14.032 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-14T22:36:14.033 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-14T22:36:14.033 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-14T22:36:14.033 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-14T22:36:14.033 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-14T22:36:14.033 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-14T22:36:14.033 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-14T22:36:14.033 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-14T22:36:14.033 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_max_gc": "300", 2024-09-14T22:36:14.033 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-14T22:36:14.033 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-14T22:36:14.033 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-14T22:36:14.033 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-14T22:36:14.033 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-14T22:36:14.033 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-14T22:36:14.033 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-14T22:36:14.034 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_num_async_rados_threads": "32", 2024-09-14T22:36:14.034 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_num_control_oids": "8", 2024-09-14T22:36:14.034 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_numa_node": "-1", 2024-09-14T22:36:14.034 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-14T22:36:14.034 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-14T22:36:14.034 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_objexp_chunk_size": "100", 2024-09-14T22:36:14.034 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_objexp_gc_interval": "600", 2024-09-14T22:36:14.034 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-14T22:36:14.034 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-14T22:36:14.034 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-14T22:36:14.034 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_op_thread_timeout": "600", 2024-09-14T22:36:14.034 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_op_tracing": "false", 2024-09-14T22:36:14.034 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_opa_token": "", 2024-09-14T22:36:14.034 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_opa_url": "", 2024-09-14T22:36:14.035 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_opa_verify_ssl": "true", 2024-09-14T22:36:14.035 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-14T22:36:14.035 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-14T22:36:14.035 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_ops_log_rados": "false", 2024-09-14T22:36:14.035 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_ops_log_socket_path": "", 2024-09-14T22:36:14.035 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-14T22:36:14.035 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-14T22:36:14.035 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-14T22:36:14.035 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_period_push_interval": "2.000000", 2024-09-14T22:36:14.035 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-14T22:36:14.035 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-14T22:36:14.035 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-14T22:36:14.035 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-14T22:36:14.035 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_posix_cache_lanes": "3", 2024-09-14T22:36:14.036 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-14T22:36:14.036 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-14T22:36:14.036 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_posix_cache_partitions": "3", 2024-09-14T22:36:14.036 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-14T22:36:14.036 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_print_continue": "true", 2024-09-14T22:36:14.036 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-14T22:36:14.036 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-14T22:36:14.036 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-14T22:36:14.036 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-14T22:36:14.036 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-14T22:36:14.036 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_rados_tracing": "false", 2024-09-14T22:36:14.036 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_realm": "", 2024-09-14T22:36:14.036 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_realm_id": "", 2024-09-14T22:36:14.037 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-14T22:36:14.037 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_region": "", 2024-09-14T22:36:14.037 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-14T22:36:14.037 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-14T22:36:14.037 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-14T22:36:14.037 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-14T22:36:14.037 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_request_uri": "", 2024-09-14T22:36:14.037 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_reshard_batch_size": "64", 2024-09-14T22:36:14.037 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-14T22:36:14.037 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_reshard_max_aio": "128", 2024-09-14T22:36:14.037 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_reshard_num_logs": "16", 2024-09-14T22:36:14.037 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_reshard_thread_interval": "600", 2024-09-14T22:36:14.037 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_resolve_cname": "false", 2024-09-14T22:36:14.037 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-14T22:36:14.038 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_run_sync_thread": "true", 2024-09-14T22:36:14.038 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-14T22:36:14.038 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-14T22:36:14.038 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-14T22:36:14.038 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-14T22:36:14.038 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-14T22:36:14.038 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-14T22:36:14.038 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-14T22:36:14.038 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-14T22:36:14.038 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-14T22:36:14.038 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_scheduler_type": "throttler", 2024-09-14T22:36:14.038 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_script_uri": "", 2024-09-14T22:36:14.038 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_service_provider_name": "", 2024-09-14T22:36:14.038 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-14T22:36:14.039 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sts_client_id": "", 2024-09-14T22:36:14.039 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sts_client_secret": "", 2024-09-14T22:36:14.039 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sts_entry": "sts", 2024-09-14T22:36:14.039 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sts_key": "sts", 2024-09-14T22:36:14.039 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-14T22:36:14.039 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sts_min_session_duration": "900", 2024-09-14T22:36:14.039 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sts_token_introspection_url": "", 2024-09-14T22:36:14.039 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_swift_account_in_url": "false", 2024-09-14T22:36:14.039 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_swift_auth_entry": "auth", 2024-09-14T22:36:14.039 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_swift_auth_url": "", 2024-09-14T22:36:14.039 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_swift_custom_header": "", 2024-09-14T22:36:14.039 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-14T22:36:14.039 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_swift_need_stats": "true", 2024-09-14T22:36:14.039 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_swift_tenant_name": "", 2024-09-14T22:36:14.040 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_swift_token_expiration": "86400", 2024-09-14T22:36:14.040 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_swift_url": "", 2024-09-14T22:36:14.040 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_swift_url_prefix": "swift", 2024-09-14T22:36:14.040 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-14T22:36:14.040 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-14T22:36:14.040 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-14T22:36:14.040 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_lease_period": "120", 2024-09-14T22:36:14.040 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-14T22:36:14.040 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-14T22:36:14.040 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-14T22:36:14.040 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-14T22:36:14.040 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-14T22:36:14.040 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-14T22:36:14.041 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-14T22:36:14.041 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-14T22:36:14.041 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-14T22:36:14.041 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_thread_pool_size": "512", 2024-09-14T22:36:14.041 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-14T22:36:14.041 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-14T22:36:14.041 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-14T22:36:14.041 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-14T22:36:14.041 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_torrent_comment": "", 2024-09-14T22:36:14.041 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_torrent_createby": "", 2024-09-14T22:36:14.041 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_torrent_encoding": "", 2024-09-14T22:36:14.041 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_torrent_flag": "false", 2024-09-14T22:36:14.041 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-14T22:36:14.041 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_torrent_origin": "", 2024-09-14T22:36:14.041 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-14T22:36:14.042 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_torrent_tracker": "", 2024-09-14T22:36:14.042 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_trust_forwarded_https": "false", 2024-09-14T22:36:14.042 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-14T22:36:14.042 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-14T22:36:14.042 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_usage_max_shards": "32", 2024-09-14T22:36:14.042 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_usage_max_user_shards": "1", 2024-09-14T22:36:14.042 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_use_opa_authz": "false", 2024-09-14T22:36:14.042 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_user_counters_cache": "false", 2024-09-14T22:36:14.042 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-14T22:36:14.042 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-14T22:36:14.042 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-14T22:36:14.042 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_user_max_buckets": "1000", 2024-09-14T22:36:14.042 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_user_policies_max_num": "100", 2024-09-14T22:36:14.042 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-14T22:36:14.043 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-14T22:36:14.043 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-14T22:36:14.043 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-14T22:36:14.043 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_user_unique_email": "true", 2024-09-14T22:36:14.043 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_verify_ssl": "true", 2024-09-14T22:36:14.043 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-14T22:36:14.043 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_zone": "", 2024-09-14T22:36:14.043 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_zone_id": "", 2024-09-14T22:36:14.043 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-14T22:36:14.043 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_zonegroup": "", 2024-09-14T22:36:14.043 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_zonegroup_id": "", 2024-09-14T22:36:14.043 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-14T22:36:14.043 INFO:teuthology.orchestra.run.smithi001.stdout: "rgwlc_auto_session_clear": "true", 2024-09-14T22:36:14.043 INFO:teuthology.orchestra.run.smithi001.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-14T22:36:14.044 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_block_size": "4096", 2024-09-14T22:36:14.044 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-14T22:36:14.044 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-14T22:36:14.044 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-14T22:36:14.044 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-14T22:36:14.044 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-14T22:36:14.044 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_cache_size": "536870912", 2024-09-14T22:36:14.044 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-14T22:36:14.044 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-14T22:36:14.044 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-14T22:36:14.044 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-14T22:36:14.044 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-14T22:36:14.044 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-14T22:36:14.044 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-14T22:36:14.045 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-14T22:36:14.045 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_index_type": "binary_search", 2024-09-14T22:36:14.045 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-14T22:36:14.045 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-14T22:36:14.045 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_partition_filters": "false", 2024-09-14T22:36:14.045 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_perf": "false", 2024-09-14T22:36:14.045 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-14T22:36:14.045 INFO:teuthology.orchestra.run.smithi001.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-14T22:36:14.045 INFO:teuthology.orchestra.run.smithi001.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-14T22:36:14.045 INFO:teuthology.orchestra.run.smithi001.stdout: "run_dir": "/var/run/ceph", 2024-09-14T22:36:14.045 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_block_create": "true", 2024-09-14T22:36:14.045 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_cache_lru_size": "67108864", 2024-09-14T22:36:14.045 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-14T22:36:14.045 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-14T22:36:14.046 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_default_max_object_size": "16777216", 2024-09-14T22:36:14.046 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-14T22:36:14.046 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_device_size": "53687091200", 2024-09-14T22:36:14.046 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-14T22:36:14.046 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_full_integrity_check": "false", 2024-09-14T22:36:14.046 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_journal_batch_capacity": "16", 2024-09-14T22:36:14.046 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-14T22:36:14.046 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-14T22:36:14.046 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-14T22:36:14.046 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_main_device_type": "SSD", 2024-09-14T22:36:14.046 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-14T22:36:14.046 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-14T22:36:14.046 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-14T22:36:14.046 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-14T22:36:14.046 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-14T22:36:14.047 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-14T22:36:14.047 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_segment_size": "67108864", 2024-09-14T22:36:14.047 INFO:teuthology.orchestra.run.smithi001.stdout: "set_keepcaps": "false", 2024-09-14T22:36:14.047 INFO:teuthology.orchestra.run.smithi001.stdout: "setgroup": "", 2024-09-14T22:36:14.047 INFO:teuthology.orchestra.run.smithi001.stdout: "setuser": "", 2024-09-14T22:36:14.047 INFO:teuthology.orchestra.run.smithi001.stdout: "setuser_match_path": "", 2024-09-14T22:36:14.047 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-14T22:36:14.047 INFO:teuthology.orchestra.run.smithi001.stdout: "thp": "false", 2024-09-14T22:36:14.047 INFO:teuthology.orchestra.run.smithi001.stdout: "threadpool_default_timeout": "60", 2024-09-14T22:36:14.047 INFO:teuthology.orchestra.run.smithi001.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-14T22:36:14.047 INFO:teuthology.orchestra.run.smithi001.stdout: "throttler_perf_counter": "true" 2024-09-14T22:36:14.047 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:14.049 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-14T22:36:14.193 INFO:teuthology.orchestra.run.smithi001.stdout:mon.a: {} 2024-09-14T22:36:14.193 INFO:teuthology.orchestra.run.smithi001.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T22:36:14.206 INFO:teuthology.orchestra.run.smithi001.stdout:mon.b: {} 2024-09-14T22:36:14.206 INFO:teuthology.orchestra.run.smithi001.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T22:36:14.216 INFO:teuthology.orchestra.run.smithi001.stdout:mon.c: {} 2024-09-14T22:36:14.216 INFO:teuthology.orchestra.run.smithi001.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T22:36:14.232 DEBUG:teuthology.orchestra.run.smithi176:> type ceph-objectstore-tool 2024-09-14T22:36:14.239 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-14T22:36:14.239 DEBUG:teuthology.orchestra.run.smithi001:> type ceph-objectstore-tool 2024-09-14T22:36:14.243 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-14T22:36:14.243 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-14T22:36:14.252 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-14T22:36:14.253 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-14T22:36:14.253 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T22:36:14.253 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:36:14.253 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-14T22:36:14.253 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 1.000000 2024-09-14T22:36:14.253 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-14T22:36:14.254 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-14T22:36:14.254 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-14T22:36:14.254 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:36:14.254 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-14T22:36:14.254 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:36:14.255 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-14T22:36:14.255 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T22:36:14.255 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-14T22:36:14.283 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:14.279+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:14.317 INFO:teuthology.orchestra.run.smithi001.stdout:mapped writes DISABLED 2024-09-14T22:36:14.317 INFO:teuthology.orchestra.run.smithi001.stdout:mapped reads DISABLED 2024-09-14T22:36:14.317 INFO:teuthology.orchestra.run.smithi001.stdout:Seed set to 6584 2024-09-14T22:36:14.356 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:14.351+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:14.448 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:36:14.448 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:36:14.457 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:14.450+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:14.462 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:14.462 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:14.473 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:36:14.473 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:36:14.476 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:36:14.490 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:36:14.490 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:36:14.522 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:36:14.522 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:36:14.557 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:36:14.557 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:36:14.558 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:14.551+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:14.592 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:36:14.593 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:36:14.601 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:14.601 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:14.614 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:36:14.628 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:36:14.628 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:36:14.658 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:36:14.658 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:36:14.658 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:14.651+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:14.746 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:14.746 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:14.758 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:36:14.759 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:36:14.754+0000 7f82f11cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26333) UID: 0 2024-09-14T22:36:14.860 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:14.854+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:14.921 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:14.922 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:14.935 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:36:14.978 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:14.974+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:15.062 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:15.055+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:15.092 INFO:teuthology.orchestra.run.smithi001.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-14T22:36:15.092 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2024-09-14T22:36:15.104 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:15.104 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:15.120 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:36:15.163 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:15.159+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:15.265 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:15.259+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:15.282 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:15.283 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:15.297 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:36:15.365 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:15.358+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:15.418 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:15.418 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:15.433 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:36:15.474 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:15.470+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:15.567 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:15.562+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:15.600 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:15.601 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:15.616 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:36:15.668 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:15.663+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:15.770 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:15.763+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:15.783 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:15.783 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:15.797 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:36:15.870 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:36:15.866+0000 7f82f11cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26333) UID: 0 2024-09-14T22:36:15.923 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:15.923 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:15.937 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:36:15.972 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:15.967+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:16.073 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:16.067+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:16.106 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:16.106 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:16.121 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:36:16.173 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:16.167+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:16.248 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:16.248 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:16.263 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:36:16.311 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:16.307+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:16.374 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:16.367+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:16.393 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:16.393 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:16.407 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:36:16.495 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:16.491+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:16.576 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:36:16.574+0000 7f82f11cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26333) UID: 0 2024-09-14T22:36:16.576 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:16.577 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:16.590 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:36:16.675 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:36:16.679 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:16.675+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:16.755 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:16.755 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:16.769 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:36:16.777 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:16.770+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:16.835 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:36:16.835 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:36:16.852 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:36:16.852 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:36:16.869 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:36:16.869 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:36:16.878 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:16.871+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:16.886 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:36:16.886 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:36:16.905 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:36:16.906 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:36:16.926 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:36:16.926 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:36:16.941 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:16.942 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:16.942 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:36:16.942 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:16.948 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:36:16.948 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:36:16.956 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:36:16.968 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:36:16.968 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:36:16.980 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:16.974+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:17.081 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:17.074+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:17.117 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T22:36:17.122 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:17.122 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:17.122 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:36:17.123 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:36:17.123 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:17.136 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:36:17.183 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:17.179+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:17.256 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:17.256 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:36:17.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:36:17.256 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:36:17.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.256 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4235.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T22:36:17.256 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:06.035535+0000", 2024-09-14T22:36:17.257 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 11.216167670000001, 2024-09-14T22:36:17.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.007582231, 2024-09-14T22:36:17.257 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:17.257 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:17.257 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:17.257 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:17.257 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4235", 2024-09-14T22:36:17.257 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4072861539", 2024-09-14T22:36:17.257 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 1 2024-09-14T22:36:17.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.257 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:17.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:17.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.035535+0000", 2024-09-14T22:36:17.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:17.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:17.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.035535+0000", 2024-09-14T22:36:17.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:17.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:17.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.035544+0000", 2024-09-14T22:36:17.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4800000000000001e-06 2024-09-14T22:36:17.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:17.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.035558+0000", 2024-09-14T22:36:17.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.393e-05 2024-09-14T22:36:17.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:17.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.035588+0000", 2024-09-14T22:36:17.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9913e-05 2024-09-14T22:36:17.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:17.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.035631+0000", 2024-09-14T22:36:17.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3205999999999999e-05 2024-09-14T22:36:17.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:17.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.036779+0000", 2024-09-14T22:36:17.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001148165 2024-09-14T22:36:17.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:36:17.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.036813+0000", 2024-09-14T22:36:17.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3510000000000003e-05 2024-09-14T22:36:17.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:17.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.039306+0000", 2024-09-14T22:36:17.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0024931770000000001 2024-09-14T22:36:17.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:17.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.039719+0000", 2024-09-14T22:36:17.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00041307699999999998 2024-09-14T22:36:17.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 2", 2024-09-14T22:36:17.261 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.040064+0000", 2024-09-14T22:36:17.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00034538400000000001 2024-09-14T22:36:17.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:17.261 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.041904+0000", 2024-09-14T22:36:17.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001839949 2024-09-14T22:36:17.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:17.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.042895+0000", 2024-09-14T22:36:17.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000990606 2024-09-14T22:36:17.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:17.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.043021+0000", 2024-09-14T22:36:17.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012573900000000001 2024-09-14T22:36:17.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:17.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.043118+0000", 2024-09-14T22:36:17.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7095000000000005e-05 2024-09-14T22:36:17.262 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:17.263 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:17.263 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:17.263 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.263 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.263 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4235.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-14T22:36:17.263 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:06.042625+0000", 2024-09-14T22:36:17.263 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 11.209077754999999, 2024-09-14T22:36:17.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00050796299999999995, 2024-09-14T22:36:17.263 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:17.263 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:17.263 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:17.263 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:17.263 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:17.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.042625+0000", 2024-09-14T22:36:17.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:17.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:17.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.042625+0000", 2024-09-14T22:36:17.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:17.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:17.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.042627+0000", 2024-09-14T22:36:17.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9069999999999999e-06 2024-09-14T22:36:17.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:17.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.042640+0000", 2024-09-14T22:36:17.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.312e-05 2024-09-14T22:36:17.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:17.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.042672+0000", 2024-09-14T22:36:17.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1427000000000003e-05 2024-09-14T22:36:17.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:17.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.042693+0000", 2024-09-14T22:36:17.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1305999999999999e-05 2024-09-14T22:36:17.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:17.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.042849+0000", 2024-09-14T22:36:17.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000156012 2024-09-14T22:36:17.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:17.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.042870+0000", 2024-09-14T22:36:17.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0706000000000001e-05 2024-09-14T22:36:17.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:17.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.043133+0000", 2024-09-14T22:36:17.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026348499999999999 2024-09-14T22:36:17.267 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:17.267 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:17.267 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:17.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.267 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4235.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-14T22:36:17.267 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:06.047757+0000", 2024-09-14T22:36:17.267 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 11.203945974, 2024-09-14T22:36:17.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00042420000000000001, 2024-09-14T22:36:17.267 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:17.267 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:17.267 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T22:36:17.267 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:17.268 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4235", 2024-09-14T22:36:17.268 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4072861539", 2024-09-14T22:36:17.268 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2 2024-09-14T22:36:17.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.268 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:17.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.268 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:17.268 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.047757+0000", 2024-09-14T22:36:17.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:17.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.268 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:17.268 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.047757+0000", 2024-09-14T22:36:17.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:17.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:17.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.047760+0000", 2024-09-14T22:36:17.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3300000000000001e-06 2024-09-14T22:36:17.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:17.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.047772+0000", 2024-09-14T22:36:17.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2543e-05 2024-09-14T22:36:17.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:17.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.047804+0000", 2024-09-14T22:36:17.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1860000000000003e-05 2024-09-14T22:36:17.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:17.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.047831+0000", 2024-09-14T22:36:17.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7236000000000001e-05 2024-09-14T22:36:17.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:17.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.047975+0000", 2024-09-14T22:36:17.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014358900000000001 2024-09-14T22:36:17.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:17.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.048181+0000", 2024-09-14T22:36:17.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020664199999999999 2024-09-14T22:36:17.271 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:17.271 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:17.271 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:17.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.271 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4235.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-14T22:36:17.271 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:06.048586+0000", 2024-09-14T22:36:17.271 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 11.203116801, 2024-09-14T22:36:17.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035711500000000003, 2024-09-14T22:36:17.271 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:17.271 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:17.271 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T22:36:17.271 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:17.271 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4235", 2024-09-14T22:36:17.272 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4072861539", 2024-09-14T22:36:17.272 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 3 2024-09-14T22:36:17.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.272 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:17.272 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.272 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:17.272 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.048586+0000", 2024-09-14T22:36:17.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:17.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.272 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.272 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:17.272 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.048586+0000", 2024-09-14T22:36:17.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:17.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:17.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.048589+0000", 2024-09-14T22:36:17.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1639999999999999e-06 2024-09-14T22:36:17.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:17.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.048599+0000", 2024-09-14T22:36:17.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.065e-05 2024-09-14T22:36:17.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:17.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.048631+0000", 2024-09-14T22:36:17.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1773000000000002e-05 2024-09-14T22:36:17.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:17.274 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.048652+0000", 2024-09-14T22:36:17.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1376999999999999e-05 2024-09-14T22:36:17.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:17.274 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.048755+0000", 2024-09-14T22:36:17.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102425 2024-09-14T22:36:17.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:17.274 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:06.048943+0000", 2024-09-14T22:36:17.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018872600000000001 2024-09-14T22:36:17.274 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:17.275 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:17.275 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:17.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.275 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4235.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-14T22:36:17.275 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:07.043076+0000", 2024-09-14T22:36:17.275 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.208627487999999, 2024-09-14T22:36:17.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0063476080000000002, 2024-09-14T22:36:17.275 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:17.275 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:17.275 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:17.275 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:17.275 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4235", 2024-09-14T22:36:17.275 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4072861539", 2024-09-14T22:36:17.276 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 5 2024-09-14T22:36:17.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.276 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:17.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:17.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:07.043076+0000", 2024-09-14T22:36:17.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:17.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:17.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:07.043076+0000", 2024-09-14T22:36:17.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:17.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:17.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:07.043079+0000", 2024-09-14T22:36:17.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4070000000000002e-06 2024-09-14T22:36:17.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:17.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:07.043095+0000", 2024-09-14T22:36:17.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5763000000000001e-05 2024-09-14T22:36:17.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:17.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:07.043129+0000", 2024-09-14T22:36:17.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4233000000000002e-05 2024-09-14T22:36:17.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:17.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:07.043155+0000", 2024-09-14T22:36:17.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5503999999999998e-05 2024-09-14T22:36:17.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:17.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:07.043301+0000", 2024-09-14T22:36:17.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014679800000000001 2024-09-14T22:36:17.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op must wait for map", 2024-09-14T22:36:17.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:07.043312+0000", 2024-09-14T22:36:17.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0943e-05 2024-09-14T22:36:17.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:17.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:07.046497+0000", 2024-09-14T22:36:17.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0031841859999999999 2024-09-14T22:36:17.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:17.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:07.046715+0000", 2024-09-14T22:36:17.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000218612 2024-09-14T22:36:17.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T22:36:17.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:07.046948+0000", 2024-09-14T22:36:17.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000233048 2024-09-14T22:36:17.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.280 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:17.280 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:07.048264+0000", 2024-09-14T22:36:17.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0013159599999999999 2024-09-14T22:36:17.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.280 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:17.280 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:07.049285+0000", 2024-09-14T22:36:17.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0010208089999999999 2024-09-14T22:36:17.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.280 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:17.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:07.049365+0000", 2024-09-14T22:36:17.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0506000000000005e-05 2024-09-14T22:36:17.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:17.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:07.049423+0000", 2024-09-14T22:36:17.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7839e-05 2024-09-14T22:36:17.281 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:17.281 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:17.281 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:17.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.281 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4235.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-14T22:36:17.282 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:07.049120+0000", 2024-09-14T22:36:17.282 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.202583594, 2024-09-14T22:36:17.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00031422399999999999, 2024-09-14T22:36:17.282 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:17.282 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:17.282 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:17.282 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:17.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:17.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:07.049120+0000", 2024-09-14T22:36:17.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:17.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:17.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:07.049120+0000", 2024-09-14T22:36:17.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:17.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:17.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:07.049121+0000", 2024-09-14T22:36:17.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.75e-06 2024-09-14T22:36:17.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:17.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:07.049133+0000", 2024-09-14T22:36:17.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1633e-05 2024-09-14T22:36:17.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:17.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:07.049158+0000", 2024-09-14T22:36:17.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4913e-05 2024-09-14T22:36:17.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:17.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:07.049178+0000", 2024-09-14T22:36:17.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9763e-05 2024-09-14T22:36:17.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:17.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:07.049265+0000", 2024-09-14T22:36:17.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7470000000000001e-05 2024-09-14T22:36:17.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.285 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:17.285 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:07.049277+0000", 2024-09-14T22:36:17.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.172e-05 2024-09-14T22:36:17.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.285 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:17.285 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:07.049434+0000", 2024-09-14T22:36:17.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015697500000000001 2024-09-14T22:36:17.285 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:17.285 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:17.285 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:17.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.286 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4235.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-14T22:36:17.286 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:08.045646+0000", 2024-09-14T22:36:17.286 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.2060568160000003, 2024-09-14T22:36:17.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012159399, 2024-09-14T22:36:17.286 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:17.286 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:17.286 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:17.286 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:17.286 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4235", 2024-09-14T22:36:17.286 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4072861539", 2024-09-14T22:36:17.286 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 7 2024-09-14T22:36:17.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.287 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:17.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.287 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:17.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:08.045646+0000", 2024-09-14T22:36:17.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:17.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.287 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:17.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:08.045646+0000", 2024-09-14T22:36:17.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:17.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:17.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:08.045649+0000", 2024-09-14T22:36:17.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0029999999999999e-06 2024-09-14T22:36:17.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:17.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:08.045680+0000", 2024-09-14T22:36:17.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0496000000000001e-05 2024-09-14T22:36:17.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:17.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:08.045712+0000", 2024-09-14T22:36:17.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1919999999999999e-05 2024-09-14T22:36:17.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:17.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:08.045730+0000", 2024-09-14T22:36:17.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7770000000000001e-05 2024-09-14T22:36:17.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:17.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:08.045896+0000", 2024-09-14T22:36:17.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016617499999999999 2024-09-14T22:36:17.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op must wait for map", 2024-09-14T22:36:17.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:08.045907+0000", 2024-09-14T22:36:17.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.133e-05 2024-09-14T22:36:17.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:17.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:08.054288+0000", 2024-09-14T22:36:17.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0083806260000000004 2024-09-14T22:36:17.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:17.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:08.054461+0000", 2024-09-14T22:36:17.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017358900000000001 2024-09-14T22:36:17.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T22:36:17.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:08.054660+0000", 2024-09-14T22:36:17.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000199104 2024-09-14T22:36:17.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:17.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:08.056282+0000", 2024-09-14T22:36:17.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001622002 2024-09-14T22:36:17.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:17.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:08.057651+0000", 2024-09-14T22:36:17.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001368232 2024-09-14T22:36:17.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:17.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:08.057736+0000", 2024-09-14T22:36:17.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5179000000000006e-05 2024-09-14T22:36:17.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:17.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:08.057806+0000", 2024-09-14T22:36:17.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9973000000000002e-05 2024-09-14T22:36:17.292 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:17.292 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:17.293 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:17.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.293 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4235.0:7 2.2 e22/19 ondisk, result = 0)", 2024-09-14T22:36:17.293 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:08.057471+0000", 2024-09-14T22:36:17.293 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.1942323540000004, 2024-09-14T22:36:17.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00034714099999999998, 2024-09-14T22:36:17.293 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:17.293 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:17.293 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:17.293 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:17.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:17.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:08.057471+0000", 2024-09-14T22:36:17.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:17.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:17.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:08.057471+0000", 2024-09-14T22:36:17.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:17.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:17.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:08.057477+0000", 2024-09-14T22:36:17.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5899999999999996e-06 2024-09-14T22:36:17.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:17.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:08.057487+0000", 2024-09-14T22:36:17.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3389999999999999e-06 2024-09-14T22:36:17.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:17.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:08.057515+0000", 2024-09-14T22:36:17.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7954e-05 2024-09-14T22:36:17.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:17.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:08.057534+0000", 2024-09-14T22:36:17.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9720000000000001e-05 2024-09-14T22:36:17.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:17.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:08.057628+0000", 2024-09-14T22:36:17.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4055000000000007e-05 2024-09-14T22:36:17.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:17.296 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:08.057642+0000", 2024-09-14T22:36:17.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3267e-05 2024-09-14T22:36:17.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:17.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:17.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:17.296 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:08.057818+0000", 2024-09-14T22:36:17.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017621600000000001 2024-09-14T22:36:17.296 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:17.296 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:17.296 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:17.296 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:17.297 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:17.297 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:17.298 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:36:17.298 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:17.290+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:17.399 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:17.391+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:17.461 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:17.461 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:17.461 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:36:17.461 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:17.475 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:36:17.519 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:17.515+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:17.601 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:36:17.594+0000 7f82f11cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26333) UID: 0 2024-09-14T22:36:17.642 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:17.642 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:17.642 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:36:17.642 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:36:17.642 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:17.656 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:36:17.702 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:17.694+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:17.708 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2024-09-14T22:36:17.777 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:17.777 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:36:17.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:36:17.777 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [] 2024-09-14T22:36:17.777 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:17.792 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:36:17.839 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:17.835+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:17.904 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:17.899+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:17.915 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:17.915 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:17.915 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:36:17.915 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:17.929 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:36:18.015 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:18.011+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:18.091 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:18.091 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:18.091 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:36:18.091 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:36:18.091 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:18.105 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:36:18.106 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:36:18.098+0000 7f82f11cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26333) UID: 0 2024-09-14T22:36:18.206 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:18.199+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:18.281 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:18.281 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:36:18.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:36:18.281 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:36:18.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.281 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:36:18.282 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:36:18.282 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.26560448, 2024-09-14T22:36:18.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.398029152, 2024-09-14T22:36:18.282 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:18.282 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:18.282 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.282 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:18.282 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:36:18.282 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:36:18.282 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 1 2024-09-14T22:36:18.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.282 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:18.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:18.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:36:18.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:18.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:36:18.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:18.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006183+0000", 2024-09-14T22:36:18.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9059999999999998e-06 2024-09-14T22:36:18.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:18.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006198+0000", 2024-09-14T22:36:18.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.523e-05 2024-09-14T22:36:18.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:18.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006219+0000", 2024-09-14T22:36:18.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1342999999999999e-05 2024-09-14T22:36:18.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.285 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.285 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006240+0000", 2024-09-14T22:36:18.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1186999999999999e-05 2024-09-14T22:36:18.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.285 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:18.285 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006363+0000", 2024-09-14T22:36:18.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012258200000000001 2024-09-14T22:36:18.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.285 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:36:18.285 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006392+0000", 2024-09-14T22:36:18.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9607e-05 2024-09-14T22:36:18.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:18.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401084+0000", 2024-09-14T22:36:18.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.39469142699999998 2024-09-14T22:36:18.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:18.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401394+0000", 2024-09-14T22:36:18.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030987799999999998 2024-09-14T22:36:18.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:18.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401860+0000", 2024-09-14T22:36:18.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00046654299999999998 2024-09-14T22:36:18.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.287 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:18.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.403386+0000", 2024-09-14T22:36:18.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001525322 2024-09-14T22:36:18.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.287 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:18.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404042+0000", 2024-09-14T22:36:18.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065676099999999996 2024-09-14T22:36:18.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:18.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404136+0000", 2024-09-14T22:36:18.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4032999999999999e-05 2024-09-14T22:36:18.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:18.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404209+0000", 2024-09-14T22:36:18.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2333e-05 2024-09-14T22:36:18.288 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.288 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:18.288 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.289 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:36:18.289 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.460632+0000", 2024-09-14T22:36:18.289 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 17.811151901999999, 2024-09-14T22:36:18.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011006678000000001, 2024-09-14T22:36:18.289 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:18.289 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:18.289 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.289 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:18.289 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:36:18.289 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:36:18.289 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 34 2024-09-14T22:36:18.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.290 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:18.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:18.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.460632+0000", 2024-09-14T22:36:18.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:18.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.460632+0000", 2024-09-14T22:36:18.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:18.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.460633+0000", 2024-09-14T22:36:18.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2699999999999999e-06 2024-09-14T22:36:18.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:18.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.460644+0000", 2024-09-14T22:36:18.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0227000000000001e-05 2024-09-14T22:36:18.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:18.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.460662+0000", 2024-09-14T22:36:18.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8773000000000001e-05 2024-09-14T22:36:18.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.460676+0000", 2024-09-14T22:36:18.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3978999999999999e-05 2024-09-14T22:36:18.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:18.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.466013+0000", 2024-09-14T22:36:18.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00533607 2024-09-14T22:36:18.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:18.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.466165+0000", 2024-09-14T22:36:18.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000152499 2024-09-14T22:36:18.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:18.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.466364+0000", 2024-09-14T22:36:18.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000198681 2024-09-14T22:36:18.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:18.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.471349+0000", 2024-09-14T22:36:18.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0049848410000000003 2024-09-14T22:36:18.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:18.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.471513+0000", 2024-09-14T22:36:18.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016470300000000001 2024-09-14T22:36:18.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:18.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.471573+0000", 2024-09-14T22:36:18.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9376000000000002e-05 2024-09-14T22:36:18.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:18.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.471639+0000", 2024-09-14T22:36:18.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6259000000000004e-05 2024-09-14T22:36:18.294 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.294 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:18.295 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.295 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:36:18.295 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.460698+0000", 2024-09-14T22:36:18.295 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 17.811086003, 2024-09-14T22:36:18.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011240623999999999, 2024-09-14T22:36:18.295 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:18.295 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:18.295 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.295 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:18.295 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:36:18.295 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:36:18.296 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 35 2024-09-14T22:36:18.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.296 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:18.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:18.296 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.460698+0000", 2024-09-14T22:36:18.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:18.296 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.460698+0000", 2024-09-14T22:36:18.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:18.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.460699+0000", 2024-09-14T22:36:18.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.263e-06 2024-09-14T22:36:18.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:18.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.460928+0000", 2024-09-14T22:36:18.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228495 2024-09-14T22:36:18.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:18.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.460951+0000", 2024-09-14T22:36:18.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2929999999999999e-05 2024-09-14T22:36:18.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.460970+0000", 2024-09-14T22:36:18.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8723e-05 2024-09-14T22:36:18.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:18.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.467280+0000", 2024-09-14T22:36:18.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0063105820000000003 2024-09-14T22:36:18.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:18.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.467408+0000", 2024-09-14T22:36:18.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012784299999999999 2024-09-14T22:36:18.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.299 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:18.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.467616+0000", 2024-09-14T22:36:18.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020798799999999999 2024-09-14T22:36:18.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.299 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:18.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.471661+0000", 2024-09-14T22:36:18.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0040454089999999998 2024-09-14T22:36:18.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.300 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:18.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.471794+0000", 2024-09-14T22:36:18.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000132349 2024-09-14T22:36:18.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.300 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:18.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.471877+0000", 2024-09-14T22:36:18.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3372000000000001e-05 2024-09-14T22:36:18.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.300 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:18.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.471939+0000", 2024-09-14T22:36:18.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1669999999999997e-05 2024-09-14T22:36:18.300 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.301 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:18.301 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.301 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:36:18.301 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.461060+0000", 2024-09-14T22:36:18.301 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 17.810723891999999, 2024-09-14T22:36:18.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011091278, 2024-09-14T22:36:18.301 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:18.301 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:18.301 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.301 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:18.301 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:36:18.302 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:36:18.302 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 36 2024-09-14T22:36:18.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.302 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:18.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:18.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.461060+0000", 2024-09-14T22:36:18.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:18.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.461060+0000", 2024-09-14T22:36:18.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.303 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:18.303 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.461062+0000", 2024-09-14T22:36:18.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8700000000000001e-06 2024-09-14T22:36:18.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.303 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:18.303 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.461089+0000", 2024-09-14T22:36:18.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6923e-05 2024-09-14T22:36:18.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.303 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:18.303 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.461110+0000", 2024-09-14T22:36:18.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.103e-05 2024-09-14T22:36:18.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.304 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.461126+0000", 2024-09-14T22:36:18.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5693e-05 2024-09-14T22:36:18.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:18.304 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.469084+0000", 2024-09-14T22:36:18.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0079581419999999996 2024-09-14T22:36:18.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:18.304 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.469271+0000", 2024-09-14T22:36:18.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018663499999999999 2024-09-14T22:36:18.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:18.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.469486+0000", 2024-09-14T22:36:18.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000215262 2024-09-14T22:36:18.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:18.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.472003+0000", 2024-09-14T22:36:18.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002517334 2024-09-14T22:36:18.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:18.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.472026+0000", 2024-09-14T22:36:18.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3336000000000002e-05 2024-09-14T22:36:18.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:18.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.472097+0000", 2024-09-14T22:36:18.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0660000000000004e-05 2024-09-14T22:36:18.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:18.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.472152+0000", 2024-09-14T22:36:18.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4392999999999999e-05 2024-09-14T22:36:18.307 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.307 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:18.307 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.307 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:36:18.307 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.461152+0000", 2024-09-14T22:36:18.307 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 17.810632126000002, 2024-09-14T22:36:18.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011740629000000001, 2024-09-14T22:36:18.307 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:18.307 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:18.307 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.307 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:18.307 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:36:18.308 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:36:18.308 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 37 2024-09-14T22:36:18.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.308 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:18.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.308 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:18.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.461152+0000", 2024-09-14T22:36:18.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.308 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:18.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.461152+0000", 2024-09-14T22:36:18.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:18.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.461153+0000", 2024-09-14T22:36:18.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4929999999999999e-06 2024-09-14T22:36:18.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:18.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.461165+0000", 2024-09-14T22:36:18.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1399999999999999e-05 2024-09-14T22:36:18.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:18.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.461186+0000", 2024-09-14T22:36:18.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0686000000000002e-05 2024-09-14T22:36:18.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.461200+0000", 2024-09-14T22:36:18.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4630000000000001e-05 2024-09-14T22:36:18.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:18.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.470222+0000", 2024-09-14T22:36:18.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0090216040000000008 2024-09-14T22:36:18.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:18.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.470460+0000", 2024-09-14T22:36:18.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00023860799999999999 2024-09-14T22:36:18.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:18.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.470828+0000", 2024-09-14T22:36:18.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036771699999999999 2024-09-14T22:36:18.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:18.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.472164+0000", 2024-09-14T22:36:18.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0013361200000000001 2024-09-14T22:36:18.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.312 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:18.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.472771+0000", 2024-09-14T22:36:18.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060630199999999997 2024-09-14T22:36:18.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.312 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:18.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.472841+0000", 2024-09-14T22:36:18.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0162999999999999e-05 2024-09-14T22:36:18.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:18.313 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.472893+0000", 2024-09-14T22:36:18.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1906000000000001e-05 2024-09-14T22:36:18.313 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.313 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:18.313 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.313 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:36:18.313 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.737928+0000", 2024-09-14T22:36:18.313 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 17.533855991999999, 2024-09-14T22:36:18.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011824962, 2024-09-14T22:36:18.314 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:18.314 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:18.314 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.314 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:18.314 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:36:18.314 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:36:18.314 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 109 2024-09-14T22:36:18.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.314 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:18.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:18.314 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.737928+0000", 2024-09-14T22:36:18.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:18.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.737928+0000", 2024-09-14T22:36:18.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:18.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.737930+0000", 2024-09-14T22:36:18.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6670000000000001e-06 2024-09-14T22:36:18.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:18.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738258+0000", 2024-09-14T22:36:18.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00032793399999999999 2024-09-14T22:36:18.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:18.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738287+0000", 2024-09-14T22:36:18.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9099e-05 2024-09-14T22:36:18.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738306+0000", 2024-09-14T22:36:18.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9137e-05 2024-09-14T22:36:18.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:18.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.744741+0000", 2024-09-14T22:36:18.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0064347680000000004 2024-09-14T22:36:18.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:18.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.744884+0000", 2024-09-14T22:36:18.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014345199999999999 2024-09-14T22:36:18.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:18.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.745082+0000", 2024-09-14T22:36:18.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000198001 2024-09-14T22:36:18.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:18.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.749461+0000", 2024-09-14T22:36:18.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0043784200000000001 2024-09-14T22:36:18.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:18.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.749503+0000", 2024-09-14T22:36:18.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2456000000000001e-05 2024-09-14T22:36:18.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:18.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.749643+0000", 2024-09-14T22:36:18.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013962299999999999 2024-09-14T22:36:18.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:18.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.749753+0000", 2024-09-14T22:36:18.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110405 2024-09-14T22:36:18.319 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.319 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:18.319 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.319 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:36:18.319 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.738342+0000", 2024-09-14T22:36:18.320 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 17.533441627999999, 2024-09-14T22:36:18.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013481445999999999, 2024-09-14T22:36:18.320 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:18.320 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:18.320 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.320 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:18.320 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:36:18.320 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:36:18.320 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 110 2024-09-14T22:36:18.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.320 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:18.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:18.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738342+0000", 2024-09-14T22:36:18.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:18.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738342+0000", 2024-09-14T22:36:18.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:18.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738344+0000", 2024-09-14T22:36:18.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6899999999999999e-06 2024-09-14T22:36:18.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:18.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738356+0000", 2024-09-14T22:36:18.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1749999999999999e-05 2024-09-14T22:36:18.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:18.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738377+0000", 2024-09-14T22:36:18.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1152999999999999e-05 2024-09-14T22:36:18.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738392+0000", 2024-09-14T22:36:18.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4555999999999999e-05 2024-09-14T22:36:18.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:18.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.746751+0000", 2024-09-14T22:36:18.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0083588619999999999 2024-09-14T22:36:18.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:18.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.746975+0000", 2024-09-14T22:36:18.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000224985 2024-09-14T22:36:18.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:18.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.747274+0000", 2024-09-14T22:36:18.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029865499999999999 2024-09-14T22:36:18.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:18.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.749883+0000", 2024-09-14T22:36:18.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002608366 2024-09-14T22:36:18.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:18.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.751530+0000", 2024-09-14T22:36:18.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001647744 2024-09-14T22:36:18.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:18.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.751680+0000", 2024-09-14T22:36:18.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014964899999999999 2024-09-14T22:36:18.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:18.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.751824+0000", 2024-09-14T22:36:18.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000144036 2024-09-14T22:36:18.325 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.325 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:18.325 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.326 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:36:18.326 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.738420+0000", 2024-09-14T22:36:18.326 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 17.533363825999999, 2024-09-14T22:36:18.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014877125999999999, 2024-09-14T22:36:18.326 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:18.326 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:18.326 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.326 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:18.326 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:36:18.326 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:36:18.326 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 111 2024-09-14T22:36:18.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.327 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:18.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:18.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738420+0000", 2024-09-14T22:36:18.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:18.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738420+0000", 2024-09-14T22:36:18.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:18.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738421+0000", 2024-09-14T22:36:18.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0270000000000001e-06 2024-09-14T22:36:18.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:18.328 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738429+0000", 2024-09-14T22:36:18.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3329999999999999e-06 2024-09-14T22:36:18.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:18.328 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738445+0000", 2024-09-14T22:36:18.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6167e-05 2024-09-14T22:36:18.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.328 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738454+0000", 2024-09-14T22:36:18.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6430000000000007e-06 2024-09-14T22:36:18.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:18.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.748449+0000", 2024-09-14T22:36:18.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0099941500000000003 2024-09-14T22:36:18.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:18.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.748586+0000", 2024-09-14T22:36:18.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000137262 2024-09-14T22:36:18.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:18.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.748785+0000", 2024-09-14T22:36:18.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019905500000000001 2024-09-14T22:36:18.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:18.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.753020+0000", 2024-09-14T22:36:18.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0042351180000000004 2024-09-14T22:36:18.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:18.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.753058+0000", 2024-09-14T22:36:18.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8309999999999997e-05 2024-09-14T22:36:18.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:18.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.753203+0000", 2024-09-14T22:36:18.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000144802 2024-09-14T22:36:18.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:18.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.753297+0000", 2024-09-14T22:36:18.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4259000000000007e-05 2024-09-14T22:36:18.331 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.331 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:18.331 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.332 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:36:18.332 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.738469+0000", 2024-09-14T22:36:18.332 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 17.533315293000001, 2024-09-14T22:36:18.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.016596349, 2024-09-14T22:36:18.332 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:18.332 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:18.332 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.332 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:18.332 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:36:18.332 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:36:18.332 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112 2024-09-14T22:36:18.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.332 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:18.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:18.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738469+0000", 2024-09-14T22:36:18.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:18.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738469+0000", 2024-09-14T22:36:18.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:18.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738470+0000", 2024-09-14T22:36:18.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1999999999999999e-07 2024-09-14T22:36:18.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:18.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738618+0000", 2024-09-14T22:36:18.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014805600000000001 2024-09-14T22:36:18.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:18.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738632+0000", 2024-09-14T22:36:18.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4276e-05 2024-09-14T22:36:18.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738644+0000", 2024-09-14T22:36:18.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.185e-05 2024-09-14T22:36:18.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:18.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.750006+0000", 2024-09-14T22:36:18.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011362179 2024-09-14T22:36:18.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:18.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.750230+0000", 2024-09-14T22:36:18.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000224405 2024-09-14T22:36:18.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:18.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.750535+0000", 2024-09-14T22:36:18.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030511500000000001 2024-09-14T22:36:18.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:18.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.753432+0000", 2024-09-14T22:36:18.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0028961439999999998 2024-09-14T22:36:18.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:18.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.754795+0000", 2024-09-14T22:36:18.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0013635730000000001 2024-09-14T22:36:18.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:18.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.754959+0000", 2024-09-14T22:36:18.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016403500000000001 2024-09-14T22:36:18.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:18.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.755065+0000", 2024-09-14T22:36:18.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010599600000000001 2024-09-14T22:36:18.337 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.337 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:18.337 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.337 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:36:18.337 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.738662+0000", 2024-09-14T22:36:18.337 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 17.533122288000001, 2024-09-14T22:36:18.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018194993999999999, 2024-09-14T22:36:18.338 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:18.338 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:18.338 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.338 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:18.338 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:36:18.338 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:36:18.338 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 113 2024-09-14T22:36:18.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.338 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:18.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:18.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738662+0000", 2024-09-14T22:36:18.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:18.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738662+0000", 2024-09-14T22:36:18.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:18.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738663+0000", 2024-09-14T22:36:18.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0300000000000001e-06 2024-09-14T22:36:18.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:18.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738669+0000", 2024-09-14T22:36:18.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.037e-06 2024-09-14T22:36:18.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:18.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738683+0000", 2024-09-14T22:36:18.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4012999999999999e-05 2024-09-14T22:36:18.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738691+0000", 2024-09-14T22:36:18.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9100000000000005e-06 2024-09-14T22:36:18.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:18.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.751940+0000", 2024-09-14T22:36:18.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013248898 2024-09-14T22:36:18.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:18.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.752098+0000", 2024-09-14T22:36:18.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015850200000000001 2024-09-14T22:36:18.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:18.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.752304+0000", 2024-09-14T22:36:18.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020529500000000001 2024-09-14T22:36:18.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:18.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.756560+0000", 2024-09-14T22:36:18.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0042563410000000003 2024-09-14T22:36:18.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:18.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.756605+0000", 2024-09-14T22:36:18.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4863000000000001e-05 2024-09-14T22:36:18.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:18.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.756747+0000", 2024-09-14T22:36:18.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014277199999999999 2024-09-14T22:36:18.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:18.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.756857+0000", 2024-09-14T22:36:18.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010933300000000001 2024-09-14T22:36:18.344 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.344 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:18.344 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.344 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:36:18.344 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.738705+0000", 2024-09-14T22:36:18.344 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 17.533079284999999, 2024-09-14T22:36:18.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.019871403999999999, 2024-09-14T22:36:18.344 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:18.344 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:18.345 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.345 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:18.345 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:36:18.345 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:36:18.345 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 114 2024-09-14T22:36:18.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.345 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:18.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:18.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738705+0000", 2024-09-14T22:36:18.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:18.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738705+0000", 2024-09-14T22:36:18.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:18.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738706+0000", 2024-09-14T22:36:18.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3e-07 2024-09-14T22:36:18.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:18.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738711+0000", 2024-09-14T22:36:18.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.767e-06 2024-09-14T22:36:18.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.347 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:18.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738745+0000", 2024-09-14T22:36:18.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3362999999999999e-05 2024-09-14T22:36:18.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.347 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738754+0000", 2024-09-14T22:36:18.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2439999999999999e-06 2024-09-14T22:36:18.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.347 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:18.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.753553+0000", 2024-09-14T22:36:18.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014799132 2024-09-14T22:36:18.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:18.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.753763+0000", 2024-09-14T22:36:18.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021000200000000001 2024-09-14T22:36:18.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:18.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.754101+0000", 2024-09-14T22:36:18.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000337664 2024-09-14T22:36:18.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:18.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.756983+0000", 2024-09-14T22:36:18.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0028820180000000001 2024-09-14T22:36:18.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:18.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.758300+0000", 2024-09-14T22:36:18.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0013173060000000001 2024-09-14T22:36:18.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:18.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.758463+0000", 2024-09-14T22:36:18.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000162752 2024-09-14T22:36:18.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:18.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.758576+0000", 2024-09-14T22:36:18.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011342599999999999 2024-09-14T22:36:18.350 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.350 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:18.350 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.350 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:36:18.350 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.738768+0000", 2024-09-14T22:36:18.350 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 17.533015774999999, 2024-09-14T22:36:18.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020973018, 2024-09-14T22:36:18.350 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:18.350 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:18.351 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.351 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:18.351 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:36:18.351 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:36:18.351 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 115 2024-09-14T22:36:18.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.351 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:18.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:18.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738768+0000", 2024-09-14T22:36:18.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:18.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738768+0000", 2024-09-14T22:36:18.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:18.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738769+0000", 2024-09-14T22:36:18.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.23e-07 2024-09-14T22:36:18.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:18.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738945+0000", 2024-09-14T22:36:18.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017596199999999999 2024-09-14T22:36:18.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:18.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738974+0000", 2024-09-14T22:36:18.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9183000000000002e-05 2024-09-14T22:36:18.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.353 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738995+0000", 2024-09-14T22:36:18.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0934e-05 2024-09-14T22:36:18.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.353 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:18.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.755191+0000", 2024-09-14T22:36:18.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.016195378 2024-09-14T22:36:18.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:18.354 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.755331+0000", 2024-09-14T22:36:18.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014080599999999999 2024-09-14T22:36:18.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:18.354 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.755543+0000", 2024-09-14T22:36:18.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021108800000000001 2024-09-14T22:36:18.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:18.354 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.759583+0000", 2024-09-14T22:36:18.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0040401020000000003 2024-09-14T22:36:18.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:18.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.759603+0000", 2024-09-14T22:36:18.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0803000000000001e-05 2024-09-14T22:36:18.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:18.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.759673+0000", 2024-09-14T22:36:18.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9900000000000005e-05 2024-09-14T22:36:18.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.356 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:18.356 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.759741+0000", 2024-09-14T22:36:18.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8039000000000004e-05 2024-09-14T22:36:18.356 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.356 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:18.356 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.356 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:36:18.356 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.739034+0000", 2024-09-14T22:36:18.356 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 17.532750354000001, 2024-09-14T22:36:18.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021026605, 2024-09-14T22:36:18.356 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:18.357 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:18.357 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.357 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:18.357 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:36:18.357 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:36:18.357 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 116 2024-09-14T22:36:18.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.357 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:18.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:18.357 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.739034+0000", 2024-09-14T22:36:18.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:18.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.739034+0000", 2024-09-14T22:36:18.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:18.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.739036+0000", 2024-09-14T22:36:18.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0040000000000002e-06 2024-09-14T22:36:18.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:18.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.739048+0000", 2024-09-14T22:36:18.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1793e-05 2024-09-14T22:36:18.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:18.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.739068+0000", 2024-09-14T22:36:18.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0006000000000001e-05 2024-09-14T22:36:18.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.739090+0000", 2024-09-14T22:36:18.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2087e-05 2024-09-14T22:36:18.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:18.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.757100+0000", 2024-09-14T22:36:18.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018010004 2024-09-14T22:36:18.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:18.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.757308+0000", 2024-09-14T22:36:18.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020808600000000001 2024-09-14T22:36:18.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:18.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.757601+0000", 2024-09-14T22:36:18.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029366400000000001 2024-09-14T22:36:18.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:18.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.759771+0000", 2024-09-14T22:36:18.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0021695400000000002 2024-09-14T22:36:18.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:18.361 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.759863+0000", 2024-09-14T22:36:18.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2388999999999999e-05 2024-09-14T22:36:18.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:18.361 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.759955+0000", 2024-09-14T22:36:18.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1608999999999997e-05 2024-09-14T22:36:18.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:18.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.760060+0000", 2024-09-14T22:36:18.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105423 2024-09-14T22:36:18.362 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.362 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:18.362 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.362 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:36:18.362 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.739135+0000", 2024-09-14T22:36:18.362 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 17.532648838, 2024-09-14T22:36:18.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021794035999999999, 2024-09-14T22:36:18.362 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:18.362 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:18.362 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.362 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:18.363 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:36:18.363 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:36:18.363 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117 2024-09-14T22:36:18.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.363 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:18.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:18.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.739135+0000", 2024-09-14T22:36:18.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:18.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.739135+0000", 2024-09-14T22:36:18.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:18.364 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.739137+0000", 2024-09-14T22:36:18.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3069999999999999e-06 2024-09-14T22:36:18.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:18.364 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.739148+0000", 2024-09-14T22:36:18.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1077000000000001e-05 2024-09-14T22:36:18.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:18.364 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.739180+0000", 2024-09-14T22:36:18.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2356e-05 2024-09-14T22:36:18.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.739221+0000", 2024-09-14T22:36:18.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0568999999999997e-05 2024-09-14T22:36:18.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:18.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.758699+0000", 2024-09-14T22:36:18.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.019478763999999999 2024-09-14T22:36:18.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:18.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.758835+0000", 2024-09-14T22:36:18.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013597899999999999 2024-09-14T22:36:18.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:18.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.759049+0000", 2024-09-14T22:36:18.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021361399999999999 2024-09-14T22:36:18.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:18.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.760246+0000", 2024-09-14T22:36:18.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0011972249999999999 2024-09-14T22:36:18.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:18.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.760790+0000", 2024-09-14T22:36:18.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00054360900000000002 2024-09-14T22:36:18.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:18.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.760870+0000", 2024-09-14T22:36:18.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0389000000000006e-05 2024-09-14T22:36:18.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:18.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.760929+0000", 2024-09-14T22:36:18.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9147000000000001e-05 2024-09-14T22:36:18.367 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.367 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:18.367 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.368 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-14T22:36:18.368 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:01.239224+0000", 2024-09-14T22:36:18.368 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 17.032560022999998, 2024-09-14T22:36:18.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012678655, 2024-09-14T22:36:18.368 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:18.368 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:18.368 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.368 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:18.368 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:36:18.368 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:36:18.368 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 136 2024-09-14T22:36:18.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.368 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:18.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:18.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239224+0000", 2024-09-14T22:36:18.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:18.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239224+0000", 2024-09-14T22:36:18.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:18.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239225+0000", 2024-09-14T22:36:18.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2399999999999998e-07 2024-09-14T22:36:18.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:18.370 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239228+0000", 2024-09-14T22:36:18.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2899999999999998e-06 2024-09-14T22:36:18.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:18.370 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239237+0000", 2024-09-14T22:36:18.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8140000000000007e-06 2024-09-14T22:36:18.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.370 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239243+0000", 2024-09-14T22:36:18.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8429999999999998e-06 2024-09-14T22:36:18.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:18.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.243290+0000", 2024-09-14T22:36:18.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0040476970000000003 2024-09-14T22:36:18.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:18.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.243344+0000", 2024-09-14T22:36:18.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3233999999999999e-05 2024-09-14T22:36:18.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:18.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.243428+0000", 2024-09-14T22:36:18.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4189000000000003e-05 2024-09-14T22:36:18.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:18.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.248224+0000", 2024-09-14T22:36:18.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0047960310000000001 2024-09-14T22:36:18.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:18.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.251856+0000", 2024-09-14T22:36:18.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0036324650000000001 2024-09-14T22:36:18.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:18.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.251882+0000", 2024-09-14T22:36:18.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6128000000000001e-05 2024-09-14T22:36:18.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:18.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.251903+0000", 2024-09-14T22:36:18.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0440000000000001e-05 2024-09-14T22:36:18.373 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.373 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:18.373 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.374 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-14T22:36:18.374 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:01.239250+0000", 2024-09-14T22:36:18.374 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 17.032533870999998, 2024-09-14T22:36:18.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012923653, 2024-09-14T22:36:18.374 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:18.374 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:18.374 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.374 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:18.374 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:36:18.374 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:36:18.374 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 137 2024-09-14T22:36:18.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.375 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:18.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:18.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239250+0000", 2024-09-14T22:36:18.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:18.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239250+0000", 2024-09-14T22:36:18.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:18.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239251+0000", 2024-09-14T22:36:18.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8299999999999998e-07 2024-09-14T22:36:18.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:18.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239291+0000", 2024-09-14T22:36:18.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0364000000000002e-05 2024-09-14T22:36:18.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:18.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239304+0000", 2024-09-14T22:36:18.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2529e-05 2024-09-14T22:36:18.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239308+0000", 2024-09-14T22:36:18.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1520000000000002e-06 2024-09-14T22:36:18.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:18.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.243677+0000", 2024-09-14T22:36:18.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0043689419999999998 2024-09-14T22:36:18.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:18.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.243718+0000", 2024-09-14T22:36:18.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1130000000000001e-05 2024-09-14T22:36:18.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:18.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.243787+0000", 2024-09-14T22:36:18.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9416000000000001e-05 2024-09-14T22:36:18.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:18.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.248234+0000", 2024-09-14T22:36:18.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004446841 2024-09-14T22:36:18.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:18.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.251985+0000", 2024-09-14T22:36:18.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0037513440000000002 2024-09-14T22:36:18.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:18.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.252097+0000", 2024-09-14T22:36:18.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112043 2024-09-14T22:36:18.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.379 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:18.379 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.252174+0000", 2024-09-14T22:36:18.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6508999999999995e-05 2024-09-14T22:36:18.379 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.379 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:18.379 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.379 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-14T22:36:18.379 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:01.239315+0000", 2024-09-14T22:36:18.379 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 17.032469002999999, 2024-09-14T22:36:18.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013021549, 2024-09-14T22:36:18.380 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:18.380 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:18.380 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.380 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:18.380 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:36:18.380 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:36:18.380 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 138 2024-09-14T22:36:18.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.380 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:18.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:18.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239315+0000", 2024-09-14T22:36:18.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:18.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239315+0000", 2024-09-14T22:36:18.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:18.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239316+0000", 2024-09-14T22:36:18.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.89e-07 2024-09-14T22:36:18.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:18.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239318+0000", 2024-09-14T22:36:18.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7640000000000001e-06 2024-09-14T22:36:18.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:18.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239323+0000", 2024-09-14T22:36:18.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6689999999999997e-06 2024-09-14T22:36:18.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239329+0000", 2024-09-14T22:36:18.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8610000000000003e-06 2024-09-14T22:36:18.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:18.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.244231+0000", 2024-09-14T22:36:18.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0049019600000000003 2024-09-14T22:36:18.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:18.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.244275+0000", 2024-09-14T22:36:18.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4027999999999999e-05 2024-09-14T22:36:18.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:18.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.244346+0000", 2024-09-14T22:36:18.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0728000000000005e-05 2024-09-14T22:36:18.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:18.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.248244+0000", 2024-09-14T22:36:18.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0038985059999999999 2024-09-14T22:36:18.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:18.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.252241+0000", 2024-09-14T22:36:18.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0039967170000000003 2024-09-14T22:36:18.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:18.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.252282+0000", 2024-09-14T22:36:18.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1071e-05 2024-09-14T22:36:18.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:18.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.252337+0000", 2024-09-14T22:36:18.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4756000000000002e-05 2024-09-14T22:36:18.385 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.385 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:18.385 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.385 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-14T22:36:18.385 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:01.239343+0000", 2024-09-14T22:36:18.385 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 17.032440859000001, 2024-09-14T22:36:18.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013206142000000001, 2024-09-14T22:36:18.386 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:18.386 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:18.386 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.386 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:18.386 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:36:18.386 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:36:18.386 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 139 2024-09-14T22:36:18.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.386 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:18.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:18.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239343+0000", 2024-09-14T22:36:18.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:18.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239343+0000", 2024-09-14T22:36:18.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:18.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239344+0000", 2024-09-14T22:36:18.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1899999999999998e-07 2024-09-14T22:36:18.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:18.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239350+0000", 2024-09-14T22:36:18.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4729999999999997e-06 2024-09-14T22:36:18.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:18.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239357+0000", 2024-09-14T22:36:18.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1169999999999998e-06 2024-09-14T22:36:18.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239361+0000", 2024-09-14T22:36:18.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8480000000000003e-06 2024-09-14T22:36:18.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:18.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.244614+0000", 2024-09-14T22:36:18.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.005252942 2024-09-14T22:36:18.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:18.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.244658+0000", 2024-09-14T22:36:18.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4323999999999998e-05 2024-09-14T22:36:18.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:18.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.244770+0000", 2024-09-14T22:36:18.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000111455 2024-09-14T22:36:18.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:18.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.248254+0000", 2024-09-14T22:36:18.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0034843399999999998 2024-09-14T22:36:18.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:18.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.252405+0000", 2024-09-14T22:36:18.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0041505580000000004 2024-09-14T22:36:18.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:18.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.252489+0000", 2024-09-14T22:36:18.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3825999999999993e-05 2024-09-14T22:36:18.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:18.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.252549+0000", 2024-09-14T22:36:18.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.084e-05 2024-09-14T22:36:18.391 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.391 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:18.391 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.391 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-14T22:36:18.391 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:01.239379+0000", 2024-09-14T22:36:18.392 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 17.032405507, 2024-09-14T22:36:18.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013638361, 2024-09-14T22:36:18.392 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:18.392 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:18.392 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.392 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:18.392 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:36:18.392 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:36:18.392 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 140 2024-09-14T22:36:18.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.392 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:18.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:18.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239379+0000", 2024-09-14T22:36:18.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:18.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239379+0000", 2024-09-14T22:36:18.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:18.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239379+0000", 2024-09-14T22:36:18.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.58e-07 2024-09-14T22:36:18.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:18.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239417+0000", 2024-09-14T22:36:18.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7966999999999997e-05 2024-09-14T22:36:18.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:18.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239425+0000", 2024-09-14T22:36:18.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2810000000000008e-06 2024-09-14T22:36:18.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239430+0000", 2024-09-14T22:36:18.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9130000000000002e-06 2024-09-14T22:36:18.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:18.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.245019+0000", 2024-09-14T22:36:18.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0055885639999999999 2024-09-14T22:36:18.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:18.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.245064+0000", 2024-09-14T22:36:18.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5587000000000002e-05 2024-09-14T22:36:18.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:18.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.245137+0000", 2024-09-14T22:36:18.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2525000000000001e-05 2024-09-14T22:36:18.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:18.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.248264+0000", 2024-09-14T22:36:18.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0031275080000000002 2024-09-14T22:36:18.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:18.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.252929+0000", 2024-09-14T22:36:18.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004664329 2024-09-14T22:36:18.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:18.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.252967+0000", 2024-09-14T22:36:18.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7796999999999997e-05 2024-09-14T22:36:18.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:18.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.253017+0000", 2024-09-14T22:36:18.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0432000000000001e-05 2024-09-14T22:36:18.397 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.397 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:18.397 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.397 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-14T22:36:18.397 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:01.239438+0000", 2024-09-14T22:36:18.397 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 17.032346233999998, 2024-09-14T22:36:18.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014419557, 2024-09-14T22:36:18.397 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:18.398 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:18.398 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.398 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:18.398 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:36:18.398 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:36:18.398 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 141 2024-09-14T22:36:18.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.398 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:18.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:18.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239438+0000", 2024-09-14T22:36:18.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:18.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239438+0000", 2024-09-14T22:36:18.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:18.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:18.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239438+0000", 2024-09-14T22:36:18.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2e-07 2024-09-14T22:36:18.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:18.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239441+0000", 2024-09-14T22:36:18.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.926e-06 2024-09-14T22:36:18.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:18.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239446+0000", 2024-09-14T22:36:18.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5859999999999998e-06 2024-09-14T22:36:18.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.239449+0000", 2024-09-14T22:36:18.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.534e-06 2024-09-14T22:36:18.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:18.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.245630+0000", 2024-09-14T22:36:18.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0061806600000000001 2024-09-14T22:36:18.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:18.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.245857+0000", 2024-09-14T22:36:18.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022733900000000001 2024-09-14T22:36:18.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:18.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.246262+0000", 2024-09-14T22:36:18.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040470000000000002 2024-09-14T22:36:18.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:18.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.248369+0000", 2024-09-14T22:36:18.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0021068800000000002 2024-09-14T22:36:18.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:18.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.253716+0000", 2024-09-14T22:36:18.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.005347503 2024-09-14T22:36:18.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:18.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.253797+0000", 2024-09-14T22:36:18.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0746000000000002e-05 2024-09-14T22:36:18.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:18.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:18.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:18.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:01.253857+0000", 2024-09-14T22:36:18.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0263000000000002e-05 2024-09-14T22:36:18.403 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.403 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:18.403 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.403 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:18.403 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:18.403 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:18.404 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:36:18.405 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:18.398+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:18.505 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:18.498+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:18.529 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:18.529 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:36:18.529 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:36:18.529 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:18.542 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:36:18.606 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:18.603+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:18.707 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:18.702+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:18.715 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:18.716 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:36:18.716 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:36:18.716 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:36:18.716 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:18.729 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:36:18.808 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:18.803+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:18.891 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:18.891 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:36:18.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:36:18.891 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:36:18.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.891 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4201.0:31 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'17, mlcod=17'14)", 2024-09-14T22:36:18.892 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:00.462598+0000", 2024-09-14T22:36:18.892 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.425925137, 2024-09-14T22:36:18.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002577233, 2024-09-14T22:36:18.892 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:18.892 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:18.892 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.892 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:18.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:18.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.462598+0000", 2024-09-14T22:36:18.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:18.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.462598+0000", 2024-09-14T22:36:18.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:18.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.462600+0000", 2024-09-14T22:36:18.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.86e-06 2024-09-14T22:36:18.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:18.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.462617+0000", 2024-09-14T22:36:18.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7122999999999999e-05 2024-09-14T22:36:18.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:18.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.462652+0000", 2024-09-14T22:36:18.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5089000000000002e-05 2024-09-14T22:36:18.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.462675+0000", 2024-09-14T22:36:18.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2900000000000001e-05 2024-09-14T22:36:18.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:18.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.462814+0000", 2024-09-14T22:36:18.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000139099 2024-09-14T22:36:18.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:18.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.462849+0000", 2024-09-14T22:36:18.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5193e-05 2024-09-14T22:36:18.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:18.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.465093+0000", 2024-09-14T22:36:18.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0022436159999999999 2024-09-14T22:36:18.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:18.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.465175+0000", 2024-09-14T22:36:18.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2353000000000002e-05 2024-09-14T22:36:18.900 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.900 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:18.900 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.900 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4201.0:54 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'29)", 2024-09-14T22:36:18.900 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:00.503107+0000", 2024-09-14T22:36:18.900 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.385416543000002, 2024-09-14T22:36:18.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002677399, 2024-09-14T22:36:18.901 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:18.901 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:18.901 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.901 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:18.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:18.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.503107+0000", 2024-09-14T22:36:18.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:18.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.503107+0000", 2024-09-14T22:36:18.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:18.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.503109+0000", 2024-09-14T22:36:18.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.863e-06 2024-09-14T22:36:18.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:18.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.503121+0000", 2024-09-14T22:36:18.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2289999999999999e-05 2024-09-14T22:36:18.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:18.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.503150+0000", 2024-09-14T22:36:18.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9056000000000001e-05 2024-09-14T22:36:18.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.904 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.503163+0000", 2024-09-14T22:36:18.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3393999999999999e-05 2024-09-14T22:36:18.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.904 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:18.904 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.503315+0000", 2024-09-14T22:36:18.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000151645 2024-09-14T22:36:18.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.904 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:18.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.503349+0000", 2024-09-14T22:36:18.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3772999999999997e-05 2024-09-14T22:36:18.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:18.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.505668+0000", 2024-09-14T22:36:18.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0023190020000000001 2024-09-14T22:36:18.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:18.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.505784+0000", 2024-09-14T22:36:18.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116376 2024-09-14T22:36:18.906 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.906 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:18.906 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.907 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4201.0:57 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'35, mlcod=17'33)", 2024-09-14T22:36:18.907 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:00.509264+0000", 2024-09-14T22:36:18.907 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.379259631, 2024-09-14T22:36:18.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0026069029999999998, 2024-09-14T22:36:18.907 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:18.907 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:18.908 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.908 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:18.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:18.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.509264+0000", 2024-09-14T22:36:18.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:18.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.509264+0000", 2024-09-14T22:36:18.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:18.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.509266+0000", 2024-09-14T22:36:18.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.83e-06 2024-09-14T22:36:18.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:18.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.509317+0000", 2024-09-14T22:36:18.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0099999999999998e-05 2024-09-14T22:36:18.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:18.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.509346+0000", 2024-09-14T22:36:18.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9546000000000001e-05 2024-09-14T22:36:18.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.509360+0000", 2024-09-14T22:36:18.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3933e-05 2024-09-14T22:36:18.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:18.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.509535+0000", 2024-09-14T22:36:18.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000174659 2024-09-14T22:36:18.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:18.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.509574+0000", 2024-09-14T22:36:18.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9762999999999999e-05 2024-09-14T22:36:18.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:18.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.511800+0000", 2024-09-14T22:36:18.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0022257100000000001 2024-09-14T22:36:18.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:18.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.511871+0000", 2024-09-14T22:36:18.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0362000000000002e-05 2024-09-14T22:36:18.912 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.912 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:18.913 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.913 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4201.0:71 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'45, mlcod=17'43)", 2024-09-14T22:36:18.913 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:00.539599+0000", 2024-09-14T22:36:18.913 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.348923808999999, 2024-09-14T22:36:18.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0024941149999999999, 2024-09-14T22:36:18.913 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:18.913 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:18.913 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.913 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:18.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:18.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.539599+0000", 2024-09-14T22:36:18.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:18.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.539599+0000", 2024-09-14T22:36:18.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:18.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.539601+0000", 2024-09-14T22:36:18.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6640000000000001e-06 2024-09-14T22:36:18.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:18.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.539656+0000", 2024-09-14T22:36:18.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4525999999999999e-05 2024-09-14T22:36:18.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:18.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.539685+0000", 2024-09-14T22:36:18.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9830000000000001e-05 2024-09-14T22:36:18.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.539701+0000", 2024-09-14T22:36:18.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5290000000000001e-05 2024-09-14T22:36:18.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:18.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.539858+0000", 2024-09-14T22:36:18.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015751200000000001 2024-09-14T22:36:18.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:18.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.539898+0000", 2024-09-14T22:36:18.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9635999999999998e-05 2024-09-14T22:36:18.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:18.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.541975+0000", 2024-09-14T22:36:18.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020770139999999999 2024-09-14T22:36:18.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:18.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.542094+0000", 2024-09-14T22:36:18.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118643 2024-09-14T22:36:18.918 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.918 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:18.918 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.919 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4201.0:72 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'46, mlcod=17'43)", 2024-09-14T22:36:18.919 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:00.541038+0000", 2024-09-14T22:36:18.919 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.347484829999999, 2024-09-14T22:36:18.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0029050980000000001, 2024-09-14T22:36:18.919 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:18.919 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:18.919 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.919 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:18.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:18.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.541038+0000", 2024-09-14T22:36:18.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:18.920 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.541038+0000", 2024-09-14T22:36:18.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:18.920 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.541040+0000", 2024-09-14T22:36:18.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.654e-06 2024-09-14T22:36:18.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:18.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.541146+0000", 2024-09-14T22:36:18.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105962 2024-09-14T22:36:18.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:18.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.541175+0000", 2024-09-14T22:36:18.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8946999999999999e-05 2024-09-14T22:36:18.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.541189+0000", 2024-09-14T22:36:18.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4365999999999999e-05 2024-09-14T22:36:18.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:18.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.541340+0000", 2024-09-14T22:36:18.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015090899999999999 2024-09-14T22:36:18.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:18.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.541377+0000", 2024-09-14T22:36:18.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6866999999999998e-05 2024-09-14T22:36:18.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:18.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.543847+0000", 2024-09-14T22:36:18.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0024699869999999999 2024-09-14T22:36:18.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:18.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.543944+0000", 2024-09-14T22:36:18.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6405999999999998e-05 2024-09-14T22:36:18.923 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.924 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:18.924 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.924 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4201.0:87 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'57, mlcod=17'53)", 2024-09-14T22:36:18.924 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:00.567997+0000", 2024-09-14T22:36:18.924 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.320525824000001, 2024-09-14T22:36:18.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002556577, 2024-09-14T22:36:18.924 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:18.924 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:18.924 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.925 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:18.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:18.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.567997+0000", 2024-09-14T22:36:18.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:18.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.567997+0000", 2024-09-14T22:36:18.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:18.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.567999+0000", 2024-09-14T22:36:18.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.906e-06 2024-09-14T22:36:18.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:18.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.568050+0000", 2024-09-14T22:36:18.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0742999999999999e-05 2024-09-14T22:36:18.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:18.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.568079+0000", 2024-09-14T22:36:18.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8986999999999999e-05 2024-09-14T22:36:18.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.568092+0000", 2024-09-14T22:36:18.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3226e-05 2024-09-14T22:36:18.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:18.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.568190+0000", 2024-09-14T22:36:18.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7472999999999993e-05 2024-09-14T22:36:18.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:18.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.568229+0000", 2024-09-14T22:36:18.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9378999999999997e-05 2024-09-14T22:36:18.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:18.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.570461+0000", 2024-09-14T22:36:18.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0022319330000000002 2024-09-14T22:36:18.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:18.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.570554+0000", 2024-09-14T22:36:18.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2930000000000006e-05 2024-09-14T22:36:18.929 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.929 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:18.929 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.929 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4201.0:106 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'64)", 2024-09-14T22:36:18.929 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:00.740740+0000", 2024-09-14T22:36:18.929 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.147783304000001, 2024-09-14T22:36:18.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0026186669999999999, 2024-09-14T22:36:18.930 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:18.930 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:18.930 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.930 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:18.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:18.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.740740+0000", 2024-09-14T22:36:18.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:18.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.740740+0000", 2024-09-14T22:36:18.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:18.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.740742+0000", 2024-09-14T22:36:18.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.747e-06 2024-09-14T22:36:18.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:18.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.740873+0000", 2024-09-14T22:36:18.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000131669 2024-09-14T22:36:18.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:18.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.740959+0000", 2024-09-14T22:36:18.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5655000000000006e-05 2024-09-14T22:36:18.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.741001+0000", 2024-09-14T22:36:18.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1539999999999999e-05 2024-09-14T22:36:18.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:18.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.741125+0000", 2024-09-14T22:36:18.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000124573 2024-09-14T22:36:18.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:18.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.741161+0000", 2024-09-14T22:36:18.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6362000000000002e-05 2024-09-14T22:36:18.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:18.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.743311+0000", 2024-09-14T22:36:18.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0021490530000000002 2024-09-14T22:36:18.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:18.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:00.743359+0000", 2024-09-14T22:36:18.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8068000000000001e-05 2024-09-14T22:36:18.934 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.934 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:18.935 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.935 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4201.0:131 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'89, mlcod=17'85)", 2024-09-14T22:36:18.935 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:01.241292+0000", 2024-09-14T22:36:18.935 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 17.647230805, 2024-09-14T22:36:18.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0043515339999999998, 2024-09-14T22:36:18.936 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:18.936 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:18.936 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.936 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:18.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:18.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.241292+0000", 2024-09-14T22:36:18.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:18.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.241292+0000", 2024-09-14T22:36:18.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:18.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.241294+0000", 2024-09-14T22:36:18.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4139999999999999e-06 2024-09-14T22:36:18.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:18.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.241422+0000", 2024-09-14T22:36:18.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012855200000000001 2024-09-14T22:36:18.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:18.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.241463+0000", 2024-09-14T22:36:18.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0200000000000001e-05 2024-09-14T22:36:18.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.241475+0000", 2024-09-14T22:36:18.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2483000000000001e-05 2024-09-14T22:36:18.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:18.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.241581+0000", 2024-09-14T22:36:18.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010630599999999999 2024-09-14T22:36:18.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:18.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.241640+0000", 2024-09-14T22:36:18.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8452999999999998e-05 2024-09-14T22:36:18.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:18.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245568+0000", 2024-09-14T22:36:18.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0039277139999999997 2024-09-14T22:36:18.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:18.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245644+0000", 2024-09-14T22:36:18.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6412e-05 2024-09-14T22:36:18.944 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.945 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:18.945 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.945 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4201.0:132 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'90, mlcod=17'85)", 2024-09-14T22:36:18.945 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:01.241802+0000", 2024-09-14T22:36:18.945 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 17.646721622000001, 2024-09-14T22:36:18.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0055789150000000003, 2024-09-14T22:36:18.945 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:18.945 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:18.946 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.946 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:18.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:18.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.241802+0000", 2024-09-14T22:36:18.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:18.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.241802+0000", 2024-09-14T22:36:18.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:18.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.241803+0000", 2024-09-14T22:36:18.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.08e-06 2024-09-14T22:36:18.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:18.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.241811+0000", 2024-09-14T22:36:18.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3000000000000002e-06 2024-09-14T22:36:18.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:18.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.241827+0000", 2024-09-14T22:36:18.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.592e-05 2024-09-14T22:36:18.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.241837+0000", 2024-09-14T22:36:18.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7399999999999999e-06 2024-09-14T22:36:18.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:18.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.242796+0000", 2024-09-14T22:36:18.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00095904599999999997 2024-09-14T22:36:18.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:18.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.242832+0000", 2024-09-14T22:36:18.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5952999999999999e-05 2024-09-14T22:36:18.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:18.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.247209+0000", 2024-09-14T22:36:18.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0043776930000000002 2024-09-14T22:36:18.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:18.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.247381+0000", 2024-09-14T22:36:18.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017118299999999999 2024-09-14T22:36:18.950 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.950 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:18.950 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.951 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4201.0:133 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'91, mlcod=18'86)", 2024-09-14T22:36:18.951 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:01.242750+0000", 2024-09-14T22:36:18.951 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 17.645773682000002, 2024-09-14T22:36:18.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0047279399999999999, 2024-09-14T22:36:18.951 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:18.951 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:18.951 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.951 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:18.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:18.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.242750+0000", 2024-09-14T22:36:18.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:18.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.242750+0000", 2024-09-14T22:36:18.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:18.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.242751+0000", 2024-09-14T22:36:18.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2929999999999999e-06 2024-09-14T22:36:18.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:18.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.242760+0000", 2024-09-14T22:36:18.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1470000000000007e-06 2024-09-14T22:36:18.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:18.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.242776+0000", 2024-09-14T22:36:18.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6466000000000001e-05 2024-09-14T22:36:18.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.954 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.954 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.242786+0000", 2024-09-14T22:36:18.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6600000000000007e-06 2024-09-14T22:36:18.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.954 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:18.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.243514+0000", 2024-09-14T22:36:18.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00072809399999999999 2024-09-14T22:36:18.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:18.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.243565+0000", 2024-09-14T22:36:18.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0874000000000001e-05 2024-09-14T22:36:18.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:18.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.247388+0000", 2024-09-14T22:36:18.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0038229570000000001 2024-09-14T22:36:18.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:18.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.247478+0000", 2024-09-14T22:36:18.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9449000000000004e-05 2024-09-14T22:36:18.956 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.956 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:18.956 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.956 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4201.0:134 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'92, mlcod=18'86)", 2024-09-14T22:36:18.957 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:01.243214+0000", 2024-09-14T22:36:18.957 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 17.645309052000002, 2024-09-14T22:36:18.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0067041150000000001, 2024-09-14T22:36:18.957 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:18.957 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:18.957 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.957 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:18.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:18.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.243214+0000", 2024-09-14T22:36:18.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:18.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.243214+0000", 2024-09-14T22:36:18.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:18.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.243215+0000", 2024-09-14T22:36:18.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2359999999999999e-06 2024-09-14T22:36:18.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:18.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.243264+0000", 2024-09-14T22:36:18.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8850000000000002e-05 2024-09-14T22:36:18.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:18.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.243287+0000", 2024-09-14T22:36:18.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2823000000000002e-05 2024-09-14T22:36:18.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.243301+0000", 2024-09-14T22:36:18.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3963e-05 2024-09-14T22:36:18.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:18.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.244365+0000", 2024-09-14T22:36:18.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001064389 2024-09-14T22:36:18.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:18.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.244423+0000", 2024-09-14T22:36:18.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7235999999999999e-05 2024-09-14T22:36:18.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:18.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.249785+0000", 2024-09-14T22:36:18.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0053624689999999999 2024-09-14T22:36:18.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:18.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.249918+0000", 2024-09-14T22:36:18.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000133149 2024-09-14T22:36:18.961 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.961 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:18.961 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.962 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4201.0:135 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'93, mlcod=18'86)", 2024-09-14T22:36:18.962 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:01.243652+0000", 2024-09-14T22:36:18.962 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 17.644871392999999, 2024-09-14T22:36:18.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.006393925, 2024-09-14T22:36:18.962 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:18.962 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:18.962 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.962 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:18.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:18.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.243652+0000", 2024-09-14T22:36:18.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:18.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.243652+0000", 2024-09-14T22:36:18.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:18.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.243653+0000", 2024-09-14T22:36:18.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3540000000000001e-06 2024-09-14T22:36:18.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:18.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.243663+0000", 2024-09-14T22:36:18.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7259999999999997e-06 2024-09-14T22:36:18.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:18.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.243684+0000", 2024-09-14T22:36:18.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0934e-05 2024-09-14T22:36:18.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.243695+0000", 2024-09-14T22:36:18.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1116e-05 2024-09-14T22:36:18.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:18.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245746+0000", 2024-09-14T22:36:18.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020508050000000002 2024-09-14T22:36:18.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:18.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245785+0000", 2024-09-14T22:36:18.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9536000000000002e-05 2024-09-14T22:36:18.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:18.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.249926+0000", 2024-09-14T22:36:18.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0041406710000000003 2024-09-14T22:36:18.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:18.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.250046+0000", 2024-09-14T22:36:18.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119783 2024-09-14T22:36:18.967 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.967 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:18.967 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.967 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4201.0:136 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'94, mlcod=18'86)", 2024-09-14T22:36:18.967 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:01.244082+0000", 2024-09-14T22:36:18.967 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 17.644441436000001, 2024-09-14T22:36:18.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0081452039999999996, 2024-09-14T22:36:18.967 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:18.967 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:18.968 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.968 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:18.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:18.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.244082+0000", 2024-09-14T22:36:18.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:18.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.244082+0000", 2024-09-14T22:36:18.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:18.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.244083+0000", 2024-09-14T22:36:18.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1039999999999999e-06 2024-09-14T22:36:18.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:18.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.244091+0000", 2024-09-14T22:36:18.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1859999999999992e-06 2024-09-14T22:36:18.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:18.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.244108+0000", 2024-09-14T22:36:18.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6634e-05 2024-09-14T22:36:18.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.244118+0000", 2024-09-14T22:36:18.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0043e-05 2024-09-14T22:36:18.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:18.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.246410+0000", 2024-09-14T22:36:18.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0022924849999999999 2024-09-14T22:36:18.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:18.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.246469+0000", 2024-09-14T22:36:18.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8903999999999997e-05 2024-09-14T22:36:18.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:18.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.252090+0000", 2024-09-14T22:36:18.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0056204560000000002 2024-09-14T22:36:18.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:18.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.252227+0000", 2024-09-14T22:36:18.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013739199999999999 2024-09-14T22:36:18.972 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.972 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:18.972 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.973 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4201.0:137 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'95, mlcod=18'86)", 2024-09-14T22:36:18.973 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:01.244490+0000", 2024-09-14T22:36:18.973 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 17.644033071999999, 2024-09-14T22:36:18.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0078329290000000006, 2024-09-14T22:36:18.973 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:18.973 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:18.973 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.973 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:18.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:18.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.244490+0000", 2024-09-14T22:36:18.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:18.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.244490+0000", 2024-09-14T22:36:18.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:18.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.244491+0000", 2024-09-14T22:36:18.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0529999999999999e-06 2024-09-14T22:36:18.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:18.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.244542+0000", 2024-09-14T22:36:18.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0550000000000002e-05 2024-09-14T22:36:18.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:18.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.244558+0000", 2024-09-14T22:36:18.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6633000000000002e-05 2024-09-14T22:36:18.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.244573+0000", 2024-09-14T22:36:18.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4513e-05 2024-09-14T22:36:18.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:18.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.247604+0000", 2024-09-14T22:36:18.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030310200000000002 2024-09-14T22:36:18.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:18.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.247664+0000", 2024-09-14T22:36:18.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9892999999999997e-05 2024-09-14T22:36:18.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:18.978 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.252235+0000", 2024-09-14T22:36:18.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.004571065 2024-09-14T22:36:18.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:18.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.252323+0000", 2024-09-14T22:36:18.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8201999999999993e-05 2024-09-14T22:36:18.979 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.979 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:18.979 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.979 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4201.0:138 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'96, mlcod=18'86)", 2024-09-14T22:36:18.979 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:01.245027+0000", 2024-09-14T22:36:18.979 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 17.643495846, 2024-09-14T22:36:18.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0073863990000000001, 2024-09-14T22:36:18.980 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:18.980 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:18.980 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.980 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:18.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:18.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245027+0000", 2024-09-14T22:36:18.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:18.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245027+0000", 2024-09-14T22:36:18.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:18.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245029+0000", 2024-09-14T22:36:18.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1629999999999999e-06 2024-09-14T22:36:18.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:18.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245038+0000", 2024-09-14T22:36:18.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8670000000000006e-06 2024-09-14T22:36:18.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:18.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245058+0000", 2024-09-14T22:36:18.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0015999999999999e-05 2024-09-14T22:36:18.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245068+0000", 2024-09-14T22:36:18.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7100000000000002e-06 2024-09-14T22:36:18.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:18.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.248834+0000", 2024-09-14T22:36:18.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003765698 2024-09-14T22:36:18.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:18.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.248913+0000", 2024-09-14T22:36:18.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9503000000000001e-05 2024-09-14T22:36:18.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:18.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.252334+0000", 2024-09-14T22:36:18.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0034203559999999998 2024-09-14T22:36:18.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:18.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.252414+0000", 2024-09-14T22:36:18.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0086000000000005e-05 2024-09-14T22:36:18.984 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.984 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:18.984 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.984 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4201.0:139 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'97, mlcod=18'86)", 2024-09-14T22:36:18.984 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:01.245437+0000", 2024-09-14T22:36:18.985 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 17.643086046000001, 2024-09-14T22:36:18.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0070633689999999999, 2024-09-14T22:36:18.985 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:18.985 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:18.985 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.985 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:18.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:18.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245437+0000", 2024-09-14T22:36:18.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:18.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245437+0000", 2024-09-14T22:36:18.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:18.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245438+0000", 2024-09-14T22:36:18.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2839999999999999e-06 2024-09-14T22:36:18.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:18.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245448+0000", 2024-09-14T22:36:18.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5999999999999996e-06 2024-09-14T22:36:18.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:18.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245463+0000", 2024-09-14T22:36:18.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5156e-05 2024-09-14T22:36:18.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245474+0000", 2024-09-14T22:36:18.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1126e-05 2024-09-14T22:36:18.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:18.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.250162+0000", 2024-09-14T22:36:18.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0046874109999999998 2024-09-14T22:36:18.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:18.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.250205+0000", 2024-09-14T22:36:18.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2945999999999998e-05 2024-09-14T22:36:18.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:18.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.252420+0000", 2024-09-14T22:36:18.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0022156260000000001 2024-09-14T22:36:18.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:18.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.252501+0000", 2024-09-14T22:36:18.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0220000000000001e-05 2024-09-14T22:36:18.990 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.990 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:18.990 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.990 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4201.0:140 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'98, mlcod=18'86)", 2024-09-14T22:36:18.990 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:01.245849+0000", 2024-09-14T22:36:18.990 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 17.642673882, 2024-09-14T22:36:18.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0074467750000000001, 2024-09-14T22:36:18.990 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:18.991 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:18.991 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.991 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:18.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:18.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245849+0000", 2024-09-14T22:36:18.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:18.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245849+0000", 2024-09-14T22:36:18.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:18.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245850+0000", 2024-09-14T22:36:18.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5300000000000003e-07 2024-09-14T22:36:18.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:18.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245924+0000", 2024-09-14T22:36:18.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3895999999999999e-05 2024-09-14T22:36:18.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:18.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245947+0000", 2024-09-14T22:36:18.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3213e-05 2024-09-14T22:36:18.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:18.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245957+0000", 2024-09-14T22:36:18.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7040000000000006e-06 2024-09-14T22:36:18.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:18.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.250889+0000", 2024-09-14T22:36:18.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049324679999999997 2024-09-14T22:36:18.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:18.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.250945+0000", 2024-09-14T22:36:18.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5155999999999998e-05 2024-09-14T22:36:18.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:18.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.253190+0000", 2024-09-14T22:36:18.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0022449530000000001 2024-09-14T22:36:18.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:18.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.253296+0000", 2024-09-14T22:36:18.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106532 2024-09-14T22:36:18.996 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.996 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:18.996 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:18.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.996 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4201.0:141 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'99, mlcod=18'86)", 2024-09-14T22:36:18.996 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:01.247004+0000", 2024-09-14T22:36:18.996 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 17.641518851000001, 2024-09-14T22:36:18.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0070737550000000001, 2024-09-14T22:36:18.997 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:18.997 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:18.997 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:18.997 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:18.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:18.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.247004+0000", 2024-09-14T22:36:18.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:18.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.247004+0000", 2024-09-14T22:36:18.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:18.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:18.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.247006+0000", 2024-09-14T22:36:18.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1659999999999999e-06 2024-09-14T22:36:18.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:18.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.247015+0000", 2024-09-14T22:36:18.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.747e-06 2024-09-14T22:36:18.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:18.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:18.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.247047+0000", 2024-09-14T22:36:18.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1656000000000003e-05 2024-09-14T22:36:18.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:18.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:19.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:19.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.247057+0000", 2024-09-14T22:36:19.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0263e-05 2024-09-14T22:36:19.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:19.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:19.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:19.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.252592+0000", 2024-09-14T22:36:19.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0055350440000000002 2024-09-14T22:36:19.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:19.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:19.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:19.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.252637+0000", 2024-09-14T22:36:19.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4369999999999997e-05 2024-09-14T22:36:19.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:19.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:19.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:19.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.254007+0000", 2024-09-14T22:36:19.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0013700590000000001 2024-09-14T22:36:19.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:19.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:19.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:19.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.254078+0000", 2024-09-14T22:36:19.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1450000000000002e-05 2024-09-14T22:36:19.002 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:19.002 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:19.002 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:19.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:19.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:19.002 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4201.0:143 1.0 e18/16 1:30306672:devicehealth::main.db.0000000000000000:head v 18'101, mlcod=18'98)", 2024-09-14T22:36:19.002 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:01.256378+0000", 2024-09-14T22:36:19.002 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 17.632144920999998, 2024-09-14T22:36:19.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0026272539999999999, 2024-09-14T22:36:19.002 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:19.003 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:19.003 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:19.003 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:19.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:19.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:19.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.256378+0000", 2024-09-14T22:36:19.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:19.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:19.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:19.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:19.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.256378+0000", 2024-09-14T22:36:19.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:19.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:19.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:19.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:19.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.256380+0000", 2024-09-14T22:36:19.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.787e-06 2024-09-14T22:36:19.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:19.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:19.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:19.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.256468+0000", 2024-09-14T22:36:19.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7979999999999995e-05 2024-09-14T22:36:19.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:19.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:19.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:19.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.256492+0000", 2024-09-14T22:36:19.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4136000000000001e-05 2024-09-14T22:36:19.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:19.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:19.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:19.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.256505+0000", 2024-09-14T22:36:19.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.327e-05 2024-09-14T22:36:19.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:19.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:19.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:19.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.256639+0000", 2024-09-14T22:36:19.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013336199999999999 2024-09-14T22:36:19.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:19.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:19.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:19.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.256678+0000", 2024-09-14T22:36:19.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9449999999999997e-05 2024-09-14T22:36:19.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:19.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:19.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:19.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.258928+0000", 2024-09-14T22:36:19.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0022500430000000002 2024-09-14T22:36:19.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:19.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:19.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:19.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.259006+0000", 2024-09-14T22:36:19.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7225999999999995e-05 2024-09-14T22:36:19.007 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:19.008 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:19.008 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:19.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:19.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:19.008 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T22:36:19.008 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:17.716282+0000", 2024-09-14T22:36:19.008 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.172241141, 2024-09-14T22:36:19.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0088192600000000006, 2024-09-14T22:36:19.008 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:19.008 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:19.008 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:19.008 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:19.009 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:19.009 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:19.009 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 11 2024-09-14T22:36:19.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:19.009 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:19.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:19.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:19.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:17.716282+0000", 2024-09-14T22:36:19.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:19.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:19.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:19.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:19.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:17.716282+0000", 2024-09-14T22:36:19.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:19.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:19.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:19.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:19.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:17.716284+0000", 2024-09-14T22:36:19.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1890000000000001e-06 2024-09-14T22:36:19.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:19.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:19.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:19.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:17.716297+0000", 2024-09-14T22:36:19.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2308e-05 2024-09-14T22:36:19.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:19.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:19.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:19.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:17.716317+0000", 2024-09-14T22:36:19.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0018999999999999e-05 2024-09-14T22:36:19.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:19.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:19.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:19.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:17.716327+0000", 2024-09-14T22:36:19.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0086e-05 2024-09-14T22:36:19.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:19.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:19.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:19.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:17.716469+0000", 2024-09-14T22:36:19.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000142084 2024-09-14T22:36:19.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:19.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:19.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2024-09-14T22:36:19.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:17.716480+0000", 2024-09-14T22:36:19.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1086e-05 2024-09-14T22:36:19.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:19.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:19.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:19.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:17.720350+0000", 2024-09-14T22:36:19.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0038704270000000001 2024-09-14T22:36:19.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:19.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:19.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:19.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:17.720559+0000", 2024-09-14T22:36:19.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020853500000000001 2024-09-14T22:36:19.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:19.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:19.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-14T22:36:19.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:17.720884+0000", 2024-09-14T22:36:19.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032519299999999999 2024-09-14T22:36:19.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:19.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:19.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:36:19.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:17.722401+0000", 2024-09-14T22:36:19.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001517191 2024-09-14T22:36:19.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:19.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:19.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:19.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:17.724983+0000", 2024-09-14T22:36:19.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025813889999999999 2024-09-14T22:36:19.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:19.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:19.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:19.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:17.725052+0000", 2024-09-14T22:36:19.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9549999999999996e-05 2024-09-14T22:36:19.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:19.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:19.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:19.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:17.725101+0000", 2024-09-14T22:36:19.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9203000000000003e-05 2024-09-14T22:36:19.016 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:19.016 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:19.016 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:19.016 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:19.017 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:19.017 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:19.017 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:36:19.019 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:36:19.019 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:36:19.014+0000 7f82f11cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26333) UID: 0 2024-09-14T22:36:19.120 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:19.115+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:19.184 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:19.184 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:36:19.184 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:36:19.184 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:19.200 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:36:19.216 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:36:19.216 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:36:19.220 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:19.214+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:19.232 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:36:19.233 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:36:19.249 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:36:19.249 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:36:19.265 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:36:19.265 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:36:19.286 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:36:19.286 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:36:19.306 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:36:19.306 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:36:19.310 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:19.310 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:19.321 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:36:19.314+0000 7f82f11cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26333) UID: 0 2024-09-14T22:36:19.323 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:36:19.327 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:36:19.327 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:36:19.348 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:36:19.349 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:36:19.422 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:36:19.418+0000 7f82f11cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26333) UID: 0 2024-09-14T22:36:19.437 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:19.437 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:19.450 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:36:19.523 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:19.519+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:19.617 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:19.617 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:19.625 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:19.619+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:19.632 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:36:19.691 INFO:teuthology.orchestra.run.smithi001.stdout:1 writesame 0x7c46c14 thru 0x7c48141 data_size 0x1(0x152e bytes) 2024-09-14T22:36:19.700 INFO:teuthology.orchestra.run.smithi001.stdout:3 write 0x87b70f6 thru 0x87bcc7c (0x5b87 bytes) 2024-09-14T22:36:19.707 INFO:teuthology.orchestra.run.smithi001.stdout:truncating to largest ever: 0x273527 2024-09-14T22:36:19.717 INFO:teuthology.orchestra.run.smithi001.stdout:4 trunc from 0x87bcc7d to 0x273527 2024-09-14T22:36:19.725 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:19.718+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:19.737 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T22:36:19.746 INFO:teuthology.orchestra.run.smithi001.stdout:5 write 0x1b6e1b8 thru 0x1b7cdfe (0xec47 bytes) 2024-09-14T22:36:19.752 INFO:teuthology.orchestra.run.smithi001.stdout:6 read 0x16a6b6f thru 0x16a97bb (0x2c4d bytes) 2024-09-14T22:36:19.752 INFO:teuthology.orchestra.run.smithi001.stdout:7 read 0x11840f3 thru 0x118798e (0x389c bytes) 2024-09-14T22:36:19.761 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:19.761 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:19.763 INFO:teuthology.orchestra.run.smithi001.stdout:8 write 0x5407473 thru 0x540c887 (0x5415 bytes) 2024-09-14T22:36:19.767 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:19.767 INFO:teuthology.orchestra.run.smithi001.stdout:truncating to largest ever: 0xcac7d1b 2024-09-14T22:36:19.775 INFO:teuthology.orchestra.run.smithi001.stdout:10 trunc from 0x540c888 to 0xcac7d1b 2024-09-14T22:36:19.781 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:36:19.782 INFO:teuthology.orchestra.run.smithi001.stdout:11 read 0xbd6984f thru 0xbd7122d (0x79df bytes) 2024-09-14T22:36:19.792 INFO:teuthology.orchestra.run.smithi001.stdout:12 write 0xe478b96 thru 0xe479e94 (0x12ff bytes) 2024-09-14T22:36:19.798 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:19.798 INFO:teuthology.orchestra.run.smithi001.stdout:15 write 0x64a156 thru 0x65a0e1 (0xff8c bytes) 2024-09-14T22:36:19.802 INFO:teuthology.orchestra.run.smithi001.stdout:16 punch from 0x1d7e277 to 0x1d8cb44, (0xe8cd bytes) 2024-09-14T22:36:19.802 INFO:teuthology.orchestra.run.smithi001.stdout:17 write 0x9f125e thru 0x9f9218 (0x7fbb bytes) 2024-09-14T22:36:19.806 INFO:teuthology.orchestra.run.smithi001.stdout:18 read 0xd727812 thru 0xd727b15 (0x304 bytes) 2024-09-14T22:36:19.806 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:19.806 INFO:teuthology.orchestra.run.smithi001.stdout:21 punch from 0xa06da49 to 0xa0703d6, (0x298d bytes) 2024-09-14T22:36:19.807 INFO:teuthology.orchestra.run.smithi001.stdout:22 read 0xbf43acb thru 0xbf523c5 (0xe8fb bytes) 2024-09-14T22:36:19.807 INFO:teuthology.orchestra.run.smithi001.stdout:23 write 0x1388c31 thru 0x1395ba9 (0xcf79 bytes) 2024-09-14T22:36:19.812 INFO:teuthology.orchestra.run.smithi001.stdout:24 read 0xa48261c thru 0xa48d4a2 (0xae87 bytes) 2024-09-14T22:36:19.813 INFO:teuthology.orchestra.run.smithi001.stdout:27 punch from 0x8aa0e44 to 0x8aaeedf, (0xe09b bytes) 2024-09-14T22:36:19.813 INFO:teuthology.orchestra.run.smithi001.stdout:28 punch from 0x6460d18 to 0x646c3b4, (0xb69c bytes) 2024-09-14T22:36:19.813 INFO:teuthology.orchestra.run.smithi001.stdout:30 writesame 0xd646296 thru 0xd64d624 data_size 0x39(0x738f bytes) 2024-09-14T22:36:19.817 INFO:teuthology.orchestra.run.smithi001.stdout:31 write 0xb9533d7 thru 0xb95ed32 (0xb95c bytes) 2024-09-14T22:36:19.822 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:19.822 INFO:teuthology.orchestra.run.smithi001.stdout:33 punch from 0x7194ed6 to 0x719a097, (0x51c1 bytes) 2024-09-14T22:36:19.822 INFO:teuthology.orchestra.run.smithi001.stdout:34 write 0x4f0a9b thru 0x4f4954 (0x3eba bytes) 2024-09-14T22:36:19.824 INFO:teuthology.orchestra.run.smithi001.stdout:37 read 0xce830d9 thru 0xce917b6 (0xe6de bytes) 2024-09-14T22:36:19.824 INFO:teuthology.orchestra.run.smithi001.stdout:39 punch from 0x5932d1 to 0x599e4e, (0x6b7d bytes) 2024-09-14T22:36:19.824 INFO:teuthology.orchestra.run.smithi001.stdout:41 read 0x83e7771 thru 0x83ec44e (0x4cde bytes) 2024-09-14T22:36:19.824 INFO:teuthology.orchestra.run.smithi001.stdout:43 read 0x8dd44a8 thru 0x8de1672 (0xd1cb bytes) 2024-09-14T22:36:19.824 INFO:teuthology.orchestra.run.smithi001.stdout:45 read 0x9a6e8e1 thru 0x9a71776 (0x2e96 bytes) 2024-09-14T22:36:19.824 INFO:teuthology.orchestra.run.smithi001.stdout:46 read 0xde7431d thru 0xde7a653 (0x6337 bytes) 2024-09-14T22:36:19.824 INFO:teuthology.orchestra.run.smithi001.stdout:47 write 0xbdfff81 thru 0xbe022fc (0x237c bytes) 2024-09-14T22:36:19.826 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:36:19.818+0000 7f82f11cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26333) UID: 0 2024-09-14T22:36:19.829 INFO:teuthology.orchestra.run.smithi001.stdout:48 write 0x18d9dd3 thru 0x18e552b (0xb759 bytes) 2024-09-14T22:36:19.833 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:19.846 INFO:teuthology.orchestra.run.smithi001.stdout:50 trunc from 0xe479e95 to 0x3a3b1b8 2024-09-14T22:36:19.861 INFO:teuthology.orchestra.run.smithi001.stdout:51 writesame 0x42bd74 thru 0x42f28d data_size 0x7(0x351a bytes) 2024-09-14T22:36:19.863 INFO:teuthology.orchestra.run.smithi001.stdout:52 read 0xfdf10a thru 0xfe17ac (0x26a3 bytes) 2024-09-14T22:36:19.863 INFO:teuthology.orchestra.run.smithi001.stdout:53 punch from 0xe798fb to 0xe79bd0, (0x2d5 bytes) 2024-09-14T22:36:19.866 INFO:teuthology.orchestra.run.smithi001.stdout:54 trunc from 0x3a3b1b8 to 0x67d82eb 2024-09-14T22:36:19.872 INFO:teuthology.orchestra.run.smithi001.stdout:55 writesame 0x53df3b0 thru 0x53e490f data_size 0x10(0x5560 bytes) 2024-09-14T22:36:19.877 INFO:teuthology.orchestra.run.smithi001.stdout:56 punch from 0x170860b to 0x170d829, (0x521e bytes) 2024-09-14T22:36:19.888 INFO:teuthology.orchestra.run.smithi001.stdout:57 write 0x8f07ca2 thru 0x8f15a99 (0xddf8 bytes) 2024-09-14T22:36:19.894 INFO:teuthology.orchestra.run.smithi001.stdout:58 read 0x119cf72 thru 0x11ace1d (0xfeac bytes) 2024-09-14T22:36:19.895 INFO:teuthology.orchestra.run.smithi001.stdout:59 punch from 0x32a16fd to 0x32acefa, (0xb7fd bytes) 2024-09-14T22:36:19.896 INFO:teuthology.orchestra.run.smithi001.stdout:60 read 0x6bf6c86 thru 0x6bfa574 (0x38ef bytes) 2024-09-14T22:36:19.896 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:19.896 INFO:teuthology.orchestra.run.smithi001.stdout:63 writesame 0x465650e thru 0x465f1f1 data_size 0x47(0x8ce4 bytes) 2024-09-14T22:36:19.907 INFO:teuthology.orchestra.run.smithi001.stdout:64 write 0xaaf835f thru 0xab0138d (0x902f bytes) 2024-09-14T22:36:19.912 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:19.912 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:19.925 INFO:teuthology.orchestra.run.smithi001.stdout:65 trunc from 0xab0138e to 0x2b03bf 2024-09-14T22:36:19.926 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:19.922+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:19.936 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:36:19.954 INFO:teuthology.orchestra.run.smithi001.stdout:66 writesame 0x7609918 thru 0x760c301 data_size 0x3a(0x29ea bytes) 2024-09-14T22:36:19.954 INFO:teuthology.orchestra.run.smithi001.stdout:67 writesame 0x58dea34 thru 0x58e88f8 data_size 0xb(0x9ec5 bytes) 2024-09-14T22:36:19.964 INFO:teuthology.orchestra.run.smithi001.stdout:68 trunc from 0x760c302 to 0x8f4141e 2024-09-14T22:36:19.970 INFO:teuthology.orchestra.run.smithi001.stdout:69 write 0x5fd9cc0 thru 0x5fda9aa (0xceb bytes) 2024-09-14T22:36:19.976 INFO:teuthology.orchestra.run.smithi001.stdout:70 trunc from 0x8f4141e to 0xba19074 2024-09-14T22:36:19.983 INFO:teuthology.orchestra.run.smithi001.stdout:71 read 0x9ae0856 thru 0x9aea0c9 (0x9874 bytes) 2024-09-14T22:36:19.983 INFO:teuthology.orchestra.run.smithi001.stdout:74 write 0x43872f8 thru 0x438be4a (0x4b53 bytes) 2024-09-14T22:36:19.987 INFO:teuthology.orchestra.run.smithi001.stdout:76 read 0x6b7fa0d thru 0x6b828ce (0x2ec2 bytes) 2024-09-14T22:36:19.987 INFO:teuthology.orchestra.run.smithi001.stdout:77 writesame 0x98956b0 thru 0x98a387b data_size 0x6(0xe1cc bytes) 2024-09-14T22:36:20.001 INFO:teuthology.orchestra.run.smithi001.stdout:78 writesame 0xcae720a thru 0xcaf05c3 data_size 0x63(0x93ba bytes) 2024-09-14T22:36:20.009 INFO:teuthology.orchestra.run.smithi001.stdout:80 write 0xe07455f thru 0xe07b81f (0x72c1 bytes) 2024-09-14T22:36:20.014 INFO:teuthology.orchestra.run.smithi001.stdout:81 writesame 0x3ec9c6a thru 0x3ed89eb data_size 0x2b(0xed82 bytes) 2024-09-14T22:36:20.019 INFO:teuthology.orchestra.run.smithi001.stdout:82 trunc from 0xe07b820 to 0xbfd81ec 2024-09-14T22:36:20.027 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:20.022+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:20.031 INFO:teuthology.orchestra.run.smithi001.stdout:83 write 0x86ad8bd thru 0x86b5f63 (0x86a7 bytes) 2024-09-14T22:36:20.036 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:20.036 INFO:teuthology.orchestra.run.smithi001.stdout:87 read 0x4af9b54 thru 0x4b08a5a (0xef07 bytes) 2024-09-14T22:36:20.036 INFO:teuthology.orchestra.run.smithi001.stdout:89 write 0x1554a17 thru 0x155adac (0x6396 bytes) 2024-09-14T22:36:20.040 INFO:teuthology.orchestra.run.smithi001.stdout:91 write 0x6a00801 thru 0x6a02a85 (0x2285 bytes) 2024-09-14T22:36:20.044 INFO:teuthology.orchestra.run.smithi001.stdout:94 clone 1 order 24 su 65536 sc 6 2024-09-14T22:36:20.077 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:20.078 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:20.090 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:36:20.116 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T22:36:20.116 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:36:20.116 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-14T22:36:20.116 INFO:tasks.thrashosds.thrasher:Setting osd 3 primary_affinity to 0.852767 2024-09-14T22:36:20.117 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 0.8527674218472764 2024-09-14T22:36:20.135 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:36:20.130+0000 7f82f11cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26333) UID: 0 2024-09-14T22:36:20.230 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:20.227+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:20.264 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:20.265 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:20.277 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:36:20.333 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:36:20.326+0000 7f82f11cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26333) UID: 0 2024-09-14T22:36:20.388 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:20.389 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:20.401 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:36:20.447 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:20.443+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:20.536 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:20.536 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:20.538 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:20.530+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:20.548 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:36:20.639 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:20.635+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:20.712 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2024-09-14T22:36:20.712 INFO:teuthology.orchestra.run.smithi001.stderr:set osd.3 primary-affinity to 0.852767 (8558862) 2024-09-14T22:36:20.714 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:20.714 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:20.731 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:36:20.775 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:20.771+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:20.841 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:20.834+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:20.899 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:20.899 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:20.913 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:36:20.942 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:20.934+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:21.032 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:21.032 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:21.043 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:36:21.038+0000 7f82f11cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26333) UID: 0 2024-09-14T22:36:21.047 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:36:21.144 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:36:21.138+0000 7f82f11cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26333) UID: 0 2024-09-14T22:36:21.168 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:21.168 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:21.182 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:36:21.245 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:21.239+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:21.306 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:21.306 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:21.320 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:36:21.363 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:21.359+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:21.365 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:36:21.447 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:21.443+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:21.488 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:21.488 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:21.503 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:36:21.523 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:36:21.524 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:36:21.540 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:36:21.540 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:36:21.548 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:21.542+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:21.556 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:36:21.557 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:36:21.573 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:36:21.573 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:36:21.591 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:36:21.591 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:36:21.649 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:36:21.646+0000 7f82f11cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26333) UID: 0 2024-09-14T22:36:21.669 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:21.670 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:21.684 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:36:21.749 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:21.742+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:21.812 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:21.812 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:21.817 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:36:21.818 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:36:21.826 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:36:21.844 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:36:21.844 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:36:21.868 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:36:21.869 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:36:21.871 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:21.867+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:21.949 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:21.949 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:21.949 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:36:21.949 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:21.952 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:21.947+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:21.964 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:36:21.976 INFO:teuthology.orchestra.run.smithi001.stdout:95 write 0x4e4acf7 thru 0x4e4b568 (0x872 bytes) 2024-09-14T22:36:21.986 INFO:teuthology.orchestra.run.smithi001.stdout:96 read 0x48bfa18 thru 0x48c4a5b (0x5044 bytes) 2024-09-14T22:36:21.986 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:21.993 INFO:teuthology.orchestra.run.smithi001.stdout:98 trunc from 0xbfd81ec to 0x78614af 2024-09-14T22:36:22.030 INFO:teuthology.orchestra.run.smithi001.stdout:99 read 0x3f545fb thru 0x3f56f2c (0x2932 bytes) 2024-09-14T22:36:22.038 INFO:teuthology.orchestra.run.smithi001.stdout:100 trunc from 0x78614af to 0x3b1ba35 2024-09-14T22:36:22.055 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:22.051+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:22.086 INFO:teuthology.orchestra.run.smithi001.stdout:101 trunc from 0x3b1ba35 to 0xa9e9eaa 2024-09-14T22:36:22.093 INFO:teuthology.orchestra.run.smithi001.stdout:102 read 0x7bbd0c1 thru 0x7bc216a (0x50aa bytes) 2024-09-14T22:36:22.093 INFO:teuthology.orchestra.run.smithi001.stdout:103 writesame 0x43f9d47 thru 0x44034c5 data_size 0x1(0x977f bytes) 2024-09-14T22:36:22.115 INFO:teuthology.orchestra.run.smithi001.stdout:104 write 0x1037375 thru 0x103f0d8 (0x7d64 bytes) 2024-09-14T22:36:22.118 INFO:teuthology.orchestra.run.smithi001.stdout:105 write 0x3e7aeeb thru 0x3e7c67d (0x1793 bytes) 2024-09-14T22:36:22.121 INFO:teuthology.orchestra.run.smithi001.stdout:106 writesame 0x2233fe8 thru 0x223bd8f data_size 0x4(0x7da8 bytes) 2024-09-14T22:36:22.130 INFO:teuthology.orchestra.run.smithi001.stdout:107 punch from 0x75d655a to 0x75df5b1, (0x9057 bytes) 2024-09-14T22:36:22.133 INFO:teuthology.orchestra.run.smithi001.stdout:109 trunc from 0xa9e9eaa to 0x90e48b0 2024-09-14T22:36:22.138 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:22.138 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:22.138 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:36:22.138 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:36:22.138 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:22.142 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:22.153 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:36:22.154 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:22.150+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:22.164 INFO:teuthology.orchestra.run.smithi001.stdout:113 trunc from 0x90e48b0 to 0x2af7d5 2024-09-14T22:36:22.178 INFO:teuthology.orchestra.run.smithi001.stdout:114 trunc from 0x2af7d5 to 0x63196a 2024-09-14T22:36:22.189 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:22.190 INFO:teuthology.orchestra.run.smithi001.stdout:116 punch from 0x1a59bf to 0x1b30b5, (0xd6f6 bytes) 2024-09-14T22:36:22.190 INFO:teuthology.orchestra.run.smithi001.stdout:117 read 0xf3bac thru 0x10103e (0xd493 bytes) 2024-09-14T22:36:22.191 INFO:teuthology.orchestra.run.smithi001.stdout:118 read 0x1886a2 thru 0x19856c (0xfecb bytes) 2024-09-14T22:36:22.234 INFO:teuthology.orchestra.run.smithi001.stdout:119 writesame 0xb6cef3e thru 0xb6d698c data_size 0x47(0x7a4f bytes) 2024-09-14T22:36:22.234 INFO:teuthology.orchestra.run.smithi001.stdout:120 punch from 0x7b609ed to 0x7b6b739, (0xad4c bytes) 2024-09-14T22:36:22.234 INFO:teuthology.orchestra.run.smithi001.stdout:122 punch from 0x2d91877 to 0x2da0eef, (0xf678 bytes) 2024-09-14T22:36:22.234 INFO:teuthology.orchestra.run.smithi001.stdout:124 punch from 0xb2a9b8e to 0xb2b1335, (0x77a7 bytes) 2024-09-14T22:36:22.234 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:22.234 INFO:teuthology.orchestra.run.smithi001.stdout:128 trunc from 0xb6d698d to 0xb2570b9 2024-09-14T22:36:22.247 INFO:teuthology.orchestra.run.smithi001.stdout:129 read 0x558c028 thru 0x5592b27 (0x6b00 bytes) 2024-09-14T22:36:22.248 INFO:teuthology.orchestra.run.smithi001.stdout:130 write 0x17547b4 thru 0x1756a27 (0x2274 bytes) 2024-09-14T22:36:22.251 INFO:teuthology.orchestra.run.smithi001.stdout:131 writesame 0x3fb54b5 thru 0x3fc4e1e data_size 0x2(0xf96a bytes) 2024-09-14T22:36:22.254 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:36:22.250+0000 7f82f11cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26333) UID: 0 2024-09-14T22:36:22.293 INFO:teuthology.orchestra.run.smithi001.stdout:132 punch from 0x8c832d7 to 0x8c91c20, (0xe949 bytes) 2024-09-14T22:36:22.293 INFO:teuthology.orchestra.run.smithi001.stdout:134 read 0x902ef0b thru 0x9034c56 (0x5d4c bytes) 2024-09-14T22:36:22.293 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:22.293 INFO:teuthology.orchestra.run.smithi001.stdout:137 read 0x1c447e thru 0x1cbeb4 (0x7a37 bytes) 2024-09-14T22:36:22.319 INFO:teuthology.orchestra.run.smithi001.stdout:138 write 0xe703504 thru 0xe703aa8 (0x5a5 bytes) 2024-09-14T22:36:22.327 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:22.327 INFO:teuthology.orchestra.run.smithi001.stdout:140 punch from 0xb7eb937 to 0xb7f5b0d, (0xa1d6 bytes) 2024-09-14T22:36:22.327 INFO:teuthology.orchestra.run.smithi001.stdout:142 read 0x30e05d7 thru 0x30eedad (0xe7d7 bytes) 2024-09-14T22:36:22.328 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:22.328 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:36:22.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:36:22.328 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:36:22.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.328 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:36:22.328 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:36:22.328 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 22.316463545000001, 2024-09-14T22:36:22.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.398029152, 2024-09-14T22:36:22.329 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:22.329 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:22.329 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:22.329 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:22.329 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:36:22.329 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:36:22.329 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 1 2024-09-14T22:36:22.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.329 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:22.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:22.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:36:22.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:22.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:36:22.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:22.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006183+0000", 2024-09-14T22:36:22.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9059999999999998e-06 2024-09-14T22:36:22.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:22.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006198+0000", 2024-09-14T22:36:22.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.523e-05 2024-09-14T22:36:22.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:22.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006219+0000", 2024-09-14T22:36:22.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1342999999999999e-05 2024-09-14T22:36:22.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:22.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006240+0000", 2024-09-14T22:36:22.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1186999999999999e-05 2024-09-14T22:36:22.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:22.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006363+0000", 2024-09-14T22:36:22.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012258200000000001 2024-09-14T22:36:22.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:36:22.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006392+0000", 2024-09-14T22:36:22.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9607e-05 2024-09-14T22:36:22.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:22.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401084+0000", 2024-09-14T22:36:22.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.39469142699999998 2024-09-14T22:36:22.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:22.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401394+0000", 2024-09-14T22:36:22.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030987799999999998 2024-09-14T22:36:22.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:22.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401860+0000", 2024-09-14T22:36:22.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00046654299999999998 2024-09-14T22:36:22.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:22.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.403386+0000", 2024-09-14T22:36:22.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001525322 2024-09-14T22:36:22.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:22.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404042+0000", 2024-09-14T22:36:22.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065676099999999996 2024-09-14T22:36:22.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:22.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404136+0000", 2024-09-14T22:36:22.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4032999999999999e-05 2024-09-14T22:36:22.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:22.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404209+0000", 2024-09-14T22:36:22.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2333e-05 2024-09-14T22:36:22.335 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.335 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:22.335 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.335 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:36:22.335 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.738662+0000", 2024-09-14T22:36:22.335 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 21.583981352999999, 2024-09-14T22:36:22.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018194993999999999, 2024-09-14T22:36:22.335 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:22.335 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:22.336 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:22.336 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:22.336 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:36:22.336 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:36:22.336 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 113 2024-09-14T22:36:22.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.336 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:22.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:22.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738662+0000", 2024-09-14T22:36:22.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:22.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738662+0000", 2024-09-14T22:36:22.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:22.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738663+0000", 2024-09-14T22:36:22.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0300000000000001e-06 2024-09-14T22:36:22.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:22.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738669+0000", 2024-09-14T22:36:22.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.037e-06 2024-09-14T22:36:22.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:22.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738683+0000", 2024-09-14T22:36:22.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4012999999999999e-05 2024-09-14T22:36:22.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:22.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738691+0000", 2024-09-14T22:36:22.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9100000000000005e-06 2024-09-14T22:36:22.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:22.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.751940+0000", 2024-09-14T22:36:22.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013248898 2024-09-14T22:36:22.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:22.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.752098+0000", 2024-09-14T22:36:22.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015850200000000001 2024-09-14T22:36:22.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:22.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.752304+0000", 2024-09-14T22:36:22.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020529500000000001 2024-09-14T22:36:22.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:22.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.756560+0000", 2024-09-14T22:36:22.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0042563410000000003 2024-09-14T22:36:22.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:22.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.756605+0000", 2024-09-14T22:36:22.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4863000000000001e-05 2024-09-14T22:36:22.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:22.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.756747+0000", 2024-09-14T22:36:22.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014277199999999999 2024-09-14T22:36:22.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:22.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.756857+0000", 2024-09-14T22:36:22.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010933300000000001 2024-09-14T22:36:22.341 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.341 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:22.341 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.341 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:36:22.341 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.738705+0000", 2024-09-14T22:36:22.341 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 21.58393835, 2024-09-14T22:36:22.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.019871403999999999, 2024-09-14T22:36:22.341 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:22.341 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:22.341 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:22.341 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:22.341 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:36:22.342 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:36:22.342 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 114 2024-09-14T22:36:22.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.342 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:22.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:22.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738705+0000", 2024-09-14T22:36:22.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:22.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738705+0000", 2024-09-14T22:36:22.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:22.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738706+0000", 2024-09-14T22:36:22.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3e-07 2024-09-14T22:36:22.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:22.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738711+0000", 2024-09-14T22:36:22.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.767e-06 2024-09-14T22:36:22.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:22.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738745+0000", 2024-09-14T22:36:22.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3362999999999999e-05 2024-09-14T22:36:22.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:22.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738754+0000", 2024-09-14T22:36:22.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2439999999999999e-06 2024-09-14T22:36:22.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:22.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.753553+0000", 2024-09-14T22:36:22.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014799132 2024-09-14T22:36:22.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:22.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.753763+0000", 2024-09-14T22:36:22.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021000200000000001 2024-09-14T22:36:22.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:22.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.754101+0000", 2024-09-14T22:36:22.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000337664 2024-09-14T22:36:22.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:22.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.756983+0000", 2024-09-14T22:36:22.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0028820180000000001 2024-09-14T22:36:22.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:22.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.758300+0000", 2024-09-14T22:36:22.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0013173060000000001 2024-09-14T22:36:22.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:22.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.758463+0000", 2024-09-14T22:36:22.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000162752 2024-09-14T22:36:22.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:22.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.758576+0000", 2024-09-14T22:36:22.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011342599999999999 2024-09-14T22:36:22.346 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.347 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:22.347 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.347 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:36:22.347 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.738768+0000", 2024-09-14T22:36:22.347 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 21.58387484, 2024-09-14T22:36:22.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020973018, 2024-09-14T22:36:22.347 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:22.347 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:22.347 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:22.347 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:22.347 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:36:22.347 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:36:22.347 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 115 2024-09-14T22:36:22.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.348 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:22.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:22.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738768+0000", 2024-09-14T22:36:22.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:22.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738768+0000", 2024-09-14T22:36:22.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:22.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738769+0000", 2024-09-14T22:36:22.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.23e-07 2024-09-14T22:36:22.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:22.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738945+0000", 2024-09-14T22:36:22.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017596199999999999 2024-09-14T22:36:22.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:22.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738974+0000", 2024-09-14T22:36:22.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9183000000000002e-05 2024-09-14T22:36:22.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:22.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.738995+0000", 2024-09-14T22:36:22.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0934e-05 2024-09-14T22:36:22.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:22.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.755191+0000", 2024-09-14T22:36:22.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.016195378 2024-09-14T22:36:22.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:22.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.755331+0000", 2024-09-14T22:36:22.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014080599999999999 2024-09-14T22:36:22.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:22.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.755543+0000", 2024-09-14T22:36:22.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021108800000000001 2024-09-14T22:36:22.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:22.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.759583+0000", 2024-09-14T22:36:22.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0040401020000000003 2024-09-14T22:36:22.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:22.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.759603+0000", 2024-09-14T22:36:22.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0803000000000001e-05 2024-09-14T22:36:22.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:22.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.759673+0000", 2024-09-14T22:36:22.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9900000000000005e-05 2024-09-14T22:36:22.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:22.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.759741+0000", 2024-09-14T22:36:22.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8039000000000004e-05 2024-09-14T22:36:22.352 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.352 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:22.352 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.353 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:36:22.353 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.739034+0000", 2024-09-14T22:36:22.353 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 21.583609418999998, 2024-09-14T22:36:22.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021026605, 2024-09-14T22:36:22.353 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:22.353 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:22.353 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:22.353 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:22.353 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:36:22.353 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:36:22.353 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 116 2024-09-14T22:36:22.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.354 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:22.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:22.354 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.739034+0000", 2024-09-14T22:36:22.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:22.354 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.739034+0000", 2024-09-14T22:36:22.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:22.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.739036+0000", 2024-09-14T22:36:22.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0040000000000002e-06 2024-09-14T22:36:22.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:22.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.739048+0000", 2024-09-14T22:36:22.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1793e-05 2024-09-14T22:36:22.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:22.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.739068+0000", 2024-09-14T22:36:22.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0006000000000001e-05 2024-09-14T22:36:22.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.356 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:22.356 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.739090+0000", 2024-09-14T22:36:22.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2087e-05 2024-09-14T22:36:22.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.356 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:22.356 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.757100+0000", 2024-09-14T22:36:22.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018010004 2024-09-14T22:36:22.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:22.357 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.757308+0000", 2024-09-14T22:36:22.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020808600000000001 2024-09-14T22:36:22.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:22.357 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.757601+0000", 2024-09-14T22:36:22.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029366400000000001 2024-09-14T22:36:22.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:22.357 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.759771+0000", 2024-09-14T22:36:22.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0021695400000000002 2024-09-14T22:36:22.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:22.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.759863+0000", 2024-09-14T22:36:22.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2388999999999999e-05 2024-09-14T22:36:22.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:22.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.759955+0000", 2024-09-14T22:36:22.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1608999999999997e-05 2024-09-14T22:36:22.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:22.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.760060+0000", 2024-09-14T22:36:22.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105423 2024-09-14T22:36:22.359 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.359 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:22.359 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.359 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:36:22.359 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.739135+0000", 2024-09-14T22:36:22.359 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 21.583507903000001, 2024-09-14T22:36:22.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021794035999999999, 2024-09-14T22:36:22.359 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:22.359 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:22.359 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:22.360 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:22.360 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:36:22.360 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:36:22.360 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117 2024-09-14T22:36:22.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.360 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:22.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:22.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.739135+0000", 2024-09-14T22:36:22.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:22.361 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.739135+0000", 2024-09-14T22:36:22.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:22.361 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.739137+0000", 2024-09-14T22:36:22.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3069999999999999e-06 2024-09-14T22:36:22.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:22.361 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.739148+0000", 2024-09-14T22:36:22.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1077000000000001e-05 2024-09-14T22:36:22.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:22.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.739180+0000", 2024-09-14T22:36:22.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2356e-05 2024-09-14T22:36:22.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:22.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.739221+0000", 2024-09-14T22:36:22.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0568999999999997e-05 2024-09-14T22:36:22.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:22.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.758699+0000", 2024-09-14T22:36:22.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.019478763999999999 2024-09-14T22:36:22.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:22.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.758835+0000", 2024-09-14T22:36:22.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013597899999999999 2024-09-14T22:36:22.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:22.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.759049+0000", 2024-09-14T22:36:22.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021361399999999999 2024-09-14T22:36:22.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:22.364 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.760246+0000", 2024-09-14T22:36:22.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0011972249999999999 2024-09-14T22:36:22.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:22.364 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.760790+0000", 2024-09-14T22:36:22.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00054360900000000002 2024-09-14T22:36:22.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:22.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.760870+0000", 2024-09-14T22:36:22.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0389000000000006e-05 2024-09-14T22:36:22.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:22.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.760929+0000", 2024-09-14T22:36:22.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9147000000000001e-05 2024-09-14T22:36:22.365 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.365 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:22.365 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.366 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:603 3.11 3:88cdb54f:::rbd_data.113cec2505f0.0000000000000021:3 [sparse-read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T22:36:22.366 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:21.998748+0000", 2024-09-14T22:36:22.366 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.32389555599999997, 2024-09-14T22:36:22.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017501243, 2024-09-14T22:36:22.366 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:22.366 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:22.366 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:22.366 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:22.366 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:22.366 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:22.366 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 603 2024-09-14T22:36:22.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.367 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:22.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:22.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998748+0000", 2024-09-14T22:36:22.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:22.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998748+0000", 2024-09-14T22:36:22.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:22.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998748+0000", 2024-09-14T22:36:22.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9299999999999999e-07 2024-09-14T22:36:22.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:22.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998755+0000", 2024-09-14T22:36:22.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3649999999999998e-06 2024-09-14T22:36:22.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:22.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998760+0000", 2024-09-14T22:36:22.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3109999999999999e-06 2024-09-14T22:36:22.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:22.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998763+0000", 2024-09-14T22:36:22.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.977e-06 2024-09-14T22:36:22.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:22.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.016119+0000", 2024-09-14T22:36:22.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017356598000000001 2024-09-14T22:36:22.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:22.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.016173+0000", 2024-09-14T22:36:22.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3396999999999997e-05 2024-09-14T22:36:22.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:22.370 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.016249+0000", 2024-09-14T22:36:22.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6302000000000002e-05 2024-09-14T22:36:22.370 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.370 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:22.370 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.370 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:604 3.11 3:88cdb54f:::rbd_data.113cec2505f0.0000000000000021:3 [sparse-read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T22:36:22.370 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:21.998801+0000", 2024-09-14T22:36:22.370 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.32384196799999998, 2024-09-14T22:36:22.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018019774999999998, 2024-09-14T22:36:22.370 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:22.370 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:22.371 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:22.371 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:22.371 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:22.371 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:22.371 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 604 2024-09-14T22:36:22.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.371 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:22.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:22.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998801+0000", 2024-09-14T22:36:22.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:22.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998801+0000", 2024-09-14T22:36:22.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:22.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998802+0000", 2024-09-14T22:36:22.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0400000000000002e-07 2024-09-14T22:36:22.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:22.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998803+0000", 2024-09-14T22:36:22.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4929999999999999e-06 2024-09-14T22:36:22.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:22.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998807+0000", 2024-09-14T22:36:22.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.856e-06 2024-09-14T22:36:22.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:22.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998810+0000", 2024-09-14T22:36:22.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0230000000000001e-06 2024-09-14T22:36:22.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:22.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.016339+0000", 2024-09-14T22:36:22.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017528873 2024-09-14T22:36:22.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.374 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:22.374 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.016562+0000", 2024-09-14T22:36:22.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022315200000000001 2024-09-14T22:36:22.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.374 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:22.374 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.016821+0000", 2024-09-14T22:36:22.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00025897399999999998 2024-09-14T22:36:22.374 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.374 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:22.374 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.374 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:605 3.11 3:88cdb54f:::rbd_data.113cec2505f0.0000000000000021:3 [sparse-read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T22:36:22.375 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:21.998814+0000", 2024-09-14T22:36:22.375 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.32382938100000003, 2024-09-14T22:36:22.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018229691999999999, 2024-09-14T22:36:22.375 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:22.375 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:22.375 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:22.375 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:22.375 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:22.375 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:22.375 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 605 2024-09-14T22:36:22.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.375 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:22.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:22.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998814+0000", 2024-09-14T22:36:22.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:22.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998814+0000", 2024-09-14T22:36:22.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:22.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998814+0000", 2024-09-14T22:36:22.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1199999999999999e-07 2024-09-14T22:36:22.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:22.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998815+0000", 2024-09-14T22:36:22.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1379999999999999e-06 2024-09-14T22:36:22.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:22.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998818+0000", 2024-09-14T22:36:22.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2380000000000002e-06 2024-09-14T22:36:22.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:22.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998826+0000", 2024-09-14T22:36:22.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2919999999999998e-06 2024-09-14T22:36:22.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:22.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.016888+0000", 2024-09-14T22:36:22.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018062575000000001 2024-09-14T22:36:22.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:22.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.016961+0000", 2024-09-14T22:36:22.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3040999999999994e-05 2024-09-14T22:36:22.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:22.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.017043+0000", 2024-09-14T22:36:22.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2195999999999997e-05 2024-09-14T22:36:22.380 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.380 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:22.380 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.380 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:606 3.11 3:88cdb54f:::rbd_data.113cec2505f0.0000000000000021:3 [sparse-read 3801088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T22:36:22.380 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:21.998843+0000", 2024-09-14T22:36:22.380 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.32380053800000003, 2024-09-14T22:36:22.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018386486, 2024-09-14T22:36:22.381 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:22.381 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:22.381 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:22.381 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:22.381 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:22.381 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:22.381 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 606 2024-09-14T22:36:22.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.381 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:22.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:22.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998843+0000", 2024-09-14T22:36:22.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:22.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998843+0000", 2024-09-14T22:36:22.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:22.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998843+0000", 2024-09-14T22:36:22.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5499999999999998e-07 2024-09-14T22:36:22.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:22.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998845+0000", 2024-09-14T22:36:22.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5030000000000001e-06 2024-09-14T22:36:22.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:22.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998849+0000", 2024-09-14T22:36:22.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0539999999999996e-06 2024-09-14T22:36:22.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:22.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998856+0000", 2024-09-14T22:36:22.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2559999999999996e-06 2024-09-14T22:36:22.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:22.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.017083+0000", 2024-09-14T22:36:22.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018226735000000001 2024-09-14T22:36:22.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:22.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.017151+0000", 2024-09-14T22:36:22.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8638000000000003e-05 2024-09-14T22:36:22.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:22.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.017229+0000", 2024-09-14T22:36:22.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7844999999999998e-05 2024-09-14T22:36:22.385 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.385 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:22.385 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.385 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:629 3.11 3:88cdb54f:::rbd_data.113cec2505f0.0000000000000021:3 [sparse-read 327680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T22:36:22.385 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:21.998880+0000", 2024-09-14T22:36:22.385 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.32376359199999999, 2024-09-14T22:36:22.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018572608000000001, 2024-09-14T22:36:22.385 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:22.385 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:22.385 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:22.386 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:22.386 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:22.386 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:22.386 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 629 2024-09-14T22:36:22.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.386 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:22.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:22.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998880+0000", 2024-09-14T22:36:22.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:22.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998880+0000", 2024-09-14T22:36:22.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:22.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998880+0000", 2024-09-14T22:36:22.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4799999999999999e-07 2024-09-14T22:36:22.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:22.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998882+0000", 2024-09-14T22:36:22.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3870000000000002e-06 2024-09-14T22:36:22.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:22.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998890+0000", 2024-09-14T22:36:22.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1620000000000002e-06 2024-09-14T22:36:22.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:22.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998895+0000", 2024-09-14T22:36:22.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9760000000000003e-06 2024-09-14T22:36:22.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:22.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.017265+0000", 2024-09-14T22:36:22.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018370173 2024-09-14T22:36:22.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:22.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.017318+0000", 2024-09-14T22:36:22.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3081999999999998e-05 2024-09-14T22:36:22.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:22.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.017452+0000", 2024-09-14T22:36:22.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013438 2024-09-14T22:36:22.389 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.389 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:22.389 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.389 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:630 3.11 3:88cdb54f:::rbd_data.113cec2505f0.0000000000000021:3 [sparse-read 720896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T22:36:22.389 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:21.998900+0000", 2024-09-14T22:36:22.390 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.32374328800000002, 2024-09-14T22:36:22.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018748135999999999, 2024-09-14T22:36:22.390 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:22.390 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:22.390 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:22.390 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:22.390 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:22.390 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:22.390 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 630 2024-09-14T22:36:22.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.390 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:22.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:22.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998900+0000", 2024-09-14T22:36:22.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:22.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998900+0000", 2024-09-14T22:36:22.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:22.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998900+0000", 2024-09-14T22:36:22.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0899999999999997e-07 2024-09-14T22:36:22.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:22.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998902+0000", 2024-09-14T22:36:22.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4309999999999999e-06 2024-09-14T22:36:22.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:22.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998905+0000", 2024-09-14T22:36:22.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.737e-06 2024-09-14T22:36:22.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:22.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998908+0000", 2024-09-14T22:36:22.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5390000000000002e-06 2024-09-14T22:36:22.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:22.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.017491+0000", 2024-09-14T22:36:22.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018583054000000002 2024-09-14T22:36:22.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:22.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.017547+0000", 2024-09-14T22:36:22.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5702000000000003e-05 2024-09-14T22:36:22.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:22.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.017648+0000", 2024-09-14T22:36:22.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101364 2024-09-14T22:36:22.393 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.394 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:22.394 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.394 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:631 3.11 3:88cdb54f:::rbd_data.113cec2505f0.0000000000000021:3 [sparse-read 1114112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T22:36:22.394 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:21.998911+0000", 2024-09-14T22:36:22.394 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.32373176999999997, 2024-09-14T22:36:22.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018998009, 2024-09-14T22:36:22.394 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:22.394 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:22.394 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:22.394 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:22.394 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:22.394 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:22.395 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 631 2024-09-14T22:36:22.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.395 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:22.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:22.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998911+0000", 2024-09-14T22:36:22.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:22.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998911+0000", 2024-09-14T22:36:22.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:22.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998912+0000", 2024-09-14T22:36:22.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1899999999999999e-07 2024-09-14T22:36:22.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:22.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998913+0000", 2024-09-14T22:36:22.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1650000000000001e-06 2024-09-14T22:36:22.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:22.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998916+0000", 2024-09-14T22:36:22.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.241e-06 2024-09-14T22:36:22.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:22.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998919+0000", 2024-09-14T22:36:22.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6649999999999999e-06 2024-09-14T22:36:22.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:22.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.017701+0000", 2024-09-14T22:36:22.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018782554999999999 2024-09-14T22:36:22.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:22.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.017789+0000", 2024-09-14T22:36:22.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7715999999999998e-05 2024-09-14T22:36:22.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:22.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.017909+0000", 2024-09-14T22:36:22.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120448 2024-09-14T22:36:22.398 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.398 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:22.398 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.399 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:632 3.11 3:88cdb54f:::rbd_data.113cec2505f0.0000000000000021:3 [sparse-read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T22:36:22.399 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:21.998922+0000", 2024-09-14T22:36:22.399 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.323721127, 2024-09-14T22:36:22.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.019183357000000002, 2024-09-14T22:36:22.399 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:22.399 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:22.399 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:22.399 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:22.399 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:22.399 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:22.399 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 632 2024-09-14T22:36:22.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.399 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:22.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:22.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998922+0000", 2024-09-14T22:36:22.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:22.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998922+0000", 2024-09-14T22:36:22.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:22.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998922+0000", 2024-09-14T22:36:22.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3300000000000001e-07 2024-09-14T22:36:22.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:22.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998924+0000", 2024-09-14T22:36:22.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.215e-06 2024-09-14T22:36:22.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:22.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998929+0000", 2024-09-14T22:36:22.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2449999999999998e-06 2024-09-14T22:36:22.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:22.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998933+0000", 2024-09-14T22:36:22.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4390000000000002e-06 2024-09-14T22:36:22.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:22.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.017952+0000", 2024-09-14T22:36:22.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.019018692 2024-09-14T22:36:22.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:22.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.018012+0000", 2024-09-14T22:36:22.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9815000000000002e-05 2024-09-14T22:36:22.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:22.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.018105+0000", 2024-09-14T22:36:22.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3718e-05 2024-09-14T22:36:22.403 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.403 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:22.403 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.403 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:633 3.11 3:88cdb54f:::rbd_data.113cec2505f0.0000000000000021:3 [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T22:36:22.403 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:21.998972+0000", 2024-09-14T22:36:22.403 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.32367110799999999, 2024-09-14T22:36:22.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.019307583999999999, 2024-09-14T22:36:22.403 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:22.403 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:22.403 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:22.403 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:22.404 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:22.404 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:22.404 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 633 2024-09-14T22:36:22.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.404 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:22.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:22.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998972+0000", 2024-09-14T22:36:22.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:22.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998972+0000", 2024-09-14T22:36:22.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:22.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998973+0000", 2024-09-14T22:36:22.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5400000000000002e-07 2024-09-14T22:36:22.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:22.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998974+0000", 2024-09-14T22:36:22.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5099999999999999e-06 2024-09-14T22:36:22.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:22.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998978+0000", 2024-09-14T22:36:22.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2640000000000002e-06 2024-09-14T22:36:22.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:22.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998981+0000", 2024-09-14T22:36:22.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0319999999999999e-06 2024-09-14T22:36:22.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:22.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.018147+0000", 2024-09-14T22:36:22.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.019165762999999999 2024-09-14T22:36:22.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:22.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.018204+0000", 2024-09-14T22:36:22.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6913999999999998e-05 2024-09-14T22:36:22.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:22.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.018280+0000", 2024-09-14T22:36:22.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5647000000000003e-05 2024-09-14T22:36:22.407 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.407 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:22.407 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.407 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:634 3.11 3:88cdb54f:::rbd_data.113cec2505f0.0000000000000021:3 [sparse-read 2293760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T22:36:22.407 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:21.998985+0000", 2024-09-14T22:36:22.407 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.32365806699999999, 2024-09-14T22:36:22.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01945906, 2024-09-14T22:36:22.408 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:22.408 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:22.408 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:22.408 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:22.408 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:22.408 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:22.408 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 634 2024-09-14T22:36:22.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.408 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:22.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:22.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998985+0000", 2024-09-14T22:36:22.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:22.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998985+0000", 2024-09-14T22:36:22.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:22.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998985+0000", 2024-09-14T22:36:22.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.28e-07 2024-09-14T22:36:22.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:22.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998990+0000", 2024-09-14T22:36:22.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.967e-06 2024-09-14T22:36:22.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:22.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998994+0000", 2024-09-14T22:36:22.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7579999999999998e-06 2024-09-14T22:36:22.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:22.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.998997+0000", 2024-09-14T22:36:22.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8439999999999998e-06 2024-09-14T22:36:22.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:22.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.018317+0000", 2024-09-14T22:36:22.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.019320284 2024-09-14T22:36:22.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:22.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.018370+0000", 2024-09-14T22:36:22.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2861000000000001e-05 2024-09-14T22:36:22.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:22.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.018444+0000", 2024-09-14T22:36:22.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4117999999999997e-05 2024-09-14T22:36:22.411 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.411 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:22.411 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.412 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:635 3.11 3:88cdb54f:::rbd_data.113cec2505f0.0000000000000021:3 [sparse-read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T22:36:22.412 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:21.999012+0000", 2024-09-14T22:36:22.412 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.32363118800000001, 2024-09-14T22:36:22.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.019597217, 2024-09-14T22:36:22.412 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:22.412 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:22.412 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:22.412 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:22.412 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:22.412 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:22.412 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 635 2024-09-14T22:36:22.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.412 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:22.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:22.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.999012+0000", 2024-09-14T22:36:22.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:22.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.999012+0000", 2024-09-14T22:36:22.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:22.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.999013+0000", 2024-09-14T22:36:22.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3499999999999996e-07 2024-09-14T22:36:22.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:22.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.999015+0000", 2024-09-14T22:36:22.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2079999999999999e-06 2024-09-14T22:36:22.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:22.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.999021+0000", 2024-09-14T22:36:22.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2920000000000001e-06 2024-09-14T22:36:22.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:22.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.999026+0000", 2024-09-14T22:36:22.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6280000000000004e-06 2024-09-14T22:36:22.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:22.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.018483+0000", 2024-09-14T22:36:22.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.019457447999999999 2024-09-14T22:36:22.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:22.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.018536+0000", 2024-09-14T22:36:22.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2534000000000002e-05 2024-09-14T22:36:22.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:22.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.018609+0000", 2024-09-14T22:36:22.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3472000000000004e-05 2024-09-14T22:36:22.415 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.415 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:22.416 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.416 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:636 3.11 3:88cdb54f:::rbd_data.113cec2505f0.0000000000000021:3 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T22:36:22.416 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:21.999049+0000", 2024-09-14T22:36:22.416 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.32359372199999997, 2024-09-14T22:36:22.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.019724251000000002, 2024-09-14T22:36:22.416 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:22.416 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:22.416 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:22.416 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:22.416 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:22.416 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:22.416 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 636 2024-09-14T22:36:22.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.417 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:22.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:22.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.999049+0000", 2024-09-14T22:36:22.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:22.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.999049+0000", 2024-09-14T22:36:22.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:22.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.999050+0000", 2024-09-14T22:36:22.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3999999999999997e-07 2024-09-14T22:36:22.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:22.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.999059+0000", 2024-09-14T22:36:22.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7120000000000006e-06 2024-09-14T22:36:22.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:22.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.999063+0000", 2024-09-14T22:36:22.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9779999999999996e-06 2024-09-14T22:36:22.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:22.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.999067+0000", 2024-09-14T22:36:22.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.484e-06 2024-09-14T22:36:22.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:22.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.018649+0000", 2024-09-14T22:36:22.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.019582404000000001 2024-09-14T22:36:22.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:22.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.018701+0000", 2024-09-14T22:36:22.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1755999999999997e-05 2024-09-14T22:36:22.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:22.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.018774+0000", 2024-09-14T22:36:22.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2577000000000007e-05 2024-09-14T22:36:22.419 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.420 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:22.420 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.420 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:637 3.11 3:88cdb54f:::rbd_data.113cec2505f0.0000000000000021:3 [sparse-read 3473408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T22:36:22.420 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:21.999072+0000", 2024-09-14T22:36:22.420 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.32357123900000001, 2024-09-14T22:36:22.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.019861798, 2024-09-14T22:36:22.420 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:22.420 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:22.420 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:22.420 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:22.420 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:22.420 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:22.420 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 637 2024-09-14T22:36:22.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.421 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:22.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:22.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.999072+0000", 2024-09-14T22:36:22.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:22.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.999072+0000", 2024-09-14T22:36:22.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:22.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.999072+0000", 2024-09-14T22:36:22.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5400000000000002e-07 2024-09-14T22:36:22.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:22.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.999073+0000", 2024-09-14T22:36:22.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2890000000000001e-06 2024-09-14T22:36:22.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:22.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.999077+0000", 2024-09-14T22:36:22.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4970000000000001e-06 2024-09-14T22:36:22.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:22.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.999084+0000", 2024-09-14T22:36:22.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.691e-06 2024-09-14T22:36:22.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:22.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.018809+0000", 2024-09-14T22:36:22.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.019725012 2024-09-14T22:36:22.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:22.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.018862+0000", 2024-09-14T22:36:22.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3637000000000002e-05 2024-09-14T22:36:22.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:22.424 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.018934+0000", 2024-09-14T22:36:22.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1418e-05 2024-09-14T22:36:22.424 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.424 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:22.424 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.424 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:638 3.11 3:88cdb54f:::rbd_data.113cec2505f0.0000000000000021:3 [sparse-read 3866624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T22:36:22.424 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:21.999087+0000", 2024-09-14T22:36:22.424 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.32355581700000002, 2024-09-14T22:36:22.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.019991981999999998, 2024-09-14T22:36:22.424 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:22.424 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:22.424 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:22.425 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:22.425 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:22.425 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:22.425 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 638 2024-09-14T22:36:22.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.425 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:22.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:22.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.999087+0000", 2024-09-14T22:36:22.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:22.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.999087+0000", 2024-09-14T22:36:22.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:22.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:22.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.999088+0000", 2024-09-14T22:36:22.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2499999999999999e-07 2024-09-14T22:36:22.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:22.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.999089+0000", 2024-09-14T22:36:22.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1799999999999999e-06 2024-09-14T22:36:22.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:22.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.999092+0000", 2024-09-14T22:36:22.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4039999999999999e-06 2024-09-14T22:36:22.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:22.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:21.999095+0000", 2024-09-14T22:36:22.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7149999999999998e-06 2024-09-14T22:36:22.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:22.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.018957+0000", 2024-09-14T22:36:22.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.019862323000000001 2024-09-14T22:36:22.428 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.428 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:22.428 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.019010+0000", 2024-09-14T22:36:22.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3103000000000003e-05 2024-09-14T22:36:22.428 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:22.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:22.428 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:22.428 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.019079+0000", 2024-09-14T22:36:22.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9031999999999999e-05 2024-09-14T22:36:22.428 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.428 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:22.428 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.428 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:22.429 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:22.429 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:22.430 INFO:teuthology.orchestra.run.smithi001.stdout:143 write 0x63a0371 thru 0x63a0ae0 (0x770 bytes) 2024-09-14T22:36:22.430 INFO:teuthology.orchestra.run.smithi001.stdout:144 trunc from 0xe703aa9 to 0x5430099 2024-09-14T22:36:22.430 INFO:teuthology.orchestra.run.smithi001.stdout:145 read 0x1aabc75 thru 0x1ab1ab7 (0x5e43 bytes) 2024-09-14T22:36:22.431 INFO:teuthology.orchestra.run.smithi001.stdout:148 trunc from 0x5430099 to 0x637cae6 2024-09-14T22:36:22.431 INFO:teuthology.orchestra.run.smithi001.stdout:150 punch from 0x140f272 to 0x141ad7f, (0xbb0d bytes) 2024-09-14T22:36:22.431 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:22.426+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:22.431 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:36:22.445 INFO:teuthology.orchestra.run.smithi001.stdout:152 write 0xeb95117 thru 0xeb964ee (0x13d8 bytes) 2024-09-14T22:36:22.452 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:22.452 INFO:teuthology.orchestra.run.smithi001.stdout:155 write 0x749de20 thru 0x74a90be (0xb29f bytes) 2024-09-14T22:36:22.468 INFO:teuthology.orchestra.run.smithi001.stdout:158 trunc from 0xeb964ef to 0xac7d18c 2024-09-14T22:36:22.494 INFO:teuthology.orchestra.run.smithi001.stdout:159 trunc from 0xac7d18c to 0x6fdab66 2024-09-14T22:36:22.509 INFO:teuthology.orchestra.run.smithi001.stdout:161 trunc from 0x6fdab66 to 0x54059fa 2024-09-14T22:36:22.531 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:22.523+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:22.531 INFO:teuthology.orchestra.run.smithi001.stdout:162 punch from 0x401c29c to 0x4021a0a, (0x576e bytes) 2024-09-14T22:36:22.531 INFO:teuthology.orchestra.run.smithi001.stdout:163 read 0x501ace0 thru 0x501d55d (0x287e bytes) 2024-09-14T22:36:22.547 INFO:teuthology.orchestra.run.smithi001.stdout:164 writesame 0x89266ee thru 0x892deac data_size 0x1(0x77bf bytes) 2024-09-14T22:36:22.547 INFO:teuthology.orchestra.run.smithi001.stdout:165 read 0x1e1ffba thru 0x1e21176 (0x11bd bytes) 2024-09-14T22:36:22.547 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:22.547 INFO:teuthology.orchestra.run.smithi001.stdout:167 write 0x76d05e5 thru 0x76dfdc4 (0xf7e0 bytes) 2024-09-14T22:36:22.554 INFO:teuthology.orchestra.run.smithi001.stdout:169 writesame 0x4fc7aea thru 0x4fcf107 data_size 0xd(0x761e bytes) 2024-09-14T22:36:22.557 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:22.557 INFO:teuthology.orchestra.run.smithi001.stdout:171 punch from 0x353384a to 0x35377f8, (0x3fae bytes) 2024-09-14T22:36:22.557 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:22.558 INFO:teuthology.orchestra.run.smithi001.stdout:176 read 0x149dfa1 thru 0x14aa494 (0xc4f4 bytes) 2024-09-14T22:36:22.558 INFO:teuthology.orchestra.run.smithi001.stdout:177 read 0x5eeb10a thru 0x5ef0b8c (0x5a83 bytes) 2024-09-14T22:36:22.559 INFO:teuthology.orchestra.run.smithi001.stdout:178 write 0x46c351e thru 0x46c4195 (0xc78 bytes) 2024-09-14T22:36:22.562 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:22.562 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:22.562 INFO:teuthology.orchestra.run.smithi001.stdout:181 write 0x4bd61f4 thru 0x4be3c82 (0xda8f bytes) 2024-09-14T22:36:22.565 INFO:teuthology.orchestra.run.smithi001.stdout:182 read 0x2256fa5 thru 0x225a21f (0x327b bytes) 2024-09-14T22:36:22.566 INFO:teuthology.orchestra.run.smithi001.stdout:183 read 0x4f1962a thru 0x4f1f3f3 (0x5dca bytes) 2024-09-14T22:36:22.567 INFO:teuthology.orchestra.run.smithi001.stdout:184 punch from 0x1252610 to 0x1258e51, (0x6841 bytes) 2024-09-14T22:36:22.579 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:22.579 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:36:22.579 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:36:22.579 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:22.584 INFO:teuthology.orchestra.run.smithi001.stdout:185 write 0xad934df thru 0xad9fda8 (0xc8ca bytes) 2024-09-14T22:36:22.586 INFO:teuthology.orchestra.run.smithi001.stdout:186 read 0xabf1c04 thru 0xac00e13 (0xf210 bytes) 2024-09-14T22:36:22.586 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:22.586 INFO:teuthology.orchestra.run.smithi001.stdout:188 read 0x739cbab thru 0x73a9eeb (0xd341 bytes) 2024-09-14T22:36:22.587 INFO:teuthology.orchestra.run.smithi001.stdout:189 write 0x3c08730 thru 0x3c0916e (0xa3f bytes) 2024-09-14T22:36:22.590 INFO:teuthology.orchestra.run.smithi001.stdout:191 read 0x71ca8d8 thru 0x71d4a61 (0xa18a bytes) 2024-09-14T22:36:22.590 INFO:teuthology.orchestra.run.smithi001.stdout:193 punch from 0x4b5c637 to 0x4b604af, (0x3e78 bytes) 2024-09-14T22:36:22.590 INFO:teuthology.orchestra.run.smithi001.stdout:195 read 0x447881d thru 0x4483e22 (0xb606 bytes) 2024-09-14T22:36:22.596 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:36:22.603 INFO:teuthology.orchestra.run.smithi001.stdout:196 trunc from 0xad9fda9 to 0x33b2772 2024-09-14T22:36:22.628 INFO:teuthology.orchestra.run.smithi001.stdout:197 write 0x3666920 thru 0x366f703 (0x8de4 bytes) 2024-09-14T22:36:22.632 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:36:22.626+0000 7f82f11cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26333) UID: 0 2024-09-14T22:36:22.655 INFO:teuthology.orchestra.run.smithi001.stdout:198 write 0xac071e5 thru 0xac13b82 (0xc99e bytes) 2024-09-14T22:36:22.663 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:22.677 INFO:teuthology.orchestra.run.smithi001.stdout:200 write 0xe299873 thru 0xe2a48f7 (0xb085 bytes) 2024-09-14T22:36:22.684 INFO:teuthology.orchestra.run.smithi001.stdout:203 write 0x19c3d65 thru 0x19c8d98 (0x5034 bytes) 2024-09-14T22:36:22.687 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:22.687 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:22.687 INFO:teuthology.orchestra.run.smithi001.stdout:207 read 0x7058c86 thru 0x705c497 (0x3812 bytes) 2024-09-14T22:36:22.688 INFO:teuthology.orchestra.run.smithi001.stdout:208 clone 2 order 19 su 2048 sc 4 2024-09-14T22:36:22.733 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:22.727+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:22.739 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:22.739 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:36:22.739 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:36:22.739 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:36:22.739 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:22.746 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T22:36:22.756 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:36:22.834 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:22.827+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:22.842 INFO:teuthology.orchestra.run.smithi001.stdout:leaving image image_client.0-clone1 intact 2024-09-14T22:36:22.930 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:36:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:36:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:36:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4201.0:132 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'90, mlcod=17'85)", 2024-09-14T22:36:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:01.241802+0000", 2024-09-14T22:36:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 21.685986734, 2024-09-14T22:36:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0055789150000000003, 2024-09-14T22:36:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.241802+0000", 2024-09-14T22:36:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.241802+0000", 2024-09-14T22:36:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.241803+0000", 2024-09-14T22:36:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.08e-06 2024-09-14T22:36:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.241811+0000", 2024-09-14T22:36:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3000000000000002e-06 2024-09-14T22:36:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.241827+0000", 2024-09-14T22:36:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.592e-05 2024-09-14T22:36:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.241837+0000", 2024-09-14T22:36:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7399999999999999e-06 2024-09-14T22:36:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.242796+0000", 2024-09-14T22:36:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00095904599999999997 2024-09-14T22:36:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.242832+0000", 2024-09-14T22:36:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5952999999999999e-05 2024-09-14T22:36:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.247209+0000", 2024-09-14T22:36:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0043776930000000002 2024-09-14T22:36:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.247381+0000", 2024-09-14T22:36:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017118299999999999 2024-09-14T22:36:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4201.0:134 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'92, mlcod=18'86)", 2024-09-14T22:36:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:01.243214+0000", 2024-09-14T22:36:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 21.684574164000001, 2024-09-14T22:36:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0067041150000000001, 2024-09-14T22:36:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.243214+0000", 2024-09-14T22:36:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.243214+0000", 2024-09-14T22:36:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.243215+0000", 2024-09-14T22:36:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2359999999999999e-06 2024-09-14T22:36:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.243264+0000", 2024-09-14T22:36:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8850000000000002e-05 2024-09-14T22:36:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.243287+0000", 2024-09-14T22:36:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2823000000000002e-05 2024-09-14T22:36:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.243301+0000", 2024-09-14T22:36:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3963e-05 2024-09-14T22:36:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.244365+0000", 2024-09-14T22:36:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001064389 2024-09-14T22:36:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.244423+0000", 2024-09-14T22:36:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7235999999999999e-05 2024-09-14T22:36:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.249785+0000", 2024-09-14T22:36:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0053624689999999999 2024-09-14T22:36:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.249918+0000", 2024-09-14T22:36:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000133149 2024-09-14T22:36:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4201.0:135 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'93, mlcod=18'86)", 2024-09-14T22:36:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:01.243652+0000", 2024-09-14T22:36:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 21.684136505000001, 2024-09-14T22:36:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.006393925, 2024-09-14T22:36:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.243652+0000", 2024-09-14T22:36:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.243652+0000", 2024-09-14T22:36:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.243653+0000", 2024-09-14T22:36:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3540000000000001e-06 2024-09-14T22:36:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.243663+0000", 2024-09-14T22:36:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7259999999999997e-06 2024-09-14T22:36:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.243684+0000", 2024-09-14T22:36:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0934e-05 2024-09-14T22:36:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.243695+0000", 2024-09-14T22:36:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1116e-05 2024-09-14T22:36:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245746+0000", 2024-09-14T22:36:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020508050000000002 2024-09-14T22:36:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245785+0000", 2024-09-14T22:36:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9536000000000002e-05 2024-09-14T22:36:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.249926+0000", 2024-09-14T22:36:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0041406710000000003 2024-09-14T22:36:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.250046+0000", 2024-09-14T22:36:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119783 2024-09-14T22:36:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4201.0:136 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'94, mlcod=18'86)", 2024-09-14T22:36:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:01.244082+0000", 2024-09-14T22:36:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 21.683706548, 2024-09-14T22:36:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0081452039999999996, 2024-09-14T22:36:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.244082+0000", 2024-09-14T22:36:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.244082+0000", 2024-09-14T22:36:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.244083+0000", 2024-09-14T22:36:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1039999999999999e-06 2024-09-14T22:36:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.244091+0000", 2024-09-14T22:36:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1859999999999992e-06 2024-09-14T22:36:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.244108+0000", 2024-09-14T22:36:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6634e-05 2024-09-14T22:36:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.244118+0000", 2024-09-14T22:36:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0043e-05 2024-09-14T22:36:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.246410+0000", 2024-09-14T22:36:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0022924849999999999 2024-09-14T22:36:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.246469+0000", 2024-09-14T22:36:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8903999999999997e-05 2024-09-14T22:36:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.252090+0000", 2024-09-14T22:36:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0056204560000000002 2024-09-14T22:36:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.252227+0000", 2024-09-14T22:36:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013739199999999999 2024-09-14T22:36:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4201.0:137 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'95, mlcod=18'86)", 2024-09-14T22:36:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:01.244490+0000", 2024-09-14T22:36:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 21.683298184000002, 2024-09-14T22:36:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0078329290000000006, 2024-09-14T22:36:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.244490+0000", 2024-09-14T22:36:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.244490+0000", 2024-09-14T22:36:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.244491+0000", 2024-09-14T22:36:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0529999999999999e-06 2024-09-14T22:36:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.244542+0000", 2024-09-14T22:36:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0550000000000002e-05 2024-09-14T22:36:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.244558+0000", 2024-09-14T22:36:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6633000000000002e-05 2024-09-14T22:36:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.244573+0000", 2024-09-14T22:36:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4513e-05 2024-09-14T22:36:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.247604+0000", 2024-09-14T22:36:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030310200000000002 2024-09-14T22:36:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.247664+0000", 2024-09-14T22:36:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9892999999999997e-05 2024-09-14T22:36:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.252235+0000", 2024-09-14T22:36:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.004571065 2024-09-14T22:36:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.252323+0000", 2024-09-14T22:36:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8201999999999993e-05 2024-09-14T22:36:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4201.0:138 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'96, mlcod=18'86)", 2024-09-14T22:36:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:01.245027+0000", 2024-09-14T22:36:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 21.682760957999999, 2024-09-14T22:36:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0073863990000000001, 2024-09-14T22:36:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245027+0000", 2024-09-14T22:36:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245027+0000", 2024-09-14T22:36:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245029+0000", 2024-09-14T22:36:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1629999999999999e-06 2024-09-14T22:36:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245038+0000", 2024-09-14T22:36:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8670000000000006e-06 2024-09-14T22:36:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245058+0000", 2024-09-14T22:36:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0015999999999999e-05 2024-09-14T22:36:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245068+0000", 2024-09-14T22:36:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7100000000000002e-06 2024-09-14T22:36:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.248834+0000", 2024-09-14T22:36:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003765698 2024-09-14T22:36:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.248913+0000", 2024-09-14T22:36:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9503000000000001e-05 2024-09-14T22:36:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.252334+0000", 2024-09-14T22:36:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0034203559999999998 2024-09-14T22:36:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.252414+0000", 2024-09-14T22:36:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0086000000000005e-05 2024-09-14T22:36:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4201.0:139 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'97, mlcod=18'86)", 2024-09-14T22:36:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:01.245437+0000", 2024-09-14T22:36:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 21.682351157999999, 2024-09-14T22:36:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0070633689999999999, 2024-09-14T22:36:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245437+0000", 2024-09-14T22:36:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245437+0000", 2024-09-14T22:36:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245438+0000", 2024-09-14T22:36:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2839999999999999e-06 2024-09-14T22:36:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245448+0000", 2024-09-14T22:36:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5999999999999996e-06 2024-09-14T22:36:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245463+0000", 2024-09-14T22:36:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5156e-05 2024-09-14T22:36:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245474+0000", 2024-09-14T22:36:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1126e-05 2024-09-14T22:36:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.250162+0000", 2024-09-14T22:36:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0046874109999999998 2024-09-14T22:36:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.250205+0000", 2024-09-14T22:36:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2945999999999998e-05 2024-09-14T22:36:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.252420+0000", 2024-09-14T22:36:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0022156260000000001 2024-09-14T22:36:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.252501+0000", 2024-09-14T22:36:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0220000000000001e-05 2024-09-14T22:36:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4201.0:140 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'98, mlcod=18'86)", 2024-09-14T22:36:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:01.245849+0000", 2024-09-14T22:36:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 21.681938993999999, 2024-09-14T22:36:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0074467750000000001, 2024-09-14T22:36:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245849+0000", 2024-09-14T22:36:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245849+0000", 2024-09-14T22:36:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245850+0000", 2024-09-14T22:36:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5300000000000003e-07 2024-09-14T22:36:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245924+0000", 2024-09-14T22:36:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3895999999999999e-05 2024-09-14T22:36:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245947+0000", 2024-09-14T22:36:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3213e-05 2024-09-14T22:36:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.245957+0000", 2024-09-14T22:36:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7040000000000006e-06 2024-09-14T22:36:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.250889+0000", 2024-09-14T22:36:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049324679999999997 2024-09-14T22:36:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.250945+0000", 2024-09-14T22:36:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5155999999999998e-05 2024-09-14T22:36:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.253190+0000", 2024-09-14T22:36:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0022449530000000001 2024-09-14T22:36:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.253296+0000", 2024-09-14T22:36:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106532 2024-09-14T22:36:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4201.0:141 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'99, mlcod=18'86)", 2024-09-14T22:36:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:01.247004+0000", 2024-09-14T22:36:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 21.680783963, 2024-09-14T22:36:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0070737550000000001, 2024-09-14T22:36:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.247004+0000", 2024-09-14T22:36:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.247004+0000", 2024-09-14T22:36:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.247006+0000", 2024-09-14T22:36:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1659999999999999e-06 2024-09-14T22:36:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.247015+0000", 2024-09-14T22:36:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.747e-06 2024-09-14T22:36:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.247047+0000", 2024-09-14T22:36:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1656000000000003e-05 2024-09-14T22:36:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.247057+0000", 2024-09-14T22:36:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0263e-05 2024-09-14T22:36:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.252592+0000", 2024-09-14T22:36:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0055350440000000002 2024-09-14T22:36:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.252637+0000", 2024-09-14T22:36:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4369999999999997e-05 2024-09-14T22:36:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.254007+0000", 2024-09-14T22:36:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0013700590000000001 2024-09-14T22:36:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:01.254078+0000", 2024-09-14T22:36:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1450000000000002e-05 2024-09-14T22:36:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T22:36:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:17.716282+0000", 2024-09-14T22:36:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.2115062529999996, 2024-09-14T22:36:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0088192600000000006, 2024-09-14T22:36:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 11 2024-09-14T22:36:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:17.716282+0000", 2024-09-14T22:36:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:17.716282+0000", 2024-09-14T22:36:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:17.716284+0000", 2024-09-14T22:36:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1890000000000001e-06 2024-09-14T22:36:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:17.716297+0000", 2024-09-14T22:36:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2308e-05 2024-09-14T22:36:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:17.716317+0000", 2024-09-14T22:36:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0018999999999999e-05 2024-09-14T22:36:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:17.716327+0000", 2024-09-14T22:36:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0086e-05 2024-09-14T22:36:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:17.716469+0000", 2024-09-14T22:36:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000142084 2024-09-14T22:36:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2024-09-14T22:36:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:17.716480+0000", 2024-09-14T22:36:22.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1086e-05 2024-09-14T22:36:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:17.720350+0000", 2024-09-14T22:36:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0038704270000000001 2024-09-14T22:36:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:17.720559+0000", 2024-09-14T22:36:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020853500000000001 2024-09-14T22:36:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-14T22:36:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:17.720884+0000", 2024-09-14T22:36:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032519299999999999 2024-09-14T22:36:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:36:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:17.722401+0000", 2024-09-14T22:36:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001517191 2024-09-14T22:36:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:17.724983+0000", 2024-09-14T22:36:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025813889999999999 2024-09-14T22:36:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:17.725052+0000", 2024-09-14T22:36:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9549999999999996e-05 2024-09-14T22:36:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:17.725101+0000", 2024-09-14T22:36:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9203000000000003e-05 2024-09-14T22:36:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:907 3.6 3:6733bf24:::rbd_data.113cec2505f0.000000000000000f:3 [sparse-read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T22:36:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:22.055977+0000", 2024-09-14T22:36:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.87181099100000004, 2024-09-14T22:36:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050059270000000003, 2024-09-14T22:36:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 907 2024-09-14T22:36:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.055977+0000", 2024-09-14T22:36:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.055977+0000", 2024-09-14T22:36:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.055978+0000", 2024-09-14T22:36:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8899999999999999e-07 2024-09-14T22:36:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.055982+0000", 2024-09-14T22:36:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6289999999999999e-06 2024-09-14T22:36:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.055993+0000", 2024-09-14T22:36:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1752e-05 2024-09-14T22:36:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056001+0000", 2024-09-14T22:36:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1270000000000003e-06 2024-09-14T22:36:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.060594+0000", 2024-09-14T22:36:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0045926860000000003 2024-09-14T22:36:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.060701+0000", 2024-09-14T22:36:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107113 2024-09-14T22:36:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.060983+0000", 2024-09-14T22:36:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00028203100000000002 2024-09-14T22:36:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:908 3.6 3:6733bf24:::rbd_data.113cec2505f0.000000000000000f:3 [sparse-read 1900544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T22:36:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:22.056012+0000", 2024-09-14T22:36:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.87177596099999999, 2024-09-14T22:36:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0053923, 2024-09-14T22:36:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 908 2024-09-14T22:36:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056012+0000", 2024-09-14T22:36:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056012+0000", 2024-09-14T22:36:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056013+0000", 2024-09-14T22:36:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2e-07 2024-09-14T22:36:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056016+0000", 2024-09-14T22:36:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.74e-06 2024-09-14T22:36:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056025+0000", 2024-09-14T22:36:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1800000000000002e-06 2024-09-14T22:36:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056032+0000", 2024-09-14T22:36:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6369999999999999e-06 2024-09-14T22:36:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.061063+0000", 2024-09-14T22:36:22.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050304759999999999 2024-09-14T22:36:22.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:22.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.061207+0000", 2024-09-14T22:36:22.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000144289 2024-09-14T22:36:22.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:22.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.061405+0000", 2024-09-14T22:36:22.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00019745799999999999 2024-09-14T22:36:22.992 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:22.993 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:22.993 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:22.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.993 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:909 3.6 3:6733bf24:::rbd_data.113cec2505f0.000000000000000f:3 [sparse-read 2293760~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T22:36:22.993 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:22.056043+0000", 2024-09-14T22:36:22.993 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.87174559299999999, 2024-09-14T22:36:22.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0057350480000000004, 2024-09-14T22:36:22.993 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:22.993 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:22.993 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:22.993 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:22.994 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:22.994 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:22.994 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 909 2024-09-14T22:36:22.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.994 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:22.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:22.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056043+0000", 2024-09-14T22:36:22.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:22.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:22.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056043+0000", 2024-09-14T22:36:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056043+0000", 2024-09-14T22:36:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.51e-07 2024-09-14T22:36:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056047+0000", 2024-09-14T22:36:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3680000000000002e-06 2024-09-14T22:36:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056056+0000", 2024-09-14T22:36:22.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8789999999999992e-06 2024-09-14T22:36:22.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:22.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056064+0000", 2024-09-14T22:36:22.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9250000000000001e-06 2024-09-14T22:36:22.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:22.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.061591+0000", 2024-09-14T22:36:22.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0055265599999999998 2024-09-14T22:36:22.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:22.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.061674+0000", 2024-09-14T22:36:22.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2888999999999999e-05 2024-09-14T22:36:22.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:22.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.061778+0000", 2024-09-14T22:36:22.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010387600000000001 2024-09-14T22:36:22.997 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:22.997 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:910 3.6 3:6733bf24:::rbd_data.113cec2505f0.000000000000000f:3 [sparse-read 2686976~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T22:36:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:22.056075+0000", 2024-09-14T22:36:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.87171380600000004, 2024-09-14T22:36:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0060015800000000003, 2024-09-14T22:36:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 910 2024-09-14T22:36:22.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.999 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:22.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:22.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056075+0000", 2024-09-14T22:36:22.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:22.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:22.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056075+0000", 2024-09-14T22:36:22.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:22.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:22.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:22.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:22.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056075+0000", 2024-09-14T22:36:23.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7100000000000002e-07 2024-09-14T22:36:23.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:23.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056078+0000", 2024-09-14T22:36:23.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3529999999999999e-06 2024-09-14T22:36:23.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:23.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056088+0000", 2024-09-14T22:36:23.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6469999999999993e-06 2024-09-14T22:36:23.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:23.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056096+0000", 2024-09-14T22:36:23.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6290000000000001e-06 2024-09-14T22:36:23.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:23.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.061851+0000", 2024-09-14T22:36:23.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0057549519999999998 2024-09-14T22:36:23.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:23.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.061954+0000", 2024-09-14T22:36:23.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000103465 2024-09-14T22:36:23.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.002 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:23.002 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.062076+0000", 2024-09-14T22:36:23.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121963 2024-09-14T22:36:23.002 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.002 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:23.002 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.002 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:911 3.6 3:6733bf24:::rbd_data.113cec2505f0.000000000000000f:3 [sparse-read 3080192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T22:36:23.002 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:22.056107+0000", 2024-09-14T22:36:23.003 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.87168126199999996, 2024-09-14T22:36:23.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.006296027, 2024-09-14T22:36:23.003 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:23.003 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:23.003 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:23.003 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:23.003 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:23.003 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:23.003 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 911 2024-09-14T22:36:23.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.003 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:23.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:23.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056107+0000", 2024-09-14T22:36:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056107+0000", 2024-09-14T22:36:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056108+0000", 2024-09-14T22:36:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6000000000000003e-07 2024-09-14T22:36:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056112+0000", 2024-09-14T22:36:23.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9759999999999997e-06 2024-09-14T22:36:23.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:23.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056125+0000", 2024-09-14T22:36:23.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3074999999999999e-05 2024-09-14T22:36:23.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:23.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056133+0000", 2024-09-14T22:36:23.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0919999999999998e-06 2024-09-14T22:36:23.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:23.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.062151+0000", 2024-09-14T22:36:23.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0060175760000000002 2024-09-14T22:36:23.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:23.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.062272+0000", 2024-09-14T22:36:23.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121452 2024-09-14T22:36:23.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:23.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.062403+0000", 2024-09-14T22:36:23.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013109600000000001 2024-09-14T22:36:23.006 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.006 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:23.006 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.007 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:912 3.6 3:6733bf24:::rbd_data.113cec2505f0.000000000000000f:3 [sparse-read 3473408~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T22:36:23.007 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:22.056145+0000", 2024-09-14T22:36:23.007 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.87164365099999996, 2024-09-14T22:36:23.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0064695990000000004, 2024-09-14T22:36:23.007 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:23.007 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:23.007 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:23.007 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:23.007 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:23.007 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:23.007 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 912 2024-09-14T22:36:23.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.008 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:23.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:23.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056145+0000", 2024-09-14T22:36:23.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:23.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056145+0000", 2024-09-14T22:36:23.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:23.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056145+0000", 2024-09-14T22:36:23.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2900000000000003e-07 2024-09-14T22:36:23.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:23.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056155+0000", 2024-09-14T22:36:23.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3300000000000005e-06 2024-09-14T22:36:23.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:23.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056166+0000", 2024-09-14T22:36:23.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0967e-05 2024-09-14T22:36:23.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056174+0000", 2024-09-14T22:36:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5399999999999996e-06 2024-09-14T22:36:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.062461+0000", 2024-09-14T22:36:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0062872420000000002 2024-09-14T22:36:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.062530+0000", 2024-09-14T22:36:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8843000000000005e-05 2024-09-14T22:36:23.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:23.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.062614+0000", 2024-09-14T22:36:23.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4048000000000006e-05 2024-09-14T22:36:23.011 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.011 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:23.011 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.011 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:913 3.6 3:6733bf24:::rbd_data.113cec2505f0.000000000000000f:3 [sparse-read 3866624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T22:36:23.011 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:22.056186+0000", 2024-09-14T22:36:23.012 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.87160204299999999, 2024-09-14T22:36:23.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0066994389999999997, 2024-09-14T22:36:23.012 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:23.012 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:23.012 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:23.012 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:23.012 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:23.012 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:23.012 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 913 2024-09-14T22:36:23.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.012 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:23.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:23.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056186+0000", 2024-09-14T22:36:23.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:23.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056186+0000", 2024-09-14T22:36:23.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:23.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056187+0000", 2024-09-14T22:36:23.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1200000000000003e-07 2024-09-14T22:36:23.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056190+0000", 2024-09-14T22:36:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.281e-06 2024-09-14T22:36:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056200+0000", 2024-09-14T22:36:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7089999999999997e-06 2024-09-14T22:36:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.056208+0000", 2024-09-14T22:36:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6879999999999999e-06 2024-09-14T22:36:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.062657+0000", 2024-09-14T22:36:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0064498489999999997 2024-09-14T22:36:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.062758+0000", 2024-09-14T22:36:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010058800000000001 2024-09-14T22:36:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.062886+0000", 2024-09-14T22:36:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012771200000000001 2024-09-14T22:36:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.016 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:23.016 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.016 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:975 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-14T22:36:23.016 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:22.093208+0000", 2024-09-14T22:36:23.016 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.83458031099999996, 2024-09-14T22:36:23.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022098585, 2024-09-14T22:36:23.016 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:23.016 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:23.016 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:23.016 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:23.016 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:23.016 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:23.017 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 975 2024-09-14T22:36:23.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.017 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:23.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:23.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.093208+0000", 2024-09-14T22:36:23.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:23.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.093208+0000", 2024-09-14T22:36:23.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:23.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.093210+0000", 2024-09-14T22:36:23.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8700000000000001e-06 2024-09-14T22:36:23.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:23.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.093218+0000", 2024-09-14T22:36:23.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0700000000000007e-06 2024-09-14T22:36:23.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:23.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.093236+0000", 2024-09-14T22:36:23.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8104000000000001e-05 2024-09-14T22:36:23.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.093245+0000", 2024-09-14T22:36:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1540000000000008e-06 2024-09-14T22:36:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.093317+0000", 2024-09-14T22:36:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1279000000000007e-05 2024-09-14T22:36:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.093428+0000", 2024-09-14T22:36:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111724 2024-09-14T22:36:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.020 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-14T22:36:23.020 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.098058+0000", 2024-09-14T22:36:23.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0046298310000000001 2024-09-14T22:36:23.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.020 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:23.020 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.115033+0000", 2024-09-14T22:36:23.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016975105000000001 2024-09-14T22:36:23.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.020 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:36:23.020 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.115091+0000", 2024-09-14T22:36:23.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7713000000000001e-05 2024-09-14T22:36:23.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:23.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.115224+0000", 2024-09-14T22:36:23.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013274700000000001 2024-09-14T22:36:23.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:23.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.115307+0000", 2024-09-14T22:36:23.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2987999999999999e-05 2024-09-14T22:36:23.021 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.021 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:23.021 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.021 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop_reply(client.4412.0:975 3.1f e28/23 ondisk, result = 0)", 2024-09-14T22:36:23.022 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:22.109016+0000", 2024-09-14T22:36:23.022 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.81877198600000001, 2024-09-14T22:36:23.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0060488219999999997, 2024-09-14T22:36:23.022 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:23.022 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:23.022 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:23.022 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:23.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:23.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.109016+0000", 2024-09-14T22:36:23.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.023 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:23.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.109016+0000", 2024-09-14T22:36:23.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.023 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:23.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.109018+0000", 2024-09-14T22:36:23.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4479999999999999e-06 2024-09-14T22:36:23.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.023 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:23.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.109024+0000", 2024-09-14T22:36:23.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4110000000000001e-06 2024-09-14T22:36:23.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.024 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:23.024 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.109044+0000", 2024-09-14T22:36:23.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0128000000000001e-05 2024-09-14T22:36:23.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.024 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:23.024 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.109054+0000", 2024-09-14T22:36:23.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7620000000000007e-06 2024-09-14T22:36:23.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:23.025 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.114981+0000", 2024-09-14T22:36:23.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0059266140000000002 2024-09-14T22:36:23.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:23.025 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.115014+0000", 2024-09-14T22:36:23.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3278999999999998e-05 2024-09-14T22:36:23.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:23.025 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.115065+0000", 2024-09-14T22:36:23.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1180000000000001e-05 2024-09-14T22:36:23.026 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.026 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:23.026 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.026 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:979 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-14T22:36:23.026 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:22.121037+0000", 2024-09-14T22:36:23.026 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.80675138599999996, 2024-09-14T22:36:23.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0085130640000000007, 2024-09-14T22:36:23.026 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:23.027 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:23.027 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:23.027 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:23.027 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:23.027 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:23.027 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 979 2024-09-14T22:36:23.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.027 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:23.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:23.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.121037+0000", 2024-09-14T22:36:23.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:23.028 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.121037+0000", 2024-09-14T22:36:23.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:23.028 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.121038+0000", 2024-09-14T22:36:23.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.384e-06 2024-09-14T22:36:23.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:23.028 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.121045+0000", 2024-09-14T22:36:23.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7700000000000004e-06 2024-09-14T22:36:23.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:23.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.121062+0000", 2024-09-14T22:36:23.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7116999999999999e-05 2024-09-14T22:36:23.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:23.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.121071+0000", 2024-09-14T22:36:23.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0329999999999997e-06 2024-09-14T22:36:23.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:23.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.121152+0000", 2024-09-14T22:36:23.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1207999999999999e-05 2024-09-14T22:36:23.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:23.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.121290+0000", 2024-09-14T22:36:23.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013705299999999999 2024-09-14T22:36:23.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-14T22:36:23.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.122522+0000", 2024-09-14T22:36:23.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001232791 2024-09-14T22:36:23.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:23.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.129353+0000", 2024-09-14T22:36:23.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0068307189999999999 2024-09-14T22:36:23.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:36:23.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.129383+0000", 2024-09-14T22:36:23.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9686e-05 2024-09-14T22:36:23.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:23.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.129473+0000", 2024-09-14T22:36:23.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0680000000000006e-05 2024-09-14T22:36:23.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:23.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.129550+0000", 2024-09-14T22:36:23.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6623000000000001e-05 2024-09-14T22:36:23.032 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.032 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:23.032 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.032 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.032 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:23.032 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:23.034 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:36:23.035 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-14T22:36:23.035 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:36:23.026+0000 7f82f11cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26333) UID: 0 2024-09-14T22:36:23.135 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:23.131+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:23.203 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:23.204 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:36:23.204 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:36:23.204 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:23.219 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:36:23.262 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:23.258+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:23.337 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:23.331+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:23.351 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:23.352 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:36:23.352 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:36:23.352 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:36:23.352 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:23.353 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:23.353 INFO:teuthology.orchestra.run.smithi001.stdout:210 writesame 0x60f533e thru 0x61052c7 data_size 0x3(0xff8a bytes) 2024-09-14T22:36:23.367 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:36:23.437 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:23.437 INFO:teuthology.orchestra.run.smithi001.stdout:212 punch from 0x7e37e8c to 0x7e40c86, (0x8dfa bytes) 2024-09-14T22:36:23.437 INFO:teuthology.orchestra.run.smithi001.stdout:214 writesame 0xa0cef8f thru 0xa0d96c1 data_size 0x1(0xa733 bytes) 2024-09-14T22:36:23.438 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:23.431+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:23.528 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:23.528 INFO:teuthology.orchestra.run.smithi001.stdout:216 read 0x688fa78 thru 0x689dc14 (0xe19d bytes) 2024-09-14T22:36:23.538 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:23.531+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:23.540 INFO:teuthology.orchestra.run.smithi001.stdout:217 write 0x50d11a7 thru 0x50dae28 (0x9c82 bytes) 2024-09-14T22:36:23.564 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:23.564 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:36:23.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:36:23.565 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:36:23.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.565 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:334 3.19 3:98b0b825:::rbd_header.113c3b74a67f:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-14T22:36:23.565 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:21.727377+0000", 2024-09-14T22:36:23.565 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.8337783430000001, 2024-09-14T22:36:23.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0097123780000000007, 2024-09-14T22:36:23.565 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:23.565 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:23.565 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:23.565 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:23.565 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:23.565 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:23.565 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 334 2024-09-14T22:36:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:21.727377+0000", 2024-09-14T22:36:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:21.727377+0000", 2024-09-14T22:36:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:21.727381+0000", 2024-09-14T22:36:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0539999999999996e-06 2024-09-14T22:36:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:21.727400+0000", 2024-09-14T22:36:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9273e-05 2024-09-14T22:36:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:21.727446+0000", 2024-09-14T22:36:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5905999999999997e-05 2024-09-14T22:36:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:23.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:21.727473+0000", 2024-09-14T22:36:23.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6829999999999999e-05 2024-09-14T22:36:23.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:23.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:21.727660+0000", 2024-09-14T22:36:23.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000186665 2024-09-14T22:36:23.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2024-09-14T22:36:23.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:21.727678+0000", 2024-09-14T22:36:23.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7657000000000001e-05 2024-09-14T22:36:23.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:23.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:21.734335+0000", 2024-09-14T22:36:23.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0066579509999999996 2024-09-14T22:36:23.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:23.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:21.734549+0000", 2024-09-14T22:36:23.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000213752 2024-09-14T22:36:23.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-14T22:36:23.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:21.734897+0000", 2024-09-14T22:36:23.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00034778400000000002 2024-09-14T22:36:23.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:36:23.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:21.736491+0000", 2024-09-14T22:36:23.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001594264 2024-09-14T22:36:23.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:23.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:21.736936+0000", 2024-09-14T22:36:23.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044442400000000001 2024-09-14T22:36:23.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:23.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:21.737022+0000", 2024-09-14T22:36:23.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6335000000000006e-05 2024-09-14T22:36:23.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:23.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:21.737090+0000", 2024-09-14T22:36:23.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7483000000000004e-05 2024-09-14T22:36:23.571 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.571 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:23.571 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.572 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:976 3.0 3:07c1eb37:::rbd_data.113c3b74a67f.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-14T22:36:23.572 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:22.093271+0000", 2024-09-14T22:36:23.572 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.467884014, 2024-09-14T22:36:23.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013790089, 2024-09-14T22:36:23.572 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:23.572 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:23.572 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:23.572 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:23.572 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:23.572 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:23.572 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 976 2024-09-14T22:36:23.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.572 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:23.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:23.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.093271+0000", 2024-09-14T22:36:23.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:23.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.093271+0000", 2024-09-14T22:36:23.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:23.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.093273+0000", 2024-09-14T22:36:23.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9929999999999998e-06 2024-09-14T22:36:23.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:23.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.093282+0000", 2024-09-14T22:36:23.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0240000000000003e-06 2024-09-14T22:36:23.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:23.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.093304+0000", 2024-09-14T22:36:23.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1554000000000001e-05 2024-09-14T22:36:23.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:23.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.093317+0000", 2024-09-14T22:36:23.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2469999999999999e-05 2024-09-14T22:36:23.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:23.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.093404+0000", 2024-09-14T22:36:23.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7579999999999998e-05 2024-09-14T22:36:23.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:23.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.093527+0000", 2024-09-14T22:36:23.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012325199999999999 2024-09-14T22:36:23.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 2", 2024-09-14T22:36:23.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.095919+0000", 2024-09-14T22:36:23.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0023913490000000001 2024-09-14T22:36:23.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:23.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.106951+0000", 2024-09-14T22:36:23.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01103232 2024-09-14T22:36:23.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:36:23.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.106967+0000", 2024-09-14T22:36:23.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6042999999999999e-05 2024-09-14T22:36:23.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:23.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.107004+0000", 2024-09-14T22:36:23.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6715999999999999e-05 2024-09-14T22:36:23.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:23.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:22.107062+0000", 2024-09-14T22:36:23.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7788000000000003e-05 2024-09-14T22:36:23.577 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.577 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:23.577 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.577 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2310 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13967360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:23.577 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.546065+0000", 2024-09-14T22:36:23.577 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.015090912999999999, 2024-09-14T22:36:23.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0094245389999999991, 2024-09-14T22:36:23.578 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:23.578 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:23.578 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:23.578 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:23.578 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:23.578 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:23.578 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2310 2024-09-14T22:36:23.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.578 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:23.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:23.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546065+0000", 2024-09-14T22:36:23.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:23.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546065+0000", 2024-09-14T22:36:23.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:23.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546065+0000", 2024-09-14T22:36:23.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1999999999999998e-07 2024-09-14T22:36:23.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:23.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546071+0000", 2024-09-14T22:36:23.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1200000000000001e-06 2024-09-14T22:36:23.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:23.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546087+0000", 2024-09-14T22:36:23.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6833999999999998e-05 2024-09-14T22:36:23.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:23.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546100+0000", 2024-09-14T22:36:23.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2757e-05 2024-09-14T22:36:23.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:23.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.555337+0000", 2024-09-14T22:36:23.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0092371209999999992 2024-09-14T22:36:23.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:23.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.555412+0000", 2024-09-14T22:36:23.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4710999999999997e-05 2024-09-14T22:36:23.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:23.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.555489+0000", 2024-09-14T22:36:23.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7076000000000005e-05 2024-09-14T22:36:23.581 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.581 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:23.582 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.582 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2311 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13975552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:23.582 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.546116+0000", 2024-09-14T22:36:23.582 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.015039749999999999, 2024-09-14T22:36:23.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0095781760000000007, 2024-09-14T22:36:23.582 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:23.582 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:23.582 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:23.582 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:23.582 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:23.582 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:23.582 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2311 2024-09-14T22:36:23.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.583 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:23.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:23.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546116+0000", 2024-09-14T22:36:23.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:23.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546116+0000", 2024-09-14T22:36:23.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:23.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546117+0000", 2024-09-14T22:36:23.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1000000000000001e-06 2024-09-14T22:36:23.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:23.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546122+0000", 2024-09-14T22:36:23.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9049999999999996e-06 2024-09-14T22:36:23.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:23.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546139+0000", 2024-09-14T22:36:23.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7085999999999999e-05 2024-09-14T22:36:23.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:23.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546151+0000", 2024-09-14T22:36:23.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2565000000000001e-05 2024-09-14T22:36:23.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:23.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.555542+0000", 2024-09-14T22:36:23.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093902640000000006 2024-09-14T22:36:23.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:23.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.555617+0000", 2024-09-14T22:36:23.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4992000000000003e-05 2024-09-14T22:36:23.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:23.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.555694+0000", 2024-09-14T22:36:23.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7263999999999998e-05 2024-09-14T22:36:23.586 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.586 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:23.586 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.586 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2312 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13983744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:23.586 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.546169+0000", 2024-09-14T22:36:23.586 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.014986464999999999, 2024-09-14T22:36:23.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0097278590000000002, 2024-09-14T22:36:23.586 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:23.586 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:23.586 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:23.587 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:23.587 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:23.587 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:23.587 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2312 2024-09-14T22:36:23.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.587 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:23.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:23.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546169+0000", 2024-09-14T22:36:23.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:23.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546169+0000", 2024-09-14T22:36:23.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:23.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546170+0000", 2024-09-14T22:36:23.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.017e-06 2024-09-14T22:36:23.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:23.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546240+0000", 2024-09-14T22:36:23.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9678999999999995e-05 2024-09-14T22:36:23.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:23.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546260+0000", 2024-09-14T22:36:23.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0723000000000001e-05 2024-09-14T22:36:23.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:23.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546275+0000", 2024-09-14T22:36:23.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4469e-05 2024-09-14T22:36:23.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:23.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.555728+0000", 2024-09-14T22:36:23.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0094527299999999995 2024-09-14T22:36:23.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:23.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.555804+0000", 2024-09-14T22:36:23.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6775999999999998e-05 2024-09-14T22:36:23.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:23.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.555897+0000", 2024-09-14T22:36:23.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2465000000000003e-05 2024-09-14T22:36:23.590 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.590 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:23.590 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.590 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2313 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13991936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:23.590 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.546296+0000", 2024-09-14T22:36:23.591 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.014859386, 2024-09-14T22:36:23.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0097792190000000005, 2024-09-14T22:36:23.591 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:23.591 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:23.591 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:23.591 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:23.591 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:23.591 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:23.591 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2313 2024-09-14T22:36:23.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.591 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:23.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:23.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546296+0000", 2024-09-14T22:36:23.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:23.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546296+0000", 2024-09-14T22:36:23.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:23.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546297+0000", 2024-09-14T22:36:23.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.06e-06 2024-09-14T22:36:23.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:23.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546303+0000", 2024-09-14T22:36:23.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3909999999999997e-06 2024-09-14T22:36:23.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:23.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546320+0000", 2024-09-14T22:36:23.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7263000000000001e-05 2024-09-14T22:36:23.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:23.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546333+0000", 2024-09-14T22:36:23.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2765999999999999e-05 2024-09-14T22:36:23.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:23.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.555927+0000", 2024-09-14T22:36:23.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0095946529999999999 2024-09-14T22:36:23.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:23.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.556002+0000", 2024-09-14T22:36:23.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4653000000000005e-05 2024-09-14T22:36:23.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:23.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.556075+0000", 2024-09-14T22:36:23.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3433e-05 2024-09-14T22:36:23.595 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.595 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:23.595 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.595 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2314 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 14000128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:23.595 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.546383+0000", 2024-09-14T22:36:23.595 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.014772112, 2024-09-14T22:36:23.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0098989730000000001, 2024-09-14T22:36:23.595 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:23.595 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:23.595 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:23.595 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:23.596 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:23.596 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:23.596 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2314 2024-09-14T22:36:23.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.596 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:23.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:23.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546383+0000", 2024-09-14T22:36:23.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:23.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546383+0000", 2024-09-14T22:36:23.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:23.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546385+0000", 2024-09-14T22:36:23.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.24e-06 2024-09-14T22:36:23.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:23.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546391+0000", 2024-09-14T22:36:23.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2369999999999997e-06 2024-09-14T22:36:23.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:23.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546410+0000", 2024-09-14T22:36:23.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9276999999999999e-05 2024-09-14T22:36:23.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:23.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546430+0000", 2024-09-14T22:36:23.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9579e-05 2024-09-14T22:36:23.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:23.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.556107+0000", 2024-09-14T22:36:23.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0096775639999999996 2024-09-14T22:36:23.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:23.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.556182+0000", 2024-09-14T22:36:23.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4876000000000006e-05 2024-09-14T22:36:23.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:23.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.556282+0000", 2024-09-14T22:36:23.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0001002 2024-09-14T22:36:23.599 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.599 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:23.599 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.599 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2315 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 14008320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:23.599 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.546449+0000", 2024-09-14T22:36:23.599 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.014706535999999999, 2024-09-14T22:36:23.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010556062, 2024-09-14T22:36:23.599 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:23.599 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:23.600 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:23.600 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:23.600 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:23.600 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:23.600 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2315 2024-09-14T22:36:23.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.600 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:23.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:23.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546449+0000", 2024-09-14T22:36:23.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:23.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546449+0000", 2024-09-14T22:36:23.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:23.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546450+0000", 2024-09-14T22:36:23.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7999999999999993e-07 2024-09-14T22:36:23.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:23.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546455+0000", 2024-09-14T22:36:23.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1340000000000003e-06 2024-09-14T22:36:23.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:23.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546473+0000", 2024-09-14T22:36:23.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7706e-05 2024-09-14T22:36:23.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:23.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546486+0000", 2024-09-14T22:36:23.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3393999999999999e-05 2024-09-14T22:36:23.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:23.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.556315+0000", 2024-09-14T22:36:23.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0098291379999999994 2024-09-14T22:36:23.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:23.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.556399+0000", 2024-09-14T22:36:23.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3498999999999995e-05 2024-09-14T22:36:23.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:23.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.557005+0000", 2024-09-14T22:36:23.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00060621099999999995 2024-09-14T22:36:23.603 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.603 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:23.603 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.603 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2316 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 14016512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:23.603 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.546508+0000", 2024-09-14T22:36:23.604 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.014647333, 2024-09-14T22:36:23.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010712418, 2024-09-14T22:36:23.604 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:23.604 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:23.604 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:23.604 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:23.604 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:23.604 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:23.604 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2316 2024-09-14T22:36:23.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.604 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:23.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:23.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546508+0000", 2024-09-14T22:36:23.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:23.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546508+0000", 2024-09-14T22:36:23.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:23.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546509+0000", 2024-09-14T22:36:23.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3099999999999996e-07 2024-09-14T22:36:23.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:23.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546514+0000", 2024-09-14T22:36:23.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1089999999999997e-06 2024-09-14T22:36:23.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:23.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546531+0000", 2024-09-14T22:36:23.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6857000000000001e-05 2024-09-14T22:36:23.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:23.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546544+0000", 2024-09-14T22:36:23.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2468e-05 2024-09-14T22:36:23.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:23.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.557039+0000", 2024-09-14T22:36:23.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010495734 2024-09-14T22:36:23.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:23.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.557134+0000", 2024-09-14T22:36:23.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4888000000000004e-05 2024-09-14T22:36:23.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:23.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.557221+0000", 2024-09-14T22:36:23.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6431e-05 2024-09-14T22:36:23.607 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.607 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:23.607 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.608 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2317 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:23.608 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.546560+0000", 2024-09-14T22:36:23.608 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.01459562, 2024-09-14T22:36:23.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010838366, 2024-09-14T22:36:23.608 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:23.608 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:23.608 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:23.608 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:23.608 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:23.608 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:23.608 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2317 2024-09-14T22:36:23.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.608 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:23.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:23.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546560+0000", 2024-09-14T22:36:23.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:23.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546560+0000", 2024-09-14T22:36:23.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:23.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546561+0000", 2024-09-14T22:36:23.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0599999999999999e-07 2024-09-14T22:36:23.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:23.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546566+0000", 2024-09-14T22:36:23.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1800000000000004e-06 2024-09-14T22:36:23.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:23.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546582+0000", 2024-09-14T22:36:23.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6113999999999999e-05 2024-09-14T22:36:23.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:23.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546595+0000", 2024-09-14T22:36:23.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2734e-05 2024-09-14T22:36:23.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:23.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.557251+0000", 2024-09-14T22:36:23.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010656696 2024-09-14T22:36:23.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:23.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.557327+0000", 2024-09-14T22:36:23.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5916999999999998e-05 2024-09-14T22:36:23.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:23.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.557398+0000", 2024-09-14T22:36:23.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0819000000000001e-05 2024-09-14T22:36:23.611 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.612 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:23.612 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.612 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2318 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 14032896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:23.612 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.546649+0000", 2024-09-14T22:36:23.612 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.014506754, 2024-09-14T22:36:23.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010937116, 2024-09-14T22:36:23.612 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:23.612 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:23.612 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:23.612 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:23.612 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:23.612 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:23.612 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2318 2024-09-14T22:36:23.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.613 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:23.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:23.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546649+0000", 2024-09-14T22:36:23.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:23.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546649+0000", 2024-09-14T22:36:23.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:23.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546650+0000", 2024-09-14T22:36:23.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.088e-06 2024-09-14T22:36:23.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:23.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546656+0000", 2024-09-14T22:36:23.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9859999999999999e-06 2024-09-14T22:36:23.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:23.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546676+0000", 2024-09-14T22:36:23.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9721999999999999e-05 2024-09-14T22:36:23.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:23.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546689+0000", 2024-09-14T22:36:23.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.366e-05 2024-09-14T22:36:23.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:23.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.557429+0000", 2024-09-14T22:36:23.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010739336 2024-09-14T22:36:23.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:23.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.557507+0000", 2024-09-14T22:36:23.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8066999999999996e-05 2024-09-14T22:36:23.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.616 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:23.616 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.557586+0000", 2024-09-14T22:36:23.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9257000000000004e-05 2024-09-14T22:36:23.616 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.616 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:23.616 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.616 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2319 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 14041088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:23.616 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.546708+0000", 2024-09-14T22:36:23.616 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.014447898000000001, 2024-09-14T22:36:23.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011066878, 2024-09-14T22:36:23.617 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:23.617 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:23.617 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:23.617 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:23.617 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:23.617 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:23.617 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2319 2024-09-14T22:36:23.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.617 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:23.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:23.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546708+0000", 2024-09-14T22:36:23.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:23.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546708+0000", 2024-09-14T22:36:23.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:23.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546709+0000", 2024-09-14T22:36:23.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.569999999999999e-07 2024-09-14T22:36:23.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:23.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546714+0000", 2024-09-14T22:36:23.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5550000000000002e-06 2024-09-14T22:36:23.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:23.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546731+0000", 2024-09-14T22:36:23.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6756999999999999e-05 2024-09-14T22:36:23.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:23.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546744+0000", 2024-09-14T22:36:23.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3081999999999999e-05 2024-09-14T22:36:23.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:23.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.557630+0000", 2024-09-14T22:36:23.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010886142999999999 2024-09-14T22:36:23.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:23.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.557705+0000", 2024-09-14T22:36:23.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4863000000000005e-05 2024-09-14T22:36:23.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:23.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.557774+0000", 2024-09-14T22:36:23.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9521000000000001e-05 2024-09-14T22:36:23.620 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.620 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:23.620 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.620 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2320 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 14049280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:23.620 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.546764+0000", 2024-09-14T22:36:23.621 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.014391173, 2024-09-14T22:36:23.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011248493, 2024-09-14T22:36:23.621 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:23.621 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:23.621 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:23.621 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:23.621 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:23.621 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:23.621 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2320 2024-09-14T22:36:23.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.621 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:23.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:23.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546764+0000", 2024-09-14T22:36:23.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:23.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546764+0000", 2024-09-14T22:36:23.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:23.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546765+0000", 2024-09-14T22:36:23.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9500000000000001e-07 2024-09-14T22:36:23.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:23.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546770+0000", 2024-09-14T22:36:23.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7709999999999997e-06 2024-09-14T22:36:23.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:23.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546787+0000", 2024-09-14T22:36:23.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6826000000000001e-05 2024-09-14T22:36:23.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:23.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546799+0000", 2024-09-14T22:36:23.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2503e-05 2024-09-14T22:36:23.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:23.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.557818+0000", 2024-09-14T22:36:23.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011019166 2024-09-14T22:36:23.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:23.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.557893+0000", 2024-09-14T22:36:23.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4989999999999999e-05 2024-09-14T22:36:23.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:23.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.558013+0000", 2024-09-14T22:36:23.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119342 2024-09-14T22:36:23.624 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.624 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:23.624 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.625 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2321 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 14057472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:23.625 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.546816+0000", 2024-09-14T22:36:23.625 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.01433947, 2024-09-14T22:36:23.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011379897, 2024-09-14T22:36:23.625 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:23.625 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:23.625 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:23.625 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:23.625 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:23.625 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:23.625 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2321 2024-09-14T22:36:23.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.625 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:23.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:23.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546816+0000", 2024-09-14T22:36:23.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:23.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546816+0000", 2024-09-14T22:36:23.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:23.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546817+0000", 2024-09-14T22:36:23.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1399999999999995e-07 2024-09-14T22:36:23.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:23.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546824+0000", 2024-09-14T22:36:23.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4739999999999999e-06 2024-09-14T22:36:23.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:23.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546840+0000", 2024-09-14T22:36:23.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5994999999999998e-05 2024-09-14T22:36:23.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:23.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546853+0000", 2024-09-14T22:36:23.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2277e-05 2024-09-14T22:36:23.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:23.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.558047+0000", 2024-09-14T22:36:23.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011194503 2024-09-14T22:36:23.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:23.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.558124+0000", 2024-09-14T22:36:23.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6614999999999997e-05 2024-09-14T22:36:23.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:23.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.558196+0000", 2024-09-14T22:36:23.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2119000000000005e-05 2024-09-14T22:36:23.628 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.628 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:23.628 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.629 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2322 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 14065664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:23.629 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.546911+0000", 2024-09-14T22:36:23.629 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.014244305000000001, 2024-09-14T22:36:23.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011777905, 2024-09-14T22:36:23.629 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:23.629 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:23.629 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:23.629 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:23.629 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:23.629 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:23.629 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2322 2024-09-14T22:36:23.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.629 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:23.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:23.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546911+0000", 2024-09-14T22:36:23.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:23.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546911+0000", 2024-09-14T22:36:23.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:23.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546913+0000", 2024-09-14T22:36:23.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5689999999999999e-06 2024-09-14T22:36:23.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:23.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546919+0000", 2024-09-14T22:36:23.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.879e-06 2024-09-14T22:36:23.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:23.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546937+0000", 2024-09-14T22:36:23.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8212000000000001e-05 2024-09-14T22:36:23.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:23.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546951+0000", 2024-09-14T22:36:23.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3784999999999999e-05 2024-09-14T22:36:23.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:23.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.558316+0000", 2024-09-14T22:36:23.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011365060999999999 2024-09-14T22:36:23.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:23.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.558529+0000", 2024-09-14T22:36:23.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021355699999999999 2024-09-14T22:36:23.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:23.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.558689+0000", 2024-09-14T22:36:23.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015984199999999999 2024-09-14T22:36:23.633 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.633 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:23.633 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.633 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2323 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 14073856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:23.633 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.546969+0000", 2024-09-14T22:36:23.633 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.01418678, 2024-09-14T22:36:23.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01210048, 2024-09-14T22:36:23.633 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:23.633 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:23.633 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:23.634 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:23.634 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:23.634 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:23.634 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2323 2024-09-14T22:36:23.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.634 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:23.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:23.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546969+0000", 2024-09-14T22:36:23.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:23.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546969+0000", 2024-09-14T22:36:23.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:23.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546970+0000", 2024-09-14T22:36:23.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.015e-06 2024-09-14T22:36:23.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:23.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546975+0000", 2024-09-14T22:36:23.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7450000000000001e-06 2024-09-14T22:36:23.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:23.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.546992+0000", 2024-09-14T22:36:23.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6759999999999999e-05 2024-09-14T22:36:23.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:23.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.547005+0000", 2024-09-14T22:36:23.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2619999999999999e-05 2024-09-14T22:36:23.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:23.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.558754+0000", 2024-09-14T22:36:23.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011748711 2024-09-14T22:36:23.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:23.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.558914+0000", 2024-09-14T22:36:23.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016069500000000001 2024-09-14T22:36:23.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:23.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559069+0000", 2024-09-14T22:36:23.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015493399999999999 2024-09-14T22:36:23.637 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.637 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:23.637 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.637 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2324 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 14082048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:23.637 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.547021+0000", 2024-09-14T22:36:23.638 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.014134020000000001, 2024-09-14T22:36:23.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012355955, 2024-09-14T22:36:23.638 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:23.638 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:23.638 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:23.638 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:23.638 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:23.638 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:23.638 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2324 2024-09-14T22:36:23.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.638 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:23.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:23.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.547021+0000", 2024-09-14T22:36:23.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:23.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.547021+0000", 2024-09-14T22:36:23.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:23.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.547022+0000", 2024-09-14T22:36:23.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7700000000000003e-07 2024-09-14T22:36:23.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:23.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.547080+0000", 2024-09-14T22:36:23.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8165000000000003e-05 2024-09-14T22:36:23.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:23.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.547100+0000", 2024-09-14T22:36:23.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.984e-05 2024-09-14T22:36:23.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:23.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.547114+0000", 2024-09-14T22:36:23.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4089e-05 2024-09-14T22:36:23.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:23.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559155+0000", 2024-09-14T22:36:23.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012041058 2024-09-14T22:36:23.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:23.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559264+0000", 2024-09-14T22:36:23.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108545 2024-09-14T22:36:23.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:23.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559377+0000", 2024-09-14T22:36:23.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000113381 2024-09-14T22:36:23.642 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.642 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:23.642 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.642 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2325 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 14090240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:23.642 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.547165+0000", 2024-09-14T22:36:23.642 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.013990607, 2024-09-14T22:36:23.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012454015000000001, 2024-09-14T22:36:23.642 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:23.642 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:23.642 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:23.642 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:23.643 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:23.643 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:23.643 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2325 2024-09-14T22:36:23.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.643 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:23.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:23.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.547165+0000", 2024-09-14T22:36:23.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:23.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.547165+0000", 2024-09-14T22:36:23.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.644 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:23.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.547166+0000", 2024-09-14T22:36:23.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1540000000000001e-06 2024-09-14T22:36:23.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.644 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:23.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.547172+0000", 2024-09-14T22:36:23.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5199999999999997e-06 2024-09-14T22:36:23.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.644 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:23.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.547190+0000", 2024-09-14T22:36:23.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.842e-05 2024-09-14T22:36:23.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:23.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.547204+0000", 2024-09-14T22:36:23.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3606e-05 2024-09-14T22:36:23.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:23.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559436+0000", 2024-09-14T22:36:23.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012232061000000001 2024-09-14T22:36:23.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:23.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559520+0000", 2024-09-14T22:36:23.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4779999999999998e-05 2024-09-14T22:36:23.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:23.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559619+0000", 2024-09-14T22:36:23.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8474000000000006e-05 2024-09-14T22:36:23.646 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.646 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:23.646 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.646 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2326 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 14098432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:23.646 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.547222+0000", 2024-09-14T22:36:23.646 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.013933667, 2024-09-14T22:36:23.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012581986999999999, 2024-09-14T22:36:23.646 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:23.647 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:23.647 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:23.647 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:23.647 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:23.647 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:23.647 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2326 2024-09-14T22:36:23.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.647 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:23.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:23.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.547222+0000", 2024-09-14T22:36:23.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:23.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.547222+0000", 2024-09-14T22:36:23.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:23.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.547223+0000", 2024-09-14T22:36:23.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.037e-06 2024-09-14T22:36:23.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:23.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.547228+0000", 2024-09-14T22:36:23.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1830000000000002e-06 2024-09-14T22:36:23.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:23.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.547244+0000", 2024-09-14T22:36:23.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6047999999999999e-05 2024-09-14T22:36:23.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:23.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.547256+0000", 2024-09-14T22:36:23.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2277e-05 2024-09-14T22:36:23.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:23.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559654+0000", 2024-09-14T22:36:23.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012397523000000001 2024-09-14T22:36:23.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:23.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559730+0000", 2024-09-14T22:36:23.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5856e-05 2024-09-14T22:36:23.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:23.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559804+0000", 2024-09-14T22:36:23.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4062999999999999e-05 2024-09-14T22:36:23.650 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.650 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:23.650 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.651 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2327 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 14106624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:23.651 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.547273+0000", 2024-09-14T22:36:23.651 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.013882693, 2024-09-14T22:36:23.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012707919, 2024-09-14T22:36:23.651 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:23.651 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:23.651 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:23.651 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:23.651 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:23.651 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:23.651 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2327 2024-09-14T22:36:23.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.651 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:23.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:23.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.547273+0000", 2024-09-14T22:36:23.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:23.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.547273+0000", 2024-09-14T22:36:23.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:23.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:23.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.547274+0000", 2024-09-14T22:36:23.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7400000000000002e-07 2024-09-14T22:36:23.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:23.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.547279+0000", 2024-09-14T22:36:23.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2249999999999999e-06 2024-09-14T22:36:23.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:23.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.547295+0000", 2024-09-14T22:36:23.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6254999999999999e-05 2024-09-14T22:36:23.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:23.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.547308+0000", 2024-09-14T22:36:23.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3193999999999999e-05 2024-09-14T22:36:23.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:23.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559837+0000", 2024-09-14T22:36:23.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012528676000000001 2024-09-14T22:36:23.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:23.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559911+0000", 2024-09-14T22:36:23.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3665999999999996e-05 2024-09-14T22:36:23.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:23.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:23.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:23.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559981+0000", 2024-09-14T22:36:23.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0029000000000003e-05 2024-09-14T22:36:23.655 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.655 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:23.655 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.655 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:23.655 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:23.655 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:23.656 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:36:23.657 INFO:teuthology.orchestra.run.smithi001.stdout:218 trunc from 0xe2a48f8 to 0x4b5c10b 2024-09-14T22:36:23.657 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:23.650+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:23.735 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2024-09-14T22:36:23.757 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:23.750+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:23.766 INFO:teuthology.orchestra.run.smithi001.stdout:219 punch from 0x3d71fdb to 0x3d74b96, (0x2bbb bytes) 2024-09-14T22:36:23.766 INFO:teuthology.orchestra.run.smithi001.stdout:220 read 0x1fd03ab thru 0x1fd3e3f (0x3a95 bytes) 2024-09-14T22:36:23.786 INFO:teuthology.orchestra.run.smithi001.stdout:221 writesame 0x991e79d thru 0x9920a5d data_size 0x1f(0x22c1 bytes) 2024-09-14T22:36:23.786 INFO:teuthology.orchestra.run.smithi001.stdout:222 read 0x729e7ce thru 0x72ae633 (0xfe66 bytes) 2024-09-14T22:36:23.786 INFO:teuthology.orchestra.run.smithi001.stdout:223 read 0x2bea597 thru 0x2bfa1fd (0xfc67 bytes) 2024-09-14T22:36:23.793 INFO:teuthology.orchestra.run.smithi001.stdout:226 punch from 0x7f76835 to 0x7f81dba, (0xb585 bytes) 2024-09-14T22:36:23.793 INFO:teuthology.orchestra.run.smithi001.stdout:227 write 0x3fcbd20 thru 0x3fd05c6 (0x48a7 bytes) 2024-09-14T22:36:23.857 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:23.858 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:36:23.858 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:36:23.858 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:23.858 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:23.850+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:23.875 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:36:23.884 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:23.884 INFO:teuthology.orchestra.run.smithi001.stdout:230 writesame 0x7a35c57 thru 0x7a37fee data_size 0x11(0x2398 bytes) 2024-09-14T22:36:23.896 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:36:23.905 INFO:teuthology.orchestra.run.smithi001.stdout:232 trunc from 0x9920a5e to 0x2eb9524 2024-09-14T22:36:23.958 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:23.951+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:24.029 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:24.029 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:36:24.030 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:36:24.030 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:36:24.030 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:24.030 INFO:teuthology.orchestra.run.smithi001.stdout:233 read 0x29a160 thru 0x2a32c9 (0x916a bytes) 2024-09-14T22:36:24.035 INFO:teuthology.orchestra.run.smithi001.stdout:234 read 0xbc7f28 thru 0xbcd0a5 (0x517e bytes) 2024-09-14T22:36:24.047 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:36:24.058 INFO:teuthology.orchestra.run.smithi001.stdout:237 write 0x847a771 thru 0x8488732 (0xdfc2 bytes) 2024-09-14T22:36:24.058 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:24.054+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:24.071 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:24.071 INFO:teuthology.orchestra.run.smithi001.stdout:240 write 0x37572f4 thru 0x3765ff1 (0xecfe bytes) 2024-09-14T22:36:24.083 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:24.100 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:36:24.100 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:36:24.103 INFO:teuthology.orchestra.run.smithi001.stdout:242 write 0xeb1ad4b thru 0xeb1f4b6 (0x476c bytes) 2024-09-14T22:36:24.114 INFO:teuthology.orchestra.run.smithi001.stdout:243 punch from 0xd0e3a11 to 0xd0edd40, (0xa32f bytes) 2024-09-14T22:36:24.114 INFO:teuthology.orchestra.run.smithi001.stdout:245 write 0xc242176 thru 0xc24a995 (0x8820 bytes) 2024-09-14T22:36:24.122 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:36:24.122 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:36:24.126 INFO:teuthology.orchestra.run.smithi001.stdout:247 punch from 0x897cce7 to 0x898cabf, (0xfdd8 bytes) 2024-09-14T22:36:24.126 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:24.126 INFO:teuthology.orchestra.run.smithi001.stdout:249 punch from 0x14f2776 to 0x14f765b, (0x4ee5 bytes) 2024-09-14T22:36:24.126 INFO:teuthology.orchestra.run.smithi001.stdout:250 punch from 0xc1ef278 to 0xc1f1801, (0x2589 bytes) 2024-09-14T22:36:24.126 INFO:teuthology.orchestra.run.smithi001.stdout:252 writesame 0x59fe31e thru 0x5a08bfb data_size 0x83(0xa8de bytes) 2024-09-14T22:36:24.143 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:36:24.143 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:36:24.144 INFO:teuthology.orchestra.run.smithi001.stdout:253 write 0x6223020 thru 0x622c8c3 (0x98a4 bytes) 2024-09-14T22:36:24.155 INFO:teuthology.orchestra.run.smithi001.stdout:254 writesame 0x34a4b74 thru 0x34aa112 data_size 0x1(0x559f bytes) 2024-09-14T22:36:24.159 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:24.154+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:24.164 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:36:24.164 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:36:24.169 INFO:teuthology.orchestra.run.smithi001.stdout:256 read 0x3a49be5 thru 0x3a512ee (0x770a bytes) 2024-09-14T22:36:24.169 INFO:teuthology.orchestra.run.smithi001.stdout:257 read 0x2ffd6ae thru 0x300149a (0x3ded bytes) 2024-09-14T22:36:24.183 INFO:teuthology.orchestra.run.smithi001.stdout:258 write 0xee38e12 thru 0xee3d46d (0x465c bytes) 2024-09-14T22:36:24.186 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:36:24.187 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:36:24.194 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:24.194 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:24.208 INFO:teuthology.orchestra.run.smithi001.stdout:260 trunc from 0xee3d46e to 0x5deb9ca 2024-09-14T22:36:24.209 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:36:24.209 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:36:24.219 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:36:24.233 INFO:teuthology.orchestra.run.smithi001.stdout:262 read 0x523fad2 thru 0x5242044 (0x2573 bytes) 2024-09-14T22:36:24.233 INFO:teuthology.orchestra.run.smithi001.stdout:263 read 0x5b4bb84 thru 0x5b5b88a (0xfd07 bytes) 2024-09-14T22:36:24.233 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:36:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:36:24.234 INFO:teuthology.orchestra.run.smithi001.stdout:264 read 0x50a771c thru 0x50a97e3 (0x20c8 bytes) 2024-09-14T22:36:24.234 INFO:teuthology.orchestra.run.smithi001.stdout:265 read 0x2be0daf thru 0x2be471a (0x396c bytes) 2024-09-14T22:36:24.237 INFO:teuthology.orchestra.run.smithi001.stdout:266 punch from 0xf3843d to 0xf474be, (0xf081 bytes) 2024-09-14T22:36:24.257 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:36:24.257 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:36:24.260 INFO:teuthology.orchestra.run.smithi001.stdout:268 write 0xec4ff12 thru 0xec548e0 (0x49cf bytes) 2024-09-14T22:36:24.260 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:24.254+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:24.270 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:24.270 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:24.271 INFO:teuthology.orchestra.run.smithi001.stdout:275 write 0xab77eca thru 0xab80d5d (0x8e94 bytes) 2024-09-14T22:36:24.286 INFO:teuthology.orchestra.run.smithi001.stdout:276 trunc from 0xec548e1 to 0xc394d6e 2024-09-14T22:36:24.308 INFO:teuthology.orchestra.run.smithi001.stdout:277 read 0x55fb0d0 thru 0x5607d6c (0xcc9d bytes) 2024-09-14T22:36:24.308 INFO:teuthology.orchestra.run.smithi001.stdout:278 write 0x436472c thru 0x436deff (0x97d4 bytes) 2024-09-14T22:36:24.321 INFO:teuthology.orchestra.run.smithi001.stdout:279 punch from 0x95f66bf to 0x95f6718, (0x59 bytes) 2024-09-14T22:36:24.321 INFO:teuthology.orchestra.run.smithi001.stdout:280 writesame 0x2b3eeb thru 0x2be64a data_size 0x10(0xa760 bytes) 2024-09-14T22:36:24.361 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:36:24.354+0000 7f82f11cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26333) UID: 0 2024-09-14T22:36:24.375 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:24.376 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:24.394 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:36:24.406 INFO:teuthology.orchestra.run.smithi001.stdout:282 read 0x36d98b7 thru 0x36e1420 (0x7b6a bytes) 2024-09-14T22:36:24.420 INFO:teuthology.orchestra.run.smithi001.stdout:283 write 0xea1a213 thru 0xea286ed (0xe4db bytes) 2024-09-14T22:36:24.448 INFO:teuthology.orchestra.run.smithi001.stdout:284 trunc from 0xea286ee to 0x1f677e8 2024-09-14T22:36:24.462 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:36:24.454+0000 7f82f11cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26333) UID: 0 2024-09-14T22:36:24.553 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:24.554 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:24.554 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:24.562 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:24.558+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:24.571 INFO:teuthology.orchestra.run.smithi001.stdout:287 writesame 0xa95e946 thru 0xa965809 data_size 0x22(0x6ec4 bytes) 2024-09-14T22:36:24.574 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:36:24.583 INFO:teuthology.orchestra.run.smithi001.stdout:288 writesame 0xe1b86d3 thru 0xe1bbebf data_size 0x1(0x37ed bytes) 2024-09-14T22:36:24.585 INFO:teuthology.orchestra.run.smithi001.stdout:289 trunc from 0xe1bbec0 to 0xc855eed 2024-09-14T22:36:24.594 INFO:teuthology.orchestra.run.smithi001.stdout:290 punch from 0x18c126a to 0x18d02ab, (0xf041 bytes) 2024-09-14T22:36:24.594 INFO:teuthology.orchestra.run.smithi001.stdout:291 read 0xe67755 thru 0xe68194 (0xa40 bytes) 2024-09-14T22:36:24.605 INFO:teuthology.orchestra.run.smithi001.stdout:292 write 0xe187d5f thru 0xe18af60 (0x3202 bytes) 2024-09-14T22:36:24.627 INFO:teuthology.orchestra.run.smithi001.stdout:295 trunc from 0xe18af61 to 0x49ca483 2024-09-14T22:36:24.663 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:24.658+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:24.666 INFO:teuthology.orchestra.run.smithi001.stdout:297 write 0xc3e90a8 thru 0xc3eb606 (0x255f bytes) 2024-09-14T22:36:24.691 INFO:teuthology.orchestra.run.smithi001.stdout:298 trunc from 0xc3eb607 to 0x72468b 2024-09-14T22:36:24.705 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:24.705 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:24.726 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:36:24.771 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:24.767+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:24.810 INFO:teuthology.orchestra.run.smithi001.stdout:299 write 0xe63d7d9 thru 0xe642960 (0x5188 bytes) 2024-09-14T22:36:24.824 INFO:teuthology.orchestra.run.smithi001.stdout:300 read 0xaa1c19 thru 0xaa390b (0x1cf3 bytes) 2024-09-14T22:36:24.824 INFO:teuthology.orchestra.run.smithi001.stdout:301 punch from 0x50799d to 0x515724, (0xdd87 bytes) 2024-09-14T22:36:24.824 INFO:teuthology.orchestra.run.smithi001.stdout:302 punch from 0xadad708 to 0xadbd2c8, (0xfbc0 bytes) 2024-09-14T22:36:24.835 INFO:teuthology.orchestra.run.smithi001.stdout:306 trunc from 0xe642961 to 0x84561ec 2024-09-14T22:36:24.863 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:24.863 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:24.866 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:24.858+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:24.867 INFO:teuthology.orchestra.run.smithi001.stdout:307 trunc from 0x84561ec to 0x4d2c80f 2024-09-14T22:36:24.877 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:36:24.882 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:24.883 INFO:teuthology.orchestra.run.smithi001.stdout:310 writesame 0x2c77c95 thru 0x2c7a40d data_size 0x2b(0x2779 bytes) 2024-09-14T22:36:24.919 INFO:teuthology.orchestra.run.smithi001.stdout:312 writesame 0x80fff9f thru 0x810c196 data_size 0xc(0xc1f8 bytes) 2024-09-14T22:36:24.919 INFO:teuthology.orchestra.run.smithi001.stdout:313 read 0x441fe10 thru 0x4425d24 (0x5f15 bytes) 2024-09-14T22:36:24.919 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:24.919 INFO:teuthology.orchestra.run.smithi001.stdout:315 trunc from 0x810c197 to 0x7a6badf 2024-09-14T22:36:24.934 INFO:teuthology.orchestra.run.smithi001.stdout:316 read 0x388239b thru 0x388ba42 (0x96a8 bytes) 2024-09-14T22:36:24.934 INFO:teuthology.orchestra.run.smithi001.stdout:317 writesame 0x6b8bcd1 thru 0x6b927e5 data_size 0x1(0x6b15 bytes) 2024-09-14T22:36:24.957 INFO:teuthology.orchestra.run.smithi001.stdout:319 read 0x1affa63 thru 0x1b0f7b5 (0xfd53 bytes) 2024-09-14T22:36:24.958 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:24.967 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:24.962+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:24.977 INFO:teuthology.orchestra.run.smithi001.stdout:321 write 0xa7271e2 thru 0xa7308f3 (0x9712 bytes) 2024-09-14T22:36:24.992 INFO:teuthology.orchestra.run.smithi001.stdout:322 writesame 0x4c5bba4 thru 0x4c688b6 data_size 0x1(0xcd13 bytes) 2024-09-14T22:36:25.033 INFO:teuthology.orchestra.run.smithi001.stdout:323 write 0xa76ccd5 thru 0xa76fc5f (0x2f8b bytes) 2024-09-14T22:36:25.039 INFO:teuthology.orchestra.run.smithi001.stdout:325 write 0x91ac4f8 thru 0x91ae325 (0x1e2e bytes) 2024-09-14T22:36:25.057 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:25.057 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:25.068 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:25.063+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:25.069 INFO:teuthology.orchestra.run.smithi001.stdout:326 writesame 0xcb0984b thru 0xcb19573 data_size 0x13(0xfd29 bytes) 2024-09-14T22:36:25.069 INFO:teuthology.orchestra.run.smithi001.stdout:328 punch from 0x3df7cb1 to 0x3e03a82, (0xbdd1 bytes) 2024-09-14T22:36:25.072 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:36:25.086 INFO:teuthology.orchestra.run.smithi001.stdout:329 writesame 0xe43a3cc thru 0xe448931 data_size 0x1(0xe566 bytes) 2024-09-14T22:36:25.099 INFO:teuthology.orchestra.run.smithi001.stdout:331 write 0xe792348 thru 0xe794d6a (0x2a23 bytes) 2024-09-14T22:36:25.110 INFO:teuthology.orchestra.run.smithi001.stdout:332 read 0x4e757db thru 0x4e82618 (0xce3e bytes) 2024-09-14T22:36:25.110 INFO:teuthology.orchestra.run.smithi001.stdout:333 read 0xbee4c1 thru 0xbf7a67 (0x95a7 bytes) 2024-09-14T22:36:25.111 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:25.120 INFO:teuthology.orchestra.run.smithi001.stdout:336 trunc from 0xe794d6b to 0xb45bfe2 2024-09-14T22:36:25.140 INFO:teuthology.orchestra.run.smithi001.stdout:337 writesame 0x62db24 thru 0x63847f data_size 0x6(0xa95c bytes) 2024-09-14T22:36:25.168 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:25.163+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:25.174 INFO:teuthology.orchestra.run.smithi001.stdout:340 writesame 0xe8fe25e thru 0xe9093a0 data_size 0x1(0xb143 bytes) 2024-09-14T22:36:25.174 INFO:teuthology.orchestra.run.smithi001.stdout:342 write 0xb2c2e40 thru 0xb2caadc (0x7c9d bytes) 2024-09-14T22:36:25.184 INFO:teuthology.orchestra.run.smithi001.stdout:343 punch from 0x2adf21c to 0x2ae934f, (0xa133 bytes) 2024-09-14T22:36:25.184 INFO:teuthology.orchestra.run.smithi001.stdout:344 read 0x3d8dd5e thru 0x3d8eeb7 (0x115a bytes) 2024-09-14T22:36:25.184 INFO:teuthology.orchestra.run.smithi001.stdout:345 write 0xb0f0122 thru 0xb0f19ce (0x18ad bytes) 2024-09-14T22:36:25.194 INFO:teuthology.orchestra.run.smithi001.stdout:347 read 0xbbb8a9b thru 0xbbbb006 (0x256c bytes) 2024-09-14T22:36:25.194 INFO:teuthology.orchestra.run.smithi001.stdout:348 read 0x57921eb thru 0x579bf90 (0x9da6 bytes) 2024-09-14T22:36:25.194 INFO:teuthology.orchestra.run.smithi001.stdout:351 writesame 0xd15bf1e thru 0xd162fd5 data_size 0x4(0x70b8 bytes) 2024-09-14T22:36:25.208 INFO:teuthology.orchestra.run.smithi001.stdout:353 read 0x66f365d thru 0x66f6557 (0x2efb bytes) 2024-09-14T22:36:25.208 INFO:teuthology.orchestra.run.smithi001.stdout:354 punch from 0x7f7b92f to 0x7f8184e, (0x5f1f bytes) 2024-09-14T22:36:25.209 INFO:teuthology.orchestra.run.smithi001.stdout:356 read 0xa860d08 thru 0xa860dee (0xe7 bytes) 2024-09-14T22:36:25.209 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:25.209 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:25.209 INFO:teuthology.orchestra.run.smithi001.stdout:360 read 0x4d7a86 thru 0x4e330d (0xb888 bytes) 2024-09-14T22:36:25.212 INFO:teuthology.orchestra.run.smithi001.stdout:361 read 0x7455653 thru 0x745efee (0x999c bytes) 2024-09-14T22:36:25.239 INFO:teuthology.orchestra.run.smithi001.stdout:362 trunc from 0xe9093a1 to 0x507bd8c 2024-09-14T22:36:25.257 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:25.257 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:25.269 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:25.262+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:25.271 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:36:25.286 INFO:teuthology.orchestra.run.smithi001.stdout:363 trunc from 0x507bd8c to 0x1260c01 2024-09-14T22:36:25.310 INFO:teuthology.orchestra.run.smithi001.stdout:364 read 0x6d4281 thru 0x6dc1c2 (0x7f42 bytes) 2024-09-14T22:36:25.341 INFO:teuthology.orchestra.run.smithi001.stdout:365 trunc from 0x1260c01 to 0xbf91e4b 2024-09-14T22:36:25.353 INFO:teuthology.orchestra.run.smithi001.stdout:366 writesame 0x88fef1c thru 0x8905c5c data_size 0x1(0x6d41 bytes) 2024-09-14T22:36:25.367 INFO:teuthology.orchestra.run.smithi001.stdout:367 read 0x6cc3eef thru 0x6ccba9e (0x7bb0 bytes) 2024-09-14T22:36:25.367 INFO:teuthology.orchestra.run.smithi001.stdout:368 read 0xbacc1dc thru 0xbadba47 (0xf86c bytes) 2024-09-14T22:36:25.368 INFO:teuthology.orchestra.run.smithi001.stdout:369 write 0x835e968 thru 0x83669eb (0x8084 bytes) 2024-09-14T22:36:25.370 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:25.363+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:25.380 INFO:teuthology.orchestra.run.smithi001.stdout:370 read 0x70f3c6f thru 0x70fcaaa (0x8e3c bytes) 2024-09-14T22:36:25.398 INFO:teuthology.orchestra.run.smithi001.stdout:371 trunc from 0xbf91e4b to 0x485b61b 2024-09-14T22:36:25.445 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:25.446 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:25.447 INFO:teuthology.orchestra.run.smithi001.stdout:373 write 0x83da8be thru 0x83e81a9 (0xd8ec bytes) 2024-09-14T22:36:25.459 INFO:teuthology.orchestra.run.smithi001.stdout:374 write 0x71bad3e thru 0x71beb0a (0x3dcd bytes) 2024-09-14T22:36:25.462 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:36:25.470 INFO:teuthology.orchestra.run.smithi001.stdout:375 punch from 0x1b9ac17 to 0x1ba53e1, (0xa7ca bytes) 2024-09-14T22:36:25.470 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:25.471 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:25.466+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:25.497 INFO:teuthology.orchestra.run.smithi001.stdout:377 write 0xd1ab086 thru 0xd1b57bd (0xa738 bytes) 2024-09-14T22:36:25.537 INFO:teuthology.orchestra.run.smithi001.stdout:380 trunc from 0xd1b57be to 0x36d050c 2024-09-14T22:36:25.572 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:25.566+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:25.580 INFO:teuthology.orchestra.run.smithi001.stdout:381 write 0x432bb8f thru 0x43378ce (0xbd40 bytes) 2024-09-14T22:36:25.591 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:25.592 INFO:teuthology.orchestra.run.smithi001.stdout:384 read 0x1f4fab7 thru 0x1f58c74 (0x91be bytes) 2024-09-14T22:36:25.592 INFO:teuthology.orchestra.run.smithi001.stdout:385 punch from 0x3337fd0 to 0x333d85b, (0x588b bytes) 2024-09-14T22:36:25.592 INFO:teuthology.orchestra.run.smithi001.stdout:386 writesame 0x26971fa thru 0x26a709e data_size 0x2f(0xfea5 bytes) 2024-09-14T22:36:25.602 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:25.602 INFO:teuthology.orchestra.run.smithi001.stdout:390 write 0x3b8d708 thru 0x3b9a245 (0xcb3e bytes) 2024-09-14T22:36:25.614 INFO:teuthology.orchestra.run.smithi001.stdout:391 read 0x37ba66d thru 0x37bc0f4 (0x1a88 bytes) 2024-09-14T22:36:25.614 INFO:teuthology.orchestra.run.smithi001.stdout:394 write 0x299a438 thru 0x29a75d4 (0xd19d bytes) 2024-09-14T22:36:25.634 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:25.634 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:25.649 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:36:25.657 INFO:teuthology.orchestra.run.smithi001.stdout:395 write 0xa8bc193 thru 0xa8cbfdd (0xfe4b bytes) 2024-09-14T22:36:25.668 INFO:teuthology.orchestra.run.smithi001.stdout:396 punch from 0xa43cd2 to 0xa4b960, (0x7c8e bytes) 2024-09-14T22:36:25.668 INFO:teuthology.orchestra.run.smithi001.stdout:397 writesame 0x6fc519 thru 0x702f1b data_size 0x1(0x6a03 bytes) 2024-09-14T22:36:25.678 INFO:teuthology.orchestra.run.smithi001.stdout:truncating to largest ever: 0xeda9248 2024-09-14T22:36:25.689 INFO:teuthology.orchestra.run.smithi001.stdout:398 trunc from 0xa8cbfde to 0xeda9248 2024-09-14T22:36:25.699 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:25.695+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:25.700 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:25.700 INFO:teuthology.orchestra.run.smithi001.stdout:401 punch from 0xba00efe to 0xba070fb, (0x61fd bytes) 2024-09-14T22:36:25.702 INFO:teuthology.orchestra.run.smithi001.stdout:402 trunc from 0xeda9248 to 0xe2731d5 2024-09-14T22:36:25.712 INFO:teuthology.orchestra.run.smithi001.stdout:403 writesame 0x9156713 thru 0x9159666 data_size 0x1a(0x2f54 bytes) 2024-09-14T22:36:25.723 INFO:teuthology.orchestra.run.smithi001.stdout:404 punch from 0xca2c969 to 0xca3a5d2, (0xdc69 bytes) 2024-09-14T22:36:25.723 INFO:teuthology.orchestra.run.smithi001.stdout:405 writesame 0x4a0d42 thru 0x4ac5b2 data_size 0x1(0xb871 bytes) 2024-09-14T22:36:25.746 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T22:36:25.747 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:36:25.747 INFO:tasks.thrashosds.thrasher:Killing osd 2, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T22:36:25.747 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-14T22:36:25.747 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T22:36:25.763 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T22:36:25.786 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:25.786 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:25.794 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:25.790+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:25.801 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:36:25.821 INFO:tasks.ceph.osd.2:Stopped 2024-09-14T22:36:25.821 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.2 2024-09-14T22:36:25.822 INFO:tasks.thrashosds.thrasher:remote for osd 2 is ubuntu@smithi001.front.sepia.ceph.com 2024-09-14T22:36:25.822 DEBUG:teuthology.orchestra.run.smithi001:> type ceph-objectstore-tool 2024-09-14T22:36:25.858 INFO:teuthology.orchestra.run.smithi001.stdout:406 write 0x3462d57 thru 0x346eae4 (0xbd8e bytes) 2024-09-14T22:36:25.861 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-14T22:36:25.861 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal --op list-pgs 2024-09-14T22:36:25.902 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:25.898+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:25.961 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:25.961 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:25.974 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:25.967+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:25.975 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:36:26.074 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:26.067+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:26.137 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:26.137 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:26.151 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:36:26.195 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:26.191+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:26.277 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:26.271+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:26.278 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:36:26.317 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:26.317 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:26.331 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:36:26.378 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:26.374+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:26.428 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:36:26.428 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:36:26.445 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:36:26.445 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:36:26.454 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:26.454 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:36:26.454 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:36:26.454 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:26.468 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:36:26.479 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:26.471+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:26.579 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:26.575+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:26.635 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:26.636 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:36:26.636 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:36:26.636 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:36:26.636 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:26.651 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:36:26.694 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:26.690+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:26.766 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2024-09-14T22:36:26.781 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:26.775+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:26.795 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:26.795 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:36:26.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:36:26.795 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:36:26.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.795 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2769 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14323712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:26.796 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.551427+0000", 2024-09-14T22:36:26.796 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.240590187, 2024-09-14T22:36:26.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028484874, 2024-09-14T22:36:26.796 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:26.796 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:26.796 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:26.796 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:26.796 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:26.796 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:26.796 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2769 2024-09-14T22:36:26.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.796 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:26.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:26.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.551427+0000", 2024-09-14T22:36:26.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:26.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.551427+0000", 2024-09-14T22:36:26.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:26.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.551427+0000", 2024-09-14T22:36:26.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9700000000000002e-07 2024-09-14T22:36:26.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:26.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.551434+0000", 2024-09-14T22:36:26.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8589999999999999e-06 2024-09-14T22:36:26.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:26.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.551442+0000", 2024-09-14T22:36:26.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4429999999999997e-06 2024-09-14T22:36:26.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:26.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.551447+0000", 2024-09-14T22:36:26.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5169999999999999e-06 2024-09-14T22:36:26.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:26.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.579693+0000", 2024-09-14T22:36:26.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028246130000000001 2024-09-14T22:36:26.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:26.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.579799+0000", 2024-09-14T22:36:26.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010518600000000001 2024-09-14T22:36:26.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:26.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.579912+0000", 2024-09-14T22:36:26.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000113342 2024-09-14T22:36:26.799 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.799 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:26.800 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.800 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2770 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14331904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:26.800 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.551455+0000", 2024-09-14T22:36:26.800 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.2405627190000001, 2024-09-14T22:36:26.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028737275, 2024-09-14T22:36:26.800 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:26.800 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:26.800 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:26.800 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:26.800 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:26.800 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:26.800 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2770 2024-09-14T22:36:26.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.801 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:26.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:26.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.551455+0000", 2024-09-14T22:36:26.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:26.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.551455+0000", 2024-09-14T22:36:26.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:26.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.551455+0000", 2024-09-14T22:36:26.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.39e-07 2024-09-14T22:36:26.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:26.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.551457+0000", 2024-09-14T22:36:26.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2000000000000001e-06 2024-09-14T22:36:26.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:26.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.551464+0000", 2024-09-14T22:36:26.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3180000000000003e-06 2024-09-14T22:36:26.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:26.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.551476+0000", 2024-09-14T22:36:26.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2345e-05 2024-09-14T22:36:26.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:26.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.579963+0000", 2024-09-14T22:36:26.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028487545999999999 2024-09-14T22:36:26.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:26.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.580075+0000", 2024-09-14T22:36:26.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111292 2024-09-14T22:36:26.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:26.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.580192+0000", 2024-09-14T22:36:26.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000117135 2024-09-14T22:36:26.804 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.804 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:26.804 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.804 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3020 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14284800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:26.804 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559505+0000", 2024-09-14T22:36:26.804 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.2325124949999999, 2024-09-14T22:36:26.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028481824999999999, 2024-09-14T22:36:26.804 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:26.804 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:26.804 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:26.805 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:26.805 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:26.805 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:26.805 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3020 2024-09-14T22:36:26.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.805 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:26.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:26.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559505+0000", 2024-09-14T22:36:26.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:26.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559505+0000", 2024-09-14T22:36:26.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:26.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559506+0000", 2024-09-14T22:36:26.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.113e-06 2024-09-14T22:36:26.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:26.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559512+0000", 2024-09-14T22:36:26.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1149999999999999e-06 2024-09-14T22:36:26.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:26.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559528+0000", 2024-09-14T22:36:26.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6149000000000001e-05 2024-09-14T22:36:26.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:26.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559539+0000", 2024-09-14T22:36:26.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1257000000000001e-05 2024-09-14T22:36:26.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:26.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.587819+0000", 2024-09-14T22:36:26.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028279955999999998 2024-09-14T22:36:26.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:26.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.587900+0000", 2024-09-14T22:36:26.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0345000000000004e-05 2024-09-14T22:36:26.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:26.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.587987+0000", 2024-09-14T22:36:26.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6890000000000003e-05 2024-09-14T22:36:26.809 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.809 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:26.809 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.809 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3021 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14292992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:26.809 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559557+0000", 2024-09-14T22:36:26.809 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.2324601130000001, 2024-09-14T22:36:26.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028720181000000001, 2024-09-14T22:36:26.809 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:26.809 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:26.809 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:26.809 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:26.809 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:26.810 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:26.810 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3021 2024-09-14T22:36:26.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.810 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:26.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:26.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559557+0000", 2024-09-14T22:36:26.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:26.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559557+0000", 2024-09-14T22:36:26.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:26.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559558+0000", 2024-09-14T22:36:26.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4499999999999996e-07 2024-09-14T22:36:26.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:26.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559562+0000", 2024-09-14T22:36:26.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.065e-06 2024-09-14T22:36:26.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:26.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559573+0000", 2024-09-14T22:36:26.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1398e-05 2024-09-14T22:36:26.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:26.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559582+0000", 2024-09-14T22:36:26.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6100000000000006e-06 2024-09-14T22:36:26.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:26.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588042+0000", 2024-09-14T22:36:26.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028460388999999999 2024-09-14T22:36:26.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:26.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588147+0000", 2024-09-14T22:36:26.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000104727 2024-09-14T22:36:26.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:26.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588277+0000", 2024-09-14T22:36:26.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000130147 2024-09-14T22:36:26.813 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.813 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:26.814 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.814 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3022 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14301184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:26.814 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559594+0000", 2024-09-14T22:36:26.814 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.2324237230000001, 2024-09-14T22:36:26.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028916481000000001, 2024-09-14T22:36:26.814 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:26.814 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:26.814 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:26.814 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:26.814 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:26.814 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:26.814 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3022 2024-09-14T22:36:26.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.815 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:26.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:26.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559594+0000", 2024-09-14T22:36:26.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:26.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559594+0000", 2024-09-14T22:36:26.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:26.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559594+0000", 2024-09-14T22:36:26.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7000000000000004e-07 2024-09-14T22:36:26.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:26.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559604+0000", 2024-09-14T22:36:26.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6020000000000006e-06 2024-09-14T22:36:26.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:26.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559615+0000", 2024-09-14T22:36:26.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1032999999999999e-05 2024-09-14T22:36:26.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:26.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559638+0000", 2024-09-14T22:36:26.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3255999999999999e-05 2024-09-14T22:36:26.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:26.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588323+0000", 2024-09-14T22:36:26.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.02868447 2024-09-14T22:36:26.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:26.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588443+0000", 2024-09-14T22:36:26.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120056 2024-09-14T22:36:26.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:26.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588510+0000", 2024-09-14T22:36:26.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7393999999999999e-05 2024-09-14T22:36:26.819 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.819 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:26.819 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.819 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3023 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14309376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:26.819 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559651+0000", 2024-09-14T22:36:26.819 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.2323662369999999, 2024-09-14T22:36:26.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029146231000000002, 2024-09-14T22:36:26.819 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:26.819 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:26.819 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:26.819 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:26.819 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:26.820 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:26.820 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3023 2024-09-14T22:36:26.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.820 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:26.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:26.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559651+0000", 2024-09-14T22:36:26.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:26.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559651+0000", 2024-09-14T22:36:26.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:26.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559652+0000", 2024-09-14T22:36:26.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2699999999999998e-07 2024-09-14T22:36:26.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:26.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559656+0000", 2024-09-14T22:36:26.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7019999999999999e-06 2024-09-14T22:36:26.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:26.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559666+0000", 2024-09-14T22:36:26.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0400999999999999e-05 2024-09-14T22:36:26.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:26.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559674+0000", 2024-09-14T22:36:26.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8890000000000002e-06 2024-09-14T22:36:26.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:26.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588565+0000", 2024-09-14T22:36:26.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028891217 2024-09-14T22:36:26.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:26.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588679+0000", 2024-09-14T22:36:26.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114058 2024-09-14T22:36:26.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:26.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588797+0000", 2024-09-14T22:36:26.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118137 2024-09-14T22:36:26.823 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.823 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:26.823 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.824 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3024 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14317568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:26.824 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559685+0000", 2024-09-14T22:36:26.824 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.2323321260000002, 2024-09-14T22:36:26.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029326498999999999, 2024-09-14T22:36:26.824 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:26.824 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:26.824 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:26.824 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:26.824 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:26.824 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:26.824 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3024 2024-09-14T22:36:26.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.824 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:26.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:26.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559685+0000", 2024-09-14T22:36:26.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:26.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559685+0000", 2024-09-14T22:36:26.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:26.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559686+0000", 2024-09-14T22:36:26.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0999999999999998e-07 2024-09-14T22:36:26.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:26.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559697+0000", 2024-09-14T22:36:26.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.147e-05 2024-09-14T22:36:26.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:26.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559708+0000", 2024-09-14T22:36:26.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0546e-05 2024-09-14T22:36:26.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:26.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559716+0000", 2024-09-14T22:36:26.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.704e-06 2024-09-14T22:36:26.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:26.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588833+0000", 2024-09-14T22:36:26.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029116442999999999 2024-09-14T22:36:26.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:26.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588920+0000", 2024-09-14T22:36:26.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6916999999999995e-05 2024-09-14T22:36:26.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:26.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589012+0000", 2024-09-14T22:36:26.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1809000000000001e-05 2024-09-14T22:36:26.828 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.828 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:26.828 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.828 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3025 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14325760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:26.828 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559729+0000", 2024-09-14T22:36:26.828 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.2322882270000002, 2024-09-14T22:36:26.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029422501, 2024-09-14T22:36:26.828 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:26.828 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3025 2024-09-14T22:36:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559729+0000", 2024-09-14T22:36:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559729+0000", 2024-09-14T22:36:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559730+0000", 2024-09-14T22:36:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8100000000000002e-07 2024-09-14T22:36:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559733+0000", 2024-09-14T22:36:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4740000000000001e-06 2024-09-14T22:36:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559744+0000", 2024-09-14T22:36:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0489000000000001e-05 2024-09-14T22:36:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559760+0000", 2024-09-14T22:36:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5817000000000001e-05 2024-09-14T22:36:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589050+0000", 2024-09-14T22:36:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.02929056 2024-09-14T22:36:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589101+0000", 2024-09-14T22:36:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0992000000000002e-05 2024-09-14T22:36:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589152+0000", 2024-09-14T22:36:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0387999999999999e-05 2024-09-14T22:36:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3026 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14333952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559795+0000", 2024-09-14T22:36:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.2322224030000002, 2024-09-14T22:36:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029483812000000002, 2024-09-14T22:36:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3026 2024-09-14T22:36:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559795+0000", 2024-09-14T22:36:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559795+0000", 2024-09-14T22:36:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559796+0000", 2024-09-14T22:36:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3200000000000003e-07 2024-09-14T22:36:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559800+0000", 2024-09-14T22:36:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2930000000000002e-06 2024-09-14T22:36:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559812+0000", 2024-09-14T22:36:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2053e-05 2024-09-14T22:36:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559821+0000", 2024-09-14T22:36:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8869999999999998e-06 2024-09-14T22:36:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589177+0000", 2024-09-14T22:36:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029356308000000001 2024-09-14T22:36:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589229+0000", 2024-09-14T22:36:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1326000000000003e-05 2024-09-14T22:36:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589279+0000", 2024-09-14T22:36:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0012999999999997e-05 2024-09-14T22:36:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3027 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14342144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559833+0000", 2024-09-14T22:36:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.2321846750000001, 2024-09-14T22:36:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029619187000000002, 2024-09-14T22:36:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3027 2024-09-14T22:36:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559833+0000", 2024-09-14T22:36:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559833+0000", 2024-09-14T22:36:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:26.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559833+0000", 2024-09-14T22:36:26.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9400000000000005e-07 2024-09-14T22:36:26.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:26.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559845+0000", 2024-09-14T22:36:26.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1623e-05 2024-09-14T22:36:26.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:26.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559856+0000", 2024-09-14T22:36:26.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0832999999999999e-05 2024-09-14T22:36:26.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.840 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:26.840 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559864+0000", 2024-09-14T22:36:26.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3119999999999994e-06 2024-09-14T22:36:26.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.840 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:26.840 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589299+0000", 2024-09-14T22:36:26.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029434699000000002 2024-09-14T22:36:26.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:26.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589354+0000", 2024-09-14T22:36:26.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5467999999999998e-05 2024-09-14T22:36:26.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:26.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589452+0000", 2024-09-14T22:36:26.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7558000000000003e-05 2024-09-14T22:36:26.841 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.841 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:26.841 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.841 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3028 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14350336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:26.841 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559875+0000", 2024-09-14T22:36:26.842 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.2321417869999998, 2024-09-14T22:36:26.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029775586999999999, 2024-09-14T22:36:26.842 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:26.842 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:26.842 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:26.842 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:26.842 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:26.842 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:26.842 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3028 2024-09-14T22:36:26.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.842 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:26.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:26.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559875+0000", 2024-09-14T22:36:26.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:26.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559875+0000", 2024-09-14T22:36:26.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:26.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559876+0000", 2024-09-14T22:36:26.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5199999999999996e-07 2024-09-14T22:36:26.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:26.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559879+0000", 2024-09-14T22:36:26.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.095e-06 2024-09-14T22:36:26.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:26.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559889+0000", 2024-09-14T22:36:26.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0019e-05 2024-09-14T22:36:26.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:26.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559897+0000", 2024-09-14T22:36:26.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9189999999999999e-06 2024-09-14T22:36:26.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:26.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589486+0000", 2024-09-14T22:36:26.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029588482999999999 2024-09-14T22:36:26.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:26.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589578+0000", 2024-09-14T22:36:26.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2329000000000003e-05 2024-09-14T22:36:26.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:26.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589651+0000", 2024-09-14T22:36:26.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2990000000000004e-05 2024-09-14T22:36:26.845 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.845 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:26.846 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.846 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4066 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13236224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:26.846 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680054+0000", 2024-09-14T22:36:26.846 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.1119632319999999, 2024-09-14T22:36:26.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028524247999999999, 2024-09-14T22:36:26.846 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:26.846 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:26.846 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:26.846 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:26.846 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:26.846 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:26.846 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4066 2024-09-14T22:36:26.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.846 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:26.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:26.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680054+0000", 2024-09-14T22:36:26.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:26.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680054+0000", 2024-09-14T22:36:26.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:26.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680055+0000", 2024-09-14T22:36:26.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6199999999999998e-07 2024-09-14T22:36:26.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:26.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680057+0000", 2024-09-14T22:36:26.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.463e-06 2024-09-14T22:36:26.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:26.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680064+0000", 2024-09-14T22:36:26.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6749999999999996e-06 2024-09-14T22:36:26.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:26.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680069+0000", 2024-09-14T22:36:26.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3399999999999997e-06 2024-09-14T22:36:26.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:26.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.708369+0000", 2024-09-14T22:36:26.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028299714 2024-09-14T22:36:26.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:26.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.708462+0000", 2024-09-14T22:36:26.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3152000000000005e-05 2024-09-14T22:36:26.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:26.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.708578+0000", 2024-09-14T22:36:26.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011644199999999999 2024-09-14T22:36:26.849 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.850 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:26.850 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.850 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4067 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13244416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:26.850 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680076+0000", 2024-09-14T22:36:26.850 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.1119414989999998, 2024-09-14T22:36:26.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028684398, 2024-09-14T22:36:26.850 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:26.850 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:26.850 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:26.850 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:26.850 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:26.850 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:26.850 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4067 2024-09-14T22:36:26.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.851 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:26.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:26.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680076+0000", 2024-09-14T22:36:26.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:26.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680076+0000", 2024-09-14T22:36:26.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:26.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680076+0000", 2024-09-14T22:36:26.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9000000000000002e-07 2024-09-14T22:36:26.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:26.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680079+0000", 2024-09-14T22:36:26.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.616e-06 2024-09-14T22:36:26.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:26.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680087+0000", 2024-09-14T22:36:26.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7059999999999993e-06 2024-09-14T22:36:26.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:26.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680093+0000", 2024-09-14T22:36:26.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.626e-06 2024-09-14T22:36:26.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:26.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.708640+0000", 2024-09-14T22:36:26.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028546892000000001 2024-09-14T22:36:26.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:26.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.708701+0000", 2024-09-14T22:36:26.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0582999999999999e-05 2024-09-14T22:36:26.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:26.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.708760+0000", 2024-09-14T22:36:26.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9585e-05 2024-09-14T22:36:26.854 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.854 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:26.854 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.854 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4068 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13252608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:26.854 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680102+0000", 2024-09-14T22:36:26.854 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.1119149679999998, 2024-09-14T22:36:26.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028907656, 2024-09-14T22:36:26.854 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:26.855 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:26.855 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:26.855 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:26.855 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:26.855 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:26.855 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4068 2024-09-14T22:36:26.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.855 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:26.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:26.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680102+0000", 2024-09-14T22:36:26.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:26.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680102+0000", 2024-09-14T22:36:26.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:26.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680103+0000", 2024-09-14T22:36:26.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6600000000000002e-07 2024-09-14T22:36:26.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:26.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680105+0000", 2024-09-14T22:36:26.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.616e-06 2024-09-14T22:36:26.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:26.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680114+0000", 2024-09-14T22:36:26.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4120000000000001e-06 2024-09-14T22:36:26.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:26.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680119+0000", 2024-09-14T22:36:26.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6210000000000001e-06 2024-09-14T22:36:26.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:26.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.708813+0000", 2024-09-14T22:36:26.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028693330999999999 2024-09-14T22:36:26.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:26.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.708903+0000", 2024-09-14T22:36:26.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0297000000000006e-05 2024-09-14T22:36:26.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:26.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709010+0000", 2024-09-14T22:36:26.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106913 2024-09-14T22:36:26.858 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.858 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:26.858 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.858 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4069 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13260800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:26.858 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680128+0000", 2024-09-14T22:36:26.858 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.1118896930000002, 2024-09-14T22:36:26.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029145476, 2024-09-14T22:36:26.859 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:26.859 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:26.859 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:26.859 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:26.859 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:26.859 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:26.859 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4069 2024-09-14T22:36:26.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.859 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:26.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:26.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680128+0000", 2024-09-14T22:36:26.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:26.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680128+0000", 2024-09-14T22:36:26.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:26.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680128+0000", 2024-09-14T22:36:26.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7599999999999998e-07 2024-09-14T22:36:26.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:26.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680131+0000", 2024-09-14T22:36:26.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9050000000000001e-06 2024-09-14T22:36:26.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:26.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680138+0000", 2024-09-14T22:36:26.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2559999999999996e-06 2024-09-14T22:36:26.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:26.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680144+0000", 2024-09-14T22:36:26.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4779999999999999e-06 2024-09-14T22:36:26.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:26.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709054+0000", 2024-09-14T22:36:26.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028910207 2024-09-14T22:36:26.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:26.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709180+0000", 2024-09-14T22:36:26.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012608699999999999 2024-09-14T22:36:26.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:26.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709273+0000", 2024-09-14T22:36:26.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3166999999999998e-05 2024-09-14T22:36:26.863 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.863 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:26.863 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.863 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4070 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13268992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:26.863 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680151+0000", 2024-09-14T22:36:26.863 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.1118664109999998, 2024-09-14T22:36:26.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029327986, 2024-09-14T22:36:26.863 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:26.863 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:26.864 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:26.864 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:26.864 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:26.864 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:26.864 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4070 2024-09-14T22:36:26.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.864 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:26.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:26.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680151+0000", 2024-09-14T22:36:26.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:26.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680151+0000", 2024-09-14T22:36:26.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:26.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680151+0000", 2024-09-14T22:36:26.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:36:26.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:26.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680154+0000", 2024-09-14T22:36:26.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3350000000000001e-06 2024-09-14T22:36:26.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:26.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680162+0000", 2024-09-14T22:36:26.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.473e-06 2024-09-14T22:36:26.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:26.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680177+0000", 2024-09-14T22:36:26.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.465e-05 2024-09-14T22:36:26.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:26.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709356+0000", 2024-09-14T22:36:26.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029178744 2024-09-14T22:36:26.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:26.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709428+0000", 2024-09-14T22:36:26.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2428000000000005e-05 2024-09-14T22:36:26.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:26.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709479+0000", 2024-09-14T22:36:26.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0915999999999998e-05 2024-09-14T22:36:26.868 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.868 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:26.868 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.868 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4071 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13277184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:26.868 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680185+0000", 2024-09-14T22:36:26.868 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.1118319109999999, 2024-09-14T22:36:26.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029452392000000001, 2024-09-14T22:36:26.868 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:26.868 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:26.869 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:26.869 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:26.869 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:26.869 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:26.869 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4071 2024-09-14T22:36:26.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.869 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:26.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:26.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680185+0000", 2024-09-14T22:36:26.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:26.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680185+0000", 2024-09-14T22:36:26.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:26.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680186+0000", 2024-09-14T22:36:26.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6100000000000001e-07 2024-09-14T22:36:26.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:26.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680188+0000", 2024-09-14T22:36:26.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.509e-06 2024-09-14T22:36:26.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:26.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680197+0000", 2024-09-14T22:36:26.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8550000000000008e-06 2024-09-14T22:36:26.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:26.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680203+0000", 2024-09-14T22:36:26.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5849999999999999e-06 2024-09-14T22:36:26.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:26.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709514+0000", 2024-09-14T22:36:26.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029311054 2024-09-14T22:36:26.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:26.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709585+0000", 2024-09-14T22:36:26.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0875000000000002e-05 2024-09-14T22:36:26.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:26.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709638+0000", 2024-09-14T22:36:26.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3053000000000002e-05 2024-09-14T22:36:26.872 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.872 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:26.872 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.872 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4072 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13285376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:26.872 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680211+0000", 2024-09-14T22:36:26.872 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.1118066299999998, 2024-09-14T22:36:26.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029591177999999999, 2024-09-14T22:36:26.872 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:26.873 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:26.873 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:26.873 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:26.873 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:26.873 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:26.873 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4072 2024-09-14T22:36:26.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.873 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:26.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:26.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680211+0000", 2024-09-14T22:36:26.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:26.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680211+0000", 2024-09-14T22:36:26.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:26.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680211+0000", 2024-09-14T22:36:26.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6699999999999999e-07 2024-09-14T22:36:26.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:26.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680214+0000", 2024-09-14T22:36:26.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0149999999999999e-06 2024-09-14T22:36:26.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:26.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680222+0000", 2024-09-14T22:36:26.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.371e-06 2024-09-14T22:36:26.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:26.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680228+0000", 2024-09-14T22:36:26.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6230000000000002e-06 2024-09-14T22:36:26.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:26.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709670+0000", 2024-09-14T22:36:26.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029441694000000001 2024-09-14T22:36:26.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:26.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709741+0000", 2024-09-14T22:36:26.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1668999999999994e-05 2024-09-14T22:36:26.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:26.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709802+0000", 2024-09-14T22:36:26.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0338999999999999e-05 2024-09-14T22:36:26.876 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.876 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:26.877 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.877 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4073 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13293568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:26.877 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680236+0000", 2024-09-14T22:36:26.877 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.1117812709999999, 2024-09-14T22:36:26.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029755238999999999, 2024-09-14T22:36:26.877 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:26.877 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:26.877 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:26.877 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:26.877 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:26.877 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:26.878 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4073 2024-09-14T22:36:26.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.878 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:26.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:26.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680236+0000", 2024-09-14T22:36:26.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:26.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680236+0000", 2024-09-14T22:36:26.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:26.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680236+0000", 2024-09-14T22:36:26.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5499999999999998e-07 2024-09-14T22:36:26.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:26.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680239+0000", 2024-09-14T22:36:26.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4380000000000002e-06 2024-09-14T22:36:26.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:26.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680248+0000", 2024-09-14T22:36:26.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6319999999999997e-06 2024-09-14T22:36:26.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:26.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680261+0000", 2024-09-14T22:36:26.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3193000000000001e-05 2024-09-14T22:36:26.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:26.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709831+0000", 2024-09-14T22:36:26.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029570486 2024-09-14T22:36:26.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:26.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709883+0000", 2024-09-14T22:36:26.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2290999999999998e-05 2024-09-14T22:36:26.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:26.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709991+0000", 2024-09-14T22:36:26.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107744 2024-09-14T22:36:26.881 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.881 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:26.881 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.881 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4074 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13301760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:26.881 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680269+0000", 2024-09-14T22:36:26.881 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.1117483730000002, 2024-09-14T22:36:26.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030007875999999999, 2024-09-14T22:36:26.881 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:26.882 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:26.882 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:26.882 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:26.882 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:26.882 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:26.882 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4074 2024-09-14T22:36:26.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.882 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:26.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:26.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680269+0000", 2024-09-14T22:36:26.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:26.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680269+0000", 2024-09-14T22:36:26.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:26.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:26.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680269+0000", 2024-09-14T22:36:26.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7300000000000001e-07 2024-09-14T22:36:26.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:26.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680277+0000", 2024-09-14T22:36:26.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8480000000000001e-06 2024-09-14T22:36:26.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:26.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680286+0000", 2024-09-14T22:36:26.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0529999999999996e-06 2024-09-14T22:36:26.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:26.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680292+0000", 2024-09-14T22:36:26.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.728e-06 2024-09-14T22:36:26.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:26.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.710024+0000", 2024-09-14T22:36:26.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029732176999999999 2024-09-14T22:36:26.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:26.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.710109+0000", 2024-09-14T22:36:26.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4715000000000005e-05 2024-09-14T22:36:26.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:26.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:26.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:26.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.710277+0000", 2024-09-14T22:36:26.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000167882 2024-09-14T22:36:26.885 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.885 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:26.885 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.885 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:26.885 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:26.885 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:26.889 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:26.879+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:26.889 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:36:26.989 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:26.983+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:27.054 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:27.054 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:36:27.055 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:36:27.055 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:27.069 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:36:27.090 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:27.083+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:27.191 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:27.187+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:27.193 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:27.193 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:36:27.193 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:36:27.194 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:36:27.194 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:27.209 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:36:27.293 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:27.287+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:27.330 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:27.330 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:36:27.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:36:27.330 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:36:27.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.331 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4149 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13129728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:27.331 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703281+0000", 2024-09-14T22:36:27.331 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.6235179730000002, 2024-09-14T22:36:27.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045957114, 2024-09-14T22:36:27.331 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:27.331 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:27.331 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:27.331 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:27.331 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:27.331 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:27.331 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4149 2024-09-14T22:36:27.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.331 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:27.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:27.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703281+0000", 2024-09-14T22:36:27.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:27.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703281+0000", 2024-09-14T22:36:27.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:27.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703282+0000", 2024-09-14T22:36:27.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4199999999999996e-07 2024-09-14T22:36:27.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:27.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703289+0000", 2024-09-14T22:36:27.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0770000000000002e-06 2024-09-14T22:36:27.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:27.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703296+0000", 2024-09-14T22:36:27.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8129999999999998e-06 2024-09-14T22:36:27.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:27.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703307+0000", 2024-09-14T22:36:27.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1663e-05 2024-09-14T22:36:27.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:27.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749073+0000", 2024-09-14T22:36:27.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.04576537 2024-09-14T22:36:27.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:27.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749156+0000", 2024-09-14T22:36:27.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3399999999999994e-05 2024-09-14T22:36:27.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:27.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749239+0000", 2024-09-14T22:36:27.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2249000000000005e-05 2024-09-14T22:36:27.335 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.335 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:27.335 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.335 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4150 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13137920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:27.335 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703318+0000", 2024-09-14T22:36:27.335 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.623481779, 2024-09-14T22:36:27.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046170585, 2024-09-14T22:36:27.335 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:27.336 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:27.336 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:27.336 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:27.336 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:27.336 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:27.336 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4150 2024-09-14T22:36:27.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.336 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:27.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:27.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703318+0000", 2024-09-14T22:36:27.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:27.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703318+0000", 2024-09-14T22:36:27.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:27.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703318+0000", 2024-09-14T22:36:27.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8800000000000003e-07 2024-09-14T22:36:27.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:27.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703321+0000", 2024-09-14T22:36:27.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7860000000000001e-06 2024-09-14T22:36:27.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:27.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703328+0000", 2024-09-14T22:36:27.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2810000000000003e-06 2024-09-14T22:36:27.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:27.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703340+0000", 2024-09-14T22:36:27.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1745e-05 2024-09-14T22:36:27.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:27.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749286+0000", 2024-09-14T22:36:27.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045946222000000002 2024-09-14T22:36:27.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:27.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749390+0000", 2024-09-14T22:36:27.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000103614 2024-09-14T22:36:27.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:27.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749488+0000", 2024-09-14T22:36:27.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8449000000000006e-05 2024-09-14T22:36:27.340 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.340 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:27.340 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.340 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4151 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13146112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:27.340 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703348+0000", 2024-09-14T22:36:27.340 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.6234516160000001, 2024-09-14T22:36:27.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046349509999999997, 2024-09-14T22:36:27.340 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:27.340 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:27.340 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:27.341 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:27.341 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:27.341 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:27.341 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4151 2024-09-14T22:36:27.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.341 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:27.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:27.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703348+0000", 2024-09-14T22:36:27.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:27.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703348+0000", 2024-09-14T22:36:27.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:27.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703348+0000", 2024-09-14T22:36:27.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6899999999999998e-07 2024-09-14T22:36:27.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:27.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703351+0000", 2024-09-14T22:36:27.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5950000000000001e-06 2024-09-14T22:36:27.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:27.343 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703357+0000", 2024-09-14T22:36:27.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4150000000000004e-06 2024-09-14T22:36:27.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.343 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:27.343 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703364+0000", 2024-09-14T22:36:27.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4069999999999998e-06 2024-09-14T22:36:27.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.343 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:27.343 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749530+0000", 2024-09-14T22:36:27.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046166281000000003 2024-09-14T22:36:27.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:27.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749615+0000", 2024-09-14T22:36:27.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5409999999999997e-05 2024-09-14T22:36:27.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:27.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749697+0000", 2024-09-14T22:36:27.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1933000000000003e-05 2024-09-14T22:36:27.344 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.344 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:27.344 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.345 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4152 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13154304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:27.345 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703371+0000", 2024-09-14T22:36:27.345 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.623428214, 2024-09-14T22:36:27.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046552233999999998, 2024-09-14T22:36:27.345 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:27.345 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:27.345 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:27.345 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:27.345 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:27.345 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:27.345 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4152 2024-09-14T22:36:27.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.346 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:27.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:27.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703371+0000", 2024-09-14T22:36:27.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:27.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703371+0000", 2024-09-14T22:36:27.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:27.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703371+0000", 2024-09-14T22:36:27.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.34e-07 2024-09-14T22:36:27.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:27.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703374+0000", 2024-09-14T22:36:27.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.227e-06 2024-09-14T22:36:27.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:27.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703380+0000", 2024-09-14T22:36:27.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4699999999999999e-06 2024-09-14T22:36:27.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:27.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703385+0000", 2024-09-14T22:36:27.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.259e-06 2024-09-14T22:36:27.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:27.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749742+0000", 2024-09-14T22:36:27.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046357031 2024-09-14T22:36:27.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:27.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749825+0000", 2024-09-14T22:36:27.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2674000000000002e-05 2024-09-14T22:36:27.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:27.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749923+0000", 2024-09-14T22:36:27.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8239000000000006e-05 2024-09-14T22:36:27.349 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.349 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:27.349 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.349 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4153 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13162496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:27.349 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703392+0000", 2024-09-14T22:36:27.349 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.6234069180000001, 2024-09-14T22:36:27.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046675756999999998, 2024-09-14T22:36:27.350 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:27.350 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:27.350 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:27.350 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:27.350 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:27.350 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:27.350 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4153 2024-09-14T22:36:27.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.350 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:27.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:27.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703392+0000", 2024-09-14T22:36:27.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:27.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703392+0000", 2024-09-14T22:36:27.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:27.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.705934+0000", 2024-09-14T22:36:27.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025418540000000001 2024-09-14T22:36:27.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:27.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.705940+0000", 2024-09-14T22:36:27.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0950000000000001e-06 2024-09-14T22:36:27.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:27.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.705952+0000", 2024-09-14T22:36:27.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1973e-05 2024-09-14T22:36:27.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:27.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.705958+0000", 2024-09-14T22:36:27.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.541e-06 2024-09-14T22:36:27.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:27.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749969+0000", 2024-09-14T22:36:27.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044011300000000003 2024-09-14T22:36:27.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:27.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.750020+0000", 2024-09-14T22:36:27.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1054000000000003e-05 2024-09-14T22:36:27.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:27.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.750068+0000", 2024-09-14T22:36:27.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7939999999999998e-05 2024-09-14T22:36:27.354 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.354 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:27.354 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.354 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4164 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13252608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:27.354 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706129+0000", 2024-09-14T22:36:27.354 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.6206704410000001, 2024-09-14T22:36:27.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045833060000000002, 2024-09-14T22:36:27.355 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:27.355 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:27.355 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:27.355 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:27.355 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:27.355 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:27.355 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4164 2024-09-14T22:36:27.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.355 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:27.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:27.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706129+0000", 2024-09-14T22:36:27.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:27.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706129+0000", 2024-09-14T22:36:27.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:27.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706129+0000", 2024-09-14T22:36:27.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0699999999999998e-07 2024-09-14T22:36:27.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:27.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706132+0000", 2024-09-14T22:36:27.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3420000000000002e-06 2024-09-14T22:36:27.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:27.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706139+0000", 2024-09-14T22:36:27.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8619999999999997e-06 2024-09-14T22:36:27.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:27.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706143+0000", 2024-09-14T22:36:27.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8060000000000002e-06 2024-09-14T22:36:27.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:27.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.751792+0000", 2024-09-14T22:36:27.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045648954999999998 2024-09-14T22:36:27.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:27.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.751878+0000", 2024-09-14T22:36:27.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5316999999999996e-05 2024-09-14T22:36:27.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:27.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.751962+0000", 2024-09-14T22:36:27.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4370999999999996e-05 2024-09-14T22:36:27.358 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.358 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:27.358 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.359 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4165 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13260800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:27.359 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706178+0000", 2024-09-14T22:36:27.359 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.6206210059999999, 2024-09-14T22:36:27.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045915594999999997, 2024-09-14T22:36:27.359 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:27.359 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:27.359 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:27.359 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:27.359 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:27.359 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:27.359 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4165 2024-09-14T22:36:27.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.360 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:27.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:27.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706178+0000", 2024-09-14T22:36:27.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:27.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706178+0000", 2024-09-14T22:36:27.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:27.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706179+0000", 2024-09-14T22:36:27.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4999999999999998e-07 2024-09-14T22:36:27.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:27.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706181+0000", 2024-09-14T22:36:27.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4679999999999999e-06 2024-09-14T22:36:27.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:27.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706189+0000", 2024-09-14T22:36:27.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.994e-06 2024-09-14T22:36:27.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:27.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706195+0000", 2024-09-14T22:36:27.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6119999999999998e-06 2024-09-14T22:36:27.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:27.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752003+0000", 2024-09-14T22:36:27.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045807618000000001 2024-09-14T22:36:27.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:27.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752050+0000", 2024-09-14T22:36:27.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7301e-05 2024-09-14T22:36:27.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:27.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752094+0000", 2024-09-14T22:36:27.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4152e-05 2024-09-14T22:36:27.363 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.363 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:27.363 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.364 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4166 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13268992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:27.364 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706205+0000", 2024-09-14T22:36:27.364 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.6205948169999997, 2024-09-14T22:36:27.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046092665999999997, 2024-09-14T22:36:27.364 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:27.364 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:27.364 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:27.364 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:27.364 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:27.364 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:27.364 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4166 2024-09-14T22:36:27.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.364 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:27.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:27.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706205+0000", 2024-09-14T22:36:27.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:27.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706205+0000", 2024-09-14T22:36:27.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:27.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706205+0000", 2024-09-14T22:36:27.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:36:27.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:27.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706208+0000", 2024-09-14T22:36:27.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8049999999999998e-06 2024-09-14T22:36:27.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:27.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706215+0000", 2024-09-14T22:36:27.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2010000000000002e-06 2024-09-14T22:36:27.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:27.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706223+0000", 2024-09-14T22:36:27.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7260000000000002e-06 2024-09-14T22:36:27.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:27.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752129+0000", 2024-09-14T22:36:27.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045906532 2024-09-14T22:36:27.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:27.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752213+0000", 2024-09-14T22:36:27.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4060000000000005e-05 2024-09-14T22:36:27.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:27.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752297+0000", 2024-09-14T22:36:27.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3807999999999994e-05 2024-09-14T22:36:27.368 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.368 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:27.368 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.368 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4167 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13277184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:27.368 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706233+0000", 2024-09-14T22:36:27.368 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.6205662059999999, 2024-09-14T22:36:27.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046190500000000002, 2024-09-14T22:36:27.369 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:27.369 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:27.369 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:27.369 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:27.369 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:27.369 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:27.369 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4167 2024-09-14T22:36:27.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.369 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:27.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:27.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706233+0000", 2024-09-14T22:36:27.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:27.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706233+0000", 2024-09-14T22:36:27.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:27.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706234+0000", 2024-09-14T22:36:27.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.75e-07 2024-09-14T22:36:27.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706237+0000", 2024-09-14T22:36:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9890000000000001e-06 2024-09-14T22:36:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706244+0000", 2024-09-14T22:36:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6240000000000002e-06 2024-09-14T22:36:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706255+0000", 2024-09-14T22:36:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0328e-05 2024-09-14T22:36:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752338+0000", 2024-09-14T22:36:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046083172999999998 2024-09-14T22:36:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752381+0000", 2024-09-14T22:36:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2973999999999999e-05 2024-09-14T22:36:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752424+0000", 2024-09-14T22:36:27.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2936999999999999e-05 2024-09-14T22:36:27.373 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.373 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:27.373 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.373 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4168 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13285376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:27.373 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706261+0000", 2024-09-14T22:36:27.373 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.6205388620000001, 2024-09-14T22:36:27.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046378635000000001, 2024-09-14T22:36:27.373 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:27.373 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:27.374 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:27.374 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:27.374 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:27.374 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:27.374 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4168 2024-09-14T22:36:27.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.374 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:27.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:27.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706261+0000", 2024-09-14T22:36:27.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:27.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706261+0000", 2024-09-14T22:36:27.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:27.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706261+0000", 2024-09-14T22:36:27.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2899999999999999e-07 2024-09-14T22:36:27.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:27.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706264+0000", 2024-09-14T22:36:27.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.774e-06 2024-09-14T22:36:27.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:27.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706267+0000", 2024-09-14T22:36:27.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7179999999999998e-06 2024-09-14T22:36:27.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:27.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706274+0000", 2024-09-14T22:36:27.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7599999999999997e-06 2024-09-14T22:36:27.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:27.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752458+0000", 2024-09-14T22:36:27.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046183825999999997 2024-09-14T22:36:27.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:27.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752545+0000", 2024-09-14T22:36:27.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7272000000000001e-05 2024-09-14T22:36:27.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:27.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752639+0000", 2024-09-14T22:36:27.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3956000000000006e-05 2024-09-14T22:36:27.378 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.378 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:27.378 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.378 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4169 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13293568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:27.378 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706278+0000", 2024-09-14T22:36:27.378 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.6205211349999997, 2024-09-14T22:36:27.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046489181999999997, 2024-09-14T22:36:27.378 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:27.378 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:27.378 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:27.379 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:27.379 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:27.379 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:27.379 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4169 2024-09-14T22:36:27.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.379 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:27.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.379 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:27.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706278+0000", 2024-09-14T22:36:27.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:27.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706278+0000", 2024-09-14T22:36:27.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:27.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706279+0000", 2024-09-14T22:36:27.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6100000000000002e-07 2024-09-14T22:36:27.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:27.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706280+0000", 2024-09-14T22:36:27.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2589999999999999e-06 2024-09-14T22:36:27.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:27.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706283+0000", 2024-09-14T22:36:27.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.501e-06 2024-09-14T22:36:27.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:27.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706286+0000", 2024-09-14T22:36:27.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5409999999999999e-06 2024-09-14T22:36:27.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:27.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752680+0000", 2024-09-14T22:36:27.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046393886000000002 2024-09-14T22:36:27.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:27.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752723+0000", 2024-09-14T22:36:27.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2870000000000001e-05 2024-09-14T22:36:27.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:27.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752767+0000", 2024-09-14T22:36:27.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4864000000000003e-05 2024-09-14T22:36:27.383 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.383 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:27.383 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.383 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4170 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13301760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:27.383 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706289+0000", 2024-09-14T22:36:27.383 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.620510237, 2024-09-14T22:36:27.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046691115999999998, 2024-09-14T22:36:27.384 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:27.384 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:27.384 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:27.384 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:27.384 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:27.384 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:27.384 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4170 2024-09-14T22:36:27.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.384 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:27.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:27.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706289+0000", 2024-09-14T22:36:27.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.385 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:27.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706289+0000", 2024-09-14T22:36:27.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.385 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:27.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706289+0000", 2024-09-14T22:36:27.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3200000000000001e-07 2024-09-14T22:36:27.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:27.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706296+0000", 2024-09-14T22:36:27.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.922e-06 2024-09-14T22:36:27.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:27.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706303+0000", 2024-09-14T22:36:27.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3980000000000004e-06 2024-09-14T22:36:27.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:27.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706307+0000", 2024-09-14T22:36:27.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6750000000000001e-06 2024-09-14T22:36:27.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:27.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752799+0000", 2024-09-14T22:36:27.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046491609000000003 2024-09-14T22:36:27.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:27.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752884+0000", 2024-09-14T22:36:27.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4876999999999994e-05 2024-09-14T22:36:27.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:27.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752980+0000", 2024-09-14T22:36:27.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6403000000000005e-05 2024-09-14T22:36:27.387 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.388 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:27.388 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.388 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6292 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8329216~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:27.388 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966066+0000", 2024-09-14T22:36:27.388 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.3607332990000001, 2024-09-14T22:36:27.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045967367000000002, 2024-09-14T22:36:27.388 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:27.388 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:27.388 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:27.388 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:27.388 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:27.388 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:27.389 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6292 2024-09-14T22:36:27.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.389 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:27.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:27.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966066+0000", 2024-09-14T22:36:27.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:27.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966066+0000", 2024-09-14T22:36:27.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:27.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966067+0000", 2024-09-14T22:36:27.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6499999999999999e-07 2024-09-14T22:36:27.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:27.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966069+0000", 2024-09-14T22:36:27.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4059999999999999e-06 2024-09-14T22:36:27.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:27.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966077+0000", 2024-09-14T22:36:27.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5120000000000002e-06 2024-09-14T22:36:27.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:27.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966083+0000", 2024-09-14T22:36:27.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5590000000000002e-06 2024-09-14T22:36:27.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:27.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.011820+0000", 2024-09-14T22:36:27.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045737281999999997 2024-09-14T22:36:27.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:27.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.011911+0000", 2024-09-14T22:36:27.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0153000000000002e-05 2024-09-14T22:36:27.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:27.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012033+0000", 2024-09-14T22:36:27.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012289000000000001 2024-09-14T22:36:27.392 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.392 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:27.392 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.392 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6293 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8337408~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:27.392 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966091+0000", 2024-09-14T22:36:27.393 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.3607082049999999, 2024-09-14T22:36:27.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046181672, 2024-09-14T22:36:27.393 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:27.393 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:27.393 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:27.393 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:27.393 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:27.393 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:27.393 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6293 2024-09-14T22:36:27.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.393 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:27.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:27.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966091+0000", 2024-09-14T22:36:27.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:27.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966091+0000", 2024-09-14T22:36:27.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:27.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966092+0000", 2024-09-14T22:36:27.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:36:27.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:27.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966094+0000", 2024-09-14T22:36:27.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5830000000000001e-06 2024-09-14T22:36:27.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:27.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966102+0000", 2024-09-14T22:36:27.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7470000000000005e-06 2024-09-14T22:36:27.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:27.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966107+0000", 2024-09-14T22:36:27.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1209999999999998e-06 2024-09-14T22:36:27.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:27.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012079+0000", 2024-09-14T22:36:27.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045971953000000003 2024-09-14T22:36:27.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:27.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012168+0000", 2024-09-14T22:36:27.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9398e-05 2024-09-14T22:36:27.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:27.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012273+0000", 2024-09-14T22:36:27.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000104483 2024-09-14T22:36:27.396 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.396 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:27.396 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.397 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6294 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8345600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:27.397 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966115+0000", 2024-09-14T22:36:27.397 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.3606848419999999, 2024-09-14T22:36:27.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046297036999999999, 2024-09-14T22:36:27.397 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:27.397 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:27.397 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:27.397 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:27.397 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:27.397 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:27.397 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6294 2024-09-14T22:36:27.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.398 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:27.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:27.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966115+0000", 2024-09-14T22:36:27.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:27.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966115+0000", 2024-09-14T22:36:27.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:27.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966115+0000", 2024-09-14T22:36:27.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7e-07 2024-09-14T22:36:27.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:27.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966118+0000", 2024-09-14T22:36:27.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.571e-06 2024-09-14T22:36:27.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:27.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966125+0000", 2024-09-14T22:36:27.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1690000000000003e-06 2024-09-14T22:36:27.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:27.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966130+0000", 2024-09-14T22:36:27.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3290000000000001e-06 2024-09-14T22:36:27.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:27.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012298+0000", 2024-09-14T22:36:27.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046168168000000002 2024-09-14T22:36:27.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:27.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012348+0000", 2024-09-14T22:36:27.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9663999999999997e-05 2024-09-14T22:36:27.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:27.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012412+0000", 2024-09-14T22:36:27.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3665999999999997e-05 2024-09-14T22:36:27.401 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.401 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:27.401 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.401 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6295 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8353792~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:27.401 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966138+0000", 2024-09-14T22:36:27.402 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.3606618639999999, 2024-09-14T22:36:27.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046505112000000001, 2024-09-14T22:36:27.402 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:27.402 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:27.402 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:27.402 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:27.402 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:27.402 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:27.402 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6295 2024-09-14T22:36:27.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.402 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:27.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:27.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966138+0000", 2024-09-14T22:36:27.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:27.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966138+0000", 2024-09-14T22:36:27.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:27.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966138+0000", 2024-09-14T22:36:27.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5900000000000002e-07 2024-09-14T22:36:27.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:27.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966140+0000", 2024-09-14T22:36:27.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4710000000000001e-06 2024-09-14T22:36:27.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:27.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966147+0000", 2024-09-14T22:36:27.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6760000000000001e-06 2024-09-14T22:36:27.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:27.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966153+0000", 2024-09-14T22:36:27.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4779999999999999e-06 2024-09-14T22:36:27.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:27.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012452+0000", 2024-09-14T22:36:27.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046299298000000003 2024-09-14T22:36:27.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:27.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012537+0000", 2024-09-14T22:36:27.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5439999999999995e-05 2024-09-14T22:36:27.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:27.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012643+0000", 2024-09-14T22:36:27.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010529 2024-09-14T22:36:27.406 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.406 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:27.406 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.406 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6296 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8361984~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:27.406 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966160+0000", 2024-09-14T22:36:27.406 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.3606398099999999, 2024-09-14T22:36:27.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046601423000000003, 2024-09-14T22:36:27.406 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:27.406 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:27.406 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:27.407 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:27.407 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:27.407 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:27.407 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6296 2024-09-14T22:36:27.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.407 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:27.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:27.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966160+0000", 2024-09-14T22:36:27.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:27.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966160+0000", 2024-09-14T22:36:27.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:27.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966160+0000", 2024-09-14T22:36:27.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7999999999999995e-07 2024-09-14T22:36:27.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:27.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966163+0000", 2024-09-14T22:36:27.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.514e-06 2024-09-14T22:36:27.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:27.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966169+0000", 2024-09-14T22:36:27.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8880000000000002e-06 2024-09-14T22:36:27.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:27.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966173+0000", 2024-09-14T22:36:27.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9189999999999998e-06 2024-09-14T22:36:27.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:27.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012660+0000", 2024-09-14T22:36:27.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046486522000000002 2024-09-14T22:36:27.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:27.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012703+0000", 2024-09-14T22:36:27.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2715e-05 2024-09-14T22:36:27.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:27.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012761+0000", 2024-09-14T22:36:27.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8285000000000002e-05 2024-09-14T22:36:27.410 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.410 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:27.410 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.411 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6297 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8370176~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:27.411 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966180+0000", 2024-09-14T22:36:27.411 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.3606190059999999, 2024-09-14T22:36:27.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046808482999999998, 2024-09-14T22:36:27.411 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:27.411 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:27.411 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:27.411 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:27.411 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:27.411 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:27.411 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6297 2024-09-14T22:36:27.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.412 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:27.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:27.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966180+0000", 2024-09-14T22:36:27.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:27.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966180+0000", 2024-09-14T22:36:27.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:27.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966181+0000", 2024-09-14T22:36:27.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0600000000000001e-07 2024-09-14T22:36:27.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:27.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966183+0000", 2024-09-14T22:36:27.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3539999999999998e-06 2024-09-14T22:36:27.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:27.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966190+0000", 2024-09-14T22:36:27.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.489e-06 2024-09-14T22:36:27.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:27.414 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966195+0000", 2024-09-14T22:36:27.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3890000000000004e-06 2024-09-14T22:36:27.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:27.414 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012799+0000", 2024-09-14T22:36:27.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046604200999999998 2024-09-14T22:36:27.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:27.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012883+0000", 2024-09-14T22:36:27.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3714000000000005e-05 2024-09-14T22:36:27.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:27.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012989+0000", 2024-09-14T22:36:27.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010593 2024-09-14T22:36:27.415 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.415 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:27.415 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.415 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6298 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8378368~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:27.415 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966203+0000", 2024-09-14T22:36:27.415 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.360595945, 2024-09-14T22:36:27.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046904649999999999, 2024-09-14T22:36:27.416 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:27.416 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:27.416 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:27.416 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:27.416 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:27.416 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:27.416 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6298 2024-09-14T22:36:27.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.416 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:27.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:27.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966203+0000", 2024-09-14T22:36:27.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:27.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966203+0000", 2024-09-14T22:36:27.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:27.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966205+0000", 2024-09-14T22:36:27.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5009999999999999e-06 2024-09-14T22:36:27.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:27.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966207+0000", 2024-09-14T22:36:27.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2019999999999998e-06 2024-09-14T22:36:27.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:27.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966214+0000", 2024-09-14T22:36:27.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.815e-06 2024-09-14T22:36:27.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:27.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966219+0000", 2024-09-14T22:36:27.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0590000000000002e-06 2024-09-14T22:36:27.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:27.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013005+0000", 2024-09-14T22:36:27.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046785852000000003 2024-09-14T22:36:27.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:27.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013050+0000", 2024-09-14T22:36:27.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.511e-05 2024-09-14T22:36:27.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:27.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013108+0000", 2024-09-14T22:36:27.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8111e-05 2024-09-14T22:36:27.419 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.419 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:27.419 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.420 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6299 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8386560~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:27.420 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966238+0000", 2024-09-14T22:36:27.420 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.3605616239999998, 2024-09-14T22:36:27.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047091099999999997, 2024-09-14T22:36:27.420 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:27.420 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:27.420 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:27.420 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:27.420 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:27.420 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:27.420 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6299 2024-09-14T22:36:27.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.420 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:27.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966238+0000", 2024-09-14T22:36:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966238+0000", 2024-09-14T22:36:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966238+0000", 2024-09-14T22:36:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1500000000000005e-07 2024-09-14T22:36:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:27.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966241+0000", 2024-09-14T22:36:27.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8270000000000002e-06 2024-09-14T22:36:27.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:27.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966249+0000", 2024-09-14T22:36:27.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0029999999999994e-06 2024-09-14T22:36:27.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:27.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966255+0000", 2024-09-14T22:36:27.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7760000000000003e-06 2024-09-14T22:36:27.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:27.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013136+0000", 2024-09-14T22:36:27.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046881379000000001 2024-09-14T22:36:27.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:27.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013223+0000", 2024-09-14T22:36:27.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6681000000000006e-05 2024-09-14T22:36:27.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:27.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:27.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:27.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013329+0000", 2024-09-14T22:36:27.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105919 2024-09-14T22:36:27.423 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.424 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:27.424 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.424 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:27.424 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:27.424 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:27.425 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:36:27.426 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:27.423+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:27.503 INFO:teuthology.orchestra.run.smithi001.stdout:3.9 2024-09-14T22:36:27.503 INFO:teuthology.orchestra.run.smithi001.stdout:3.4 2024-09-14T22:36:27.503 INFO:teuthology.orchestra.run.smithi001.stdout:2.3 2024-09-14T22:36:27.503 INFO:teuthology.orchestra.run.smithi001.stdout:3.0 2024-09-14T22:36:27.503 INFO:teuthology.orchestra.run.smithi001.stdout:2.1 2024-09-14T22:36:27.526 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:27.519+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:27.549 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:27.549 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:27.564 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:36:27.627 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:27.623+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:27.693 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:27.693 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:27.707 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:36:27.758 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:27.754+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:27.780 INFO:teuthology.orchestra.run.smithi001.stdout:407 write 0x3f9f353 thru 0x3fabdf0 (0xca9e bytes) 2024-09-14T22:36:27.806 INFO:teuthology.orchestra.run.smithi001.stdout:408 trunc from 0xe2731d5 to 0xbd0cd89 2024-09-14T22:36:27.819 INFO:teuthology.orchestra.run.smithi001.stdout:410 write 0x7e641cb thru 0x7e6e9ac (0xa7e2 bytes) 2024-09-14T22:36:27.829 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:27.823+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:27.834 INFO:teuthology.orchestra.run.smithi001.stdout:412 read 0x1467d86 thru 0x1477649 (0xf8c4 bytes) 2024-09-14T22:36:27.834 INFO:teuthology.orchestra.run.smithi001.stdout:413 writesame 0x1f6d026 thru 0x1f75a80 data_size 0x1(0x8a5b bytes) 2024-09-14T22:36:27.843 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:27.843 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:27.858 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:36:27.861 INFO:teuthology.orchestra.run.smithi001.stdout:414 write 0x3b76512 thru 0x3b7e83d (0x832c bytes) 2024-09-14T22:36:27.865 INFO:teuthology.orchestra.run.smithi001.stdout:416 punch from 0x7f37a34 to 0x7f45296, (0xd862 bytes) 2024-09-14T22:36:27.865 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:27.865 INFO:teuthology.orchestra.run.smithi001.stdout:418 write 0x3cdbc53 thru 0x3cdcd9a (0x1148 bytes) 2024-09-14T22:36:27.887 INFO:teuthology.orchestra.run.smithi001.stdout:419 trunc from 0xbd0cd89 to 0x313aa91 2024-09-14T22:36:27.929 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:27.922+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:27.937 INFO:teuthology.orchestra.run.smithi001.stdout:420 writesame 0xd53518f thru 0xd53e2ee data_size 0x10(0x9160 bytes) 2024-09-14T22:36:27.937 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:27.947 INFO:teuthology.orchestra.run.smithi001.stdout:423 write 0xe65e3de thru 0xe6683ed (0xa010 bytes) 2024-09-14T22:36:27.955 INFO:teuthology.orchestra.run.smithi001.stdout:425 read 0x823b3e4 thru 0x823eb76 (0x3793 bytes) 2024-09-14T22:36:27.955 INFO:teuthology.orchestra.run.smithi001.stdout:426 write 0x29f31c4 thru 0x29f97b3 (0x65f0 bytes) 2024-09-14T22:36:27.959 INFO:teuthology.orchestra.run.smithi001.stdout:427 write 0x3d59851 thru 0x3d696af (0xfe5f bytes) 2024-09-14T22:36:27.969 INFO:teuthology.orchestra.run.smithi001.stdout:428 read 0x29209ba thru 0x2926cb6 (0x62fd bytes) 2024-09-14T22:36:27.969 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:27.970 INFO:teuthology.orchestra.run.smithi001.stdout:431 write 0x638de0f thru 0x63927f8 (0x49ea bytes) 2024-09-14T22:36:27.979 INFO:teuthology.orchestra.run.smithi001.stdout:435 write 0x5772c9e thru 0x5780049 (0xd3ac bytes) 2024-09-14T22:36:27.990 INFO:teuthology.orchestra.run.smithi001.stdout:437 read 0x8ba48b5 thru 0x8bb1c87 (0xd3d3 bytes) 2024-09-14T22:36:27.990 INFO:teuthology.orchestra.run.smithi001.stdout:438 punch from 0x1aa3d29 to 0x1aa9a31, (0x5d08 bytes) 2024-09-14T22:36:27.990 INFO:teuthology.orchestra.run.smithi001.stdout:439 read 0x7a362c1 thru 0x7a4343e (0xd17e bytes) 2024-09-14T22:36:28.002 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:28.003 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:28.010 INFO:teuthology.orchestra.run.smithi001.stdout:440 trunc from 0xe6683ee to 0x5d2dcf0 2024-09-14T22:36:28.017 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:36:28.035 INFO:teuthology.orchestra.run.smithi001.stdout:441 read 0x3186b05 thru 0x3191d42 (0xb23e bytes) 2024-09-14T22:36:28.045 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal --op export --pgid 3.4 --file /var/log/ceph/exp.3.4.2 2024-09-14T22:36:28.051 INFO:teuthology.orchestra.run.smithi001.stdout:442 trunc from 0x5d2dcf0 to 0xc507ecd 2024-09-14T22:36:28.059 INFO:teuthology.orchestra.run.smithi001.stdout:443 read 0x53ade37 thru 0x53ba225 (0xc3ef bytes) 2024-09-14T22:36:28.059 INFO:teuthology.orchestra.run.smithi001.stdout:444 write 0x899ccd thru 0x8a5937 (0xbc6b bytes) 2024-09-14T22:36:28.067 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:28.063+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:28.069 INFO:teuthology.orchestra.run.smithi001.stdout:445 writesame 0x51d4e57 thru 0x51d6c64 data_size 0x1(0x1e0e bytes) 2024-09-14T22:36:28.079 INFO:teuthology.orchestra.run.smithi001.stdout:447 writesame 0xf71e32 thru 0xf73902 data_size 0x1(0x1ad1 bytes) 2024-09-14T22:36:28.086 INFO:teuthology.orchestra.run.smithi001.stdout:448 write 0x55b374a thru 0x55b67e4 (0x309b bytes) 2024-09-14T22:36:28.093 INFO:teuthology.orchestra.run.smithi001.stdout:451 read 0x818fd27 thru 0x81988f5 (0x8bcf bytes) 2024-09-14T22:36:28.093 INFO:teuthology.orchestra.run.smithi001.stdout:452 punch from 0x855198e to 0x8553322, (0x1994 bytes) 2024-09-14T22:36:28.093 INFO:teuthology.orchestra.run.smithi001.stdout:455 punch from 0xb3023c9 to 0xb305f61, (0x3b98 bytes) 2024-09-14T22:36:28.093 INFO:teuthology.orchestra.run.smithi001.stdout:456 write 0x31709a9 thru 0x3173d60 (0x33b8 bytes) 2024-09-14T22:36:28.100 INFO:teuthology.orchestra.run.smithi001.stdout:457 read 0x6322297 thru 0x6330fdd (0xed47 bytes) 2024-09-14T22:36:28.100 INFO:teuthology.orchestra.run.smithi001.stdout:458 read 0x8c82510 thru 0x8c8bbe8 (0x96d9 bytes) 2024-09-14T22:36:28.100 INFO:teuthology.orchestra.run.smithi001.stdout:460 read 0xeb2deb thru 0xebd67f (0xa895 bytes) 2024-09-14T22:36:28.100 INFO:teuthology.orchestra.run.smithi001.stdout:461 read 0xa2843db thru 0xa28a4fc (0x6122 bytes) 2024-09-14T22:36:28.101 INFO:teuthology.orchestra.run.smithi001.stdout:462 write 0xbc090e0 thru 0xbc17726 (0xe647 bytes) 2024-09-14T22:36:28.107 INFO:teuthology.orchestra.run.smithi001.stdout:463 punch from 0xb82bf91 to 0xb8387c8, (0xc837 bytes) 2024-09-14T22:36:28.107 INFO:teuthology.orchestra.run.smithi001.stdout:464 write 0xb085f2a thru 0xb08ee4d (0x8f24 bytes) 2024-09-14T22:36:28.113 INFO:teuthology.orchestra.run.smithi001.stdout:465 write 0x667d3f2 thru 0x66820ae (0x4cbd bytes) 2024-09-14T22:36:28.122 INFO:teuthology.orchestra.run.smithi001.stdout:466 read 0xb097d01 thru 0xb0a24c8 (0xa7c8 bytes) 2024-09-14T22:36:28.122 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:28.125 INFO:teuthology.orchestra.run.smithi001.stdout:468 trunc from 0xc507ecd to 0xdb79616 2024-09-14T22:36:28.130 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:28.123+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:28.133 INFO:teuthology.orchestra.run.smithi001.stdout:469 read 0xb815f56 thru 0xb81f206 (0x92b1 bytes) 2024-09-14T22:36:28.134 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:28.151 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:28.151 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:28.153 INFO:teuthology.orchestra.run.smithi001.stdout:474 trunc from 0xdb79616 to 0x22029e4 2024-09-14T22:36:28.168 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:36:28.200 INFO:teuthology.orchestra.run.smithi001.stdout:476 trunc from 0x22029e4 to 0xb90c8a1 2024-09-14T22:36:28.209 INFO:teuthology.orchestra.run.smithi001.stdout:477 write 0x8fcde54 thru 0x8fd4706 (0x68b3 bytes) 2024-09-14T22:36:28.218 INFO:teuthology.orchestra.run.smithi001.stdout:478 write 0x98dc397 thru 0x98e3a25 (0x768f bytes) 2024-09-14T22:36:28.227 INFO:teuthology.orchestra.run.smithi001.stdout:481 read 0x176fd29 thru 0x17778bf (0x7b97 bytes) 2024-09-14T22:36:28.227 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:28.227 INFO:teuthology.orchestra.run.smithi001.stdout:484 read 0x50522f9 thru 0x5057b67 (0x586f bytes) 2024-09-14T22:36:28.227 INFO:teuthology.orchestra.run.smithi001.stdout:485 writesame 0x8f91b64 thru 0x8f9a4c8 data_size 0x1(0x8965 bytes) 2024-09-14T22:36:28.237 INFO:teuthology.orchestra.run.smithi001.stdout:486 read 0x89ff80e thru 0x8a00847 (0x103a bytes) 2024-09-14T22:36:28.237 INFO:teuthology.orchestra.run.smithi001.stdout:487 read 0x432e339 thru 0x4330f35 (0x2bfd bytes) 2024-09-14T22:36:28.237 INFO:teuthology.orchestra.run.smithi001.stdout:489 write 0x272dbfe thru 0x2733473 (0x5876 bytes) 2024-09-14T22:36:28.248 INFO:teuthology.orchestra.run.smithi001.stdout:490 read 0x12cbb51 thru 0x12cf634 (0x3ae4 bytes) 2024-09-14T22:36:28.248 INFO:teuthology.orchestra.run.smithi001.stdout:491 read 0x7e4fa2c thru 0x7e54b68 (0x513d bytes) 2024-09-14T22:36:28.248 INFO:teuthology.orchestra.run.smithi001.stdout:492 punch from 0x88456f1 to 0x8850014, (0xa923 bytes) 2024-09-14T22:36:28.248 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:28.255 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:28.251+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:28.271 INFO:teuthology.orchestra.run.smithi001.stdout:494 trunc from 0xb90c8a1 to 0xe374b 2024-09-14T22:36:28.332 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:28.326+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:28.353 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:28.353 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:28.367 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:36:28.400 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:28.410 INFO:teuthology.orchestra.run.smithi001.stdout:496 writesame 0x19c646c thru 0x19c7217 data_size 0x23(0xdac bytes) 2024-09-14T22:36:28.440 INFO:teuthology.orchestra.run.smithi001.stdout:497 write 0xe169590 thru 0xe16b139 (0x1baa bytes) 2024-09-14T22:36:28.447 INFO:teuthology.orchestra.run.smithi001.stdout:498 punch from 0xcef89d5 to 0xcf07874, (0xee9f bytes) 2024-09-14T22:36:28.447 INFO:teuthology.orchestra.run.smithi001.stdout:499 write 0xcfaf6a5 thru 0xcfb4945 (0x52a1 bytes) 2024-09-14T22:36:28.455 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:28.456 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:28.459 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:28.455+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:28.467 INFO:teuthology.orchestra.run.smithi001.stdout:502 trunc from 0xe16b13a to 0x7560259 2024-09-14T22:36:28.489 INFO:teuthology.orchestra.run.smithi001.stdout:503 read 0x566fec7 thru 0x56761b0 (0x62ea bytes) 2024-09-14T22:36:28.510 INFO:teuthology.orchestra.run.smithi001.stdout:504 write 0xbe848df thru 0xbe85637 (0xd59 bytes) 2024-09-14T22:36:28.529 INFO:teuthology.orchestra.run.smithi001.stdout:505 trunc from 0xbe85638 to 0x57df1ad 2024-09-14T22:36:28.534 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:28.534 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:28.535 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:28.526+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:28.550 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:36:28.555 INFO:teuthology.orchestra.run.smithi001.stdout:506 flatten 2024-09-14T22:36:28.569 INFO:teuthology.orchestra.run.smithi001.stdout:508 read 0x16afee8 thru 0x16b42c5 (0x43de bytes) 2024-09-14T22:36:28.597 INFO:teuthology.orchestra.run.smithi001.stdout:509 writesame 0xa232e56 thru 0xa233bc9 data_size 0x15(0xd74 bytes) 2024-09-14T22:36:28.597 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:28.597 INFO:teuthology.orchestra.run.smithi001.stdout:511 read 0x593aaa1 thru 0x593f2b8 (0x4818 bytes) 2024-09-14T22:36:28.597 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:28.597 INFO:teuthology.orchestra.run.smithi001.stdout:513 punch from 0x64ecd9 to 0x6574a9, (0x87d0 bytes) 2024-09-14T22:36:28.597 INFO:teuthology.orchestra.run.smithi001.stdout:514 punch from 0x3f44f4 to 0x3fdbda, (0x96e6 bytes) 2024-09-14T22:36:28.609 INFO:teuthology.orchestra.run.smithi001.stdout:516 write 0xaa7b644 thru 0xaa8b0ed (0xfaaa bytes) 2024-09-14T22:36:28.620 INFO:teuthology.orchestra.run.smithi001.stdout:518 write 0x4cd8188 thru 0x4ce6a45 (0xe8be bytes) 2024-09-14T22:36:28.632 INFO:teuthology.orchestra.run.smithi001.stdout:519 writesame 0x74e2d80 thru 0x74f0605 data_size 0x2e(0xd886 bytes) 2024-09-14T22:36:28.639 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:28.635+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:28.643 INFO:teuthology.orchestra.run.smithi001.stdout:520 write 0x8f24575 thru 0x8f306ee (0xc17a bytes) 2024-09-14T22:36:28.654 INFO:teuthology.orchestra.run.smithi001.stdout:521 read 0x7224ed2 thru 0x722544a (0x579 bytes) 2024-09-14T22:36:28.654 INFO:teuthology.orchestra.run.smithi001.stdout:522 punch from 0x7834a09 to 0x783cf7b, (0x8572 bytes) 2024-09-14T22:36:28.654 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:28.654 INFO:teuthology.orchestra.run.smithi001.stdout:527 read 0x2349c9a thru 0x2358b54 (0xeebb bytes) 2024-09-14T22:36:28.654 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:28.654 INFO:teuthology.orchestra.run.smithi001.stdout:533 writesame 0x4a4b81b thru 0x4a4f3b6 data_size 0x1c(0x3b9c bytes) 2024-09-14T22:36:28.664 INFO:teuthology.orchestra.run.smithi001.stdout:534 read 0x5aeed03 thru 0x5afc4c8 (0xd7c6 bytes) 2024-09-14T22:36:28.666 INFO:teuthology.orchestra.run.smithi001.stdout:535 trunc from 0xaa8b0ee to 0xb296769 2024-09-14T22:36:28.696 INFO:teuthology.orchestra.run.smithi001.stdout:536 trunc from 0xb296769 to 0x25c41eb 2024-09-14T22:36:28.715 INFO:teuthology.orchestra.run.smithi001.stdout:537 punch from 0x148e242 to 0x14986fd, (0xa4bb bytes) 2024-09-14T22:36:28.728 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:28.728 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:28.735 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:28.731+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:28.742 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:36:28.753 INFO:teuthology.orchestra.run.smithi001.stdout:538 writesame 0xc4e14e5 thru 0xc4efb78 data_size 0x6(0xe694 bytes) 2024-09-14T22:36:28.753 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:28.753 INFO:teuthology.orchestra.run.smithi001.stdout:540 read 0xa02c050 thru 0xa0322d1 (0x6282 bytes) 2024-09-14T22:36:28.753 INFO:teuthology.orchestra.run.smithi001.stdout:541 write 0xa60cc67 thru 0xa61a268 (0xd602 bytes) 2024-09-14T22:36:28.767 INFO:teuthology.orchestra.run.smithi001.stdout:542 read 0x1805775 thru 0x18065ec (0xe78 bytes) 2024-09-14T22:36:28.767 INFO:teuthology.orchestra.run.smithi001.stdout:543 read 0x4fd8fc1 thru 0x4fdf5dc (0x661c bytes) 2024-09-14T22:36:28.783 INFO:teuthology.orchestra.run.smithi001.stdout:544 trunc from 0xc4efb79 to 0x6afe64c 2024-09-14T22:36:28.793 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T22:36:28.805 INFO:teuthology.orchestra.run.smithi001.stdout:546 read 0x5fd124 thru 0x60cca0 (0xfb7d bytes) 2024-09-14T22:36:28.806 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:28.806 INFO:teuthology.orchestra.run.smithi001.stdout:548 read 0x1133318 thru 0x113d995 (0xa67e bytes) 2024-09-14T22:36:28.819 INFO:teuthology.orchestra.run.smithi001.stdout:552 write 0x829363d thru 0x82a1416 (0xddda bytes) 2024-09-14T22:36:28.827 INFO:teuthology.orchestra.run.smithi001.stdout:554 read 0x73fe590 thru 0x7402845 (0x42b6 bytes) 2024-09-14T22:36:28.827 INFO:teuthology.orchestra.run.smithi001.stdout:557 writesame 0x79a1a1a thru 0x79afb25 data_size 0x6(0xe10c bytes) 2024-09-14T22:36:28.836 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:28.831+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:28.838 INFO:teuthology.orchestra.run.smithi001.stdout:558 punch from 0x504f0d7 to 0x505f03b, (0xff64 bytes) 2024-09-14T22:36:28.851 INFO:teuthology.orchestra.run.smithi001.stdout:559 trunc from 0x82a1417 to 0x1679e7e 2024-09-14T22:36:28.873 INFO:teuthology.orchestra.run.smithi001.stdout:560 trunc from 0x1679e7e to 0x58d842e 2024-09-14T22:36:28.898 INFO:teuthology.orchestra.run.smithi001.stdout:561 write 0x703c2e2 thru 0x704b933 (0xf652 bytes) 2024-09-14T22:36:28.918 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:28.918 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:28.924 INFO:teuthology.orchestra.run.smithi001.stdout:562 trunc from 0x704b934 to 0x1b2bc4e 2024-09-14T22:36:28.933 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:36:28.943 INFO:teuthology.orchestra.run.smithi001.stdout:563 trunc from 0x1b2bc4e to 0x28617aa 2024-09-14T22:36:28.953 INFO:teuthology.orchestra.run.smithi001.stdout:564 read 0x118e4fb thru 0x1190af7 (0x25fd bytes) 2024-09-14T22:36:28.954 INFO:teuthology.orchestra.run.smithi001.stdout:565 read 0xd34ea3 thru 0xd3d837 (0x8995 bytes) 2024-09-14T22:36:28.973 INFO:teuthology.orchestra.run.smithi001.stdout:567 write 0x8fa83c1 thru 0x8fab7d4 (0x3414 bytes) 2024-09-14T22:36:28.979 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:28.975+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:28.984 INFO:teuthology.orchestra.run.smithi001.stdout:568 writesame 0x4eedb5c thru 0x4eefc56 data_size 0x1(0x20fb bytes) 2024-09-14T22:36:28.994 INFO:teuthology.orchestra.run.smithi001.stdout:569 punch from 0x4ed6ca7 to 0x4eda05b, (0x33b4 bytes) 2024-09-14T22:36:28.994 INFO:teuthology.orchestra.run.smithi001.stdout:570 punch from 0x797ad3e to 0x79812b4, (0x6576 bytes) 2024-09-14T22:36:28.994 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:28.994 INFO:teuthology.orchestra.run.smithi001.stdout:575 read 0x20295d5 thru 0x202b3c0 (0x1dec bytes) 2024-09-14T22:36:28.994 INFO:teuthology.orchestra.run.smithi001.stdout:576 read 0x62c6bdd thru 0x62cef46 (0x836a bytes) 2024-09-14T22:36:29.012 INFO:teuthology.orchestra.run.smithi001.stdout:577 write 0xdf53443 thru 0xdf629eb (0xf5a9 bytes) 2024-09-14T22:36:29.022 INFO:teuthology.orchestra.run.smithi001.stdout:578 read 0x5391317 thru 0x539e9e3 (0xd6cd bytes) 2024-09-14T22:36:29.022 INFO:teuthology.orchestra.run.smithi001.stdout:580 punch from 0x537e4fd to 0x5388bee, (0xa6f1 bytes) 2024-09-14T22:36:29.022 INFO:teuthology.orchestra.run.smithi001.stdout:582 write 0xa2b3f86 thru 0xa2ba024 (0x609f bytes) 2024-09-14T22:36:29.030 INFO:teuthology.orchestra.run.smithi001.stdout:583 punch from 0x200a828 to 0x2017de4, (0xd5bc bytes) 2024-09-14T22:36:29.030 INFO:teuthology.orchestra.run.smithi001.stdout:584 read 0x280023e thru 0x2806c78 (0x6a3b bytes) 2024-09-14T22:36:29.030 INFO:teuthology.orchestra.run.smithi001.stdout:585 write 0x4e2a71b thru 0x4e3739c (0xcc82 bytes) 2024-09-14T22:36:29.033 INFO:teuthology.orchestra.run.smithi001.stdout:589 write 0x1581a25 thru 0x158ec51 (0xd22d bytes) 2024-09-14T22:36:29.038 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:29.030+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:29.040 INFO:teuthology.orchestra.run.smithi001.stdout:590 punch from 0x33f606e to 0x33ff608, (0x959a bytes) 2024-09-14T22:36:29.040 INFO:teuthology.orchestra.run.smithi001.stdout:591 write 0x3fcc6c7 thru 0x3fd82ea (0xbc24 bytes) 2024-09-14T22:36:29.050 INFO:teuthology.orchestra.run.smithi001.stdout:592 punch from 0x1069186 to 0x107703d, (0xdeb7 bytes) 2024-09-14T22:36:29.050 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:29.050 INFO:teuthology.orchestra.run.smithi001.stdout:595 write 0x76f87c4 thru 0x76fc9ac (0x41e9 bytes) 2024-09-14T22:36:29.057 INFO:teuthology.orchestra.run.smithi001.stdout:596 write 0x7d09fa9 thru 0x7d127bc (0x8814 bytes) 2024-09-14T22:36:29.065 INFO:teuthology.orchestra.run.smithi001.stdout:597 write 0x919e8b6 thru 0x919e961 (0xac bytes) 2024-09-14T22:36:29.069 INFO:teuthology.orchestra.run.smithi001.stdout:598 write 0x858e54c thru 0x859115c (0x2c11 bytes) 2024-09-14T22:36:29.079 INFO:teuthology.orchestra.run.smithi001.stdout:599 writesame 0xbb9829 thru 0xbc71bc data_size 0x19(0xd994 bytes) 2024-09-14T22:36:29.099 INFO:teuthology.orchestra.run.smithi001.stdout:600 trunc from 0xdf629ec to 0x8b0e570 2024-09-14T22:36:29.113 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:29.114 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:29.116 INFO:teuthology.orchestra.run.smithi001.stdout:601 punch from 0x587423b to 0x5882ae7, (0xe8ac bytes) 2024-09-14T22:36:29.117 INFO:teuthology.orchestra.run.smithi001.stdout:602 trunc from 0x8b0e570 to 0x944637b 2024-09-14T22:36:29.129 INFO:teuthology.orchestra.run.smithi001.stdout:604 write 0x79ea26d thru 0x79ec1b6 (0x1f4a bytes) 2024-09-14T22:36:29.130 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:36:29.138 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:29.134+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:29.143 INFO:teuthology.orchestra.run.smithi001.stdout:605 trunc from 0x944637b to 0xed09eb9 2024-09-14T22:36:29.151 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:29.151 INFO:teuthology.orchestra.run.smithi001.stdout:607 punch from 0xd060202 to 0xd063b11, (0x390f bytes) 2024-09-14T22:36:29.151 INFO:teuthology.orchestra.run.smithi001.stdout:609 punch from 0xabe339a to 0xabed3dd, (0xa043 bytes) 2024-09-14T22:36:29.151 INFO:teuthology.orchestra.run.smithi001.stdout:610 punch from 0x598ac82 to 0x598fb6d, (0x4eeb bytes) 2024-09-14T22:36:29.170 INFO:teuthology.orchestra.run.smithi001.stdout:612 trunc from 0xed09eb9 to 0x1db44c2 2024-09-14T22:36:29.209 INFO:teuthology.orchestra.run.smithi001.stdout:614 writesame 0x84746be thru 0x847688a data_size 0x1(0x21cd bytes) 2024-09-14T22:36:29.218 INFO:teuthology.orchestra.run.smithi001.stdout:616 trunc from 0x847688b to 0xd2a7d05 2024-09-14T22:36:29.240 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:29.234+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:29.245 INFO:teuthology.orchestra.run.smithi001.stdout:617 trunc from 0xd2a7d05 to 0x301d5c5 2024-09-14T22:36:29.256 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:29.256 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:29.258 INFO:teuthology.orchestra.run.smithi001.stdout:618 write 0x1bf2d8d thru 0x1bf7aa1 (0x4d15 bytes) 2024-09-14T22:36:29.272 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:36:29.275 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:29.275 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:29.307 INFO:teuthology.orchestra.run.smithi001.stdout:623 write 0xdc72c3d thru 0xdc749a6 (0x1d6a bytes) 2024-09-14T22:36:29.316 INFO:teuthology.orchestra.run.smithi001.stdout:624 read 0x5896620 thru 0x5899a48 (0x3429 bytes) 2024-09-14T22:36:29.316 INFO:teuthology.orchestra.run.smithi001.stdout:626 writesame 0xaaa4421 thru 0xaaac346 data_size 0x5d(0x7f26 bytes) 2024-09-14T22:36:29.334 INFO:teuthology.orchestra.run.smithi001.stdout:628 trunc from 0xdc749a7 to 0x9063c83 2024-09-14T22:36:29.341 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:29.334+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:29.353 INFO:teuthology.orchestra.run.smithi001.stdout:630 writesame 0x3d20bce thru 0x3d2da7e data_size 0x1(0xceb1 bytes) 2024-09-14T22:36:29.382 INFO:teuthology.orchestra.run.smithi001.stdout:632 read 0x260c988 thru 0x2610b15 (0x418e bytes) 2024-09-14T22:36:29.383 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:29.401 INFO:teuthology.orchestra.run.smithi001.stdout:636 writesame 0xb6d9484 thru 0xb6d9b7f data_size 0x6(0x6fc bytes) 2024-09-14T22:36:29.401 INFO:teuthology.orchestra.run.smithi001.stdout:639 read 0x3b8aa5d thru 0x3b8e10b (0x36af bytes) 2024-09-14T22:36:29.401 INFO:teuthology.orchestra.run.smithi001.stdout:641 writesame 0x177deef thru 0x177e7d5 data_size 0x1(0x8e7 bytes) 2024-09-14T22:36:29.405 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:29.405 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:29.410 INFO:teuthology.orchestra.run.smithi001.stdout:642 read 0x4ae6788 thru 0x4aed4bb (0x6d34 bytes) 2024-09-14T22:36:29.410 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:29.410 INFO:teuthology.orchestra.run.smithi001.stdout:644 read 0xaff9276 thru 0xb007293 (0xe01e bytes) 2024-09-14T22:36:29.411 INFO:teuthology.orchestra.run.smithi001.stdout:645 read 0xaff4791 thru 0xaff9d09 (0x5579 bytes) 2024-09-14T22:36:29.411 INFO:teuthology.orchestra.run.smithi001.stdout:646 writesame 0x5891d72 thru 0x5896fc8 data_size 0x1(0x5257 bytes) 2024-09-14T22:36:29.421 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:36:29.431 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:29.431 INFO:teuthology.orchestra.run.smithi001.stdout:648 punch from 0x83fda80 to 0x840973b, (0xbcbb bytes) 2024-09-14T22:36:29.431 INFO:teuthology.orchestra.run.smithi001.stdout:649 write 0x397e59e thru 0x3987408 (0x8e6b bytes) 2024-09-14T22:36:29.443 INFO:teuthology.orchestra.run.smithi001.stdout:650 writesame 0x43c5056 thru 0x43d27eb data_size 0x5(0xd796 bytes) 2024-09-14T22:36:29.471 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:29.467+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:29.481 INFO:teuthology.orchestra.run.smithi001.stdout:651 write 0xdb38a27 thru 0xdb3cc41 (0x421b bytes) 2024-09-14T22:36:29.491 INFO:teuthology.orchestra.run.smithi001.stdout:655 punch from 0x68a4739 to 0x68b0e20, (0xc6e7 bytes) 2024-09-14T22:36:29.491 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:29.491 INFO:teuthology.orchestra.run.smithi001.stdout:657 read 0x1a33ed8 thru 0x1a3e926 (0xaa4f bytes) 2024-09-14T22:36:29.492 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:29.492 INFO:teuthology.orchestra.run.smithi001.stdout:660 read 0xc373c6c thru 0xc37b778 (0x7b0d bytes) 2024-09-14T22:36:29.492 INFO:teuthology.orchestra.run.smithi001.stdout:661 punch from 0x888f6e0 to 0x889ea81, (0xf3a1 bytes) 2024-09-14T22:36:29.517 INFO:teuthology.orchestra.run.smithi001.stdout:662 trunc from 0xdb3cc42 to 0x17b8abe 2024-09-14T22:36:29.545 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:29.539+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:29.547 INFO:teuthology.orchestra.run.smithi001.stdout:663 read 0x109b775 thru 0x10a9d9f (0xe62b bytes) 2024-09-14T22:36:29.547 INFO:teuthology.orchestra.run.smithi001.stdout:664 punch from 0xaefc5 to 0xb4949, (0x5984 bytes) 2024-09-14T22:36:29.547 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:29.548 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:29.548 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:29.557 INFO:teuthology.orchestra.run.smithi001.stdout:666 write 0x2de8971 thru 0x2df2934 (0x9fc4 bytes) 2024-09-14T22:36:29.562 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:36:29.584 INFO:teuthology.orchestra.run.smithi001.stdout:667 writesame 0x8fc92f4 thru 0x8fca63f data_size 0x1a(0x134c bytes) 2024-09-14T22:36:29.584 INFO:teuthology.orchestra.run.smithi001.stdout:668 punch from 0x3f6d868 to 0x3f6fa04, (0x219c bytes) 2024-09-14T22:36:29.590 INFO:teuthology.orchestra.run.smithi001.stdout:669 write 0x91f160f thru 0x91f242e (0xe20 bytes) 2024-09-14T22:36:29.596 INFO:teuthology.orchestra.run.smithi001.stdout:670 read 0x2e5562c thru 0x2e5bee1 (0x68b6 bytes) 2024-09-14T22:36:29.612 INFO:teuthology.orchestra.run.smithi001.stdout:672 write 0xbd10878 thru 0xbd18c5f (0x83e8 bytes) 2024-09-14T22:36:29.642 INFO:teuthology.orchestra.run.smithi001.stdout:673 trunc from 0xbd18c60 to 0x112b7d0 2024-09-14T22:36:29.646 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:29.642+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:29.699 INFO:teuthology.orchestra.run.smithi001.stdout:674 writesame 0xb941ac9 thru 0xb9430ad data_size 0x5(0x15e5 bytes) 2024-09-14T22:36:29.699 INFO:teuthology.orchestra.run.smithi001.stdout:675 read 0x2ca444d thru 0x2cad9df (0x9593 bytes) 2024-09-14T22:36:29.703 INFO:teuthology.orchestra.run.smithi001.stdout:676 trunc from 0xb9430ae to 0xd138ab4 2024-09-14T22:36:29.717 INFO:teuthology.orchestra.run.smithi001.stdout:678 read 0x5ef335d thru 0x5eff891 (0xc535 bytes) 2024-09-14T22:36:29.718 INFO:teuthology.orchestra.run.smithi001.stdout:679 punch from 0x9cede69 to 0x9cef8a8, (0x1a3f bytes) 2024-09-14T22:36:29.718 INFO:teuthology.orchestra.run.smithi001.stdout:680 read 0xa5dceee thru 0xa5e1843 (0x4956 bytes) 2024-09-14T22:36:29.718 INFO:teuthology.orchestra.run.smithi001.stdout:681 punch from 0xbd74aed to 0xbd7b9c7, (0x6eda bytes) 2024-09-14T22:36:29.718 INFO:teuthology.orchestra.run.smithi001.stdout:682 write 0x22400ce thru 0x22403dc (0x30f bytes) 2024-09-14T22:36:29.725 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:29.725 INFO:teuthology.orchestra.run.smithi001.stdout:684 punch from 0x489c580 to 0x48a053e, (0x3fbe bytes) 2024-09-14T22:36:29.730 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:29.730 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:29.733 INFO:teuthology.orchestra.run.smithi001.stdout:685 trunc from 0xd138ab4 to 0x9e92f44 2024-09-14T22:36:29.745 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:36:29.748 INFO:teuthology.orchestra.run.smithi001.stdout:686 writesame 0xbd851e thru 0xbe4083 data_size 0x22(0xbb66 bytes) 2024-09-14T22:36:29.748 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:29.742+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:29.760 INFO:teuthology.orchestra.run.smithi001.stdout:687 trunc from 0x9e92f44 to 0x7e11756 2024-09-14T22:36:29.772 INFO:teuthology.orchestra.run.smithi001.stdout:688 clone 3 order 23 su 524288 sc 12 2024-09-14T22:36:29.781 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2024-09-14T22:36:29.850 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:29.846+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:29.868 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:29.868 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:29.869 INFO:teuthology.orchestra.run.smithi001.stderr:Exporting 3.4 info 3.4( v 30'20 (0'0,30'20] local-lis/les=23/24 n=2 ec=23/23 lis/c=23/23 les/c/f=24/24/0 sis=23) 2024-09-14T22:36:29.875 INFO:teuthology.orchestra.run.smithi001.stderr:Read #3:2104c016:::rbd_data.113c3b74a67f.0000000000000004:head# 2024-09-14T22:36:29.882 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:36:29.883 INFO:teuthology.orchestra.run.smithi001.stderr:Read #3:25583457:::rbd_data.113cde4a8b6b.0000000000000074:head# 2024-09-14T22:36:29.883 INFO:teuthology.orchestra.run.smithi001.stderr:Export successful 2024-09-14T22:36:29.948 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:29.942+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:30.008 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:30.008 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:36:30.008 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:36:30.008 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:30.022 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:36:30.070 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:30.066+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:30.150 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:30.146+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:30.192 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:30.192 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:36:30.192 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:36:30.192 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:36:30.192 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:30.206 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:36:30.250 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:30.246+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:30.352 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:30.347+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:30.379 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:30.379 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:36:30.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:36:30.379 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:36:30.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.379 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2770 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14331904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:30.380 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.551455+0000", 2024-09-14T22:36:30.380 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.8253539410000004, 2024-09-14T22:36:30.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028737275, 2024-09-14T22:36:30.380 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:30.380 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:30.380 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:30.380 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:30.380 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:30.380 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:30.380 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2770 2024-09-14T22:36:30.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.380 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:30.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:30.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.551455+0000", 2024-09-14T22:36:30.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:30.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.551455+0000", 2024-09-14T22:36:30.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:30.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.551455+0000", 2024-09-14T22:36:30.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.39e-07 2024-09-14T22:36:30.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:30.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.551457+0000", 2024-09-14T22:36:30.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2000000000000001e-06 2024-09-14T22:36:30.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:30.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.551464+0000", 2024-09-14T22:36:30.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3180000000000003e-06 2024-09-14T22:36:30.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:30.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.551476+0000", 2024-09-14T22:36:30.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2345e-05 2024-09-14T22:36:30.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:30.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.579963+0000", 2024-09-14T22:36:30.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028487545999999999 2024-09-14T22:36:30.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:30.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.580075+0000", 2024-09-14T22:36:30.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111292 2024-09-14T22:36:30.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:30.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.580192+0000", 2024-09-14T22:36:30.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000117135 2024-09-14T22:36:30.384 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.384 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:30.384 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.384 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3021 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14292992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:30.384 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559557+0000", 2024-09-14T22:36:30.384 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.8172513349999999, 2024-09-14T22:36:30.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028720181000000001, 2024-09-14T22:36:30.384 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:30.384 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:30.385 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:30.385 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:30.385 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:30.385 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:30.385 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3021 2024-09-14T22:36:30.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.385 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:30.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.385 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:30.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559557+0000", 2024-09-14T22:36:30.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:30.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559557+0000", 2024-09-14T22:36:30.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:30.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559558+0000", 2024-09-14T22:36:30.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4499999999999996e-07 2024-09-14T22:36:30.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:30.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559562+0000", 2024-09-14T22:36:30.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.065e-06 2024-09-14T22:36:30.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:30.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559573+0000", 2024-09-14T22:36:30.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1398e-05 2024-09-14T22:36:30.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:30.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559582+0000", 2024-09-14T22:36:30.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6100000000000006e-06 2024-09-14T22:36:30.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:30.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588042+0000", 2024-09-14T22:36:30.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028460388999999999 2024-09-14T22:36:30.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:30.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588147+0000", 2024-09-14T22:36:30.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000104727 2024-09-14T22:36:30.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:30.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588277+0000", 2024-09-14T22:36:30.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000130147 2024-09-14T22:36:30.389 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.389 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:30.389 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.389 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3022 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14301184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:30.389 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559594+0000", 2024-09-14T22:36:30.389 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.8172149449999999, 2024-09-14T22:36:30.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028916481000000001, 2024-09-14T22:36:30.389 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:30.389 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:30.389 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:30.390 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:30.390 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:30.390 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:30.390 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3022 2024-09-14T22:36:30.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.390 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:30.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:30.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559594+0000", 2024-09-14T22:36:30.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:30.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559594+0000", 2024-09-14T22:36:30.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:30.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559594+0000", 2024-09-14T22:36:30.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7000000000000004e-07 2024-09-14T22:36:30.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:30.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559604+0000", 2024-09-14T22:36:30.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6020000000000006e-06 2024-09-14T22:36:30.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:30.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559615+0000", 2024-09-14T22:36:30.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1032999999999999e-05 2024-09-14T22:36:30.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:30.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559638+0000", 2024-09-14T22:36:30.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3255999999999999e-05 2024-09-14T22:36:30.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:30.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588323+0000", 2024-09-14T22:36:30.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.02868447 2024-09-14T22:36:30.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:30.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588443+0000", 2024-09-14T22:36:30.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120056 2024-09-14T22:36:30.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:30.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588510+0000", 2024-09-14T22:36:30.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7393999999999999e-05 2024-09-14T22:36:30.394 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.394 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:30.394 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.394 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3023 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14309376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:30.394 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559651+0000", 2024-09-14T22:36:30.394 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.8171574589999997, 2024-09-14T22:36:30.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029146231000000002, 2024-09-14T22:36:30.394 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:30.394 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:30.394 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:30.395 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:30.395 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:30.395 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:30.395 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3023 2024-09-14T22:36:30.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.395 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:30.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:30.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559651+0000", 2024-09-14T22:36:30.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:30.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559651+0000", 2024-09-14T22:36:30.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:30.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559652+0000", 2024-09-14T22:36:30.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2699999999999998e-07 2024-09-14T22:36:30.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:30.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559656+0000", 2024-09-14T22:36:30.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7019999999999999e-06 2024-09-14T22:36:30.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.397 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:30.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559666+0000", 2024-09-14T22:36:30.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0400999999999999e-05 2024-09-14T22:36:30.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.397 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:30.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559674+0000", 2024-09-14T22:36:30.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8890000000000002e-06 2024-09-14T22:36:30.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.397 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:30.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588565+0000", 2024-09-14T22:36:30.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028891217 2024-09-14T22:36:30.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:30.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588679+0000", 2024-09-14T22:36:30.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114058 2024-09-14T22:36:30.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:30.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588797+0000", 2024-09-14T22:36:30.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118137 2024-09-14T22:36:30.398 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.398 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:30.398 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.399 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3024 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14317568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:30.399 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559685+0000", 2024-09-14T22:36:30.399 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.817123348, 2024-09-14T22:36:30.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029326498999999999, 2024-09-14T22:36:30.399 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:30.399 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:30.399 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:30.399 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:30.399 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:30.399 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:30.399 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3024 2024-09-14T22:36:30.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.400 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:30.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:30.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559685+0000", 2024-09-14T22:36:30.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:30.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559685+0000", 2024-09-14T22:36:30.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:30.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559686+0000", 2024-09-14T22:36:30.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0999999999999998e-07 2024-09-14T22:36:30.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:30.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559697+0000", 2024-09-14T22:36:30.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.147e-05 2024-09-14T22:36:30.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:30.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559708+0000", 2024-09-14T22:36:30.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0546e-05 2024-09-14T22:36:30.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:30.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559716+0000", 2024-09-14T22:36:30.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.704e-06 2024-09-14T22:36:30.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:30.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588833+0000", 2024-09-14T22:36:30.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029116442999999999 2024-09-14T22:36:30.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:30.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588920+0000", 2024-09-14T22:36:30.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6916999999999995e-05 2024-09-14T22:36:30.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:30.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589012+0000", 2024-09-14T22:36:30.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1809000000000001e-05 2024-09-14T22:36:30.403 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.404 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:30.404 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.404 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3025 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14325760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:30.404 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559729+0000", 2024-09-14T22:36:30.404 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.8170794489999995, 2024-09-14T22:36:30.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029422501, 2024-09-14T22:36:30.404 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:30.404 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:30.404 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:30.404 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:30.404 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:30.404 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:30.405 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3025 2024-09-14T22:36:30.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.405 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:30.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:30.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559729+0000", 2024-09-14T22:36:30.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:30.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559729+0000", 2024-09-14T22:36:30.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:30.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559730+0000", 2024-09-14T22:36:30.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8100000000000002e-07 2024-09-14T22:36:30.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:30.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559733+0000", 2024-09-14T22:36:30.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4740000000000001e-06 2024-09-14T22:36:30.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:30.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559744+0000", 2024-09-14T22:36:30.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0489000000000001e-05 2024-09-14T22:36:30.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:30.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559760+0000", 2024-09-14T22:36:30.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5817000000000001e-05 2024-09-14T22:36:30.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:30.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589050+0000", 2024-09-14T22:36:30.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.02929056 2024-09-14T22:36:30.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:30.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589101+0000", 2024-09-14T22:36:30.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0992000000000002e-05 2024-09-14T22:36:30.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:30.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589152+0000", 2024-09-14T22:36:30.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0387999999999999e-05 2024-09-14T22:36:30.408 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.408 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:30.408 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.409 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3026 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14333952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:30.409 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559795+0000", 2024-09-14T22:36:30.409 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.8170136249999995, 2024-09-14T22:36:30.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029483812000000002, 2024-09-14T22:36:30.409 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:30.409 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:30.409 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:30.409 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:30.409 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:30.409 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:30.409 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3026 2024-09-14T22:36:30.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.409 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:30.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:30.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559795+0000", 2024-09-14T22:36:30.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:30.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559795+0000", 2024-09-14T22:36:30.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:30.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559796+0000", 2024-09-14T22:36:30.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3200000000000003e-07 2024-09-14T22:36:30.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:30.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559800+0000", 2024-09-14T22:36:30.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2930000000000002e-06 2024-09-14T22:36:30.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:30.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559812+0000", 2024-09-14T22:36:30.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2053e-05 2024-09-14T22:36:30.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:30.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559821+0000", 2024-09-14T22:36:30.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8869999999999998e-06 2024-09-14T22:36:30.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:30.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589177+0000", 2024-09-14T22:36:30.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029356308000000001 2024-09-14T22:36:30.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:30.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589229+0000", 2024-09-14T22:36:30.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1326000000000003e-05 2024-09-14T22:36:30.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:30.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589279+0000", 2024-09-14T22:36:30.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0012999999999997e-05 2024-09-14T22:36:30.413 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.413 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:30.413 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.413 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3027 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14342144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:30.413 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559833+0000", 2024-09-14T22:36:30.413 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.8169758969999998, 2024-09-14T22:36:30.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029619187000000002, 2024-09-14T22:36:30.413 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:30.413 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:30.414 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:30.414 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:30.414 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:30.414 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:30.414 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3027 2024-09-14T22:36:30.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.414 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:30.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:30.414 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559833+0000", 2024-09-14T22:36:30.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:30.414 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559833+0000", 2024-09-14T22:36:30.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:30.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559833+0000", 2024-09-14T22:36:30.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9400000000000005e-07 2024-09-14T22:36:30.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:30.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559845+0000", 2024-09-14T22:36:30.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1623e-05 2024-09-14T22:36:30.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:30.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559856+0000", 2024-09-14T22:36:30.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0832999999999999e-05 2024-09-14T22:36:30.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:30.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559864+0000", 2024-09-14T22:36:30.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3119999999999994e-06 2024-09-14T22:36:30.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:30.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589299+0000", 2024-09-14T22:36:30.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029434699000000002 2024-09-14T22:36:30.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:30.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589354+0000", 2024-09-14T22:36:30.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5467999999999998e-05 2024-09-14T22:36:30.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:30.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589452+0000", 2024-09-14T22:36:30.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7558000000000003e-05 2024-09-14T22:36:30.417 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.417 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:30.417 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.418 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3028 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14350336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:30.418 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559875+0000", 2024-09-14T22:36:30.418 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.8169330089999995, 2024-09-14T22:36:30.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029775586999999999, 2024-09-14T22:36:30.418 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:30.418 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:30.418 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:30.418 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:30.418 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:30.418 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:30.418 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3028 2024-09-14T22:36:30.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.418 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:30.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:30.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559875+0000", 2024-09-14T22:36:30.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:30.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559875+0000", 2024-09-14T22:36:30.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:30.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559876+0000", 2024-09-14T22:36:30.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5199999999999996e-07 2024-09-14T22:36:30.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:30.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559879+0000", 2024-09-14T22:36:30.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.095e-06 2024-09-14T22:36:30.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:30.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559889+0000", 2024-09-14T22:36:30.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0019e-05 2024-09-14T22:36:30.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:30.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559897+0000", 2024-09-14T22:36:30.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9189999999999999e-06 2024-09-14T22:36:30.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:30.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589486+0000", 2024-09-14T22:36:30.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029588482999999999 2024-09-14T22:36:30.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:30.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589578+0000", 2024-09-14T22:36:30.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2329000000000003e-05 2024-09-14T22:36:30.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:30.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589651+0000", 2024-09-14T22:36:30.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2990000000000004e-05 2024-09-14T22:36:30.422 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.422 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:30.422 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.422 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4067 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13244416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:30.422 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680076+0000", 2024-09-14T22:36:30.422 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.6967327210000001, 2024-09-14T22:36:30.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028684398, 2024-09-14T22:36:30.423 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:30.423 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:30.423 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:30.423 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:30.423 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:30.423 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:30.423 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4067 2024-09-14T22:36:30.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.423 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:30.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:30.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680076+0000", 2024-09-14T22:36:30.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:30.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680076+0000", 2024-09-14T22:36:30.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:30.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680076+0000", 2024-09-14T22:36:30.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9000000000000002e-07 2024-09-14T22:36:30.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:30.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680079+0000", 2024-09-14T22:36:30.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.616e-06 2024-09-14T22:36:30.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:30.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680087+0000", 2024-09-14T22:36:30.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7059999999999993e-06 2024-09-14T22:36:30.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:30.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680093+0000", 2024-09-14T22:36:30.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.626e-06 2024-09-14T22:36:30.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:30.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.708640+0000", 2024-09-14T22:36:30.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028546892000000001 2024-09-14T22:36:30.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:30.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.708701+0000", 2024-09-14T22:36:30.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0582999999999999e-05 2024-09-14T22:36:30.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:30.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.708760+0000", 2024-09-14T22:36:30.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9585e-05 2024-09-14T22:36:30.426 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.427 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:30.427 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.427 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4068 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13252608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:30.427 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680102+0000", 2024-09-14T22:36:30.427 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.6967061900000004, 2024-09-14T22:36:30.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028907656, 2024-09-14T22:36:30.427 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:30.427 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:30.427 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:30.427 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:30.428 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:30.428 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:30.428 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4068 2024-09-14T22:36:30.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.428 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:30.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:30.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680102+0000", 2024-09-14T22:36:30.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:30.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680102+0000", 2024-09-14T22:36:30.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:30.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680103+0000", 2024-09-14T22:36:30.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6600000000000002e-07 2024-09-14T22:36:30.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:30.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680105+0000", 2024-09-14T22:36:30.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.616e-06 2024-09-14T22:36:30.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:30.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680114+0000", 2024-09-14T22:36:30.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4120000000000001e-06 2024-09-14T22:36:30.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:30.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680119+0000", 2024-09-14T22:36:30.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6210000000000001e-06 2024-09-14T22:36:30.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:30.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.708813+0000", 2024-09-14T22:36:30.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028693330999999999 2024-09-14T22:36:30.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:30.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.708903+0000", 2024-09-14T22:36:30.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0297000000000006e-05 2024-09-14T22:36:30.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:30.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709010+0000", 2024-09-14T22:36:30.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106913 2024-09-14T22:36:30.431 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.431 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:30.431 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.432 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4069 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13260800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:30.432 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680128+0000", 2024-09-14T22:36:30.432 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.696680915, 2024-09-14T22:36:30.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029145476, 2024-09-14T22:36:30.432 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:30.432 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:30.432 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:30.432 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:30.432 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:30.432 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:30.432 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4069 2024-09-14T22:36:30.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.433 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:30.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:30.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680128+0000", 2024-09-14T22:36:30.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:30.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680128+0000", 2024-09-14T22:36:30.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:30.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680128+0000", 2024-09-14T22:36:30.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7599999999999998e-07 2024-09-14T22:36:30.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:30.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680131+0000", 2024-09-14T22:36:30.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9050000000000001e-06 2024-09-14T22:36:30.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:30.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680138+0000", 2024-09-14T22:36:30.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2559999999999996e-06 2024-09-14T22:36:30.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:30.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680144+0000", 2024-09-14T22:36:30.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4779999999999999e-06 2024-09-14T22:36:30.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:30.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709054+0000", 2024-09-14T22:36:30.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028910207 2024-09-14T22:36:30.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:30.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709180+0000", 2024-09-14T22:36:30.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012608699999999999 2024-09-14T22:36:30.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:30.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709273+0000", 2024-09-14T22:36:30.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3166999999999998e-05 2024-09-14T22:36:30.436 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.436 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:30.436 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.436 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4070 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13268992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:30.437 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680151+0000", 2024-09-14T22:36:30.437 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.6966576330000001, 2024-09-14T22:36:30.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029327986, 2024-09-14T22:36:30.437 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:30.437 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:30.437 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:30.437 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:30.437 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:30.437 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:30.437 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4070 2024-09-14T22:36:30.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.437 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:30.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:30.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680151+0000", 2024-09-14T22:36:30.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:30.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680151+0000", 2024-09-14T22:36:30.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:30.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680151+0000", 2024-09-14T22:36:30.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:36:30.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:30.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680154+0000", 2024-09-14T22:36:30.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3350000000000001e-06 2024-09-14T22:36:30.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:30.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680162+0000", 2024-09-14T22:36:30.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.473e-06 2024-09-14T22:36:30.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:30.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680177+0000", 2024-09-14T22:36:30.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.465e-05 2024-09-14T22:36:30.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:30.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709356+0000", 2024-09-14T22:36:30.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029178744 2024-09-14T22:36:30.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:30.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709428+0000", 2024-09-14T22:36:30.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2428000000000005e-05 2024-09-14T22:36:30.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:30.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709479+0000", 2024-09-14T22:36:30.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0915999999999998e-05 2024-09-14T22:36:30.440 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.440 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:30.441 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.441 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4071 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13277184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:30.441 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680185+0000", 2024-09-14T22:36:30.441 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.6966231330000001, 2024-09-14T22:36:30.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029452392000000001, 2024-09-14T22:36:30.441 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:30.441 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:30.441 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:30.441 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:30.441 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:30.442 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:30.442 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4071 2024-09-14T22:36:30.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.442 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:30.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:30.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680185+0000", 2024-09-14T22:36:30.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:30.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680185+0000", 2024-09-14T22:36:30.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:30.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680186+0000", 2024-09-14T22:36:30.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6100000000000001e-07 2024-09-14T22:36:30.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:30.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680188+0000", 2024-09-14T22:36:30.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.509e-06 2024-09-14T22:36:30.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:30.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680197+0000", 2024-09-14T22:36:30.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8550000000000008e-06 2024-09-14T22:36:30.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:30.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680203+0000", 2024-09-14T22:36:30.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5849999999999999e-06 2024-09-14T22:36:30.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:30.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709514+0000", 2024-09-14T22:36:30.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029311054 2024-09-14T22:36:30.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:30.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709585+0000", 2024-09-14T22:36:30.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0875000000000002e-05 2024-09-14T22:36:30.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:30.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709638+0000", 2024-09-14T22:36:30.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3053000000000002e-05 2024-09-14T22:36:30.445 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.445 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:30.445 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.445 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4072 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13285376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:30.445 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680211+0000", 2024-09-14T22:36:30.445 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.696597852, 2024-09-14T22:36:30.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029591177999999999, 2024-09-14T22:36:30.446 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:30.446 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:30.446 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:30.446 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:30.446 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:30.446 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:30.446 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4072 2024-09-14T22:36:30.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.446 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:30.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:30.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680211+0000", 2024-09-14T22:36:30.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680211+0000", 2024-09-14T22:36:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680211+0000", 2024-09-14T22:36:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6699999999999999e-07 2024-09-14T22:36:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680214+0000", 2024-09-14T22:36:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0149999999999999e-06 2024-09-14T22:36:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:30.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680222+0000", 2024-09-14T22:36:30.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.371e-06 2024-09-14T22:36:30.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:30.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680228+0000", 2024-09-14T22:36:30.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6230000000000002e-06 2024-09-14T22:36:30.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:30.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709670+0000", 2024-09-14T22:36:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029441694000000001 2024-09-14T22:36:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709741+0000", 2024-09-14T22:36:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1668999999999994e-05 2024-09-14T22:36:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709802+0000", 2024-09-14T22:36:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0338999999999999e-05 2024-09-14T22:36:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.450 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4073 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13293568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:30.450 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680236+0000", 2024-09-14T22:36:30.450 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.6965724929999997, 2024-09-14T22:36:30.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029755238999999999, 2024-09-14T22:36:30.450 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:30.450 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:30.450 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:30.450 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:30.450 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:30.450 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:30.450 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4073 2024-09-14T22:36:30.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680236+0000", 2024-09-14T22:36:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680236+0000", 2024-09-14T22:36:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680236+0000", 2024-09-14T22:36:30.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5499999999999998e-07 2024-09-14T22:36:30.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:30.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680239+0000", 2024-09-14T22:36:30.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4380000000000002e-06 2024-09-14T22:36:30.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:30.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680248+0000", 2024-09-14T22:36:30.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6319999999999997e-06 2024-09-14T22:36:30.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:30.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680261+0000", 2024-09-14T22:36:30.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3193000000000001e-05 2024-09-14T22:36:30.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:30.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709831+0000", 2024-09-14T22:36:30.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029570486 2024-09-14T22:36:30.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:30.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709883+0000", 2024-09-14T22:36:30.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2290999999999998e-05 2024-09-14T22:36:30.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:30.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709991+0000", 2024-09-14T22:36:30.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107744 2024-09-14T22:36:30.454 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.454 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:30.454 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4074 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13301760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680269+0000", 2024-09-14T22:36:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.696539595, 2024-09-14T22:36:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030007875999999999, 2024-09-14T22:36:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4074 2024-09-14T22:36:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:30.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:30.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680269+0000", 2024-09-14T22:36:30.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:30.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680269+0000", 2024-09-14T22:36:30.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:30.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680269+0000", 2024-09-14T22:36:30.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7300000000000001e-07 2024-09-14T22:36:30.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:30.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680277+0000", 2024-09-14T22:36:30.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8480000000000001e-06 2024-09-14T22:36:30.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:30.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680286+0000", 2024-09-14T22:36:30.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0529999999999996e-06 2024-09-14T22:36:30.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:30.458 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680292+0000", 2024-09-14T22:36:30.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.728e-06 2024-09-14T22:36:30.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:30.458 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.710024+0000", 2024-09-14T22:36:30.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029732176999999999 2024-09-14T22:36:30.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:30.459 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.710109+0000", 2024-09-14T22:36:30.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4715000000000005e-05 2024-09-14T22:36:30.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.459 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:30.459 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.710277+0000", 2024-09-14T22:36:30.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000167882 2024-09-14T22:36:30.459 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.459 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:30.459 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.460 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:12663 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 374784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T22:36:30.460 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:28.325542+0000", 2024-09-14T22:36:30.460 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 2.051266585, 2024-09-14T22:36:30.460 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028843810000000001, 2024-09-14T22:36:30.460 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:30.460 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:30.460 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:30.460 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:30.460 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:30.460 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:30.460 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12663 2024-09-14T22:36:30.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.460 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:30.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:30.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325542+0000", 2024-09-14T22:36:30.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:30.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325542+0000", 2024-09-14T22:36:30.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:30.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325542+0000", 2024-09-14T22:36:30.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.44e-07 2024-09-14T22:36:30.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:30.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325545+0000", 2024-09-14T22:36:30.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.796e-06 2024-09-14T22:36:30.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:30.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325553+0000", 2024-09-14T22:36:30.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8569999999999995e-06 2024-09-14T22:36:30.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.463 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:30.463 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325560+0000", 2024-09-14T22:36:30.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5379999999999999e-06 2024-09-14T22:36:30.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.463 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:30.463 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354196+0000", 2024-09-14T22:36:30.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028635864 2024-09-14T22:36:30.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.463 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:30.463 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354280+0000", 2024-09-14T22:36:30.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4324000000000001e-05 2024-09-14T22:36:30.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.464 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:30.464 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354386+0000", 2024-09-14T22:36:30.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105887 2024-09-14T22:36:30.464 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.464 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:30.464 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.464 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:12664 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T22:36:30.464 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:28.325596+0000", 2024-09-14T22:36:30.465 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 2.0512120810000001, 2024-09-14T22:36:30.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028964581, 2024-09-14T22:36:30.465 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:30.465 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:30.465 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:30.465 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:30.465 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:30.465 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:30.465 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12664 2024-09-14T22:36:30.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.465 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:30.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:30.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325596+0000", 2024-09-14T22:36:30.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:30.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325596+0000", 2024-09-14T22:36:30.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:30.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325597+0000", 2024-09-14T22:36:30.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4300000000000002e-07 2024-09-14T22:36:30.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:30.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325600+0000", 2024-09-14T22:36:30.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2219999999999998e-06 2024-09-14T22:36:30.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:30.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325610+0000", 2024-09-14T22:36:30.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2650000000000002e-06 2024-09-14T22:36:30.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:30.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325617+0000", 2024-09-14T22:36:30.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1199999999999996e-06 2024-09-14T22:36:30.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:30.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354438+0000", 2024-09-14T22:36:30.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028821205999999999 2024-09-14T22:36:30.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:30.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354495+0000", 2024-09-14T22:36:30.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6752999999999997e-05 2024-09-14T22:36:30.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:30.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354561+0000", 2024-09-14T22:36:30.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6272000000000005e-05 2024-09-14T22:36:30.468 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.469 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:30.469 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.469 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:12665 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 391168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T22:36:30.469 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:28.325626+0000", 2024-09-14T22:36:30.469 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 2.051182759, 2024-09-14T22:36:30.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029080366, 2024-09-14T22:36:30.469 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:30.469 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:30.469 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:30.469 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:30.469 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:30.469 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:30.470 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12665 2024-09-14T22:36:30.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.470 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:30.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:30.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325626+0000", 2024-09-14T22:36:30.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:30.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325626+0000", 2024-09-14T22:36:30.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:30.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325626+0000", 2024-09-14T22:36:30.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.06e-07 2024-09-14T22:36:30.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:30.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325629+0000", 2024-09-14T22:36:30.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7149999999999998e-06 2024-09-14T22:36:30.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:30.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325637+0000", 2024-09-14T22:36:30.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7360000000000001e-06 2024-09-14T22:36:30.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:30.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325643+0000", 2024-09-14T22:36:30.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2480000000000001e-06 2024-09-14T22:36:30.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:30.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354589+0000", 2024-09-14T22:36:30.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028945538 2024-09-14T22:36:30.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:30.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354635+0000", 2024-09-14T22:36:30.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6374e-05 2024-09-14T22:36:30.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:30.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354706+0000", 2024-09-14T22:36:30.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1049000000000004e-05 2024-09-14T22:36:30.473 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.473 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:30.473 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.473 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.473 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:30.473 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:30.475 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:36:30.476 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal --force --op remove --pgid 3.4 2024-09-14T22:36:30.476 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:30.471+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:30.576 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:30.571+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:30.648 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:30.648 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:36:30.648 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:36:30.648 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:30.662 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:36:30.706 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:30.702+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:30.778 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:30.771+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:30.797 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:30.797 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:36:30.797 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:36:30.797 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:36:30.797 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:30.812 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:36:30.879 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:30.875+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:30.972 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-14T22:36:30.980 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:30.974+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:30.986 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:30.986 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:36:30.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:36:30.986 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:36:30.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.986 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4149 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13129728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:30.986 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703281+0000", 2024-09-14T22:36:30.987 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 7.2790424949999997, 2024-09-14T22:36:30.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045957114, 2024-09-14T22:36:30.987 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:30.987 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:30.987 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:30.987 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:30.987 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:30.987 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:30.987 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4149 2024-09-14T22:36:30.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.987 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:30.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:30.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703281+0000", 2024-09-14T22:36:30.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:30.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703281+0000", 2024-09-14T22:36:30.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:30.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703282+0000", 2024-09-14T22:36:30.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4199999999999996e-07 2024-09-14T22:36:30.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:30.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703289+0000", 2024-09-14T22:36:30.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0770000000000002e-06 2024-09-14T22:36:30.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:30.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703296+0000", 2024-09-14T22:36:30.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8129999999999998e-06 2024-09-14T22:36:30.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:30.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703307+0000", 2024-09-14T22:36:30.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1663e-05 2024-09-14T22:36:30.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:30.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749073+0000", 2024-09-14T22:36:30.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.04576537 2024-09-14T22:36:30.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:30.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749156+0000", 2024-09-14T22:36:30.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3399999999999994e-05 2024-09-14T22:36:30.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:30.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749239+0000", 2024-09-14T22:36:30.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2249000000000005e-05 2024-09-14T22:36:30.990 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.990 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:30.990 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.991 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4150 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13137920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:30.991 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703318+0000", 2024-09-14T22:36:30.991 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 7.2790063009999999, 2024-09-14T22:36:30.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046170585, 2024-09-14T22:36:30.991 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:30.991 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:30.991 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:30.991 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:30.991 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:30.991 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:30.991 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4150 2024-09-14T22:36:30.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.991 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:30.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:30.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703318+0000", 2024-09-14T22:36:30.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:30.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703318+0000", 2024-09-14T22:36:30.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:30.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703318+0000", 2024-09-14T22:36:30.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8800000000000003e-07 2024-09-14T22:36:30.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:30.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703321+0000", 2024-09-14T22:36:30.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7860000000000001e-06 2024-09-14T22:36:30.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:30.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703328+0000", 2024-09-14T22:36:30.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2810000000000003e-06 2024-09-14T22:36:30.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:30.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703340+0000", 2024-09-14T22:36:30.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1745e-05 2024-09-14T22:36:30.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:30.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749286+0000", 2024-09-14T22:36:30.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045946222000000002 2024-09-14T22:36:30.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:30.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749390+0000", 2024-09-14T22:36:30.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000103614 2024-09-14T22:36:30.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:30.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749488+0000", 2024-09-14T22:36:30.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8449000000000006e-05 2024-09-14T22:36:30.995 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.995 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:30.995 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.995 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4151 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13146112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:30.995 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703348+0000", 2024-09-14T22:36:30.995 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 7.278976138, 2024-09-14T22:36:30.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046349509999999997, 2024-09-14T22:36:30.995 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:30.995 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:30.996 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:30.996 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:30.996 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:30.996 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:30.996 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4151 2024-09-14T22:36:30.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.996 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:30.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:30.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703348+0000", 2024-09-14T22:36:30.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:30.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703348+0000", 2024-09-14T22:36:30.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:30.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:30.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703348+0000", 2024-09-14T22:36:30.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6899999999999998e-07 2024-09-14T22:36:30.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:30.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703351+0000", 2024-09-14T22:36:30.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5950000000000001e-06 2024-09-14T22:36:30.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:30.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703357+0000", 2024-09-14T22:36:30.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4150000000000004e-06 2024-09-14T22:36:30.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:30.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703364+0000", 2024-09-14T22:36:30.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4069999999999998e-06 2024-09-14T22:36:30.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:30.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749530+0000", 2024-09-14T22:36:30.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046166281000000003 2024-09-14T22:36:30.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:30.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749615+0000", 2024-09-14T22:36:30.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5409999999999997e-05 2024-09-14T22:36:30.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:30.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:30.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:30.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749697+0000", 2024-09-14T22:36:30.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1933000000000003e-05 2024-09-14T22:36:30.999 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:30.999 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:30.999 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.000 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4152 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13154304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:31.000 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703371+0000", 2024-09-14T22:36:31.000 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 7.2789527359999999, 2024-09-14T22:36:31.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046552233999999998, 2024-09-14T22:36:31.000 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.000 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.000 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.000 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.000 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.000 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.000 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4152 2024-09-14T22:36:31.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.001 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703371+0000", 2024-09-14T22:36:31.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703371+0000", 2024-09-14T22:36:31.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.002 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703371+0000", 2024-09-14T22:36:31.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.34e-07 2024-09-14T22:36:31.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.002 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.002 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703374+0000", 2024-09-14T22:36:31.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.227e-06 2024-09-14T22:36:31.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.002 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.002 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703380+0000", 2024-09-14T22:36:31.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4699999999999999e-06 2024-09-14T22:36:31.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703385+0000", 2024-09-14T22:36:31.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.259e-06 2024-09-14T22:36:31.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749742+0000", 2024-09-14T22:36:31.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046357031 2024-09-14T22:36:31.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749825+0000", 2024-09-14T22:36:31.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2674000000000002e-05 2024-09-14T22:36:31.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749923+0000", 2024-09-14T22:36:31.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8239000000000006e-05 2024-09-14T22:36:31.004 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.004 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.004 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.004 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4153 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13162496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:31.005 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703392+0000", 2024-09-14T22:36:31.005 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 7.27893144, 2024-09-14T22:36:31.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046675756999999998, 2024-09-14T22:36:31.005 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.005 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.005 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.005 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.005 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.005 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.005 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4153 2024-09-14T22:36:31.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.005 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703392+0000", 2024-09-14T22:36:31.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703392+0000", 2024-09-14T22:36:31.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.705934+0000", 2024-09-14T22:36:31.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025418540000000001 2024-09-14T22:36:31.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.705940+0000", 2024-09-14T22:36:31.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0950000000000001e-06 2024-09-14T22:36:31.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.705952+0000", 2024-09-14T22:36:31.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1973e-05 2024-09-14T22:36:31.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.705958+0000", 2024-09-14T22:36:31.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.541e-06 2024-09-14T22:36:31.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749969+0000", 2024-09-14T22:36:31.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044011300000000003 2024-09-14T22:36:31.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.750020+0000", 2024-09-14T22:36:31.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1054000000000003e-05 2024-09-14T22:36:31.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.750068+0000", 2024-09-14T22:36:31.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7939999999999998e-05 2024-09-14T22:36:31.009 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.009 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.009 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.009 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4164 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13252608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:31.009 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706129+0000", 2024-09-14T22:36:31.009 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 7.276194963, 2024-09-14T22:36:31.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045833060000000002, 2024-09-14T22:36:31.010 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.010 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.010 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.010 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.010 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.010 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.010 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4164 2024-09-14T22:36:31.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.010 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706129+0000", 2024-09-14T22:36:31.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706129+0000", 2024-09-14T22:36:31.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706129+0000", 2024-09-14T22:36:31.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0699999999999998e-07 2024-09-14T22:36:31.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706132+0000", 2024-09-14T22:36:31.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3420000000000002e-06 2024-09-14T22:36:31.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706139+0000", 2024-09-14T22:36:31.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8619999999999997e-06 2024-09-14T22:36:31.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706143+0000", 2024-09-14T22:36:31.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8060000000000002e-06 2024-09-14T22:36:31.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.751792+0000", 2024-09-14T22:36:31.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045648954999999998 2024-09-14T22:36:31.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.751878+0000", 2024-09-14T22:36:31.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5316999999999996e-05 2024-09-14T22:36:31.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.751962+0000", 2024-09-14T22:36:31.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4370999999999996e-05 2024-09-14T22:36:31.014 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.014 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.014 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.014 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4165 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13260800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:31.014 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706178+0000", 2024-09-14T22:36:31.014 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 7.2761455279999998, 2024-09-14T22:36:31.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045915594999999997, 2024-09-14T22:36:31.014 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.014 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.014 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.014 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.015 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.015 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.015 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4165 2024-09-14T22:36:31.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.015 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706178+0000", 2024-09-14T22:36:31.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706178+0000", 2024-09-14T22:36:31.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706179+0000", 2024-09-14T22:36:31.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4999999999999998e-07 2024-09-14T22:36:31.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706181+0000", 2024-09-14T22:36:31.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4679999999999999e-06 2024-09-14T22:36:31.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706189+0000", 2024-09-14T22:36:31.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.994e-06 2024-09-14T22:36:31.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706195+0000", 2024-09-14T22:36:31.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6119999999999998e-06 2024-09-14T22:36:31.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752003+0000", 2024-09-14T22:36:31.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045807618000000001 2024-09-14T22:36:31.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752050+0000", 2024-09-14T22:36:31.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7301e-05 2024-09-14T22:36:31.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752094+0000", 2024-09-14T22:36:31.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4152e-05 2024-09-14T22:36:31.018 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.018 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.018 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.018 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4166 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13268992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:31.018 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706205+0000", 2024-09-14T22:36:31.018 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 7.2761193390000001, 2024-09-14T22:36:31.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046092665999999997, 2024-09-14T22:36:31.019 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.019 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.019 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.019 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.019 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.019 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.019 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4166 2024-09-14T22:36:31.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.019 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.019 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706205+0000", 2024-09-14T22:36:31.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.020 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.020 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706205+0000", 2024-09-14T22:36:31.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.020 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.020 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706205+0000", 2024-09-14T22:36:31.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:36:31.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.020 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.020 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706208+0000", 2024-09-14T22:36:31.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8049999999999998e-06 2024-09-14T22:36:31.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706215+0000", 2024-09-14T22:36:31.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2010000000000002e-06 2024-09-14T22:36:31.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706223+0000", 2024-09-14T22:36:31.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7260000000000002e-06 2024-09-14T22:36:31.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752129+0000", 2024-09-14T22:36:31.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045906532 2024-09-14T22:36:31.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752213+0000", 2024-09-14T22:36:31.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4060000000000005e-05 2024-09-14T22:36:31.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752297+0000", 2024-09-14T22:36:31.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3807999999999994e-05 2024-09-14T22:36:31.023 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.023 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.023 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.023 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4167 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13277184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:31.023 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706233+0000", 2024-09-14T22:36:31.023 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 7.2760907279999998, 2024-09-14T22:36:31.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046190500000000002, 2024-09-14T22:36:31.023 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.023 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.023 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.023 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.023 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.024 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.024 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4167 2024-09-14T22:36:31.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.024 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.024 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.024 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706233+0000", 2024-09-14T22:36:31.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.024 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.024 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706233+0000", 2024-09-14T22:36:31.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.025 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706234+0000", 2024-09-14T22:36:31.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.75e-07 2024-09-14T22:36:31.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.025 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706237+0000", 2024-09-14T22:36:31.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9890000000000001e-06 2024-09-14T22:36:31.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.025 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706244+0000", 2024-09-14T22:36:31.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6240000000000002e-06 2024-09-14T22:36:31.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706255+0000", 2024-09-14T22:36:31.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0328e-05 2024-09-14T22:36:31.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752338+0000", 2024-09-14T22:36:31.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046083172999999998 2024-09-14T22:36:31.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752381+0000", 2024-09-14T22:36:31.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2973999999999999e-05 2024-09-14T22:36:31.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752424+0000", 2024-09-14T22:36:31.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2936999999999999e-05 2024-09-14T22:36:31.027 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.027 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.027 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.027 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4168 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13285376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:31.027 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706261+0000", 2024-09-14T22:36:31.027 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 7.2760633840000004, 2024-09-14T22:36:31.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046378635000000001, 2024-09-14T22:36:31.028 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.028 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.028 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.028 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.028 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.028 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.028 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4168 2024-09-14T22:36:31.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.028 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.028 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706261+0000", 2024-09-14T22:36:31.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706261+0000", 2024-09-14T22:36:31.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706261+0000", 2024-09-14T22:36:31.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2899999999999999e-07 2024-09-14T22:36:31.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706264+0000", 2024-09-14T22:36:31.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.774e-06 2024-09-14T22:36:31.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706267+0000", 2024-09-14T22:36:31.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7179999999999998e-06 2024-09-14T22:36:31.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706274+0000", 2024-09-14T22:36:31.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7599999999999997e-06 2024-09-14T22:36:31.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752458+0000", 2024-09-14T22:36:31.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046183825999999997 2024-09-14T22:36:31.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752545+0000", 2024-09-14T22:36:31.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7272000000000001e-05 2024-09-14T22:36:31.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752639+0000", 2024-09-14T22:36:31.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3956000000000006e-05 2024-09-14T22:36:31.031 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.032 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.032 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.032 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4169 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13293568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:31.032 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706278+0000", 2024-09-14T22:36:31.032 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 7.2760456570000001, 2024-09-14T22:36:31.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046489181999999997, 2024-09-14T22:36:31.032 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.032 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.032 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.032 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.032 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.033 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.033 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4169 2024-09-14T22:36:31.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.033 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.033 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.033 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706278+0000", 2024-09-14T22:36:31.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.033 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.033 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706278+0000", 2024-09-14T22:36:31.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706279+0000", 2024-09-14T22:36:31.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6100000000000002e-07 2024-09-14T22:36:31.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706280+0000", 2024-09-14T22:36:31.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2589999999999999e-06 2024-09-14T22:36:31.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706283+0000", 2024-09-14T22:36:31.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.501e-06 2024-09-14T22:36:31.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706286+0000", 2024-09-14T22:36:31.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5409999999999999e-06 2024-09-14T22:36:31.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752680+0000", 2024-09-14T22:36:31.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046393886000000002 2024-09-14T22:36:31.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752723+0000", 2024-09-14T22:36:31.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2870000000000001e-05 2024-09-14T22:36:31.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752767+0000", 2024-09-14T22:36:31.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4864000000000003e-05 2024-09-14T22:36:31.036 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.037 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.037 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.037 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4170 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13301760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:31.037 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706289+0000", 2024-09-14T22:36:31.037 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 7.2760347589999999, 2024-09-14T22:36:31.037 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046691115999999998, 2024-09-14T22:36:31.037 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.037 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.037 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.037 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.037 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.038 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.038 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4170 2024-09-14T22:36:31.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.038 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706289+0000", 2024-09-14T22:36:31.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.041 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.041 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706289+0000", 2024-09-14T22:36:31.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.041 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.041 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706289+0000", 2024-09-14T22:36:31.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3200000000000001e-07 2024-09-14T22:36:31.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.042 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.042 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706296+0000", 2024-09-14T22:36:31.042 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.922e-06 2024-09-14T22:36:31.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.042 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.042 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706303+0000", 2024-09-14T22:36:31.042 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3980000000000004e-06 2024-09-14T22:36:31.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.042 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.042 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706307+0000", 2024-09-14T22:36:31.042 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6750000000000001e-06 2024-09-14T22:36:31.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.043 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752799+0000", 2024-09-14T22:36:31.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046491609000000003 2024-09-14T22:36:31.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.043 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752884+0000", 2024-09-14T22:36:31.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4876999999999994e-05 2024-09-14T22:36:31.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.043 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752980+0000", 2024-09-14T22:36:31.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6403000000000005e-05 2024-09-14T22:36:31.044 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.044 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.044 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.044 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6292 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8329216~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:31.044 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966066+0000", 2024-09-14T22:36:31.044 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 7.016257821, 2024-09-14T22:36:31.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045967367000000002, 2024-09-14T22:36:31.044 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.044 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.044 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.044 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.045 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.045 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.045 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6292 2024-09-14T22:36:31.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.045 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.045 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.045 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966066+0000", 2024-09-14T22:36:31.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.045 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.045 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966066+0000", 2024-09-14T22:36:31.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966067+0000", 2024-09-14T22:36:31.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6499999999999999e-07 2024-09-14T22:36:31.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966069+0000", 2024-09-14T22:36:31.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4059999999999999e-06 2024-09-14T22:36:31.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966077+0000", 2024-09-14T22:36:31.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5120000000000002e-06 2024-09-14T22:36:31.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966083+0000", 2024-09-14T22:36:31.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5590000000000002e-06 2024-09-14T22:36:31.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.011820+0000", 2024-09-14T22:36:31.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045737281999999997 2024-09-14T22:36:31.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.011911+0000", 2024-09-14T22:36:31.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0153000000000002e-05 2024-09-14T22:36:31.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012033+0000", 2024-09-14T22:36:31.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012289000000000001 2024-09-14T22:36:31.048 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.048 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.048 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.048 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6293 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8337408~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:31.048 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966091+0000", 2024-09-14T22:36:31.049 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 7.0162327270000002, 2024-09-14T22:36:31.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046181672, 2024-09-14T22:36:31.049 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.049 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.049 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.049 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.049 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.049 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.049 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6293 2024-09-14T22:36:31.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.049 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966091+0000", 2024-09-14T22:36:31.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.050 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.050 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966091+0000", 2024-09-14T22:36:31.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.050 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.050 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966092+0000", 2024-09-14T22:36:31.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:36:31.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.051 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.051 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966094+0000", 2024-09-14T22:36:31.051 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5830000000000001e-06 2024-09-14T22:36:31.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.051 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.051 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966102+0000", 2024-09-14T22:36:31.051 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7470000000000005e-06 2024-09-14T22:36:31.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.051 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.051 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966107+0000", 2024-09-14T22:36:31.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1209999999999998e-06 2024-09-14T22:36:31.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.052 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.052 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012079+0000", 2024-09-14T22:36:31.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045971953000000003 2024-09-14T22:36:31.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.052 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.052 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012168+0000", 2024-09-14T22:36:31.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9398e-05 2024-09-14T22:36:31.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.052 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012273+0000", 2024-09-14T22:36:31.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000104483 2024-09-14T22:36:31.053 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.053 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.053 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.053 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6294 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8345600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:31.053 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966115+0000", 2024-09-14T22:36:31.053 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 7.0162093639999998, 2024-09-14T22:36:31.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046297036999999999, 2024-09-14T22:36:31.053 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.053 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.054 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.054 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.054 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.054 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.054 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6294 2024-09-14T22:36:31.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.054 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966115+0000", 2024-09-14T22:36:31.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966115+0000", 2024-09-14T22:36:31.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966115+0000", 2024-09-14T22:36:31.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7e-07 2024-09-14T22:36:31.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966118+0000", 2024-09-14T22:36:31.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.571e-06 2024-09-14T22:36:31.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.056 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.056 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966125+0000", 2024-09-14T22:36:31.056 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1690000000000003e-06 2024-09-14T22:36:31.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.056 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.056 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966130+0000", 2024-09-14T22:36:31.056 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3290000000000001e-06 2024-09-14T22:36:31.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.056 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.056 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012298+0000", 2024-09-14T22:36:31.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046168168000000002 2024-09-14T22:36:31.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.057 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.057 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012348+0000", 2024-09-14T22:36:31.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9663999999999997e-05 2024-09-14T22:36:31.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.057 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.057 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012412+0000", 2024-09-14T22:36:31.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3665999999999997e-05 2024-09-14T22:36:31.057 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.057 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.058 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.058 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6295 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8353792~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:31.058 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966138+0000", 2024-09-14T22:36:31.058 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 7.0161863860000002, 2024-09-14T22:36:31.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046505112000000001, 2024-09-14T22:36:31.058 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.058 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.058 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.058 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.058 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.058 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.058 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6295 2024-09-14T22:36:31.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.059 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.059 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966138+0000", 2024-09-14T22:36:31.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.059 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966138+0000", 2024-09-14T22:36:31.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.060 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966138+0000", 2024-09-14T22:36:31.060 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5900000000000002e-07 2024-09-14T22:36:31.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.060 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.060 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966140+0000", 2024-09-14T22:36:31.060 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4710000000000001e-06 2024-09-14T22:36:31.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.060 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.060 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966147+0000", 2024-09-14T22:36:31.060 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6760000000000001e-06 2024-09-14T22:36:31.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.061 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.061 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.061 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966153+0000", 2024-09-14T22:36:31.061 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4779999999999999e-06 2024-09-14T22:36:31.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.061 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.061 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.061 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012452+0000", 2024-09-14T22:36:31.061 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046299298000000003 2024-09-14T22:36:31.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.061 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.061 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.061 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012537+0000", 2024-09-14T22:36:31.062 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5439999999999995e-05 2024-09-14T22:36:31.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.062 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.062 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012643+0000", 2024-09-14T22:36:31.062 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010529 2024-09-14T22:36:31.062 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.062 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.062 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.062 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6296 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8361984~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:31.062 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966160+0000", 2024-09-14T22:36:31.063 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 7.0161643319999998, 2024-09-14T22:36:31.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046601423000000003, 2024-09-14T22:36:31.063 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.063 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.063 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.063 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.063 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.063 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.063 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6296 2024-09-14T22:36:31.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.063 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.064 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.064 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966160+0000", 2024-09-14T22:36:31.064 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.064 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.064 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966160+0000", 2024-09-14T22:36:31.064 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.064 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.064 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966160+0000", 2024-09-14T22:36:31.064 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7999999999999995e-07 2024-09-14T22:36:31.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.065 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.065 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966163+0000", 2024-09-14T22:36:31.065 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.514e-06 2024-09-14T22:36:31.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.065 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.065 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966169+0000", 2024-09-14T22:36:31.065 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8880000000000002e-06 2024-09-14T22:36:31.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.065 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.066 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966173+0000", 2024-09-14T22:36:31.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9189999999999998e-06 2024-09-14T22:36:31.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.066 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.066 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012660+0000", 2024-09-14T22:36:31.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046486522000000002 2024-09-14T22:36:31.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.066 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.066 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012703+0000", 2024-09-14T22:36:31.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2715e-05 2024-09-14T22:36:31.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.067 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.067 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012761+0000", 2024-09-14T22:36:31.067 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8285000000000002e-05 2024-09-14T22:36:31.067 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.067 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.067 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.067 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.067 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6297 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8370176~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:31.067 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966180+0000", 2024-09-14T22:36:31.067 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 7.0161435279999997, 2024-09-14T22:36:31.067 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046808482999999998, 2024-09-14T22:36:31.067 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.067 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.068 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.068 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.068 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.068 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.068 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6297 2024-09-14T22:36:31.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.068 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.068 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.068 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966180+0000", 2024-09-14T22:36:31.068 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.068 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.069 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966180+0000", 2024-09-14T22:36:31.069 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.069 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.069 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966181+0000", 2024-09-14T22:36:31.069 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0600000000000001e-07 2024-09-14T22:36:31.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.069 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.069 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966183+0000", 2024-09-14T22:36:31.069 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3539999999999998e-06 2024-09-14T22:36:31.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.070 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966190+0000", 2024-09-14T22:36:31.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.489e-06 2024-09-14T22:36:31.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.070 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966195+0000", 2024-09-14T22:36:31.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3890000000000004e-06 2024-09-14T22:36:31.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.070 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012799+0000", 2024-09-14T22:36:31.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046604200999999998 2024-09-14T22:36:31.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.071 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.071 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012883+0000", 2024-09-14T22:36:31.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3714000000000005e-05 2024-09-14T22:36:31.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.071 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.071 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012989+0000", 2024-09-14T22:36:31.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010593 2024-09-14T22:36:31.071 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.071 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.071 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.072 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6298 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8378368~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:31.072 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966203+0000", 2024-09-14T22:36:31.072 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 7.0161204670000004, 2024-09-14T22:36:31.072 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046904649999999999, 2024-09-14T22:36:31.072 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.072 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.072 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.072 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.072 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.073 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.073 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6298 2024-09-14T22:36:31.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.073 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.073 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966203+0000", 2024-09-14T22:36:31.073 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.073 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966203+0000", 2024-09-14T22:36:31.073 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.074 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966205+0000", 2024-09-14T22:36:31.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5009999999999999e-06 2024-09-14T22:36:31.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.074 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966207+0000", 2024-09-14T22:36:31.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2019999999999998e-06 2024-09-14T22:36:31.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.074 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966214+0000", 2024-09-14T22:36:31.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.815e-06 2024-09-14T22:36:31.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.075 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966219+0000", 2024-09-14T22:36:31.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0590000000000002e-06 2024-09-14T22:36:31.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.075 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013005+0000", 2024-09-14T22:36:31.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046785852000000003 2024-09-14T22:36:31.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.076 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013050+0000", 2024-09-14T22:36:31.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.511e-05 2024-09-14T22:36:31.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.076 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013108+0000", 2024-09-14T22:36:31.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8111e-05 2024-09-14T22:36:31.076 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.076 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.076 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.077 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6299 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8386560~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:31.077 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966238+0000", 2024-09-14T22:36:31.077 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 7.0160861460000001, 2024-09-14T22:36:31.077 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047091099999999997, 2024-09-14T22:36:31.077 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.077 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.077 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.077 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.077 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.077 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.077 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6299 2024-09-14T22:36:31.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.077 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.078 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.078 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966238+0000", 2024-09-14T22:36:31.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.078 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.078 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966238+0000", 2024-09-14T22:36:31.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.078 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.078 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966238+0000", 2024-09-14T22:36:31.079 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1500000000000005e-07 2024-09-14T22:36:31.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.079 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.079 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966241+0000", 2024-09-14T22:36:31.079 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8270000000000002e-06 2024-09-14T22:36:31.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.079 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.079 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966249+0000", 2024-09-14T22:36:31.079 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0029999999999994e-06 2024-09-14T22:36:31.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.079 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.079 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966255+0000", 2024-09-14T22:36:31.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7760000000000003e-06 2024-09-14T22:36:31.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013136+0000", 2024-09-14T22:36:31.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046881379000000001 2024-09-14T22:36:31.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013223+0000", 2024-09-14T22:36:31.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6681000000000006e-05 2024-09-14T22:36:31.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013329+0000", 2024-09-14T22:36:31.081 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105919 2024-09-14T22:36:31.081 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.081 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.081 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.081 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.081 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.081 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:31.083 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:36:31.083 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:31.075+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:31.183 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:31.179+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:31.259 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:31.259 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:31.276 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:36:31.326 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:31.322+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:31.363 INFO:teuthology.orchestra.run.smithi001.stdout:689 writesame 0x365ef55 thru 0x366418f data_size 0x3(0x523b bytes) 2024-09-14T22:36:31.378 INFO:teuthology.orchestra.run.smithi001.stdout:690 read 0x26945ea thru 0x269a9a6 (0x63bd bytes) 2024-09-14T22:36:31.385 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:31.378+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:31.389 INFO:teuthology.orchestra.run.smithi001.stdout:691 write 0xa10d482 thru 0xa11a540 (0xd0bf bytes) 2024-09-14T22:36:31.397 INFO:teuthology.orchestra.run.smithi001.stdout:692 read 0x635510b thru 0x6363da8 (0xec9e bytes) 2024-09-14T22:36:31.397 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:31.397 INFO:teuthology.orchestra.run.smithi001.stdout:695 write 0x9d32847 thru 0x9d3399a (0x1154 bytes) 2024-09-14T22:36:31.404 INFO:teuthology.orchestra.run.smithi001.stdout:696 writesame 0x6b64646 thru 0x6b66303 data_size 0xd(0x1cbe bytes) 2024-09-14T22:36:31.407 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-14T22:36:31.419 INFO:teuthology.orchestra.run.smithi001.stdout:697 trunc from 0xa11a541 to 0x18dc7a8 2024-09-14T22:36:31.422 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:31.423 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:31.425 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:36:31.426 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:36:31.444 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:36:31.454 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:36:31.455 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:36:31.477 INFO:teuthology.orchestra.run.smithi001.stdout:698 writesame 0x17614f7 thru 0x176fb52 data_size 0x17(0xe65c bytes) 2024-09-14T22:36:31.479 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:36:31.479 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:36:31.487 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:31.482+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:31.504 INFO:teuthology.orchestra.run.smithi001.stdout:699 write 0x88d429b thru 0x88d51d7 (0xf3d bytes) 2024-09-14T22:36:31.510 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:36:31.511 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:36:31.511 INFO:teuthology.orchestra.run.smithi001.stdout:700 read 0x30b49ff thru 0x30c05b9 (0xbbbb bytes) 2024-09-14T22:36:31.525 INFO:teuthology.orchestra.run.smithi001.stdout:702 write 0xb69bbed thru 0xb6a43c7 (0x87db bytes) 2024-09-14T22:36:31.533 INFO:teuthology.orchestra.run.smithi001.stdout:704 punch from 0xa96c336 to 0xa9745c5, (0x828f bytes) 2024-09-14T22:36:31.533 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:31.533 INFO:teuthology.orchestra.run.smithi001.stdout:706 read 0x15d210 thru 0x1647f4 (0x75e5 bytes) 2024-09-14T22:36:31.533 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:36:31.533 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:36:31.534 INFO:teuthology.orchestra.run.smithi001.stdout:707 punch from 0x466f47 to 0x471fac, (0xb065 bytes) 2024-09-14T22:36:31.534 INFO:teuthology.orchestra.run.smithi001.stdout:708 punch from 0x6652585 to 0x665ad15, (0x8790 bytes) 2024-09-14T22:36:31.534 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:31.534 INFO:teuthology.orchestra.run.smithi001.stdout:711 write 0x1d1c56f thru 0x1d22c83 (0x6715 bytes) 2024-09-14T22:36:31.536 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:36:31.537 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:36:31.537 INFO:teuthology.orchestra.run.smithi001.stdout:712 read 0x8296ce7 thru 0x82a5bc2 (0xeedc bytes) 2024-09-14T22:36:31.538 INFO:teuthology.orchestra.run.smithi001.stdout:715 writesame 0x468fa9f thru 0x469f7e9 data_size 0x1(0xfd4b bytes) 2024-09-14T22:36:31.570 INFO:teuthology.orchestra.run.smithi001.stdout:716 read 0x89a3a9 thru 0x8a62cb (0xbf23 bytes) 2024-09-14T22:36:31.577 INFO:teuthology.orchestra.run.smithi001.stdout:717 trunc from 0xb6a43c8 to 0x74bfbff 2024-09-14T22:36:31.588 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:31.583+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:31.592 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:31.592 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:36:31.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:36:31.592 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:36:31.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.592 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2124 3.1e 3:79220a5a:::rbd_data.113c3b74a67f.000000000000000b:4 [read 11376640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:31.592 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.447210+0000", 2024-09-14T22:36:31.593 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.1426094379999991, 2024-09-14T22:36:31.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033826508999999998, 2024-09-14T22:36:31.593 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.593 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.593 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.593 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.593 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.593 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.593 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2124 2024-09-14T22:36:31.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.593 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447210+0000", 2024-09-14T22:36:31.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447210+0000", 2024-09-14T22:36:31.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447211+0000", 2024-09-14T22:36:31.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4300000000000003e-07 2024-09-14T22:36:31.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447214+0000", 2024-09-14T22:36:31.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1329999999999999e-06 2024-09-14T22:36:31.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447225+0000", 2024-09-14T22:36:31.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1314999999999999e-05 2024-09-14T22:36:31.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447233+0000", 2024-09-14T22:36:31.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6190000000000002e-06 2024-09-14T22:36:31.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.480915+0000", 2024-09-14T22:36:31.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033682221999999998 2024-09-14T22:36:31.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.480965+0000", 2024-09-14T22:36:31.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9938000000000002e-05 2024-09-14T22:36:31.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.481037+0000", 2024-09-14T22:36:31.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1638999999999996e-05 2024-09-14T22:36:31.597 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.597 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.597 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.597 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2125 3.1e 3:79220a5a:::rbd_data.113c3b74a67f.000000000000000b:4 [read 11384832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:31.597 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.447246+0000", 2024-09-14T22:36:31.597 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.1425737569999992, 2024-09-14T22:36:31.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033991511000000002, 2024-09-14T22:36:31.597 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.597 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.597 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.597 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.597 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.598 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.598 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2125 2024-09-14T22:36:31.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.598 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447246+0000", 2024-09-14T22:36:31.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447246+0000", 2024-09-14T22:36:31.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447247+0000", 2024-09-14T22:36:31.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6899999999999997e-07 2024-09-14T22:36:31.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447251+0000", 2024-09-14T22:36:31.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4449999999999998e-06 2024-09-14T22:36:31.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447267+0000", 2024-09-14T22:36:31.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6032000000000002e-05 2024-09-14T22:36:31.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447275+0000", 2024-09-14T22:36:31.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8439999999999998e-06 2024-09-14T22:36:31.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.481082+0000", 2024-09-14T22:36:31.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033807401000000001 2024-09-14T22:36:31.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.481152+0000", 2024-09-14T22:36:31.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0081999999999997e-05 2024-09-14T22:36:31.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.481237+0000", 2024-09-14T22:36:31.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5137999999999997e-05 2024-09-14T22:36:31.601 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.601 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.601 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.602 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2126 3.1e 3:79220a5a:::rbd_data.113c3b74a67f.000000000000000b:4 [read 11393024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:31.602 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.447300+0000", 2024-09-14T22:36:31.602 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.1425196260000003, 2024-09-14T22:36:31.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034116058999999997, 2024-09-14T22:36:31.602 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.602 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.602 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.602 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.602 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.602 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.602 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2126 2024-09-14T22:36:31.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.602 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447300+0000", 2024-09-14T22:36:31.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447300+0000", 2024-09-14T22:36:31.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447301+0000", 2024-09-14T22:36:31.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.33e-07 2024-09-14T22:36:31.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447305+0000", 2024-09-14T22:36:31.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1239999999999998e-06 2024-09-14T22:36:31.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447317+0000", 2024-09-14T22:36:31.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1557e-05 2024-09-14T22:36:31.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447324+0000", 2024-09-14T22:36:31.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5800000000000003e-06 2024-09-14T22:36:31.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.481273+0000", 2024-09-14T22:36:31.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033948700999999998 2024-09-14T22:36:31.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.481321+0000", 2024-09-14T22:36:31.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8183999999999998e-05 2024-09-14T22:36:31.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.481416+0000", 2024-09-14T22:36:31.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4980000000000002e-05 2024-09-14T22:36:31.606 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.606 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.606 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.606 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2127 3.1e 3:79220a5a:::rbd_data.113c3b74a67f.000000000000000b:4 [read 11401216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:31.606 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.447335+0000", 2024-09-14T22:36:31.606 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.1424847959999997, 2024-09-14T22:36:31.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034280315999999998, 2024-09-14T22:36:31.606 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.606 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.606 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.606 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.607 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.607 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.607 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2127 2024-09-14T22:36:31.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.607 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447335+0000", 2024-09-14T22:36:31.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447335+0000", 2024-09-14T22:36:31.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447336+0000", 2024-09-14T22:36:31.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8999999999999996e-07 2024-09-14T22:36:31.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447339+0000", 2024-09-14T22:36:31.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.038e-06 2024-09-14T22:36:31.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447349+0000", 2024-09-14T22:36:31.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9329999999999996e-06 2024-09-14T22:36:31.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447356+0000", 2024-09-14T22:36:31.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3470000000000001e-06 2024-09-14T22:36:31.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.481462+0000", 2024-09-14T22:36:31.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034106118999999997 2024-09-14T22:36:31.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.481531+0000", 2024-09-14T22:36:31.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9096000000000004e-05 2024-09-14T22:36:31.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.481615+0000", 2024-09-14T22:36:31.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4092999999999996e-05 2024-09-14T22:36:31.610 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.610 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.610 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.610 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2128 3.1e 3:79220a5a:::rbd_data.113c3b74a67f.000000000000000b:4 [read 11409408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:31.611 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.447366+0000", 2024-09-14T22:36:31.611 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.1424538510000009, 2024-09-14T22:36:31.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034418836000000001, 2024-09-14T22:36:31.611 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.611 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.611 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.611 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.611 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.611 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.611 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2128 2024-09-14T22:36:31.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.611 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447366+0000", 2024-09-14T22:36:31.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447366+0000", 2024-09-14T22:36:31.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447366+0000", 2024-09-14T22:36:31.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9999999999999997e-07 2024-09-14T22:36:31.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447369+0000", 2024-09-14T22:36:31.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9569999999999998e-06 2024-09-14T22:36:31.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447377+0000", 2024-09-14T22:36:31.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7689999999999996e-06 2024-09-14T22:36:31.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447383+0000", 2024-09-14T22:36:31.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1380000000000004e-06 2024-09-14T22:36:31.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.481652+0000", 2024-09-14T22:36:31.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034268224999999999 2024-09-14T22:36:31.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.481700+0000", 2024-09-14T22:36:31.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8223999999999998e-05 2024-09-14T22:36:31.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.481785+0000", 2024-09-14T22:36:31.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4923e-05 2024-09-14T22:36:31.615 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.615 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.615 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.615 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2129 3.1e 3:79220a5a:::rbd_data.113c3b74a67f.000000000000000b:4 [read 11417600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:31.615 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.447393+0000", 2024-09-14T22:36:31.615 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.1424271679999993, 2024-09-14T22:36:31.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034605863000000001, 2024-09-14T22:36:31.615 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.615 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.615 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.616 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.616 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.616 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.616 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2129 2024-09-14T22:36:31.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.616 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.616 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.616 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447393+0000", 2024-09-14T22:36:31.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.616 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.616 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447393+0000", 2024-09-14T22:36:31.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447393+0000", 2024-09-14T22:36:31.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7100000000000002e-07 2024-09-14T22:36:31.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447396+0000", 2024-09-14T22:36:31.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9550000000000001e-06 2024-09-14T22:36:31.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447406+0000", 2024-09-14T22:36:31.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8409999999999995e-06 2024-09-14T22:36:31.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447425+0000", 2024-09-14T22:36:31.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9514e-05 2024-09-14T22:36:31.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.481832+0000", 2024-09-14T22:36:31.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034406398999999997 2024-09-14T22:36:31.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.481912+0000", 2024-09-14T22:36:31.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0316999999999997e-05 2024-09-14T22:36:31.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.481998+0000", 2024-09-14T22:36:31.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6266000000000004e-05 2024-09-14T22:36:31.619 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.619 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.619 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.620 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2130 3.1e 3:79220a5a:::rbd_data.113c3b74a67f.000000000000000b:4 [read 11425792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:31.620 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.447479+0000", 2024-09-14T22:36:31.620 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.1423410959999991, 2024-09-14T22:36:31.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034706394000000002, 2024-09-14T22:36:31.620 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.620 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.620 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.620 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.620 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.620 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.620 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2130 2024-09-14T22:36:31.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.620 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447479+0000", 2024-09-14T22:36:31.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447479+0000", 2024-09-14T22:36:31.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447480+0000", 2024-09-14T22:36:31.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.540000000000001e-07 2024-09-14T22:36:31.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447492+0000", 2024-09-14T22:36:31.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2403e-05 2024-09-14T22:36:31.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447505+0000", 2024-09-14T22:36:31.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3363000000000001e-05 2024-09-14T22:36:31.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447526+0000", 2024-09-14T22:36:31.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0557e-05 2024-09-14T22:36:31.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.482054+0000", 2024-09-14T22:36:31.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034528498999999997 2024-09-14T22:36:31.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.482112+0000", 2024-09-14T22:36:31.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8074e-05 2024-09-14T22:36:31.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.482185+0000", 2024-09-14T22:36:31.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2544000000000002e-05 2024-09-14T22:36:31.624 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.624 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.624 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.624 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2131 3.1e 3:79220a5a:::rbd_data.113c3b74a67f.000000000000000b:4 [read 11433984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:31.624 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.447543+0000", 2024-09-14T22:36:31.624 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.1422769680000009, 2024-09-14T22:36:31.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034855945999999999, 2024-09-14T22:36:31.625 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.625 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.625 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.625 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.625 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.625 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.625 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2131 2024-09-14T22:36:31.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.625 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.625 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.625 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447543+0000", 2024-09-14T22:36:31.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447543+0000", 2024-09-14T22:36:31.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447544+0000", 2024-09-14T22:36:31.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0599999999999999e-07 2024-09-14T22:36:31.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447547+0000", 2024-09-14T22:36:31.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7440000000000001e-06 2024-09-14T22:36:31.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447558+0000", 2024-09-14T22:36:31.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0504e-05 2024-09-14T22:36:31.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447565+0000", 2024-09-14T22:36:31.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.52e-06 2024-09-14T22:36:31.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.482239+0000", 2024-09-14T22:36:31.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034673253000000001 2024-09-14T22:36:31.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.482305+0000", 2024-09-14T22:36:31.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6533999999999997e-05 2024-09-14T22:36:31.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.482399+0000", 2024-09-14T22:36:31.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3485000000000004e-05 2024-09-14T22:36:31.629 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.629 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.629 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.629 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2132 3.1e 3:79220a5a:::rbd_data.113c3b74a67f.000000000000000b:4 [read 11442176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:31.629 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.447580+0000", 2024-09-14T22:36:31.629 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.1422395600000002, 2024-09-14T22:36:31.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034975214999999997, 2024-09-14T22:36:31.629 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.629 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.629 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.630 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.630 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.630 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.630 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2132 2024-09-14T22:36:31.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.630 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447580+0000", 2024-09-14T22:36:31.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447580+0000", 2024-09-14T22:36:31.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447581+0000", 2024-09-14T22:36:31.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4600000000000004e-07 2024-09-14T22:36:31.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447609+0000", 2024-09-14T22:36:31.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7840000000000001e-05 2024-09-14T22:36:31.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447640+0000", 2024-09-14T22:36:31.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1557000000000003e-05 2024-09-14T22:36:31.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447650+0000", 2024-09-14T22:36:31.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5070000000000006e-06 2024-09-14T22:36:31.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.482437+0000", 2024-09-14T22:36:31.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034787106999999998 2024-09-14T22:36:31.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.482485+0000", 2024-09-14T22:36:31.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8170000000000001e-05 2024-09-14T22:36:31.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.482555+0000", 2024-09-14T22:36:31.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0388000000000004e-05 2024-09-14T22:36:31.633 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.633 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.633 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.634 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2133 3.1e 3:79220a5a:::rbd_data.113c3b74a67f.000000000000000b:4 [read 11450368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:31.634 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.447670+0000", 2024-09-14T22:36:31.634 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.1421493619999996, 2024-09-14T22:36:31.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035086766999999998, 2024-09-14T22:36:31.634 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.634 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.634 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.634 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.634 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.634 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.634 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2133 2024-09-14T22:36:31.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.634 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447670+0000", 2024-09-14T22:36:31.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447670+0000", 2024-09-14T22:36:31.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447671+0000", 2024-09-14T22:36:31.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9599999999999998e-07 2024-09-14T22:36:31.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447675+0000", 2024-09-14T22:36:31.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9859999999999996e-06 2024-09-14T22:36:31.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447687+0000", 2024-09-14T22:36:31.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2340999999999999e-05 2024-09-14T22:36:31.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447705+0000", 2024-09-14T22:36:31.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7778000000000002e-05 2024-09-14T22:36:31.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.482606+0000", 2024-09-14T22:36:31.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034900492999999998 2024-09-14T22:36:31.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.482673+0000", 2024-09-14T22:36:31.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6973999999999999e-05 2024-09-14T22:36:31.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.482757+0000", 2024-09-14T22:36:31.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4399000000000003e-05 2024-09-14T22:36:31.638 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.638 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.638 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.638 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2134 3.1e 3:79220a5a:::rbd_data.113c3b74a67f.000000000000000b:4 [read 11458560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:31.638 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.447718+0000", 2024-09-14T22:36:31.638 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.1421013949999992, 2024-09-14T22:36:31.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035217466000000003, 2024-09-14T22:36:31.638 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.638 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.638 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.639 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.639 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.639 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.639 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2134 2024-09-14T22:36:31.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.639 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447718+0000", 2024-09-14T22:36:31.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447718+0000", 2024-09-14T22:36:31.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447719+0000", 2024-09-14T22:36:31.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2399999999999997e-07 2024-09-14T22:36:31.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447723+0000", 2024-09-14T22:36:31.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6789999999999998e-06 2024-09-14T22:36:31.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447747+0000", 2024-09-14T22:36:31.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4601e-05 2024-09-14T22:36:31.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447757+0000", 2024-09-14T22:36:31.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0142999999999999e-05 2024-09-14T22:36:31.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.482801+0000", 2024-09-14T22:36:31.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035043188000000003 2024-09-14T22:36:31.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.482848+0000", 2024-09-14T22:36:31.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7463999999999998e-05 2024-09-14T22:36:31.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.482936+0000", 2024-09-14T22:36:31.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7666999999999999e-05 2024-09-14T22:36:31.642 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.642 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.642 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.642 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2135 3.1e 3:79220a5a:::rbd_data.113c3b74a67f.000000000000000b:4 [read 11466752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:31.643 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.447772+0000", 2024-09-14T22:36:31.643 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.1420481739999992, 2024-09-14T22:36:31.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035359571999999999, 2024-09-14T22:36:31.643 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.643 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.643 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.643 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.643 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.643 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.643 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2135 2024-09-14T22:36:31.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.643 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447772+0000", 2024-09-14T22:36:31.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.644 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447772+0000", 2024-09-14T22:36:31.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.644 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447772+0000", 2024-09-14T22:36:31.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.23e-07 2024-09-14T22:36:31.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447776+0000", 2024-09-14T22:36:31.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8759999999999998e-06 2024-09-14T22:36:31.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447786+0000", 2024-09-14T22:36:31.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0190999999999999e-05 2024-09-14T22:36:31.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447796+0000", 2024-09-14T22:36:31.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8460000000000003e-06 2024-09-14T22:36:31.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.482984+0000", 2024-09-14T22:36:31.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035187820000000002 2024-09-14T22:36:31.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.483048+0000", 2024-09-14T22:36:31.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3985000000000005e-05 2024-09-14T22:36:31.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.483131+0000", 2024-09-14T22:36:31.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3131000000000001e-05 2024-09-14T22:36:31.647 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.647 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.647 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.647 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2136 3.1e 3:79220a5a:::rbd_data.113c3b74a67f.000000000000000b:4 [read 11474944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:31.647 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.447807+0000", 2024-09-14T22:36:31.647 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.1420128070000004, 2024-09-14T22:36:31.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035482360999999997, 2024-09-14T22:36:31.647 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.647 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.647 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.647 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.648 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.648 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.648 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2136 2024-09-14T22:36:31.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.648 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447807+0000", 2024-09-14T22:36:31.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447807+0000", 2024-09-14T22:36:31.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447807+0000", 2024-09-14T22:36:31.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5000000000000003e-07 2024-09-14T22:36:31.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447811+0000", 2024-09-14T22:36:31.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.163e-06 2024-09-14T22:36:31.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447825+0000", 2024-09-14T22:36:31.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.487e-05 2024-09-14T22:36:31.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447845+0000", 2024-09-14T22:36:31.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9809e-05 2024-09-14T22:36:31.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.483165+0000", 2024-09-14T22:36:31.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035319982999999999 2024-09-14T22:36:31.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.483222+0000", 2024-09-14T22:36:31.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6301000000000002e-05 2024-09-14T22:36:31.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.483289+0000", 2024-09-14T22:36:31.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7685e-05 2024-09-14T22:36:31.651 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.651 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.652 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.652 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2137 3.1e 3:79220a5a:::rbd_data.113c3b74a67f.000000000000000b:4 [read 11483136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:31.652 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.447885+0000", 2024-09-14T22:36:31.652 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.1419352079999996, 2024-09-14T22:36:31.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035603003000000001, 2024-09-14T22:36:31.652 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.652 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.652 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.652 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.652 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.652 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.653 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2137 2024-09-14T22:36:31.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.653 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447885+0000", 2024-09-14T22:36:31.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447885+0000", 2024-09-14T22:36:31.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447886+0000", 2024-09-14T22:36:31.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0330000000000001e-06 2024-09-14T22:36:31.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447890+0000", 2024-09-14T22:36:31.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5519999999999998e-06 2024-09-14T22:36:31.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447904+0000", 2024-09-14T22:36:31.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3582e-05 2024-09-14T22:36:31.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447913+0000", 2024-09-14T22:36:31.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2259999999999994e-06 2024-09-14T22:36:31.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.483340+0000", 2024-09-14T22:36:31.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035426641000000002 2024-09-14T22:36:31.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.483406+0000", 2024-09-14T22:36:31.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6724999999999996e-05 2024-09-14T22:36:31.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.483488+0000", 2024-09-14T22:36:31.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1243999999999997e-05 2024-09-14T22:36:31.656 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.656 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.656 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.656 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2138 3.1e 3:79220a5a:::rbd_data.113c3b74a67f.000000000000000b:4 [read 11491328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:31.656 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.447926+0000", 2024-09-14T22:36:31.657 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.1418935710000007, 2024-09-14T22:36:31.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035728593000000003, 2024-09-14T22:36:31.657 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.657 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.657 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.657 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.657 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.657 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.657 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2138 2024-09-14T22:36:31.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.657 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447926+0000", 2024-09-14T22:36:31.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447926+0000", 2024-09-14T22:36:31.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447927+0000", 2024-09-14T22:36:31.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6199999999999996e-07 2024-09-14T22:36:31.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447930+0000", 2024-09-14T22:36:31.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.377e-06 2024-09-14T22:36:31.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447942+0000", 2024-09-14T22:36:31.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2065000000000001e-05 2024-09-14T22:36:31.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447960+0000", 2024-09-14T22:36:31.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7903000000000001e-05 2024-09-14T22:36:31.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.483522+0000", 2024-09-14T22:36:31.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035561317000000002 2024-09-14T22:36:31.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.483584+0000", 2024-09-14T22:36:31.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2646999999999998e-05 2024-09-14T22:36:31.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.483655+0000", 2024-09-14T22:36:31.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0421999999999998e-05 2024-09-14T22:36:31.661 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.661 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.661 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.661 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2139 3.1e 3:79220a5a:::rbd_data.113c3b74a67f.000000000000000b:4 [read 11499520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:31.661 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.447985+0000", 2024-09-14T22:36:31.661 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.1418342760000009, 2024-09-14T22:36:31.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035850057999999997, 2024-09-14T22:36:31.661 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.661 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.661 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.661 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.661 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.662 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.662 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2139 2024-09-14T22:36:31.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.662 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447985+0000", 2024-09-14T22:36:31.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447985+0000", 2024-09-14T22:36:31.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447986+0000", 2024-09-14T22:36:31.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2500000000000005e-07 2024-09-14T22:36:31.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.447990+0000", 2024-09-14T22:36:31.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0799999999999999e-06 2024-09-14T22:36:31.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.448002+0000", 2024-09-14T22:36:31.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1265e-05 2024-09-14T22:36:31.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.448009+0000", 2024-09-14T22:36:31.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3390000000000004e-06 2024-09-14T22:36:31.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.483700+0000", 2024-09-14T22:36:31.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035691187999999999 2024-09-14T22:36:31.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.483757+0000", 2024-09-14T22:36:31.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7176000000000003e-05 2024-09-14T22:36:31.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.483835+0000", 2024-09-14T22:36:31.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8285e-05 2024-09-14T22:36:31.665 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.665 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.665 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.665 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2140 3.1e 3:79220a5a:::rbd_data.113c3b74a67f.000000000000000b:4 [read 11507712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:31.666 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.448071+0000", 2024-09-14T22:36:31.666 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.1417487479999995, 2024-09-14T22:36:31.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035927279999999999, 2024-09-14T22:36:31.666 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.666 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.666 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.666 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.666 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.666 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.666 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2140 2024-09-14T22:36:31.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.666 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.666 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.448071+0000", 2024-09-14T22:36:31.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.448071+0000", 2024-09-14T22:36:31.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.448072+0000", 2024-09-14T22:36:31.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1699999999999997e-07 2024-09-14T22:36:31.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.448077+0000", 2024-09-14T22:36:31.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.729e-06 2024-09-14T22:36:31.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.448094+0000", 2024-09-14T22:36:31.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7178000000000001e-05 2024-09-14T22:36:31.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.448102+0000", 2024-09-14T22:36:31.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2199999999999992e-06 2024-09-14T22:36:31.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.483873+0000", 2024-09-14T22:36:31.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035770798999999999 2024-09-14T22:36:31.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.483918+0000", 2024-09-14T22:36:31.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5649999999999998e-05 2024-09-14T22:36:31.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.483998+0000", 2024-09-14T22:36:31.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9887000000000003e-05 2024-09-14T22:36:31.670 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.670 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.670 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.670 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2141 3.1e 3:79220a5a:::rbd_data.113c3b74a67f.000000000000000b:4 [read 11515904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:31.670 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.448118+0000", 2024-09-14T22:36:31.670 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.1417018589999994, 2024-09-14T22:36:31.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.036077233, 2024-09-14T22:36:31.670 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.670 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.670 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.670 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.670 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.671 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.671 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2141 2024-09-14T22:36:31.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.671 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.448118+0000", 2024-09-14T22:36:31.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.448118+0000", 2024-09-14T22:36:31.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.448119+0000", 2024-09-14T22:36:31.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4499999999999996e-07 2024-09-14T22:36:31.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.448131+0000", 2024-09-14T22:36:31.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1959e-05 2024-09-14T22:36:31.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.448155+0000", 2024-09-14T22:36:31.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3986999999999999e-05 2024-09-14T22:36:31.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.448166+0000", 2024-09-14T22:36:31.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1644999999999999e-05 2024-09-14T22:36:31.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.484044+0000", 2024-09-14T22:36:31.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035877932000000001 2024-09-14T22:36:31.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.484114+0000", 2024-09-14T22:36:31.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.957e-05 2024-09-14T22:36:31.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.484195+0000", 2024-09-14T22:36:31.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1295e-05 2024-09-14T22:36:31.674 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.674 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.674 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.674 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2142 3.1e 3:79220a5a:::rbd_data.113c3b74a67f.000000000000000b:4 [read 11524096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:31.675 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.448179+0000", 2024-09-14T22:36:31.675 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.1416407629999998, 2024-09-14T22:36:31.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.036169468000000003, 2024-09-14T22:36:31.675 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.675 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.675 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.675 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.675 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.675 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.675 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2142 2024-09-14T22:36:31.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.675 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.448179+0000", 2024-09-14T22:36:31.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.448179+0000", 2024-09-14T22:36:31.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.448180+0000", 2024-09-14T22:36:31.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2900000000000003e-07 2024-09-14T22:36:31.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.448183+0000", 2024-09-14T22:36:31.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.805e-06 2024-09-14T22:36:31.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.448193+0000", 2024-09-14T22:36:31.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7529999999999996e-06 2024-09-14T22:36:31.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.448202+0000", 2024-09-14T22:36:31.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9330000000000007e-06 2024-09-14T22:36:31.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.484229+0000", 2024-09-14T22:36:31.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.036026662000000001 2024-09-14T22:36:31.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.484281+0000", 2024-09-14T22:36:31.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2354000000000001e-05 2024-09-14T22:36:31.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.484348+0000", 2024-09-14T22:36:31.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7231999999999996e-05 2024-09-14T22:36:31.679 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.679 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.679 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.679 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2143 3.1e 3:79220a5a:::rbd_data.113c3b74a67f.000000000000000b:4 [read 11532288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:31.679 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.448212+0000", 2024-09-14T22:36:31.679 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.1416079470000007, 2024-09-14T22:36:31.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.036321040999999998, 2024-09-14T22:36:31.679 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:31.679 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:31.679 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:31.679 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:31.679 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:31.680 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:31.680 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2143 2024-09-14T22:36:31.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.680 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:31.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:31.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.448212+0000", 2024-09-14T22:36:31.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:31.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.448212+0000", 2024-09-14T22:36:31.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:31.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.681 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:31.681 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.448212+0000", 2024-09-14T22:36:31.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1699999999999998e-07 2024-09-14T22:36:31.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.681 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:31.681 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.448215+0000", 2024-09-14T22:36:31.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1769999999999998e-06 2024-09-14T22:36:31.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.681 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:31.681 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.448224+0000", 2024-09-14T22:36:31.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7159999999999992e-06 2024-09-14T22:36:31.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:31.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.448233+0000", 2024-09-14T22:36:31.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.867e-06 2024-09-14T22:36:31.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:31.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.484384+0000", 2024-09-14T22:36:31.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.036150733999999997 2024-09-14T22:36:31.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:31.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.484451+0000", 2024-09-14T22:36:31.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7088999999999994e-05 2024-09-14T22:36:31.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:31.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:31.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:31.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.484533+0000", 2024-09-14T22:36:31.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1940999999999994e-05 2024-09-14T22:36:31.683 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.683 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.683 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.683 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:31.683 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:31.683 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:31.684 INFO:teuthology.orchestra.run.smithi001.stdout:719 write 0xbc35d2f thru 0xbc3f9de (0x9cb0 bytes) 2024-09-14T22:36:31.685 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:36:31.686 INFO:teuthology.orchestra.run.smithi001.stdout:721 writesame 0xc70494a thru 0xc7119df data_size 0x1(0xd096 bytes) 2024-09-14T22:36:31.686 INFO:teuthology.orchestra.run.smithi001.stdout:722 trunc from 0xc7119e0 to 0x1659eeb 2024-09-14T22:36:31.686 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:31.686 INFO:teuthology.orchestra.run.smithi001.stdout:724 write 0x5be74ed thru 0x5bec3da (0x4eee bytes) 2024-09-14T22:36:31.687 INFO:teuthology.orchestra.run.smithi001.stdout:725 read 0x1b46487 thru 0x1b4afc1 (0x4b3b bytes) 2024-09-14T22:36:31.687 INFO:teuthology.orchestra.run.smithi001.stdout:726 clone 4 order 19 su 2048 sc 14 2024-09-14T22:36:31.688 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:31.682+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:31.789 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:31.783+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:31.805 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T22:36:31.822 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:31.822 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:31.822 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:36:31.822 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:31.837 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:36:31.896 INFO:teuthology.orchestra.run.smithi001.stdout:leaving image image_client.0-clone3 intact 2024-09-14T22:36:31.923 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:31.919+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:31.991 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-14T22:36:31.991 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:31.983+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:32.006 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:32.006 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:32.006 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:36:32.006 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:36:32.007 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:32.020 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:36:32.092 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:32.086+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:32.189 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:32.189 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:36:32.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:36:32.190 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:36:32.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.190 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6376 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8230912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:32.190 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932578+0000", 2024-09-14T22:36:32.190 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.2521329449999996, 2024-09-14T22:36:32.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051558771000000003, 2024-09-14T22:36:32.190 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.190 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.190 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:32.190 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.190 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.190 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.191 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6376 2024-09-14T22:36:32.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.191 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932578+0000", 2024-09-14T22:36:32.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932578+0000", 2024-09-14T22:36:32.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932579+0000", 2024-09-14T22:36:32.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8599999999999996e-07 2024-09-14T22:36:32.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932597+0000", 2024-09-14T22:36:32.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7771e-05 2024-09-14T22:36:32.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932618+0000", 2024-09-14T22:36:32.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1026000000000002e-05 2024-09-14T22:36:32.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932635+0000", 2024-09-14T22:36:32.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.702e-05 2024-09-14T22:36:32.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:32.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.983951+0000", 2024-09-14T22:36:32.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051316639999999997 2024-09-14T22:36:32.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:32.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984031+0000", 2024-09-14T22:36:32.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0085000000000003e-05 2024-09-14T22:36:32.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:32.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984137+0000", 2024-09-14T22:36:32.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105243 2024-09-14T22:36:32.194 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.194 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.194 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.195 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6377 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8239104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:32.195 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932688+0000", 2024-09-14T22:36:32.195 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.2520229139999994, 2024-09-14T22:36:32.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051650696000000003, 2024-09-14T22:36:32.195 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.195 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.195 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:32.195 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.195 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.195 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.195 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6377 2024-09-14T22:36:32.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.195 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932688+0000", 2024-09-14T22:36:32.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932688+0000", 2024-09-14T22:36:32.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932689+0000", 2024-09-14T22:36:32.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4309999999999999e-06 2024-09-14T22:36:32.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932695+0000", 2024-09-14T22:36:32.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8889999999999999e-06 2024-09-14T22:36:32.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932713+0000", 2024-09-14T22:36:32.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7416999999999999e-05 2024-09-14T22:36:32.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932726+0000", 2024-09-14T22:36:32.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3145e-05 2024-09-14T22:36:32.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:32.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984181+0000", 2024-09-14T22:36:32.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051455630000000002 2024-09-14T22:36:32.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:32.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984247+0000", 2024-09-14T22:36:32.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5350000000000003e-05 2024-09-14T22:36:32.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:32.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984339+0000", 2024-09-14T22:36:32.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1834000000000002e-05 2024-09-14T22:36:32.199 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.199 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.199 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.199 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6378 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8247296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:32.199 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932743+0000", 2024-09-14T22:36:32.199 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.2519676799999999, 2024-09-14T22:36:32.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051819373000000002, 2024-09-14T22:36:32.199 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.200 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.200 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:32.200 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.200 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.200 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.200 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6378 2024-09-14T22:36:32.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.200 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932743+0000", 2024-09-14T22:36:32.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932743+0000", 2024-09-14T22:36:32.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932745+0000", 2024-09-14T22:36:32.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.637e-06 2024-09-14T22:36:32.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932750+0000", 2024-09-14T22:36:32.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7859999999999996e-06 2024-09-14T22:36:32.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932767+0000", 2024-09-14T22:36:32.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7621999999999998e-05 2024-09-14T22:36:32.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932781+0000", 2024-09-14T22:36:32.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3403e-05 2024-09-14T22:36:32.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:32.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984383+0000", 2024-09-14T22:36:32.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051602249000000003 2024-09-14T22:36:32.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:32.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984461+0000", 2024-09-14T22:36:32.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8467000000000006e-05 2024-09-14T22:36:32.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:32.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984562+0000", 2024-09-14T22:36:32.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101209 2024-09-14T22:36:32.203 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.203 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.203 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.204 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6379 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8255488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:32.204 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932796+0000", 2024-09-14T22:36:32.204 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.2519142890000001, 2024-09-14T22:36:32.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051961245000000003, 2024-09-14T22:36:32.204 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.204 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.204 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:32.204 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.204 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.204 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.204 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6379 2024-09-14T22:36:32.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.205 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932796+0000", 2024-09-14T22:36:32.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932796+0000", 2024-09-14T22:36:32.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932797+0000", 2024-09-14T22:36:32.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9700000000000005e-07 2024-09-14T22:36:32.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932823+0000", 2024-09-14T22:36:32.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5273999999999999e-05 2024-09-14T22:36:32.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932840+0000", 2024-09-14T22:36:32.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7774e-05 2024-09-14T22:36:32.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932862+0000", 2024-09-14T22:36:32.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1957e-05 2024-09-14T22:36:32.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:32.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984603+0000", 2024-09-14T22:36:32.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051740543 2024-09-14T22:36:32.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:32.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984669+0000", 2024-09-14T22:36:32.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.567e-05 2024-09-14T22:36:32.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:32.208 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984758+0000", 2024-09-14T22:36:32.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9129999999999995e-05 2024-09-14T22:36:32.208 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.208 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.208 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.208 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6380 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8263680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:32.208 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932879+0000", 2024-09-14T22:36:32.208 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.2518322099999999, 2024-09-14T22:36:32.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052102860000000001, 2024-09-14T22:36:32.209 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.209 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.209 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:32.209 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.209 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.209 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.209 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6380 2024-09-14T22:36:32.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.209 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932879+0000", 2024-09-14T22:36:32.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932879+0000", 2024-09-14T22:36:32.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932880+0000", 2024-09-14T22:36:32.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.054e-06 2024-09-14T22:36:32.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932885+0000", 2024-09-14T22:36:32.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1290000000000004e-06 2024-09-14T22:36:32.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932901+0000", 2024-09-14T22:36:32.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5908000000000001e-05 2024-09-14T22:36:32.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932913+0000", 2024-09-14T22:36:32.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2408999999999999e-05 2024-09-14T22:36:32.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:32.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984800+0000", 2024-09-14T22:36:32.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051886671000000002 2024-09-14T22:36:32.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.212 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:32.212 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984879+0000", 2024-09-14T22:36:32.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9200000000000001e-05 2024-09-14T22:36:32.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.212 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:32.212 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984981+0000", 2024-09-14T22:36:32.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102489 2024-09-14T22:36:32.212 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.212 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.213 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.213 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6381 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8271872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:32.213 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932929+0000", 2024-09-14T22:36:32.213 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.2517818789999993, 2024-09-14T22:36:32.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052251484000000001, 2024-09-14T22:36:32.213 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.213 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.213 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:32.213 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.213 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.213 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.214 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6381 2024-09-14T22:36:32.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.214 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932929+0000", 2024-09-14T22:36:32.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932929+0000", 2024-09-14T22:36:32.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932930+0000", 2024-09-14T22:36:32.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3399999999999998e-07 2024-09-14T22:36:32.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932934+0000", 2024-09-14T22:36:32.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5719999999999996e-06 2024-09-14T22:36:32.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932961+0000", 2024-09-14T22:36:32.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6437000000000001e-05 2024-09-14T22:36:32.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932973+0000", 2024-09-14T22:36:32.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2097e-05 2024-09-14T22:36:32.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:32.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985021+0000", 2024-09-14T22:36:32.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052047955999999999 2024-09-14T22:36:32.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:32.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985089+0000", 2024-09-14T22:36:32.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7725999999999995e-05 2024-09-14T22:36:32.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:32.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985180+0000", 2024-09-14T22:36:32.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1861999999999996e-05 2024-09-14T22:36:32.217 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.217 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.217 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.217 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6382 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8280064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:32.217 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933013+0000", 2024-09-14T22:36:32.217 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.2516977170000008, 2024-09-14T22:36:32.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052391118, 2024-09-14T22:36:32.218 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.218 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.218 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:32.218 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.218 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.218 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.218 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6382 2024-09-14T22:36:32.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.218 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933013+0000", 2024-09-14T22:36:32.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933013+0000", 2024-09-14T22:36:32.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933014+0000", 2024-09-14T22:36:32.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.375e-06 2024-09-14T22:36:32.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933020+0000", 2024-09-14T22:36:32.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0220000000000001e-06 2024-09-14T22:36:32.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933038+0000", 2024-09-14T22:36:32.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7694999999999999e-05 2024-09-14T22:36:32.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933051+0000", 2024-09-14T22:36:32.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2721999999999999e-05 2024-09-14T22:36:32.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:32.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985224+0000", 2024-09-14T22:36:32.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052172757 2024-09-14T22:36:32.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:32.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985303+0000", 2024-09-14T22:36:32.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9426999999999997e-05 2024-09-14T22:36:32.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:32.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985404+0000", 2024-09-14T22:36:32.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010111999999999999 2024-09-14T22:36:32.222 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.222 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.222 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.222 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6383 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8288256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:32.222 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933082+0000", 2024-09-14T22:36:32.222 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.2516287370000008, 2024-09-14T22:36:32.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052525761999999997, 2024-09-14T22:36:32.222 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.222 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.222 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:32.222 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.223 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.223 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.223 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6383 2024-09-14T22:36:32.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.223 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933082+0000", 2024-09-14T22:36:32.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933082+0000", 2024-09-14T22:36:32.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933084+0000", 2024-09-14T22:36:32.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5659999999999999e-06 2024-09-14T22:36:32.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933089+0000", 2024-09-14T22:36:32.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.271e-06 2024-09-14T22:36:32.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933105+0000", 2024-09-14T22:36:32.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5685999999999999e-05 2024-09-14T22:36:32.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933118+0000", 2024-09-14T22:36:32.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3188e-05 2024-09-14T22:36:32.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:32.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985448+0000", 2024-09-14T22:36:32.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052330176999999999 2024-09-14T22:36:32.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:32.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985514+0000", 2024-09-14T22:36:32.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6029000000000001e-05 2024-09-14T22:36:32.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:32.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985608+0000", 2024-09-14T22:36:32.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3844999999999993e-05 2024-09-14T22:36:32.226 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.226 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.226 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.226 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6384 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8296448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:32.227 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933134+0000", 2024-09-14T22:36:32.227 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.2515763779999993, 2024-09-14T22:36:32.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052688835000000003, 2024-09-14T22:36:32.227 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.227 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.227 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:32.227 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.227 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.227 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.227 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6384 2024-09-14T22:36:32.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.227 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933134+0000", 2024-09-14T22:36:32.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933134+0000", 2024-09-14T22:36:32.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933135+0000", 2024-09-14T22:36:32.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.78e-07 2024-09-14T22:36:32.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933141+0000", 2024-09-14T22:36:32.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1540000000000001e-06 2024-09-14T22:36:32.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933156+0000", 2024-09-14T22:36:32.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5488000000000001e-05 2024-09-14T22:36:32.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933169+0000", 2024-09-14T22:36:32.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2649e-05 2024-09-14T22:36:32.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:32.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985650+0000", 2024-09-14T22:36:32.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052481112000000003 2024-09-14T22:36:32.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:32.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985724+0000", 2024-09-14T22:36:32.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3953000000000002e-05 2024-09-14T22:36:32.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:32.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985823+0000", 2024-09-14T22:36:32.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9500999999999995e-05 2024-09-14T22:36:32.231 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.231 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.231 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.231 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6385 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8304640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:32.231 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933185+0000", 2024-09-14T22:36:32.231 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.2515256749999999, 2024-09-14T22:36:32.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052892844000000001, 2024-09-14T22:36:32.231 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.231 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.232 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:32.232 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.232 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.232 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.232 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6385 2024-09-14T22:36:32.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.232 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933185+0000", 2024-09-14T22:36:32.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933185+0000", 2024-09-14T22:36:32.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933186+0000", 2024-09-14T22:36:32.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0139999999999999e-06 2024-09-14T22:36:32.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933191+0000", 2024-09-14T22:36:32.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7230000000000004e-06 2024-09-14T22:36:32.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933207+0000", 2024-09-14T22:36:32.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.658e-05 2024-09-14T22:36:32.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933220+0000", 2024-09-14T22:36:32.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2583e-05 2024-09-14T22:36:32.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:32.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985880+0000", 2024-09-14T22:36:32.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052660140000000001 2024-09-14T22:36:32.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:32.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985965+0000", 2024-09-14T22:36:32.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4472e-05 2024-09-14T22:36:32.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:32.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986078+0000", 2024-09-14T22:36:32.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011333200000000001 2024-09-14T22:36:32.235 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.235 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.235 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.236 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6386 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8312832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:32.236 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933236+0000", 2024-09-14T22:36:32.236 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.2514747810000006, 2024-09-14T22:36:32.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053075342999999997, 2024-09-14T22:36:32.236 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.236 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.236 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:32.236 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.236 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.236 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.236 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6386 2024-09-14T22:36:32.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.237 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933236+0000", 2024-09-14T22:36:32.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933236+0000", 2024-09-14T22:36:32.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933237+0000", 2024-09-14T22:36:32.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6500000000000008e-07 2024-09-14T22:36:32.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933242+0000", 2024-09-14T22:36:32.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.938e-06 2024-09-14T22:36:32.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933258+0000", 2024-09-14T22:36:32.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6334e-05 2024-09-14T22:36:32.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933270+0000", 2024-09-14T22:36:32.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1973999999999999e-05 2024-09-14T22:36:32.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:32.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986123+0000", 2024-09-14T22:36:32.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052852769000000001 2024-09-14T22:36:32.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:32.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986204+0000", 2024-09-14T22:36:32.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1458999999999994e-05 2024-09-14T22:36:32.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:32.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986311+0000", 2024-09-14T22:36:32.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010690400000000001 2024-09-14T22:36:32.240 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.240 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.240 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.240 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6387 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8321024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:32.240 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933335+0000", 2024-09-14T22:36:32.240 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.2513755109999991, 2024-09-14T22:36:32.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053182363000000003, 2024-09-14T22:36:32.240 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.240 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.241 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:32.241 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.241 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.241 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.241 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6387 2024-09-14T22:36:32.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.241 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933335+0000", 2024-09-14T22:36:32.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933335+0000", 2024-09-14T22:36:32.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933337+0000", 2024-09-14T22:36:32.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3829999999999999e-06 2024-09-14T22:36:32.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933343+0000", 2024-09-14T22:36:32.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1489999999999999e-06 2024-09-14T22:36:32.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933361+0000", 2024-09-14T22:36:32.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8105e-05 2024-09-14T22:36:32.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933374+0000", 2024-09-14T22:36:32.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.33e-05 2024-09-14T22:36:32.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:32.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986355+0000", 2024-09-14T22:36:32.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052981053 2024-09-14T22:36:32.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:32.244 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986424+0000", 2024-09-14T22:36:32.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8864999999999999e-05 2024-09-14T22:36:32.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:32.244 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986518+0000", 2024-09-14T22:36:32.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3508e-05 2024-09-14T22:36:32.244 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.244 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.244 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.245 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6388 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8329216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:32.245 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933390+0000", 2024-09-14T22:36:32.245 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.2513209249999999, 2024-09-14T22:36:32.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053371605000000003, 2024-09-14T22:36:32.245 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.245 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.245 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:32.245 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.245 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.245 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.245 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6388 2024-09-14T22:36:32.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.245 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933390+0000", 2024-09-14T22:36:32.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933390+0000", 2024-09-14T22:36:32.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933391+0000", 2024-09-14T22:36:32.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.443e-06 2024-09-14T22:36:32.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933396+0000", 2024-09-14T22:36:32.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1680000000000003e-06 2024-09-14T22:36:32.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933412+0000", 2024-09-14T22:36:32.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5302999999999999e-05 2024-09-14T22:36:32.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933439+0000", 2024-09-14T22:36:32.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6747999999999999e-05 2024-09-14T22:36:32.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.248 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:32.248 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986562+0000", 2024-09-14T22:36:32.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053123161000000002 2024-09-14T22:36:32.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.248 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:32.248 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986641+0000", 2024-09-14T22:36:32.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9086999999999997e-05 2024-09-14T22:36:32.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.249 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:32.249 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986761+0000", 2024-09-14T22:36:32.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120695 2024-09-14T22:36:32.249 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.249 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.249 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.249 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6389 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8337408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:32.249 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933456+0000", 2024-09-14T22:36:32.249 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.2512550739999995, 2024-09-14T22:36:32.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053514182, 2024-09-14T22:36:32.249 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.250 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.250 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:32.250 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.250 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.250 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.250 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6389 2024-09-14T22:36:32.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.250 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933456+0000", 2024-09-14T22:36:32.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933456+0000", 2024-09-14T22:36:32.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933457+0000", 2024-09-14T22:36:32.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0020000000000001e-06 2024-09-14T22:36:32.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933462+0000", 2024-09-14T22:36:32.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8400000000000002e-06 2024-09-14T22:36:32.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933477+0000", 2024-09-14T22:36:32.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.592e-05 2024-09-14T22:36:32.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933489+0000", 2024-09-14T22:36:32.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1955e-05 2024-09-14T22:36:32.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:32.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986798+0000", 2024-09-14T22:36:32.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053308263000000002 2024-09-14T22:36:32.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:32.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986871+0000", 2024-09-14T22:36:32.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3518999999999999e-05 2024-09-14T22:36:32.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:32.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986970+0000", 2024-09-14T22:36:32.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8683000000000004e-05 2024-09-14T22:36:32.253 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.254 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.254 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.254 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6390 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8345600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:32.254 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933504+0000", 2024-09-14T22:36:32.254 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.2512062890000006, 2024-09-14T22:36:32.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053719579000000003, 2024-09-14T22:36:32.254 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.254 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.254 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:32.254 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.255 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.255 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.255 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6390 2024-09-14T22:36:32.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.255 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933504+0000", 2024-09-14T22:36:32.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933504+0000", 2024-09-14T22:36:32.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933505+0000", 2024-09-14T22:36:32.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8299999999999995e-07 2024-09-14T22:36:32.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933520+0000", 2024-09-14T22:36:32.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4948e-05 2024-09-14T22:36:32.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933538+0000", 2024-09-14T22:36:32.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7286e-05 2024-09-14T22:36:32.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933550+0000", 2024-09-14T22:36:32.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2223e-05 2024-09-14T22:36:32.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:32.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987028+0000", 2024-09-14T22:36:32.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053478500999999998 2024-09-14T22:36:32.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:32.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987111+0000", 2024-09-14T22:36:32.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3158999999999994e-05 2024-09-14T22:36:32.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:32.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987224+0000", 2024-09-14T22:36:32.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011257899999999999 2024-09-14T22:36:32.258 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.258 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.258 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.259 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6391 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8353792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:32.259 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933576+0000", 2024-09-14T22:36:32.259 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.2511348439999992, 2024-09-14T22:36:32.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053858196999999997, 2024-09-14T22:36:32.259 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.259 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.259 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:32.259 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.259 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.259 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.259 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6391 2024-09-14T22:36:32.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.259 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933576+0000", 2024-09-14T22:36:32.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933576+0000", 2024-09-14T22:36:32.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933578+0000", 2024-09-14T22:36:32.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9539999999999998e-06 2024-09-14T22:36:32.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.261 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933584+0000", 2024-09-14T22:36:32.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9030000000000001e-06 2024-09-14T22:36:32.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.261 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933601+0000", 2024-09-14T22:36:32.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7703e-05 2024-09-14T22:36:32.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933614+0000", 2024-09-14T22:36:32.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2330999999999999e-05 2024-09-14T22:36:32.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:32.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987275+0000", 2024-09-14T22:36:32.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053661470000000003 2024-09-14T22:36:32.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:32.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987342+0000", 2024-09-14T22:36:32.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6266000000000006e-05 2024-09-14T22:36:32.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.263 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.263 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:32.263 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987434+0000", 2024-09-14T22:36:32.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2570000000000003e-05 2024-09-14T22:36:32.263 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.263 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.263 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.263 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.263 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.263 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6392 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8361984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:32.263 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933648+0000", 2024-09-14T22:36:32.263 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.2510624559999997, 2024-09-14T22:36:32.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054032430999999999, 2024-09-14T22:36:32.263 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.264 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.264 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:32.264 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.264 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.264 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.264 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6392 2024-09-14T22:36:32.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.264 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933648+0000", 2024-09-14T22:36:32.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933648+0000", 2024-09-14T22:36:32.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933650+0000", 2024-09-14T22:36:32.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3969999999999999e-06 2024-09-14T22:36:32.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933655+0000", 2024-09-14T22:36:32.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7690000000000002e-06 2024-09-14T22:36:32.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933672+0000", 2024-09-14T22:36:32.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6359999999999999e-05 2024-09-14T22:36:32.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933696+0000", 2024-09-14T22:36:32.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4630999999999998e-05 2024-09-14T22:36:32.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:32.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987490+0000", 2024-09-14T22:36:32.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053793256999999997 2024-09-14T22:36:32.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:32.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987572+0000", 2024-09-14T22:36:32.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1897999999999994e-05 2024-09-14T22:36:32.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:32.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987681+0000", 2024-09-14T22:36:32.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109119 2024-09-14T22:36:32.267 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.267 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.267 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.268 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6393 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8370176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:32.268 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933713+0000", 2024-09-14T22:36:32.268 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.2509975020000006, 2024-09-14T22:36:32.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054194111000000003, 2024-09-14T22:36:32.268 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.268 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.268 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:32.268 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.268 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.268 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.268 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6393 2024-09-14T22:36:32.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.269 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933713+0000", 2024-09-14T22:36:32.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933713+0000", 2024-09-14T22:36:32.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933714+0000", 2024-09-14T22:36:32.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2109999999999999e-06 2024-09-14T22:36:32.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933721+0000", 2024-09-14T22:36:32.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2489999999999998e-06 2024-09-14T22:36:32.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933751+0000", 2024-09-14T22:36:32.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0239999999999998e-05 2024-09-14T22:36:32.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933764+0000", 2024-09-14T22:36:32.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.322e-05 2024-09-14T22:36:32.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:32.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987737+0000", 2024-09-14T22:36:32.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053972539 2024-09-14T22:36:32.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:32.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987813+0000", 2024-09-14T22:36:32.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6635000000000001e-05 2024-09-14T22:36:32.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.272 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:32.272 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987907+0000", 2024-09-14T22:36:32.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4017000000000005e-05 2024-09-14T22:36:32.272 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.272 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.272 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.272 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.272 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6394 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8378368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:32.272 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933781+0000", 2024-09-14T22:36:32.272 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.2509297430000004, 2024-09-14T22:36:32.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054378276000000003, 2024-09-14T22:36:32.272 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.272 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.273 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:32.273 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.273 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.273 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.273 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6394 2024-09-14T22:36:32.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.273 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933781+0000", 2024-09-14T22:36:32.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.274 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933781+0000", 2024-09-14T22:36:32.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.274 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933782+0000", 2024-09-14T22:36:32.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9199999999999999e-07 2024-09-14T22:36:32.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.274 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933788+0000", 2024-09-14T22:36:32.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9340000000000003e-06 2024-09-14T22:36:32.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933803+0000", 2024-09-14T22:36:32.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5362999999999998e-05 2024-09-14T22:36:32.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933816+0000", 2024-09-14T22:36:32.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2456999999999999e-05 2024-09-14T22:36:32.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:32.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987961+0000", 2024-09-14T22:36:32.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054145581999999998 2024-09-14T22:36:32.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:32.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988042+0000", 2024-09-14T22:36:32.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1075000000000006e-05 2024-09-14T22:36:32.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:32.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988159+0000", 2024-09-14T22:36:32.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116873 2024-09-14T22:36:32.276 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.276 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.276 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.277 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6395 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8386560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:32.277 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:36:32.277 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.2508788489999993, 2024-09-14T22:36:32.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054534379000000001, 2024-09-14T22:36:32.277 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.277 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.277 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:32.277 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.277 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.277 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.277 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6395 2024-09-14T22:36:32.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.278 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:36:32.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:36:32.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933833+0000", 2024-09-14T22:36:32.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3859999999999999e-06 2024-09-14T22:36:32.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933838+0000", 2024-09-14T22:36:32.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8709999999999996e-06 2024-09-14T22:36:32.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933854+0000", 2024-09-14T22:36:32.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6189e-05 2024-09-14T22:36:32.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.280 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.280 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933871+0000", 2024-09-14T22:36:32.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.626e-05 2024-09-14T22:36:32.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.280 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:32.280 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988198+0000", 2024-09-14T22:36:32.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054327581999999999 2024-09-14T22:36:32.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.280 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:32.280 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988274+0000", 2024-09-14T22:36:32.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5759000000000004e-05 2024-09-14T22:36:32.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:32.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988366+0000", 2024-09-14T22:36:32.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2331999999999996e-05 2024-09-14T22:36:32.281 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.281 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.281 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.281 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.281 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.281 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:32.283 INFO:teuthology.orchestra.run.smithi001.stdout: marking collection for removal 2024-09-14T22:36:32.283 INFO:teuthology.orchestra.run.smithi001.stdout:setting '_remove' omap key 2024-09-14T22:36:32.283 INFO:teuthology.orchestra.run.smithi001.stdout:finish_remove_pgs 3.4_head removing 3.4 2024-09-14T22:36:32.283 INFO:teuthology.orchestra.run.smithi001.stdout:Remove successful 2024-09-14T22:36:32.284 INFO:teuthology.orchestra.run.smithi001.stdout:727 read 0x5be0a45 thru 0x5be7a7c (0x7038 bytes) 2024-09-14T22:36:32.284 INFO:teuthology.orchestra.run.smithi001.stdout:728 write 0x3f8cea8 thru 0x3f9acd8 (0xde31 bytes) 2024-09-14T22:36:32.284 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:36:32.285 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:32.278+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:32.383 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:32.379+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:32.414 INFO:teuthology.orchestra.run.smithi001.stdout:730 write 0x42f5452 thru 0x42fe35b (0x8f0a bytes) 2024-09-14T22:36:32.417 INFO:teuthology.orchestra.run.smithi001.stdout:731 read 0x2affa0a thru 0x2b0e46b (0xea62 bytes) 2024-09-14T22:36:32.422 INFO:teuthology.orchestra.run.smithi001.stdout:734 writesame 0x53a442e thru 0x53aba84 data_size 0x5(0x7657 bytes) 2024-09-14T22:36:32.476 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:32.476 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:36:32.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.476 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19571 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.476 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.445968+0000", 2024-09-14T22:36:32.476 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.019446839, 2024-09-14T22:36:32.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.019474583, 2024-09-14T22:36:32.476 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.476 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.476 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.476 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.477 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.477 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.477 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19571 2024-09-14T22:36:32.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.477 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.445968+0000", 2024-09-14T22:36:32.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.445968+0000", 2024-09-14T22:36:32.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.445969+0000", 2024-09-14T22:36:32.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6000000000000004e-07 2024-09-14T22:36:32.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.445971+0000", 2024-09-14T22:36:32.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4329999999999998e-06 2024-09-14T22:36:32.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.445978+0000", 2024-09-14T22:36:32.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4080000000000003e-06 2024-09-14T22:36:32.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.445983+0000", 2024-09-14T22:36:32.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0679999999999996e-06 2024-09-14T22:36:32.479 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.479 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.479 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.479 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19836 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.479 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.455030+0000", 2024-09-14T22:36:32.479 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.010385686, 2024-09-14T22:36:32.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010484439, 2024-09-14T22:36:32.479 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.480 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.480 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.480 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.480 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.480 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.480 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19836 2024-09-14T22:36:32.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.480 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.480 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455030+0000", 2024-09-14T22:36:32.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.481 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455030+0000", 2024-09-14T22:36:32.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.481 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455030+0000", 2024-09-14T22:36:32.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8299999999999998e-07 2024-09-14T22:36:32.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.481 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455032+0000", 2024-09-14T22:36:32.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.125e-06 2024-09-14T22:36:32.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455039+0000", 2024-09-14T22:36:32.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5590000000000002e-06 2024-09-14T22:36:32.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455050+0000", 2024-09-14T22:36:32.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0902e-05 2024-09-14T22:36:32.482 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.482 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.482 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.483 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20069 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.483 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.461419+0000", 2024-09-14T22:36:32.483 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0039959399999999999, 2024-09-14T22:36:32.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0041526139999999998, 2024-09-14T22:36:32.483 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.483 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.483 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.483 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.483 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.483 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.483 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20069 2024-09-14T22:36:32.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.483 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.484 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461419+0000", 2024-09-14T22:36:32.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.484 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461419+0000", 2024-09-14T22:36:32.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.484 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461420+0000", 2024-09-14T22:36:32.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6400000000000002e-07 2024-09-14T22:36:32.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.485 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.485 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461422+0000", 2024-09-14T22:36:32.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2520000000000002e-06 2024-09-14T22:36:32.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.485 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.485 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461429+0000", 2024-09-14T22:36:32.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.905e-06 2024-09-14T22:36:32.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.485 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.485 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461434+0000", 2024-09-14T22:36:32.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1170000000000003e-06 2024-09-14T22:36:32.485 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.485 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.486 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.486 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20101 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.486 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462195+0000", 2024-09-14T22:36:32.486 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0032198800000000001, 2024-09-14T22:36:32.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0034360609999999998, 2024-09-14T22:36:32.486 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.486 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.486 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.486 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.486 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.487 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.487 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20101 2024-09-14T22:36:32.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.487 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462195+0000", 2024-09-14T22:36:32.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462195+0000", 2024-09-14T22:36:32.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.488 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.488 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462196+0000", 2024-09-14T22:36:32.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8000000000000001e-07 2024-09-14T22:36:32.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.488 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.488 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462198+0000", 2024-09-14T22:36:32.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0439999999999998e-06 2024-09-14T22:36:32.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.488 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462205+0000", 2024-09-14T22:36:32.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3359999999999997e-06 2024-09-14T22:36:32.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462210+0000", 2024-09-14T22:36:32.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0649999999999998e-06 2024-09-14T22:36:32.489 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.489 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.489 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.489 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19572 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.489 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.445989+0000", 2024-09-14T22:36:32.490 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.019426387999999999, 2024-09-14T22:36:32.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.019701840000000002, 2024-09-14T22:36:32.490 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.490 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.490 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.490 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.490 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.490 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.490 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19572 2024-09-14T22:36:32.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.490 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.445989+0000", 2024-09-14T22:36:32.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.445989+0000", 2024-09-14T22:36:32.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.445989+0000", 2024-09-14T22:36:32.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.72e-07 2024-09-14T22:36:32.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.492 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.492 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.445991+0000", 2024-09-14T22:36:32.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0370000000000001e-06 2024-09-14T22:36:32.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.492 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.492 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.445997+0000", 2024-09-14T22:36:32.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0859999999999998e-06 2024-09-14T22:36:32.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.492 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446002+0000", 2024-09-14T22:36:32.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8999999999999997e-06 2024-09-14T22:36:32.493 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.493 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.493 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.493 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19837 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.493 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.455070+0000", 2024-09-14T22:36:32.493 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.01034538, 2024-09-14T22:36:32.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010680904, 2024-09-14T22:36:32.493 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.493 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.493 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.494 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.494 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.494 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.494 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19837 2024-09-14T22:36:32.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.494 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.494 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455070+0000", 2024-09-14T22:36:32.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.494 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455070+0000", 2024-09-14T22:36:32.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.495 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.495 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.495 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455071+0000", 2024-09-14T22:36:32.495 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4700000000000001e-07 2024-09-14T22:36:32.495 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.495 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.495 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455073+0000", 2024-09-14T22:36:32.495 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4959999999999999e-06 2024-09-14T22:36:32.495 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.496 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.496 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455081+0000", 2024-09-14T22:36:32.496 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5660000000000001e-06 2024-09-14T22:36:32.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.496 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.496 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455086+0000", 2024-09-14T22:36:32.496 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3229999999999997e-06 2024-09-14T22:36:32.496 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.496 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.496 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.497 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20070 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.497 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.461443+0000", 2024-09-14T22:36:32.497 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0039726229999999998, 2024-09-14T22:36:32.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0043659160000000001, 2024-09-14T22:36:32.497 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.497 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.497 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.497 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.497 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.497 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.497 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20070 2024-09-14T22:36:32.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.497 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.497 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461443+0000", 2024-09-14T22:36:32.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461443+0000", 2024-09-14T22:36:32.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461443+0000", 2024-09-14T22:36:32.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1800000000000001e-07 2024-09-14T22:36:32.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461451+0000", 2024-09-14T22:36:32.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.515e-06 2024-09-14T22:36:32.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461458+0000", 2024-09-14T22:36:32.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2799999999999998e-06 2024-09-14T22:36:32.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461463+0000", 2024-09-14T22:36:32.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.378e-06 2024-09-14T22:36:32.499 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.499 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.499 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.500 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20102 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.500 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462216+0000", 2024-09-14T22:36:32.500 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0031989319999999998, 2024-09-14T22:36:32.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0036491879999999998, 2024-09-14T22:36:32.500 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.500 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.500 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.500 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.500 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.500 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.500 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20102 2024-09-14T22:36:32.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.501 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.501 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.501 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462216+0000", 2024-09-14T22:36:32.501 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.501 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.501 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.501 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462216+0000", 2024-09-14T22:36:32.501 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.501 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.501 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.502 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462217+0000", 2024-09-14T22:36:32.502 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9100000000000003e-07 2024-09-14T22:36:32.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.502 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.502 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462219+0000", 2024-09-14T22:36:32.502 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9740000000000001e-06 2024-09-14T22:36:32.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.502 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.502 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462235+0000", 2024-09-14T22:36:32.502 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6050999999999999e-05 2024-09-14T22:36:32.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.502 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462247+0000", 2024-09-14T22:36:32.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2245999999999999e-05 2024-09-14T22:36:32.503 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.503 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.503 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.503 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19573 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.503 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446008+0000", 2024-09-14T22:36:32.503 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.019406907000000001, 2024-09-14T22:36:32.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.019912175000000001, 2024-09-14T22:36:32.503 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.503 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.504 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.504 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.504 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.504 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.504 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19573 2024-09-14T22:36:32.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.504 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446008+0000", 2024-09-14T22:36:32.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446008+0000", 2024-09-14T22:36:32.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446009+0000", 2024-09-14T22:36:32.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6399999999999998e-07 2024-09-14T22:36:32.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446011+0000", 2024-09-14T22:36:32.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9120000000000001e-06 2024-09-14T22:36:32.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446016+0000", 2024-09-14T22:36:32.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.755e-06 2024-09-14T22:36:32.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.506 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.506 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446021+0000", 2024-09-14T22:36:32.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7639999999999996e-06 2024-09-14T22:36:32.506 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.506 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.506 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.506 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19838 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.506 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.455093+0000", 2024-09-14T22:36:32.507 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.010322064000000001, 2024-09-14T22:36:32.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010885475, 2024-09-14T22:36:32.507 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.507 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.507 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.507 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.507 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.507 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.507 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19838 2024-09-14T22:36:32.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.507 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.507 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.507 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455093+0000", 2024-09-14T22:36:32.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455093+0000", 2024-09-14T22:36:32.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455094+0000", 2024-09-14T22:36:32.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2600000000000002e-07 2024-09-14T22:36:32.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455096+0000", 2024-09-14T22:36:32.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3149999999999999e-06 2024-09-14T22:36:32.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455102+0000", 2024-09-14T22:36:32.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3579999999999999e-06 2024-09-14T22:36:32.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455114+0000", 2024-09-14T22:36:32.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1671000000000001e-05 2024-09-14T22:36:32.509 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.509 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.509 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.510 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20071 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.510 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.461471+0000", 2024-09-14T22:36:32.510 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0039446359999999996, 2024-09-14T22:36:32.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0045717800000000001, 2024-09-14T22:36:32.510 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.510 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.510 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.510 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.510 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.510 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.510 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20071 2024-09-14T22:36:32.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.510 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.511 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.511 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461471+0000", 2024-09-14T22:36:32.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.511 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.511 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461471+0000", 2024-09-14T22:36:32.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.511 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.511 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461471+0000", 2024-09-14T22:36:32.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9999999999999998e-07 2024-09-14T22:36:32.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461473+0000", 2024-09-14T22:36:32.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.384e-06 2024-09-14T22:36:32.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461480+0000", 2024-09-14T22:36:32.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4230000000000002e-06 2024-09-14T22:36:32.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461485+0000", 2024-09-14T22:36:32.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3750000000000002e-06 2024-09-14T22:36:32.512 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.512 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.513 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.513 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20103 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.513 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462256+0000", 2024-09-14T22:36:32.513 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0031597259999999999, 2024-09-14T22:36:32.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0038396200000000002, 2024-09-14T22:36:32.513 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.513 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.513 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.513 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.513 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.513 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.514 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20103 2024-09-14T22:36:32.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.514 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462256+0000", 2024-09-14T22:36:32.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462256+0000", 2024-09-14T22:36:32.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462256+0000", 2024-09-14T22:36:32.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1399999999999997e-07 2024-09-14T22:36:32.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462259+0000", 2024-09-14T22:36:32.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.694e-06 2024-09-14T22:36:32.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462266+0000", 2024-09-14T22:36:32.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8940000000000004e-06 2024-09-14T22:36:32.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462271+0000", 2024-09-14T22:36:32.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0459999999999996e-06 2024-09-14T22:36:32.516 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.516 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.516 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.516 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19574 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.516 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446027+0000", 2024-09-14T22:36:32.516 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.019388579, 2024-09-14T22:36:32.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020123531, 2024-09-14T22:36:32.516 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.516 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.516 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.516 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.517 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.517 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.517 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19574 2024-09-14T22:36:32.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.517 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.517 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.517 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446027+0000", 2024-09-14T22:36:32.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.517 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.517 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446027+0000", 2024-09-14T22:36:32.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446035+0000", 2024-09-14T22:36:32.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7120000000000006e-06 2024-09-14T22:36:32.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446045+0000", 2024-09-14T22:36:32.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7210000000000006e-06 2024-09-14T22:36:32.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446053+0000", 2024-09-14T22:36:32.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7959999999999996e-06 2024-09-14T22:36:32.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446066+0000", 2024-09-14T22:36:32.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3192e-05 2024-09-14T22:36:32.519 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.519 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.519 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.519 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19839 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.519 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.455122+0000", 2024-09-14T22:36:32.520 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.010293566000000001, 2024-09-14T22:36:32.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011083366000000001, 2024-09-14T22:36:32.520 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.520 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.520 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.520 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.520 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.520 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.520 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19839 2024-09-14T22:36:32.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.520 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455122+0000", 2024-09-14T22:36:32.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.521 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455122+0000", 2024-09-14T22:36:32.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.521 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455122+0000", 2024-09-14T22:36:32.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6899999999999997e-07 2024-09-14T22:36:32.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455125+0000", 2024-09-14T22:36:32.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3879999999999998e-06 2024-09-14T22:36:32.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455134+0000", 2024-09-14T22:36:32.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4520000000000003e-06 2024-09-14T22:36:32.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455140+0000", 2024-09-14T22:36:32.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3639999999999998e-06 2024-09-14T22:36:32.522 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.523 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.523 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.523 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20072 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.523 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.461492+0000", 2024-09-14T22:36:32.523 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0039229269999999997, 2024-09-14T22:36:32.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0047684349999999997, 2024-09-14T22:36:32.523 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.523 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.523 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.523 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.524 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.524 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.524 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20072 2024-09-14T22:36:32.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.524 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461492+0000", 2024-09-14T22:36:32.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461492+0000", 2024-09-14T22:36:32.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.525 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461493+0000", 2024-09-14T22:36:32.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7700000000000004e-07 2024-09-14T22:36:32.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.525 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461499+0000", 2024-09-14T22:36:32.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0360000000000003e-06 2024-09-14T22:36:32.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.526 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.526 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461507+0000", 2024-09-14T22:36:32.526 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.52e-06 2024-09-14T22:36:32.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.526 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.526 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461513+0000", 2024-09-14T22:36:32.526 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1700000000000002e-06 2024-09-14T22:36:32.526 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.526 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.526 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.527 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20104 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.527 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462277+0000", 2024-09-14T22:36:32.527 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0031381899999999999, 2024-09-14T22:36:32.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0040444160000000003, 2024-09-14T22:36:32.527 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.527 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.527 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.527 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.527 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.527 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.527 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20104 2024-09-14T22:36:32.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.527 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.528 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462277+0000", 2024-09-14T22:36:32.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.528 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462277+0000", 2024-09-14T22:36:32.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.528 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462277+0000", 2024-09-14T22:36:32.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2800000000000002e-07 2024-09-14T22:36:32.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.529 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462280+0000", 2024-09-14T22:36:32.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2570000000000002e-06 2024-09-14T22:36:32.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.529 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462286+0000", 2024-09-14T22:36:32.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2199999999999997e-06 2024-09-14T22:36:32.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462291+0000", 2024-09-14T22:36:32.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5689999999999998e-06 2024-09-14T22:36:32.530 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.530 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.530 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.530 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19575 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.530 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446074+0000", 2024-09-14T22:36:32.530 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.019341198, 2024-09-14T22:36:32.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020302522999999999, 2024-09-14T22:36:32.530 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.530 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.530 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.531 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.531 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.531 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.531 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19575 2024-09-14T22:36:32.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.531 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446074+0000", 2024-09-14T22:36:32.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446074+0000", 2024-09-14T22:36:32.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.532 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.532 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446075+0000", 2024-09-14T22:36:32.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9800000000000004e-07 2024-09-14T22:36:32.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.532 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.532 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446081+0000", 2024-09-14T22:36:32.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9539999999999998e-06 2024-09-14T22:36:32.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.532 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446088+0000", 2024-09-14T22:36:32.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3720000000000001e-06 2024-09-14T22:36:32.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446093+0000", 2024-09-14T22:36:32.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9710000000000003e-06 2024-09-14T22:36:32.533 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.533 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.533 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.533 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19840 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.534 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.455146+0000", 2024-09-14T22:36:32.534 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.010269046, 2024-09-14T22:36:32.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011309161, 2024-09-14T22:36:32.534 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.534 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.534 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.534 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.534 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.534 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.534 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19840 2024-09-14T22:36:32.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.534 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.535 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455146+0000", 2024-09-14T22:36:32.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.535 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455146+0000", 2024-09-14T22:36:32.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.535 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455147+0000", 2024-09-14T22:36:32.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5499999999999998e-07 2024-09-14T22:36:32.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455149+0000", 2024-09-14T22:36:32.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1229999999999998e-06 2024-09-14T22:36:32.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455155+0000", 2024-09-14T22:36:32.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6059999999999999e-06 2024-09-14T22:36:32.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455160+0000", 2024-09-14T22:36:32.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0080000000000002e-06 2024-09-14T22:36:32.536 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.537 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.537 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.537 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20073 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.537 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.461521+0000", 2024-09-14T22:36:32.537 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0038946369999999998, 2024-09-14T22:36:32.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0049909760000000003, 2024-09-14T22:36:32.537 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.537 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.537 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.537 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.537 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.538 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.538 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20073 2024-09-14T22:36:32.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.538 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.538 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461521+0000", 2024-09-14T22:36:32.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.538 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461521+0000", 2024-09-14T22:36:32.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461521+0000", 2024-09-14T22:36:32.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1800000000000001e-07 2024-09-14T22:36:32.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461523+0000", 2024-09-14T22:36:32.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2960000000000001e-06 2024-09-14T22:36:32.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461530+0000", 2024-09-14T22:36:32.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7510000000000002e-06 2024-09-14T22:36:32.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461542+0000", 2024-09-14T22:36:32.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2279999999999999e-05 2024-09-14T22:36:32.540 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.540 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.540 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.540 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19077 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7211008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.540 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.443038+0000", 2024-09-14T22:36:32.541 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.022376745, 2024-09-14T22:36:32.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023535157000000001, 2024-09-14T22:36:32.541 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.541 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.541 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:32.541 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.541 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.541 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.541 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19077 2024-09-14T22:36:32.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.541 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.443038+0000", 2024-09-14T22:36:32.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.443038+0000", 2024-09-14T22:36:32.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.443039+0000", 2024-09-14T22:36:32.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2899999999999999e-07 2024-09-14T22:36:32.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.443040+0000", 2024-09-14T22:36:32.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.437e-06 2024-09-14T22:36:32.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.443045+0000", 2024-09-14T22:36:32.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5290000000000002e-06 2024-09-14T22:36:32.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.443048+0000", 2024-09-14T22:36:32.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6679999999999999e-06 2024-09-14T22:36:32.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.544 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:32.544 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.459277+0000", 2024-09-14T22:36:32.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.016228455999999999 2024-09-14T22:36:32.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.544 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:32.544 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.459344+0000", 2024-09-14T22:36:32.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7348999999999995e-05 2024-09-14T22:36:32.544 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.544 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.544 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.545 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19576 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.545 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446099+0000", 2024-09-14T22:36:32.545 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.019316494, 2024-09-14T22:36:32.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020552245, 2024-09-14T22:36:32.545 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.545 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.545 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.545 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.545 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.545 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.545 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19576 2024-09-14T22:36:32.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.545 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446099+0000", 2024-09-14T22:36:32.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446099+0000", 2024-09-14T22:36:32.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446099+0000", 2024-09-14T22:36:32.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3e-07 2024-09-14T22:36:32.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.547 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.547 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446101+0000", 2024-09-14T22:36:32.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.81e-06 2024-09-14T22:36:32.547 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.547 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.547 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446107+0000", 2024-09-14T22:36:32.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1380000000000004e-06 2024-09-14T22:36:32.547 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.547 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.547 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446112+0000", 2024-09-14T22:36:32.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3900000000000003e-06 2024-09-14T22:36:32.547 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.547 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.547 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.548 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.548 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19841 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.548 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.455167+0000", 2024-09-14T22:36:32.548 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.010247996000000001, 2024-09-14T22:36:32.548 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011546529999999999, 2024-09-14T22:36:32.548 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.548 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.548 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.548 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.548 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.548 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.548 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19841 2024-09-14T22:36:32.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.549 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.549 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.549 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455167+0000", 2024-09-14T22:36:32.549 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.549 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.549 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455167+0000", 2024-09-14T22:36:32.549 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.549 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.549 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455168+0000", 2024-09-14T22:36:32.549 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.96e-07 2024-09-14T22:36:32.550 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.550 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.550 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455170+0000", 2024-09-14T22:36:32.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.131e-06 2024-09-14T22:36:32.550 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.550 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.550 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455176+0000", 2024-09-14T22:36:32.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1260000000000003e-06 2024-09-14T22:36:32.550 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.550 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.551 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455181+0000", 2024-09-14T22:36:32.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8470000000000003e-06 2024-09-14T22:36:32.551 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.551 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.551 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.551 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.551 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20074 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.551 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.461549+0000", 2024-09-14T22:36:32.551 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0038658189999999999, 2024-09-14T22:36:32.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0052206320000000002, 2024-09-14T22:36:32.551 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.551 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.551 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.552 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.552 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.552 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.552 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20074 2024-09-14T22:36:32.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.552 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.552 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.552 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461549+0000", 2024-09-14T22:36:32.552 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.552 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.552 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461549+0000", 2024-09-14T22:36:32.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.553 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.553 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461550+0000", 2024-09-14T22:36:32.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6199999999999998e-07 2024-09-14T22:36:32.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.553 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.553 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461552+0000", 2024-09-14T22:36:32.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1739999999999999e-06 2024-09-14T22:36:32.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.553 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.553 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461559+0000", 2024-09-14T22:36:32.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4699999999999999e-06 2024-09-14T22:36:32.554 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.554 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.554 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461564+0000", 2024-09-14T22:36:32.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.972e-06 2024-09-14T22:36:32.554 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.554 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.554 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.554 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.554 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19577 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.554 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446118+0000", 2024-09-14T22:36:32.554 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.019297684999999998, 2024-09-14T22:36:32.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020709318000000001, 2024-09-14T22:36:32.555 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.555 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.555 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.555 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.555 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.555 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.555 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19577 2024-09-14T22:36:32.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.555 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.555 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.555 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.555 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446118+0000", 2024-09-14T22:36:32.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.556 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.556 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.556 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446118+0000", 2024-09-14T22:36:32.556 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.556 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.556 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.556 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446118+0000", 2024-09-14T22:36:32.556 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9799999999999999e-07 2024-09-14T22:36:32.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.556 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.556 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.556 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446126+0000", 2024-09-14T22:36:32.556 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1159999999999999e-06 2024-09-14T22:36:32.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.557 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.557 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446133+0000", 2024-09-14T22:36:32.557 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4050000000000003e-06 2024-09-14T22:36:32.557 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.557 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.557 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446139+0000", 2024-09-14T22:36:32.557 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2780000000000001e-06 2024-09-14T22:36:32.557 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.557 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.557 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.557 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.557 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19842 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.558 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.455187+0000", 2024-09-14T22:36:32.558 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.010228381, 2024-09-14T22:36:32.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011700960999999999, 2024-09-14T22:36:32.558 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.558 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.558 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.558 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.558 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.558 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.558 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19842 2024-09-14T22:36:32.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.558 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.558 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.558 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455187+0000", 2024-09-14T22:36:32.559 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.559 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.559 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.559 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455187+0000", 2024-09-14T22:36:32.559 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.559 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.559 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.559 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455187+0000", 2024-09-14T22:36:32.559 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9400000000000001e-07 2024-09-14T22:36:32.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.559 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.559 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455202+0000", 2024-09-14T22:36:32.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4443999999999999e-05 2024-09-14T22:36:32.560 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.560 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.560 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455212+0000", 2024-09-14T22:36:32.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.049e-05 2024-09-14T22:36:32.560 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.560 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.560 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455218+0000", 2024-09-14T22:36:32.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4389999999999999e-06 2024-09-14T22:36:32.560 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.561 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.561 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.561 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.561 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.561 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20075 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.561 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.461571+0000", 2024-09-14T22:36:32.561 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0038438460000000002, 2024-09-14T22:36:32.561 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0053740200000000002, 2024-09-14T22:36:32.561 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.561 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.561 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.561 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.562 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.562 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.562 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20075 2024-09-14T22:36:32.562 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.562 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.562 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.562 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.562 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461571+0000", 2024-09-14T22:36:32.562 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.562 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.562 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.562 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.562 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461571+0000", 2024-09-14T22:36:32.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.563 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.563 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.563 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461572+0000", 2024-09-14T22:36:32.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0699999999999997e-07 2024-09-14T22:36:32.563 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.563 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.563 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461574+0000", 2024-09-14T22:36:32.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.04e-06 2024-09-14T22:36:32.563 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.563 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.564 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461580+0000", 2024-09-14T22:36:32.564 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0039999999999996e-06 2024-09-14T22:36:32.564 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.564 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.564 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.564 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461590+0000", 2024-09-14T22:36:32.564 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0023999999999999e-05 2024-09-14T22:36:32.564 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.564 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.564 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.564 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.564 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.564 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19578 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.564 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446146+0000", 2024-09-14T22:36:32.564 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.019269616999999999, 2024-09-14T22:36:32.565 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020863244, 2024-09-14T22:36:32.565 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.565 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.565 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.565 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.565 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.565 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.565 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19578 2024-09-14T22:36:32.565 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.565 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.565 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.565 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.565 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446146+0000", 2024-09-14T22:36:32.565 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.566 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.566 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.566 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.566 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446146+0000", 2024-09-14T22:36:32.566 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.566 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.566 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.566 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.566 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446146+0000", 2024-09-14T22:36:32.566 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.37e-07 2024-09-14T22:36:32.566 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.566 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.567 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.567 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446148+0000", 2024-09-14T22:36:32.567 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.227e-06 2024-09-14T22:36:32.567 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.567 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.567 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.567 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446155+0000", 2024-09-14T22:36:32.567 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5969999999999997e-06 2024-09-14T22:36:32.567 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.567 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.567 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.567 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446160+0000", 2024-09-14T22:36:32.567 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2299999999999999e-06 2024-09-14T22:36:32.567 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.567 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.568 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.568 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.568 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.568 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19843 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.568 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.455226+0000", 2024-09-14T22:36:32.568 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.010189192, 2024-09-14T22:36:32.568 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011841908, 2024-09-14T22:36:32.568 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.568 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.568 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.568 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.568 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.568 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.568 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19843 2024-09-14T22:36:32.569 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.569 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.569 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.569 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.569 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455226+0000", 2024-09-14T22:36:32.569 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.569 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.569 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.569 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.569 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455226+0000", 2024-09-14T22:36:32.569 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.569 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.569 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.570 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.570 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455227+0000", 2024-09-14T22:36:32.570 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.89e-07 2024-09-14T22:36:32.570 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.570 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.570 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.570 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455229+0000", 2024-09-14T22:36:32.570 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.452e-06 2024-09-14T22:36:32.570 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.570 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.570 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.570 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455244+0000", 2024-09-14T22:36:32.570 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4732999999999999e-05 2024-09-14T22:36:32.571 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.571 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.571 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.571 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455249+0000", 2024-09-14T22:36:32.571 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4450000000000004e-06 2024-09-14T22:36:32.571 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.571 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.571 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.571 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.571 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.571 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20076 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.571 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.461596+0000", 2024-09-14T22:36:32.571 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.003818891, 2024-09-14T22:36:32.571 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.005530877, 2024-09-14T22:36:32.572 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.572 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.572 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.572 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.572 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.572 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.572 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20076 2024-09-14T22:36:32.572 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.572 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.572 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.572 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.572 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461596+0000", 2024-09-14T22:36:32.572 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.573 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.573 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.573 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.573 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461596+0000", 2024-09-14T22:36:32.573 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.573 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.573 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.573 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.573 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461597+0000", 2024-09-14T22:36:32.573 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0999999999999999e-07 2024-09-14T22:36:32.573 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.573 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.574 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.574 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461599+0000", 2024-09-14T22:36:32.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0420000000000001e-06 2024-09-14T22:36:32.574 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.574 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.574 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.574 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461605+0000", 2024-09-14T22:36:32.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.178e-06 2024-09-14T22:36:32.574 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.574 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.574 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.574 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461610+0000", 2024-09-14T22:36:32.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6770000000000003e-06 2024-09-14T22:36:32.574 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.575 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.575 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.575 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.575 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.575 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19579 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.575 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446167+0000", 2024-09-14T22:36:32.575 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.019248463, 2024-09-14T22:36:32.575 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021017708999999999, 2024-09-14T22:36:32.575 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.575 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.575 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.575 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.575 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.575 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.576 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19579 2024-09-14T22:36:32.576 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.576 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.576 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.576 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.576 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446167+0000", 2024-09-14T22:36:32.576 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.576 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.576 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.576 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.576 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446167+0000", 2024-09-14T22:36:32.576 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.576 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.576 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.577 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.577 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446167+0000", 2024-09-14T22:36:32.577 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8799999999999998e-07 2024-09-14T22:36:32.577 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.577 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.577 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.577 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446169+0000", 2024-09-14T22:36:32.577 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0880000000000002e-06 2024-09-14T22:36:32.577 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.577 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.577 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.577 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446175+0000", 2024-09-14T22:36:32.577 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1829999999999999e-06 2024-09-14T22:36:32.578 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.578 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.578 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446180+0000", 2024-09-14T22:36:32.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9620000000000001e-06 2024-09-14T22:36:32.578 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.578 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.578 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.578 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.578 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19812 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.578 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.454232+0000", 2024-09-14T22:36:32.578 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.011183139, 2024-09-14T22:36:32.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013011964000000001, 2024-09-14T22:36:32.579 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.579 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.579 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.579 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.579 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.579 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.579 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19812 2024-09-14T22:36:32.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.579 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.579 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.579 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454232+0000", 2024-09-14T22:36:32.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.580 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.580 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454232+0000", 2024-09-14T22:36:32.580 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.580 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.580 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454233+0000", 2024-09-14T22:36:32.580 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.316e-06 2024-09-14T22:36:32.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.580 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454238+0000", 2024-09-14T22:36:32.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7809999999999996e-06 2024-09-14T22:36:32.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.581 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454265+0000", 2024-09-14T22:36:32.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6727999999999999e-05 2024-09-14T22:36:32.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.581 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454273+0000", 2024-09-14T22:36:32.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5189999999999992e-06 2024-09-14T22:36:32.581 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.581 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.582 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.582 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.582 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19844 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.582 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.455256+0000", 2024-09-14T22:36:32.582 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.010158896000000001, 2024-09-14T22:36:32.582 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012045528, 2024-09-14T22:36:32.582 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.582 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.582 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.582 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.582 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.582 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.582 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19844 2024-09-14T22:36:32.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.583 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.583 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.583 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455256+0000", 2024-09-14T22:36:32.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.583 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.583 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455256+0000", 2024-09-14T22:36:32.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.583 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455257+0000", 2024-09-14T22:36:32.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0999999999999998e-07 2024-09-14T22:36:32.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455266+0000", 2024-09-14T22:36:32.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8249999999999994e-06 2024-09-14T22:36:32.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455273+0000", 2024-09-14T22:36:32.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5349999999999999e-06 2024-09-14T22:36:32.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.585 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455279+0000", 2024-09-14T22:36:32.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5489999999999997e-06 2024-09-14T22:36:32.585 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.585 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.585 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.585 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20077 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.585 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.461615+0000", 2024-09-14T22:36:32.585 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0038000130000000001, 2024-09-14T22:36:32.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0057481030000000001, 2024-09-14T22:36:32.585 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.585 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.586 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.586 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.586 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.586 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.586 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20077 2024-09-14T22:36:32.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.586 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.586 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461615+0000", 2024-09-14T22:36:32.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.586 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461615+0000", 2024-09-14T22:36:32.587 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.587 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.587 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461616+0000", 2024-09-14T22:36:32.587 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8500000000000002e-07 2024-09-14T22:36:32.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.587 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.587 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461618+0000", 2024-09-14T22:36:32.587 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.872e-06 2024-09-14T22:36:32.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.587 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.587 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461624+0000", 2024-09-14T22:36:32.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9970000000000004e-06 2024-09-14T22:36:32.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461628+0000", 2024-09-14T22:36:32.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5229999999999997e-06 2024-09-14T22:36:32.588 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.588 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.588 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.588 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19580 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.588 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446186+0000", 2024-09-14T22:36:32.588 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.019229191999999999, 2024-09-14T22:36:32.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021235911, 2024-09-14T22:36:32.589 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.589 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.589 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.589 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.589 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.589 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.589 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19580 2024-09-14T22:36:32.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.589 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446186+0000", 2024-09-14T22:36:32.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.590 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446186+0000", 2024-09-14T22:36:32.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.590 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446186+0000", 2024-09-14T22:36:32.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7500000000000001e-07 2024-09-14T22:36:32.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.590 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446188+0000", 2024-09-14T22:36:32.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.725e-06 2024-09-14T22:36:32.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.591 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.591 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446194+0000", 2024-09-14T22:36:32.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.789e-06 2024-09-14T22:36:32.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.591 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.591 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446199+0000", 2024-09-14T22:36:32.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7809999999999996e-06 2024-09-14T22:36:32.591 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.591 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.592 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.592 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19813 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.592 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.454308+0000", 2024-09-14T22:36:32.592 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.011106886999999999, 2024-09-14T22:36:32.592 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01321335, 2024-09-14T22:36:32.592 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.592 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.592 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.592 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.592 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.592 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.593 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19813 2024-09-14T22:36:32.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.593 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.593 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454308+0000", 2024-09-14T22:36:32.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.593 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454308+0000", 2024-09-14T22:36:32.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454309+0000", 2024-09-14T22:36:32.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3799999999999997e-07 2024-09-14T22:36:32.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454313+0000", 2024-09-14T22:36:32.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.534e-06 2024-09-14T22:36:32.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454330+0000", 2024-09-14T22:36:32.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7333000000000002e-05 2024-09-14T22:36:32.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.595 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.595 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454345+0000", 2024-09-14T22:36:32.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5262999999999999e-05 2024-09-14T22:36:32.595 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.595 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.595 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.595 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19845 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.595 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.455286+0000", 2024-09-14T22:36:32.595 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.010129173999999999, 2024-09-14T22:36:32.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012291668, 2024-09-14T22:36:32.596 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.596 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.596 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.596 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.596 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.596 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.596 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19845 2024-09-14T22:36:32.596 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.596 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.596 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.596 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.596 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455286+0000", 2024-09-14T22:36:32.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.596 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.596 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.597 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.597 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455286+0000", 2024-09-14T22:36:32.597 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.597 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.597 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455287+0000", 2024-09-14T22:36:32.597 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9699999999999995e-07 2024-09-14T22:36:32.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.597 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.597 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455289+0000", 2024-09-14T22:36:32.597 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.12e-06 2024-09-14T22:36:32.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455295+0000", 2024-09-14T22:36:32.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9150000000000001e-06 2024-09-14T22:36:32.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455300+0000", 2024-09-14T22:36:32.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8219999999999997e-06 2024-09-14T22:36:32.598 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.598 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.598 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.598 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20078 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.598 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.461634+0000", 2024-09-14T22:36:32.598 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0037815959999999999, 2024-09-14T22:36:32.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0060047829999999996, 2024-09-14T22:36:32.599 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.599 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.599 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.599 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.599 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.599 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.599 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20078 2024-09-14T22:36:32.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.599 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461634+0000", 2024-09-14T22:36:32.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.600 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461634+0000", 2024-09-14T22:36:32.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.600 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461634+0000", 2024-09-14T22:36:32.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.58e-07 2024-09-14T22:36:32.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.600 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.601 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461641+0000", 2024-09-14T22:36:32.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0029999999999997e-06 2024-09-14T22:36:32.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.601 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461647+0000", 2024-09-14T22:36:32.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4699999999999999e-06 2024-09-14T22:36:32.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.601 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461652+0000", 2024-09-14T22:36:32.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6589999999999998e-06 2024-09-14T22:36:32.601 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.601 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.602 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.602 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19581 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.602 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446204+0000", 2024-09-14T22:36:32.602 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.019211057, 2024-09-14T22:36:32.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021492951999999999, 2024-09-14T22:36:32.602 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.602 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.602 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.602 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.602 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.602 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.602 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19581 2024-09-14T22:36:32.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.603 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446204+0000", 2024-09-14T22:36:32.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446204+0000", 2024-09-14T22:36:32.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446205+0000", 2024-09-14T22:36:32.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7899999999999999e-07 2024-09-14T22:36:32.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.604 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.604 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446206+0000", 2024-09-14T22:36:32.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.781e-06 2024-09-14T22:36:32.604 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.604 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.604 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446212+0000", 2024-09-14T22:36:32.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7610000000000004e-06 2024-09-14T22:36:32.604 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.604 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.604 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446225+0000", 2024-09-14T22:36:32.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2480000000000001e-05 2024-09-14T22:36:32.605 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.605 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.605 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.605 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.605 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.605 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19814 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.605 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.454385+0000", 2024-09-14T22:36:32.605 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.011030223, 2024-09-14T22:36:32.605 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013370107000000001, 2024-09-14T22:36:32.605 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.605 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.605 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.605 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.605 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.605 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.606 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19814 2024-09-14T22:36:32.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.606 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.606 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.606 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454385+0000", 2024-09-14T22:36:32.606 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.606 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.606 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454385+0000", 2024-09-14T22:36:32.606 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.606 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454386+0000", 2024-09-14T22:36:32.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.68e-07 2024-09-14T22:36:32.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454396+0000", 2024-09-14T22:36:32.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9690000000000006e-06 2024-09-14T22:36:32.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454404+0000", 2024-09-14T22:36:32.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.721e-06 2024-09-14T22:36:32.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454421+0000", 2024-09-14T22:36:32.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7113e-05 2024-09-14T22:36:32.608 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.608 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.608 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.608 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19846 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.608 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.455306+0000", 2024-09-14T22:36:32.608 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.010108742, 2024-09-14T22:36:32.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012510854, 2024-09-14T22:36:32.608 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.608 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.608 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.608 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.608 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.608 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.609 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19846 2024-09-14T22:36:32.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.609 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.609 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.609 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.609 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455306+0000", 2024-09-14T22:36:32.609 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.609 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.609 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.609 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455306+0000", 2024-09-14T22:36:32.609 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.609 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.610 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.610 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455307+0000", 2024-09-14T22:36:32.610 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9999999999999998e-07 2024-09-14T22:36:32.610 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.610 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.610 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.610 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455309+0000", 2024-09-14T22:36:32.610 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9690000000000001e-06 2024-09-14T22:36:32.610 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.610 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.610 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.610 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455325+0000", 2024-09-14T22:36:32.610 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6458999999999999e-05 2024-09-14T22:36:32.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.611 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.611 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455333+0000", 2024-09-14T22:36:32.611 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1200000000000002e-06 2024-09-14T22:36:32.611 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.611 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.611 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.611 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20079 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.611 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.461658+0000", 2024-09-14T22:36:32.611 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.003756845, 2024-09-14T22:36:32.611 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0062140870000000001, 2024-09-14T22:36:32.612 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.612 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.612 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.612 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.612 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.612 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.612 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20079 2024-09-14T22:36:32.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.612 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.612 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461658+0000", 2024-09-14T22:36:32.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.613 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461658+0000", 2024-09-14T22:36:32.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.613 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461659+0000", 2024-09-14T22:36:32.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7100000000000001e-07 2024-09-14T22:36:32.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.613 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461661+0000", 2024-09-14T22:36:32.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1780000000000002e-06 2024-09-14T22:36:32.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.614 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461668+0000", 2024-09-14T22:36:32.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7859999999999999e-06 2024-09-14T22:36:32.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.614 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461673+0000", 2024-09-14T22:36:32.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1379999999999998e-06 2024-09-14T22:36:32.614 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.614 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.614 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.615 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.615 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19582 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.615 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446231+0000", 2024-09-14T22:36:32.615 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.019184507999999999, 2024-09-14T22:36:32.615 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021701152000000001, 2024-09-14T22:36:32.615 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.615 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.615 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.615 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.615 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.615 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.615 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19582 2024-09-14T22:36:32.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.616 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.616 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.616 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.616 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446231+0000", 2024-09-14T22:36:32.616 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.616 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.616 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.616 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446231+0000", 2024-09-14T22:36:32.616 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.616 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.616 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.617 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446231+0000", 2024-09-14T22:36:32.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9400000000000001e-07 2024-09-14T22:36:32.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.617 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.617 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446233+0000", 2024-09-14T22:36:32.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0870000000000001e-06 2024-09-14T22:36:32.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.617 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.617 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446239+0000", 2024-09-14T22:36:32.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9000000000000003e-06 2024-09-14T22:36:32.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.618 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.618 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446244+0000", 2024-09-14T22:36:32.618 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6469999999999997e-06 2024-09-14T22:36:32.618 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.618 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.618 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.618 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19815 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.618 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.454435+0000", 2024-09-14T22:36:32.618 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.01098062, 2024-09-14T22:36:32.618 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013554353999999999, 2024-09-14T22:36:32.618 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.619 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.619 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.619 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.619 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.619 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.619 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19815 2024-09-14T22:36:32.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.619 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454435+0000", 2024-09-14T22:36:32.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.620 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454435+0000", 2024-09-14T22:36:32.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.620 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454435+0000", 2024-09-14T22:36:32.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0500000000000004e-07 2024-09-14T22:36:32.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.620 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454438+0000", 2024-09-14T22:36:32.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6410000000000002e-06 2024-09-14T22:36:32.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.621 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.621 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454454+0000", 2024-09-14T22:36:32.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6245000000000001e-05 2024-09-14T22:36:32.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.621 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.621 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454460+0000", 2024-09-14T22:36:32.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0419999999999999e-06 2024-09-14T22:36:32.621 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.621 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.622 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.622 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19847 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.622 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.455344+0000", 2024-09-14T22:36:32.622 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.010071297999999999, 2024-09-14T22:36:32.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012706237, 2024-09-14T22:36:32.622 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.622 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.622 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.622 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.622 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.622 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.622 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19847 2024-09-14T22:36:32.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.623 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.623 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.623 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455344+0000", 2024-09-14T22:36:32.623 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.623 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.623 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455344+0000", 2024-09-14T22:36:32.623 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.624 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.624 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455344+0000", 2024-09-14T22:36:32.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3900000000000005e-07 2024-09-14T22:36:32.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.624 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.624 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.624 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455348+0000", 2024-09-14T22:36:32.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3009999999999998e-06 2024-09-14T22:36:32.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.624 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.624 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.624 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455355+0000", 2024-09-14T22:36:32.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.221e-06 2024-09-14T22:36:32.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.625 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.625 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.625 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455367+0000", 2024-09-14T22:36:32.625 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2211e-05 2024-09-14T22:36:32.625 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.625 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.625 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.625 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.625 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20080 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.625 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.461680+0000", 2024-09-14T22:36:32.625 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0037355159999999999, 2024-09-14T22:36:32.625 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0064288360000000003, 2024-09-14T22:36:32.626 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.626 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.626 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.626 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.626 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.626 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.626 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20080 2024-09-14T22:36:32.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.626 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.626 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461680+0000", 2024-09-14T22:36:32.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.627 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.627 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461680+0000", 2024-09-14T22:36:32.627 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.627 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.627 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461680+0000", 2024-09-14T22:36:32.627 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.58e-07 2024-09-14T22:36:32.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.627 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.627 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461682+0000", 2024-09-14T22:36:32.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0779999999999998e-06 2024-09-14T22:36:32.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.628 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461689+0000", 2024-09-14T22:36:32.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3659999999999997e-06 2024-09-14T22:36:32.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.628 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461693+0000", 2024-09-14T22:36:32.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7269999999999998e-06 2024-09-14T22:36:32.628 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.628 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.628 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.629 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.629 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19583 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.629 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446249+0000", 2024-09-14T22:36:32.629 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.019166239000000002, 2024-09-14T22:36:32.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021921125, 2024-09-14T22:36:32.629 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.629 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.629 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.629 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.629 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.629 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.629 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19583 2024-09-14T22:36:32.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.630 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.630 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.630 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.630 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446249+0000", 2024-09-14T22:36:32.630 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.630 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.630 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.630 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446249+0000", 2024-09-14T22:36:32.630 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.630 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.630 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.630 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446250+0000", 2024-09-14T22:36:32.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3499999999999996e-07 2024-09-14T22:36:32.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446251+0000", 2024-09-14T22:36:32.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7680000000000001e-06 2024-09-14T22:36:32.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446257+0000", 2024-09-14T22:36:32.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6720000000000001e-06 2024-09-14T22:36:32.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.632 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.632 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446267+0000", 2024-09-14T22:36:32.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5670000000000001e-06 2024-09-14T22:36:32.632 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.632 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.632 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.632 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19816 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.632 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.454485+0000", 2024-09-14T22:36:32.632 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.010930251, 2024-09-14T22:36:32.633 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013741681, 2024-09-14T22:36:32.633 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.633 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.633 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.633 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.633 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.633 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.633 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19816 2024-09-14T22:36:32.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.633 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.633 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.633 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454485+0000", 2024-09-14T22:36:32.634 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.634 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.634 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.634 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454485+0000", 2024-09-14T22:36:32.634 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.634 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.634 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.634 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454486+0000", 2024-09-14T22:36:32.634 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8299999999999996e-07 2024-09-14T22:36:32.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.634 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.634 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.635 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454488+0000", 2024-09-14T22:36:32.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6340000000000001e-06 2024-09-14T22:36:32.635 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.635 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.635 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.635 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454502+0000", 2024-09-14T22:36:32.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3862e-05 2024-09-14T22:36:32.635 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.635 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.635 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.635 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454507+0000", 2024-09-14T22:36:32.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2569999999999998e-06 2024-09-14T22:36:32.635 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.635 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.636 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.636 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.636 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19848 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.636 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.455374+0000", 2024-09-14T22:36:32.636 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.010041347000000001, 2024-09-14T22:36:32.636 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012911706, 2024-09-14T22:36:32.636 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.636 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.636 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.636 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.636 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.636 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.636 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19848 2024-09-14T22:36:32.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.637 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.637 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.637 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.637 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455374+0000", 2024-09-14T22:36:32.637 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.637 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.637 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.637 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455374+0000", 2024-09-14T22:36:32.637 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.637 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.637 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455374+0000", 2024-09-14T22:36:32.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.08e-07 2024-09-14T22:36:32.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455377+0000", 2024-09-14T22:36:32.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2970000000000002e-06 2024-09-14T22:36:32.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455383+0000", 2024-09-14T22:36:32.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1519999999999997e-06 2024-09-14T22:36:32.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.639 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.639 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.639 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455387+0000", 2024-09-14T22:36:32.639 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5329999999999996e-06 2024-09-14T22:36:32.639 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.639 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.639 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.639 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.639 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20081 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.639 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.461699+0000", 2024-09-14T22:36:32.639 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0037157729999999999, 2024-09-14T22:36:32.639 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0066426300000000001, 2024-09-14T22:36:32.640 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.640 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.640 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.640 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.640 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.640 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.640 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20081 2024-09-14T22:36:32.640 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.640 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.640 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.640 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.640 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461699+0000", 2024-09-14T22:36:32.640 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.640 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.640 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.641 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.641 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461699+0000", 2024-09-14T22:36:32.641 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.641 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.641 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.641 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.641 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461700+0000", 2024-09-14T22:36:32.641 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2600000000000002e-07 2024-09-14T22:36:32.641 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.641 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.641 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.641 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461702+0000", 2024-09-14T22:36:32.641 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9719999999999999e-06 2024-09-14T22:36:32.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.642 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.642 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.642 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461708+0000", 2024-09-14T22:36:32.642 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7919999999999998e-06 2024-09-14T22:36:32.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.642 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.642 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.642 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461712+0000", 2024-09-14T22:36:32.642 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4499999999999997e-06 2024-09-14T22:36:32.642 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.642 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.642 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.642 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.643 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19584 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.643 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446273+0000", 2024-09-14T22:36:32.643 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.019142681000000002, 2024-09-14T22:36:32.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022131163999999998, 2024-09-14T22:36:32.643 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.643 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.643 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.643 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.643 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.643 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.643 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19584 2024-09-14T22:36:32.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.643 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.643 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.644 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446273+0000", 2024-09-14T22:36:32.644 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.644 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.644 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.644 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.644 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446273+0000", 2024-09-14T22:36:32.644 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.644 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.644 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.644 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.644 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446273+0000", 2024-09-14T22:36:32.644 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4300000000000003e-07 2024-09-14T22:36:32.644 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.644 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.644 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.645 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446275+0000", 2024-09-14T22:36:32.645 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.917e-06 2024-09-14T22:36:32.645 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.645 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.645 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.645 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446281+0000", 2024-09-14T22:36:32.645 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8119999999999996e-06 2024-09-14T22:36:32.645 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.645 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.645 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.645 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446285+0000", 2024-09-14T22:36:32.645 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4700000000000004e-06 2024-09-14T22:36:32.645 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.645 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.646 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.646 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.646 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.646 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19817 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.646 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.454515+0000", 2024-09-14T22:36:32.646 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.010900218, 2024-09-14T22:36:32.646 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01394771, 2024-09-14T22:36:32.646 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.646 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.646 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.646 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.646 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.646 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.646 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19817 2024-09-14T22:36:32.646 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.646 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.647 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.647 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454515+0000", 2024-09-14T22:36:32.647 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.647 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.647 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454515+0000", 2024-09-14T22:36:32.647 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.647 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.647 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454516+0000", 2024-09-14T22:36:32.647 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4799999999999998e-07 2024-09-14T22:36:32.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.648 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.648 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.648 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454518+0000", 2024-09-14T22:36:32.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5100000000000001e-06 2024-09-14T22:36:32.648 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.648 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.648 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.648 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454525+0000", 2024-09-14T22:36:32.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5899999999999996e-06 2024-09-14T22:36:32.648 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.648 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.648 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.648 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454529+0000", 2024-09-14T22:36:32.649 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6720000000000003e-06 2024-09-14T22:36:32.649 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.649 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.649 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.649 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.649 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.649 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20082 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.649 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.461718+0000", 2024-09-14T22:36:32.649 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0036975739999999999, 2024-09-14T22:36:32.649 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0068066180000000004, 2024-09-14T22:36:32.649 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.649 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.649 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.650 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.650 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.650 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.650 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20082 2024-09-14T22:36:32.650 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.650 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.650 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.650 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.650 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461718+0000", 2024-09-14T22:36:32.650 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.650 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.650 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.650 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.650 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461718+0000", 2024-09-14T22:36:32.651 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.651 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.651 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.651 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.651 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461718+0000", 2024-09-14T22:36:32.651 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9200000000000002e-07 2024-09-14T22:36:32.651 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.651 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.651 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.651 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461724+0000", 2024-09-14T22:36:32.651 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3090000000000001e-06 2024-09-14T22:36:32.651 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.651 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.652 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461731+0000", 2024-09-14T22:36:32.652 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3400000000000003e-06 2024-09-14T22:36:32.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.652 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.652 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461735+0000", 2024-09-14T22:36:32.652 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7269999999999998e-06 2024-09-14T22:36:32.652 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.652 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.652 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.652 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.652 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19585 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.653 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446291+0000", 2024-09-14T22:36:32.653 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.019124395999999998, 2024-09-14T22:36:32.653 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022290040000000001, 2024-09-14T22:36:32.653 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.653 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.653 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.653 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.653 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.653 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.653 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19585 2024-09-14T22:36:32.653 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.653 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.653 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.653 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.653 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446291+0000", 2024-09-14T22:36:32.654 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.654 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.654 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.654 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.654 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446291+0000", 2024-09-14T22:36:32.654 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.654 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.654 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.654 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.654 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446291+0000", 2024-09-14T22:36:32.654 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9200000000000002e-07 2024-09-14T22:36:32.654 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.654 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.654 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.654 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446293+0000", 2024-09-14T22:36:32.654 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7099999999999999e-06 2024-09-14T22:36:32.655 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.655 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.655 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.655 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446299+0000", 2024-09-14T22:36:32.655 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5940000000000001e-06 2024-09-14T22:36:32.655 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.655 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.655 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.655 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446303+0000", 2024-09-14T22:36:32.655 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5789999999999997e-06 2024-09-14T22:36:32.655 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.655 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.655 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.655 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.655 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.656 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19818 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.656 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.454536+0000", 2024-09-14T22:36:32.656 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.010879464, 2024-09-14T22:36:32.656 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014104032000000001, 2024-09-14T22:36:32.656 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.656 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.656 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.656 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.656 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.656 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.656 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19818 2024-09-14T22:36:32.656 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.656 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.657 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.657 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.657 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454536+0000", 2024-09-14T22:36:32.657 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.657 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.657 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.657 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.657 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454536+0000", 2024-09-14T22:36:32.657 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.657 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.657 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.657 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.657 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454536+0000", 2024-09-14T22:36:32.658 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8699999999999995e-07 2024-09-14T22:36:32.658 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.658 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.658 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.658 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454538+0000", 2024-09-14T22:36:32.658 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.871e-06 2024-09-14T22:36:32.658 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.658 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.658 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.658 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454546+0000", 2024-09-14T22:36:32.658 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7610000000000007e-06 2024-09-14T22:36:32.658 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.658 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.658 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.658 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454560+0000", 2024-09-14T22:36:32.658 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3526999999999999e-05 2024-09-14T22:36:32.659 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.659 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.659 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.659 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.659 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.659 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20083 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.659 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.461755+0000", 2024-09-14T22:36:32.659 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0036603019999999998, 2024-09-14T22:36:32.659 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0069416180000000001, 2024-09-14T22:36:32.659 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.659 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.659 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.659 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.659 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.660 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.660 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20083 2024-09-14T22:36:32.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.660 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.660 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.660 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461755+0000", 2024-09-14T22:36:32.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.660 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.660 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461755+0000", 2024-09-14T22:36:32.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.661 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.661 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461756+0000", 2024-09-14T22:36:32.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1500000000000004e-07 2024-09-14T22:36:32.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.661 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.661 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461758+0000", 2024-09-14T22:36:32.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4389999999999999e-06 2024-09-14T22:36:32.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.661 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.661 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461768+0000", 2024-09-14T22:36:32.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8810000000000008e-06 2024-09-14T22:36:32.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461773+0000", 2024-09-14T22:36:32.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1839999999999998e-06 2024-09-14T22:36:32.662 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.662 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.662 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.662 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19586 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.662 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446309+0000", 2024-09-14T22:36:32.662 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.019106465999999999, 2024-09-14T22:36:32.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.02244529, 2024-09-14T22:36:32.663 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.663 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.663 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.663 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.663 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.663 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.663 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19586 2024-09-14T22:36:32.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.663 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.663 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446309+0000", 2024-09-14T22:36:32.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.664 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.664 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446309+0000", 2024-09-14T22:36:32.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.664 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.664 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446309+0000", 2024-09-14T22:36:32.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.77e-07 2024-09-14T22:36:32.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.664 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446315+0000", 2024-09-14T22:36:32.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8769999999999998e-06 2024-09-14T22:36:32.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446321+0000", 2024-09-14T22:36:32.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1210000000000003e-06 2024-09-14T22:36:32.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446326+0000", 2024-09-14T22:36:32.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6709999999999998e-06 2024-09-14T22:36:32.665 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.665 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.666 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.666 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19819 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.666 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.454590+0000", 2024-09-14T22:36:32.666 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.010824798, 2024-09-14T22:36:32.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014228322, 2024-09-14T22:36:32.666 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.666 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.666 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.666 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.666 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.666 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.667 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19819 2024-09-14T22:36:32.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.667 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454590+0000", 2024-09-14T22:36:32.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454590+0000", 2024-09-14T22:36:32.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454591+0000", 2024-09-14T22:36:32.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.75e-07 2024-09-14T22:36:32.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454599+0000", 2024-09-14T22:36:32.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0830000000000004e-06 2024-09-14T22:36:32.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454615+0000", 2024-09-14T22:36:32.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5976000000000001e-05 2024-09-14T22:36:32.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454626+0000", 2024-09-14T22:36:32.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0713999999999999e-05 2024-09-14T22:36:32.669 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.669 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.669 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.669 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20084 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.669 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.461779+0000", 2024-09-14T22:36:32.669 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0036359280000000001, 2024-09-14T22:36:32.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0070973720000000002, 2024-09-14T22:36:32.670 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.670 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.670 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.670 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.670 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.670 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.670 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20084 2024-09-14T22:36:32.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.670 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461779+0000", 2024-09-14T22:36:32.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461779+0000", 2024-09-14T22:36:32.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461780+0000", 2024-09-14T22:36:32.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8299999999999998e-07 2024-09-14T22:36:32.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461782+0000", 2024-09-14T22:36:32.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9269999999999999e-06 2024-09-14T22:36:32.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461787+0000", 2024-09-14T22:36:32.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4689999999999996e-06 2024-09-14T22:36:32.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461799+0000", 2024-09-14T22:36:32.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2016e-05 2024-09-14T22:36:32.672 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.673 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.673 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.673 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19587 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.673 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446332+0000", 2024-09-14T22:36:32.673 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.019083678999999999, 2024-09-14T22:36:32.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022608243, 2024-09-14T22:36:32.673 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.673 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.673 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.673 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.673 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.673 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.674 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19587 2024-09-14T22:36:32.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.674 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446332+0000", 2024-09-14T22:36:32.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446332+0000", 2024-09-14T22:36:32.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446332+0000", 2024-09-14T22:36:32.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9700000000000002e-07 2024-09-14T22:36:32.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446334+0000", 2024-09-14T22:36:32.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8330000000000001e-06 2024-09-14T22:36:32.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446339+0000", 2024-09-14T22:36:32.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6810000000000003e-06 2024-09-14T22:36:32.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446344+0000", 2024-09-14T22:36:32.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5759999999999999e-06 2024-09-14T22:36:32.676 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.676 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.676 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.676 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19820 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.676 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.454634+0000", 2024-09-14T22:36:32.676 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.010781363, 2024-09-14T22:36:32.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014363459, 2024-09-14T22:36:32.676 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.676 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.676 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.676 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.676 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.677 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.677 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19820 2024-09-14T22:36:32.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.677 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454634+0000", 2024-09-14T22:36:32.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454634+0000", 2024-09-14T22:36:32.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.678 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454635+0000", 2024-09-14T22:36:32.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9400000000000005e-07 2024-09-14T22:36:32.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.678 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454643+0000", 2024-09-14T22:36:32.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.755e-06 2024-09-14T22:36:32.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.678 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454650+0000", 2024-09-14T22:36:32.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8340000000000001e-06 2024-09-14T22:36:32.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454667+0000", 2024-09-14T22:36:32.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7297999999999999e-05 2024-09-14T22:36:32.679 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.679 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.679 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.679 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20085 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.679 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.461806+0000", 2024-09-14T22:36:32.679 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0036089310000000001, 2024-09-14T22:36:32.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0072528230000000003, 2024-09-14T22:36:32.680 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.680 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.680 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.680 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.680 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.680 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.680 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20085 2024-09-14T22:36:32.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.680 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461806+0000", 2024-09-14T22:36:32.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461806+0000", 2024-09-14T22:36:32.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461807+0000", 2024-09-14T22:36:32.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3399999999999998e-07 2024-09-14T22:36:32.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461809+0000", 2024-09-14T22:36:32.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4030000000000001e-06 2024-09-14T22:36:32.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461816+0000", 2024-09-14T22:36:32.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6719999999999998e-06 2024-09-14T22:36:32.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461821+0000", 2024-09-14T22:36:32.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9479999999999999e-06 2024-09-14T22:36:32.682 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.682 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.682 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.683 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19588 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.683 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446349+0000", 2024-09-14T22:36:32.683 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.019065822999999999, 2024-09-14T22:36:32.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022768503999999998, 2024-09-14T22:36:32.683 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.683 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.683 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.683 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.683 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.683 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.683 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19588 2024-09-14T22:36:32.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.683 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446349+0000", 2024-09-14T22:36:32.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446349+0000", 2024-09-14T22:36:32.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446350+0000", 2024-09-14T22:36:32.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.96e-07 2024-09-14T22:36:32.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446357+0000", 2024-09-14T22:36:32.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1019999999999999e-06 2024-09-14T22:36:32.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446363+0000", 2024-09-14T22:36:32.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1709999999999999e-06 2024-09-14T22:36:32.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446368+0000", 2024-09-14T22:36:32.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6859999999999997e-06 2024-09-14T22:36:32.686 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.686 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.686 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.686 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19821 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.686 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.454675+0000", 2024-09-14T22:36:32.686 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.010740536, 2024-09-14T22:36:32.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014504991, 2024-09-14T22:36:32.686 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.686 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.686 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.686 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.686 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.686 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.687 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19821 2024-09-14T22:36:32.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.687 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454675+0000", 2024-09-14T22:36:32.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454675+0000", 2024-09-14T22:36:32.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454675+0000", 2024-09-14T22:36:32.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2900000000000004e-07 2024-09-14T22:36:32.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454678+0000", 2024-09-14T22:36:32.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3989999999999998e-06 2024-09-14T22:36:32.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454684+0000", 2024-09-14T22:36:32.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4030000000000004e-06 2024-09-14T22:36:32.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454691+0000", 2024-09-14T22:36:32.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8349999999999998e-06 2024-09-14T22:36:32.689 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.689 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.689 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.689 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20086 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.689 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.461827+0000", 2024-09-14T22:36:32.689 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.003587926, 2024-09-14T22:36:32.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.007411188, 2024-09-14T22:36:32.689 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.689 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.690 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.690 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.690 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.690 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.690 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20086 2024-09-14T22:36:32.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.690 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461827+0000", 2024-09-14T22:36:32.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461827+0000", 2024-09-14T22:36:32.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461828+0000", 2024-09-14T22:36:32.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1899999999999998e-07 2024-09-14T22:36:32.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461830+0000", 2024-09-14T22:36:32.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.046e-06 2024-09-14T22:36:32.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.692 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.692 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461836+0000", 2024-09-14T22:36:32.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9240000000000004e-06 2024-09-14T22:36:32.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.692 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.692 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461840+0000", 2024-09-14T22:36:32.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5029999999999999e-06 2024-09-14T22:36:32.692 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.692 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.692 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.693 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19589 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.693 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446374+0000", 2024-09-14T22:36:32.693 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.019041572999999999, 2024-09-14T22:36:32.693 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022926044, 2024-09-14T22:36:32.693 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.693 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.693 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.693 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.693 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.693 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.693 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19589 2024-09-14T22:36:32.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.693 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.694 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446374+0000", 2024-09-14T22:36:32.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.694 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446374+0000", 2024-09-14T22:36:32.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.694 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446374+0000", 2024-09-14T22:36:32.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0600000000000001e-07 2024-09-14T22:36:32.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.695 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.695 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.695 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446376+0000", 2024-09-14T22:36:32.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.792e-06 2024-09-14T22:36:32.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.695 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.695 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.695 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446381+0000", 2024-09-14T22:36:32.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5389999999999998e-06 2024-09-14T22:36:32.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.695 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.695 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.695 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446386+0000", 2024-09-14T22:36:32.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5159999999999996e-06 2024-09-14T22:36:32.695 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.696 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.696 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.696 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.696 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19822 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.696 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.454698+0000", 2024-09-14T22:36:32.696 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.010717725000000001, 2024-09-14T22:36:32.696 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014660240999999999, 2024-09-14T22:36:32.696 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.696 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.696 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.696 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.696 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.696 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.696 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19822 2024-09-14T22:36:32.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.697 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454698+0000", 2024-09-14T22:36:32.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454698+0000", 2024-09-14T22:36:32.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454698+0000", 2024-09-14T22:36:32.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0399999999999996e-07 2024-09-14T22:36:32.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454708+0000", 2024-09-14T22:36:32.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8269999999999993e-06 2024-09-14T22:36:32.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454715+0000", 2024-09-14T22:36:32.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6830000000000002e-06 2024-09-14T22:36:32.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454720+0000", 2024-09-14T22:36:32.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0499999999999999e-06 2024-09-14T22:36:32.699 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.699 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.699 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.699 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20087 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.699 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.461846+0000", 2024-09-14T22:36:32.699 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0035695869999999999, 2024-09-14T22:36:32.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0075737139999999996, 2024-09-14T22:36:32.699 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.699 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.700 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.700 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.700 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.700 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.700 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20087 2024-09-14T22:36:32.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.700 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.700 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.700 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461846+0000", 2024-09-14T22:36:32.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.700 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461846+0000", 2024-09-14T22:36:32.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461846+0000", 2024-09-14T22:36:32.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9099999999999999e-07 2024-09-14T22:36:32.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461848+0000", 2024-09-14T22:36:32.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8160000000000001e-06 2024-09-14T22:36:32.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461854+0000", 2024-09-14T22:36:32.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8089999999999998e-06 2024-09-14T22:36:32.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461858+0000", 2024-09-14T22:36:32.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.532e-06 2024-09-14T22:36:32.702 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.702 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.702 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.702 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19590 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.703 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446391+0000", 2024-09-14T22:36:32.703 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.019023900999999999, 2024-09-14T22:36:32.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023085853, 2024-09-14T22:36:32.703 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.703 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.703 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.703 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.703 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.703 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.703 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19590 2024-09-14T22:36:32.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.703 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446391+0000", 2024-09-14T22:36:32.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446391+0000", 2024-09-14T22:36:32.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446392+0000", 2024-09-14T22:36:32.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4299999999999999e-07 2024-09-14T22:36:32.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446393+0000", 2024-09-14T22:36:32.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8059999999999999e-06 2024-09-14T22:36:32.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446399+0000", 2024-09-14T22:36:32.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6740000000000002e-06 2024-09-14T22:36:32.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446404+0000", 2024-09-14T22:36:32.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6240000000000001e-06 2024-09-14T22:36:32.706 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.706 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.706 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.706 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19823 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.706 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.454727+0000", 2024-09-14T22:36:32.706 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.010688028, 2024-09-14T22:36:32.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014904107999999999, 2024-09-14T22:36:32.706 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.706 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.706 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.706 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.706 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.707 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.707 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19823 2024-09-14T22:36:32.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.707 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454727+0000", 2024-09-14T22:36:32.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454727+0000", 2024-09-14T22:36:32.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454728+0000", 2024-09-14T22:36:32.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8099999999999998e-07 2024-09-14T22:36:32.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454730+0000", 2024-09-14T22:36:32.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1689999999999999e-06 2024-09-14T22:36:32.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454737+0000", 2024-09-14T22:36:32.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0890000000000002e-06 2024-09-14T22:36:32.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454742+0000", 2024-09-14T22:36:32.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.865e-06 2024-09-14T22:36:32.709 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.709 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.709 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.709 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20088 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.709 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.461864+0000", 2024-09-14T22:36:32.709 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0035516459999999999, 2024-09-14T22:36:32.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0078287099999999991, 2024-09-14T22:36:32.710 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.710 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.710 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.710 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.710 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.710 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.710 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20088 2024-09-14T22:36:32.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.710 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461864+0000", 2024-09-14T22:36:32.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461864+0000", 2024-09-14T22:36:32.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461864+0000", 2024-09-14T22:36:32.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2000000000000001e-07 2024-09-14T22:36:32.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461866+0000", 2024-09-14T22:36:32.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.81e-06 2024-09-14T22:36:32.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461872+0000", 2024-09-14T22:36:32.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8270000000000003e-06 2024-09-14T22:36:32.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461876+0000", 2024-09-14T22:36:32.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5680000000000001e-06 2024-09-14T22:36:32.712 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.712 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.712 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.713 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19591 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.713 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446415+0000", 2024-09-14T22:36:32.713 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.019000547, 2024-09-14T22:36:32.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023334852, 2024-09-14T22:36:32.713 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.713 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.713 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.713 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.713 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.713 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.713 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19591 2024-09-14T22:36:32.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.714 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446415+0000", 2024-09-14T22:36:32.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446415+0000", 2024-09-14T22:36:32.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446415+0000", 2024-09-14T22:36:32.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8800000000000002e-07 2024-09-14T22:36:32.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446417+0000", 2024-09-14T22:36:32.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7990000000000001e-06 2024-09-14T22:36:32.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446423+0000", 2024-09-14T22:36:32.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.066e-06 2024-09-14T22:36:32.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446428+0000", 2024-09-14T22:36:32.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5889999999999996e-06 2024-09-14T22:36:32.716 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.716 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.716 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.716 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19824 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.716 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.454748+0000", 2024-09-14T22:36:32.716 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.010667243, 2024-09-14T22:36:32.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015063429, 2024-09-14T22:36:32.716 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.716 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.716 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.716 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.717 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.717 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.717 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19824 2024-09-14T22:36:32.717 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.717 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.717 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.717 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.717 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454748+0000", 2024-09-14T22:36:32.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.717 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.717 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.717 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.717 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454748+0000", 2024-09-14T22:36:32.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454748+0000", 2024-09-14T22:36:32.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0299999999999999e-07 2024-09-14T22:36:32.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454751+0000", 2024-09-14T22:36:32.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0090000000000002e-06 2024-09-14T22:36:32.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454756+0000", 2024-09-14T22:36:32.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9270000000000002e-06 2024-09-14T22:36:32.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454763+0000", 2024-09-14T22:36:32.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2890000000000003e-06 2024-09-14T22:36:32.719 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.719 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.719 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.719 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20089 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.719 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.461882+0000", 2024-09-14T22:36:32.719 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.003533399, 2024-09-14T22:36:32.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0079873219999999998, 2024-09-14T22:36:32.720 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.720 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.720 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.720 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.720 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.720 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.720 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20089 2024-09-14T22:36:32.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.720 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461882+0000", 2024-09-14T22:36:32.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.721 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.721 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461882+0000", 2024-09-14T22:36:32.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.721 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.721 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461882+0000", 2024-09-14T22:36:32.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0600000000000001e-07 2024-09-14T22:36:32.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.722 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.722 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461890+0000", 2024-09-14T22:36:32.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6320000000000008e-06 2024-09-14T22:36:32.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.722 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.722 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461896+0000", 2024-09-14T22:36:32.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1460000000000001e-06 2024-09-14T22:36:32.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.722 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.722 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461901+0000", 2024-09-14T22:36:32.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0250000000000002e-06 2024-09-14T22:36:32.722 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.723 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.723 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.723 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19592 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.723 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446433+0000", 2024-09-14T22:36:32.723 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.018981886, 2024-09-14T22:36:32.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023496125, 2024-09-14T22:36:32.723 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.723 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.723 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.723 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.723 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.723 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.724 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19592 2024-09-14T22:36:32.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.724 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446433+0000", 2024-09-14T22:36:32.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446433+0000", 2024-09-14T22:36:32.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446434+0000", 2024-09-14T22:36:32.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3600000000000004e-07 2024-09-14T22:36:32.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446436+0000", 2024-09-14T22:36:32.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.796e-06 2024-09-14T22:36:32.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446441+0000", 2024-09-14T22:36:32.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7209999999999999e-06 2024-09-14T22:36:32.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.726 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.726 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446450+0000", 2024-09-14T22:36:32.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0780000000000002e-06 2024-09-14T22:36:32.726 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.726 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.726 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.726 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19825 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.726 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.454768+0000", 2024-09-14T22:36:32.726 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.01064696, 2024-09-14T22:36:32.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015216185, 2024-09-14T22:36:32.727 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.727 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.727 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.727 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.727 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.727 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.727 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19825 2024-09-14T22:36:32.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.727 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454768+0000", 2024-09-14T22:36:32.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.728 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454768+0000", 2024-09-14T22:36:32.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.728 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454769+0000", 2024-09-14T22:36:32.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3299999999999998e-07 2024-09-14T22:36:32.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.728 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454770+0000", 2024-09-14T22:36:32.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.854e-06 2024-09-14T22:36:32.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454777+0000", 2024-09-14T22:36:32.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4869999999999999e-06 2024-09-14T22:36:32.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454782+0000", 2024-09-14T22:36:32.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1959999999999999e-06 2024-09-14T22:36:32.729 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.729 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.729 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.730 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20090 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.730 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.461908+0000", 2024-09-14T22:36:32.730 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.003507198, 2024-09-14T22:36:32.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0081417580000000007, 2024-09-14T22:36:32.730 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.730 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.730 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.730 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.730 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.730 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.730 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20090 2024-09-14T22:36:32.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.730 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461908+0000", 2024-09-14T22:36:32.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461908+0000", 2024-09-14T22:36:32.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461908+0000", 2024-09-14T22:36:32.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9299999999999999e-07 2024-09-14T22:36:32.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461911+0000", 2024-09-14T22:36:32.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.441e-06 2024-09-14T22:36:32.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461917+0000", 2024-09-14T22:36:32.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.268e-06 2024-09-14T22:36:32.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461922+0000", 2024-09-14T22:36:32.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7360000000000001e-06 2024-09-14T22:36:32.733 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.733 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.733 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.733 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19826 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.733 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.454788+0000", 2024-09-14T22:36:32.733 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0106268, 2024-09-14T22:36:32.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015323148, 2024-09-14T22:36:32.733 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.733 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.733 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.734 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.734 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.734 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.734 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19826 2024-09-14T22:36:32.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.734 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454788+0000", 2024-09-14T22:36:32.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454788+0000", 2024-09-14T22:36:32.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454789+0000", 2024-09-14T22:36:32.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4499999999999997e-07 2024-09-14T22:36:32.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454791+0000", 2024-09-14T22:36:32.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1239999999999999e-06 2024-09-14T22:36:32.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454797+0000", 2024-09-14T22:36:32.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9859999999999999e-06 2024-09-14T22:36:32.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454802+0000", 2024-09-14T22:36:32.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7140000000000001e-06 2024-09-14T22:36:32.736 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.736 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.736 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.736 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20091 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.737 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.461928+0000", 2024-09-14T22:36:32.737 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0034872309999999999, 2024-09-14T22:36:32.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0082484029999999996, 2024-09-14T22:36:32.737 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.737 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.737 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.737 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.737 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.737 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.737 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20091 2024-09-14T22:36:32.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.737 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461928+0000", 2024-09-14T22:36:32.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461928+0000", 2024-09-14T22:36:32.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461928+0000", 2024-09-14T22:36:32.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0499999999999999e-07 2024-09-14T22:36:32.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461930+0000", 2024-09-14T22:36:32.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8789999999999999e-06 2024-09-14T22:36:32.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461936+0000", 2024-09-14T22:36:32.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8719999999999999e-06 2024-09-14T22:36:32.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461946+0000", 2024-09-14T22:36:32.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0277e-05 2024-09-14T22:36:32.739 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.740 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.740 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.740 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19827 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.740 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.454807+0000", 2024-09-14T22:36:32.740 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.010607904, 2024-09-14T22:36:32.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015427349999999999, 2024-09-14T22:36:32.740 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.740 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.740 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.740 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.740 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.740 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.741 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19827 2024-09-14T22:36:32.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.741 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454807+0000", 2024-09-14T22:36:32.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454807+0000", 2024-09-14T22:36:32.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454808+0000", 2024-09-14T22:36:32.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3300000000000002e-07 2024-09-14T22:36:32.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454810+0000", 2024-09-14T22:36:32.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9e-06 2024-09-14T22:36:32.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454816+0000", 2024-09-14T22:36:32.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4439999999999996e-06 2024-09-14T22:36:32.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.743 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454821+0000", 2024-09-14T22:36:32.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9599999999999999e-06 2024-09-14T22:36:32.743 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.743 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.743 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.743 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20092 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.743 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.461953+0000", 2024-09-14T22:36:32.743 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0034623879999999998, 2024-09-14T22:36:32.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.008344031, 2024-09-14T22:36:32.744 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.744 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.744 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.744 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.744 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.744 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.744 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20092 2024-09-14T22:36:32.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.744 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.744 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.744 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461953+0000", 2024-09-14T22:36:32.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461953+0000", 2024-09-14T22:36:32.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461953+0000", 2024-09-14T22:36:32.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.58e-07 2024-09-14T22:36:32.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461956+0000", 2024-09-14T22:36:32.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2840000000000001e-06 2024-09-14T22:36:32.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461962+0000", 2024-09-14T22:36:32.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9e-06 2024-09-14T22:36:32.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461967+0000", 2024-09-14T22:36:32.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7589999999999997e-06 2024-09-14T22:36:32.746 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.746 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.746 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.747 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19828 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.747 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.454829+0000", 2024-09-14T22:36:32.747 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.010586665, 2024-09-14T22:36:32.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015526386, 2024-09-14T22:36:32.747 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.747 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.747 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.747 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.747 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.747 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.747 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19828 2024-09-14T22:36:32.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.748 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454829+0000", 2024-09-14T22:36:32.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454829+0000", 2024-09-14T22:36:32.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454829+0000", 2024-09-14T22:36:32.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.03e-07 2024-09-14T22:36:32.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454831+0000", 2024-09-14T22:36:32.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.12e-06 2024-09-14T22:36:32.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454837+0000", 2024-09-14T22:36:32.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.976e-06 2024-09-14T22:36:32.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454842+0000", 2024-09-14T22:36:32.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4859999999999999e-06 2024-09-14T22:36:32.750 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.750 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.750 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.750 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20093 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.750 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.461973+0000", 2024-09-14T22:36:32.750 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0034418029999999998, 2024-09-14T22:36:32.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0084436809999999998, 2024-09-14T22:36:32.751 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.751 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.751 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.751 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.751 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.751 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.751 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20093 2024-09-14T22:36:32.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.751 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461973+0000", 2024-09-14T22:36:32.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461973+0000", 2024-09-14T22:36:32.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461974+0000", 2024-09-14T22:36:32.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7300000000000002e-07 2024-09-14T22:36:32.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461976+0000", 2024-09-14T22:36:32.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.091e-06 2024-09-14T22:36:32.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.753 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.753 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461982+0000", 2024-09-14T22:36:32.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3219999999999998e-06 2024-09-14T22:36:32.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.753 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.753 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461998+0000", 2024-09-14T22:36:32.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5290000000000001e-05 2024-09-14T22:36:32.753 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.754 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.754 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.754 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19829 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.754 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.454848+0000", 2024-09-14T22:36:32.754 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.010567557999999999, 2024-09-14T22:36:32.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015628825999999998, 2024-09-14T22:36:32.754 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.754 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.754 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.754 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.754 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.754 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.755 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19829 2024-09-14T22:36:32.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.755 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454848+0000", 2024-09-14T22:36:32.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454848+0000", 2024-09-14T22:36:32.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454848+0000", 2024-09-14T22:36:32.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1699999999999999e-07 2024-09-14T22:36:32.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454850+0000", 2024-09-14T22:36:32.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9479999999999998e-06 2024-09-14T22:36:32.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454856+0000", 2024-09-14T22:36:32.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1689999999999997e-06 2024-09-14T22:36:32.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454867+0000", 2024-09-14T22:36:32.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0875e-05 2024-09-14T22:36:32.757 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.757 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.757 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.758 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20094 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.758 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462005+0000", 2024-09-14T22:36:32.758 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0034105920000000001, 2024-09-14T22:36:32.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0085343880000000004, 2024-09-14T22:36:32.758 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.758 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.758 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.758 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.758 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.758 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.758 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20094 2024-09-14T22:36:32.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.758 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462005+0000", 2024-09-14T22:36:32.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462005+0000", 2024-09-14T22:36:32.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462005+0000", 2024-09-14T22:36:32.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4099999999999998e-07 2024-09-14T22:36:32.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462016+0000", 2024-09-14T22:36:32.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0516999999999999e-05 2024-09-14T22:36:32.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462023+0000", 2024-09-14T22:36:32.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1670000000000002e-06 2024-09-14T22:36:32.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462053+0000", 2024-09-14T22:36:32.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0267999999999999e-05 2024-09-14T22:36:32.761 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.761 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.761 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.761 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19830 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.761 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.454888+0000", 2024-09-14T22:36:32.761 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.010527503000000001, 2024-09-14T22:36:32.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015710637, 2024-09-14T22:36:32.761 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.761 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.762 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.762 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.762 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.762 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.762 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19830 2024-09-14T22:36:32.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.762 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454888+0000", 2024-09-14T22:36:32.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454888+0000", 2024-09-14T22:36:32.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454888+0000", 2024-09-14T22:36:32.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2900000000000003e-07 2024-09-14T22:36:32.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454895+0000", 2024-09-14T22:36:32.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4960000000000001e-06 2024-09-14T22:36:32.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.764 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.764 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454902+0000", 2024-09-14T22:36:32.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.126e-06 2024-09-14T22:36:32.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.764 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.764 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454909+0000", 2024-09-14T22:36:32.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.754e-06 2024-09-14T22:36:32.764 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.764 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.764 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.764 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20095 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.765 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462063+0000", 2024-09-14T22:36:32.765 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.003352416, 2024-09-14T22:36:32.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0085935370000000001, 2024-09-14T22:36:32.765 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.765 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.765 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.765 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.765 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.765 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.765 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20095 2024-09-14T22:36:32.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.765 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462063+0000", 2024-09-14T22:36:32.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462063+0000", 2024-09-14T22:36:32.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462064+0000", 2024-09-14T22:36:32.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4300000000000002e-07 2024-09-14T22:36:32.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462066+0000", 2024-09-14T22:36:32.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7810000000000001e-06 2024-09-14T22:36:32.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462074+0000", 2024-09-14T22:36:32.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3289999999999996e-06 2024-09-14T22:36:32.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462079+0000", 2024-09-14T22:36:32.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4909999999999996e-06 2024-09-14T22:36:32.768 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.768 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.768 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.768 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19831 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.768 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.454916+0000", 2024-09-14T22:36:32.768 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.010498845999999999, 2024-09-14T22:36:32.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015801808000000001, 2024-09-14T22:36:32.768 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.768 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.768 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.769 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.769 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.769 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.769 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19831 2024-09-14T22:36:32.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.769 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454916+0000", 2024-09-14T22:36:32.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454916+0000", 2024-09-14T22:36:32.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454917+0000", 2024-09-14T22:36:32.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4999999999999998e-07 2024-09-14T22:36:32.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454921+0000", 2024-09-14T22:36:32.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8789999999999996e-06 2024-09-14T22:36:32.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454927+0000", 2024-09-14T22:36:32.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5350000000000001e-06 2024-09-14T22:36:32.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454933+0000", 2024-09-14T22:36:32.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0580000000000003e-06 2024-09-14T22:36:32.771 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.771 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.771 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.771 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20096 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.772 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462086+0000", 2024-09-14T22:36:32.772 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.003329186, 2024-09-14T22:36:32.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0086909150000000004, 2024-09-14T22:36:32.772 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.772 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.772 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.772 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.772 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.772 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.772 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20096 2024-09-14T22:36:32.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.772 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462086+0000", 2024-09-14T22:36:32.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462086+0000", 2024-09-14T22:36:32.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462086+0000", 2024-09-14T22:36:32.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8000000000000001e-07 2024-09-14T22:36:32.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462090+0000", 2024-09-14T22:36:32.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1839999999999999e-06 2024-09-14T22:36:32.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462096+0000", 2024-09-14T22:36:32.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5459999999999997e-06 2024-09-14T22:36:32.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462101+0000", 2024-09-14T22:36:32.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.977e-06 2024-09-14T22:36:32.775 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.775 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.775 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.775 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19832 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.775 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.454940+0000", 2024-09-14T22:36:32.775 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.010475384000000001, 2024-09-14T22:36:32.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015898058999999999, 2024-09-14T22:36:32.775 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.775 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.775 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.775 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.776 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.776 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.776 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19832 2024-09-14T22:36:32.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.776 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.776 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454940+0000", 2024-09-14T22:36:32.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.776 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454940+0000", 2024-09-14T22:36:32.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454940+0000", 2024-09-14T22:36:32.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2099999999999997e-07 2024-09-14T22:36:32.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454943+0000", 2024-09-14T22:36:32.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3479999999999998e-06 2024-09-14T22:36:32.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454949+0000", 2024-09-14T22:36:32.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.533e-06 2024-09-14T22:36:32.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454954+0000", 2024-09-14T22:36:32.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0429999999999998e-06 2024-09-14T22:36:32.778 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.778 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.778 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.778 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20097 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.778 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462107+0000", 2024-09-14T22:36:32.778 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0033080060000000001, 2024-09-14T22:36:32.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0087892019999999994, 2024-09-14T22:36:32.779 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.779 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.779 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.779 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.779 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.779 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.779 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20097 2024-09-14T22:36:32.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.779 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462107+0000", 2024-09-14T22:36:32.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462107+0000", 2024-09-14T22:36:32.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462108+0000", 2024-09-14T22:36:32.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6399999999999998e-07 2024-09-14T22:36:32.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462110+0000", 2024-09-14T22:36:32.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0229999999999999e-06 2024-09-14T22:36:32.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462116+0000", 2024-09-14T22:36:32.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4690000000000002e-06 2024-09-14T22:36:32.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462121+0000", 2024-09-14T22:36:32.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9089999999999999e-06 2024-09-14T22:36:32.781 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.782 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.782 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.782 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19833 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.782 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.454960+0000", 2024-09-14T22:36:32.782 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.010454985999999999, 2024-09-14T22:36:32.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.016000298999999999, 2024-09-14T22:36:32.782 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.782 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.782 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.782 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.782 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.783 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.783 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19833 2024-09-14T22:36:32.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.783 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454960+0000", 2024-09-14T22:36:32.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454960+0000", 2024-09-14T22:36:32.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454961+0000", 2024-09-14T22:36:32.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8000000000000001e-07 2024-09-14T22:36:32.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454969+0000", 2024-09-14T22:36:32.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5269999999999998e-06 2024-09-14T22:36:32.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454976+0000", 2024-09-14T22:36:32.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9430000000000003e-06 2024-09-14T22:36:32.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454981+0000", 2024-09-14T22:36:32.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.203e-06 2024-09-14T22:36:32.785 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.785 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.785 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.785 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20098 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.785 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462127+0000", 2024-09-14T22:36:32.786 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0032878579999999998, 2024-09-14T22:36:32.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0088925649999999998, 2024-09-14T22:36:32.786 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.786 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.786 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.786 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.786 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.786 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.786 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20098 2024-09-14T22:36:32.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.786 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462127+0000", 2024-09-14T22:36:32.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462127+0000", 2024-09-14T22:36:32.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462128+0000", 2024-09-14T22:36:32.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1699999999999999e-07 2024-09-14T22:36:32.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462130+0000", 2024-09-14T22:36:32.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9230000000000001e-06 2024-09-14T22:36:32.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462136+0000", 2024-09-14T22:36:32.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1299999999999998e-06 2024-09-14T22:36:32.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462141+0000", 2024-09-14T22:36:32.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8199999999999996e-06 2024-09-14T22:36:32.788 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.788 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.788 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.789 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19834 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.789 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.454989+0000", 2024-09-14T22:36:32.789 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.010426695, 2024-09-14T22:36:32.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.016091095999999999, 2024-09-14T22:36:32.789 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.789 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.789 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.789 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.789 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.789 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.789 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19834 2024-09-14T22:36:32.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.790 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454989+0000", 2024-09-14T22:36:32.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454989+0000", 2024-09-14T22:36:32.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454989+0000", 2024-09-14T22:36:32.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.51e-07 2024-09-14T22:36:32.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454991+0000", 2024-09-14T22:36:32.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3259999999999998e-06 2024-09-14T22:36:32.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.454999+0000", 2024-09-14T22:36:32.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2849999999999997e-06 2024-09-14T22:36:32.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455004+0000", 2024-09-14T22:36:32.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0000000000000004e-06 2024-09-14T22:36:32.792 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.792 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.792 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.792 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20099 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.792 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462148+0000", 2024-09-14T22:36:32.792 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0032674890000000002, 2024-09-14T22:36:32.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0089908820000000004, 2024-09-14T22:36:32.792 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.793 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.793 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.793 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.793 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.793 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.793 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20099 2024-09-14T22:36:32.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.793 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462148+0000", 2024-09-14T22:36:32.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462148+0000", 2024-09-14T22:36:32.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462148+0000", 2024-09-14T22:36:32.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6399999999999998e-07 2024-09-14T22:36:32.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462150+0000", 2024-09-14T22:36:32.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.922e-06 2024-09-14T22:36:32.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462156+0000", 2024-09-14T22:36:32.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9889999999999997e-06 2024-09-14T22:36:32.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462161+0000", 2024-09-14T22:36:32.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.763e-06 2024-09-14T22:36:32.795 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.795 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.796 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.796 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19835 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.796 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.455010+0000", 2024-09-14T22:36:32.796 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.010405388999999999, 2024-09-14T22:36:32.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.016187178, 2024-09-14T22:36:32.796 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.796 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.796 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.796 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.796 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.796 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.797 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19835 2024-09-14T22:36:32.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.797 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455010+0000", 2024-09-14T22:36:32.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455010+0000", 2024-09-14T22:36:32.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455010+0000", 2024-09-14T22:36:32.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7800000000000002e-07 2024-09-14T22:36:32.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455012+0000", 2024-09-14T22:36:32.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0420000000000001e-06 2024-09-14T22:36:32.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455019+0000", 2024-09-14T22:36:32.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5440000000000004e-06 2024-09-14T22:36:32.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.455024+0000", 2024-09-14T22:36:32.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7049999999999998e-06 2024-09-14T22:36:32.799 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.799 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.799 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.799 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20068 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.799 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.461396+0000", 2024-09-14T22:36:32.799 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0040191960000000001, 2024-09-14T22:36:32.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0098578850000000003, 2024-09-14T22:36:32.800 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.800 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.800 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.800 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.800 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.800 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.800 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20068 2024-09-14T22:36:32.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.800 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461396+0000", 2024-09-14T22:36:32.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461396+0000", 2024-09-14T22:36:32.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461397+0000", 2024-09-14T22:36:32.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3099999999999997e-07 2024-09-14T22:36:32.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461399+0000", 2024-09-14T22:36:32.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.481e-06 2024-09-14T22:36:32.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461406+0000", 2024-09-14T22:36:32.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2259999999999999e-06 2024-09-14T22:36:32.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.461412+0000", 2024-09-14T22:36:32.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3820000000000003e-06 2024-09-14T22:36:32.802 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.802 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.802 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.803 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20100 3.c 3.b416eaac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:32.803 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462167+0000", 2024-09-14T22:36:32.803 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0032481630000000001, 2024-09-14T22:36:32.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0091493210000000002, 2024-09-14T22:36:32.803 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:32.803 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:32.803 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:36:32.803 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:32.803 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:32.803 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:32.803 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20100 2024-09-14T22:36:32.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.804 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:32.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:32.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462167+0000", 2024-09-14T22:36:32.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:32.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462167+0000", 2024-09-14T22:36:32.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:32.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:32.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462168+0000", 2024-09-14T22:36:32.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8800000000000002e-07 2024-09-14T22:36:32.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:32.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462177+0000", 2024-09-14T22:36:32.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9330000000000007e-06 2024-09-14T22:36:32.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:32.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462183+0000", 2024-09-14T22:36:32.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6200000000000001e-06 2024-09-14T22:36:32.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:32.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:32.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:32.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462188+0000", 2024-09-14T22:36:32.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1590000000000001e-06 2024-09-14T22:36:32.806 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.806 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:32.806 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.806 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:32.806 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T22:36:32.806 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 97 2024-09-14T22:36:32.806 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:32.808 INFO:teuthology.orchestra.run.smithi001.stdout:737 write 0xac1aca5 thru 0xac24250 (0x95ac bytes) 2024-09-14T22:36:32.808 INFO:teuthology.orchestra.run.smithi001.stdout:738 write 0x7a8b5fe thru 0x7a8da92 (0x2495 bytes) 2024-09-14T22:36:32.808 INFO:teuthology.orchestra.run.smithi001.stdout:739 punch from 0x656dc8b to 0x6570671, (0x29e6 bytes) 2024-09-14T22:36:32.808 INFO:teuthology.orchestra.run.smithi001.stdout:741 write 0x38424f4 thru 0x384b08b (0x8b98 bytes) 2024-09-14T22:36:32.809 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:36:32.809 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:32.803+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:32.810 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.4.2 2024-09-14T22:36:32.810 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2024-09-14T22:36:32.838 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:32.846 INFO:teuthology.orchestra.run.smithi001.stdout:743 write 0xc3211fc thru 0xc329034 (0x7e39 bytes) 2024-09-14T22:36:32.860 INFO:teuthology.orchestra.run.smithi001.stdout:744 write 0x4d2a9f4 thru 0x4d3a415 (0xfa22 bytes) 2024-09-14T22:36:32.865 INFO:teuthology.orchestra.run.smithi001.stdout:745 write 0x880fbb3 thru 0x881dc25 (0xe073 bytes) 2024-09-14T22:36:32.880 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:32.880 INFO:teuthology.orchestra.run.smithi001.stdout:747 read 0x71fe8c5 thru 0x720e037 (0xf773 bytes) 2024-09-14T22:36:32.896 INFO:teuthology.orchestra.run.smithi001.stdout:748 trunc from 0xc329035 to 0xe60e41 2024-09-14T22:36:32.908 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:32.902+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:33.009 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:33.003+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:33.025 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:33.025 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:33.025 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:36:33.025 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:36:33.025 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:33.042 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:36:33.092 INFO:teuthology.orchestra.run.smithi001.stdout:749 read 0xd9391a thru 0xd9a5e0 (0x6cc7 bytes) 2024-09-14T22:36:33.095 INFO:teuthology.orchestra.run.smithi001.stdout:750 read 0x70ace2 thru 0x715ab2 (0xadd1 bytes) 2024-09-14T22:36:33.099 INFO:teuthology.orchestra.run.smithi001.stdout:751 read 0x34eb60 thru 0x3509d7 (0x1e78 bytes) 2024-09-14T22:36:33.108 INFO:teuthology.orchestra.run.smithi001.stdout:753 write 0x1e4e2ff thru 0x1e4ebd2 (0x8d4 bytes) 2024-09-14T22:36:33.112 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:33.112 INFO:teuthology.orchestra.run.smithi001.stdout:755 read 0x19a517c thru 0x19a95cc (0x4451 bytes) 2024-09-14T22:36:33.132 INFO:teuthology.orchestra.run.smithi001.stdout:756 writesame 0x898f1b9 thru 0x898f7da data_size 0x5(0x622 bytes) 2024-09-14T22:36:33.135 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:33.131+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:33.138 INFO:teuthology.orchestra.run.smithi001.stdout:757 write 0x8d2b8a4 thru 0x8d3896b (0xd0c8 bytes) 2024-09-14T22:36:33.161 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:33.161 INFO:teuthology.orchestra.run.smithi001.stdout:759 write 0x577b028 thru 0x57819e4 (0x69bd bytes) 2024-09-14T22:36:33.181 INFO:teuthology.orchestra.run.smithi001.stdout:760 read 0x56d60d7 thru 0x56e251f (0xc449 bytes) 2024-09-14T22:36:33.190 INFO:teuthology.orchestra.run.smithi001.stdout:763 trunc from 0x8d3896c to 0x3dbbb6f 2024-09-14T22:36:33.208 INFO:teuthology.orchestra.run.smithi001.stdout:764 write 0x27ba4a8 thru 0x27c6e1d (0xc976 bytes) 2024-09-14T22:36:33.210 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:33.203+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:33.216 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:33.216 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:36:33.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:36:33.216 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:36:33.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.216 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19077 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7211008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:33.216 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.443038+0000", 2024-09-14T22:36:33.216 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.76894646300000002, 2024-09-14T22:36:33.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027007455, 2024-09-14T22:36:33.217 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:33.217 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:33.217 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:33.217 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:33.217 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:33.217 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:33.217 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19077 2024-09-14T22:36:33.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.217 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:33.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:33.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.443038+0000", 2024-09-14T22:36:33.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:33.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.443038+0000", 2024-09-14T22:36:33.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:33.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.443039+0000", 2024-09-14T22:36:33.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2899999999999999e-07 2024-09-14T22:36:33.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:33.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.443040+0000", 2024-09-14T22:36:33.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.437e-06 2024-09-14T22:36:33.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:33.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.443045+0000", 2024-09-14T22:36:33.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5290000000000002e-06 2024-09-14T22:36:33.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:33.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.443048+0000", 2024-09-14T22:36:33.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6679999999999999e-06 2024-09-14T22:36:33.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:33.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.459277+0000", 2024-09-14T22:36:33.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.016228455999999999 2024-09-14T22:36:33.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:33.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.459344+0000", 2024-09-14T22:36:33.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7348999999999995e-05 2024-09-14T22:36:33.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:33.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.470046+0000", 2024-09-14T22:36:33.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010701687 2024-09-14T22:36:33.220 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.221 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:33.221 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.221 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19589 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7243776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:33.221 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446374+0000", 2024-09-14T22:36:33.221 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.76561129100000003, 2024-09-14T22:36:33.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026980818, 2024-09-14T22:36:33.221 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:33.221 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:33.221 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:33.221 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:33.221 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:33.222 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:33.222 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19589 2024-09-14T22:36:33.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.222 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:33.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:33.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446374+0000", 2024-09-14T22:36:33.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:33.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446374+0000", 2024-09-14T22:36:33.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:33.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446374+0000", 2024-09-14T22:36:33.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0600000000000001e-07 2024-09-14T22:36:33.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:33.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446376+0000", 2024-09-14T22:36:33.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.792e-06 2024-09-14T22:36:33.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:33.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446381+0000", 2024-09-14T22:36:33.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5389999999999998e-06 2024-09-14T22:36:33.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:33.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446386+0000", 2024-09-14T22:36:33.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5159999999999996e-06 2024-09-14T22:36:33.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:33.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473225+0000", 2024-09-14T22:36:33.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026838951 2024-09-14T22:36:33.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:33.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473292+0000", 2024-09-14T22:36:33.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7086000000000001e-05 2024-09-14T22:36:33.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:33.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473354+0000", 2024-09-14T22:36:33.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2527999999999995e-05 2024-09-14T22:36:33.225 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.225 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:33.225 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.225 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19590 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7272448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:33.225 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446391+0000", 2024-09-14T22:36:33.225 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.765593619, 2024-09-14T22:36:33.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027116533000000002, 2024-09-14T22:36:33.226 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:33.226 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:33.226 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:33.226 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:33.226 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:33.226 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:33.226 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19590 2024-09-14T22:36:33.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.226 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:33.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:33.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446391+0000", 2024-09-14T22:36:33.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:33.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446391+0000", 2024-09-14T22:36:33.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:33.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446392+0000", 2024-09-14T22:36:33.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4299999999999999e-07 2024-09-14T22:36:33.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:33.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446393+0000", 2024-09-14T22:36:33.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8059999999999999e-06 2024-09-14T22:36:33.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:33.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446399+0000", 2024-09-14T22:36:33.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6740000000000002e-06 2024-09-14T22:36:33.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:33.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446404+0000", 2024-09-14T22:36:33.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6240000000000001e-06 2024-09-14T22:36:33.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:33.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473382+0000", 2024-09-14T22:36:33.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026978716999999999 2024-09-14T22:36:33.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:33.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473446+0000", 2024-09-14T22:36:33.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3971000000000001e-05 2024-09-14T22:36:33.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:33.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473508+0000", 2024-09-14T22:36:33.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1397999999999997e-05 2024-09-14T22:36:33.229 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.230 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:33.230 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.230 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19591 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7301120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:33.230 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446415+0000", 2024-09-14T22:36:33.230 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.76557026500000003, 2024-09-14T22:36:33.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027255773, 2024-09-14T22:36:33.230 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:33.230 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:33.230 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:33.230 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:33.231 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:33.231 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:33.231 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19591 2024-09-14T22:36:33.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.231 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:33.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:33.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446415+0000", 2024-09-14T22:36:33.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:33.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446415+0000", 2024-09-14T22:36:33.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:33.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446415+0000", 2024-09-14T22:36:33.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8800000000000002e-07 2024-09-14T22:36:33.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:33.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446417+0000", 2024-09-14T22:36:33.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7990000000000001e-06 2024-09-14T22:36:33.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:33.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446423+0000", 2024-09-14T22:36:33.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.066e-06 2024-09-14T22:36:33.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:33.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446428+0000", 2024-09-14T22:36:33.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5889999999999996e-06 2024-09-14T22:36:33.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:33.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473535+0000", 2024-09-14T22:36:33.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027107220000000001 2024-09-14T22:36:33.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:33.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473602+0000", 2024-09-14T22:36:33.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6482999999999993e-05 2024-09-14T22:36:33.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:33.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473670+0000", 2024-09-14T22:36:33.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8928000000000001e-05 2024-09-14T22:36:33.234 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.234 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:33.234 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.235 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19592 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7329792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:33.235 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446433+0000", 2024-09-14T22:36:33.235 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.765551604, 2024-09-14T22:36:33.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027394960999999999, 2024-09-14T22:36:33.235 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:33.235 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:33.235 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:33.235 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:33.235 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:33.235 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:33.235 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19592 2024-09-14T22:36:33.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.235 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:33.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:33.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446433+0000", 2024-09-14T22:36:33.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:33.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446433+0000", 2024-09-14T22:36:33.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:33.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446434+0000", 2024-09-14T22:36:33.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3600000000000004e-07 2024-09-14T22:36:33.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:33.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446436+0000", 2024-09-14T22:36:33.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.796e-06 2024-09-14T22:36:33.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:33.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446441+0000", 2024-09-14T22:36:33.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7209999999999999e-06 2024-09-14T22:36:33.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:33.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446450+0000", 2024-09-14T22:36:33.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0780000000000002e-06 2024-09-14T22:36:33.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:33.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473705+0000", 2024-09-14T22:36:33.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027254246999999999 2024-09-14T22:36:33.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:33.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473769+0000", 2024-09-14T22:36:33.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4101000000000002e-05 2024-09-14T22:36:33.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:33.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473828+0000", 2024-09-14T22:36:33.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9481999999999997e-05 2024-09-14T22:36:33.239 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.239 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:33.239 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.239 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20099 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7190528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:33.240 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462148+0000", 2024-09-14T22:36:33.240 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.74983720700000001, 2024-09-14T22:36:33.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026931119, 2024-09-14T22:36:33.240 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:33.240 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:33.240 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:33.240 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:33.240 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:33.240 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:33.240 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20099 2024-09-14T22:36:33.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.240 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:33.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:33.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462148+0000", 2024-09-14T22:36:33.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:33.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462148+0000", 2024-09-14T22:36:33.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:33.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462148+0000", 2024-09-14T22:36:33.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6399999999999998e-07 2024-09-14T22:36:33.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:33.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462150+0000", 2024-09-14T22:36:33.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.922e-06 2024-09-14T22:36:33.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:33.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462156+0000", 2024-09-14T22:36:33.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9889999999999997e-06 2024-09-14T22:36:33.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:33.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462161+0000", 2024-09-14T22:36:33.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.763e-06 2024-09-14T22:36:33.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:33.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.488945+0000", 2024-09-14T22:36:33.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026784136 2024-09-14T22:36:33.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:33.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489007+0000", 2024-09-14T22:36:33.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2564000000000006e-05 2024-09-14T22:36:33.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:33.244 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489079+0000", 2024-09-14T22:36:33.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1381e-05 2024-09-14T22:36:33.244 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.244 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:33.244 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.244 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20100 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7219200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:33.245 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462167+0000", 2024-09-14T22:36:33.245 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.74981788100000002, 2024-09-14T22:36:33.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027082189999999999, 2024-09-14T22:36:33.245 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:33.245 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:33.245 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:33.245 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:33.245 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:33.245 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:33.245 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20100 2024-09-14T22:36:33.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.245 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:33.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:33.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462167+0000", 2024-09-14T22:36:33.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:33.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462167+0000", 2024-09-14T22:36:33.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:33.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462168+0000", 2024-09-14T22:36:33.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8800000000000002e-07 2024-09-14T22:36:33.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:33.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462177+0000", 2024-09-14T22:36:33.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9330000000000007e-06 2024-09-14T22:36:33.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:33.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462183+0000", 2024-09-14T22:36:33.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6200000000000001e-06 2024-09-14T22:36:33.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:33.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462188+0000", 2024-09-14T22:36:33.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1590000000000001e-06 2024-09-14T22:36:33.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.248 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:33.248 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489109+0000", 2024-09-14T22:36:33.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026920881000000001 2024-09-14T22:36:33.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.248 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:33.248 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489181+0000", 2024-09-14T22:36:33.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2035999999999999e-05 2024-09-14T22:36:33.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.249 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:33.249 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489249+0000", 2024-09-14T22:36:33.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7972999999999994e-05 2024-09-14T22:36:33.249 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.249 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:33.249 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.249 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20101 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7247872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:33.249 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462195+0000", 2024-09-14T22:36:33.249 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.749789598, 2024-09-14T22:36:33.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027205994000000001, 2024-09-14T22:36:33.249 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:33.250 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:33.250 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:33.250 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:33.250 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:33.250 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:33.250 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20101 2024-09-14T22:36:33.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.250 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:33.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:33.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462195+0000", 2024-09-14T22:36:33.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:33.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462195+0000", 2024-09-14T22:36:33.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:33.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462196+0000", 2024-09-14T22:36:33.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8000000000000001e-07 2024-09-14T22:36:33.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:33.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462198+0000", 2024-09-14T22:36:33.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0439999999999998e-06 2024-09-14T22:36:33.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:33.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462205+0000", 2024-09-14T22:36:33.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3359999999999997e-06 2024-09-14T22:36:33.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:33.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462210+0000", 2024-09-14T22:36:33.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0649999999999998e-06 2024-09-14T22:36:33.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:33.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489279+0000", 2024-09-14T22:36:33.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027069123 2024-09-14T22:36:33.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:33.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489342+0000", 2024-09-14T22:36:33.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2759e-05 2024-09-14T22:36:33.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:33.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489401+0000", 2024-09-14T22:36:33.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9286999999999997e-05 2024-09-14T22:36:33.253 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.253 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:33.253 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.254 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20102 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7276544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:33.254 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462216+0000", 2024-09-14T22:36:33.254 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.74976865000000004, 2024-09-14T22:36:33.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027352785000000001, 2024-09-14T22:36:33.254 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:33.254 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:33.254 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:33.254 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:33.254 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:33.254 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:33.254 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20102 2024-09-14T22:36:33.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.255 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:33.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:33.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462216+0000", 2024-09-14T22:36:33.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:33.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462216+0000", 2024-09-14T22:36:33.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:33.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462217+0000", 2024-09-14T22:36:33.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9100000000000003e-07 2024-09-14T22:36:33.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:33.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462219+0000", 2024-09-14T22:36:33.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9740000000000001e-06 2024-09-14T22:36:33.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:33.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462235+0000", 2024-09-14T22:36:33.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6050999999999999e-05 2024-09-14T22:36:33.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:33.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462247+0000", 2024-09-14T22:36:33.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2245999999999999e-05 2024-09-14T22:36:33.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:33.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489431+0000", 2024-09-14T22:36:33.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027183624999999999 2024-09-14T22:36:33.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:33.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489504+0000", 2024-09-14T22:36:33.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3498999999999995e-05 2024-09-14T22:36:33.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:33.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489569+0000", 2024-09-14T22:36:33.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4698999999999998e-05 2024-09-14T22:36:33.258 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.258 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:33.258 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.258 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20103 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7305216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:33.258 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462256+0000", 2024-09-14T22:36:33.258 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.74972944399999997, 2024-09-14T22:36:33.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027473543, 2024-09-14T22:36:33.258 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:33.258 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:33.259 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:33.259 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:33.259 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:33.259 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:33.259 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20103 2024-09-14T22:36:33.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.259 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:33.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:33.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462256+0000", 2024-09-14T22:36:33.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:33.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462256+0000", 2024-09-14T22:36:33.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:33.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462256+0000", 2024-09-14T22:36:33.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1399999999999997e-07 2024-09-14T22:36:33.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:33.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462259+0000", 2024-09-14T22:36:33.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.694e-06 2024-09-14T22:36:33.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:33.261 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462266+0000", 2024-09-14T22:36:33.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8940000000000004e-06 2024-09-14T22:36:33.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:33.261 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462271+0000", 2024-09-14T22:36:33.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0459999999999996e-06 2024-09-14T22:36:33.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:33.261 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489601+0000", 2024-09-14T22:36:33.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027330027 2024-09-14T22:36:33.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:33.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489663+0000", 2024-09-14T22:36:33.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2807999999999999e-05 2024-09-14T22:36:33.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:33.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489729+0000", 2024-09-14T22:36:33.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5560000000000002e-05 2024-09-14T22:36:33.262 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.262 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:33.262 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.263 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20104 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7333888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:33.263 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462277+0000", 2024-09-14T22:36:33.263 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.74970790799999998, 2024-09-14T22:36:33.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027652614999999998, 2024-09-14T22:36:33.263 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:33.263 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:33.263 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:33.263 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:33.263 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:33.263 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:33.263 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20104 2024-09-14T22:36:33.263 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.263 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:33.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:33.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462277+0000", 2024-09-14T22:36:33.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:33.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462277+0000", 2024-09-14T22:36:33.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:33.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462277+0000", 2024-09-14T22:36:33.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2800000000000002e-07 2024-09-14T22:36:33.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:33.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462280+0000", 2024-09-14T22:36:33.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2570000000000002e-06 2024-09-14T22:36:33.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:33.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462286+0000", 2024-09-14T22:36:33.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2199999999999997e-06 2024-09-14T22:36:33.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:33.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462291+0000", 2024-09-14T22:36:33.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5689999999999998e-06 2024-09-14T22:36:33.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:33.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489760+0000", 2024-09-14T22:36:33.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027469345999999999 2024-09-14T22:36:33.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:33.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489852+0000", 2024-09-14T22:36:33.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1941999999999995e-05 2024-09-14T22:36:33.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:33.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489930+0000", 2024-09-14T22:36:33.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7853000000000002e-05 2024-09-14T22:36:33.267 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.267 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:33.267 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.267 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20867 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7110656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:33.267 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483587+0000", 2024-09-14T22:36:33.267 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.72839807899999998, 2024-09-14T22:36:33.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026913502999999998, 2024-09-14T22:36:33.267 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:33.267 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:33.267 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:33.268 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:33.268 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:33.268 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:33.268 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20867 2024-09-14T22:36:33.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.268 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:33.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.268 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:33.268 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483587+0000", 2024-09-14T22:36:33.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.268 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:33.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483587+0000", 2024-09-14T22:36:33.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:33.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483587+0000", 2024-09-14T22:36:33.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0399999999999996e-07 2024-09-14T22:36:33.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:33.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483590+0000", 2024-09-14T22:36:33.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.65e-06 2024-09-14T22:36:33.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:33.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483597+0000", 2024-09-14T22:36:33.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1350000000000003e-06 2024-09-14T22:36:33.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:33.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483603+0000", 2024-09-14T22:36:33.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2910000000000004e-06 2024-09-14T22:36:33.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:33.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.510363+0000", 2024-09-14T22:36:33.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026759593000000002 2024-09-14T22:36:33.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:33.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.510418+0000", 2024-09-14T22:36:33.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5245999999999999e-05 2024-09-14T22:36:33.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:33.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.510500+0000", 2024-09-14T22:36:33.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2083999999999995e-05 2024-09-14T22:36:33.271 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.271 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:33.271 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.271 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20868 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7139328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:33.272 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483611+0000", 2024-09-14T22:36:33.272 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.72837345600000003, 2024-09-14T22:36:33.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027036665000000001, 2024-09-14T22:36:33.272 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:33.272 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:33.272 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:33.272 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:33.272 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:33.272 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:33.272 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20868 2024-09-14T22:36:33.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.272 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:33.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:33.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483611+0000", 2024-09-14T22:36:33.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:33.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483611+0000", 2024-09-14T22:36:33.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:33.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483612+0000", 2024-09-14T22:36:33.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3300000000000003e-07 2024-09-14T22:36:33.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:33.274 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483614+0000", 2024-09-14T22:36:33.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4480000000000001e-06 2024-09-14T22:36:33.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:33.274 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483621+0000", 2024-09-14T22:36:33.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9920000000000001e-06 2024-09-14T22:36:33.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:33.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483626+0000", 2024-09-14T22:36:33.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7899999999999999e-06 2024-09-14T22:36:33.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:33.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.510528+0000", 2024-09-14T22:36:33.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026901682999999999 2024-09-14T22:36:33.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:33.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.510587+0000", 2024-09-14T22:36:33.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9137e-05 2024-09-14T22:36:33.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:33.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.510648+0000", 2024-09-14T22:36:33.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1181999999999998e-05 2024-09-14T22:36:33.276 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.276 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:33.276 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.276 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20869 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7168000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:33.276 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483633+0000", 2024-09-14T22:36:33.276 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.72835155299999998, 2024-09-14T22:36:33.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028181490999999999, 2024-09-14T22:36:33.277 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:33.277 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:33.277 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:33.277 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:33.277 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:33.277 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:33.277 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20869 2024-09-14T22:36:33.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.277 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:33.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:33.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483633+0000", 2024-09-14T22:36:33.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:33.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483633+0000", 2024-09-14T22:36:33.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:33.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483634+0000", 2024-09-14T22:36:33.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2599999999999998e-07 2024-09-14T22:36:33.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:33.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483636+0000", 2024-09-14T22:36:33.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1679999999999998e-06 2024-09-14T22:36:33.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:33.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483642+0000", 2024-09-14T22:36:33.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2149999999999998e-06 2024-09-14T22:36:33.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.280 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:33.280 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483647+0000", 2024-09-14T22:36:33.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7679999999999999e-06 2024-09-14T22:36:33.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.280 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:33.280 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.510694+0000", 2024-09-14T22:36:33.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027047073000000001 2024-09-14T22:36:33.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.280 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:33.280 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.510771+0000", 2024-09-14T22:36:33.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6814e-05 2024-09-14T22:36:33.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:33.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511815+0000", 2024-09-14T22:36:33.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0010440270000000001 2024-09-14T22:36:33.281 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.281 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:33.281 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.281 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20870 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7196672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:33.281 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483652+0000", 2024-09-14T22:36:33.281 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.72833255100000005, 2024-09-14T22:36:33.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028301248000000001, 2024-09-14T22:36:33.282 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:33.282 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:33.282 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:33.282 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:33.282 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:33.282 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:33.282 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20870 2024-09-14T22:36:33.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.282 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:33.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:33.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483652+0000", 2024-09-14T22:36:33.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:33.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483652+0000", 2024-09-14T22:36:33.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:33.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483653+0000", 2024-09-14T22:36:33.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3099999999999997e-07 2024-09-14T22:36:33.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:33.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483655+0000", 2024-09-14T22:36:33.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.877e-06 2024-09-14T22:36:33.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:33.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483661+0000", 2024-09-14T22:36:33.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6049999999999997e-06 2024-09-14T22:36:33.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:33.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483666+0000", 2024-09-14T22:36:33.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3410000000000002e-06 2024-09-14T22:36:33.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:33.285 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511854+0000", 2024-09-14T22:36:33.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028187773999999999 2024-09-14T22:36:33.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.285 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:33.285 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511906+0000", 2024-09-14T22:36:33.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2012000000000003e-05 2024-09-14T22:36:33.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.285 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:33.285 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511954+0000", 2024-09-14T22:36:33.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8007999999999998e-05 2024-09-14T22:36:33.285 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.286 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:33.286 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.286 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20871 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7225344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:33.286 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483671+0000", 2024-09-14T22:36:33.286 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.72831362799999999, 2024-09-14T22:36:33.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028405392000000002, 2024-09-14T22:36:33.286 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:33.286 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:33.286 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:33.286 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:33.286 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:33.286 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:33.286 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20871 2024-09-14T22:36:33.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.287 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:33.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.287 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:33.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483671+0000", 2024-09-14T22:36:33.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.287 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:33.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483671+0000", 2024-09-14T22:36:33.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.287 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:33.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483672+0000", 2024-09-14T22:36:33.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:36:33.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:33.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483674+0000", 2024-09-14T22:36:33.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3190000000000002e-06 2024-09-14T22:36:33.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:33.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483680+0000", 2024-09-14T22:36:33.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6860000000000003e-06 2024-09-14T22:36:33.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:33.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483684+0000", 2024-09-14T22:36:33.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3769999999999998e-06 2024-09-14T22:36:33.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:33.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511982+0000", 2024-09-14T22:36:33.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028297478000000001 2024-09-14T22:36:33.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:33.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512027+0000", 2024-09-14T22:36:33.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5290000000000002e-05 2024-09-14T22:36:33.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:33.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512077+0000", 2024-09-14T22:36:33.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9876000000000002e-05 2024-09-14T22:36:33.290 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.290 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:33.290 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.290 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20872 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7254016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:33.290 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483689+0000", 2024-09-14T22:36:33.290 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.72829555800000001, 2024-09-14T22:36:33.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028494049, 2024-09-14T22:36:33.290 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:33.290 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:33.291 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:33.291 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:33.291 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:33.291 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:33.291 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20872 2024-09-14T22:36:33.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.291 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:33.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:33.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483689+0000", 2024-09-14T22:36:33.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:33.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483689+0000", 2024-09-14T22:36:33.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:33.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483690+0000", 2024-09-14T22:36:33.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:36:33.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:33.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483692+0000", 2024-09-14T22:36:33.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.778e-06 2024-09-14T22:36:33.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:33.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483697+0000", 2024-09-14T22:36:33.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7069999999999997e-06 2024-09-14T22:36:33.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:33.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483702+0000", 2024-09-14T22:36:33.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4689999999999999e-06 2024-09-14T22:36:33.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:33.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512104+0000", 2024-09-14T22:36:33.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028401948999999999 2024-09-14T22:36:33.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:33.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512142+0000", 2024-09-14T22:36:33.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8114999999999997e-05 2024-09-14T22:36:33.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:33.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512183+0000", 2024-09-14T22:36:33.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1496999999999999e-05 2024-09-14T22:36:33.294 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.294 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:33.294 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.295 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20873 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7282688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:33.295 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483707+0000", 2024-09-14T22:36:33.295 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.72827784399999995, 2024-09-14T22:36:33.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028581662000000001, 2024-09-14T22:36:33.295 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:33.295 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:33.295 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:33.295 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:33.295 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:33.295 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:33.295 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20873 2024-09-14T22:36:33.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.295 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:33.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:33.296 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483707+0000", 2024-09-14T22:36:33.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:33.296 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483707+0000", 2024-09-14T22:36:33.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:33.296 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483707+0000", 2024-09-14T22:36:33.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2500000000000001e-07 2024-09-14T22:36:33.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:33.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483709+0000", 2024-09-14T22:36:33.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.891e-06 2024-09-14T22:36:33.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:33.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483715+0000", 2024-09-14T22:36:33.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7019999999999998e-06 2024-09-14T22:36:33.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:33.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483720+0000", 2024-09-14T22:36:33.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4900000000000002e-06 2024-09-14T22:36:33.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:33.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512213+0000", 2024-09-14T22:36:33.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028493622 2024-09-14T22:36:33.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:33.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512248+0000", 2024-09-14T22:36:33.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5228000000000002e-05 2024-09-14T22:36:33.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:33.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512289+0000", 2024-09-14T22:36:33.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0404000000000001e-05 2024-09-14T22:36:33.299 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.299 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:33.299 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.299 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20874 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7311360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:33.299 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483725+0000", 2024-09-14T22:36:33.299 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.72826020499999999, 2024-09-14T22:36:33.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028777193999999999, 2024-09-14T22:36:33.299 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:33.299 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:33.299 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:33.300 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:33.300 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:33.300 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:33.300 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20874 2024-09-14T22:36:33.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.300 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:33.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.300 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:33.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483725+0000", 2024-09-14T22:36:33.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:33.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483725+0000", 2024-09-14T22:36:33.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:33.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483725+0000", 2024-09-14T22:36:33.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8000000000000001e-07 2024-09-14T22:36:33.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:33.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483727+0000", 2024-09-14T22:36:33.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8920000000000001e-06 2024-09-14T22:36:33.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:33.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483733+0000", 2024-09-14T22:36:33.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6840000000000001e-06 2024-09-14T22:36:33.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:33.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483737+0000", 2024-09-14T22:36:33.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4440000000000001e-06 2024-09-14T22:36:33.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:33.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512326+0000", 2024-09-14T22:36:33.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028588612999999999 2024-09-14T22:36:33.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.303 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:33.303 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512399+0000", 2024-09-14T22:36:33.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2782999999999998e-05 2024-09-14T22:36:33.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.303 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:33.303 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512502+0000", 2024-09-14T22:36:33.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000103398 2024-09-14T22:36:33.303 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.303 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:33.303 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.304 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20912 3.1c 3:3b9f022e:::rbd_data.113c4558bb1.000000000000009a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 497664~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:33.304 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.484800+0000", 2024-09-14T22:36:33.304 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.727184677, 2024-09-14T22:36:33.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045787688999999999, 2024-09-14T22:36:33.304 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:33.304 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:33.304 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:33.304 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:33.304 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:33.304 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:33.304 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20912 2024-09-14T22:36:33.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.305 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:33.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:33.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484800+0000", 2024-09-14T22:36:33.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:33.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484800+0000", 2024-09-14T22:36:33.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:33.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:33.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484801+0000", 2024-09-14T22:36:33.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1939999999999999e-06 2024-09-14T22:36:33.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:33.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484809+0000", 2024-09-14T22:36:33.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2989999999999999e-06 2024-09-14T22:36:33.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:33.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484823+0000", 2024-09-14T22:36:33.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3986e-05 2024-09-14T22:36:33.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:33.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484831+0000", 2024-09-14T22:36:33.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4009999999999997e-06 2024-09-14T22:36:33.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:33.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512318+0000", 2024-09-14T22:36:33.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027487208999999999 2024-09-14T22:36:33.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:33.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512450+0000", 2024-09-14T22:36:33.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000131681 2024-09-14T22:36:33.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:36:33.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512666+0000", 2024-09-14T22:36:33.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021547 2024-09-14T22:36:33.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.308 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:33.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.513788+0000", 2024-09-14T22:36:33.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001122975 2024-09-14T22:36:33.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.308 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:33.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.530473+0000", 2024-09-14T22:36:33.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.016684984999999999 2024-09-14T22:36:33.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.308 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:33.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.530533+0000", 2024-09-14T22:36:33.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9296999999999998e-05 2024-09-14T22:36:33.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:33.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:33.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:33.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.530588+0000", 2024-09-14T22:36:33.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5192000000000003e-05 2024-09-14T22:36:33.309 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.309 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:33.309 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.309 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:33.309 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:33.309 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:33.311 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:36:33.312 INFO:teuthology.orchestra.run.smithi001.stdout:765 write 0x3d457fb thru 0x3d4a8a5 (0x50ab bytes) 2024-09-14T22:36:33.312 INFO:teuthology.orchestra.run.smithi001.stdout:766 write 0x19c1937 thru 0x19c5846 (0x3f10 bytes) 2024-09-14T22:36:33.312 INFO:teuthology.orchestra.run.smithi001.stdout:767 read 0x3267537 thru 0x327526f (0xdd39 bytes) 2024-09-14T22:36:33.313 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:33.313 INFO:teuthology.orchestra.run.smithi001.stdout:769 write 0x371845a thru 0x371b225 (0x2dcc bytes) 2024-09-14T22:36:33.313 INFO:teuthology.orchestra.run.smithi001.stdout:770 read 0x1bcc23c thru 0x1bd2889 (0x664e bytes) 2024-09-14T22:36:33.313 INFO:teuthology.orchestra.run.smithi001.stdout:771 trunc from 0x3dbbb6f to 0xae17b3f 2024-09-14T22:36:33.313 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:33.313 INFO:teuthology.orchestra.run.smithi001.stdout:773 punch from 0xabc8ff2 to 0xabc90a0, (0xae bytes) 2024-09-14T22:36:33.313 INFO:teuthology.orchestra.run.smithi001.stdout:774 read 0x1f247f1 thru 0x1f2e6e5 (0x9ef5 bytes) 2024-09-14T22:36:33.313 INFO:teuthology.orchestra.run.smithi001.stdout:775 writesame 0x636c616 thru 0x6377c28 data_size 0x3(0xb613 bytes) 2024-09-14T22:36:33.313 INFO:teuthology.orchestra.run.smithi001.stdout:777 punch from 0x92219ee to 0x9224a94, (0x30a6 bytes) 2024-09-14T22:36:33.313 INFO:teuthology.orchestra.run.smithi001.stdout:782 punch from 0x224222b to 0x224af0e, (0x8ce3 bytes) 2024-09-14T22:36:33.313 INFO:teuthology.orchestra.run.smithi001.stdout:783 write 0x660e239 thru 0x66185c0 (0xa388 bytes) 2024-09-14T22:36:33.313 INFO:teuthology.orchestra.run.smithi001.stdout:784 read 0x81fd7e9 thru 0x8205736 (0x7f4e bytes) 2024-09-14T22:36:33.314 INFO:teuthology.orchestra.run.smithi001.stdout:785 writesame 0x6dc142f thru 0x6dce40a data_size 0x6a(0xcfdc bytes) 2024-09-14T22:36:33.314 INFO:teuthology.orchestra.run.smithi001.stdout:786 trunc from 0xae17b3f to 0xbd8bd91 2024-09-14T22:36:33.314 INFO:teuthology.orchestra.run.smithi001.stdout:787 read 0x7c8fce3 thru 0x7c9b590 (0xb8ae bytes) 2024-09-14T22:36:33.314 INFO:teuthology.orchestra.run.smithi001.stdout:789 punch from 0x908d876 to 0x9090bf1, (0x337b bytes) 2024-09-14T22:36:33.315 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:33.315 INFO:teuthology.orchestra.run.smithi001.stdout:791 writesame 0x25d3f3c thru 0x25d9ceb data_size 0x8(0x5db0 bytes) 2024-09-14T22:36:33.319 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:33.319 INFO:teuthology.orchestra.run.smithi001.stdout:793 write 0x29b1d9 thru 0x2a9e53 (0xec7b bytes) 2024-09-14T22:36:33.362 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:33.355+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:33.415 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:33.410+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:33.454 INFO:teuthology.orchestra.run.smithi001.stdout:795 write 0xaff6ef8 thru 0xb005bdd (0xece6 bytes) 2024-09-14T22:36:33.466 INFO:teuthology.orchestra.run.smithi001.stdout:796 read 0xaec3b96 thru 0xaec8c0b (0x5076 bytes) 2024-09-14T22:36:33.466 INFO:teuthology.orchestra.run.smithi001.stdout:797 write 0xc8d66c thru 0xc919c3 (0x4358 bytes) 2024-09-14T22:36:33.516 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:33.510+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:33.539 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:36:33.542 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:33.542 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:33.542 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:36:33.543 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:33.557 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:36:33.588 INFO:teuthology.orchestra.run.smithi001.stdout:799 write 0xdc74545 thru 0xdc7a30a (0x5dc6 bytes) 2024-09-14T22:36:33.603 INFO:teuthology.orchestra.run.smithi001.stdout:800 read 0x43c1c6a thru 0x43cf5d8 (0xd96f bytes) 2024-09-14T22:36:33.603 INFO:teuthology.orchestra.run.smithi001.stdout:801 read 0xbce8614 thru 0xbcf44a7 (0xbe94 bytes) 2024-09-14T22:36:33.613 INFO:teuthology.orchestra.run.smithi001.stdout:803 trunc from 0xdc7a30b to 0x61881bc 2024-09-14T22:36:33.634 INFO:teuthology.orchestra.run.smithi001.stdout:804 writesame 0x2e7a6ee thru 0x2e7b03f data_size 0x1(0x952 bytes) 2024-09-14T22:36:33.640 INFO:teuthology.orchestra.run.smithi001.stdout:805 read 0x36a66da thru 0x36afdbc (0x96e3 bytes) 2024-09-14T22:36:33.646 INFO:teuthology.orchestra.run.smithi001.stdout:806 read 0x163072a thru 0x1635313 (0x4bea bytes) 2024-09-14T22:36:33.646 INFO:teuthology.orchestra.run.smithi001.stdout:807 read 0x5892d64 thru 0x5893413 (0x6b0 bytes) 2024-09-14T22:36:33.646 INFO:teuthology.orchestra.run.smithi001.stdout:808 punch from 0x224a9c4 to 0x224d510, (0x2b4c bytes) 2024-09-14T22:36:33.646 INFO:teuthology.orchestra.run.smithi001.stdout:809 punch from 0x1c57f2a to 0x1c65177, (0xd24d bytes) 2024-09-14T22:36:33.646 INFO:teuthology.orchestra.run.smithi001.stdout:810 read 0x4fa13f1 thru 0x4faa7a5 (0x93b5 bytes) 2024-09-14T22:36:33.646 INFO:teuthology.orchestra.run.smithi001.stdout:811 punch from 0x5a66733 to 0x5a66dc9, (0x696 bytes) 2024-09-14T22:36:33.646 INFO:teuthology.orchestra.run.smithi001.stdout:812 read 0x5d8569 thru 0x5e4a27 (0xc4bf bytes) 2024-09-14T22:36:33.646 INFO:teuthology.orchestra.run.smithi001.stdout:813 punch from 0x1d243f9 to 0x1d29a65, (0x566c bytes) 2024-09-14T22:36:33.647 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:33.643+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:33.654 INFO:teuthology.orchestra.run.smithi001.stdout:815 writesame 0xa18feb9 thru 0xa1918ab data_size 0xd(0x19f3 bytes) 2024-09-14T22:36:33.662 INFO:teuthology.orchestra.run.smithi001.stdout:817 trunc from 0xa1918ac to 0x3876424 2024-09-14T22:36:33.671 INFO:teuthology.orchestra.run.smithi001.stdout:818 punch from 0x1d30587 to 0x1d3fb91, (0xf60a bytes) 2024-09-14T22:36:33.671 INFO:teuthology.orchestra.run.smithi001.stdout:819 read 0x2e27121 thru 0x2e2c9f8 (0x58d8 bytes) 2024-09-14T22:36:33.671 INFO:teuthology.orchestra.run.smithi001.stdout:822 read 0x1d611a2 thru 0x1d680f2 (0x6f51 bytes) 2024-09-14T22:36:33.671 INFO:teuthology.orchestra.run.smithi001.stdout:823 read 0x2fceb23 thru 0x2fd4c5c (0x613a bytes) 2024-09-14T22:36:33.672 INFO:teuthology.orchestra.run.smithi001.stdout:825 read 0x1516e87 thru 0x152619a (0xf314 bytes) 2024-09-14T22:36:33.674 INFO:teuthology.orchestra.run.smithi001.stdout:826 trunc from 0x3876424 to 0x2306243 2024-09-14T22:36:33.695 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:33.695 INFO:teuthology.orchestra.run.smithi001.stdout:829 punch from 0x1b9cdba to 0x1baa1bc, (0xd402 bytes) 2024-09-14T22:36:33.695 INFO:teuthology.orchestra.run.smithi001.stdout:830 read 0x55fc1d thru 0x569549 (0x992d bytes) 2024-09-14T22:36:33.696 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:33.696 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:33.696 INFO:teuthology.orchestra.run.smithi001.stdout:833 read 0x1ff5a64 thru 0x1ff5b28 (0xc5 bytes) 2024-09-14T22:36:33.696 INFO:teuthology.orchestra.run.smithi001.stdout:834 read 0x21f1fea thru 0x21fa56c (0x8583 bytes) 2024-09-14T22:36:33.704 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:36:33.704 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:36:33.705 INFO:teuthology.orchestra.run.smithi001.stdout:835 writesame 0x4dd885c thru 0x4dd9021 data_size 0x5(0x7c6 bytes) 2024-09-14T22:36:33.705 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:33.711 INFO:teuthology.orchestra.run.smithi001.stdout:840 trunc from 0x4dd9022 to 0xa41883b 2024-09-14T22:36:33.716 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:33.711+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:33.717 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:33.717 INFO:teuthology.orchestra.run.smithi001.stdout:842 write 0x1f6c197 thru 0x1f78195 (0xbfff bytes) 2024-09-14T22:36:33.722 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:36:33.722 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:36:33.722 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-14T22:36:33.730 INFO:teuthology.orchestra.run.smithi001.stdout:843 read 0x983c088 thru 0x983cc80 (0xbf9 bytes) 2024-09-14T22:36:33.732 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:33.732 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:33.732 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:36:33.732 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:36:33.732 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:33.735 INFO:teuthology.orchestra.run.smithi001.stdout:844 trunc from 0xa41883b to 0xe21dc68 2024-09-14T22:36:33.741 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:36:33.741 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:36:33.742 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:33.743 INFO:teuthology.orchestra.run.smithi001.stdout:846 trunc from 0xe21dc68 to 0xd2b777c 2024-09-14T22:36:33.745 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:36:33.749 INFO:teuthology.orchestra.run.smithi001.stdout:847 read 0xcebbc25 thru 0xcebc1e8 (0x5c4 bytes) 2024-09-14T22:36:33.749 INFO:teuthology.orchestra.run.smithi001.stdout:848 write 0x6ce85b3 thru 0x6cf3f80 (0xb9ce bytes) 2024-09-14T22:36:33.760 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:36:33.760 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:36:33.763 INFO:teuthology.orchestra.run.smithi001.stdout:849 writesame 0x5c6ce40 thru 0x5c770be data_size 0x1(0xa27f bytes) 2024-09-14T22:36:33.778 INFO:teuthology.orchestra.run.smithi001.stdout:850 read 0xac9caa1 thru 0xaca3a6a (0x6fca bytes) 2024-09-14T22:36:33.779 INFO:teuthology.orchestra.run.smithi001.stdout:851 write 0x6e68300 thru 0x6e6adf3 (0x2af4 bytes) 2024-09-14T22:36:33.781 INFO:teuthology.orchestra.run.smithi001.stdout:852 write 0x6cf5940 thru 0x6cf7f0a (0x25cb bytes) 2024-09-14T22:36:33.782 INFO:teuthology.orchestra.run.smithi001.stdout:853 writesame 0xa12fd52 thru 0xa1321ea data_size 0x9(0x2499 bytes) 2024-09-14T22:36:33.786 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:36:33.787 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:36:33.796 INFO:teuthology.orchestra.run.smithi001.stdout:854 trunc from 0xd2b777c to 0x7a83e7e 2024-09-14T22:36:33.807 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:36:33.807 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:36:33.811 INFO:teuthology.orchestra.run.smithi001.stdout:856 trunc from 0x7a83e7e to 0xa08cfd9 2024-09-14T22:36:33.817 INFO:teuthology.orchestra.run.smithi001.stdout:858 write 0xc93db1 thru 0xc97bbc (0x3e0c bytes) 2024-09-14T22:36:33.829 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:36:33.830 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:36:33.832 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:36:33.833 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:36:33.835 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:33.831+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:33.836 INFO:teuthology.orchestra.run.smithi001.stdout:859 punch from 0x4456605 to 0x4465015, (0xea10 bytes) 2024-09-14T22:36:33.836 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:33.836 INFO:teuthology.orchestra.run.smithi001.stdout:861 read 0x27d0f87 thru 0x27d3aee (0x2b68 bytes) 2024-09-14T22:36:33.836 INFO:teuthology.orchestra.run.smithi001.stdout:862 punch from 0x43b1ee0 to 0x43b6ad4, (0x4bf4 bytes) 2024-09-14T22:36:33.844 INFO:teuthology.orchestra.run.smithi001.stdout:863 trunc from 0xa08cfd9 to 0x389b4bb 2024-09-14T22:36:33.869 INFO:teuthology.orchestra.run.smithi001.stdout:864 trunc from 0x389b4bb to 0xbc55a83 2024-09-14T22:36:33.875 INFO:teuthology.orchestra.run.smithi001.stdout:865 punch from 0xe39cba to 0xe3edce, (0x5114 bytes) 2024-09-14T22:36:33.876 INFO:teuthology.orchestra.run.smithi001.stdout:866 read 0x338a66d thru 0x339497f (0xa313 bytes) 2024-09-14T22:36:33.876 INFO:teuthology.orchestra.run.smithi001.stdout:868 write 0x4cc7a8a thru 0x4cce663 (0x6bda bytes) 2024-09-14T22:36:33.890 INFO:teuthology.orchestra.run.smithi001.stdout:869 writesame 0x19ff53d thru 0x1a06623 data_size 0x1(0x70e7 bytes) 2024-09-14T22:36:33.902 INFO:teuthology.orchestra.run.smithi001.stdout:870 trunc from 0xbc55a83 to 0x8b83a01 2024-09-14T22:36:33.908 INFO:teuthology.orchestra.run.smithi001.stdout:872 read 0x64c56fe thru 0x64cc16f (0x6a72 bytes) 2024-09-14T22:36:33.908 INFO:teuthology.orchestra.run.smithi001.stdout:873 write 0x3a6e008 thru 0x3a7a304 (0xc2fd bytes) 2024-09-14T22:36:33.911 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:33.911 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:33.917 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:33.911+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:33.923 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:33.923 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:33.923 INFO:teuthology.orchestra.run.smithi001.stdout:876 read 0x5c0e97c thru 0x5c15262 (0x68e7 bytes) 2024-09-14T22:36:33.923 INFO:teuthology.orchestra.run.smithi001.stdout:877 writesame 0x4773db8 thru 0x477ae16 data_size 0x2b(0x705f bytes) 2024-09-14T22:36:33.926 INFO:teuthology.orchestra.run.smithi001.stdout:878 read 0x1947e34 thru 0x1952077 (0xa244 bytes) 2024-09-14T22:36:33.927 INFO:teuthology.orchestra.run.smithi001.stdout:879 read 0x82c5a76 thru 0x82cebc2 (0x914d bytes) 2024-09-14T22:36:33.930 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:36:33.933 INFO:teuthology.orchestra.run.smithi001.stdout:880 write 0x986c334 thru 0x9871de8 (0x5ab5 bytes) 2024-09-14T22:36:33.949 INFO:teuthology.orchestra.run.smithi001.stdout:881 read 0x59ce5df thru 0x59d3327 (0x4d49 bytes) 2024-09-14T22:36:33.949 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:33.949 INFO:teuthology.orchestra.run.smithi001.stdout:883 write 0x73aef2 thru 0x73c5a3 (0x16b2 bytes) 2024-09-14T22:36:33.951 INFO:teuthology.orchestra.run.smithi001.stdout:885 read 0xabc57d thru 0xac7f3a (0xb9be bytes) 2024-09-14T22:36:33.953 INFO:teuthology.orchestra.run.smithi001.stdout:886 write 0x8678d85 thru 0x8682908 (0x9b84 bytes) 2024-09-14T22:36:33.977 INFO:teuthology.orchestra.run.smithi001.stdout:887 trunc from 0x9871de9 to 0x3750345 2024-09-14T22:36:34.009 INFO:teuthology.orchestra.run.smithi001.stdout:888 trunc from 0x3750345 to 0xd07b7c2 2024-09-14T22:36:34.016 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:34.016 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:34.019 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:34.014+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:34.025 INFO:teuthology.orchestra.run.smithi001.stdout:891 write 0xeca2372 thru 0xeca3853 (0x14e2 bytes) 2024-09-14T22:36:34.030 INFO:teuthology.orchestra.run.smithi001.stdout:892 writesame 0x458437d thru 0x4586474 data_size 0x14(0x20f8 bytes) 2024-09-14T22:36:34.039 INFO:teuthology.orchestra.run.smithi001.stdout:895 read 0x90b6f55 thru 0x90ba3fd (0x34a9 bytes) 2024-09-14T22:36:34.047 INFO:teuthology.orchestra.run.smithi001.stdout:897 trunc from 0xeca3854 to 0x7907bc2 2024-09-14T22:36:34.073 INFO:teuthology.orchestra.run.smithi001.stdout:899 write 0xa621f75 thru 0xa625179 (0x3205 bytes) 2024-09-14T22:36:34.090 INFO:teuthology.orchestra.run.smithi001.stdout:900 write 0x41c45ef thru 0x41c717b (0x2b8d bytes) 2024-09-14T22:36:34.091 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:34.092 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:34.098 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:34.098 INFO:teuthology.orchestra.run.smithi001.stdout:902 write 0x688c96f thru 0x68936cf (0x6d61 bytes) 2024-09-14T22:36:34.107 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:36:34.119 INFO:teuthology.orchestra.run.smithi001.stdout:903 writesame 0x7adefd6 thru 0x7aee3da data_size 0x21(0xf405 bytes) 2024-09-14T22:36:34.142 INFO:teuthology.orchestra.run.smithi001.stdout:906 punch from 0x284c292 to 0x285b00d, (0xed7b bytes) 2024-09-14T22:36:34.144 INFO:teuthology.orchestra.run.smithi001.stdout:908 trunc from 0xa62517a to 0xcc3d0ff 2024-09-14T22:36:34.151 INFO:teuthology.orchestra.run.smithi001.stdout:909 write 0x2199eb0 thru 0x21a42ec (0xa43d bytes) 2024-09-14T22:36:34.155 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:34.151+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:34.155 INFO:teuthology.orchestra.run.smithi001.stdout:910 read 0x882a27d thru 0x882eb56 (0x48da bytes) 2024-09-14T22:36:34.156 INFO:teuthology.orchestra.run.smithi001.stdout:911 write 0x2a08c34 thru 0x2a0dfdf (0x53ac bytes) 2024-09-14T22:36:34.169 INFO:teuthology.orchestra.run.smithi001.stdout:912 read 0x8333af3 thru 0x8333e3a (0x348 bytes) 2024-09-14T22:36:34.169 INFO:teuthology.orchestra.run.smithi001.stdout:913 punch from 0x1559a00 to 0x155c48d, (0x2a8d bytes) 2024-09-14T22:36:34.169 INFO:teuthology.orchestra.run.smithi001.stdout:914 read 0x762391f thru 0x762ce26 (0x9508 bytes) 2024-09-14T22:36:34.169 INFO:teuthology.orchestra.run.smithi001.stdout:915 punch from 0x5c4682e to 0x5c4d6d3, (0x6ea5 bytes) 2024-09-14T22:36:34.170 INFO:teuthology.orchestra.run.smithi001.stdout:916 write 0x9920606 thru 0x9925063 (0x4a5e bytes) 2024-09-14T22:36:34.187 INFO:teuthology.orchestra.run.smithi001.stdout:917 write 0x2a21690 thru 0x2a22b27 (0x1498 bytes) 2024-09-14T22:36:34.189 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:34.190 INFO:teuthology.orchestra.run.smithi001.stdout:919 write 0xbd7809b thru 0xbd8261d (0xa583 bytes) 2024-09-14T22:36:34.213 INFO:teuthology.orchestra.run.smithi001.stdout:920 read 0xbc1030 thru 0xbc730c (0x62dd bytes) 2024-09-14T22:36:34.220 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:34.214+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:34.229 INFO:teuthology.orchestra.run.smithi001.stdout:922 trunc from 0xcc3d0ff to 0x2b4f397 2024-09-14T22:36:34.235 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:34.235 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:34.257 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:36:34.278 INFO:teuthology.orchestra.run.smithi001.stdout:923 write 0x5b3cc2b thru 0x5b400f7 (0x34cd bytes) 2024-09-14T22:36:34.291 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:34.292 INFO:teuthology.orchestra.run.smithi001.stdout:925 write 0x3c5fd6f thru 0x3c65103 (0x5395 bytes) 2024-09-14T22:36:34.322 INFO:teuthology.orchestra.run.smithi001.stdout:927 write 0xce2011a thru 0xce2697c (0x6863 bytes) 2024-09-14T22:36:34.339 INFO:teuthology.orchestra.run.smithi001.stdout:928 read 0x185cc3f thru 0x1868930 (0xbcf2 bytes) 2024-09-14T22:36:34.339 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:34.343 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:34.339+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:34.351 INFO:teuthology.orchestra.run.smithi001.stdout:930 trunc from 0xce2697d to 0x41d6f9d 2024-09-14T22:36:34.369 INFO:teuthology.orchestra.run.smithi001.stdout:931 punch from 0x4b808d to 0x4bb8dd, (0x3850 bytes) 2024-09-14T22:36:34.382 INFO:teuthology.orchestra.run.smithi001.stdout:932 write 0x7382ccb thru 0x73850f8 (0x242e bytes) 2024-09-14T22:36:34.393 INFO:teuthology.orchestra.run.smithi001.stdout:933 punch from 0x1a972bd to 0x1aa54b4, (0xe1f7 bytes) 2024-09-14T22:36:34.393 INFO:teuthology.orchestra.run.smithi001.stdout:934 writesame 0x54ab585 thru 0x54b939c data_size 0x2e(0xde18 bytes) 2024-09-14T22:36:34.412 INFO:teuthology.orchestra.run.smithi001.stdout:935 clone 5 order 25 su 131072 sc 9 2024-09-14T22:36:34.422 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:34.414+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:34.422 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:34.422 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:34.435 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:36:34.523 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:34.519+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:34.557 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:34.557 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:34.571 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:36:34.609 INFO:teuthology.orchestra.run.smithi001.stdout:get_pg_num_history pg_num_history pg_num_history(e30 pg_nums {1={16=1},2={19=8},3={23=32}} deleted_pools ) 2024-09-14T22:36:34.609 INFO:teuthology.orchestra.run.smithi001.stdout:Importing pgid 3.4 2024-09-14T22:36:34.614 INFO:teuthology.orchestra.run.smithi001.stdout:Write #3:2104c016:::rbd_data.113c3b74a67f.0000000000000004:head# 2024-09-14T22:36:34.617 INFO:teuthology.orchestra.run.smithi001.stdout:snapset 0=[]:{} 2024-09-14T22:36:34.624 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:34.618+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:34.625 INFO:teuthology.orchestra.run.smithi001.stdout:Write #3:25583457:::rbd_data.113cde4a8b6b.0000000000000074:head# 2024-09-14T22:36:34.625 INFO:teuthology.orchestra.run.smithi001.stdout:snapset 0=[]:{} 2024-09-14T22:36:34.625 INFO:teuthology.orchestra.run.smithi001.stdout:write_pg epoch 30 info 3.4( v 30'20 (0'0,30'20] local-lis/les=23/24 n=2 ec=23/23 lis/c=23/23 les/c/f=24/24/0 sis=23) 2024-09-14T22:36:34.626 INFO:teuthology.orchestra.run.smithi001.stdout:Import successful 2024-09-14T22:36:34.698 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:34.699 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:34.712 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:36:34.754 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:34.750+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:34.817 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T22:36:34.862 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:34.858+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:34.889 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:34.889 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:34.907 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:36:34.944 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone4 from 0x73850f9 (overlap 0xe60e41) to 0x0 2024-09-14T22:36:34.944 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:34.938+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:35.027 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:35.019+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:35.127 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:35.122+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:35.134 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:35.134 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:35.155 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:36:35.190 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f /var/log/ceph/exp.3.4.2 2024-09-14T22:36:35.191 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-14T22:36:35.228 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:35.223+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:35.313 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:35.314 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:35.329 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:36:35.329 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:35.322+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:35.410 INFO:teuthology.orchestra.run.smithi001.stdout:936 read 0x3486556 thru 0x348d171 (0x6c1c bytes) 2024-09-14T22:36:35.429 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:35.423+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:35.437 INFO:teuthology.orchestra.run.smithi001.stdout:937 write 0xe4869e6 thru 0xe4927df (0xbdfa bytes) 2024-09-14T22:36:35.514 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:35.514 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:35.529 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:36:35.531 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:35.527+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:35.559 INFO:teuthology.orchestra.run.smithi001.stdout:938 read 0x616b141 thru 0x6175017 (0x9ed7 bytes) 2024-09-14T22:36:35.561 INFO:teuthology.orchestra.run.smithi001.stdout:939 flatten 2024-09-14T22:36:35.633 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:35.626+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:35.733 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:35.727+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:35.761 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:35.761 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:35.784 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:36:35.810 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2024-09-14T22:36:35.833 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:36:35.834 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:35.827+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:35.935 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:35.931+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:35.940 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:35.940 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:35.956 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:36:36.021 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:36:36.021 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:36:36.036 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:36.036 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:36.031+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:36.040 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:36:36.041 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:36:36.041 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-14T22:36:36.045 INFO:teuthology.orchestra.run.smithi001.stdout:942 trunc from 0xe4927e0 to 0x83dfb80 2024-09-14T22:36:36.057 INFO:teuthology.orchestra.run.smithi001.stdout:944 read 0x61aa409 thru 0x61af74c (0x5344 bytes) 2024-09-14T22:36:36.058 INFO:teuthology.orchestra.run.smithi001.stdout:945 write 0x3aeeb67 thru 0x3af7aa6 (0x8f40 bytes) 2024-09-14T22:36:36.063 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:36:36.063 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:36:36.075 INFO:teuthology.orchestra.run.smithi001.stdout:947 trunc from 0x83dfb80 to 0x5426ce 2024-09-14T22:36:36.089 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:36:36.089 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:36:36.102 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:36.102 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:36:36.102 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:36:36.102 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:36:36.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.103 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35300 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1253376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:36.103 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.506930+0000", 2024-09-14T22:36:36.103 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 2.5929251099999999, 2024-09-14T22:36:36.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044627325000000002, 2024-09-14T22:36:36.103 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:36.103 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:36.103 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:36.103 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:36.103 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:36.103 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.103 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35300 2024-09-14T22:36:36.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.103 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:36.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:36.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506930+0000", 2024-09-14T22:36:36.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:36.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506930+0000", 2024-09-14T22:36:36.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:36.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506930+0000", 2024-09-14T22:36:36.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2500000000000001e-07 2024-09-14T22:36:36.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.105 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:36.105 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506932+0000", 2024-09-14T22:36:36.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8300000000000001e-06 2024-09-14T22:36:36.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.105 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:36.105 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506940+0000", 2024-09-14T22:36:36.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2810000000000003e-06 2024-09-14T22:36:36.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506944+0000", 2024-09-14T22:36:36.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8049999999999997e-06 2024-09-14T22:36:36.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:36.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.551359+0000", 2024-09-14T22:36:36.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044414370000000002 2024-09-14T22:36:36.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:36.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.551432+0000", 2024-09-14T22:36:36.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3338999999999997e-05 2024-09-14T22:36:36.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:36.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.551557+0000", 2024-09-14T22:36:36.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000125275 2024-09-14T22:36:36.107 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.107 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:36.107 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.107 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35301 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1282048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:36.107 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.506950+0000", 2024-09-14T22:36:36.107 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 2.592904909, 2024-09-14T22:36:36.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044873612, 2024-09-14T22:36:36.108 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:36.108 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:36.108 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:36.108 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:36.108 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:36.108 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.108 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35301 2024-09-14T22:36:36.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.108 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:36.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:36.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506950+0000", 2024-09-14T22:36:36.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:36.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506950+0000", 2024-09-14T22:36:36.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:36.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506951+0000", 2024-09-14T22:36:36.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0400000000000002e-07 2024-09-14T22:36:36.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:36.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506953+0000", 2024-09-14T22:36:36.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9470000000000002e-06 2024-09-14T22:36:36.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:36.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506958+0000", 2024-09-14T22:36:36.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8930000000000002e-06 2024-09-14T22:36:36.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.111 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506963+0000", 2024-09-14T22:36:36.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.459e-06 2024-09-14T22:36:36.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.111 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:36.111 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.551642+0000", 2024-09-14T22:36:36.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044678766000000002 2024-09-14T22:36:36.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.111 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:36.111 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.551715+0000", 2024-09-14T22:36:36.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3603999999999995e-05 2024-09-14T22:36:36.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:36.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.551824+0000", 2024-09-14T22:36:36.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108539 2024-09-14T22:36:36.112 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.112 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:36.112 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.113 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35302 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1310720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:36.113 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.506970+0000", 2024-09-14T22:36:36.113 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 2.5928851989999999, 2024-09-14T22:36:36.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045057898999999998, 2024-09-14T22:36:36.113 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:36.113 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:36.113 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:36.113 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:36.113 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:36.113 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.113 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35302 2024-09-14T22:36:36.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.113 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:36.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:36.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506970+0000", 2024-09-14T22:36:36.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:36.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506970+0000", 2024-09-14T22:36:36.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:36.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506970+0000", 2024-09-14T22:36:36.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0999999999999999e-07 2024-09-14T22:36:36.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:36.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506973+0000", 2024-09-14T22:36:36.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4310000000000001e-06 2024-09-14T22:36:36.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:36.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506981+0000", 2024-09-14T22:36:36.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2830000000000001e-06 2024-09-14T22:36:36.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506986+0000", 2024-09-14T22:36:36.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3959999999999997e-06 2024-09-14T22:36:36.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:36.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.551856+0000", 2024-09-14T22:36:36.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044869603000000001 2024-09-14T22:36:36.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:36.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.551931+0000", 2024-09-14T22:36:36.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5253999999999995e-05 2024-09-14T22:36:36.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:36.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552028+0000", 2024-09-14T22:36:36.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6521999999999995e-05 2024-09-14T22:36:36.117 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.117 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:36.117 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.117 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35303 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1339392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:36.117 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.506994+0000", 2024-09-14T22:36:36.117 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 2.5928612910000002, 2024-09-14T22:36:36.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045224116000000002, 2024-09-14T22:36:36.118 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:36.118 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:36.118 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:36.118 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:36.118 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:36.118 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.118 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35303 2024-09-14T22:36:36.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.118 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:36.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:36.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506994+0000", 2024-09-14T22:36:36.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:36.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506994+0000", 2024-09-14T22:36:36.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:36.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506994+0000", 2024-09-14T22:36:36.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1699999999999999e-07 2024-09-14T22:36:36.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:36.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506997+0000", 2024-09-14T22:36:36.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.357e-06 2024-09-14T22:36:36.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:36.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507005+0000", 2024-09-14T22:36:36.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1939999999999998e-06 2024-09-14T22:36:36.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507013+0000", 2024-09-14T22:36:36.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3930000000000008e-06 2024-09-14T22:36:36.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:36.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552058+0000", 2024-09-14T22:36:36.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045044645000000001 2024-09-14T22:36:36.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:36.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552132+0000", 2024-09-14T22:36:36.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3831000000000005e-05 2024-09-14T22:36:36.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:36.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552218+0000", 2024-09-14T22:36:36.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6279000000000005e-05 2024-09-14T22:36:36.121 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.121 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:36.121 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.122 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35304 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1368064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:36.122 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507021+0000", 2024-09-14T22:36:36.122 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 2.5928337309999998, 2024-09-14T22:36:36.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045388246, 2024-09-14T22:36:36.122 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:36.122 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:36.122 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:36.122 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:36.122 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:36.122 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.122 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35304 2024-09-14T22:36:36.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.123 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:36.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:36.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507021+0000", 2024-09-14T22:36:36.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:36.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507021+0000", 2024-09-14T22:36:36.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:36.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507022+0000", 2024-09-14T22:36:36.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.82e-07 2024-09-14T22:36:36.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:36.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507024+0000", 2024-09-14T22:36:36.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4949999999999998e-06 2024-09-14T22:36:36.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:36.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507032+0000", 2024-09-14T22:36:36.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9449999999999993e-06 2024-09-14T22:36:36.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.125 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507039+0000", 2024-09-14T22:36:36.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2129999999999996e-06 2024-09-14T22:36:36.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:36.125 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552250+0000", 2024-09-14T22:36:36.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045211150999999998 2024-09-14T22:36:36.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:36.125 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552326+0000", 2024-09-14T22:36:36.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6008999999999997e-05 2024-09-14T22:36:36.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:36.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552410+0000", 2024-09-14T22:36:36.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3950999999999996e-05 2024-09-14T22:36:36.126 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.126 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:36.126 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.126 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35305 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1396736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:36.126 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507048+0000", 2024-09-14T22:36:36.126 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 2.5928071930000001, 2024-09-14T22:36:36.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045554728000000003, 2024-09-14T22:36:36.127 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:36.127 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:36.127 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:36.127 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:36.127 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:36.127 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.127 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35305 2024-09-14T22:36:36.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.127 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:36.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:36.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507048+0000", 2024-09-14T22:36:36.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:36.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507048+0000", 2024-09-14T22:36:36.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:36.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507048+0000", 2024-09-14T22:36:36.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7800000000000002e-07 2024-09-14T22:36:36.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:36.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507050+0000", 2024-09-14T22:36:36.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.024e-06 2024-09-14T22:36:36.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:36.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507057+0000", 2024-09-14T22:36:36.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1800000000000001e-06 2024-09-14T22:36:36.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507062+0000", 2024-09-14T22:36:36.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0930000000000002e-06 2024-09-14T22:36:36.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:36.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552439+0000", 2024-09-14T22:36:36.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045377460000000001 2024-09-14T22:36:36.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:36.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552512+0000", 2024-09-14T22:36:36.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2633999999999996e-05 2024-09-14T22:36:36.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:36.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552603+0000", 2024-09-14T22:36:36.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0958999999999994e-05 2024-09-14T22:36:36.130 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.131 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:36.131 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.131 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35306 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1425408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:36.131 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507069+0000", 2024-09-14T22:36:36.131 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 2.5927856500000002, 2024-09-14T22:36:36.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045725259999999997, 2024-09-14T22:36:36.131 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:36.131 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:36.131 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:36.131 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:36.131 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:36.131 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.131 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35306 2024-09-14T22:36:36.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.132 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:36.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:36.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507069+0000", 2024-09-14T22:36:36.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:36.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507069+0000", 2024-09-14T22:36:36.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.133 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:36.133 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507070+0000", 2024-09-14T22:36:36.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0400000000000002e-07 2024-09-14T22:36:36.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.133 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:36.133 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507072+0000", 2024-09-14T22:36:36.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.891e-06 2024-09-14T22:36:36.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.133 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:36.133 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507078+0000", 2024-09-14T22:36:36.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0279999999999997e-06 2024-09-14T22:36:36.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.134 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507082+0000", 2024-09-14T22:36:36.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4680000000000003e-06 2024-09-14T22:36:36.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:36.134 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552632+0000", 2024-09-14T22:36:36.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045550144000000001 2024-09-14T22:36:36.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:36.134 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552707+0000", 2024-09-14T22:36:36.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4148999999999998e-05 2024-09-14T22:36:36.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:36.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552795+0000", 2024-09-14T22:36:36.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8176000000000004e-05 2024-09-14T22:36:36.135 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.135 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:36.135 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.135 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35307 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1454080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:36.135 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:36:36.135 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 2.592767099, 2024-09-14T22:36:36.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045891358, 2024-09-14T22:36:36.136 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:36.136 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:36.136 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:36.136 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:36.136 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:36.136 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.136 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35307 2024-09-14T22:36:36.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.136 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:36.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:36.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:36:36.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:36.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:36:36.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:36.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:36:36.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4700000000000002e-07 2024-09-14T22:36:36.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:36.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507093+0000", 2024-09-14T22:36:36.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9520000000000002e-06 2024-09-14T22:36:36.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.138 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:36.138 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507099+0000", 2024-09-14T22:36:36.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0240000000000003e-06 2024-09-14T22:36:36.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.138 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.138 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507105+0000", 2024-09-14T22:36:36.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5520000000000004e-06 2024-09-14T22:36:36.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.138 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:36.139 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552826+0000", 2024-09-14T22:36:36.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045721494000000001 2024-09-14T22:36:36.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.139 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:36.139 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552896+0000", 2024-09-14T22:36:36.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9519999999999998e-05 2024-09-14T22:36:36.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.139 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:36.139 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552979+0000", 2024-09-14T22:36:36.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3468999999999997e-05 2024-09-14T22:36:36.139 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.139 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:36.140 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.140 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35308 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1482752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:36.140 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:36:36.140 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 2.5927441030000002, 2024-09-14T22:36:36.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046076533000000003, 2024-09-14T22:36:36.140 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:36.140 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:36.140 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:36.140 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:36.140 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:36.140 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.140 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35308 2024-09-14T22:36:36.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.141 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:36.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:36.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:36:36.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:36.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:36:36.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:36.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:36:36.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:36:36.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:36.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507113+0000", 2024-09-14T22:36:36.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9099999999999999e-06 2024-09-14T22:36:36.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:36.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507119+0000", 2024-09-14T22:36:36.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7440000000000004e-06 2024-09-14T22:36:36.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507124+0000", 2024-09-14T22:36:36.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2129999999999999e-06 2024-09-14T22:36:36.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:36.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553007+0000", 2024-09-14T22:36:36.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045882488999999999 2024-09-14T22:36:36.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:36.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553076+0000", 2024-09-14T22:36:36.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9233000000000006e-05 2024-09-14T22:36:36.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:36.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553188+0000", 2024-09-14T22:36:36.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111557 2024-09-14T22:36:36.144 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.144 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:36.144 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.144 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35309 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1511424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:36.144 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:36:36.144 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 2.5927251980000001, 2024-09-14T22:36:36.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046231869000000002, 2024-09-14T22:36:36.144 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:36.144 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:36.144 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:36.145 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:36.145 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:36.145 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.145 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35309 2024-09-14T22:36:36.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.145 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:36.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:36.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:36:36.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:36.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:36:36.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:36.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:36:36.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9099999999999999e-07 2024-09-14T22:36:36.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:36.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507132+0000", 2024-09-14T22:36:36.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8190000000000001e-06 2024-09-14T22:36:36.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:36.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507139+0000", 2024-09-14T22:36:36.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9419999999999998e-06 2024-09-14T22:36:36.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507145+0000", 2024-09-14T22:36:36.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2600000000000002e-06 2024-09-14T22:36:36.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:36.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553225+0000", 2024-09-14T22:36:36.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046079991000000001 2024-09-14T22:36:36.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:36.148 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553284+0000", 2024-09-14T22:36:36.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8638999999999999e-05 2024-09-14T22:36:36.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:36.148 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553362+0000", 2024-09-14T22:36:36.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7826999999999999e-05 2024-09-14T22:36:36.148 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.148 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:36.148 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.148 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35310 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1540096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:36.149 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507151+0000", 2024-09-14T22:36:36.149 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 2.5927037419999999, 2024-09-14T22:36:36.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046444541999999998, 2024-09-14T22:36:36.149 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:36.149 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:36.149 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:36.149 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:36.149 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:36.149 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.149 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35310 2024-09-14T22:36:36.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.149 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:36.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:36.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507151+0000", 2024-09-14T22:36:36.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:36.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507151+0000", 2024-09-14T22:36:36.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:36.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507152+0000", 2024-09-14T22:36:36.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1699999999999999e-07 2024-09-14T22:36:36.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:36.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507154+0000", 2024-09-14T22:36:36.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.902e-06 2024-09-14T22:36:36.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:36.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507159+0000", 2024-09-14T22:36:36.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.823e-06 2024-09-14T22:36:36.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507165+0000", 2024-09-14T22:36:36.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.587e-06 2024-09-14T22:36:36.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:36.152 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553395+0000", 2024-09-14T22:36:36.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046230318999999999 2024-09-14T22:36:36.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:36.152 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553434+0000", 2024-09-14T22:36:36.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8680999999999998e-05 2024-09-14T22:36:36.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:36.152 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553596+0000", 2024-09-14T22:36:36.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016191299999999999 2024-09-14T22:36:36.152 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.152 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:36.153 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.153 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35311 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1568768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:36.153 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:36:36.153 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 2.5926844889999998, 2024-09-14T22:36:36.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046623592999999998, 2024-09-14T22:36:36.153 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:36.153 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:36.153 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:36.153 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:36.153 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:36.153 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.153 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35311 2024-09-14T22:36:36.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.154 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:36.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:36.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:36:36.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:36.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:36:36.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:36.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:36:36.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:36:36.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:36.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507176+0000", 2024-09-14T22:36:36.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3349999999999997e-06 2024-09-14T22:36:36.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:36.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507200+0000", 2024-09-14T22:36:36.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4003e-05 2024-09-14T22:36:36.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507206+0000", 2024-09-14T22:36:36.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.976e-06 2024-09-14T22:36:36.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:36.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553627+0000", 2024-09-14T22:36:36.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046420284999999999 2024-09-14T22:36:36.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:36.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553696+0000", 2024-09-14T22:36:36.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9567999999999995e-05 2024-09-14T22:36:36.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.157 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:36.157 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553794+0000", 2024-09-14T22:36:36.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8060000000000006e-05 2024-09-14T22:36:36.157 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.157 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:36.157 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.157 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35562 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1370112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:36.157 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514584+0000", 2024-09-14T22:36:36.157 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 2.5852711020000001, 2024-09-14T22:36:36.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044805208999999999, 2024-09-14T22:36:36.157 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:36.158 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:36.158 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:36.158 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:36.158 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:36.158 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.158 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35562 2024-09-14T22:36:36.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.158 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:36.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:36.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514584+0000", 2024-09-14T22:36:36.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:36.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514584+0000", 2024-09-14T22:36:36.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:36.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514584+0000", 2024-09-14T22:36:36.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0900000000000002e-07 2024-09-14T22:36:36.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:36.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514587+0000", 2024-09-14T22:36:36.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1739999999999999e-06 2024-09-14T22:36:36.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:36.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514593+0000", 2024-09-14T22:36:36.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3080000000000004e-06 2024-09-14T22:36:36.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514600+0000", 2024-09-14T22:36:36.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6420000000000001e-06 2024-09-14T22:36:36.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:36.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559211+0000", 2024-09-14T22:36:36.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044611454000000002 2024-09-14T22:36:36.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.161 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:36.161 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559296+0000", 2024-09-14T22:36:36.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4676e-05 2024-09-14T22:36:36.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.161 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:36.161 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559389+0000", 2024-09-14T22:36:36.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3546000000000002e-05 2024-09-14T22:36:36.161 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.161 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:36.161 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.162 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35563 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1398784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:36.162 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514606+0000", 2024-09-14T22:36:36.162 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 2.585248773, 2024-09-14T22:36:36.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045029859999999998, 2024-09-14T22:36:36.162 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:36.162 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:36.162 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:36.162 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:36.162 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:36.162 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.162 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35563 2024-09-14T22:36:36.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.163 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:36.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:36.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514606+0000", 2024-09-14T22:36:36.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:36.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514606+0000", 2024-09-14T22:36:36.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:36.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514607+0000", 2024-09-14T22:36:36.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1199999999999998e-07 2024-09-14T22:36:36.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:36.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514609+0000", 2024-09-14T22:36:36.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.199e-06 2024-09-14T22:36:36.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:36.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514615+0000", 2024-09-14T22:36:36.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1260000000000003e-06 2024-09-14T22:36:36.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.165 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.165 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514620+0000", 2024-09-14T22:36:36.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.045e-06 2024-09-14T22:36:36.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.165 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:36.165 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559440+0000", 2024-09-14T22:36:36.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044820169999999999 2024-09-14T22:36:36.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.165 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:36.165 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559524+0000", 2024-09-14T22:36:36.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4124999999999998e-05 2024-09-14T22:36:36.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.166 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:36.166 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559636+0000", 2024-09-14T22:36:36.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111783 2024-09-14T22:36:36.166 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.166 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:36.166 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.166 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35564 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1427456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:36.166 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514627+0000", 2024-09-14T22:36:36.166 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 2.585228034, 2024-09-14T22:36:36.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045232025000000002, 2024-09-14T22:36:36.166 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:36.166 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:36.167 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:36.167 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:36.167 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:36.167 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.167 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35564 2024-09-14T22:36:36.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.167 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:36.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:36.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514627+0000", 2024-09-14T22:36:36.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:36.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514627+0000", 2024-09-14T22:36:36.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:36.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514628+0000", 2024-09-14T22:36:36.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4299999999999998e-07 2024-09-14T22:36:36.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:36.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514630+0000", 2024-09-14T22:36:36.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1579999999999999e-06 2024-09-14T22:36:36.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:36.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514636+0000", 2024-09-14T22:36:36.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5520000000000001e-06 2024-09-14T22:36:36.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514641+0000", 2024-09-14T22:36:36.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6199999999999998e-06 2024-09-14T22:36:36.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:36.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559679+0000", 2024-09-14T22:36:36.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045038474000000002 2024-09-14T22:36:36.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:36.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559763+0000", 2024-09-14T22:36:36.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3357999999999997e-05 2024-09-14T22:36:36.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:36.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559859+0000", 2024-09-14T22:36:36.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6420000000000002e-05 2024-09-14T22:36:36.170 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.170 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:36.170 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.171 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35565 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1456128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:36.171 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514648+0000", 2024-09-14T22:36:36.171 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 2.5852071360000002, 2024-09-14T22:36:36.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045442455999999999, 2024-09-14T22:36:36.171 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:36.171 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:36.171 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:36.171 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:36.171 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:36.171 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.171 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35565 2024-09-14T22:36:36.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.171 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:36.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:36.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514648+0000", 2024-09-14T22:36:36.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:36.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514648+0000", 2024-09-14T22:36:36.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:36.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514648+0000", 2024-09-14T22:36:36.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1399999999999997e-07 2024-09-14T22:36:36.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:36.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514651+0000", 2024-09-14T22:36:36.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.136e-06 2024-09-14T22:36:36.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:36.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514657+0000", 2024-09-14T22:36:36.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.241e-06 2024-09-14T22:36:36.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514662+0000", 2024-09-14T22:36:36.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9690000000000002e-06 2024-09-14T22:36:36.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.174 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:36.174 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559903+0000", 2024-09-14T22:36:36.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045241393999999997 2024-09-14T22:36:36.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.174 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:36.174 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559985+0000", 2024-09-14T22:36:36.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1460999999999998e-05 2024-09-14T22:36:36.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.174 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:36.174 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560090+0000", 2024-09-14T22:36:36.175 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010584100000000001 2024-09-14T22:36:36.175 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.175 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:36.175 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.175 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35566 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1484800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:36.175 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514668+0000", 2024-09-14T22:36:36.175 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 2.5851867120000001, 2024-09-14T22:36:36.175 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045644974999999997, 2024-09-14T22:36:36.175 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:36.175 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:36.175 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:36.175 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:36.175 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:36.176 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.176 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35566 2024-09-14T22:36:36.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.176 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:36.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:36.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514668+0000", 2024-09-14T22:36:36.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:36.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514668+0000", 2024-09-14T22:36:36.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:36.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514669+0000", 2024-09-14T22:36:36.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9099999999999999e-07 2024-09-14T22:36:36.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:36.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514671+0000", 2024-09-14T22:36:36.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3800000000000001e-06 2024-09-14T22:36:36.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:36.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514677+0000", 2024-09-14T22:36:36.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9660000000000001e-06 2024-09-14T22:36:36.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514682+0000", 2024-09-14T22:36:36.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0939999999999998e-06 2024-09-14T22:36:36.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:36.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560133+0000", 2024-09-14T22:36:36.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045450398000000003 2024-09-14T22:36:36.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:36.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560207+0000", 2024-09-14T22:36:36.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4196999999999995e-05 2024-09-14T22:36:36.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:36.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560313+0000", 2024-09-14T22:36:36.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106549 2024-09-14T22:36:36.179 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.179 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:36.179 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.180 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35567 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1513472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:36.180 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:36:36.180 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 2.5851661730000002, 2024-09-14T22:36:36.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045793215999999998, 2024-09-14T22:36:36.180 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:36.180 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:36.180 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:36.180 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:36.180 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:36.180 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.180 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35567 2024-09-14T22:36:36.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.180 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:36.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:36.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:36:36.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:36.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:36:36.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:36.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:36:36.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1899999999999998e-07 2024-09-14T22:36:36.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:36.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514692+0000", 2024-09-14T22:36:36.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2699999999999999e-06 2024-09-14T22:36:36.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:36.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514698+0000", 2024-09-14T22:36:36.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0939999999999996e-06 2024-09-14T22:36:36.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514703+0000", 2024-09-14T22:36:36.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3870000000000003e-06 2024-09-14T22:36:36.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:36.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560353+0000", 2024-09-14T22:36:36.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045649914 2024-09-14T22:36:36.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:36.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560416+0000", 2024-09-14T22:36:36.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3096999999999996e-05 2024-09-14T22:36:36.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:36.184 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560482+0000", 2024-09-14T22:36:36.184 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6035e-05 2024-09-14T22:36:36.184 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.184 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:36.184 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.184 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35568 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1542144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:36.184 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:36:36.184 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 2.5851413750000001, 2024-09-14T22:36:36.184 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045995921000000002, 2024-09-14T22:36:36.184 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:36.184 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:36.184 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:36.185 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:36.185 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:36.185 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.185 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35568 2024-09-14T22:36:36.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.185 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:36.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.185 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:36.185 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:36:36.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.185 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:36.185 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:36:36.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.186 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:36.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:36:36.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4099999999999999e-07 2024-09-14T22:36:36.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.186 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:36.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514720+0000", 2024-09-14T22:36:36.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6869999999999999e-06 2024-09-14T22:36:36.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.186 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:36.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514727+0000", 2024-09-14T22:36:36.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3540000000000002e-06 2024-09-14T22:36:36.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514732+0000", 2024-09-14T22:36:36.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7690000000000004e-06 2024-09-14T22:36:36.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:36.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560523+0000", 2024-09-14T22:36:36.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045791373000000003 2024-09-14T22:36:36.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:36.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560606+0000", 2024-09-14T22:36:36.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2690999999999998e-05 2024-09-14T22:36:36.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:36.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560710+0000", 2024-09-14T22:36:36.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000103606 2024-09-14T22:36:36.188 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.188 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:36.188 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.188 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35569 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1570816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:36.188 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:36:36.188 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 2.5851155659999998, 2024-09-14T22:36:36.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046110630999999999, 2024-09-14T22:36:36.189 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:36.189 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:36.189 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:36.189 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:36.189 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:36.189 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.189 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35569 2024-09-14T22:36:36.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.189 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:36.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:36.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:36:36.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:36.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:36:36.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:36.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:36.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:36:36.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.82e-07 2024-09-14T22:36:36.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:36.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514743+0000", 2024-09-14T22:36:36.191 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.269e-06 2024-09-14T22:36:36.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.191 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:36.191 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514750+0000", 2024-09-14T22:36:36.191 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7340000000000002e-06 2024-09-14T22:36:36.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.191 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.191 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514755+0000", 2024-09-14T22:36:36.191 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7260000000000002e-06 2024-09-14T22:36:36.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560731+0000", 2024-09-14T22:36:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045976377999999998 2024-09-14T22:36:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560777+0000", 2024-09-14T22:36:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5868999999999997e-05 2024-09-14T22:36:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560850+0000", 2024-09-14T22:36:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3172999999999999e-05 2024-09-14T22:36:36.193 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.193 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:36.193 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.193 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:36.193 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:36.193 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:36.194 INFO:teuthology.orchestra.run.smithi001.stdout:949 writesame 0x2f83fb6 thru 0x2f8f4c5 data_size 0x8(0xb510 bytes) 2024-09-14T22:36:36.194 INFO:teuthology.orchestra.run.smithi001.stdout:950 punch from 0x2974f78 to 0x2981756, (0xc7de bytes) 2024-09-14T22:36:36.194 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:36.194 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:36:36.195 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:36:36.196 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:36:36.196 INFO:teuthology.orchestra.run.smithi001.stdout:956 write 0x5d24534 thru 0x5d27120 (0x2bed bytes) 2024-09-14T22:36:36.196 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:36:36.197 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:36:36.197 INFO:teuthology.orchestra.run.smithi001.stdout:957 write 0xc9c0eef thru 0xc9ccd62 (0xbe74 bytes) 2024-09-14T22:36:36.198 INFO:teuthology.orchestra.run.smithi001.stdout:959 read 0xbd9524a thru 0xbd9f279 (0xa030 bytes) 2024-09-14T22:36:36.198 INFO:teuthology.orchestra.run.smithi001.stdout:960 write 0x7b008e9 thru 0x7b086ec (0x7e04 bytes) 2024-09-14T22:36:36.198 INFO:teuthology.orchestra.run.smithi001.stdout:961 punch from 0x960bd54 to 0x9616027, (0xa2d3 bytes) 2024-09-14T22:36:36.198 INFO:teuthology.orchestra.run.smithi001.stdout:963 trunc from 0xc9ccd63 to 0x8a77533 2024-09-14T22:36:36.198 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:36:36.198 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:36:36.199 INFO:teuthology.orchestra.run.smithi001.stdout:964 trunc from 0x8a77533 to 0x8e32609 2024-09-14T22:36:36.199 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:36:36.199 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:36:36.200 INFO:teuthology.orchestra.run.smithi001.stdout:965 read 0xa1d8db thru 0xa28727 (0xae4d bytes) 2024-09-14T22:36:36.200 INFO:teuthology.orchestra.run.smithi001.stdout:966 write 0x3e6aeb9 thru 0x3e78535 (0xd67d bytes) 2024-09-14T22:36:36.200 INFO:teuthology.orchestra.run.smithi001.stdout:967 write 0x863e3bc thru 0x864c11b (0xdd60 bytes) 2024-09-14T22:36:36.200 INFO:teuthology.orchestra.run.smithi001.stdout:969 trunc from 0x8e32609 to 0x60e7413 2024-09-14T22:36:36.200 INFO:teuthology.orchestra.run.smithi001.stdout:970 writesame 0x4a6ef thru 0x5787d data_size 0x1(0xd18f bytes) 2024-09-14T22:36:36.201 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:36.190+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:36.253 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:36.253 INFO:teuthology.orchestra.run.smithi001.stdout:973 writesame 0x5565693 thru 0x55663b0 data_size 0x17(0xd1e bytes) 2024-09-14T22:36:36.256 INFO:teuthology.orchestra.run.smithi001.stdout:974 write 0x128be52 thru 0x128ff3e (0x40ed bytes) 2024-09-14T22:36:36.258 INFO:teuthology.orchestra.run.smithi001.stdout:975 writesame 0x282119 thru 0x290d0e data_size 0x1(0xebf6 bytes) 2024-09-14T22:36:36.297 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:36.291+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:36.318 INFO:teuthology.orchestra.run.smithi001.stdout:976 punch from 0x21b27eb to 0x21c08ca, (0xe0df bytes) 2024-09-14T22:36:36.319 INFO:teuthology.orchestra.run.smithi001.stdout:977 read 0x1cbee02 thru 0x1cc8461 (0x9660 bytes) 2024-09-14T22:36:36.319 INFO:teuthology.orchestra.run.smithi001.stdout:978 write 0xb68d5 thru 0xbafc3 (0x46ef bytes) 2024-09-14T22:36:36.323 INFO:teuthology.orchestra.run.smithi001.stdout:979 writesame 0xae514e thru 0xaee0fd data_size 0x58(0x8fb0 bytes) 2024-09-14T22:36:36.334 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:36.334 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:36.334 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:36:36.334 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:36.349 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:36:36.351 INFO:teuthology.orchestra.run.smithi001.stdout:980 write 0xb4e625a thru 0xb4f2012 (0xbdb9 bytes) 2024-09-14T22:36:36.366 INFO:teuthology.orchestra.run.smithi001.stdout:981 trunc from 0xb4f2013 to 0x6dd1ced 2024-09-14T22:36:36.382 INFO:teuthology.orchestra.run.smithi001.stdout:982 clone 6 order 24 su 131072 sc 11 2024-09-14T22:36:36.396 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:36.390+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:36.497 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:36.491+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:36.513 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:36.513 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:36.514 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:36:36.514 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:36:36.514 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:36.527 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:36:36.598 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:36.590+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:36.696 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:36.697 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:36:36.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:36:36.697 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:36:36.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.697 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6376 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8230912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:36.697 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932578+0000", 2024-09-14T22:36:36.697 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.760878590000001, 2024-09-14T22:36:36.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051558771000000003, 2024-09-14T22:36:36.697 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:36.697 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:36.697 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:36.697 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:36.697 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:36.698 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.698 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6376 2024-09-14T22:36:36.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.698 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:36.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:36.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932578+0000", 2024-09-14T22:36:36.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:36.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932578+0000", 2024-09-14T22:36:36.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:36.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932579+0000", 2024-09-14T22:36:36.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8599999999999996e-07 2024-09-14T22:36:36.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:36.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932597+0000", 2024-09-14T22:36:36.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7771e-05 2024-09-14T22:36:36.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:36.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932618+0000", 2024-09-14T22:36:36.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1026000000000002e-05 2024-09-14T22:36:36.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.700 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.700 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932635+0000", 2024-09-14T22:36:36.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.702e-05 2024-09-14T22:36:36.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.700 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:36.700 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.983951+0000", 2024-09-14T22:36:36.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051316639999999997 2024-09-14T22:36:36.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.700 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:36.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984031+0000", 2024-09-14T22:36:36.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0085000000000003e-05 2024-09-14T22:36:36.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:36.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984137+0000", 2024-09-14T22:36:36.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105243 2024-09-14T22:36:36.701 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.701 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:36.701 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.701 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6377 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8239104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:36.702 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932688+0000", 2024-09-14T22:36:36.702 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.760768559000001, 2024-09-14T22:36:36.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051650696000000003, 2024-09-14T22:36:36.702 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:36.702 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:36.702 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:36.702 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:36.702 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:36.702 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.702 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6377 2024-09-14T22:36:36.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.702 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:36.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:36.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932688+0000", 2024-09-14T22:36:36.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:36.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932688+0000", 2024-09-14T22:36:36.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:36.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932689+0000", 2024-09-14T22:36:36.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4309999999999999e-06 2024-09-14T22:36:36.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:36.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932695+0000", 2024-09-14T22:36:36.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8889999999999999e-06 2024-09-14T22:36:36.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:36.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932713+0000", 2024-09-14T22:36:36.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7416999999999999e-05 2024-09-14T22:36:36.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932726+0000", 2024-09-14T22:36:36.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3145e-05 2024-09-14T22:36:36.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:36.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984181+0000", 2024-09-14T22:36:36.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051455630000000002 2024-09-14T22:36:36.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:36.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984247+0000", 2024-09-14T22:36:36.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5350000000000003e-05 2024-09-14T22:36:36.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:36.706 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984339+0000", 2024-09-14T22:36:36.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1834000000000002e-05 2024-09-14T22:36:36.706 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.706 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:36.706 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.706 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6378 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8247296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:36.706 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932743+0000", 2024-09-14T22:36:36.706 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.760713324999999, 2024-09-14T22:36:36.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051819373000000002, 2024-09-14T22:36:36.706 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:36.706 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:36.706 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:36.707 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:36.707 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:36.707 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.707 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6378 2024-09-14T22:36:36.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.707 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:36.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:36.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932743+0000", 2024-09-14T22:36:36.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:36.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932743+0000", 2024-09-14T22:36:36.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:36.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932745+0000", 2024-09-14T22:36:36.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.637e-06 2024-09-14T22:36:36.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:36.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932750+0000", 2024-09-14T22:36:36.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7859999999999996e-06 2024-09-14T22:36:36.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:36.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932767+0000", 2024-09-14T22:36:36.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7621999999999998e-05 2024-09-14T22:36:36.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932781+0000", 2024-09-14T22:36:36.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3403e-05 2024-09-14T22:36:36.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:36.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984383+0000", 2024-09-14T22:36:36.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051602249000000003 2024-09-14T22:36:36.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:36.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984461+0000", 2024-09-14T22:36:36.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8467000000000006e-05 2024-09-14T22:36:36.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:36.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984562+0000", 2024-09-14T22:36:36.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101209 2024-09-14T22:36:36.710 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.710 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:36.710 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.710 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6379 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8255488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:36.711 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932796+0000", 2024-09-14T22:36:36.711 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.760659934, 2024-09-14T22:36:36.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051961245000000003, 2024-09-14T22:36:36.712 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:36.712 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:36.712 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:36.712 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:36.713 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:36.713 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.713 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6379 2024-09-14T22:36:36.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.713 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:36.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:36.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932796+0000", 2024-09-14T22:36:36.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:36.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932796+0000", 2024-09-14T22:36:36.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:36.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932797+0000", 2024-09-14T22:36:36.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9700000000000005e-07 2024-09-14T22:36:36.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:36.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932823+0000", 2024-09-14T22:36:36.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5273999999999999e-05 2024-09-14T22:36:36.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:36.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932840+0000", 2024-09-14T22:36:36.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7774e-05 2024-09-14T22:36:36.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932862+0000", 2024-09-14T22:36:36.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1957e-05 2024-09-14T22:36:36.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:36.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984603+0000", 2024-09-14T22:36:36.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051740543 2024-09-14T22:36:36.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:36.716 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984669+0000", 2024-09-14T22:36:36.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.567e-05 2024-09-14T22:36:36.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.716 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:36.716 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984758+0000", 2024-09-14T22:36:36.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9129999999999995e-05 2024-09-14T22:36:36.716 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.716 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:36.716 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.716 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6380 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8263680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:36.717 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932879+0000", 2024-09-14T22:36:36.717 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.760577854999999, 2024-09-14T22:36:36.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052102860000000001, 2024-09-14T22:36:36.717 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:36.717 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:36.717 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:36.717 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:36.717 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:36.717 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.717 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6380 2024-09-14T22:36:36.717 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.717 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:36.717 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.717 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:36.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932879+0000", 2024-09-14T22:36:36.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:36.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932879+0000", 2024-09-14T22:36:36.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:36.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932880+0000", 2024-09-14T22:36:36.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.054e-06 2024-09-14T22:36:36.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:36.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932885+0000", 2024-09-14T22:36:36.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1290000000000004e-06 2024-09-14T22:36:36.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:36.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932901+0000", 2024-09-14T22:36:36.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5908000000000001e-05 2024-09-14T22:36:36.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932913+0000", 2024-09-14T22:36:36.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2408999999999999e-05 2024-09-14T22:36:36.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:36.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984800+0000", 2024-09-14T22:36:36.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051886671000000002 2024-09-14T22:36:36.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:36.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984879+0000", 2024-09-14T22:36:36.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9200000000000001e-05 2024-09-14T22:36:36.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:36.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984981+0000", 2024-09-14T22:36:36.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102489 2024-09-14T22:36:36.721 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.721 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:36.721 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.721 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6381 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8271872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:36.721 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932929+0000", 2024-09-14T22:36:36.721 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.760527524, 2024-09-14T22:36:36.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052251484000000001, 2024-09-14T22:36:36.721 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:36.721 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:36.721 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:36.721 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:36.721 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:36.722 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.722 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6381 2024-09-14T22:36:36.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.722 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:36.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.722 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:36.722 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932929+0000", 2024-09-14T22:36:36.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.722 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:36.722 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932929+0000", 2024-09-14T22:36:36.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:36.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932930+0000", 2024-09-14T22:36:36.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3399999999999998e-07 2024-09-14T22:36:36.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:36.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932934+0000", 2024-09-14T22:36:36.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5719999999999996e-06 2024-09-14T22:36:36.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:36.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932961+0000", 2024-09-14T22:36:36.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6437000000000001e-05 2024-09-14T22:36:36.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932973+0000", 2024-09-14T22:36:36.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2097e-05 2024-09-14T22:36:36.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:36.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985021+0000", 2024-09-14T22:36:36.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052047955999999999 2024-09-14T22:36:36.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:36.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985089+0000", 2024-09-14T22:36:36.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7725999999999995e-05 2024-09-14T22:36:36.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:36.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985180+0000", 2024-09-14T22:36:36.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1861999999999996e-05 2024-09-14T22:36:36.725 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.725 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:36.725 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.725 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6382 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8280064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:36.725 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933013+0000", 2024-09-14T22:36:36.726 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.760443362, 2024-09-14T22:36:36.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052391118, 2024-09-14T22:36:36.726 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:36.726 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:36.726 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:36.726 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:36.726 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:36.726 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.726 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6382 2024-09-14T22:36:36.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.726 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:36.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.726 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:36.726 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933013+0000", 2024-09-14T22:36:36.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:36.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933013+0000", 2024-09-14T22:36:36.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:36.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933014+0000", 2024-09-14T22:36:36.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.375e-06 2024-09-14T22:36:36.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:36.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933020+0000", 2024-09-14T22:36:36.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0220000000000001e-06 2024-09-14T22:36:36.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.728 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:36.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933038+0000", 2024-09-14T22:36:36.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7694999999999999e-05 2024-09-14T22:36:36.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.728 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933051+0000", 2024-09-14T22:36:36.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2721999999999999e-05 2024-09-14T22:36:36.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:36.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985224+0000", 2024-09-14T22:36:36.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052172757 2024-09-14T22:36:36.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:36.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985303+0000", 2024-09-14T22:36:36.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9426999999999997e-05 2024-09-14T22:36:36.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:36.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985404+0000", 2024-09-14T22:36:36.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010111999999999999 2024-09-14T22:36:36.729 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.730 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:36.730 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.730 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6383 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8288256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:36.730 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933082+0000", 2024-09-14T22:36:36.730 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.760374382, 2024-09-14T22:36:36.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052525761999999997, 2024-09-14T22:36:36.730 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:36.730 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:36.730 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:36.730 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:36.730 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:36.730 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.731 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6383 2024-09-14T22:36:36.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.731 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:36.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:36.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933082+0000", 2024-09-14T22:36:36.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:36.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933082+0000", 2024-09-14T22:36:36.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:36.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933084+0000", 2024-09-14T22:36:36.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5659999999999999e-06 2024-09-14T22:36:36.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:36.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933089+0000", 2024-09-14T22:36:36.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.271e-06 2024-09-14T22:36:36.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:36.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933105+0000", 2024-09-14T22:36:36.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5685999999999999e-05 2024-09-14T22:36:36.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933118+0000", 2024-09-14T22:36:36.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3188e-05 2024-09-14T22:36:36.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:36.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985448+0000", 2024-09-14T22:36:36.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052330176999999999 2024-09-14T22:36:36.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:36.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985514+0000", 2024-09-14T22:36:36.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6029000000000001e-05 2024-09-14T22:36:36.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:36.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985608+0000", 2024-09-14T22:36:36.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3844999999999993e-05 2024-09-14T22:36:36.734 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.734 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:36.734 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.734 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6384 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8296448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:36.734 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933134+0000", 2024-09-14T22:36:36.734 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.760322023000001, 2024-09-14T22:36:36.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052688835000000003, 2024-09-14T22:36:36.735 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:36.735 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:36.735 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:36.735 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:36.735 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:36.735 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.735 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6384 2024-09-14T22:36:36.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.735 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:36.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:36.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933134+0000", 2024-09-14T22:36:36.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:36.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933134+0000", 2024-09-14T22:36:36.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:36.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933135+0000", 2024-09-14T22:36:36.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.78e-07 2024-09-14T22:36:36.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:36.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933141+0000", 2024-09-14T22:36:36.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1540000000000001e-06 2024-09-14T22:36:36.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:36.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933156+0000", 2024-09-14T22:36:36.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5488000000000001e-05 2024-09-14T22:36:36.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933169+0000", 2024-09-14T22:36:36.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2649e-05 2024-09-14T22:36:36.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:36.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985650+0000", 2024-09-14T22:36:36.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052481112000000003 2024-09-14T22:36:36.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:36.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985724+0000", 2024-09-14T22:36:36.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3953000000000002e-05 2024-09-14T22:36:36.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:36.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985823+0000", 2024-09-14T22:36:36.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9500999999999995e-05 2024-09-14T22:36:36.738 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.738 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:36.739 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.739 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6385 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8304640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:36.739 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933185+0000", 2024-09-14T22:36:36.739 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.760271319999999, 2024-09-14T22:36:36.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052892844000000001, 2024-09-14T22:36:36.739 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:36.739 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:36.739 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:36.739 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:36.739 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:36.739 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.739 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6385 2024-09-14T22:36:36.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.740 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:36.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:36.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933185+0000", 2024-09-14T22:36:36.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:36.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933185+0000", 2024-09-14T22:36:36.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:36.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933186+0000", 2024-09-14T22:36:36.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0139999999999999e-06 2024-09-14T22:36:36.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:36.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933191+0000", 2024-09-14T22:36:36.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7230000000000004e-06 2024-09-14T22:36:36.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:36.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933207+0000", 2024-09-14T22:36:36.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.658e-05 2024-09-14T22:36:36.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933220+0000", 2024-09-14T22:36:36.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2583e-05 2024-09-14T22:36:36.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:36.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985880+0000", 2024-09-14T22:36:36.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052660140000000001 2024-09-14T22:36:36.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:36.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985965+0000", 2024-09-14T22:36:36.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4472e-05 2024-09-14T22:36:36.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.743 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:36.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986078+0000", 2024-09-14T22:36:36.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011333200000000001 2024-09-14T22:36:36.743 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.743 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:36.743 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.743 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6386 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8312832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:36.743 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933236+0000", 2024-09-14T22:36:36.743 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.760220426, 2024-09-14T22:36:36.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053075342999999997, 2024-09-14T22:36:36.743 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:36.743 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:36.744 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:36.744 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:36.744 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:36.744 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.744 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6386 2024-09-14T22:36:36.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.744 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:36.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.744 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:36.744 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933236+0000", 2024-09-14T22:36:36.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.744 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:36.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933236+0000", 2024-09-14T22:36:36.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:36.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933237+0000", 2024-09-14T22:36:36.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6500000000000008e-07 2024-09-14T22:36:36.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:36.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933242+0000", 2024-09-14T22:36:36.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.938e-06 2024-09-14T22:36:36.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:36.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933258+0000", 2024-09-14T22:36:36.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6334e-05 2024-09-14T22:36:36.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933270+0000", 2024-09-14T22:36:36.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1973999999999999e-05 2024-09-14T22:36:36.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:36.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986123+0000", 2024-09-14T22:36:36.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052852769000000001 2024-09-14T22:36:36.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:36.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986204+0000", 2024-09-14T22:36:36.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1458999999999994e-05 2024-09-14T22:36:36.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:36.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986311+0000", 2024-09-14T22:36:36.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010690400000000001 2024-09-14T22:36:36.747 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.747 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:36.747 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.748 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6387 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8321024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:36.748 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933335+0000", 2024-09-14T22:36:36.748 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.760121156, 2024-09-14T22:36:36.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053182363000000003, 2024-09-14T22:36:36.748 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:36.748 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:36.748 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:36.748 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:36.748 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:36.748 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.748 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6387 2024-09-14T22:36:36.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.749 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:36.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:36.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933335+0000", 2024-09-14T22:36:36.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:36.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933335+0000", 2024-09-14T22:36:36.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:36.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933337+0000", 2024-09-14T22:36:36.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3829999999999999e-06 2024-09-14T22:36:36.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:36.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933343+0000", 2024-09-14T22:36:36.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1489999999999999e-06 2024-09-14T22:36:36.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:36.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933361+0000", 2024-09-14T22:36:36.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8105e-05 2024-09-14T22:36:36.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933374+0000", 2024-09-14T22:36:36.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.33e-05 2024-09-14T22:36:36.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:36.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986355+0000", 2024-09-14T22:36:36.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052981053 2024-09-14T22:36:36.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:36.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986424+0000", 2024-09-14T22:36:36.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8864999999999999e-05 2024-09-14T22:36:36.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:36.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986518+0000", 2024-09-14T22:36:36.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3508e-05 2024-09-14T22:36:36.752 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.752 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:36.752 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.752 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6388 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8329216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:36.752 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933390+0000", 2024-09-14T22:36:36.752 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.760066569999999, 2024-09-14T22:36:36.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053371605000000003, 2024-09-14T22:36:36.752 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:36.753 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:36.753 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:36.753 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:36.753 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:36.753 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.753 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6388 2024-09-14T22:36:36.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.753 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:36.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.753 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:36.753 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933390+0000", 2024-09-14T22:36:36.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:36.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933390+0000", 2024-09-14T22:36:36.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:36.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933391+0000", 2024-09-14T22:36:36.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.443e-06 2024-09-14T22:36:36.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:36.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933396+0000", 2024-09-14T22:36:36.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1680000000000003e-06 2024-09-14T22:36:36.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:36.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933412+0000", 2024-09-14T22:36:36.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5302999999999999e-05 2024-09-14T22:36:36.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933439+0000", 2024-09-14T22:36:36.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6747999999999999e-05 2024-09-14T22:36:36.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:36.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986562+0000", 2024-09-14T22:36:36.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053123161000000002 2024-09-14T22:36:36.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:36.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986641+0000", 2024-09-14T22:36:36.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9086999999999997e-05 2024-09-14T22:36:36.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:36.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986761+0000", 2024-09-14T22:36:36.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120695 2024-09-14T22:36:36.756 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.756 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:36.756 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.757 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6389 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8337408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:36.757 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933456+0000", 2024-09-14T22:36:36.757 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.760000719000001, 2024-09-14T22:36:36.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053514182, 2024-09-14T22:36:36.757 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:36.757 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:36.757 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:36.757 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:36.757 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:36.757 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.757 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6389 2024-09-14T22:36:36.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.758 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:36.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.758 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:36.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933456+0000", 2024-09-14T22:36:36.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.758 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:36.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933456+0000", 2024-09-14T22:36:36.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.758 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:36.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933457+0000", 2024-09-14T22:36:36.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0020000000000001e-06 2024-09-14T22:36:36.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:36.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933462+0000", 2024-09-14T22:36:36.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8400000000000002e-06 2024-09-14T22:36:36.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:36.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933477+0000", 2024-09-14T22:36:36.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.592e-05 2024-09-14T22:36:36.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933489+0000", 2024-09-14T22:36:36.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1955e-05 2024-09-14T22:36:36.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:36.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986798+0000", 2024-09-14T22:36:36.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053308263000000002 2024-09-14T22:36:36.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:36.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986871+0000", 2024-09-14T22:36:36.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3518999999999999e-05 2024-09-14T22:36:36.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:36.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986970+0000", 2024-09-14T22:36:36.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8683000000000004e-05 2024-09-14T22:36:36.761 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.761 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:36.761 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.761 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6390 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8345600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:36.761 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933504+0000", 2024-09-14T22:36:36.761 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.759951934, 2024-09-14T22:36:36.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053719579000000003, 2024-09-14T22:36:36.761 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:36.761 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:36.761 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:36.762 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:36.762 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:36.762 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.762 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6390 2024-09-14T22:36:36.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.762 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:36.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:36.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933504+0000", 2024-09-14T22:36:36.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:36.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933504+0000", 2024-09-14T22:36:36.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:36.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933505+0000", 2024-09-14T22:36:36.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8299999999999995e-07 2024-09-14T22:36:36.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:36.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933520+0000", 2024-09-14T22:36:36.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4948e-05 2024-09-14T22:36:36.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.764 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:36.764 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933538+0000", 2024-09-14T22:36:36.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7286e-05 2024-09-14T22:36:36.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.764 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.764 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933550+0000", 2024-09-14T22:36:36.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2223e-05 2024-09-14T22:36:36.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.764 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:36.764 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987028+0000", 2024-09-14T22:36:36.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053478500999999998 2024-09-14T22:36:36.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:36.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987111+0000", 2024-09-14T22:36:36.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3158999999999994e-05 2024-09-14T22:36:36.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:36.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987224+0000", 2024-09-14T22:36:36.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011257899999999999 2024-09-14T22:36:36.765 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.765 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:36.765 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.766 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6391 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8353792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:36.766 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933576+0000", 2024-09-14T22:36:36.766 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.759880489, 2024-09-14T22:36:36.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053858196999999997, 2024-09-14T22:36:36.766 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:36.766 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:36.766 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:36.766 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:36.766 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:36.766 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.766 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6391 2024-09-14T22:36:36.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.766 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:36.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:36.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933576+0000", 2024-09-14T22:36:36.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:36.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933576+0000", 2024-09-14T22:36:36.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:36.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933578+0000", 2024-09-14T22:36:36.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9539999999999998e-06 2024-09-14T22:36:36.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:36.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933584+0000", 2024-09-14T22:36:36.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9030000000000001e-06 2024-09-14T22:36:36.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:36.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933601+0000", 2024-09-14T22:36:36.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7703e-05 2024-09-14T22:36:36.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933614+0000", 2024-09-14T22:36:36.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2330999999999999e-05 2024-09-14T22:36:36.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:36.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987275+0000", 2024-09-14T22:36:36.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053661470000000003 2024-09-14T22:36:36.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:36.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987342+0000", 2024-09-14T22:36:36.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6266000000000006e-05 2024-09-14T22:36:36.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:36.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987434+0000", 2024-09-14T22:36:36.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2570000000000003e-05 2024-09-14T22:36:36.770 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.770 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:36.770 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.770 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6392 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8361984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:36.770 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933648+0000", 2024-09-14T22:36:36.770 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.759808101000001, 2024-09-14T22:36:36.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054032430999999999, 2024-09-14T22:36:36.770 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:36.770 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:36.770 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:36.770 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:36.770 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:36.771 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.771 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6392 2024-09-14T22:36:36.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.771 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:36.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:36.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933648+0000", 2024-09-14T22:36:36.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:36.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933648+0000", 2024-09-14T22:36:36.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:36.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933650+0000", 2024-09-14T22:36:36.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3969999999999999e-06 2024-09-14T22:36:36.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:36.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933655+0000", 2024-09-14T22:36:36.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7690000000000002e-06 2024-09-14T22:36:36.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:36.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933672+0000", 2024-09-14T22:36:36.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6359999999999999e-05 2024-09-14T22:36:36.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933696+0000", 2024-09-14T22:36:36.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4630999999999998e-05 2024-09-14T22:36:36.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:36.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987490+0000", 2024-09-14T22:36:36.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053793256999999997 2024-09-14T22:36:36.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:36.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987572+0000", 2024-09-14T22:36:36.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1897999999999994e-05 2024-09-14T22:36:36.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:36.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987681+0000", 2024-09-14T22:36:36.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109119 2024-09-14T22:36:36.774 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.774 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:36.774 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.775 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6393 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8370176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:36.775 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933713+0000", 2024-09-14T22:36:36.775 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.759743147, 2024-09-14T22:36:36.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054194111000000003, 2024-09-14T22:36:36.775 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:36.775 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:36.775 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:36.775 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:36.775 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:36.775 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.775 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6393 2024-09-14T22:36:36.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.776 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:36.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.776 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:36.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933713+0000", 2024-09-14T22:36:36.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.776 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:36.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933713+0000", 2024-09-14T22:36:36.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.776 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:36.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933714+0000", 2024-09-14T22:36:36.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2109999999999999e-06 2024-09-14T22:36:36.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:36.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933721+0000", 2024-09-14T22:36:36.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2489999999999998e-06 2024-09-14T22:36:36.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:36.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933751+0000", 2024-09-14T22:36:36.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0239999999999998e-05 2024-09-14T22:36:36.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933764+0000", 2024-09-14T22:36:36.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.322e-05 2024-09-14T22:36:36.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:36.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987737+0000", 2024-09-14T22:36:36.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053972539 2024-09-14T22:36:36.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:36.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987813+0000", 2024-09-14T22:36:36.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6635000000000001e-05 2024-09-14T22:36:36.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:36.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987907+0000", 2024-09-14T22:36:36.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4017000000000005e-05 2024-09-14T22:36:36.779 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.779 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:36.779 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.779 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6394 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8378368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:36.779 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933781+0000", 2024-09-14T22:36:36.779 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.759675388, 2024-09-14T22:36:36.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054378276000000003, 2024-09-14T22:36:36.779 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:36.780 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:36.780 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:36.780 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:36.780 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:36.780 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.780 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6394 2024-09-14T22:36:36.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.780 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:36.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:36.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933781+0000", 2024-09-14T22:36:36.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:36.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933781+0000", 2024-09-14T22:36:36.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:36.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933782+0000", 2024-09-14T22:36:36.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9199999999999999e-07 2024-09-14T22:36:36.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:36.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933788+0000", 2024-09-14T22:36:36.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9340000000000003e-06 2024-09-14T22:36:36.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:36.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933803+0000", 2024-09-14T22:36:36.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5362999999999998e-05 2024-09-14T22:36:36.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933816+0000", 2024-09-14T22:36:36.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2456999999999999e-05 2024-09-14T22:36:36.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:36.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987961+0000", 2024-09-14T22:36:36.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054145581999999998 2024-09-14T22:36:36.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:36.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988042+0000", 2024-09-14T22:36:36.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1075000000000006e-05 2024-09-14T22:36:36.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:36.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988159+0000", 2024-09-14T22:36:36.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116873 2024-09-14T22:36:36.783 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.783 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:36.783 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.784 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6395 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8386560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:36.784 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:36:36.784 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.759624494000001, 2024-09-14T22:36:36.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054534379000000001, 2024-09-14T22:36:36.784 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:36.784 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:36.784 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:36.784 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:36.784 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:36.784 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:36.784 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6395 2024-09-14T22:36:36.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.784 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:36.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:36.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:36:36.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:36.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:36:36.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:36.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:36.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933833+0000", 2024-09-14T22:36:36.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3859999999999999e-06 2024-09-14T22:36:36.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:36.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933838+0000", 2024-09-14T22:36:36.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8709999999999996e-06 2024-09-14T22:36:36.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:36.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933854+0000", 2024-09-14T22:36:36.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6189e-05 2024-09-14T22:36:36.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:36.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933871+0000", 2024-09-14T22:36:36.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.626e-05 2024-09-14T22:36:36.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:36.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988198+0000", 2024-09-14T22:36:36.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054327581999999999 2024-09-14T22:36:36.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:36.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988274+0000", 2024-09-14T22:36:36.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5759000000000004e-05 2024-09-14T22:36:36.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:36.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:36.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:36.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988366+0000", 2024-09-14T22:36:36.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2331999999999996e-05 2024-09-14T22:36:36.788 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.788 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:36.788 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.788 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:36.788 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:36.788 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:36.789 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:36:36.790 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:36.782+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:36.891 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:36.883+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:36.923 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:36.923 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:36.923 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:36:36.923 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:36.938 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:36:36.992 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:36.986+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:37.094 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:37.087+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:37.101 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:37.101 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:37.101 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:36:37.101 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:36:37.101 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:37.113 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-14T22:36:37.116 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:36:37.153 INFO:teuthology.orchestra.run.smithi001.stdout:985 write 0xe2ecd37 thru 0xe2f561e (0x88e8 bytes) 2024-09-14T22:36:37.159 INFO:teuthology.orchestra.run.smithi001.stdout:986 read 0xc214913 thru 0xc21db02 (0x91f0 bytes) 2024-09-14T22:36:37.163 INFO:teuthology.orchestra.run.smithi001.stdout:987 trunc from 0xe2f561f to 0xcdf7458 2024-09-14T22:36:37.176 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:37.194 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:37.190+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:37.208 INFO:teuthology.orchestra.run.smithi001.stdout:990 trunc from 0xcdf7458 to 0xdaf6c3 2024-09-14T22:36:37.295 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:37.291+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:37.303 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:37.303 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:36:37.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:36:37.304 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:36:37.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.304 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19590 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7272448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:37.304 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446391+0000", 2024-09-14T22:36:37.304 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.8545306720000001, 2024-09-14T22:36:37.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027116533000000002, 2024-09-14T22:36:37.304 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:37.304 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:37.304 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:37.304 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:37.305 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:37.305 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:37.305 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19590 2024-09-14T22:36:37.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.305 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:37.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:37.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446391+0000", 2024-09-14T22:36:37.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:37.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446391+0000", 2024-09-14T22:36:37.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:37.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446392+0000", 2024-09-14T22:36:37.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4299999999999999e-07 2024-09-14T22:36:37.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:37.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446393+0000", 2024-09-14T22:36:37.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8059999999999999e-06 2024-09-14T22:36:37.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:37.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446399+0000", 2024-09-14T22:36:37.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6740000000000002e-06 2024-09-14T22:36:37.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:37.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446404+0000", 2024-09-14T22:36:37.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6240000000000001e-06 2024-09-14T22:36:37.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:37.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473382+0000", 2024-09-14T22:36:37.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026978716999999999 2024-09-14T22:36:37.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:37.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473446+0000", 2024-09-14T22:36:37.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3971000000000001e-05 2024-09-14T22:36:37.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.308 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:37.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473508+0000", 2024-09-14T22:36:37.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1397999999999997e-05 2024-09-14T22:36:37.308 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.308 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:37.308 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.309 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19591 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7301120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:37.309 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446415+0000", 2024-09-14T22:36:37.309 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.8545073179999996, 2024-09-14T22:36:37.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027255773, 2024-09-14T22:36:37.309 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:37.309 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:37.309 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:37.309 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:37.309 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:37.309 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:37.309 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19591 2024-09-14T22:36:37.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.310 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:37.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:37.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446415+0000", 2024-09-14T22:36:37.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:37.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446415+0000", 2024-09-14T22:36:37.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:37.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446415+0000", 2024-09-14T22:36:37.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8800000000000002e-07 2024-09-14T22:36:37.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:37.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446417+0000", 2024-09-14T22:36:37.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7990000000000001e-06 2024-09-14T22:36:37.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:37.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446423+0000", 2024-09-14T22:36:37.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.066e-06 2024-09-14T22:36:37.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.312 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:37.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446428+0000", 2024-09-14T22:36:37.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5889999999999996e-06 2024-09-14T22:36:37.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.312 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:37.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473535+0000", 2024-09-14T22:36:37.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027107220000000001 2024-09-14T22:36:37.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.312 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:37.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473602+0000", 2024-09-14T22:36:37.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6482999999999993e-05 2024-09-14T22:36:37.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:37.313 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473670+0000", 2024-09-14T22:36:37.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8928000000000001e-05 2024-09-14T22:36:37.313 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.313 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:37.313 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.313 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19592 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7329792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:37.313 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446433+0000", 2024-09-14T22:36:37.313 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.8544886570000001, 2024-09-14T22:36:37.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027394960999999999, 2024-09-14T22:36:37.314 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:37.314 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:37.314 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:37.314 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:37.314 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:37.314 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:37.314 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19592 2024-09-14T22:36:37.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.314 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:37.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:37.314 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446433+0000", 2024-09-14T22:36:37.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:37.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446433+0000", 2024-09-14T22:36:37.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:37.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446434+0000", 2024-09-14T22:36:37.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3600000000000004e-07 2024-09-14T22:36:37.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:37.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446436+0000", 2024-09-14T22:36:37.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.796e-06 2024-09-14T22:36:37.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:37.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446441+0000", 2024-09-14T22:36:37.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7209999999999999e-06 2024-09-14T22:36:37.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:37.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446450+0000", 2024-09-14T22:36:37.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0780000000000002e-06 2024-09-14T22:36:37.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:37.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473705+0000", 2024-09-14T22:36:37.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027254246999999999 2024-09-14T22:36:37.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:37.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473769+0000", 2024-09-14T22:36:37.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4101000000000002e-05 2024-09-14T22:36:37.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:37.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473828+0000", 2024-09-14T22:36:37.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9481999999999997e-05 2024-09-14T22:36:37.317 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.317 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:37.318 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.318 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20100 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7219200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:37.318 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462167+0000", 2024-09-14T22:36:37.318 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.8387549339999998, 2024-09-14T22:36:37.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027082189999999999, 2024-09-14T22:36:37.318 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:37.318 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:37.318 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:37.318 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:37.319 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:37.319 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:37.319 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20100 2024-09-14T22:36:37.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.319 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:37.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:37.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462167+0000", 2024-09-14T22:36:37.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:37.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462167+0000", 2024-09-14T22:36:37.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:37.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462168+0000", 2024-09-14T22:36:37.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8800000000000002e-07 2024-09-14T22:36:37.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:37.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462177+0000", 2024-09-14T22:36:37.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9330000000000007e-06 2024-09-14T22:36:37.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:37.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462183+0000", 2024-09-14T22:36:37.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6200000000000001e-06 2024-09-14T22:36:37.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:37.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462188+0000", 2024-09-14T22:36:37.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1590000000000001e-06 2024-09-14T22:36:37.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:37.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489109+0000", 2024-09-14T22:36:37.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026920881000000001 2024-09-14T22:36:37.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:37.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489181+0000", 2024-09-14T22:36:37.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2035999999999999e-05 2024-09-14T22:36:37.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:37.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489249+0000", 2024-09-14T22:36:37.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7972999999999994e-05 2024-09-14T22:36:37.322 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.322 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:37.322 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.323 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20101 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7247872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:37.323 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462195+0000", 2024-09-14T22:36:37.323 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.838726651, 2024-09-14T22:36:37.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027205994000000001, 2024-09-14T22:36:37.323 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:37.323 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:37.323 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:37.323 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:37.323 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:37.323 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:37.323 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20101 2024-09-14T22:36:37.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.324 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:37.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:37.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462195+0000", 2024-09-14T22:36:37.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:37.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462195+0000", 2024-09-14T22:36:37.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:37.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462196+0000", 2024-09-14T22:36:37.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8000000000000001e-07 2024-09-14T22:36:37.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:37.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462198+0000", 2024-09-14T22:36:37.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0439999999999998e-06 2024-09-14T22:36:37.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:37.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462205+0000", 2024-09-14T22:36:37.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3359999999999997e-06 2024-09-14T22:36:37.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:37.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462210+0000", 2024-09-14T22:36:37.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0649999999999998e-06 2024-09-14T22:36:37.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:37.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489279+0000", 2024-09-14T22:36:37.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027069123 2024-09-14T22:36:37.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:37.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489342+0000", 2024-09-14T22:36:37.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2759e-05 2024-09-14T22:36:37.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:37.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489401+0000", 2024-09-14T22:36:37.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9286999999999997e-05 2024-09-14T22:36:37.327 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.327 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:37.327 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.327 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20102 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7276544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:37.327 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462216+0000", 2024-09-14T22:36:37.327 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.8387057030000005, 2024-09-14T22:36:37.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027352785000000001, 2024-09-14T22:36:37.327 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:37.327 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:37.328 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:37.328 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:37.328 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:37.328 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:37.328 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20102 2024-09-14T22:36:37.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.328 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:37.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:37.328 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462216+0000", 2024-09-14T22:36:37.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:37.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462216+0000", 2024-09-14T22:36:37.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:37.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462217+0000", 2024-09-14T22:36:37.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9100000000000003e-07 2024-09-14T22:36:37.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:37.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462219+0000", 2024-09-14T22:36:37.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9740000000000001e-06 2024-09-14T22:36:37.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:37.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462235+0000", 2024-09-14T22:36:37.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6050999999999999e-05 2024-09-14T22:36:37.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:37.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462247+0000", 2024-09-14T22:36:37.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2245999999999999e-05 2024-09-14T22:36:37.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:37.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489431+0000", 2024-09-14T22:36:37.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027183624999999999 2024-09-14T22:36:37.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:37.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489504+0000", 2024-09-14T22:36:37.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3498999999999995e-05 2024-09-14T22:36:37.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:37.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489569+0000", 2024-09-14T22:36:37.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4698999999999998e-05 2024-09-14T22:36:37.331 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.331 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:37.331 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.331 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20103 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7305216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:37.331 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462256+0000", 2024-09-14T22:36:37.332 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.8386664970000002, 2024-09-14T22:36:37.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027473543, 2024-09-14T22:36:37.332 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:37.332 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:37.332 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:37.332 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:37.332 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:37.332 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:37.332 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20103 2024-09-14T22:36:37.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.332 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:37.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:37.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462256+0000", 2024-09-14T22:36:37.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:37.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462256+0000", 2024-09-14T22:36:37.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:37.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462256+0000", 2024-09-14T22:36:37.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1399999999999997e-07 2024-09-14T22:36:37.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:37.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462259+0000", 2024-09-14T22:36:37.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.694e-06 2024-09-14T22:36:37.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:37.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462266+0000", 2024-09-14T22:36:37.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8940000000000004e-06 2024-09-14T22:36:37.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:37.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462271+0000", 2024-09-14T22:36:37.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0459999999999996e-06 2024-09-14T22:36:37.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:37.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489601+0000", 2024-09-14T22:36:37.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027330027 2024-09-14T22:36:37.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:37.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489663+0000", 2024-09-14T22:36:37.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2807999999999999e-05 2024-09-14T22:36:37.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:37.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489729+0000", 2024-09-14T22:36:37.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5560000000000002e-05 2024-09-14T22:36:37.335 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.335 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:37.336 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.336 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20104 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7333888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:37.336 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462277+0000", 2024-09-14T22:36:37.336 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.838644961, 2024-09-14T22:36:37.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027652614999999998, 2024-09-14T22:36:37.336 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:37.336 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:37.336 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:37.336 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:37.336 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:37.336 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:37.336 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20104 2024-09-14T22:36:37.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.337 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:37.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:37.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462277+0000", 2024-09-14T22:36:37.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:37.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462277+0000", 2024-09-14T22:36:37.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:37.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462277+0000", 2024-09-14T22:36:37.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2800000000000002e-07 2024-09-14T22:36:37.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:37.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462280+0000", 2024-09-14T22:36:37.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2570000000000002e-06 2024-09-14T22:36:37.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:37.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462286+0000", 2024-09-14T22:36:37.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2199999999999997e-06 2024-09-14T22:36:37.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:37.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462291+0000", 2024-09-14T22:36:37.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5689999999999998e-06 2024-09-14T22:36:37.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:37.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489760+0000", 2024-09-14T22:36:37.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027469345999999999 2024-09-14T22:36:37.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:37.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489852+0000", 2024-09-14T22:36:37.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1941999999999995e-05 2024-09-14T22:36:37.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:37.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489930+0000", 2024-09-14T22:36:37.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7853000000000002e-05 2024-09-14T22:36:37.340 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.340 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:37.340 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.340 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20868 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7139328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:37.340 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483611+0000", 2024-09-14T22:36:37.340 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.8173105090000004, 2024-09-14T22:36:37.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027036665000000001, 2024-09-14T22:36:37.340 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:37.340 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:37.340 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:37.340 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:37.340 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:37.341 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:37.341 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20868 2024-09-14T22:36:37.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.341 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:37.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:37.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483611+0000", 2024-09-14T22:36:37.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:37.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483611+0000", 2024-09-14T22:36:37.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:37.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483612+0000", 2024-09-14T22:36:37.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3300000000000003e-07 2024-09-14T22:36:37.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:37.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483614+0000", 2024-09-14T22:36:37.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4480000000000001e-06 2024-09-14T22:36:37.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:37.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483621+0000", 2024-09-14T22:36:37.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9920000000000001e-06 2024-09-14T22:36:37.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:37.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483626+0000", 2024-09-14T22:36:37.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7899999999999999e-06 2024-09-14T22:36:37.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:37.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.510528+0000", 2024-09-14T22:36:37.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026901682999999999 2024-09-14T22:36:37.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:37.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.510587+0000", 2024-09-14T22:36:37.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9137e-05 2024-09-14T22:36:37.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:37.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.510648+0000", 2024-09-14T22:36:37.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1181999999999998e-05 2024-09-14T22:36:37.344 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.344 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:37.344 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.344 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20869 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7168000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:37.344 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483633+0000", 2024-09-14T22:36:37.344 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.817288606, 2024-09-14T22:36:37.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028181490999999999, 2024-09-14T22:36:37.344 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:37.344 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:37.344 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:37.344 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:37.344 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:37.345 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:37.345 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20869 2024-09-14T22:36:37.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.345 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:37.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:37.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483633+0000", 2024-09-14T22:36:37.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:37.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483633+0000", 2024-09-14T22:36:37.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:37.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483634+0000", 2024-09-14T22:36:37.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2599999999999998e-07 2024-09-14T22:36:37.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:37.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483636+0000", 2024-09-14T22:36:37.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1679999999999998e-06 2024-09-14T22:36:37.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:37.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483642+0000", 2024-09-14T22:36:37.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2149999999999998e-06 2024-09-14T22:36:37.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:37.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483647+0000", 2024-09-14T22:36:37.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7679999999999999e-06 2024-09-14T22:36:37.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.347 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:37.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.510694+0000", 2024-09-14T22:36:37.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027047073000000001 2024-09-14T22:36:37.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.347 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:37.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.510771+0000", 2024-09-14T22:36:37.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6814e-05 2024-09-14T22:36:37.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:37.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511815+0000", 2024-09-14T22:36:37.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0010440270000000001 2024-09-14T22:36:37.348 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.348 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:37.348 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.348 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20870 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7196672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:37.348 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483652+0000", 2024-09-14T22:36:37.348 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.8172696039999998, 2024-09-14T22:36:37.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028301248000000001, 2024-09-14T22:36:37.348 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:37.348 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:37.348 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:37.349 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:37.349 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:37.349 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:37.349 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20870 2024-09-14T22:36:37.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.349 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:37.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:37.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483652+0000", 2024-09-14T22:36:37.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:37.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483652+0000", 2024-09-14T22:36:37.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:37.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483653+0000", 2024-09-14T22:36:37.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3099999999999997e-07 2024-09-14T22:36:37.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:37.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483655+0000", 2024-09-14T22:36:37.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.877e-06 2024-09-14T22:36:37.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:37.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483661+0000", 2024-09-14T22:36:37.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6049999999999997e-06 2024-09-14T22:36:37.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:37.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483666+0000", 2024-09-14T22:36:37.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3410000000000002e-06 2024-09-14T22:36:37.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:37.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511854+0000", 2024-09-14T22:36:37.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028187773999999999 2024-09-14T22:36:37.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:37.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511906+0000", 2024-09-14T22:36:37.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2012000000000003e-05 2024-09-14T22:36:37.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:37.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511954+0000", 2024-09-14T22:36:37.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8007999999999998e-05 2024-09-14T22:36:37.352 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.352 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:37.352 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.352 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20871 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7225344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:37.352 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483671+0000", 2024-09-14T22:36:37.353 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.817250681, 2024-09-14T22:36:37.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028405392000000002, 2024-09-14T22:36:37.353 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:37.353 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:37.353 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:37.353 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:37.353 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:37.353 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:37.353 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20871 2024-09-14T22:36:37.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.353 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:37.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.353 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:37.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483671+0000", 2024-09-14T22:36:37.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:37.354 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483671+0000", 2024-09-14T22:36:37.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:37.354 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483672+0000", 2024-09-14T22:36:37.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:36:37.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:37.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483674+0000", 2024-09-14T22:36:37.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3190000000000002e-06 2024-09-14T22:36:37.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:37.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483680+0000", 2024-09-14T22:36:37.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6860000000000003e-06 2024-09-14T22:36:37.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:37.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483684+0000", 2024-09-14T22:36:37.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3769999999999998e-06 2024-09-14T22:36:37.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:37.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511982+0000", 2024-09-14T22:36:37.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028297478000000001 2024-09-14T22:36:37.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.356 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:37.356 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512027+0000", 2024-09-14T22:36:37.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5290000000000002e-05 2024-09-14T22:36:37.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.356 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:37.356 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512077+0000", 2024-09-14T22:36:37.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9876000000000002e-05 2024-09-14T22:36:37.356 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.356 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:37.356 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.357 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20872 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7254016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:37.357 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483689+0000", 2024-09-14T22:36:37.357 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.8172326109999997, 2024-09-14T22:36:37.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028494049, 2024-09-14T22:36:37.357 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:37.357 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:37.357 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:37.357 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:37.357 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:37.357 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:37.357 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20872 2024-09-14T22:36:37.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.357 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:37.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:37.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483689+0000", 2024-09-14T22:36:37.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:37.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483689+0000", 2024-09-14T22:36:37.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:37.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483690+0000", 2024-09-14T22:36:37.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:36:37.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:37.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483692+0000", 2024-09-14T22:36:37.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.778e-06 2024-09-14T22:36:37.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:37.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483697+0000", 2024-09-14T22:36:37.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7069999999999997e-06 2024-09-14T22:36:37.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:37.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483702+0000", 2024-09-14T22:36:37.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4689999999999999e-06 2024-09-14T22:36:37.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:37.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512104+0000", 2024-09-14T22:36:37.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028401948999999999 2024-09-14T22:36:37.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:37.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512142+0000", 2024-09-14T22:36:37.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8114999999999997e-05 2024-09-14T22:36:37.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:37.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512183+0000", 2024-09-14T22:36:37.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1496999999999999e-05 2024-09-14T22:36:37.360 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.360 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:37.360 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.361 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20873 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7282688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:37.361 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483707+0000", 2024-09-14T22:36:37.361 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.8172148970000004, 2024-09-14T22:36:37.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028581662000000001, 2024-09-14T22:36:37.361 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:37.361 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:37.361 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:37.361 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:37.361 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:37.361 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:37.361 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20873 2024-09-14T22:36:37.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.362 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:37.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:37.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483707+0000", 2024-09-14T22:36:37.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:37.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483707+0000", 2024-09-14T22:36:37.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:37.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483707+0000", 2024-09-14T22:36:37.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2500000000000001e-07 2024-09-14T22:36:37.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:37.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483709+0000", 2024-09-14T22:36:37.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.891e-06 2024-09-14T22:36:37.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:37.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483715+0000", 2024-09-14T22:36:37.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7019999999999998e-06 2024-09-14T22:36:37.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:37.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483720+0000", 2024-09-14T22:36:37.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4900000000000002e-06 2024-09-14T22:36:37.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:37.364 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512213+0000", 2024-09-14T22:36:37.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028493622 2024-09-14T22:36:37.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:37.364 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512248+0000", 2024-09-14T22:36:37.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5228000000000002e-05 2024-09-14T22:36:37.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:37.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512289+0000", 2024-09-14T22:36:37.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0404000000000001e-05 2024-09-14T22:36:37.365 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.365 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:37.365 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.365 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20874 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7311360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:37.365 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483725+0000", 2024-09-14T22:36:37.365 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.8171972580000002, 2024-09-14T22:36:37.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028777193999999999, 2024-09-14T22:36:37.365 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:37.365 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:37.365 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:37.366 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:37.366 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:37.366 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:37.366 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20874 2024-09-14T22:36:37.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.366 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:37.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:37.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483725+0000", 2024-09-14T22:36:37.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:37.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483725+0000", 2024-09-14T22:36:37.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:37.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483725+0000", 2024-09-14T22:36:37.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8000000000000001e-07 2024-09-14T22:36:37.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:37.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483727+0000", 2024-09-14T22:36:37.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8920000000000001e-06 2024-09-14T22:36:37.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:37.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483733+0000", 2024-09-14T22:36:37.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6840000000000001e-06 2024-09-14T22:36:37.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:37.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483737+0000", 2024-09-14T22:36:37.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4440000000000001e-06 2024-09-14T22:36:37.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:37.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512326+0000", 2024-09-14T22:36:37.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028588612999999999 2024-09-14T22:36:37.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:37.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512399+0000", 2024-09-14T22:36:37.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2782999999999998e-05 2024-09-14T22:36:37.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:37.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512502+0000", 2024-09-14T22:36:37.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000103398 2024-09-14T22:36:37.369 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.369 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:37.369 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.370 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20912 3.1c 3:3b9f022e:::rbd_data.113c4558bb1.000000000000009a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 497664~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:37.370 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.484800+0000", 2024-09-14T22:36:37.370 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.8161217299999999, 2024-09-14T22:36:37.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045787688999999999, 2024-09-14T22:36:37.370 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:37.370 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:37.370 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:37.370 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:37.370 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:37.370 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:37.370 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20912 2024-09-14T22:36:37.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.370 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:37.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:37.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484800+0000", 2024-09-14T22:36:37.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:37.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484800+0000", 2024-09-14T22:36:37.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:37.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484801+0000", 2024-09-14T22:36:37.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1939999999999999e-06 2024-09-14T22:36:37.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:37.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484809+0000", 2024-09-14T22:36:37.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2989999999999999e-06 2024-09-14T22:36:37.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:37.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484823+0000", 2024-09-14T22:36:37.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3986e-05 2024-09-14T22:36:37.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:37.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484831+0000", 2024-09-14T22:36:37.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4009999999999997e-06 2024-09-14T22:36:37.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:37.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512318+0000", 2024-09-14T22:36:37.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027487208999999999 2024-09-14T22:36:37.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:37.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512450+0000", 2024-09-14T22:36:37.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000131681 2024-09-14T22:36:37.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:36:37.374 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512666+0000", 2024-09-14T22:36:37.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021547 2024-09-14T22:36:37.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.374 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:37.374 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.513788+0000", 2024-09-14T22:36:37.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001122975 2024-09-14T22:36:37.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.374 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:37.374 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.530473+0000", 2024-09-14T22:36:37.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.016684984999999999 2024-09-14T22:36:37.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:37.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.530533+0000", 2024-09-14T22:36:37.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9296999999999998e-05 2024-09-14T22:36:37.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:37.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.530588+0000", 2024-09-14T22:36:37.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5192000000000003e-05 2024-09-14T22:36:37.375 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.375 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:37.375 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.375 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:30512 3.1d 3:b9e08d52:::rbd_data.113cda0183a0.0000000000000008:6 [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:37.376 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.331867+0000", 2024-09-14T22:36:37.376 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.9690547760000001, 2024-09-14T22:36:37.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027901088000000001, 2024-09-14T22:36:37.376 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:37.376 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:37.376 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:37.376 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:37.376 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:37.376 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:37.376 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 30512 2024-09-14T22:36:37.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.376 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:37.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:37.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331867+0000", 2024-09-14T22:36:37.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:37.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331867+0000", 2024-09-14T22:36:37.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:37.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331868+0000", 2024-09-14T22:36:37.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8999999999999996e-07 2024-09-14T22:36:37.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:37.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331871+0000", 2024-09-14T22:36:37.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.163e-06 2024-09-14T22:36:37.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:37.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331880+0000", 2024-09-14T22:36:37.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3449999999999995e-06 2024-09-14T22:36:37.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:37.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331888+0000", 2024-09-14T22:36:37.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5320000000000001e-06 2024-09-14T22:36:37.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.379 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:37.379 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.358443+0000", 2024-09-14T22:36:37.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026554845000000001 2024-09-14T22:36:37.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.379 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:37.379 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.359649+0000", 2024-09-14T22:36:37.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001205991 2024-09-14T22:36:37.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.379 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:37.379 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.359768+0000", 2024-09-14T22:36:37.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119522 2024-09-14T22:36:37.380 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.380 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:37.380 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.380 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:30513 3.1d 3:b9e08d52:::rbd_data.113cda0183a0.0000000000000008:6 [read 456704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:37.380 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.331897+0000", 2024-09-14T22:36:37.380 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.9690245499999999, 2024-09-14T22:36:37.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028098122999999999, 2024-09-14T22:36:37.380 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:37.380 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:37.380 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:37.380 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:37.381 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:37.381 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:37.381 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 30513 2024-09-14T22:36:37.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.381 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:37.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:37.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331897+0000", 2024-09-14T22:36:37.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:37.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331897+0000", 2024-09-14T22:36:37.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:37.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331898+0000", 2024-09-14T22:36:37.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:36:37.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:37.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331901+0000", 2024-09-14T22:36:37.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7889999999999999e-06 2024-09-14T22:36:37.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:37.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331910+0000", 2024-09-14T22:36:37.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.346e-06 2024-09-14T22:36:37.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:37.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331917+0000", 2024-09-14T22:36:37.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2239999999999998e-06 2024-09-14T22:36:37.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:37.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.359807+0000", 2024-09-14T22:36:37.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027889225 2024-09-14T22:36:37.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:37.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.359895+0000", 2024-09-14T22:36:37.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8721000000000007e-05 2024-09-14T22:36:37.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:37.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.359996+0000", 2024-09-14T22:36:37.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010022300000000001 2024-09-14T22:36:37.384 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.384 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:37.385 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.385 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:30514 3.1d 3:b9e08d52:::rbd_data.113cda0183a0.0000000000000008:6 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:37.385 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.331927+0000", 2024-09-14T22:36:37.385 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.9689947370000001, 2024-09-14T22:36:37.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028476330000000001, 2024-09-14T22:36:37.385 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:37.385 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:37.385 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:37.385 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:37.385 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:37.385 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:37.386 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 30514 2024-09-14T22:36:37.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.386 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:37.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:37.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331927+0000", 2024-09-14T22:36:37.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:37.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331927+0000", 2024-09-14T22:36:37.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:37.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331928+0000", 2024-09-14T22:36:37.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5700000000000002e-07 2024-09-14T22:36:37.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:37.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331931+0000", 2024-09-14T22:36:37.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9909999999999998e-06 2024-09-14T22:36:37.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:37.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331940+0000", 2024-09-14T22:36:37.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.312e-06 2024-09-14T22:36:37.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:37.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331948+0000", 2024-09-14T22:36:37.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5630000000000003e-06 2024-09-14T22:36:37.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:37.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360032+0000", 2024-09-14T22:36:37.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028083877 2024-09-14T22:36:37.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:37.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360282+0000", 2024-09-14T22:36:37.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00025062100000000003 2024-09-14T22:36:37.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:37.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360404+0000", 2024-09-14T22:36:37.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000121409 2024-09-14T22:36:37.389 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.389 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:37.389 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.389 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:30515 3.1d 3:b9e08d52:::rbd_data.113cda0183a0.0000000000000008:6 [read 514048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:37.389 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.331957+0000", 2024-09-14T22:36:37.390 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.9689649120000001, 2024-09-14T22:36:37.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028684729999999999, 2024-09-14T22:36:37.390 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:37.390 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:37.390 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:37.390 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:37.390 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:37.390 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:37.390 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 30515 2024-09-14T22:36:37.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.390 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:37.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:37.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331957+0000", 2024-09-14T22:36:37.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:37.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331957+0000", 2024-09-14T22:36:37.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:37.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331958+0000", 2024-09-14T22:36:37.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1200000000000003e-07 2024-09-14T22:36:37.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:37.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331960+0000", 2024-09-14T22:36:37.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6929999999999999e-06 2024-09-14T22:36:37.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:37.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331970+0000", 2024-09-14T22:36:37.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4020000000000008e-06 2024-09-14T22:36:37.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:37.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331977+0000", 2024-09-14T22:36:37.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7660000000000001e-06 2024-09-14T22:36:37.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:37.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360444+0000", 2024-09-14T22:36:37.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028467399000000001 2024-09-14T22:36:37.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:37.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360536+0000", 2024-09-14T22:36:37.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1618000000000003e-05 2024-09-14T22:36:37.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:37.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360642+0000", 2024-09-14T22:36:37.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010624 2024-09-14T22:36:37.394 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.394 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:37.394 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.394 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.394 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:37.394 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:37.395 INFO:teuthology.orchestra.run.smithi001.stdout:991 write 0xe1ef4ff thru 0xe1f101e (0x1b20 bytes) 2024-09-14T22:36:37.395 INFO:teuthology.orchestra.run.smithi001.stdout:992 trunc from 0xe1f101f to 0xd2d713a 2024-09-14T22:36:37.395 INFO:teuthology.orchestra.run.smithi001.stdout:993 punch from 0x9495874 to 0x949bc83, (0x640f bytes) 2024-09-14T22:36:37.395 INFO:teuthology.orchestra.run.smithi001.stdout:995 writesame 0x6a2c991 thru 0x6a3c973 data_size 0x1(0xffe3 bytes) 2024-09-14T22:36:37.396 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:36:37.397 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:37.390+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:37.453 INFO:teuthology.orchestra.run.smithi001.stdout:996 writesame 0xbe91637 thru 0xbe9d06b data_size 0x1(0xba35 bytes) 2024-09-14T22:36:37.487 INFO:teuthology.orchestra.run.smithi001.stdout:997 punch from 0x1abb88 to 0x1b01bb, (0x4633 bytes) 2024-09-14T22:36:37.498 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:37.491+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:37.501 INFO:teuthology.orchestra.run.smithi001.stdout:998 write 0xe7927f3 thru 0xe7a1105 (0xe913 bytes) 2024-09-14T22:36:37.519 INFO:teuthology.orchestra.run.smithi001.stdout:999 trunc from 0xe7a1106 to 0x9ed9325 2024-09-14T22:36:37.546 INFO:teuthology.orchestra.run.smithi001.stdout:1000 write 0x3bba6f2 thru 0x3bc8d17 (0xe626 bytes) 2024-09-14T22:36:37.549 INFO:teuthology.orchestra.run.smithi001.stdout:1001 read 0x26dbdaa thru 0x26de0bc (0x2313 bytes) 2024-09-14T22:36:37.568 INFO:teuthology.orchestra.run.smithi001.stdout:1002 trunc from 0x9ed9325 to 0x32d69e4 2024-09-14T22:36:37.582 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:37.582 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:36:37.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.582 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:46637 3.a 3:5177106c:::rbd_header.113c8642facc:head [call rbd.parent_get in=13b,call rbd.parent_overlap_get in=29b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T22:36:37.582 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:37.580302+0000", 2024-09-14T22:36:37.582 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.00013170900000000001, 2024-09-14T22:36:37.582 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016146699999999999, 2024-09-14T22:36:37.582 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:37.582 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:37.582 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:37.582 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:37.583 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:37.583 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:37.583 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 46637 2024-09-14T22:36:37.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.583 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:37.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.583 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:37.583 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.580302+0000", 2024-09-14T22:36:37.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.583 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:37.583 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.580302+0000", 2024-09-14T22:36:37.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:37.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.580303+0000", 2024-09-14T22:36:37.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.032e-06 2024-09-14T22:36:37.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:37.585 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.580311+0000", 2024-09-14T22:36:37.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0860000000000002e-06 2024-09-14T22:36:37.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:37.585 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.580330+0000", 2024-09-14T22:36:37.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8933999999999999e-05 2024-09-14T22:36:37.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:37.585 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.580350+0000", 2024-09-14T22:36:37.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0292000000000002e-05 2024-09-14T22:36:37.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.586 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:37.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.580417+0000", 2024-09-14T22:36:37.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6723000000000005e-05 2024-09-14T22:36:37.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.586 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:37.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.580464+0000", 2024-09-14T22:36:37.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7335e-05 2024-09-14T22:36:37.586 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.586 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:37.587 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.587 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.587 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T22:36:37.587 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 1 2024-09-14T22:36:37.587 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:37.596 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:36:37.598 INFO:teuthology.orchestra.run.smithi001.stdout:1003 trunc from 0x32d69e4 to 0x88cbf6f 2024-09-14T22:36:37.599 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:37.594+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:37.626 INFO:teuthology.orchestra.run.smithi001.stdout:1004 trunc from 0x88cbf6f to 0x2520f69 2024-09-14T22:36:37.683 INFO:teuthology.orchestra.run.smithi001.stdout:1006 writesame 0xe05422d thru 0xe05f1a3 data_size 0x5d(0xaf77 bytes) 2024-09-14T22:36:37.697 INFO:teuthology.orchestra.run.smithi001.stdout:1007 trunc from 0xe05f1a4 to 0x8899711 2024-09-14T22:36:37.700 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:37.695+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:37.710 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:37.710 INFO:teuthology.orchestra.run.smithi001.stdout:1009 writesame 0x2d25104 thru 0x2d2601b data_size 0x2a(0xf18 bytes) 2024-09-14T22:36:37.713 INFO:teuthology.orchestra.run.smithi001.stdout:1010 write 0x3ba5557 thru 0x3ba8bbe (0x3668 bytes) 2024-09-14T22:36:37.717 INFO:teuthology.orchestra.run.smithi001.stdout:1012 punch from 0x9a1f05 to 0x9a78e9, (0x59e4 bytes) 2024-09-14T22:36:37.717 INFO:teuthology.orchestra.run.smithi001.stdout:1013 read 0x1c8c4ea thru 0x1c94d75 (0x888c bytes) 2024-09-14T22:36:37.731 INFO:teuthology.orchestra.run.smithi001.stdout:1014 write 0x9edd12f thru 0x9eecccb (0xfb9d bytes) 2024-09-14T22:36:37.736 INFO:teuthology.orchestra.run.smithi001.stdout:1015 read 0x5654cb6 thru 0x565c5e4 (0x792f bytes) 2024-09-14T22:36:37.737 INFO:teuthology.orchestra.run.smithi001.stdout:1016 write 0x88d10c5 thru 0x88d3c13 (0x2b4f bytes) 2024-09-14T22:36:37.741 INFO:teuthology.orchestra.run.smithi001.stdout:1018 write 0x3031d95 thru 0x3035caa (0x3f16 bytes) 2024-09-14T22:36:37.762 INFO:teuthology.orchestra.run.smithi001.stdout:1019 trunc from 0x9eecccc to 0x27d4fe8 2024-09-14T22:36:37.764 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:37.764 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:37.764 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:36:37.764 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:36:37.764 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:37.778 INFO:teuthology.orchestra.run.smithi001.stdout:1024 write 0x100909d thru 0x10176c7 (0xe62b bytes) 2024-09-14T22:36:37.780 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:36:37.800 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:37.794+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:37.803 INFO:teuthology.orchestra.run.smithi001.stdout:1025 write 0x6827689 thru 0x6836772 (0xf0ea bytes) 2024-09-14T22:36:37.832 INFO:teuthology.orchestra.run.smithi001.stdout:1026 write 0xb949340 thru 0xb9542e4 (0xafa5 bytes) 2024-09-14T22:36:37.838 INFO:teuthology.orchestra.run.smithi001.stdout:1027 write 0x24f20f7 thru 0x24fa923 (0x882d bytes) 2024-09-14T22:36:37.841 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T22:36:37.842 INFO:teuthology.orchestra.run.smithi001.stdout:1028 writesame 0x841bf68 thru 0x842186c data_size 0x1(0x5905 bytes) 2024-09-14T22:36:37.853 INFO:teuthology.orchestra.run.smithi001.stdout:1029 read 0x31ab0fd thru 0x31b8136 (0xd03a bytes) 2024-09-14T22:36:37.854 INFO:teuthology.orchestra.run.smithi001.stdout:1030 read 0x594c0d thru 0x599635 (0x4a29 bytes) 2024-09-14T22:36:37.855 INFO:teuthology.orchestra.run.smithi001.stdout:1031 writesame 0x20a4a19 thru 0x20aabc5 data_size 0x5(0x61ad bytes) 2024-09-14T22:36:37.860 INFO:teuthology.orchestra.run.smithi001.stdout:1032 writesame 0x36f209f thru 0x36f9065 data_size 0x3b(0x6fc7 bytes) 2024-09-14T22:36:37.864 INFO:teuthology.orchestra.run.smithi001.stdout:1033 read 0x1c7973a thru 0x1c7de9d (0x4764 bytes) 2024-09-14T22:36:37.864 INFO:teuthology.orchestra.run.smithi001.stdout:1035 punch from 0x74cc8f0 to 0x74dc018, (0xf728 bytes) 2024-09-14T22:36:37.864 INFO:teuthology.orchestra.run.smithi001.stdout:1036 write 0xa04fd02 thru 0xa054a5c (0x4d5b bytes) 2024-09-14T22:36:37.867 INFO:teuthology.orchestra.run.smithi001.stdout:1037 read 0x9804f2c thru 0x981179c (0xc871 bytes) 2024-09-14T22:36:37.873 INFO:teuthology.orchestra.run.smithi001.stdout:1038 trunc from 0xb9542e5 to 0x9d10d9c 2024-09-14T22:36:37.892 INFO:teuthology.orchestra.run.smithi001.stdout:1039 read 0x145a681 thru 0x146a534 (0xfeb4 bytes) 2024-09-14T22:36:37.893 INFO:teuthology.orchestra.run.smithi001.stdout:1040 read 0x588cbbf thru 0x588da47 (0xe89 bytes) 2024-09-14T22:36:37.893 INFO:teuthology.orchestra.run.smithi001.stdout:1041 write 0x643b8c2 thru 0x644a379 (0xeab8 bytes) 2024-09-14T22:36:37.900 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:37.894+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:37.902 INFO:teuthology.orchestra.run.smithi001.stdout:1042 trunc from 0x9d10d9c to 0x76b143c 2024-09-14T22:36:37.912 INFO:teuthology.orchestra.run.smithi001.stdout:1044 read 0xceab43 thru 0xcecb13 (0x1fd1 bytes) 2024-09-14T22:36:37.913 INFO:teuthology.orchestra.run.smithi001.stdout:1047 write 0x315143f thru 0x3159449 (0x800b bytes) 2024-09-14T22:36:37.916 INFO:teuthology.orchestra.run.smithi001.stdout:1048 read 0x108a229 thru 0x10970a2 (0xce7a bytes) 2024-09-14T22:36:37.916 INFO:teuthology.orchestra.run.smithi001.stdout:1051 read 0x3b20c85 thru 0x3b24a2f (0x3dab bytes) 2024-09-14T22:36:37.921 INFO:teuthology.orchestra.run.smithi001.stdout:1052 trunc from 0x76b143c to 0x8fca93a 2024-09-14T22:36:37.928 INFO:teuthology.orchestra.run.smithi001.stdout:1053 read 0x4cf28f4 thru 0x4cf2d36 (0x443 bytes) 2024-09-14T22:36:37.939 INFO:teuthology.orchestra.run.smithi001.stdout:1054 writesame 0x9d3af3b thru 0x9d3cbe4 data_size 0x3(0x1caa bytes) 2024-09-14T22:36:37.941 INFO:teuthology.orchestra.run.smithi001.stdout:1055 writesame 0x876833e thru 0x87684a6 data_size 0x1(0x169 bytes) 2024-09-14T22:36:37.944 INFO:teuthology.orchestra.run.smithi001.stdout:1057 punch from 0x3202980 to 0x3202e3c, (0x4bc bytes) 2024-09-14T22:36:37.944 INFO:teuthology.orchestra.run.smithi001.stdout:1058 read 0x1f10e5 thru 0x1f8b18 (0x7a34 bytes) 2024-09-14T22:36:37.944 INFO:teuthology.orchestra.run.smithi001.stdout:1059 clone 7 order 21 su 8192 sc 5 2024-09-14T22:36:37.961 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:37.961 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:36:37.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:36:37.961 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:36:37.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.961 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:1241 3.10 3:0bfe6369:::rbd_object_map.113c3b74a67f.0000000000000004:head [writefull 0~39] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:37.961 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:22.744905+0000", 2024-09-14T22:36:37.961 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 15.210943136999999, 2024-09-14T22:36:37.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0063495579999999999, 2024-09-14T22:36:37.961 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:37.961 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:37.962 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:37.962 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:37.962 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:37.962 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:37.962 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 1241 2024-09-14T22:36:37.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.962 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:37.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:37.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.744905+0000", 2024-09-14T22:36:37.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:37.963 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.744905+0000", 2024-09-14T22:36:37.963 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.963 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.963 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.963 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:37.963 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.744908+0000", 2024-09-14T22:36:37.963 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2899999999999998e-06 2024-09-14T22:36:37.963 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.963 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.963 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:37.963 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.744923+0000", 2024-09-14T22:36:37.963 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4797000000000001e-05 2024-09-14T22:36:37.963 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.963 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.963 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:37.964 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.744958+0000", 2024-09-14T22:36:37.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5015999999999998e-05 2024-09-14T22:36:37.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:37.964 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.744974+0000", 2024-09-14T22:36:37.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.615e-05 2024-09-14T22:36:37.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:37.964 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.745133+0000", 2024-09-14T22:36:37.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000159145 2024-09-14T22:36:37.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op must wait for map", 2024-09-14T22:36:37.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.745145+0000", 2024-09-14T22:36:37.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1457000000000001e-05 2024-09-14T22:36:37.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:37.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.748491+0000", 2024-09-14T22:36:37.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0033466139999999999 2024-09-14T22:36:37.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:37.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.748681+0000", 2024-09-14T22:36:37.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000189638 2024-09-14T22:36:37.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:37.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.748893+0000", 2024-09-14T22:36:37.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000211855 2024-09-14T22:36:37.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:37.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.750340+0000", 2024-09-14T22:36:37.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001447126 2024-09-14T22:36:37.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:37.967 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.750975+0000", 2024-09-14T22:36:37.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00063514799999999998 2024-09-14T22:36:37.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.967 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:37.967 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.751131+0000", 2024-09-14T22:36:37.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015583900000000001 2024-09-14T22:36:37.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.967 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:37.967 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.751254+0000", 2024-09-14T22:36:37.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000123483 2024-09-14T22:36:37.967 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.967 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:37.968 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.968 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.968 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.968 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2151 3.a 3:5478be22:::rbd_data.113cde4a8b6b.0000000000000143:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:37.968 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.514710+0000", 2024-09-14T22:36:37.968 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.441137789999999, 2024-09-14T22:36:37.968 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012174763, 2024-09-14T22:36:37.968 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:37.968 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:37.968 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:37.968 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:37.968 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:37.968 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:37.969 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2151 2024-09-14T22:36:37.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.969 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:37.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:37.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.514710+0000", 2024-09-14T22:36:37.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:37.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.514710+0000", 2024-09-14T22:36:37.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:37.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.514712+0000", 2024-09-14T22:36:37.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3649999999999998e-06 2024-09-14T22:36:37.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:37.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.514723+0000", 2024-09-14T22:36:37.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0936e-05 2024-09-14T22:36:37.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:37.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.514747+0000", 2024-09-14T22:36:37.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3288000000000001e-05 2024-09-14T22:36:37.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:37.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.514756+0000", 2024-09-14T22:36:37.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6949999999999995e-06 2024-09-14T22:36:37.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:37.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.514873+0000", 2024-09-14T22:36:37.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011657700000000001 2024-09-14T22:36:37.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:37.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.515139+0000", 2024-09-14T22:36:37.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026613100000000002 2024-09-14T22:36:37.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:36:37.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.517580+0000", 2024-09-14T22:36:37.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002441365 2024-09-14T22:36:37.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:37.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.526706+0000", 2024-09-14T22:36:37.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0091256459999999994 2024-09-14T22:36:37.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:37.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.526762+0000", 2024-09-14T22:36:37.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5489999999999999e-05 2024-09-14T22:36:37.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:37.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.526815+0000", 2024-09-14T22:36:37.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3458000000000002e-05 2024-09-14T22:36:37.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:37.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.526885+0000", 2024-09-14T22:36:37.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9812000000000002e-05 2024-09-14T22:36:37.974 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.974 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:37.974 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.974 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:5363 3.10 3:0bb60ae6:::rbd_data.113cde4a8b6b.0000000000000094:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:37.974 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.907462+0000", 2024-09-14T22:36:37.974 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.048386189, 2024-09-14T22:36:37.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0080656949999999995, 2024-09-14T22:36:37.974 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:37.974 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:37.974 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:37.974 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:37.975 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:37.975 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:37.975 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 5363 2024-09-14T22:36:37.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.975 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:37.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:37.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.907462+0000", 2024-09-14T22:36:37.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:37.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.907462+0000", 2024-09-14T22:36:37.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.976 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:37.976 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.907465+0000", 2024-09-14T22:36:37.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.393e-06 2024-09-14T22:36:37.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:37.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.907479+0000", 2024-09-14T22:36:37.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3920000000000001e-05 2024-09-14T22:36:37.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:37.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.907521+0000", 2024-09-14T22:36:37.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2525999999999999e-05 2024-09-14T22:36:37.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:37.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.907546+0000", 2024-09-14T22:36:37.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4783e-05 2024-09-14T22:36:37.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:37.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.907769+0000", 2024-09-14T22:36:37.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022249499999999999 2024-09-14T22:36:37.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op must wait for map", 2024-09-14T22:36:37.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.907792+0000", 2024-09-14T22:36:37.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2912999999999999e-05 2024-09-14T22:36:37.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:37.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.913820+0000", 2024-09-14T22:36:37.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.006028828 2024-09-14T22:36:37.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.979 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:37.979 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.913959+0000", 2024-09-14T22:36:37.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013830900000000001 2024-09-14T22:36:37.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.979 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:37.979 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.914169+0000", 2024-09-14T22:36:37.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000209898 2024-09-14T22:36:37.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.979 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:37.979 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.915324+0000", 2024-09-14T22:36:37.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0011553640000000001 2024-09-14T22:36:37.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.980 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:37.980 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.915381+0000", 2024-09-14T22:36:37.980 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7086999999999997e-05 2024-09-14T22:36:37.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.980 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:37.980 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.915456+0000", 2024-09-14T22:36:37.980 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5038999999999998e-05 2024-09-14T22:36:37.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.980 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:37.980 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.915527+0000", 2024-09-14T22:36:37.980 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.114e-05 2024-09-14T22:36:37.980 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.981 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:37.981 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.981 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4412.0:25388 3.1e e36/23 3:7ee90710:::rbd_object_map.113c4558bb1:head v 36'122, mlcod=36'118)", 2024-09-14T22:36:37.981 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.849766+0000", 2024-09-14T22:36:37.981 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.1060818320000001, 2024-09-14T22:36:37.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0056330360000000001, 2024-09-14T22:36:37.981 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:37.981 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:37.981 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:37.981 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:37.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.981 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:37.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.849766+0000", 2024-09-14T22:36:37.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:37.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.849766+0000", 2024-09-14T22:36:37.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:37.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.849766+0000", 2024-09-14T22:36:37.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5600000000000001e-07 2024-09-14T22:36:37.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:37.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.849769+0000", 2024-09-14T22:36:37.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1149999999999998e-06 2024-09-14T22:36:37.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:37.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.849776+0000", 2024-09-14T22:36:37.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3729999999999998e-06 2024-09-14T22:36:37.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:37.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.849782+0000", 2024-09-14T22:36:37.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7849999999999997e-06 2024-09-14T22:36:37.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.984 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:37.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.849822+0000", 2024-09-14T22:36:37.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0584999999999999e-05 2024-09-14T22:36:37.984 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.984 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:37.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.849833+0000", 2024-09-14T22:36:37.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1280999999999999e-05 2024-09-14T22:36:37.984 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.984 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:37.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.855376+0000", 2024-09-14T22:36:37.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0055428370000000001 2024-09-14T22:36:37.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.985 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.985 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:37.985 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.855399+0000", 2024-09-14T22:36:37.985 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2704000000000001e-05 2024-09-14T22:36:37.985 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.985 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:37.985 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.986 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.986 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4412.0:25389 3.1e e36/23 3:7ee90710:::rbd_object_map.113c4558bb1:head v 36'123, mlcod=36'118)", 2024-09-14T22:36:37.986 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.850409+0000", 2024-09-14T22:36:37.986 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.1054386550000004, 2024-09-14T22:36:37.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.005003469, 2024-09-14T22:36:37.986 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:37.986 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:37.986 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:37.986 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:37.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.986 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:37.986 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.850409+0000", 2024-09-14T22:36:37.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:37.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.850409+0000", 2024-09-14T22:36:37.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:37.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.850410+0000", 2024-09-14T22:36:37.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6199999999999998e-07 2024-09-14T22:36:37.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.988 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:37.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.850413+0000", 2024-09-14T22:36:37.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8789999999999999e-06 2024-09-14T22:36:37.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.988 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:37.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.850419+0000", 2024-09-14T22:36:37.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7150000000000001e-06 2024-09-14T22:36:37.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.988 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:37.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.850423+0000", 2024-09-14T22:36:37.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3009999999999998e-06 2024-09-14T22:36:37.989 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.989 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.989 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:37.989 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.850458+0000", 2024-09-14T22:36:37.989 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5522000000000003e-05 2024-09-14T22:36:37.989 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.989 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.989 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:37.989 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.850470+0000", 2024-09-14T22:36:37.989 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1467999999999999e-05 2024-09-14T22:36:37.989 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.989 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.989 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:37.989 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.855400+0000", 2024-09-14T22:36:37.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0049306619999999997 2024-09-14T22:36:37.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.990 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:37.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.855413+0000", 2024-09-14T22:36:37.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2459999999999999e-05 2024-09-14T22:36:37.990 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.990 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:37.990 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.991 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.991 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4412.0:25431 3.1e e36/23 3:7ee90710:::rbd_object_map.113c4558bb1:head v 36'137, mlcod=36'134)", 2024-09-14T22:36:37.991 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.869784+0000", 2024-09-14T22:36:37.991 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.086063856, 2024-09-14T22:36:37.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00505085, 2024-09-14T22:36:37.991 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:37.991 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:37.991 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:37.991 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:37.991 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.991 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:37.991 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.869784+0000", 2024-09-14T22:36:37.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:37.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.869784+0000", 2024-09-14T22:36:37.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:37.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.869785+0000", 2024-09-14T22:36:37.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.37e-07 2024-09-14T22:36:37.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.993 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:37.993 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.869796+0000", 2024-09-14T22:36:37.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0866999999999999e-05 2024-09-14T22:36:37.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.993 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.993 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:37.993 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.869808+0000", 2024-09-14T22:36:37.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2128e-05 2024-09-14T22:36:37.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.993 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.993 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:37.993 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.869815+0000", 2024-09-14T22:36:37.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0650000000000001e-06 2024-09-14T22:36:37.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:37.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.869864+0000", 2024-09-14T22:36:37.994 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9168000000000001e-05 2024-09-14T22:36:37.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:37.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.869883+0000", 2024-09-14T22:36:37.994 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9091999999999999e-05 2024-09-14T22:36:37.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:37.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.874803+0000", 2024-09-14T22:36:37.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0049196800000000001 2024-09-14T22:36:37.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.995 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:37.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.874835+0000", 2024-09-14T22:36:37.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2113000000000002e-05 2024-09-14T22:36:37.995 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.995 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:37.995 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:37.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.995 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4412.0:29460 3.1e e36/23 3:7ee90710:::rbd_object_map.113c4558bb1:head v 36'251, mlcod=36'246)", 2024-09-14T22:36:37.995 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.268560+0000", 2024-09-14T22:36:37.995 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.6872877800000001, 2024-09-14T22:36:37.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0062238609999999998, 2024-09-14T22:36:37.996 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:37.996 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:37.996 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:37.996 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:37.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.996 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:37.996 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.268560+0000", 2024-09-14T22:36:37.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.996 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.996 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:37.996 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.268560+0000", 2024-09-14T22:36:37.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:37.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.997 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:37.997 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.268561+0000", 2024-09-14T22:36:37.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1999999999999998e-07 2024-09-14T22:36:37.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.997 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:37.997 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.268568+0000", 2024-09-14T22:36:37.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1069999999999999e-06 2024-09-14T22:36:37.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:37.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.268585+0000", 2024-09-14T22:36:37.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7390000000000001e-05 2024-09-14T22:36:37.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:37.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.268595+0000", 2024-09-14T22:36:37.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5929999999999995e-06 2024-09-14T22:36:37.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:37.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.268647+0000", 2024-09-14T22:36:37.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1570000000000003e-05 2024-09-14T22:36:37.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:37.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.268664+0000", 2024-09-14T22:36:37.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7946000000000001e-05 2024-09-14T22:36:37.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:37.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.274739+0000", 2024-09-14T22:36:37.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0060749549999999999 2024-09-14T22:36:37.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:37.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:37.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:37.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.274784+0000", 2024-09-14T22:36:37.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4480000000000001e-05 2024-09-14T22:36:38.000 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:38.000 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:38.000 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:38.000 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.000 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4412.0:29461 3.1e e36/23 3:7ee90710:::rbd_object_map.113c4558bb1:head v 36'252, mlcod=36'248)", 2024-09-14T22:36:38.000 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.269830+0000", 2024-09-14T22:36:38.000 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.6860181320000001, 2024-09-14T22:36:38.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0055875689999999997, 2024-09-14T22:36:38.000 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:38.000 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:38.000 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:38.000 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:38.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:38.001 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.269830+0000", 2024-09-14T22:36:38.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:38.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:38.001 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.269830+0000", 2024-09-14T22:36:38.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:38.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:38.001 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.269830+0000", 2024-09-14T22:36:38.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9400000000000005e-07 2024-09-14T22:36:38.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.002 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.002 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:38.002 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.269834+0000", 2024-09-14T22:36:38.002 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0369999999999996e-06 2024-09-14T22:36:38.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.002 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.002 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:38.002 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.269843+0000", 2024-09-14T22:36:38.002 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4940000000000003e-06 2024-09-14T22:36:38.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.002 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.002 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:38.002 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.269847+0000", 2024-09-14T22:36:38.002 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6460000000000001e-06 2024-09-14T22:36:38.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:38.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.269890+0000", 2024-09-14T22:36:38.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2527999999999997e-05 2024-09-14T22:36:38.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:38.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.269910+0000", 2024-09-14T22:36:38.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0177999999999999e-05 2024-09-14T22:36:38.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:38.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.275380+0000", 2024-09-14T22:36:38.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0054699279999999998 2024-09-14T22:36:38.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:38.004 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.275417+0000", 2024-09-14T22:36:38.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7163999999999999e-05 2024-09-14T22:36:38.004 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:38.004 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:38.004 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:38.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.004 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:29488 3.a 3:55944348:::rbd_data.113c4558bb1.00000000000000c9:head [write 303104~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:38.005 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.281270+0000", 2024-09-14T22:36:38.005 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.6745783080000001, 2024-09-14T22:36:38.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0065184809999999996, 2024-09-14T22:36:38.005 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:38.005 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:38.005 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:38.005 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:38.005 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:38.005 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:38.005 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29488 2024-09-14T22:36:38.005 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.005 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:38.005 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.005 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:38.005 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.281270+0000", 2024-09-14T22:36:38.006 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:38.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.006 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.006 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:38.006 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.281270+0000", 2024-09-14T22:36:38.006 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:38.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.006 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.006 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:38.006 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.281270+0000", 2024-09-14T22:36:38.006 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9400000000000004e-07 2024-09-14T22:36:38.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.006 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.006 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:38.006 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.281276+0000", 2024-09-14T22:36:38.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1630000000000004e-06 2024-09-14T22:36:38.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.007 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:38.007 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.281281+0000", 2024-09-14T22:36:38.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6500000000000001e-06 2024-09-14T22:36:38.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.007 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:38.007 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.281285+0000", 2024-09-14T22:36:38.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3009999999999998e-06 2024-09-14T22:36:38.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.007 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:38.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.281754+0000", 2024-09-14T22:36:38.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00046994099999999998 2024-09-14T22:36:38.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:38.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.281864+0000", 2024-09-14T22:36:38.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010964900000000001 2024-09-14T22:36:38.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:36:38.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.282018+0000", 2024-09-14T22:36:38.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015433600000000001 2024-09-14T22:36:38.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:38.009 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.285885+0000", 2024-09-14T22:36:38.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0038670309999999999 2024-09-14T22:36:38.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:38.009 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.287716+0000", 2024-09-14T22:36:38.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0018305769999999999 2024-09-14T22:36:38.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:38.009 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.287763+0000", 2024-09-14T22:36:38.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6601000000000003e-05 2024-09-14T22:36:38.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.010 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:38.010 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.287788+0000", 2024-09-14T22:36:38.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5338e-05 2024-09-14T22:36:38.010 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:38.010 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:38.010 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:38.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.010 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:29494 3.a 3:5771f379:::rbd_data.113c4558bb1.00000000000000cf:head [write 303104~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:38.010 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.281513+0000", 2024-09-14T22:36:38.010 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.6743346070000005, 2024-09-14T22:36:38.011 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0065160160000000003, 2024-09-14T22:36:38.011 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:38.011 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:38.011 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:38.011 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:38.011 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:38.011 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:38.011 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29494 2024-09-14T22:36:38.011 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.011 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:38.011 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.011 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:38.011 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.281513+0000", 2024-09-14T22:36:38.011 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:38.011 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:38.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.281513+0000", 2024-09-14T22:36:38.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:38.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:38.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.281515+0000", 2024-09-14T22:36:38.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.342e-06 2024-09-14T22:36:38.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:38.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.281522+0000", 2024-09-14T22:36:38.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4270000000000002e-06 2024-09-14T22:36:38.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:38.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.281538+0000", 2024-09-14T22:36:38.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6138e-05 2024-09-14T22:36:38.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:38.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.281549+0000", 2024-09-14T22:36:38.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0713999999999999e-05 2024-09-14T22:36:38.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:38.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.282673+0000", 2024-09-14T22:36:38.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0011243990000000001 2024-09-14T22:36:38.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:38.014 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.282776+0000", 2024-09-14T22:36:38.014 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102469 2024-09-14T22:36:38.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:36:38.014 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.282943+0000", 2024-09-14T22:36:38.014 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016728899999999999 2024-09-14T22:36:38.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:38.014 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.285931+0000", 2024-09-14T22:36:38.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0029884740000000001 2024-09-14T22:36:38.015 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.015 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.015 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:38.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.287986+0000", 2024-09-14T22:36:38.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0020541639999999998 2024-09-14T22:36:38.015 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.015 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.015 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:38.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.288011+0000", 2024-09-14T22:36:38.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5335999999999999e-05 2024-09-14T22:36:38.015 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.015 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.015 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:38.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.288029+0000", 2024-09-14T22:36:38.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8264e-05 2024-09-14T22:36:38.016 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:38.016 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:38.016 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:38.016 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.016 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.016 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4412.0:29501 3.1e e36/23 3:7ee90710:::rbd_object_map.113c4558bb1:head v 36'264, mlcod=36'259)", 2024-09-14T22:36:38.016 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.292854+0000", 2024-09-14T22:36:38.016 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.6629935959999997, 2024-09-14T22:36:38.016 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0052550180000000002, 2024-09-14T22:36:38.016 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:38.016 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:38.016 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:38.016 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:38.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:38.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.292854+0000", 2024-09-14T22:36:38.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:38.017 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:38.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.292854+0000", 2024-09-14T22:36:38.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:38.017 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:38.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.292855+0000", 2024-09-14T22:36:38.018 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7900000000000007e-07 2024-09-14T22:36:38.018 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.018 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:38.018 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.292863+0000", 2024-09-14T22:36:38.018 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5820000000000004e-06 2024-09-14T22:36:38.018 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.018 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:38.018 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.292881+0000", 2024-09-14T22:36:38.018 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7980999999999999e-05 2024-09-14T22:36:38.018 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.018 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:38.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.292891+0000", 2024-09-14T22:36:38.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8120000000000002e-06 2024-09-14T22:36:38.019 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.019 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.019 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:38.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.292946+0000", 2024-09-14T22:36:38.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5815e-05 2024-09-14T22:36:38.019 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.019 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.019 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:38.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.292959+0000", 2024-09-14T22:36:38.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2886e-05 2024-09-14T22:36:38.019 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.019 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.019 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:38.020 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.298059+0000", 2024-09-14T22:36:38.020 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0050995720000000001 2024-09-14T22:36:38.020 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.020 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.020 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:38.020 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.298109+0000", 2024-09-14T22:36:38.020 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0390999999999999e-05 2024-09-14T22:36:38.020 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:38.020 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:38.020 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:38.020 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.020 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.021 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4412.0:29504 3.1e e36/23 3:7ee90710:::rbd_object_map.113c4558bb1:head v 36'267, mlcod=36'260)", 2024-09-14T22:36:38.021 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.295410+0000", 2024-09-14T22:36:38.021 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.6604378620000002, 2024-09-14T22:36:38.021 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0054212009999999996, 2024-09-14T22:36:38.021 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:38.021 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:38.021 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:38.021 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:38.021 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.021 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:38.021 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.295410+0000", 2024-09-14T22:36:38.021 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:38.021 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.021 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:38.022 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.295410+0000", 2024-09-14T22:36:38.022 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:38.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.022 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:38.022 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.295411+0000", 2024-09-14T22:36:38.022 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3900000000000004e-07 2024-09-14T22:36:38.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.022 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:38.022 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.295419+0000", 2024-09-14T22:36:38.022 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3920000000000003e-06 2024-09-14T22:36:38.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:38.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.295425+0000", 2024-09-14T22:36:38.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1630000000000001e-06 2024-09-14T22:36:38.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:38.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.295437+0000", 2024-09-14T22:36:38.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2035000000000001e-05 2024-09-14T22:36:38.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:38.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.295478+0000", 2024-09-14T22:36:38.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1245000000000002e-05 2024-09-14T22:36:38.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.024 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:38.024 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.295494+0000", 2024-09-14T22:36:38.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5664000000000001e-05 2024-09-14T22:36:38.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.024 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:38.024 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.300779+0000", 2024-09-14T22:36:38.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0052853960000000004 2024-09-14T22:36:38.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.024 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:38.024 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.300831+0000", 2024-09-14T22:36:38.025 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1666999999999998e-05 2024-09-14T22:36:38.025 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:38.025 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:38.025 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:38.025 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.025 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.025 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4412.0:36110 3.1e e36/23 3:7ee90710:::rbd_object_map.113c4558bb1:head v 36'378, mlcod=36'371)", 2024-09-14T22:36:38.025 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.767573+0000", 2024-09-14T22:36:38.025 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.1882743539999998, 2024-09-14T22:36:38.025 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0050296840000000004, 2024-09-14T22:36:38.025 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:38.025 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:38.026 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:38.026 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:38.026 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.026 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:38.026 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.767573+0000", 2024-09-14T22:36:38.026 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:38.026 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.026 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.026 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:38.026 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.767573+0000", 2024-09-14T22:36:38.026 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:38.026 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.026 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.026 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:38.027 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.767574+0000", 2024-09-14T22:36:38.027 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9999999999999997e-07 2024-09-14T22:36:38.027 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.027 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:38.027 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.767580+0000", 2024-09-14T22:36:38.027 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.699e-06 2024-09-14T22:36:38.027 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.027 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:38.027 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.767587+0000", 2024-09-14T22:36:38.027 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4070000000000004e-06 2024-09-14T22:36:38.027 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.027 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:38.027 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.767592+0000", 2024-09-14T22:36:38.028 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.617e-06 2024-09-14T22:36:38.028 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.028 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.028 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:38.028 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.767631+0000", 2024-09-14T22:36:38.028 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8994e-05 2024-09-14T22:36:38.028 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.028 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.028 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:38.028 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.767641+0000", 2024-09-14T22:36:38.028 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0229e-05 2024-09-14T22:36:38.028 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.028 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.028 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:38.028 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.772560+0000", 2024-09-14T22:36:38.029 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004919143 2024-09-14T22:36:38.029 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.029 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.029 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:38.029 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.772603+0000", 2024-09-14T22:36:38.029 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2994999999999997e-05 2024-09-14T22:36:38.029 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:38.029 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:38.029 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:38.029 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.029 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.029 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4412.0:36111 3.1e e36/23 3:7ee90710:::rbd_object_map.113c4558bb1:head v 36'379, mlcod=36'371)", 2024-09-14T22:36:38.029 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.768076+0000", 2024-09-14T22:36:38.030 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.1877713500000002, 2024-09-14T22:36:38.030 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0053600779999999999, 2024-09-14T22:36:38.030 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:38.030 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:38.030 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:38.030 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:38.030 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.030 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:38.030 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.768076+0000", 2024-09-14T22:36:38.030 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:38.030 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.030 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.030 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:38.030 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.768076+0000", 2024-09-14T22:36:38.031 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:38.031 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.031 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.031 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:38.031 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.768077+0000", 2024-09-14T22:36:38.031 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2500000000000001e-07 2024-09-14T22:36:38.031 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.031 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.031 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:38.031 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.768081+0000", 2024-09-14T22:36:38.031 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.929e-06 2024-09-14T22:36:38.031 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.031 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.031 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:38.031 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.768089+0000", 2024-09-14T22:36:38.032 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9440000000000005e-06 2024-09-14T22:36:38.032 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.032 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.032 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:38.032 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.768093+0000", 2024-09-14T22:36:38.032 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.403e-06 2024-09-14T22:36:38.032 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.032 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.032 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:38.032 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.768126+0000", 2024-09-14T22:36:38.032 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2993000000000001e-05 2024-09-14T22:36:38.032 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.032 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.033 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:38.033 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.768138+0000", 2024-09-14T22:36:38.033 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.171e-05 2024-09-14T22:36:38.033 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.033 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.033 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:38.033 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.773413+0000", 2024-09-14T22:36:38.033 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0052751880000000001 2024-09-14T22:36:38.033 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.033 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.033 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:38.033 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.773437+0000", 2024-09-14T22:36:38.033 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3485999999999998e-05 2024-09-14T22:36:38.033 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:38.033 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:38.034 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:38.034 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.034 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.034 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:45535 3.a 3:5177106c:::rbd_header.113c8642facc:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T22:36:38.034 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:36.886213+0000", 2024-09-14T22:36:38.034 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.069634712, 2024-09-14T22:36:38.034 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0053524699999999998, 2024-09-14T22:36:38.034 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:38.034 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:38.034 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:38.034 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:38.034 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:38.034 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:38.034 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 45535 2024-09-14T22:36:38.034 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.035 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:38.035 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.035 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:38.035 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.886213+0000", 2024-09-14T22:36:38.035 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:38.035 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.035 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.035 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:38.035 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.886213+0000", 2024-09-14T22:36:38.035 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:38.035 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.035 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.035 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:38.035 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.886215+0000", 2024-09-14T22:36:38.035 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8649999999999999e-06 2024-09-14T22:36:38.036 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.036 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.036 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:38.036 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.886224+0000", 2024-09-14T22:36:38.036 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5599999999999994e-06 2024-09-14T22:36:38.036 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.036 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.036 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:38.036 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.886244+0000", 2024-09-14T22:36:38.036 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0275000000000002e-05 2024-09-14T22:36:38.036 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.036 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.036 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:38.036 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.886253+0000", 2024-09-14T22:36:38.037 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0909999999999999e-06 2024-09-14T22:36:38.037 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.037 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.037 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:38.037 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.886318+0000", 2024-09-14T22:36:38.037 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4783000000000006e-05 2024-09-14T22:36:38.037 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.037 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.037 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op must wait for map", 2024-09-14T22:36:38.037 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.886323+0000", 2024-09-14T22:36:38.037 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4829999999999999e-06 2024-09-14T22:36:38.037 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.037 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.038 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:38.038 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.888819+0000", 2024-09-14T22:36:38.038 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0024958960000000001 2024-09-14T22:36:38.038 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.038 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.038 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:38.038 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.889178+0000", 2024-09-14T22:36:38.038 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035912699999999999 2024-09-14T22:36:38.038 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.038 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.038 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:36:38.038 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.889844+0000", 2024-09-14T22:36:38.038 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00066624499999999999 2024-09-14T22:36:38.038 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.038 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.039 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:38.039 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.891052+0000", 2024-09-14T22:36:38.039 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0012071510000000001 2024-09-14T22:36:38.039 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.039 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.039 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:38.039 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.891424+0000", 2024-09-14T22:36:38.039 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000372748 2024-09-14T22:36:38.039 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.039 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.039 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:38.039 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.891506+0000", 2024-09-14T22:36:38.039 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1198000000000004e-05 2024-09-14T22:36:38.039 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.040 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.040 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:38.040 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.891566+0000", 2024-09-14T22:36:38.040 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0047999999999998e-05 2024-09-14T22:36:38.040 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:38.040 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:38.040 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:38.040 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.040 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.040 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:46461 3.10 3:09ce7ae2:::rbd_data.113cc4e35b3.0000000000000001:8 [sparse-read 5898240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T22:36:38.040 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:37.265002+0000", 2024-09-14T22:36:38.040 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.69084628699999995, 2024-09-14T22:36:38.040 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0047275579999999998, 2024-09-14T22:36:38.041 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:38.041 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:38.041 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:38.041 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:38.041 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:38.041 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:38.041 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 46461 2024-09-14T22:36:38.041 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.041 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:38.041 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.041 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:38.041 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265002+0000", 2024-09-14T22:36:38.041 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:38.042 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.042 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.042 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:38.042 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265002+0000", 2024-09-14T22:36:38.042 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:38.042 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.042 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.042 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:38.042 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265002+0000", 2024-09-14T22:36:38.042 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.34e-07 2024-09-14T22:36:38.042 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.042 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.042 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:38.042 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265004+0000", 2024-09-14T22:36:38.043 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8470000000000001e-06 2024-09-14T22:36:38.043 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.043 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.043 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:38.043 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265010+0000", 2024-09-14T22:36:38.043 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5220000000000004e-06 2024-09-14T22:36:38.043 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.043 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.043 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:38.043 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265015+0000", 2024-09-14T22:36:38.043 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.011e-06 2024-09-14T22:36:38.043 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.043 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.043 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:38.044 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.269506+0000", 2024-09-14T22:36:38.044 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0044906099999999999 2024-09-14T22:36:38.044 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.044 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.044 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:38.044 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.269617+0000", 2024-09-14T22:36:38.044 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110845 2024-09-14T22:36:38.044 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.044 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.044 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:38.044 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.269729+0000", 2024-09-14T22:36:38.044 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112389 2024-09-14T22:36:38.044 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:38.044 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:38.044 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:38.045 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.045 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.045 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:46462 3.10 3:09ce7ae2:::rbd_data.113cc4e35b3.0000000000000001:8 [sparse-read 7340032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T22:36:38.045 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:37.265051+0000", 2024-09-14T22:36:38.045 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.69079712599999998, 2024-09-14T22:36:38.045 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0049099060000000003, 2024-09-14T22:36:38.045 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:38.045 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:38.045 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:38.045 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:38.045 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:38.045 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:38.045 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 46462 2024-09-14T22:36:38.045 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.046 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:38.046 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.046 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:38.046 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265051+0000", 2024-09-14T22:36:38.046 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:38.046 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.046 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.046 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:38.046 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265051+0000", 2024-09-14T22:36:38.046 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:38.046 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.046 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.046 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:38.047 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265051+0000", 2024-09-14T22:36:38.047 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2900000000000003e-07 2024-09-14T22:36:38.047 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.047 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.047 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:38.047 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265054+0000", 2024-09-14T22:36:38.047 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7360000000000001e-06 2024-09-14T22:36:38.047 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.047 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.047 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:38.047 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265061+0000", 2024-09-14T22:36:38.047 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1430000000000001e-06 2024-09-14T22:36:38.047 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.047 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.047 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:38.048 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265067+0000", 2024-09-14T22:36:38.048 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4879999999999998e-06 2024-09-14T22:36:38.048 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.048 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.048 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:38.048 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.269772+0000", 2024-09-14T22:36:38.048 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0047057669999999996 2024-09-14T22:36:38.048 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.048 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.048 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:38.048 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.269870+0000", 2024-09-14T22:36:38.048 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7292000000000003e-05 2024-09-14T22:36:38.048 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.049 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.049 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:38.049 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.269961+0000", 2024-09-14T22:36:38.049 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0851000000000001e-05 2024-09-14T22:36:38.049 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:38.049 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:38.049 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:38.049 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.049 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.049 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:46463 3.10 3:09ce7ae2:::rbd_data.113cc4e35b3.0000000000000001:8 [sparse-read 8781824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T22:36:38.049 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:37.265074+0000", 2024-09-14T22:36:38.049 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.69077334499999998, 2024-09-14T22:36:38.049 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0052541760000000002, 2024-09-14T22:36:38.050 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:38.050 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:38.050 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:38.050 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:38.050 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:38.050 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:38.050 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 46463 2024-09-14T22:36:38.050 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.050 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:38.050 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.050 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:38.050 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265074+0000", 2024-09-14T22:36:38.050 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:38.050 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:38.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265074+0000", 2024-09-14T22:36:38.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:38.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:38.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265075+0000", 2024-09-14T22:36:38.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:36:38.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:38.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265077+0000", 2024-09-14T22:36:38.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1320000000000001e-06 2024-09-14T22:36:38.052 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.052 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:38.052 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265083+0000", 2024-09-14T22:36:38.052 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0580000000000003e-06 2024-09-14T22:36:38.052 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.052 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:38.052 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265088+0000", 2024-09-14T22:36:38.052 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8119999999999998e-06 2024-09-14T22:36:38.052 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.052 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:38.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.270026+0000", 2024-09-14T22:36:38.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0049381649999999996 2024-09-14T22:36:38.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:38.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.270138+0000", 2024-09-14T22:36:38.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112175 2024-09-14T22:36:38.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:38.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.270329+0000", 2024-09-14T22:36:38.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019029999999999999 2024-09-14T22:36:38.053 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:38.054 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:38.054 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:38.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.054 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:46464 3.10 3:09ce7ae2:::rbd_data.113cc4e35b3.0000000000000001:8 [sparse-read 10223616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T22:36:38.054 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:37.265094+0000", 2024-09-14T22:36:38.054 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.69075364500000003, 2024-09-14T22:36:38.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0055334720000000002, 2024-09-14T22:36:38.054 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:38.054 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:38.054 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:38.054 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:38.054 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:38.054 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:38.054 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 46464 2024-09-14T22:36:38.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.055 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:38.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:38.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265094+0000", 2024-09-14T22:36:38.055 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:38.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:38.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265094+0000", 2024-09-14T22:36:38.055 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:38.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:38.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265095+0000", 2024-09-14T22:36:38.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8099999999999998e-07 2024-09-14T22:36:38.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.056 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:38.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265101+0000", 2024-09-14T22:36:38.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8909999999999998e-06 2024-09-14T22:36:38.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.056 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:38.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265108+0000", 2024-09-14T22:36:38.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2899999999999999e-06 2024-09-14T22:36:38.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.056 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:38.057 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265113+0000", 2024-09-14T22:36:38.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.101e-06 2024-09-14T22:36:38.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.057 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:38.057 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.270374+0000", 2024-09-14T22:36:38.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0052610460000000001 2024-09-14T22:36:38.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.057 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:38.057 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.270472+0000", 2024-09-14T22:36:38.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7635999999999998e-05 2024-09-14T22:36:38.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.057 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:38.058 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.270628+0000", 2024-09-14T22:36:38.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015612699999999999 2024-09-14T22:36:38.058 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:38.058 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:38.058 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:38.058 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.058 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:46465 3.10 3:09ce7ae2:::rbd_data.113cc4e35b3.0000000000000001:8 [sparse-read 11665408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T22:36:38.058 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:37.265119+0000", 2024-09-14T22:36:38.058 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.69072868300000001, 2024-09-14T22:36:38.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.005788924, 2024-09-14T22:36:38.058 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:38.058 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:38.058 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:38.058 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:38.059 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:38.059 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:38.059 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 46465 2024-09-14T22:36:38.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.059 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:38.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.059 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:38.059 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265119+0000", 2024-09-14T22:36:38.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:38.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.059 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:38.059 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265119+0000", 2024-09-14T22:36:38.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:38.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:38.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265120+0000", 2024-09-14T22:36:38.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9700000000000002e-07 2024-09-14T22:36:38.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:38.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265123+0000", 2024-09-14T22:36:38.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1920000000000001e-06 2024-09-14T22:36:38.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:38.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265129+0000", 2024-09-14T22:36:38.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9769999999999997e-06 2024-09-14T22:36:38.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:38.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265133+0000", 2024-09-14T22:36:38.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7439999999999998e-06 2024-09-14T22:36:38.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:38.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.270662+0000", 2024-09-14T22:36:38.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0055282739999999997 2024-09-14T22:36:38.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:38.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.270774+0000", 2024-09-14T22:36:38.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011195499999999999 2024-09-14T22:36:38.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:38.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:38.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:38.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.270908+0000", 2024-09-14T22:36:38.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013438499999999999 2024-09-14T22:36:38.062 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:38.062 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:38.062 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:38.062 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:38.062 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:38.062 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:38.064 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:36:38.064 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:38.059+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:38.164 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:38.158+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:38.200 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:36:38.235 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:38.236 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:38.236 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:36:38.236 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:38.249 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:36:38.265 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:38.258+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:38.348 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:36:38.348 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:36:38.366 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:36:38.367 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:38.359+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:38.367 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:36:38.367 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-14T22:36:38.371 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:38.372 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:38.372 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:36:38.372 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:36:38.372 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:38.384 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:36:38.384 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:36:38.386 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:36:38.402 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:36:38.402 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:36:38.421 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:36:38.421 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:36:38.440 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:36:38.441 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:36:38.460 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:36:38.460 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:36:38.465 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:36:38.466 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:36:38.510 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:38.506+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:38.549 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:38.549 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:38.562 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:36:38.610 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:38.606+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:38.668 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:38.662+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:38.695 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:38.695 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:38.710 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:36:38.802 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:38.798+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:38.827 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2024-09-14T22:36:38.870 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:38.863+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:38.891 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:38.892 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:38.909 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:36:38.955 INFO:teuthology.orchestra.run.smithi001.stdout:flattening image image_client.0-clone6 2024-09-14T22:36:38.971 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:38.963+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:39.072 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:39.067+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:39.080 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:39.081 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:39.095 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:36:39.134 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-14T22:36:39.173 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:39.167+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:39.262 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:39.262 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:39.274 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:39.267+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:39.278 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:36:39.353 INFO:teuthology.orchestra.run.smithi001.stdout:1060 writesame 0x12ca5e2 thru 0x12ceb9e data_size 0x1(0x45bd bytes) 2024-09-14T22:36:39.378 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:39.370+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:39.418 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:39.418 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:39.435 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:36:39.453 INFO:teuthology.orchestra.run.smithi001.stdout:1061 read 0x2227ceb thru 0x222c2b1 (0x45c7 bytes) 2024-09-14T22:36:39.454 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:39.455 INFO:teuthology.orchestra.run.smithi001.stdout:1063 read 0x4a4ae1d thru 0x4a5a197 (0xf37b bytes) 2024-09-14T22:36:39.457 INFO:teuthology.orchestra.run.smithi001.stdout:1064 punch from 0x939aab to 0x93c439, (0x298e bytes) 2024-09-14T22:36:39.457 INFO:teuthology.orchestra.run.smithi001.stdout:1065 read 0x59c50a7 thru 0x59cd671 (0x85cb bytes) 2024-09-14T22:36:39.457 INFO:teuthology.orchestra.run.smithi001.stdout:1067 punch from 0x775146 to 0x778cfb, (0x3bb5 bytes) 2024-09-14T22:36:39.462 INFO:teuthology.orchestra.run.smithi001.stdout:1069 trunc from 0x9d3cbe5 to 0x70ec40e 2024-09-14T22:36:39.475 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:39.471+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:39.547 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:39.562 INFO:teuthology.orchestra.run.smithi001.stdout:1072 writesame 0xc939601 thru 0xc9488d2 data_size 0x1(0xf2d2 bytes) 2024-09-14T22:36:39.563 INFO:teuthology.orchestra.run.smithi001.stdout:1075 writesame 0x6caee3c thru 0x6cb7f82 data_size 0x4d(0x9147 bytes) 2024-09-14T22:36:39.576 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:39.571+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:39.617 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:39.618 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:39.639 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:36:39.657 INFO:teuthology.orchestra.run.smithi001.stdout:1076 write 0x9784607 thru 0x9794400 (0xfdfa bytes) 2024-09-14T22:36:39.669 INFO:teuthology.orchestra.run.smithi001.stdout:1079 write 0x4d33f92 thru 0x4d387d9 (0x4848 bytes) 2024-09-14T22:36:39.677 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:39.670+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:39.736 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:39.736 INFO:teuthology.orchestra.run.smithi001.stdout:1081 read 0x5b614dc thru 0x5b6fd88 (0xe8ad bytes) 2024-09-14T22:36:39.738 INFO:teuthology.orchestra.run.smithi001.stdout:1084 read 0x1c58217 thru 0x1c5b6cc (0x34b6 bytes) 2024-09-14T22:36:39.738 INFO:teuthology.orchestra.run.smithi001.stdout:1085 write 0x7206a33 thru 0x7216262 (0xf830 bytes) 2024-09-14T22:36:39.741 INFO:teuthology.orchestra.run.smithi001.stdout:1086 write 0xc8a3b98 thru 0xc8a8d8e (0x51f7 bytes) 2024-09-14T22:36:39.752 INFO:teuthology.orchestra.run.smithi001.stdout:1087 trunc from 0xc9488d3 to 0x93188b7 2024-09-14T22:36:39.776 INFO:teuthology.orchestra.run.smithi001.stdout:1088 write 0xc037533 thru 0xc044f2c (0xd9fa bytes) 2024-09-14T22:36:39.777 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:39.770+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:39.793 INFO:teuthology.orchestra.run.smithi001.stdout:1090 write 0xc54ffcc thru 0xc55b72e (0xb763 bytes) 2024-09-14T22:36:39.795 INFO:teuthology.orchestra.run.smithi001.stdout:1091 read 0x2c2eb29 thru 0x2c3eb03 (0xffdb bytes) 2024-09-14T22:36:39.796 INFO:teuthology.orchestra.run.smithi001.stdout:1092 punch from 0x8ded1b4 to 0x8ded54b, (0x397 bytes) 2024-09-14T22:36:39.796 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:39.796 INFO:teuthology.orchestra.run.smithi001.stdout:1094 read 0x5ef99c4 thru 0x5efaa68 (0x10a5 bytes) 2024-09-14T22:36:39.797 INFO:teuthology.orchestra.run.smithi001.stdout:1095 clone 8 order 18 su 131072 sc 8 2024-09-14T22:36:39.800 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:39.800 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:39.814 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:36:39.902 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:39.898+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:39.975 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone7 from 0xc55b72f (overlap 0x70ec40e) to 0x0 2024-09-14T22:36:39.979 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:39.975+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:40.012 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:40.012 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:40.033 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:36:40.080 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:40.075+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:40.180 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:40.175+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:40.205 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [2] live_osds: [0, 1, 3, 4, 5, 6, 7] 2024-09-14T22:36:40.205 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:36:40.205 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-14T22:36:40.205 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 0.160636 2024-09-14T22:36:40.205 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 0.16063648043448275 2024-09-14T22:36:40.276 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:40.276 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:40.281 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:40.275+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:40.308 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:36:40.383 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:40.378+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:40.467 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:36:40.481 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:40.481 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:40.483 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:40.479+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:40.495 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:36:40.584 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:40.579+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:40.617 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:36:40.617 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:36:40.633 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:36:40.634 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:36:40.634 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-14T22:36:40.660 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:36:40.661 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:36:40.674 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:40.674 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:40.681 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:36:40.681 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:36:40.685 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:40.679+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:40.691 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:36:40.701 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:36:40.701 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:36:40.721 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:36:40.721 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:36:40.738 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:36:40.738 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:36:40.741 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:36:40.741 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:36:40.760 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-14T22:36:40.786 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:40.779+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:40.847 INFO:teuthology.orchestra.run.smithi001.stderr:set osd.1 primary-affinity to 0.160636 (8105272) 2024-09-14T22:36:40.854 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T22:36:40.865 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:40.865 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:36:40.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:36:40.865 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:36:40.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.865 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6376 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8230912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:40.865 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932578+0000", 2024-09-14T22:36:40.865 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.929392908000001, 2024-09-14T22:36:40.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051558771000000003, 2024-09-14T22:36:40.866 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:40.866 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:40.866 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:40.866 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:40.866 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:40.866 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:40.866 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6376 2024-09-14T22:36:40.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.866 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:40.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:40.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932578+0000", 2024-09-14T22:36:40.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:40.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932578+0000", 2024-09-14T22:36:40.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:40.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932579+0000", 2024-09-14T22:36:40.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8599999999999996e-07 2024-09-14T22:36:40.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:40.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932597+0000", 2024-09-14T22:36:40.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7771e-05 2024-09-14T22:36:40.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:40.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932618+0000", 2024-09-14T22:36:40.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1026000000000002e-05 2024-09-14T22:36:40.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:40.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932635+0000", 2024-09-14T22:36:40.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.702e-05 2024-09-14T22:36:40.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:40.869 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.983951+0000", 2024-09-14T22:36:40.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051316639999999997 2024-09-14T22:36:40.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.869 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:40.869 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984031+0000", 2024-09-14T22:36:40.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0085000000000003e-05 2024-09-14T22:36:40.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.869 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:40.869 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984137+0000", 2024-09-14T22:36:40.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105243 2024-09-14T22:36:40.869 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.870 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:40.870 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.870 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.870 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6377 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8239104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:40.870 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932688+0000", 2024-09-14T22:36:40.870 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.929282876999999, 2024-09-14T22:36:40.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051650696000000003, 2024-09-14T22:36:40.870 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:40.870 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:40.870 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:40.870 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:40.871 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:40.871 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:40.871 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6377 2024-09-14T22:36:40.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.871 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:40.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:40.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932688+0000", 2024-09-14T22:36:40.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:40.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932688+0000", 2024-09-14T22:36:40.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:40.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932689+0000", 2024-09-14T22:36:40.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4309999999999999e-06 2024-09-14T22:36:40.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:40.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932695+0000", 2024-09-14T22:36:40.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8889999999999999e-06 2024-09-14T22:36:40.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:40.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932713+0000", 2024-09-14T22:36:40.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7416999999999999e-05 2024-09-14T22:36:40.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:40.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932726+0000", 2024-09-14T22:36:40.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3145e-05 2024-09-14T22:36:40.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:40.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984181+0000", 2024-09-14T22:36:40.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051455630000000002 2024-09-14T22:36:40.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.874 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:40.874 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984247+0000", 2024-09-14T22:36:40.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5350000000000003e-05 2024-09-14T22:36:40.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.874 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:40.874 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984339+0000", 2024-09-14T22:36:40.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1834000000000002e-05 2024-09-14T22:36:40.874 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.874 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:40.874 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.875 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6378 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8247296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:40.875 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932743+0000", 2024-09-14T22:36:40.875 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.929227643000001, 2024-09-14T22:36:40.875 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051819373000000002, 2024-09-14T22:36:40.875 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:40.875 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:40.875 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:40.875 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:40.875 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:40.875 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:40.875 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6378 2024-09-14T22:36:40.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.876 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:40.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:40.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932743+0000", 2024-09-14T22:36:40.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:40.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932743+0000", 2024-09-14T22:36:40.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:40.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932745+0000", 2024-09-14T22:36:40.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.637e-06 2024-09-14T22:36:40.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:40.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932750+0000", 2024-09-14T22:36:40.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7859999999999996e-06 2024-09-14T22:36:40.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:40.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932767+0000", 2024-09-14T22:36:40.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7621999999999998e-05 2024-09-14T22:36:40.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:40.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932781+0000", 2024-09-14T22:36:40.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3403e-05 2024-09-14T22:36:40.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:40.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984383+0000", 2024-09-14T22:36:40.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051602249000000003 2024-09-14T22:36:40.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:40.879 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984461+0000", 2024-09-14T22:36:40.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8467000000000006e-05 2024-09-14T22:36:40.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.879 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:40.879 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984562+0000", 2024-09-14T22:36:40.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101209 2024-09-14T22:36:40.879 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.879 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:40.879 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.879 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6379 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8255488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:40.880 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932796+0000", 2024-09-14T22:36:40.880 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.929174251999999, 2024-09-14T22:36:40.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051961245000000003, 2024-09-14T22:36:40.880 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:40.880 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:40.880 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:40.880 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:40.880 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:40.880 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:40.880 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6379 2024-09-14T22:36:40.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.880 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:40.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:40.881 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932796+0000", 2024-09-14T22:36:40.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:40.881 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932796+0000", 2024-09-14T22:36:40.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:40.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932797+0000", 2024-09-14T22:36:40.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9700000000000005e-07 2024-09-14T22:36:40.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:40.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932823+0000", 2024-09-14T22:36:40.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5273999999999999e-05 2024-09-14T22:36:40.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:40.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932840+0000", 2024-09-14T22:36:40.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7774e-05 2024-09-14T22:36:40.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.883 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:40.883 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932862+0000", 2024-09-14T22:36:40.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1957e-05 2024-09-14T22:36:40.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.883 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:40.883 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984603+0000", 2024-09-14T22:36:40.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051740543 2024-09-14T22:36:40.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.883 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:40.883 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984669+0000", 2024-09-14T22:36:40.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.567e-05 2024-09-14T22:36:40.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.884 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:40.884 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984758+0000", 2024-09-14T22:36:40.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9129999999999995e-05 2024-09-14T22:36:40.884 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.884 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:40.884 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.884 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6380 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8263680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:40.885 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932879+0000", 2024-09-14T22:36:40.885 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.929092173000001, 2024-09-14T22:36:40.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052102860000000001, 2024-09-14T22:36:40.885 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:40.885 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:40.885 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:40.885 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:40.885 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:40.885 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:40.885 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6380 2024-09-14T22:36:40.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.885 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:40.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:40.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932879+0000", 2024-09-14T22:36:40.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:40.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932879+0000", 2024-09-14T22:36:40.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:40.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932880+0000", 2024-09-14T22:36:40.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.054e-06 2024-09-14T22:36:40.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.887 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:40.887 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932885+0000", 2024-09-14T22:36:40.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1290000000000004e-06 2024-09-14T22:36:40.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.887 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:40.887 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932901+0000", 2024-09-14T22:36:40.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5908000000000001e-05 2024-09-14T22:36:40.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.887 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:40.887 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932913+0000", 2024-09-14T22:36:40.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2408999999999999e-05 2024-09-14T22:36:40.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.888 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:40.888 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984800+0000", 2024-09-14T22:36:40.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051886671000000002 2024-09-14T22:36:40.888 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.888 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.888 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:40.888 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984879+0000", 2024-09-14T22:36:40.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9200000000000001e-05 2024-09-14T22:36:40.888 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.888 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.888 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:40.888 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984981+0000", 2024-09-14T22:36:40.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102489 2024-09-14T22:36:40.888 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.889 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:40.889 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.889 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6381 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8271872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:40.889 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932929+0000", 2024-09-14T22:36:40.889 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.929041842, 2024-09-14T22:36:40.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052251484000000001, 2024-09-14T22:36:40.889 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:40.889 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:40.889 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:40.889 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:40.889 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:40.889 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:40.889 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6381 2024-09-14T22:36:40.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.890 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:40.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:40.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932929+0000", 2024-09-14T22:36:40.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:40.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932929+0000", 2024-09-14T22:36:40.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:40.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932930+0000", 2024-09-14T22:36:40.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3399999999999998e-07 2024-09-14T22:36:40.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:40.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932934+0000", 2024-09-14T22:36:40.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5719999999999996e-06 2024-09-14T22:36:40.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:40.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932961+0000", 2024-09-14T22:36:40.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6437000000000001e-05 2024-09-14T22:36:40.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.892 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:40.892 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932973+0000", 2024-09-14T22:36:40.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2097e-05 2024-09-14T22:36:40.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.892 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:40.892 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985021+0000", 2024-09-14T22:36:40.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052047955999999999 2024-09-14T22:36:40.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.892 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:40.892 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985089+0000", 2024-09-14T22:36:40.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7725999999999995e-05 2024-09-14T22:36:40.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:40.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985180+0000", 2024-09-14T22:36:40.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1861999999999996e-05 2024-09-14T22:36:40.893 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.893 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:40.893 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.893 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6382 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8280064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:40.893 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933013+0000", 2024-09-14T22:36:40.893 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.92895768, 2024-09-14T22:36:40.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052391118, 2024-09-14T22:36:40.894 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:40.894 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:40.894 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:40.894 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:40.894 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:40.894 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:40.894 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6382 2024-09-14T22:36:40.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.894 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:40.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.894 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:40.894 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933013+0000", 2024-09-14T22:36:40.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.895 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:40.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933013+0000", 2024-09-14T22:36:40.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.895 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:40.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933014+0000", 2024-09-14T22:36:40.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.375e-06 2024-09-14T22:36:40.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.895 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:40.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933020+0000", 2024-09-14T22:36:40.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0220000000000001e-06 2024-09-14T22:36:40.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.896 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:40.896 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933038+0000", 2024-09-14T22:36:40.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7694999999999999e-05 2024-09-14T22:36:40.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.896 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:40.896 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933051+0000", 2024-09-14T22:36:40.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2721999999999999e-05 2024-09-14T22:36:40.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.896 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:40.896 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985224+0000", 2024-09-14T22:36:40.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052172757 2024-09-14T22:36:40.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:40.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985303+0000", 2024-09-14T22:36:40.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9426999999999997e-05 2024-09-14T22:36:40.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:40.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985404+0000", 2024-09-14T22:36:40.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010111999999999999 2024-09-14T22:36:40.898 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.898 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:40.898 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.898 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6383 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8288256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:40.898 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933082+0000", 2024-09-14T22:36:40.898 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.928888700000002, 2024-09-14T22:36:40.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052525761999999997, 2024-09-14T22:36:40.899 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:40.899 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:40.899 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:40.899 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:40.899 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:40.899 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:40.899 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6383 2024-09-14T22:36:40.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.899 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:40.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.899 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:40.899 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933082+0000", 2024-09-14T22:36:40.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.900 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.900 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:40.900 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933082+0000", 2024-09-14T22:36:40.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.900 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.900 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:40.900 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933084+0000", 2024-09-14T22:36:40.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5659999999999999e-06 2024-09-14T22:36:40.900 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.900 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:40.900 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933089+0000", 2024-09-14T22:36:40.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.271e-06 2024-09-14T22:36:40.901 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.901 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.901 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:40.901 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933105+0000", 2024-09-14T22:36:40.901 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5685999999999999e-05 2024-09-14T22:36:40.901 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.901 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.901 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:40.901 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933118+0000", 2024-09-14T22:36:40.901 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3188e-05 2024-09-14T22:36:40.901 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.901 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.901 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:40.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985448+0000", 2024-09-14T22:36:40.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052330176999999999 2024-09-14T22:36:40.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:40.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985514+0000", 2024-09-14T22:36:40.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6029000000000001e-05 2024-09-14T22:36:40.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:40.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985608+0000", 2024-09-14T22:36:40.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3844999999999993e-05 2024-09-14T22:36:40.902 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.902 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:40.903 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.903 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6384 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8296448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:40.903 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933134+0000", 2024-09-14T22:36:40.903 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.928836341, 2024-09-14T22:36:40.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052688835000000003, 2024-09-14T22:36:40.903 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:40.903 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:40.903 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:40.903 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:40.903 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:40.903 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:40.903 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6384 2024-09-14T22:36:40.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.904 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:40.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.904 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:40.904 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933134+0000", 2024-09-14T22:36:40.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.904 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:40.904 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933134+0000", 2024-09-14T22:36:40.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.904 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:40.904 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933135+0000", 2024-09-14T22:36:40.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.78e-07 2024-09-14T22:36:40.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.905 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:40.905 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933141+0000", 2024-09-14T22:36:40.905 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1540000000000001e-06 2024-09-14T22:36:40.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.905 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:40.905 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933156+0000", 2024-09-14T22:36:40.905 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5488000000000001e-05 2024-09-14T22:36:40.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.905 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:40.905 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933169+0000", 2024-09-14T22:36:40.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2649e-05 2024-09-14T22:36:40.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.906 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:40.906 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985650+0000", 2024-09-14T22:36:40.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052481112000000003 2024-09-14T22:36:40.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.906 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:40.906 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985724+0000", 2024-09-14T22:36:40.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3953000000000002e-05 2024-09-14T22:36:40.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.906 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:40.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985823+0000", 2024-09-14T22:36:40.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9500999999999995e-05 2024-09-14T22:36:40.907 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.907 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:40.907 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.907 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6385 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8304640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:40.907 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933185+0000", 2024-09-14T22:36:40.907 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.928785638000001, 2024-09-14T22:36:40.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052892844000000001, 2024-09-14T22:36:40.907 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:40.907 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:40.907 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:40.908 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:40.908 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:40.908 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:40.908 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6385 2024-09-14T22:36:40.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.908 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:40.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:40.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933185+0000", 2024-09-14T22:36:40.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:40.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933185+0000", 2024-09-14T22:36:40.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.909 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:40.909 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933186+0000", 2024-09-14T22:36:40.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0139999999999999e-06 2024-09-14T22:36:40.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.909 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:40.909 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933191+0000", 2024-09-14T22:36:40.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7230000000000004e-06 2024-09-14T22:36:40.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.909 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:40.910 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933207+0000", 2024-09-14T22:36:40.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.658e-05 2024-09-14T22:36:40.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:40.910 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933220+0000", 2024-09-14T22:36:40.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2583e-05 2024-09-14T22:36:40.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:40.910 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985880+0000", 2024-09-14T22:36:40.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052660140000000001 2024-09-14T22:36:40.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:40.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985965+0000", 2024-09-14T22:36:40.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4472e-05 2024-09-14T22:36:40.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:40.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986078+0000", 2024-09-14T22:36:40.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011333200000000001 2024-09-14T22:36:40.911 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.911 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:40.911 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.911 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6386 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8312832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:40.912 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933236+0000", 2024-09-14T22:36:40.912 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.928734744, 2024-09-14T22:36:40.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053075342999999997, 2024-09-14T22:36:40.912 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:40.912 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:40.912 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:40.912 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:40.912 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:40.912 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:40.912 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6386 2024-09-14T22:36:40.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.912 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:40.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:40.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933236+0000", 2024-09-14T22:36:40.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:40.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933236+0000", 2024-09-14T22:36:40.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:40.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933237+0000", 2024-09-14T22:36:40.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6500000000000008e-07 2024-09-14T22:36:40.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.914 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:40.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933242+0000", 2024-09-14T22:36:40.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.938e-06 2024-09-14T22:36:40.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.914 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:40.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933258+0000", 2024-09-14T22:36:40.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6334e-05 2024-09-14T22:36:40.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.914 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:40.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933270+0000", 2024-09-14T22:36:40.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1973999999999999e-05 2024-09-14T22:36:40.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:40.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986123+0000", 2024-09-14T22:36:40.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052852769000000001 2024-09-14T22:36:40.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:40.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986204+0000", 2024-09-14T22:36:40.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1458999999999994e-05 2024-09-14T22:36:40.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:40.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986311+0000", 2024-09-14T22:36:40.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010690400000000001 2024-09-14T22:36:40.916 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.916 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:40.916 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.916 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6387 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8321024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:40.916 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933335+0000", 2024-09-14T22:36:40.916 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.928635474, 2024-09-14T22:36:40.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053182363000000003, 2024-09-14T22:36:40.916 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:40.916 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:40.916 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:40.917 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:40.917 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:40.917 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:40.917 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6387 2024-09-14T22:36:40.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.917 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:40.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:40.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933335+0000", 2024-09-14T22:36:40.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:40.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933335+0000", 2024-09-14T22:36:40.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:40.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933337+0000", 2024-09-14T22:36:40.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3829999999999999e-06 2024-09-14T22:36:40.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:40.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933343+0000", 2024-09-14T22:36:40.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1489999999999999e-06 2024-09-14T22:36:40.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:40.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933361+0000", 2024-09-14T22:36:40.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8105e-05 2024-09-14T22:36:40.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:40.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933374+0000", 2024-09-14T22:36:40.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.33e-05 2024-09-14T22:36:40.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:40.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986355+0000", 2024-09-14T22:36:40.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052981053 2024-09-14T22:36:40.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:40.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986424+0000", 2024-09-14T22:36:40.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8864999999999999e-05 2024-09-14T22:36:40.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:40.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986518+0000", 2024-09-14T22:36:40.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3508e-05 2024-09-14T22:36:40.920 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.920 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:40.920 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.920 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6388 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8329216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:40.920 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933390+0000", 2024-09-14T22:36:40.921 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.928580887999999, 2024-09-14T22:36:40.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053371605000000003, 2024-09-14T22:36:40.921 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:40.921 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:40.921 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:40.921 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:40.921 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:40.921 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:40.921 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6388 2024-09-14T22:36:40.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.921 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:40.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:40.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933390+0000", 2024-09-14T22:36:40.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:40.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933390+0000", 2024-09-14T22:36:40.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:40.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933391+0000", 2024-09-14T22:36:40.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.443e-06 2024-09-14T22:36:40.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.923 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:40.923 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933396+0000", 2024-09-14T22:36:40.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1680000000000003e-06 2024-09-14T22:36:40.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.923 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:40.923 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933412+0000", 2024-09-14T22:36:40.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5302999999999999e-05 2024-09-14T22:36:40.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.923 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:40.923 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933439+0000", 2024-09-14T22:36:40.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6747999999999999e-05 2024-09-14T22:36:40.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:40.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986562+0000", 2024-09-14T22:36:40.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053123161000000002 2024-09-14T22:36:40.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:40.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986641+0000", 2024-09-14T22:36:40.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9086999999999997e-05 2024-09-14T22:36:40.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:40.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986761+0000", 2024-09-14T22:36:40.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120695 2024-09-14T22:36:40.925 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.925 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:40.925 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.925 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6389 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8337408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:40.925 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933456+0000", 2024-09-14T22:36:40.925 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.928515037, 2024-09-14T22:36:40.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053514182, 2024-09-14T22:36:40.925 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:40.926 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:40.926 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:40.926 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:40.926 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:40.926 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:40.926 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6389 2024-09-14T22:36:40.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.926 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:40.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:40.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933456+0000", 2024-09-14T22:36:40.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:40.927 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933456+0000", 2024-09-14T22:36:40.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:40.927 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933457+0000", 2024-09-14T22:36:40.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0020000000000001e-06 2024-09-14T22:36:40.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:40.927 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933462+0000", 2024-09-14T22:36:40.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8400000000000002e-06 2024-09-14T22:36:40.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:40.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933477+0000", 2024-09-14T22:36:40.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.592e-05 2024-09-14T22:36:40.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:40.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933489+0000", 2024-09-14T22:36:40.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1955e-05 2024-09-14T22:36:40.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:40.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986798+0000", 2024-09-14T22:36:40.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053308263000000002 2024-09-14T22:36:40.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:40.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986871+0000", 2024-09-14T22:36:40.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3518999999999999e-05 2024-09-14T22:36:40.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:40.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986970+0000", 2024-09-14T22:36:40.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8683000000000004e-05 2024-09-14T22:36:40.929 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.929 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:40.929 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.930 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6390 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8345600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:40.930 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933504+0000", 2024-09-14T22:36:40.930 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.928466252, 2024-09-14T22:36:40.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053719579000000003, 2024-09-14T22:36:40.930 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:40.930 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:40.930 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:40.930 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:40.930 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:40.930 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:40.930 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6390 2024-09-14T22:36:40.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.931 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:40.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:40.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933504+0000", 2024-09-14T22:36:40.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:40.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933504+0000", 2024-09-14T22:36:40.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:40.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933505+0000", 2024-09-14T22:36:40.932 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8299999999999995e-07 2024-09-14T22:36:40.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.932 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.932 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:40.932 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933520+0000", 2024-09-14T22:36:40.932 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4948e-05 2024-09-14T22:36:40.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.932 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.932 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:40.932 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933538+0000", 2024-09-14T22:36:40.932 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7286e-05 2024-09-14T22:36:40.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.932 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:40.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933550+0000", 2024-09-14T22:36:40.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2223e-05 2024-09-14T22:36:40.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:40.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987028+0000", 2024-09-14T22:36:40.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053478500999999998 2024-09-14T22:36:40.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:40.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987111+0000", 2024-09-14T22:36:40.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3158999999999994e-05 2024-09-14T22:36:40.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:40.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987224+0000", 2024-09-14T22:36:40.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011257899999999999 2024-09-14T22:36:40.934 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.934 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:40.934 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.934 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6391 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8353792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:40.934 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933576+0000", 2024-09-14T22:36:40.934 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.928394807, 2024-09-14T22:36:40.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053858196999999997, 2024-09-14T22:36:40.934 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:40.935 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:40.935 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:40.935 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:40.935 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:40.935 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:40.935 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6391 2024-09-14T22:36:40.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.935 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:40.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:40.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933576+0000", 2024-09-14T22:36:40.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.936 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:40.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933576+0000", 2024-09-14T22:36:40.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.936 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:40.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933578+0000", 2024-09-14T22:36:40.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9539999999999998e-06 2024-09-14T22:36:40.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.936 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:40.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933584+0000", 2024-09-14T22:36:40.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9030000000000001e-06 2024-09-14T22:36:40.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.937 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.937 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:40.937 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933601+0000", 2024-09-14T22:36:40.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7703e-05 2024-09-14T22:36:40.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.937 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.937 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:40.937 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933614+0000", 2024-09-14T22:36:40.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2330999999999999e-05 2024-09-14T22:36:40.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.937 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.937 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:40.937 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987275+0000", 2024-09-14T22:36:40.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053661470000000003 2024-09-14T22:36:40.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.938 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:40.938 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987342+0000", 2024-09-14T22:36:40.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6266000000000006e-05 2024-09-14T22:36:40.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.938 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:40.938 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987434+0000", 2024-09-14T22:36:40.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2570000000000003e-05 2024-09-14T22:36:40.938 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.938 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:40.939 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.939 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6392 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8361984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:40.939 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933648+0000", 2024-09-14T22:36:40.939 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.928322419000001, 2024-09-14T22:36:40.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054032430999999999, 2024-09-14T22:36:40.939 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:40.939 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:40.939 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:40.939 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:40.939 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:40.939 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:40.939 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6392 2024-09-14T22:36:40.940 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.940 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:40.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.940 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:40.940 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933648+0000", 2024-09-14T22:36:40.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.940 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.940 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:40.940 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933648+0000", 2024-09-14T22:36:40.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.940 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.940 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:40.941 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933650+0000", 2024-09-14T22:36:40.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3969999999999999e-06 2024-09-14T22:36:40.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.941 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:40.941 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933655+0000", 2024-09-14T22:36:40.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7690000000000002e-06 2024-09-14T22:36:40.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.941 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:40.941 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933672+0000", 2024-09-14T22:36:40.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6359999999999999e-05 2024-09-14T22:36:40.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.942 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:40.942 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933696+0000", 2024-09-14T22:36:40.942 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4630999999999998e-05 2024-09-14T22:36:40.942 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.942 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.942 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:40.942 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987490+0000", 2024-09-14T22:36:40.942 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053793256999999997 2024-09-14T22:36:40.942 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.942 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.942 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:40.942 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987572+0000", 2024-09-14T22:36:40.942 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1897999999999994e-05 2024-09-14T22:36:40.942 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:40.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987681+0000", 2024-09-14T22:36:40.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109119 2024-09-14T22:36:40.943 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.943 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:40.943 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.943 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6393 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8370176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:40.943 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933713+0000", 2024-09-14T22:36:40.943 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.928257465000002, 2024-09-14T22:36:40.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054194111000000003, 2024-09-14T22:36:40.943 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:40.944 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:40.944 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:40.944 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:40.944 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:40.944 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:40.944 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6393 2024-09-14T22:36:40.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.944 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:40.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:40.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933713+0000", 2024-09-14T22:36:40.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:40.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933713+0000", 2024-09-14T22:36:40.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:40.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933714+0000", 2024-09-14T22:36:40.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2109999999999999e-06 2024-09-14T22:36:40.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:40.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933721+0000", 2024-09-14T22:36:40.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2489999999999998e-06 2024-09-14T22:36:40.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.946 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:40.946 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933751+0000", 2024-09-14T22:36:40.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0239999999999998e-05 2024-09-14T22:36:40.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.946 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:40.946 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933764+0000", 2024-09-14T22:36:40.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.322e-05 2024-09-14T22:36:40.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.946 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:40.946 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987737+0000", 2024-09-14T22:36:40.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053972539 2024-09-14T22:36:40.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:40.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987813+0000", 2024-09-14T22:36:40.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6635000000000001e-05 2024-09-14T22:36:40.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:40.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987907+0000", 2024-09-14T22:36:40.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4017000000000005e-05 2024-09-14T22:36:40.947 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.947 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:40.947 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.947 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6394 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8378368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:40.947 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933781+0000", 2024-09-14T22:36:40.948 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.928189706000001, 2024-09-14T22:36:40.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054378276000000003, 2024-09-14T22:36:40.948 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:40.948 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:40.948 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:40.948 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:40.948 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:40.948 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:40.948 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6394 2024-09-14T22:36:40.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.948 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:40.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:40.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933781+0000", 2024-09-14T22:36:40.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:40.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933781+0000", 2024-09-14T22:36:40.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:40.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933782+0000", 2024-09-14T22:36:40.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9199999999999999e-07 2024-09-14T22:36:40.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:40.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933788+0000", 2024-09-14T22:36:40.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9340000000000003e-06 2024-09-14T22:36:40.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.950 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:40.950 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933803+0000", 2024-09-14T22:36:40.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5362999999999998e-05 2024-09-14T22:36:40.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.950 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:40.950 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933816+0000", 2024-09-14T22:36:40.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2456999999999999e-05 2024-09-14T22:36:40.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.950 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:40.950 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987961+0000", 2024-09-14T22:36:40.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054145581999999998 2024-09-14T22:36:40.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:40.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988042+0000", 2024-09-14T22:36:40.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1075000000000006e-05 2024-09-14T22:36:40.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:40.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988159+0000", 2024-09-14T22:36:40.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116873 2024-09-14T22:36:40.951 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.951 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:40.952 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.952 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6395 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8386560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:40.952 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:36:40.952 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.928138812, 2024-09-14T22:36:40.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054534379000000001, 2024-09-14T22:36:40.952 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:40.952 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:40.952 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:40.952 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:40.952 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:40.952 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:40.952 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6395 2024-09-14T22:36:40.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.953 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:40.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:40.953 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:36:40.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:40.953 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:36:40.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:40.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:40.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933833+0000", 2024-09-14T22:36:40.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3859999999999999e-06 2024-09-14T22:36:40.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:40.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933838+0000", 2024-09-14T22:36:40.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8709999999999996e-06 2024-09-14T22:36:40.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:40.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933854+0000", 2024-09-14T22:36:40.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6189e-05 2024-09-14T22:36:40.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:40.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933871+0000", 2024-09-14T22:36:40.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.626e-05 2024-09-14T22:36:40.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:40.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988198+0000", 2024-09-14T22:36:40.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054327581999999999 2024-09-14T22:36:40.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:40.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988274+0000", 2024-09-14T22:36:40.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5759000000000004e-05 2024-09-14T22:36:40.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:40.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:40.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:40.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988366+0000", 2024-09-14T22:36:40.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2331999999999996e-05 2024-09-14T22:36:40.956 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.956 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:40.956 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.956 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:40.956 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:40.956 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:40.958 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:36:40.959 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:40.951+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:41.049 INFO:teuthology.orchestra.run.smithi001.stdout:1097 writesame 0x1caf552 thru 0x1cb271b data_size 0x1(0x31ca bytes) 2024-09-14T22:36:41.058 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:41.054+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:41.072 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:41.074 INFO:teuthology.orchestra.run.smithi001.stdout:1100 trunc from 0xc55b72f to 0xd8f5728 2024-09-14T22:36:41.084 INFO:teuthology.orchestra.run.smithi001.stdout:1101 writesame 0xcef585a thru 0xcefe756 data_size 0x1(0x8efd bytes) 2024-09-14T22:36:41.099 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:41.099 INFO:teuthology.orchestra.run.smithi001.stdout:1105 write 0x165fa48 thru 0x16688e7 (0x8ea0 bytes) 2024-09-14T22:36:41.112 INFO:teuthology.orchestra.run.smithi001.stdout:1106 punch from 0xcf5b8fe to 0xcf68184, (0xc886 bytes) 2024-09-14T22:36:41.112 INFO:teuthology.orchestra.run.smithi001.stdout:1107 write 0x8b5eb71 thru 0x8b65180 (0x6610 bytes) 2024-09-14T22:36:41.122 INFO:teuthology.orchestra.run.smithi001.stdout:1109 read 0x42ed074 thru 0x42f29ea (0x5977 bytes) 2024-09-14T22:36:41.123 INFO:teuthology.orchestra.run.smithi001.stdout:1110 write 0x9a618a7 thru 0x9a68721 (0x6e7b bytes) 2024-09-14T22:36:41.129 INFO:teuthology.orchestra.run.smithi001.stdout:1111 punch from 0x1fa0f18 to 0x1fa17c3, (0x8ab bytes) 2024-09-14T22:36:41.129 INFO:teuthology.orchestra.run.smithi001.stdout:1113 read 0x777732e thru 0x7779039 (0x1d0c bytes) 2024-09-14T22:36:41.130 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:41.130 INFO:teuthology.orchestra.run.smithi001.stdout:1115 punch from 0x9afb4a to 0x9bf3b5, (0xf86b bytes) 2024-09-14T22:36:41.130 INFO:teuthology.orchestra.run.smithi001.stdout:1116 write 0x63d8faa thru 0x63e14f0 (0x8547 bytes) 2024-09-14T22:36:41.144 INFO:teuthology.orchestra.run.smithi001.stdout:1117 writesame 0x1896138 thru 0x189b737 data_size 0x2b(0x5600 bytes) 2024-09-14T22:36:41.144 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:41.145 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:41.145 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:36:41.145 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:41.157 INFO:teuthology.orchestra.run.smithi001.stdout:1118 read 0x412e35 thru 0x4191fd (0x63c9 bytes) 2024-09-14T22:36:41.158 INFO:teuthology.orchestra.run.smithi001.stdout:1119 writesame 0x984832e thru 0x9855b1d data_size 0x28(0xd7f0 bytes) 2024-09-14T22:36:41.159 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:41.155+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:41.160 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:36:41.163 INFO:teuthology.orchestra.run.smithi001.stdout:1120 write 0x1d24723 thru 0x1d26177 (0x1a55 bytes) 2024-09-14T22:36:41.173 INFO:teuthology.orchestra.run.smithi001.stdout:1122 punch from 0x1aa5191 to 0x1aa8e54, (0x3cc3 bytes) 2024-09-14T22:36:41.173 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:41.173 INFO:teuthology.orchestra.run.smithi001.stdout:1124 write 0x6048f4f thru 0x6054aed (0xbb9f bytes) 2024-09-14T22:36:41.197 INFO:teuthology.orchestra.run.smithi001.stdout:1126 trunc from 0xd8f5728 to 0x5e25b0c 2024-09-14T22:36:41.226 INFO:teuthology.orchestra.run.smithi001.stdout:1128 punch from 0x8c25ed to 0x8c87e3, (0x61f6 bytes) 2024-09-14T22:36:41.237 INFO:teuthology.orchestra.run.smithi001.stdout:1129 write 0x7ed7d4b thru 0x7ee37c1 (0xba77 bytes) 2024-09-14T22:36:41.258 INFO:teuthology.orchestra.run.smithi001.stdout:1130 writesame 0xab14f84 thru 0xab2258c data_size 0x1(0xd609 bytes) 2024-09-14T22:36:41.260 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:41.255+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:41.273 INFO:teuthology.orchestra.run.smithi001.stdout:1132 trunc from 0xab2258d to 0x39c080d 2024-09-14T22:36:41.302 INFO:teuthology.orchestra.run.smithi001.stdout:1133 punch from 0x9cc604 to 0x9db4a3, (0xee9f bytes) 2024-09-14T22:36:41.303 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:41.303 INFO:teuthology.orchestra.run.smithi001.stdout:1135 punch from 0x32e79fd to 0x32ef84b, (0x7e4e bytes) 2024-09-14T22:36:41.303 INFO:teuthology.orchestra.run.smithi001.stdout:1136 punch from 0x3642fcb to 0x3646aab, (0x3ae0 bytes) 2024-09-14T22:36:41.303 INFO:teuthology.orchestra.run.smithi001.stdout:1138 trunc from 0x39c080d to 0x3f92c66 2024-09-14T22:36:41.322 INFO:teuthology.orchestra.run.smithi001.stdout:1139 trunc from 0x3f92c66 to 0x7436f24 2024-09-14T22:36:41.334 INFO:teuthology.orchestra.run.smithi001.stdout:1140 clone 9 order 20 su 524288 sc 4 2024-09-14T22:36:41.340 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:41.341 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:41.341 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:36:41.341 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:36:41.341 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:41.355 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:36:41.399 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:41.395+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:41.461 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:41.455+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:41.527 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:41.528 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:36:41.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:36:41.528 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:36:41.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.528 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19590 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7272448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:41.528 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446391+0000", 2024-09-14T22:36:41.528 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.0766266130000002, 2024-09-14T22:36:41.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027116533000000002, 2024-09-14T22:36:41.528 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:41.528 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:41.528 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:41.528 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:41.528 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:41.529 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:41.529 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19590 2024-09-14T22:36:41.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.529 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:41.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:41.529 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446391+0000", 2024-09-14T22:36:41.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:41.529 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446391+0000", 2024-09-14T22:36:41.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:41.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446392+0000", 2024-09-14T22:36:41.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4299999999999999e-07 2024-09-14T22:36:41.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:41.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446393+0000", 2024-09-14T22:36:41.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8059999999999999e-06 2024-09-14T22:36:41.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:41.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446399+0000", 2024-09-14T22:36:41.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6740000000000002e-06 2024-09-14T22:36:41.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:41.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446404+0000", 2024-09-14T22:36:41.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6240000000000001e-06 2024-09-14T22:36:41.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:41.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473382+0000", 2024-09-14T22:36:41.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026978716999999999 2024-09-14T22:36:41.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.532 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:41.532 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473446+0000", 2024-09-14T22:36:41.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3971000000000001e-05 2024-09-14T22:36:41.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.532 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:41.532 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473508+0000", 2024-09-14T22:36:41.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1397999999999997e-05 2024-09-14T22:36:41.532 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.532 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:41.532 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.533 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19591 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7301120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:41.533 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446415+0000", 2024-09-14T22:36:41.533 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.0766032590000005, 2024-09-14T22:36:41.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027255773, 2024-09-14T22:36:41.533 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:41.533 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:41.533 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:41.533 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:41.533 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:41.533 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:41.533 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19591 2024-09-14T22:36:41.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.534 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:41.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.534 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:41.534 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446415+0000", 2024-09-14T22:36:41.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.534 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:41.534 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446415+0000", 2024-09-14T22:36:41.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.534 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:41.535 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446415+0000", 2024-09-14T22:36:41.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8800000000000002e-07 2024-09-14T22:36:41.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:41.535 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446417+0000", 2024-09-14T22:36:41.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7990000000000001e-06 2024-09-14T22:36:41.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:41.535 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446423+0000", 2024-09-14T22:36:41.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.066e-06 2024-09-14T22:36:41.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:41.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446428+0000", 2024-09-14T22:36:41.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5889999999999996e-06 2024-09-14T22:36:41.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:41.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473535+0000", 2024-09-14T22:36:41.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027107220000000001 2024-09-14T22:36:41.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:41.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473602+0000", 2024-09-14T22:36:41.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6482999999999993e-05 2024-09-14T22:36:41.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:41.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473670+0000", 2024-09-14T22:36:41.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8928000000000001e-05 2024-09-14T22:36:41.537 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.537 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:41.537 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.537 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19592 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7329792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:41.537 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446433+0000", 2024-09-14T22:36:41.537 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.0765845980000002, 2024-09-14T22:36:41.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027394960999999999, 2024-09-14T22:36:41.537 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:41.538 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:41.538 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:41.538 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:41.538 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:41.538 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:41.538 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19592 2024-09-14T22:36:41.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.538 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:41.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.538 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:41.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446433+0000", 2024-09-14T22:36:41.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.538 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:41.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446433+0000", 2024-09-14T22:36:41.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:41.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446434+0000", 2024-09-14T22:36:41.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3600000000000004e-07 2024-09-14T22:36:41.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:41.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446436+0000", 2024-09-14T22:36:41.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.796e-06 2024-09-14T22:36:41.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:41.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446441+0000", 2024-09-14T22:36:41.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7209999999999999e-06 2024-09-14T22:36:41.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:41.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446450+0000", 2024-09-14T22:36:41.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0780000000000002e-06 2024-09-14T22:36:41.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:41.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473705+0000", 2024-09-14T22:36:41.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027254246999999999 2024-09-14T22:36:41.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:41.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473769+0000", 2024-09-14T22:36:41.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4101000000000002e-05 2024-09-14T22:36:41.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:41.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473828+0000", 2024-09-14T22:36:41.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9481999999999997e-05 2024-09-14T22:36:41.541 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.541 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:41.541 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.541 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20100 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7219200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:41.541 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462167+0000", 2024-09-14T22:36:41.542 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.0608508749999999, 2024-09-14T22:36:41.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027082189999999999, 2024-09-14T22:36:41.542 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:41.542 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:41.542 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:41.542 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:41.542 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:41.542 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:41.542 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20100 2024-09-14T22:36:41.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.542 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:41.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:41.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462167+0000", 2024-09-14T22:36:41.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:41.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462167+0000", 2024-09-14T22:36:41.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:41.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462168+0000", 2024-09-14T22:36:41.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8800000000000002e-07 2024-09-14T22:36:41.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.544 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:41.544 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462177+0000", 2024-09-14T22:36:41.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9330000000000007e-06 2024-09-14T22:36:41.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.544 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:41.544 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462183+0000", 2024-09-14T22:36:41.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6200000000000001e-06 2024-09-14T22:36:41.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:41.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462188+0000", 2024-09-14T22:36:41.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1590000000000001e-06 2024-09-14T22:36:41.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:41.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489109+0000", 2024-09-14T22:36:41.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026920881000000001 2024-09-14T22:36:41.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:41.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489181+0000", 2024-09-14T22:36:41.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2035999999999999e-05 2024-09-14T22:36:41.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:41.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489249+0000", 2024-09-14T22:36:41.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7972999999999994e-05 2024-09-14T22:36:41.546 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.546 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:41.546 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.546 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20101 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7247872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:41.546 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462195+0000", 2024-09-14T22:36:41.546 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.0608225919999992, 2024-09-14T22:36:41.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027205994000000001, 2024-09-14T22:36:41.546 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:41.547 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:41.547 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:41.547 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:41.547 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:41.547 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:41.547 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20101 2024-09-14T22:36:41.547 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.547 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:41.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.547 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:41.547 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462195+0000", 2024-09-14T22:36:41.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.547 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.548 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:41.548 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462195+0000", 2024-09-14T22:36:41.548 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.548 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.548 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:41.548 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462196+0000", 2024-09-14T22:36:41.548 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8000000000000001e-07 2024-09-14T22:36:41.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.548 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.548 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:41.548 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462198+0000", 2024-09-14T22:36:41.548 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0439999999999998e-06 2024-09-14T22:36:41.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.549 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:41.549 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462205+0000", 2024-09-14T22:36:41.549 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3359999999999997e-06 2024-09-14T22:36:41.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.549 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:41.549 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462210+0000", 2024-09-14T22:36:41.549 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0649999999999998e-06 2024-09-14T22:36:41.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.549 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:41.550 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489279+0000", 2024-09-14T22:36:41.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027069123 2024-09-14T22:36:41.550 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.550 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:41.550 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489342+0000", 2024-09-14T22:36:41.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2759e-05 2024-09-14T22:36:41.550 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.550 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:41.550 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489401+0000", 2024-09-14T22:36:41.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9286999999999997e-05 2024-09-14T22:36:41.550 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.550 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:41.551 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.551 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.551 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20102 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7276544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:41.551 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462216+0000", 2024-09-14T22:36:41.551 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.0608016439999997, 2024-09-14T22:36:41.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027352785000000001, 2024-09-14T22:36:41.551 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:41.551 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:41.551 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:41.551 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:41.551 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:41.551 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:41.551 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20102 2024-09-14T22:36:41.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.552 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:41.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.552 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:41.552 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462216+0000", 2024-09-14T22:36:41.552 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.552 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:41.552 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462216+0000", 2024-09-14T22:36:41.552 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.552 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:41.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462217+0000", 2024-09-14T22:36:41.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9100000000000003e-07 2024-09-14T22:36:41.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.553 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.553 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:41.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462219+0000", 2024-09-14T22:36:41.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9740000000000001e-06 2024-09-14T22:36:41.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.553 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.553 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:41.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462235+0000", 2024-09-14T22:36:41.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6050999999999999e-05 2024-09-14T22:36:41.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.553 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.554 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:41.554 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462247+0000", 2024-09-14T22:36:41.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2245999999999999e-05 2024-09-14T22:36:41.554 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.554 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:41.554 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489431+0000", 2024-09-14T22:36:41.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027183624999999999 2024-09-14T22:36:41.554 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.554 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:41.554 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489504+0000", 2024-09-14T22:36:41.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3498999999999995e-05 2024-09-14T22:36:41.554 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.554 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:41.555 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489569+0000", 2024-09-14T22:36:41.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4698999999999998e-05 2024-09-14T22:36:41.555 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.555 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:41.555 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.555 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.555 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20103 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7305216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:41.555 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462256+0000", 2024-09-14T22:36:41.555 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.0607624379999994, 2024-09-14T22:36:41.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027473543, 2024-09-14T22:36:41.555 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:41.555 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:41.555 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:41.555 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:41.555 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:41.556 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:41.556 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20103 2024-09-14T22:36:41.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.556 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:41.556 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.556 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:41.556 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462256+0000", 2024-09-14T22:36:41.556 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.556 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.556 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:41.556 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462256+0000", 2024-09-14T22:36:41.556 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.557 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:41.557 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462256+0000", 2024-09-14T22:36:41.557 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1399999999999997e-07 2024-09-14T22:36:41.557 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.557 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:41.557 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462259+0000", 2024-09-14T22:36:41.557 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.694e-06 2024-09-14T22:36:41.557 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.557 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:41.557 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462266+0000", 2024-09-14T22:36:41.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8940000000000004e-06 2024-09-14T22:36:41.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.558 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:41.558 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462271+0000", 2024-09-14T22:36:41.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0459999999999996e-06 2024-09-14T22:36:41.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.558 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:41.558 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489601+0000", 2024-09-14T22:36:41.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027330027 2024-09-14T22:36:41.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.558 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:41.558 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489663+0000", 2024-09-14T22:36:41.559 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2807999999999999e-05 2024-09-14T22:36:41.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.559 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.559 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:41.559 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489729+0000", 2024-09-14T22:36:41.559 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5560000000000002e-05 2024-09-14T22:36:41.559 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.559 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:41.559 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.559 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.559 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20104 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7333888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:41.559 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462277+0000", 2024-09-14T22:36:41.559 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.0607409019999992, 2024-09-14T22:36:41.559 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027652614999999998, 2024-09-14T22:36:41.560 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:41.560 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:41.560 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:41.560 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:41.560 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:41.560 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:41.560 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20104 2024-09-14T22:36:41.560 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.560 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:41.560 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.560 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:41.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462277+0000", 2024-09-14T22:36:41.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.561 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.561 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.561 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:41.561 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462277+0000", 2024-09-14T22:36:41.561 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.561 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.561 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.561 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:41.561 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462277+0000", 2024-09-14T22:36:41.561 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2800000000000002e-07 2024-09-14T22:36:41.561 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.561 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.561 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:41.562 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462280+0000", 2024-09-14T22:36:41.562 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2570000000000002e-06 2024-09-14T22:36:41.562 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.562 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.562 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:41.562 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462286+0000", 2024-09-14T22:36:41.562 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2199999999999997e-06 2024-09-14T22:36:41.562 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.562 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.562 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:41.562 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462291+0000", 2024-09-14T22:36:41.562 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5689999999999998e-06 2024-09-14T22:36:41.562 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.562 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.563 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:41.563 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489760+0000", 2024-09-14T22:36:41.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027469345999999999 2024-09-14T22:36:41.563 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.563 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:41.563 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489852+0000", 2024-09-14T22:36:41.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1941999999999995e-05 2024-09-14T22:36:41.563 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.563 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:41.563 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489930+0000", 2024-09-14T22:36:41.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7853000000000002e-05 2024-09-14T22:36:41.563 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.563 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:41.564 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.564 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.564 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.564 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20869 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7168000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:41.564 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483633+0000", 2024-09-14T22:36:41.564 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.0393845469999992, 2024-09-14T22:36:41.564 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028181490999999999, 2024-09-14T22:36:41.564 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:41.564 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:41.564 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:41.564 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:41.564 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:41.564 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:41.564 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20869 2024-09-14T22:36:41.564 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.565 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:41.565 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.565 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:41.565 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483633+0000", 2024-09-14T22:36:41.565 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.565 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.565 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.565 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:41.565 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483633+0000", 2024-09-14T22:36:41.565 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.565 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.565 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.565 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:41.565 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483634+0000", 2024-09-14T22:36:41.565 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2599999999999998e-07 2024-09-14T22:36:41.566 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.566 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.566 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:41.566 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483636+0000", 2024-09-14T22:36:41.566 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1679999999999998e-06 2024-09-14T22:36:41.566 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.566 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.566 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:41.566 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483642+0000", 2024-09-14T22:36:41.566 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2149999999999998e-06 2024-09-14T22:36:41.566 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.566 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.566 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:41.567 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483647+0000", 2024-09-14T22:36:41.567 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7679999999999999e-06 2024-09-14T22:36:41.567 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.567 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.567 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:41.567 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.510694+0000", 2024-09-14T22:36:41.567 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027047073000000001 2024-09-14T22:36:41.567 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.567 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.567 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:41.567 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.510771+0000", 2024-09-14T22:36:41.567 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6814e-05 2024-09-14T22:36:41.567 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.567 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.568 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:41.568 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511815+0000", 2024-09-14T22:36:41.568 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0010440270000000001 2024-09-14T22:36:41.568 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.568 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:41.568 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.568 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.568 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.568 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20870 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7196672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:41.568 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483652+0000", 2024-09-14T22:36:41.568 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.0393655450000008, 2024-09-14T22:36:41.568 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028301248000000001, 2024-09-14T22:36:41.568 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:41.568 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:41.569 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:41.569 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:41.569 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:41.569 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:41.569 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20870 2024-09-14T22:36:41.569 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.569 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:41.569 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.569 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:41.569 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483652+0000", 2024-09-14T22:36:41.569 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.569 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.569 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.569 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:41.569 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483652+0000", 2024-09-14T22:36:41.570 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.570 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.570 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.570 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:41.570 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483653+0000", 2024-09-14T22:36:41.570 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3099999999999997e-07 2024-09-14T22:36:41.570 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.570 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.570 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:41.570 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483655+0000", 2024-09-14T22:36:41.570 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.877e-06 2024-09-14T22:36:41.570 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.570 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.570 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:41.570 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483661+0000", 2024-09-14T22:36:41.570 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6049999999999997e-06 2024-09-14T22:36:41.571 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.571 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.571 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:41.571 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483666+0000", 2024-09-14T22:36:41.571 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3410000000000002e-06 2024-09-14T22:36:41.571 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.571 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.571 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:41.571 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511854+0000", 2024-09-14T22:36:41.571 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028187773999999999 2024-09-14T22:36:41.571 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.571 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.571 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:41.571 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511906+0000", 2024-09-14T22:36:41.572 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2012000000000003e-05 2024-09-14T22:36:41.572 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.572 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.572 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:41.572 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511954+0000", 2024-09-14T22:36:41.572 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8007999999999998e-05 2024-09-14T22:36:41.572 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.572 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:41.572 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.572 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.572 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.572 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20871 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7225344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:41.572 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483671+0000", 2024-09-14T22:36:41.572 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.0393466220000001, 2024-09-14T22:36:41.573 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028405392000000002, 2024-09-14T22:36:41.573 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:41.573 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:41.573 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:41.573 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:41.573 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:41.573 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:41.573 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20871 2024-09-14T22:36:41.573 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.573 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:41.573 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.573 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:41.573 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483671+0000", 2024-09-14T22:36:41.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.574 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.574 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.574 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:41.574 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483671+0000", 2024-09-14T22:36:41.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.574 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.574 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.574 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:41.574 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483672+0000", 2024-09-14T22:36:41.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:36:41.574 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.574 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.574 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:41.575 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483674+0000", 2024-09-14T22:36:41.575 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3190000000000002e-06 2024-09-14T22:36:41.575 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.575 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.575 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:41.575 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483680+0000", 2024-09-14T22:36:41.575 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6860000000000003e-06 2024-09-14T22:36:41.575 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.575 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.575 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:41.575 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483684+0000", 2024-09-14T22:36:41.575 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3769999999999998e-06 2024-09-14T22:36:41.575 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.575 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.575 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:41.576 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511982+0000", 2024-09-14T22:36:41.576 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028297478000000001 2024-09-14T22:36:41.576 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.576 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.576 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:41.576 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512027+0000", 2024-09-14T22:36:41.576 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5290000000000002e-05 2024-09-14T22:36:41.576 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.576 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.576 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:41.576 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512077+0000", 2024-09-14T22:36:41.576 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9876000000000002e-05 2024-09-14T22:36:41.576 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.576 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:41.577 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.577 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.577 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.577 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20872 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7254016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:41.577 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483689+0000", 2024-09-14T22:36:41.577 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.0393285520000006, 2024-09-14T22:36:41.577 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028494049, 2024-09-14T22:36:41.577 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:41.577 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:41.577 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:41.577 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:41.577 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:41.577 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:41.577 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20872 2024-09-14T22:36:41.577 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.578 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:41.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.578 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:41.578 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483689+0000", 2024-09-14T22:36:41.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.578 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.578 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:41.578 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483689+0000", 2024-09-14T22:36:41.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.578 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.578 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:41.579 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483690+0000", 2024-09-14T22:36:41.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:36:41.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.579 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:41.579 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483692+0000", 2024-09-14T22:36:41.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.778e-06 2024-09-14T22:36:41.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.579 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:41.579 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483697+0000", 2024-09-14T22:36:41.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7069999999999997e-06 2024-09-14T22:36:41.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.580 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:41.580 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483702+0000", 2024-09-14T22:36:41.580 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4689999999999999e-06 2024-09-14T22:36:41.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.580 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:41.580 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512104+0000", 2024-09-14T22:36:41.580 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028401948999999999 2024-09-14T22:36:41.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.580 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:41.580 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512142+0000", 2024-09-14T22:36:41.580 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8114999999999997e-05 2024-09-14T22:36:41.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.581 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:41.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512183+0000", 2024-09-14T22:36:41.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1496999999999999e-05 2024-09-14T22:36:41.581 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.581 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:41.581 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.581 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20873 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7282688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:41.581 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483707+0000", 2024-09-14T22:36:41.581 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.0393108380000005, 2024-09-14T22:36:41.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028581662000000001, 2024-09-14T22:36:41.581 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:41.582 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:41.582 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:41.582 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:41.582 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:41.582 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:41.582 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20873 2024-09-14T22:36:41.582 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.582 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:41.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.583 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:41.583 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483707+0000", 2024-09-14T22:36:41.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.583 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:41.583 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483707+0000", 2024-09-14T22:36:41.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.583 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:41.583 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483707+0000", 2024-09-14T22:36:41.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2500000000000001e-07 2024-09-14T22:36:41.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:41.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483709+0000", 2024-09-14T22:36:41.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.891e-06 2024-09-14T22:36:41.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:41.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483715+0000", 2024-09-14T22:36:41.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7019999999999998e-06 2024-09-14T22:36:41.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:41.585 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483720+0000", 2024-09-14T22:36:41.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4900000000000002e-06 2024-09-14T22:36:41.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:41.585 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512213+0000", 2024-09-14T22:36:41.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028493622 2024-09-14T22:36:41.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:41.585 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512248+0000", 2024-09-14T22:36:41.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5228000000000002e-05 2024-09-14T22:36:41.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.586 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:41.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512289+0000", 2024-09-14T22:36:41.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0404000000000001e-05 2024-09-14T22:36:41.586 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.586 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:41.586 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.586 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20874 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7311360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:41.586 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483725+0000", 2024-09-14T22:36:41.586 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.0392931989999994, 2024-09-14T22:36:41.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028777193999999999, 2024-09-14T22:36:41.586 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:41.587 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:41.587 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:41.587 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:41.587 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:41.587 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:41.587 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20874 2024-09-14T22:36:41.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.587 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:41.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.587 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:41.587 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483725+0000", 2024-09-14T22:36:41.587 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:41.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483725+0000", 2024-09-14T22:36:41.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:41.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483725+0000", 2024-09-14T22:36:41.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8000000000000001e-07 2024-09-14T22:36:41.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:41.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483727+0000", 2024-09-14T22:36:41.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8920000000000001e-06 2024-09-14T22:36:41.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:41.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483733+0000", 2024-09-14T22:36:41.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6840000000000001e-06 2024-09-14T22:36:41.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:41.590 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483737+0000", 2024-09-14T22:36:41.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4440000000000001e-06 2024-09-14T22:36:41.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:41.590 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512326+0000", 2024-09-14T22:36:41.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028588612999999999 2024-09-14T22:36:41.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:41.590 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512399+0000", 2024-09-14T22:36:41.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2782999999999998e-05 2024-09-14T22:36:41.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.591 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:41.591 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512502+0000", 2024-09-14T22:36:41.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000103398 2024-09-14T22:36:41.591 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.591 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:41.591 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.591 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20912 3.1c 3:3b9f022e:::rbd_data.113c4558bb1.000000000000009a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 497664~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:41.591 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.484800+0000", 2024-09-14T22:36:41.591 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.038217671, 2024-09-14T22:36:41.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045787688999999999, 2024-09-14T22:36:41.592 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:41.592 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:41.592 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:41.592 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:41.592 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:41.592 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:41.592 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20912 2024-09-14T22:36:41.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.592 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:41.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.592 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:41.592 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484800+0000", 2024-09-14T22:36:41.592 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.593 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:41.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484800+0000", 2024-09-14T22:36:41.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.593 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:41.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484801+0000", 2024-09-14T22:36:41.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1939999999999999e-06 2024-09-14T22:36:41.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.593 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:41.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484809+0000", 2024-09-14T22:36:41.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2989999999999999e-06 2024-09-14T22:36:41.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:41.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484823+0000", 2024-09-14T22:36:41.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3986e-05 2024-09-14T22:36:41.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:41.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484831+0000", 2024-09-14T22:36:41.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4009999999999997e-06 2024-09-14T22:36:41.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:41.595 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512318+0000", 2024-09-14T22:36:41.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027487208999999999 2024-09-14T22:36:41.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.595 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:41.595 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512450+0000", 2024-09-14T22:36:41.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000131681 2024-09-14T22:36:41.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.595 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:36:41.595 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512666+0000", 2024-09-14T22:36:41.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021547 2024-09-14T22:36:41.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.595 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:41.595 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.513788+0000", 2024-09-14T22:36:41.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001122975 2024-09-14T22:36:41.596 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.596 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.596 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:41.596 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.530473+0000", 2024-09-14T22:36:41.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.016684984999999999 2024-09-14T22:36:41.596 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.596 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.596 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:41.596 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.530533+0000", 2024-09-14T22:36:41.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9296999999999998e-05 2024-09-14T22:36:41.596 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.596 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.596 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:41.597 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.530588+0000", 2024-09-14T22:36:41.597 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5192000000000003e-05 2024-09-14T22:36:41.597 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.597 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:41.597 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.597 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:30512 3.1d 3:b9e08d52:::rbd_data.113cda0183a0.0000000000000008:6 [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:41.597 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.331867+0000", 2024-09-14T22:36:41.597 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.1911507169999993, 2024-09-14T22:36:41.597 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027901088000000001, 2024-09-14T22:36:41.597 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:41.597 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:41.597 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:41.598 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:41.598 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:41.598 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:41.598 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 30512 2024-09-14T22:36:41.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.598 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:41.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:41.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331867+0000", 2024-09-14T22:36:41.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:41.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331867+0000", 2024-09-14T22:36:41.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:41.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331868+0000", 2024-09-14T22:36:41.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8999999999999996e-07 2024-09-14T22:36:41.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:41.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331871+0000", 2024-09-14T22:36:41.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.163e-06 2024-09-14T22:36:41.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.600 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:41.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331880+0000", 2024-09-14T22:36:41.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3449999999999995e-06 2024-09-14T22:36:41.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.600 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:41.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331888+0000", 2024-09-14T22:36:41.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5320000000000001e-06 2024-09-14T22:36:41.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.600 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:41.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.358443+0000", 2024-09-14T22:36:41.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026554845000000001 2024-09-14T22:36:41.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:41.601 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.359649+0000", 2024-09-14T22:36:41.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001205991 2024-09-14T22:36:41.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:41.601 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.359768+0000", 2024-09-14T22:36:41.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119522 2024-09-14T22:36:41.601 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.601 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:41.601 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.602 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:30513 3.1d 3:b9e08d52:::rbd_data.113cda0183a0.0000000000000008:6 [read 456704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:41.602 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.331897+0000", 2024-09-14T22:36:41.602 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.1911204909999995, 2024-09-14T22:36:41.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028098122999999999, 2024-09-14T22:36:41.602 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:41.602 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:41.602 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:41.602 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:41.602 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:41.602 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:41.602 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 30513 2024-09-14T22:36:41.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.603 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:41.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:41.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331897+0000", 2024-09-14T22:36:41.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:41.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331897+0000", 2024-09-14T22:36:41.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.604 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:41.604 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331898+0000", 2024-09-14T22:36:41.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:36:41.604 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.604 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:41.604 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331901+0000", 2024-09-14T22:36:41.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7889999999999999e-06 2024-09-14T22:36:41.604 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.604 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:41.604 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331910+0000", 2024-09-14T22:36:41.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.346e-06 2024-09-14T22:36:41.604 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.605 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.605 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:41.605 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331917+0000", 2024-09-14T22:36:41.605 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2239999999999998e-06 2024-09-14T22:36:41.605 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.605 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.605 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:41.605 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.359807+0000", 2024-09-14T22:36:41.605 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027889225 2024-09-14T22:36:41.605 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.605 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.605 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:41.605 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.359895+0000", 2024-09-14T22:36:41.606 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8721000000000007e-05 2024-09-14T22:36:41.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.606 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:41.606 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.359996+0000", 2024-09-14T22:36:41.606 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010022300000000001 2024-09-14T22:36:41.606 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.606 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:41.606 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.606 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:30514 3.1d 3:b9e08d52:::rbd_data.113cda0183a0.0000000000000008:6 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:41.606 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.331927+0000", 2024-09-14T22:36:41.606 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.1910906780000001, 2024-09-14T22:36:41.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028476330000000001, 2024-09-14T22:36:41.607 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:41.607 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:41.607 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:41.607 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:41.607 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:41.607 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:41.607 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 30514 2024-09-14T22:36:41.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.607 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:41.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:41.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331927+0000", 2024-09-14T22:36:41.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:41.608 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331927+0000", 2024-09-14T22:36:41.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:41.608 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331928+0000", 2024-09-14T22:36:41.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5700000000000002e-07 2024-09-14T22:36:41.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:41.609 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331931+0000", 2024-09-14T22:36:41.609 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9909999999999998e-06 2024-09-14T22:36:41.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.609 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.609 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:41.609 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331940+0000", 2024-09-14T22:36:41.609 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.312e-06 2024-09-14T22:36:41.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.609 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.609 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:41.609 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331948+0000", 2024-09-14T22:36:41.609 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5630000000000003e-06 2024-09-14T22:36:41.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.609 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.610 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:41.610 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360032+0000", 2024-09-14T22:36:41.610 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028083877 2024-09-14T22:36:41.610 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.610 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.610 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:41.610 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360282+0000", 2024-09-14T22:36:41.610 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00025062100000000003 2024-09-14T22:36:41.610 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.610 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.610 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:41.610 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360404+0000", 2024-09-14T22:36:41.610 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000121409 2024-09-14T22:36:41.610 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.610 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:41.611 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.611 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:30515 3.1d 3:b9e08d52:::rbd_data.113cda0183a0.0000000000000008:6 [read 514048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:41.611 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.331957+0000", 2024-09-14T22:36:41.611 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.1910608529999998, 2024-09-14T22:36:41.611 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028684729999999999, 2024-09-14T22:36:41.611 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:41.611 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:41.611 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:41.611 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:41.611 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:41.611 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:41.611 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 30515 2024-09-14T22:36:41.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.612 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:41.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:41.612 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331957+0000", 2024-09-14T22:36:41.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:41.612 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331957+0000", 2024-09-14T22:36:41.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:41.612 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331958+0000", 2024-09-14T22:36:41.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1200000000000003e-07 2024-09-14T22:36:41.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.616 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:41.616 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331960+0000", 2024-09-14T22:36:41.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6929999999999999e-06 2024-09-14T22:36:41.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.617 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:41.617 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331970+0000", 2024-09-14T22:36:41.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4020000000000008e-06 2024-09-14T22:36:41.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.617 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:41.617 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331977+0000", 2024-09-14T22:36:41.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7660000000000001e-06 2024-09-14T22:36:41.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.617 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:41.618 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360444+0000", 2024-09-14T22:36:41.618 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028467399000000001 2024-09-14T22:36:41.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.618 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:41.618 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360536+0000", 2024-09-14T22:36:41.618 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1618000000000003e-05 2024-09-14T22:36:41.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.618 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:41.618 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360642+0000", 2024-09-14T22:36:41.618 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010624 2024-09-14T22:36:41.618 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.618 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:41.619 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.619 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:46576 3.1d 3:b9b1f07f:::rbd_data.113c8642facc.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T22:36:41.619 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:37.379223+0000", 2024-09-14T22:36:41.619 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.1437953280000004, 2024-09-14T22:36:41.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072706044999999997, 2024-09-14T22:36:41.619 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:41.619 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:41.619 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:41.619 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:41.619 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:41.619 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:41.619 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 46576 2024-09-14T22:36:41.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.620 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:41.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:41.620 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379223+0000", 2024-09-14T22:36:41.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:41.620 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379223+0000", 2024-09-14T22:36:41.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:41.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:41.621 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379226+0000", 2024-09-14T22:36:41.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0929999999999999e-06 2024-09-14T22:36:41.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.621 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:41.621 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379240+0000", 2024-09-14T22:36:41.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4606999999999999e-05 2024-09-14T22:36:41.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.621 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:41.621 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379272+0000", 2024-09-14T22:36:41.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2073000000000003e-05 2024-09-14T22:36:41.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.621 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:41.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379288+0000", 2024-09-14T22:36:41.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6036999999999999e-05 2024-09-14T22:36:41.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:41.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379451+0000", 2024-09-14T22:36:41.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000162795 2024-09-14T22:36:41.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:41.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379702+0000", 2024-09-14T22:36:41.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00025106500000000002 2024-09-14T22:36:41.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.623 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:36:41.623 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.394067+0000", 2024-09-14T22:36:41.623 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014364564999999999 2024-09-14T22:36:41.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.623 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:41.623 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.451737+0000", 2024-09-14T22:36:41.623 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057670185999999998 2024-09-14T22:36:41.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.623 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:41.623 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.451823+0000", 2024-09-14T22:36:41.623 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6075000000000005e-05 2024-09-14T22:36:41.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.623 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:41.624 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.451881+0000", 2024-09-14T22:36:41.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7454999999999998e-05 2024-09-14T22:36:41.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:41.624 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:41.624 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:41.624 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.451929+0000", 2024-09-14T22:36:41.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8093999999999997e-05 2024-09-14T22:36:41.624 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.624 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:41.624 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.624 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:41.624 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:41.624 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:41.626 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:36:41.626 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:41.622+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:41.726 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:41.719+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:41.794 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:41.794 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:41.794 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:36:41.794 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:41.808 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:36:41.846 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2024-09-14T22:36:41.851 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:41.847+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:41.928 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:41.923+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:41.978 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:41.978 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:41.978 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:36:41.978 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:36:41.978 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:41.981 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone8 from 0x7436f24 (overlap 0x39c080d) to 0x33fa0d8 2024-09-14T22:36:41.997 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:36:42.039 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:42.035+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:42.131 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:42.126+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:42.166 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:42.166 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:36:42.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:36:42.166 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:36:42.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.166 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:1241 3.10 3:0bfe6369:::rbd_object_map.113c3b74a67f.0000000000000004:head [writefull 0~39] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:42.166 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:22.744905+0000", 2024-09-14T22:36:42.166 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.415668653000001, 2024-09-14T22:36:42.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0063495579999999999, 2024-09-14T22:36:42.166 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.166 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.167 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:42.167 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.167 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.167 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.167 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 1241 2024-09-14T22:36:42.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.167 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.744905+0000", 2024-09-14T22:36:42.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.744905+0000", 2024-09-14T22:36:42.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.744908+0000", 2024-09-14T22:36:42.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2899999999999998e-06 2024-09-14T22:36:42.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.744923+0000", 2024-09-14T22:36:42.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4797000000000001e-05 2024-09-14T22:36:42.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.744958+0000", 2024-09-14T22:36:42.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5015999999999998e-05 2024-09-14T22:36:42.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.744974+0000", 2024-09-14T22:36:42.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.615e-05 2024-09-14T22:36:42.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.745133+0000", 2024-09-14T22:36:42.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000159145 2024-09-14T22:36:42.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op must wait for map", 2024-09-14T22:36:42.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.745145+0000", 2024-09-14T22:36:42.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1457000000000001e-05 2024-09-14T22:36:42.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.748491+0000", 2024-09-14T22:36:42.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0033466139999999999 2024-09-14T22:36:42.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.748681+0000", 2024-09-14T22:36:42.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000189638 2024-09-14T22:36:42.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:42.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.748893+0000", 2024-09-14T22:36:42.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000211855 2024-09-14T22:36:42.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:42.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.750340+0000", 2024-09-14T22:36:42.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001447126 2024-09-14T22:36:42.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:42.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.750975+0000", 2024-09-14T22:36:42.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00063514799999999998 2024-09-14T22:36:42.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:42.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.751131+0000", 2024-09-14T22:36:42.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015583900000000001 2024-09-14T22:36:42.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:22.751254+0000", 2024-09-14T22:36:42.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000123483 2024-09-14T22:36:42.172 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.172 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.172 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.172 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2151 3.a 3:5478be22:::rbd_data.113cde4a8b6b.0000000000000143:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:42.173 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.514710+0000", 2024-09-14T22:36:42.173 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.645863305999999, 2024-09-14T22:36:42.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012174763, 2024-09-14T22:36:42.173 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.173 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.173 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:42.173 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.173 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.173 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.173 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2151 2024-09-14T22:36:42.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.173 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.514710+0000", 2024-09-14T22:36:42.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.514710+0000", 2024-09-14T22:36:42.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.514712+0000", 2024-09-14T22:36:42.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3649999999999998e-06 2024-09-14T22:36:42.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.514723+0000", 2024-09-14T22:36:42.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0936e-05 2024-09-14T22:36:42.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.514747+0000", 2024-09-14T22:36:42.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3288000000000001e-05 2024-09-14T22:36:42.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.514756+0000", 2024-09-14T22:36:42.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6949999999999995e-06 2024-09-14T22:36:42.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.514873+0000", 2024-09-14T22:36:42.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011657700000000001 2024-09-14T22:36:42.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.515139+0000", 2024-09-14T22:36:42.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026613100000000002 2024-09-14T22:36:42.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:36:42.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.517580+0000", 2024-09-14T22:36:42.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002441365 2024-09-14T22:36:42.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:42.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.526706+0000", 2024-09-14T22:36:42.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0091256459999999994 2024-09-14T22:36:42.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:42.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.526762+0000", 2024-09-14T22:36:42.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5489999999999999e-05 2024-09-14T22:36:42.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:42.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.526815+0000", 2024-09-14T22:36:42.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3458000000000002e-05 2024-09-14T22:36:42.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.526885+0000", 2024-09-14T22:36:42.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9812000000000002e-05 2024-09-14T22:36:42.178 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.178 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.178 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.178 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:5363 3.10 3:0bb60ae6:::rbd_data.113cde4a8b6b.0000000000000094:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:42.178 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.907462+0000", 2024-09-14T22:36:42.178 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.253111704999998, 2024-09-14T22:36:42.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0080656949999999995, 2024-09-14T22:36:42.178 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.178 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.179 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:42.179 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.179 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.179 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.179 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 5363 2024-09-14T22:36:42.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.179 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.907462+0000", 2024-09-14T22:36:42.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.907462+0000", 2024-09-14T22:36:42.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.907465+0000", 2024-09-14T22:36:42.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.393e-06 2024-09-14T22:36:42.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.907479+0000", 2024-09-14T22:36:42.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3920000000000001e-05 2024-09-14T22:36:42.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.907521+0000", 2024-09-14T22:36:42.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2525999999999999e-05 2024-09-14T22:36:42.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.907546+0000", 2024-09-14T22:36:42.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4783e-05 2024-09-14T22:36:42.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.907769+0000", 2024-09-14T22:36:42.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022249499999999999 2024-09-14T22:36:42.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op must wait for map", 2024-09-14T22:36:42.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.907792+0000", 2024-09-14T22:36:42.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2912999999999999e-05 2024-09-14T22:36:42.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.913820+0000", 2024-09-14T22:36:42.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.006028828 2024-09-14T22:36:42.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.913959+0000", 2024-09-14T22:36:42.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013830900000000001 2024-09-14T22:36:42.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:42.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.914169+0000", 2024-09-14T22:36:42.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000209898 2024-09-14T22:36:42.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:42.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.915324+0000", 2024-09-14T22:36:42.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0011553640000000001 2024-09-14T22:36:42.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:42.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.915381+0000", 2024-09-14T22:36:42.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7086999999999997e-05 2024-09-14T22:36:42.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:42.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.915456+0000", 2024-09-14T22:36:42.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5038999999999998e-05 2024-09-14T22:36:42.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.915527+0000", 2024-09-14T22:36:42.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.114e-05 2024-09-14T22:36:42.184 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.184 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.184 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.184 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4412.0:25388 3.1e e36/23 3:7ee90710:::rbd_object_map.113c4558bb1:head v 36'122, mlcod=36'118)", 2024-09-14T22:36:42.184 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.849766+0000", 2024-09-14T22:36:42.184 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.3108073480000009, 2024-09-14T22:36:42.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0056330360000000001, 2024-09-14T22:36:42.185 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.185 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.185 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:42.185 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.849766+0000", 2024-09-14T22:36:42.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.849766+0000", 2024-09-14T22:36:42.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.849766+0000", 2024-09-14T22:36:42.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5600000000000001e-07 2024-09-14T22:36:42.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.849769+0000", 2024-09-14T22:36:42.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1149999999999998e-06 2024-09-14T22:36:42.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.849776+0000", 2024-09-14T22:36:42.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3729999999999998e-06 2024-09-14T22:36:42.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.849782+0000", 2024-09-14T22:36:42.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7849999999999997e-06 2024-09-14T22:36:42.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.849822+0000", 2024-09-14T22:36:42.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0584999999999999e-05 2024-09-14T22:36:42.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.849833+0000", 2024-09-14T22:36:42.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1280999999999999e-05 2024-09-14T22:36:42.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.855376+0000", 2024-09-14T22:36:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0055428370000000001 2024-09-14T22:36:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.855399+0000", 2024-09-14T22:36:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2704000000000001e-05 2024-09-14T22:36:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4412.0:29460 3.1e e36/23 3:7ee90710:::rbd_object_map.113c4558bb1:head v 36'251, mlcod=36'246)", 2024-09-14T22:36:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.268560+0000", 2024-09-14T22:36:42.189 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.892013296, 2024-09-14T22:36:42.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0062238609999999998, 2024-09-14T22:36:42.189 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.189 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.189 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:42.189 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.268560+0000", 2024-09-14T22:36:42.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.268560+0000", 2024-09-14T22:36:42.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.268561+0000", 2024-09-14T22:36:42.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1999999999999998e-07 2024-09-14T22:36:42.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.268568+0000", 2024-09-14T22:36:42.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1069999999999999e-06 2024-09-14T22:36:42.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.268585+0000", 2024-09-14T22:36:42.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7390000000000001e-05 2024-09-14T22:36:42.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.268595+0000", 2024-09-14T22:36:42.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5929999999999995e-06 2024-09-14T22:36:42.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.268647+0000", 2024-09-14T22:36:42.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1570000000000003e-05 2024-09-14T22:36:42.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.268664+0000", 2024-09-14T22:36:42.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7946000000000001e-05 2024-09-14T22:36:42.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:42.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.274739+0000", 2024-09-14T22:36:42.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0060749549999999999 2024-09-14T22:36:42.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.274784+0000", 2024-09-14T22:36:42.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4480000000000001e-05 2024-09-14T22:36:42.192 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.192 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.192 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.193 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4412.0:29461 3.1e e36/23 3:7ee90710:::rbd_object_map.113c4558bb1:head v 36'252, mlcod=36'248)", 2024-09-14T22:36:42.193 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.269830+0000", 2024-09-14T22:36:42.193 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.8907436480000008, 2024-09-14T22:36:42.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0055875689999999997, 2024-09-14T22:36:42.193 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.193 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.193 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:42.193 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.269830+0000", 2024-09-14T22:36:42.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.269830+0000", 2024-09-14T22:36:42.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.269830+0000", 2024-09-14T22:36:42.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9400000000000005e-07 2024-09-14T22:36:42.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.269834+0000", 2024-09-14T22:36:42.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0369999999999996e-06 2024-09-14T22:36:42.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.269843+0000", 2024-09-14T22:36:42.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4940000000000003e-06 2024-09-14T22:36:42.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.269847+0000", 2024-09-14T22:36:42.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6460000000000001e-06 2024-09-14T22:36:42.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.269890+0000", 2024-09-14T22:36:42.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2527999999999997e-05 2024-09-14T22:36:42.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.269910+0000", 2024-09-14T22:36:42.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0177999999999999e-05 2024-09-14T22:36:42.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:42.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.275380+0000", 2024-09-14T22:36:42.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0054699279999999998 2024-09-14T22:36:42.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.275417+0000", 2024-09-14T22:36:42.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7163999999999999e-05 2024-09-14T22:36:42.197 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.197 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.197 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.197 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:29488 3.a 3:55944348:::rbd_data.113c4558bb1.00000000000000c9:head [write 303104~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:42.197 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.281270+0000", 2024-09-14T22:36:42.197 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.8793038240000008, 2024-09-14T22:36:42.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0065184809999999996, 2024-09-14T22:36:42.197 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.197 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.197 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:42.197 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.197 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.197 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.198 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29488 2024-09-14T22:36:42.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.198 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.281270+0000", 2024-09-14T22:36:42.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.281270+0000", 2024-09-14T22:36:42.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.281270+0000", 2024-09-14T22:36:42.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9400000000000004e-07 2024-09-14T22:36:42.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.281276+0000", 2024-09-14T22:36:42.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1630000000000004e-06 2024-09-14T22:36:42.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.281281+0000", 2024-09-14T22:36:42.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6500000000000001e-06 2024-09-14T22:36:42.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.281285+0000", 2024-09-14T22:36:42.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3009999999999998e-06 2024-09-14T22:36:42.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.281754+0000", 2024-09-14T22:36:42.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00046994099999999998 2024-09-14T22:36:42.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.281864+0000", 2024-09-14T22:36:42.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010964900000000001 2024-09-14T22:36:42.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:36:42.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.282018+0000", 2024-09-14T22:36:42.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015433600000000001 2024-09-14T22:36:42.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:42.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.285885+0000", 2024-09-14T22:36:42.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0038670309999999999 2024-09-14T22:36:42.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:42.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.287716+0000", 2024-09-14T22:36:42.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0018305769999999999 2024-09-14T22:36:42.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:42.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.287763+0000", 2024-09-14T22:36:42.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6601000000000003e-05 2024-09-14T22:36:42.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.287788+0000", 2024-09-14T22:36:42.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5338e-05 2024-09-14T22:36:42.202 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.202 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.202 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.202 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:29494 3.a 3:5771f379:::rbd_data.113c4558bb1.00000000000000cf:head [write 303104~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:42.202 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.281513+0000", 2024-09-14T22:36:42.202 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.8790601230000004, 2024-09-14T22:36:42.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0065160160000000003, 2024-09-14T22:36:42.202 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.202 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.203 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:42.203 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.203 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.203 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.203 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29494 2024-09-14T22:36:42.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.203 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.281513+0000", 2024-09-14T22:36:42.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.281513+0000", 2024-09-14T22:36:42.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.281515+0000", 2024-09-14T22:36:42.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.342e-06 2024-09-14T22:36:42.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.281522+0000", 2024-09-14T22:36:42.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4270000000000002e-06 2024-09-14T22:36:42.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.281538+0000", 2024-09-14T22:36:42.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6138e-05 2024-09-14T22:36:42.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.281549+0000", 2024-09-14T22:36:42.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0713999999999999e-05 2024-09-14T22:36:42.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.282673+0000", 2024-09-14T22:36:42.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0011243990000000001 2024-09-14T22:36:42.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.282776+0000", 2024-09-14T22:36:42.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102469 2024-09-14T22:36:42.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:36:42.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.282943+0000", 2024-09-14T22:36:42.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016728899999999999 2024-09-14T22:36:42.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:42.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.285931+0000", 2024-09-14T22:36:42.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0029884740000000001 2024-09-14T22:36:42.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:42.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.287986+0000", 2024-09-14T22:36:42.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0020541639999999998 2024-09-14T22:36:42.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:42.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.288011+0000", 2024-09-14T22:36:42.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5335999999999999e-05 2024-09-14T22:36:42.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.288029+0000", 2024-09-14T22:36:42.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8264e-05 2024-09-14T22:36:42.207 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.207 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.207 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.208 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:46465 3.10 3:09ce7ae2:::rbd_data.113cc4e35b3.0000000000000001:8 [sparse-read 11665408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T22:36:42.208 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:37.265119+0000", 2024-09-14T22:36:42.208 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.8954541989999996, 2024-09-14T22:36:42.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.005788924, 2024-09-14T22:36:42.208 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.208 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.208 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:42.208 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.208 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.208 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.208 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 46465 2024-09-14T22:36:42.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.208 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265119+0000", 2024-09-14T22:36:42.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265119+0000", 2024-09-14T22:36:42.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265120+0000", 2024-09-14T22:36:42.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9700000000000002e-07 2024-09-14T22:36:42.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265123+0000", 2024-09-14T22:36:42.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1920000000000001e-06 2024-09-14T22:36:42.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265129+0000", 2024-09-14T22:36:42.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9769999999999997e-06 2024-09-14T22:36:42.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.265133+0000", 2024-09-14T22:36:42.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7439999999999998e-06 2024-09-14T22:36:42.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.270662+0000", 2024-09-14T22:36:42.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0055282739999999997 2024-09-14T22:36:42.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.270774+0000", 2024-09-14T22:36:42.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011195499999999999 2024-09-14T22:36:42.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.270908+0000", 2024-09-14T22:36:42.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013438499999999999 2024-09-14T22:36:42.211 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.211 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.212 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.212 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:63509 3.10 3:0c425a9a:::rbd_data.113cce440cee.000000000000001d:a [read 786432~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:36:42.212 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:41.992335+0000", 2024-09-14T22:36:42.212 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.16823835600000001, 2024-09-14T22:36:42.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0057088549999999997, 2024-09-14T22:36:42.212 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.212 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.212 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:42.212 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.212 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.212 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.212 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 63509 2024-09-14T22:36:42.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.213 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.992335+0000", 2024-09-14T22:36:42.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.992335+0000", 2024-09-14T22:36:42.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.992337+0000", 2024-09-14T22:36:42.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9099999999999999e-06 2024-09-14T22:36:42.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.992344+0000", 2024-09-14T22:36:42.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.737e-06 2024-09-14T22:36:42.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.992384+0000", 2024-09-14T22:36:42.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0546000000000001e-05 2024-09-14T22:36:42.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.992402+0000", 2024-09-14T22:36:42.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7730000000000001e-05 2024-09-14T22:36:42.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.997682+0000", 2024-09-14T22:36:42.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.005280547 2024-09-14T22:36:42.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.997778+0000", 2024-09-14T22:36:42.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6006000000000002e-05 2024-09-14T22:36:42.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.998044+0000", 2024-09-14T22:36:42.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026537899999999999 2024-09-14T22:36:42.215 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.215 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.215 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.216 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:63519 3.10 3:0c425a9a:::rbd_data.113cce440cee.000000000000001d:a [read 835584~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:36:42.216 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:41.992433+0000", 2024-09-14T22:36:42.216 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.16814011000000001, 2024-09-14T22:36:42.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0062459689999999997, 2024-09-14T22:36:42.216 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.216 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.216 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:42.216 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.216 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.216 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.216 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 63519 2024-09-14T22:36:42.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.216 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.992433+0000", 2024-09-14T22:36:42.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.992433+0000", 2024-09-14T22:36:42.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.992434+0000", 2024-09-14T22:36:42.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2130000000000001e-06 2024-09-14T22:36:42.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.992442+0000", 2024-09-14T22:36:42.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3069999999999997e-06 2024-09-14T22:36:42.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.992464+0000", 2024-09-14T22:36:42.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2500000000000001e-05 2024-09-14T22:36:42.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.992482+0000", 2024-09-14T22:36:42.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7866000000000001e-05 2024-09-14T22:36:42.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.998081+0000", 2024-09-14T22:36:42.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0055993220000000003 2024-09-14T22:36:42.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.998127+0000", 2024-09-14T22:36:42.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5958999999999998e-05 2024-09-14T22:36:42.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.998679+0000", 2024-09-14T22:36:42.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000551802 2024-09-14T22:36:42.219 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.220 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.220 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.220 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:63521 3.10 3:0c425a9a:::rbd_data.113cce440cee.000000000000001d:a [read 1048576~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:36:42.220 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:41.992505+0000", 2024-09-14T22:36:42.220 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.16806803400000001, 2024-09-14T22:36:42.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0065321110000000002, 2024-09-14T22:36:42.220 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.220 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.220 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:42.220 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.220 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.220 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.221 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 63521 2024-09-14T22:36:42.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.221 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.992505+0000", 2024-09-14T22:36:42.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.992505+0000", 2024-09-14T22:36:42.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.992506+0000", 2024-09-14T22:36:42.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1570000000000001e-06 2024-09-14T22:36:42.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.992513+0000", 2024-09-14T22:36:42.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1600000000000003e-06 2024-09-14T22:36:42.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.992530+0000", 2024-09-14T22:36:42.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7107000000000001e-05 2024-09-14T22:36:42.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.992544+0000", 2024-09-14T22:36:42.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4222999999999999e-05 2024-09-14T22:36:42.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.998703+0000", 2024-09-14T22:36:42.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0061592820000000003 2024-09-14T22:36:42.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.998750+0000", 2024-09-14T22:36:42.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6498000000000001e-05 2024-09-14T22:36:42.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.999037+0000", 2024-09-14T22:36:42.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00028768400000000002 2024-09-14T22:36:42.224 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.224 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.224 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.224 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:63535 3.10 3:0c425a9a:::rbd_data.113cce440cee.000000000000001d:a [read 442368~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:36:42.224 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:41.993155+0000", 2024-09-14T22:36:42.224 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.16741874900000001, 2024-09-14T22:36:42.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0063964570000000004, 2024-09-14T22:36:42.224 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.224 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.224 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:42.224 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.225 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.225 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.225 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 63535 2024-09-14T22:36:42.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.225 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993155+0000", 2024-09-14T22:36:42.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993155+0000", 2024-09-14T22:36:42.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993157+0000", 2024-09-14T22:36:42.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.903e-06 2024-09-14T22:36:42.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993170+0000", 2024-09-14T22:36:42.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2407e-05 2024-09-14T22:36:42.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993191+0000", 2024-09-14T22:36:42.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1029000000000001e-05 2024-09-14T22:36:42.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993203+0000", 2024-09-14T22:36:42.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2254e-05 2024-09-14T22:36:42.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.999106+0000", 2024-09-14T22:36:42.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0059026579999999999 2024-09-14T22:36:42.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.999297+0000", 2024-09-14T22:36:42.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000191232 2024-09-14T22:36:42.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.999551+0000", 2024-09-14T22:36:42.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00025397400000000002 2024-09-14T22:36:42.228 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.228 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.228 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.228 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:63536 3.10 3:0c425a9a:::rbd_data.113cce440cee.000000000000001d:a [read 466944~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:36:42.228 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:41.993234+0000", 2024-09-14T22:36:42.228 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.16733945, 2024-09-14T22:36:42.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0065203919999999999, 2024-09-14T22:36:42.229 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.229 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.229 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:42.229 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.229 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.229 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.229 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 63536 2024-09-14T22:36:42.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.229 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993234+0000", 2024-09-14T22:36:42.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993234+0000", 2024-09-14T22:36:42.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993235+0000", 2024-09-14T22:36:42.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3570000000000001e-06 2024-09-14T22:36:42.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993242+0000", 2024-09-14T22:36:42.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5230000000000001e-06 2024-09-14T22:36:42.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993262+0000", 2024-09-14T22:36:42.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0027e-05 2024-09-14T22:36:42.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993277+0000", 2024-09-14T22:36:42.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4863e-05 2024-09-14T22:36:42.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.999585+0000", 2024-09-14T22:36:42.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0063078880000000002 2024-09-14T22:36:42.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.999682+0000", 2024-09-14T22:36:42.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7937999999999996e-05 2024-09-14T22:36:42.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.999754+0000", 2024-09-14T22:36:42.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1796000000000002e-05 2024-09-14T22:36:42.235 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.236 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.236 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.236 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:63537 3.10 3:0c425a9a:::rbd_data.113cce440cee.000000000000001d:a [read 655360~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:36:42.236 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:41.993297+0000", 2024-09-14T22:36:42.236 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.16727610400000001, 2024-09-14T22:36:42.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0067916579999999999, 2024-09-14T22:36:42.236 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.236 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.236 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:42.236 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.236 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.237 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.237 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 63537 2024-09-14T22:36:42.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.237 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993297+0000", 2024-09-14T22:36:42.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993297+0000", 2024-09-14T22:36:42.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993298+0000", 2024-09-14T22:36:42.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7000000000000003e-07 2024-09-14T22:36:42.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993302+0000", 2024-09-14T22:36:42.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2039999999999999e-06 2024-09-14T22:36:42.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993316+0000", 2024-09-14T22:36:42.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3609e-05 2024-09-14T22:36:42.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993323+0000", 2024-09-14T22:36:42.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4599999999999997e-06 2024-09-14T22:36:42.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.999810+0000", 2024-09-14T22:36:42.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0064868039999999997 2024-09-14T22:36:42.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.999940+0000", 2024-09-14T22:36:42.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000129849 2024-09-14T22:36:42.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.000089+0000", 2024-09-14T22:36:42.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014886200000000001 2024-09-14T22:36:42.240 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.240 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.240 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.240 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:63538 3.10 3:0c425a9a:::rbd_data.113cce440cee.000000000000001d:a [read 679936~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:36:42.240 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:41.993334+0000", 2024-09-14T22:36:42.240 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.16723938699999999, 2024-09-14T22:36:42.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0068890929999999998, 2024-09-14T22:36:42.241 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.241 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.241 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:42.241 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.241 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.241 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.241 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 63538 2024-09-14T22:36:42.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.241 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993334+0000", 2024-09-14T22:36:42.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993334+0000", 2024-09-14T22:36:42.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993335+0000", 2024-09-14T22:36:42.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5300000000000004e-07 2024-09-14T22:36:42.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993338+0000", 2024-09-14T22:36:42.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4630000000000001e-06 2024-09-14T22:36:42.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993350+0000", 2024-09-14T22:36:42.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2037e-05 2024-09-14T22:36:42.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993359+0000", 2024-09-14T22:36:42.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4400000000000005e-06 2024-09-14T22:36:42.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.000106+0000", 2024-09-14T22:36:42.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0067476530000000002 2024-09-14T22:36:42.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.244 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.000158+0000", 2024-09-14T22:36:42.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2280000000000001e-05 2024-09-14T22:36:42.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.244 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.000223+0000", 2024-09-14T22:36:42.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4566999999999994e-05 2024-09-14T22:36:42.244 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.244 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.244 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.245 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:63553 3.10 3:0f1b8cf8:::rbd_data.113c8642facc.000000000000000a:9 [read 4849664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:36:42.245 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:41.993368+0000", 2024-09-14T22:36:42.245 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.167205031, 2024-09-14T22:36:42.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0071530079999999998, 2024-09-14T22:36:42.245 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.245 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.245 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:42.245 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.245 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.245 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.245 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 63553 2024-09-14T22:36:42.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.246 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993368+0000", 2024-09-14T22:36:42.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993368+0000", 2024-09-14T22:36:42.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993369+0000", 2024-09-14T22:36:42.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T22:36:42.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993373+0000", 2024-09-14T22:36:42.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5640000000000001e-06 2024-09-14T22:36:42.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993392+0000", 2024-09-14T22:36:42.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9079000000000001e-05 2024-09-14T22:36:42.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993408+0000", 2024-09-14T22:36:42.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5913000000000002e-05 2024-09-14T22:36:42.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.248 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.248 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.000271+0000", 2024-09-14T22:36:42.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0068634710000000003 2024-09-14T22:36:42.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.248 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.248 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.000376+0000", 2024-09-14T22:36:42.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000104999 2024-09-14T22:36:42.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.248 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.249 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.000521+0000", 2024-09-14T22:36:42.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000145322 2024-09-14T22:36:42.249 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.249 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.249 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.249 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:63555 3.10 3:0f1b8cf8:::rbd_data.113c8642facc.000000000000000a:9 [read 4890624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:36:42.249 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:41.993428+0000", 2024-09-14T22:36:42.249 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.16714546499999999, 2024-09-14T22:36:42.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0072193999999999999, 2024-09-14T22:36:42.249 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.249 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.250 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:42.250 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.250 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.250 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.250 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 63555 2024-09-14T22:36:42.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.250 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993428+0000", 2024-09-14T22:36:42.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993428+0000", 2024-09-14T22:36:42.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993429+0000", 2024-09-14T22:36:42.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.13e-06 2024-09-14T22:36:42.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993435+0000", 2024-09-14T22:36:42.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7529999999999998e-06 2024-09-14T22:36:42.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993455+0000", 2024-09-14T22:36:42.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0604000000000001e-05 2024-09-14T22:36:42.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993472+0000", 2024-09-14T22:36:42.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.66e-05 2024-09-14T22:36:42.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.000545+0000", 2024-09-14T22:36:42.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0070728969999999999 2024-09-14T22:36:42.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.000589+0000", 2024-09-14T22:36:42.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3758999999999999e-05 2024-09-14T22:36:42.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.000647+0000", 2024-09-14T22:36:42.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8656999999999998e-05 2024-09-14T22:36:42.253 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.253 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.253 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.253 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:63557 3.10 3:0f1b8cf8:::rbd_data.113c8642facc.000000000000000a:9 [read 4931584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:36:42.254 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:41.993494+0000", 2024-09-14T22:36:42.254 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.16707924499999999, 2024-09-14T22:36:42.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.007277576, 2024-09-14T22:36:42.254 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.254 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.254 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:42.254 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.254 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.254 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.254 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 63557 2024-09-14T22:36:42.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.254 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.254 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.254 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993494+0000", 2024-09-14T22:36:42.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993494+0000", 2024-09-14T22:36:42.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993495+0000", 2024-09-14T22:36:42.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.083e-06 2024-09-14T22:36:42.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993501+0000", 2024-09-14T22:36:42.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1240000000000002e-06 2024-09-14T22:36:42.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993522+0000", 2024-09-14T22:36:42.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1046000000000001e-05 2024-09-14T22:36:42.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993533+0000", 2024-09-14T22:36:42.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0305999999999999e-05 2024-09-14T22:36:42.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.000679+0000", 2024-09-14T22:36:42.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0071462280000000001 2024-09-14T22:36:42.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.000719+0000", 2024-09-14T22:36:42.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0201999999999999e-05 2024-09-14T22:36:42.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.000772+0000", 2024-09-14T22:36:42.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2587000000000003e-05 2024-09-14T22:36:42.257 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.257 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.257 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.258 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:63559 3.10 3:0f1b8cf8:::rbd_data.113c8642facc.000000000000000a:9 [read 4972544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:36:42.258 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:41.993547+0000", 2024-09-14T22:36:42.258 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.16702679200000001, 2024-09-14T22:36:42.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0073356990000000002, 2024-09-14T22:36:42.258 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.258 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.258 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:42.258 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.258 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.258 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.258 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 63559 2024-09-14T22:36:42.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.258 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993547+0000", 2024-09-14T22:36:42.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993547+0000", 2024-09-14T22:36:42.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993547+0000", 2024-09-14T22:36:42.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4899999999999995e-07 2024-09-14T22:36:42.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993550+0000", 2024-09-14T22:36:42.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2200000000000001e-06 2024-09-14T22:36:42.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993562+0000", 2024-09-14T22:36:42.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1164e-05 2024-09-14T22:36:42.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993568+0000", 2024-09-14T22:36:42.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7129999999999999e-06 2024-09-14T22:36:42.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.261 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.000797+0000", 2024-09-14T22:36:42.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0072290130000000003 2024-09-14T22:36:42.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.261 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.000834+0000", 2024-09-14T22:36:42.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6365999999999997e-05 2024-09-14T22:36:42.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.261 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.000882+0000", 2024-09-14T22:36:42.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8573999999999999e-05 2024-09-14T22:36:42.262 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.262 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.262 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.262 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.262 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.262 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:42.263 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-14T22:36:42.263 INFO:teuthology.orchestra.run.smithi001.stdout:1141 read 0x267a3d7 thru 0x26830c3 (0x8ced bytes) 2024-09-14T22:36:42.263 INFO:teuthology.orchestra.run.smithi001.stdout:1142 write 0x3de1a52 thru 0x3de8657 (0x6c06 bytes) 2024-09-14T22:36:42.264 INFO:teuthology.orchestra.run.smithi001.stdout:1143 write 0xc7c6cdb thru 0xc7ca42e (0x3754 bytes) 2024-09-14T22:36:42.264 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:42.258+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:42.264 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:36:42.267 INFO:teuthology.orchestra.run.smithi001.stdout:1144 read 0x87185f8 thru 0x8725a01 (0xd40a bytes) 2024-09-14T22:36:42.267 INFO:teuthology.orchestra.run.smithi001.stdout:1145 read 0x1a96a5 thru 0x1ad41a (0x3d76 bytes) 2024-09-14T22:36:42.269 INFO:teuthology.orchestra.run.smithi001.stdout:1146 writesame 0x98fc6de thru 0x9906cef data_size 0x1d(0xa612 bytes) 2024-09-14T22:36:42.279 INFO:teuthology.orchestra.run.smithi001.stdout:1148 write 0x2d975f6 thru 0x2d97f6e (0x979 bytes) 2024-09-14T22:36:42.293 INFO:teuthology.orchestra.run.smithi001.stdout:1151 write 0x9d7dfee thru 0x9d84c2d (0x6c40 bytes) 2024-09-14T22:36:42.299 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:42.300 INFO:teuthology.orchestra.run.smithi001.stdout:1153 read 0x9e6b44b thru 0x9e728bd (0x7473 bytes) 2024-09-14T22:36:42.300 INFO:teuthology.orchestra.run.smithi001.stdout:1154 punch from 0xaa3561c to 0xaa41174, (0xbb58 bytes) 2024-09-14T22:36:42.312 INFO:teuthology.orchestra.run.smithi001.stdout:1155 write 0xe2ba515 thru 0xe2c02b7 (0x5da3 bytes) 2024-09-14T22:36:42.318 INFO:teuthology.orchestra.run.smithi001.stdout:1156 write 0x6a350d6 thru 0x6a38bda (0x3b05 bytes) 2024-09-14T22:36:42.324 INFO:teuthology.orchestra.run.smithi001.stdout:1157 punch from 0x48de031 to 0x48e97d5, (0xb7a4 bytes) 2024-09-14T22:36:42.324 INFO:teuthology.orchestra.run.smithi001.stdout:1158 write 0xc6b340b thru 0xc6b8d51 (0x5947 bytes) 2024-09-14T22:36:42.331 INFO:teuthology.orchestra.run.smithi001.stdout:1160 punch from 0xcdd35af to 0xcdd45ac, (0xffd bytes) 2024-09-14T22:36:42.332 INFO:teuthology.orchestra.run.smithi001.stdout:1161 read 0x403e42c thru 0x403e523 (0xf8 bytes) 2024-09-14T22:36:42.332 INFO:teuthology.orchestra.run.smithi001.stdout:1162 write 0xa8ba520 thru 0xa8c118e (0x6c6f bytes) 2024-09-14T22:36:42.337 INFO:teuthology.orchestra.run.smithi001.stdout:1163 read 0xbf3f954 thru 0xbf42c88 (0x3335 bytes) 2024-09-14T22:36:42.338 INFO:teuthology.orchestra.run.smithi001.stdout:1164 writesame 0x61ec6ee thru 0x61f7c45 data_size 0xe(0xb558 bytes) 2024-09-14T22:36:42.345 INFO:teuthology.orchestra.run.smithi001.stdout:1165 read 0x11daced thru 0x11e5871 (0xab85 bytes) 2024-09-14T22:36:42.359 INFO:teuthology.orchestra.run.smithi001.stdout:1166 writesame 0xe5a4cb5 thru 0xe5ab3f1 data_size 0x13(0x673d bytes) 2024-09-14T22:36:42.359 INFO:teuthology.orchestra.run.smithi001.stdout:1167 write 0x6fd13f7 thru 0x6fdce06 (0xba10 bytes) 2024-09-14T22:36:42.364 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:42.358+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:42.365 INFO:teuthology.orchestra.run.smithi001.stdout:1168 read 0x310bcf9 thru 0x3117fd2 (0xc2da bytes) 2024-09-14T22:36:42.367 INFO:teuthology.orchestra.run.smithi001.stdout:1170 read 0x5033eb5 thru 0x504286f (0xe9bb bytes) 2024-09-14T22:36:42.367 INFO:teuthology.orchestra.run.smithi001.stdout:1171 write 0x5030ed2 thru 0x503784d (0x697c bytes) 2024-09-14T22:36:42.372 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:42.373 INFO:teuthology.orchestra.run.smithi001.stdout:1173 read 0x6ba6813 thru 0x6ba87fd (0x1feb bytes) 2024-09-14T22:36:42.373 INFO:teuthology.orchestra.run.smithi001.stdout:1174 read 0x36acaed thru 0x36b7037 (0xa54b bytes) 2024-09-14T22:36:42.376 INFO:teuthology.orchestra.run.smithi001.stdout:1175 trunc from 0xe5ab3f2 to 0xd7ba40b 2024-09-14T22:36:42.394 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:42.394 INFO:teuthology.orchestra.run.smithi001.stdout:1178 read 0x5707862 thru 0x570eb61 (0x7300 bytes) 2024-09-14T22:36:42.394 INFO:teuthology.orchestra.run.smithi001.stdout:1179 write 0x87d415d thru 0x87e2df0 (0xec94 bytes) 2024-09-14T22:36:42.401 INFO:teuthology.orchestra.run.smithi001.stdout:1181 write 0x4d07b3f thru 0x4d17329 (0xf7eb bytes) 2024-09-14T22:36:42.408 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:42.408 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:42.408 INFO:teuthology.orchestra.run.smithi001.stdout:1185 write 0xa0e5bb3 thru 0xa0ea884 (0x4cd2 bytes) 2024-09-14T22:36:42.411 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:42.411 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:42.411 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:36:42.411 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:42.413 INFO:teuthology.orchestra.run.smithi001.stdout:1186 writesame 0xce94710 thru 0xce983a2 data_size 0x3(0x3c93 bytes) 2024-09-14T22:36:42.424 INFO:teuthology.orchestra.run.smithi001.stdout:1187 read 0xa082857 thru 0xa09196c (0xf116 bytes) 2024-09-14T22:36:42.426 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:36:42.426 INFO:teuthology.orchestra.run.smithi001.stdout:1188 write 0x9928e2a thru 0x9933986 (0xab5d bytes) 2024-09-14T22:36:42.429 INFO:teuthology.orchestra.run.smithi001.stdout:1189 read 0x5f9d44a thru 0x5fa8e33 (0xb9ea bytes) 2024-09-14T22:36:42.429 INFO:teuthology.orchestra.run.smithi001.stdout:1190 read 0x8dd059b thru 0x8dda22d (0x9c93 bytes) 2024-09-14T22:36:42.430 INFO:teuthology.orchestra.run.smithi001.stdout:1192 punch from 0xbb86b3f to 0xbb966b5, (0xfb76 bytes) 2024-09-14T22:36:42.430 INFO:teuthology.orchestra.run.smithi001.stdout:1193 punch from 0xa58afe3 to 0xa594083, (0x90a0 bytes) 2024-09-14T22:36:42.430 INFO:teuthology.orchestra.run.smithi001.stdout:1194 read 0x5da8ed thru 0x5e89b2 (0xe0c6 bytes) 2024-09-14T22:36:42.432 INFO:teuthology.orchestra.run.smithi001.stdout:1195 read 0x78c45e7 thru 0x78c706a (0x2a84 bytes) 2024-09-14T22:36:42.441 INFO:teuthology.orchestra.run.smithi001.stdout:1197 writesame 0xdd496c4 thru 0xdd4d881 data_size 0x37(0x41be bytes) 2024-09-14T22:36:42.441 INFO:teuthology.orchestra.run.smithi001.stdout:1200 write 0xa8e7ce2 thru 0xa8e9d41 (0x2060 bytes) 2024-09-14T22:36:42.443 INFO:teuthology.orchestra.run.smithi001.stdout:1201 writesame 0xb97e471 thru 0xb981773 data_size 0x3(0x3303 bytes) 2024-09-14T22:36:42.451 INFO:teuthology.orchestra.run.smithi001.stdout:1202 read 0x32eabb9 thru 0x32fa67d (0xfac5 bytes) 2024-09-14T22:36:42.453 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:42.453 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:42.453 INFO:teuthology.orchestra.run.smithi001.stdout:1206 read 0x56de9ff thru 0x56e3398 (0x499a bytes) 2024-09-14T22:36:42.453 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:42.453 INFO:teuthology.orchestra.run.smithi001.stdout:1208 read 0xc487f3 thru 0xc4a6ae (0x1ebc bytes) 2024-09-14T22:36:42.455 INFO:teuthology.orchestra.run.smithi001.stdout:1209 read 0xb3d7f69 thru 0xb3dc0f3 (0x418b bytes) 2024-09-14T22:36:42.465 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:42.458+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:42.478 INFO:teuthology.orchestra.run.smithi001.stdout:1211 trunc from 0xdd4d882 to 0x5510478 2024-09-14T22:36:42.506 INFO:teuthology.orchestra.run.smithi001.stdout:1212 read 0xf7c809 thru 0xf7d5f7 (0xdef bytes) 2024-09-14T22:36:42.507 INFO:teuthology.orchestra.run.smithi001.stdout:1213 read 0x431021e thru 0x431b635 (0xb418 bytes) 2024-09-14T22:36:42.507 INFO:teuthology.orchestra.run.smithi001.stdout:1214 write 0x3106374 thru 0x310b31e (0x4fab bytes) 2024-09-14T22:36:42.535 INFO:teuthology.orchestra.run.smithi001.stdout:1215 trunc from 0x5510478 to 0xa9bd0fa 2024-09-14T22:36:42.548 INFO:teuthology.orchestra.run.smithi001.stdout:1216 trunc from 0xa9bd0fa to 0xc20f5e6 2024-09-14T22:36:42.552 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:42.553 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:42.553 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:36:42.553 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:36:42.553 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:42.560 INFO:teuthology.orchestra.run.smithi001.stdout:1217 write 0x605851f thru 0x60630ae (0xab90 bytes) 2024-09-14T22:36:42.566 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:42.559+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:42.566 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:42.566 INFO:teuthology.orchestra.run.smithi001.stdout:1220 punch from 0x665017b to 0x665ca37, (0xc8bc bytes) 2024-09-14T22:36:42.568 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:36:42.577 INFO:teuthology.orchestra.run.smithi001.stdout:1222 write 0xcd96dad thru 0xcd9da1e (0x6c72 bytes) 2024-09-14T22:36:42.583 INFO:teuthology.orchestra.run.smithi001.stdout:1223 write 0x17426e4 thru 0x174c096 (0x99b3 bytes) 2024-09-14T22:36:42.608 INFO:teuthology.orchestra.run.smithi001.stdout:1224 write 0x45e59c3 thru 0x45e66c8 (0xd06 bytes) 2024-09-14T22:36:42.614 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:42.628 INFO:teuthology.orchestra.run.smithi001.stdout:1226 writesame 0xea641c8 thru 0xea712b2 data_size 0x1(0xd0eb bytes) 2024-09-14T22:36:42.628 INFO:teuthology.orchestra.run.smithi001.stdout:1227 writesame 0x72c98a9 thru 0x72d0529 data_size 0x2f(0x6c81 bytes) 2024-09-14T22:36:42.633 INFO:teuthology.orchestra.run.smithi001.stdout:1230 writesame 0x68ddf56 thru 0x68e780e data_size 0x1(0x98b9 bytes) 2024-09-14T22:36:42.660 INFO:teuthology.orchestra.run.smithi001.stdout:1232 writesame 0x5fb3da3 thru 0x5fbcebc data_size 0x29(0x911a bytes) 2024-09-14T22:36:42.667 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:42.663+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:42.667 INFO:teuthology.orchestra.run.smithi001.stdout:1233 read 0x7c4a2 thru 0x838c5 (0x7424 bytes) 2024-09-14T22:36:42.669 INFO:teuthology.orchestra.run.smithi001.stdout:1235 read 0x1d63c9a thru 0x1d6d024 (0x938b bytes) 2024-09-14T22:36:42.672 INFO:teuthology.orchestra.run.smithi001.stdout:1236 writesame 0x66f9173 thru 0x6704ca9 data_size 0x1(0xbb37 bytes) 2024-09-14T22:36:42.711 INFO:teuthology.orchestra.run.smithi001.stdout:1239 write 0xec68dbc thru 0xec6faa3 (0x6ce8 bytes) 2024-09-14T22:36:42.719 INFO:teuthology.orchestra.run.smithi001.stdout:1240 read 0xc84cbb0 thru 0xc84f0dc (0x252d bytes) 2024-09-14T22:36:42.737 INFO:teuthology.orchestra.run.smithi001.stdout:1243 trunc from 0xec6faa4 to 0x83338b5 2024-09-14T22:36:42.742 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:36:42.746 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:42.746 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:36:42.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:36:42.746 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:36:42.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.746 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:36:42.746 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:36:42.746 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.735915935999998, 2024-09-14T22:36:42.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.398029152, 2024-09-14T22:36:42.746 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.747 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.747 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:42.747 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.747 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:36:42.747 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:36:42.747 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 1 2024-09-14T22:36:42.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.747 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:36:42.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:36:42.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006183+0000", 2024-09-14T22:36:42.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9059999999999998e-06 2024-09-14T22:36:42.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006198+0000", 2024-09-14T22:36:42.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.523e-05 2024-09-14T22:36:42.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006219+0000", 2024-09-14T22:36:42.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1342999999999999e-05 2024-09-14T22:36:42.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006240+0000", 2024-09-14T22:36:42.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1186999999999999e-05 2024-09-14T22:36:42.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006363+0000", 2024-09-14T22:36:42.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012258200000000001 2024-09-14T22:36:42.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:36:42.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006392+0000", 2024-09-14T22:36:42.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9607e-05 2024-09-14T22:36:42.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401084+0000", 2024-09-14T22:36:42.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.39469142699999998 2024-09-14T22:36:42.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401394+0000", 2024-09-14T22:36:42.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030987799999999998 2024-09-14T22:36:42.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:42.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401860+0000", 2024-09-14T22:36:42.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00046654299999999998 2024-09-14T22:36:42.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:42.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.403386+0000", 2024-09-14T22:36:42.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001525322 2024-09-14T22:36:42.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:42.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404042+0000", 2024-09-14T22:36:42.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065676099999999996 2024-09-14T22:36:42.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:42.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404136+0000", 2024-09-14T22:36:42.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4032999999999999e-05 2024-09-14T22:36:42.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404209+0000", 2024-09-14T22:36:42.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2333e-05 2024-09-14T22:36:42.752 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.752 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.752 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.753 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2079 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11401216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:42.753 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450429+0000", 2024-09-14T22:36:42.753 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.291666148000001, 2024-09-14T22:36:42.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055279107000000001, 2024-09-14T22:36:42.753 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.753 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.753 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:42.753 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.753 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.753 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.753 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2079 2024-09-14T22:36:42.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.753 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450429+0000", 2024-09-14T22:36:42.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450429+0000", 2024-09-14T22:36:42.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450431+0000", 2024-09-14T22:36:42.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1039999999999999e-06 2024-09-14T22:36:42.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450435+0000", 2024-09-14T22:36:42.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9300000000000002e-06 2024-09-14T22:36:42.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450455+0000", 2024-09-14T22:36:42.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9242999999999999e-05 2024-09-14T22:36:42.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450470+0000", 2024-09-14T22:36:42.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5523e-05 2024-09-14T22:36:42.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.505413+0000", 2024-09-14T22:36:42.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054943193000000001 2024-09-14T22:36:42.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.505538+0000", 2024-09-14T22:36:42.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012467199999999999 2024-09-14T22:36:42.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.505709+0000", 2024-09-14T22:36:42.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017044200000000001 2024-09-14T22:36:42.756 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.756 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.757 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.757 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2080 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11409408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:42.757 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450489+0000", 2024-09-14T22:36:42.757 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.291606400999999, 2024-09-14T22:36:42.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055572178, 2024-09-14T22:36:42.757 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.757 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.757 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:42.757 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.757 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.757 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.757 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2080 2024-09-14T22:36:42.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.757 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.758 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450489+0000", 2024-09-14T22:36:42.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.758 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450489+0000", 2024-09-14T22:36:42.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.758 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450490+0000", 2024-09-14T22:36:42.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.023e-06 2024-09-14T22:36:42.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450495+0000", 2024-09-14T22:36:42.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1399999999999999e-06 2024-09-14T22:36:42.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450512+0000", 2024-09-14T22:36:42.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6892999999999999e-05 2024-09-14T22:36:42.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450525+0000", 2024-09-14T22:36:42.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2877000000000001e-05 2024-09-14T22:36:42.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.505787+0000", 2024-09-14T22:36:42.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055261432999999999 2024-09-14T22:36:42.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.505899+0000", 2024-09-14T22:36:42.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112456 2024-09-14T22:36:42.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506061+0000", 2024-09-14T22:36:42.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000162356 2024-09-14T22:36:42.760 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.761 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.761 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.761 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2081 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11417600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:42.761 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450544+0000", 2024-09-14T22:36:42.761 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.291551082000002, 2024-09-14T22:36:42.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.05598185, 2024-09-14T22:36:42.761 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.761 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.761 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:42.761 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.761 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.761 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.761 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2081 2024-09-14T22:36:42.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.762 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450544+0000", 2024-09-14T22:36:42.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450544+0000", 2024-09-14T22:36:42.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450546+0000", 2024-09-14T22:36:42.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5370000000000001e-06 2024-09-14T22:36:42.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450553+0000", 2024-09-14T22:36:42.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8630000000000002e-06 2024-09-14T22:36:42.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450571+0000", 2024-09-14T22:36:42.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7657000000000001e-05 2024-09-14T22:36:42.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450585+0000", 2024-09-14T22:36:42.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4056000000000001e-05 2024-09-14T22:36:42.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.764 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.764 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506185+0000", 2024-09-14T22:36:42.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055599920999999997 2024-09-14T22:36:42.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.764 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.764 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506334+0000", 2024-09-14T22:36:42.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014919900000000001 2024-09-14T22:36:42.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.764 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506526+0000", 2024-09-14T22:36:42.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019261699999999999 2024-09-14T22:36:42.765 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.765 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.765 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.765 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2082 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11425792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:42.765 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450602+0000", 2024-09-14T22:36:42.765 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.291493355, 2024-09-14T22:36:42.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056238654999999999, 2024-09-14T22:36:42.765 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.765 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.765 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:42.765 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.766 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.766 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.766 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2082 2024-09-14T22:36:42.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.766 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450602+0000", 2024-09-14T22:36:42.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450602+0000", 2024-09-14T22:36:42.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450603+0000", 2024-09-14T22:36:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1060000000000001e-06 2024-09-14T22:36:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450608+0000", 2024-09-14T22:36:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1499999999999998e-06 2024-09-14T22:36:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450626+0000", 2024-09-14T22:36:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7484000000000001e-05 2024-09-14T22:36:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450639+0000", 2024-09-14T22:36:42.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3485999999999999e-05 2024-09-14T22:36:42.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506599+0000", 2024-09-14T22:36:42.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055959395000000002 2024-09-14T22:36:42.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506706+0000", 2024-09-14T22:36:42.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010703499999999999 2024-09-14T22:36:42.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506841+0000", 2024-09-14T22:36:42.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013499900000000001 2024-09-14T22:36:42.769 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.769 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.769 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.769 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2083 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11433984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:42.769 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450658+0000", 2024-09-14T22:36:42.769 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.291437929000001, 2024-09-14T22:36:42.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056459267, 2024-09-14T22:36:42.769 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.769 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.769 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:42.770 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.770 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.770 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.770 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2083 2024-09-14T22:36:42.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.770 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450658+0000", 2024-09-14T22:36:42.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450658+0000", 2024-09-14T22:36:42.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450659+0000", 2024-09-14T22:36:42.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T22:36:42.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450664+0000", 2024-09-14T22:36:42.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4700000000000001e-06 2024-09-14T22:36:42.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450684+0000", 2024-09-14T22:36:42.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.916e-05 2024-09-14T22:36:42.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450698+0000", 2024-09-14T22:36:42.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4100000000000001e-05 2024-09-14T22:36:42.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506881+0000", 2024-09-14T22:36:42.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056183250999999997 2024-09-14T22:36:42.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506982+0000", 2024-09-14T22:36:42.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010090900000000001 2024-09-14T22:36:42.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507117+0000", 2024-09-14T22:36:42.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013510400000000001 2024-09-14T22:36:42.773 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.773 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.773 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.773 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2084 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11442176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:42.773 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450716+0000", 2024-09-14T22:36:42.773 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.291380069999999, 2024-09-14T22:36:42.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056802328999999999, 2024-09-14T22:36:42.773 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.773 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.774 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:42.774 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.774 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.774 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.774 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2084 2024-09-14T22:36:42.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.774 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450716+0000", 2024-09-14T22:36:42.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450716+0000", 2024-09-14T22:36:42.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450717+0000", 2024-09-14T22:36:42.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.08e-06 2024-09-14T22:36:42.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450722+0000", 2024-09-14T22:36:42.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5099999999999998e-06 2024-09-14T22:36:42.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450740+0000", 2024-09-14T22:36:42.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7836999999999999e-05 2024-09-14T22:36:42.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.776 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450757+0000", 2024-09-14T22:36:42.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7513e-05 2024-09-14T22:36:42.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.776 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507224+0000", 2024-09-14T22:36:42.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056466954 2024-09-14T22:36:42.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.776 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507355+0000", 2024-09-14T22:36:42.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000130207 2024-09-14T22:36:42.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507518+0000", 2024-09-14T22:36:42.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016322799999999999 2024-09-14T22:36:42.777 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.777 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.777 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.777 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2085 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11450368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:42.777 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450774+0000", 2024-09-14T22:36:42.777 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.291321267000001, 2024-09-14T22:36:42.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057099952000000002, 2024-09-14T22:36:42.778 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.778 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.778 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:42.778 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.778 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.778 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.778 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2085 2024-09-14T22:36:42.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.778 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450774+0000", 2024-09-14T22:36:42.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450774+0000", 2024-09-14T22:36:42.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450775+0000", 2024-09-14T22:36:42.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1030000000000001e-06 2024-09-14T22:36:42.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450781+0000", 2024-09-14T22:36:42.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.22e-06 2024-09-14T22:36:42.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450798+0000", 2024-09-14T22:36:42.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7350000000000002e-05 2024-09-14T22:36:42.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450812+0000", 2024-09-14T22:36:42.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.358e-05 2024-09-14T22:36:42.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507576+0000", 2024-09-14T22:36:42.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056764468999999998 2024-09-14T22:36:42.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507714+0000", 2024-09-14T22:36:42.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013826800000000001 2024-09-14T22:36:42.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507874+0000", 2024-09-14T22:36:42.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015996200000000001 2024-09-14T22:36:42.781 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.781 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.781 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.781 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2086 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11458560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:42.781 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450830+0000", 2024-09-14T22:36:42.782 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.291265976999998, 2024-09-14T22:36:42.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057480495999999999, 2024-09-14T22:36:42.782 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.782 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.782 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:42.782 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.782 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.782 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.782 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2086 2024-09-14T22:36:42.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.782 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450830+0000", 2024-09-14T22:36:42.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450830+0000", 2024-09-14T22:36:42.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450831+0000", 2024-09-14T22:36:42.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0860000000000001e-06 2024-09-14T22:36:42.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450836+0000", 2024-09-14T22:36:42.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4999999999999999e-06 2024-09-14T22:36:42.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450853+0000", 2024-09-14T22:36:42.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7184e-05 2024-09-14T22:36:42.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450869+0000", 2024-09-14T22:36:42.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5965999999999999e-05 2024-09-14T22:36:42.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507972+0000", 2024-09-14T22:36:42.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057103095999999999 2024-09-14T22:36:42.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508119+0000", 2024-09-14T22:36:42.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014651200000000001 2024-09-14T22:36:42.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508310+0000", 2024-09-14T22:36:42.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019115199999999999 2024-09-14T22:36:42.785 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.785 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.785 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.786 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2087 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11466752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:42.786 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450888+0000", 2024-09-14T22:36:42.786 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.291207930999999, 2024-09-14T22:36:42.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057792806000000002, 2024-09-14T22:36:42.786 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.786 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.786 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:42.786 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.786 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.786 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.786 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2087 2024-09-14T22:36:42.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.787 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450888+0000", 2024-09-14T22:36:42.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450888+0000", 2024-09-14T22:36:42.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450889+0000", 2024-09-14T22:36:42.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4270000000000001e-06 2024-09-14T22:36:42.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450895+0000", 2024-09-14T22:36:42.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4700000000000001e-06 2024-09-14T22:36:42.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450913+0000", 2024-09-14T22:36:42.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7958999999999999e-05 2024-09-14T22:36:42.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450926+0000", 2024-09-14T22:36:42.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3842999999999999e-05 2024-09-14T22:36:42.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508394+0000", 2024-09-14T22:36:42.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057467795000000002 2024-09-14T22:36:42.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508516+0000", 2024-09-14T22:36:42.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000121822 2024-09-14T22:36:42.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508680+0000", 2024-09-14T22:36:42.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016448999999999999 2024-09-14T22:36:42.790 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.790 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.790 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.790 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2088 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11474944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:42.790 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450943+0000", 2024-09-14T22:36:42.790 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.291152318000002, 2024-09-14T22:36:42.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058110368000000003, 2024-09-14T22:36:42.790 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.790 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.790 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:42.790 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.790 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.790 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.791 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2088 2024-09-14T22:36:42.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.791 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450943+0000", 2024-09-14T22:36:42.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450943+0000", 2024-09-14T22:36:42.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450944+0000", 2024-09-14T22:36:42.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2100000000000001e-06 2024-09-14T22:36:42.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450950+0000", 2024-09-14T22:36:42.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3329999999999996e-06 2024-09-14T22:36:42.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450968+0000", 2024-09-14T22:36:42.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7802999999999999e-05 2024-09-14T22:36:42.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450981+0000", 2024-09-14T22:36:42.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.349e-05 2024-09-14T22:36:42.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508752+0000", 2024-09-14T22:36:42.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057770957999999997 2024-09-14T22:36:42.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508884+0000", 2024-09-14T22:36:42.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013186900000000001 2024-09-14T22:36:42.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509054+0000", 2024-09-14T22:36:42.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000169705 2024-09-14T22:36:42.794 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.794 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.794 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.794 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2089 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11483136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:42.794 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450998+0000", 2024-09-14T22:36:42.794 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.291097425, 2024-09-14T22:36:42.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058422566000000002, 2024-09-14T22:36:42.794 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.794 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.794 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:42.794 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.795 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.795 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.795 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2089 2024-09-14T22:36:42.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.795 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450998+0000", 2024-09-14T22:36:42.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450998+0000", 2024-09-14T22:36:42.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450999+0000", 2024-09-14T22:36:42.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0100000000000001e-06 2024-09-14T22:36:42.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451014+0000", 2024-09-14T22:36:42.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4486e-05 2024-09-14T22:36:42.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451032+0000", 2024-09-14T22:36:42.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7893e-05 2024-09-14T22:36:42.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451046+0000", 2024-09-14T22:36:42.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4783999999999999e-05 2024-09-14T22:36:42.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509131+0000", 2024-09-14T22:36:42.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058085070000000003 2024-09-14T22:36:42.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509253+0000", 2024-09-14T22:36:42.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012173299999999999 2024-09-14T22:36:42.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509421+0000", 2024-09-14T22:36:42.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016759000000000001 2024-09-14T22:36:42.798 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.798 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.798 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.798 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2090 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11491328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:42.798 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.451064+0000", 2024-09-14T22:36:42.798 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.291031649000001, 2024-09-14T22:36:42.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058723076999999999, 2024-09-14T22:36:42.798 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.798 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.799 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:42.799 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.799 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.799 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.799 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2090 2024-09-14T22:36:42.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.799 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451064+0000", 2024-09-14T22:36:42.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451064+0000", 2024-09-14T22:36:42.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451065+0000", 2024-09-14T22:36:42.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2160000000000001e-06 2024-09-14T22:36:42.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451070+0000", 2024-09-14T22:36:42.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1340000000000003e-06 2024-09-14T22:36:42.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451088+0000", 2024-09-14T22:36:42.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7476e-05 2024-09-14T22:36:42.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451101+0000", 2024-09-14T22:36:42.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3723999999999999e-05 2024-09-14T22:36:42.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509492+0000", 2024-09-14T22:36:42.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058390899000000003 2024-09-14T22:36:42.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509618+0000", 2024-09-14T22:36:42.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012593899999999999 2024-09-14T22:36:42.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509787+0000", 2024-09-14T22:36:42.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000168689 2024-09-14T22:36:42.802 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.802 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.802 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.802 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2091 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11499520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:42.803 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.451119+0000", 2024-09-14T22:36:42.803 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.290976596, 2024-09-14T22:36:42.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059045274000000002, 2024-09-14T22:36:42.803 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.803 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.803 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:42.803 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.803 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.803 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.803 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2091 2024-09-14T22:36:42.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.803 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451119+0000", 2024-09-14T22:36:42.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451119+0000", 2024-09-14T22:36:42.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451120+0000", 2024-09-14T22:36:42.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0669999999999999e-06 2024-09-14T22:36:42.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451125+0000", 2024-09-14T22:36:42.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3329999999999996e-06 2024-09-14T22:36:42.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451143+0000", 2024-09-14T22:36:42.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7309999999999999e-05 2024-09-14T22:36:42.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451157+0000", 2024-09-14T22:36:42.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3835999999999999e-05 2024-09-14T22:36:42.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509863+0000", 2024-09-14T22:36:42.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058706345 2024-09-14T22:36:42.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509985+0000", 2024-09-14T22:36:42.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000121736 2024-09-14T22:36:42.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510164+0000", 2024-09-14T22:36:42.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017964700000000001 2024-09-14T22:36:42.806 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.806 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.806 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.807 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2092 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11507712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:42.807 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.451202+0000", 2024-09-14T22:36:42.807 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.290893923999999, 2024-09-14T22:36:42.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059345031999999999, 2024-09-14T22:36:42.807 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.807 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.807 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:42.807 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.807 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.807 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.807 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2092 2024-09-14T22:36:42.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.808 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451202+0000", 2024-09-14T22:36:42.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451202+0000", 2024-09-14T22:36:42.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451203+0000", 2024-09-14T22:36:42.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6199999999999999e-06 2024-09-14T22:36:42.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451209+0000", 2024-09-14T22:36:42.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1999999999999999e-06 2024-09-14T22:36:42.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451230+0000", 2024-09-14T22:36:42.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0247000000000001e-05 2024-09-14T22:36:42.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451245+0000", 2024-09-14T22:36:42.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4993e-05 2024-09-14T22:36:42.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510239+0000", 2024-09-14T22:36:42.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058994321000000002 2024-09-14T22:36:42.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510371+0000", 2024-09-14T22:36:42.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013195899999999999 2024-09-14T22:36:42.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510547+0000", 2024-09-14T22:36:42.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000175692 2024-09-14T22:36:42.811 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.811 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.811 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.811 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2093 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11515904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:42.811 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.451264+0000", 2024-09-14T22:36:42.811 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.290831887, 2024-09-14T22:36:42.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059636293999999999, 2024-09-14T22:36:42.811 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.811 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.811 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:42.811 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.812 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.812 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.812 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2093 2024-09-14T22:36:42.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.812 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451264+0000", 2024-09-14T22:36:42.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451264+0000", 2024-09-14T22:36:42.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451265+0000", 2024-09-14T22:36:42.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3400000000000001e-06 2024-09-14T22:36:42.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451270+0000", 2024-09-14T22:36:42.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4360000000000001e-06 2024-09-14T22:36:42.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451289+0000", 2024-09-14T22:36:42.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8383e-05 2024-09-14T22:36:42.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451303+0000", 2024-09-14T22:36:42.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.455e-05 2024-09-14T22:36:42.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510627+0000", 2024-09-14T22:36:42.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059323280999999999 2024-09-14T22:36:42.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510745+0000", 2024-09-14T22:36:42.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011867900000000001 2024-09-14T22:36:42.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510900+0000", 2024-09-14T22:36:42.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000154625 2024-09-14T22:36:42.815 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.815 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.815 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.815 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2094 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11524096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:42.815 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.451321+0000", 2024-09-14T22:36:42.815 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.290774900999999, 2024-09-14T22:36:42.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059938827, 2024-09-14T22:36:42.816 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.816 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.816 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:42.816 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.816 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.816 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.816 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2094 2024-09-14T22:36:42.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.816 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451321+0000", 2024-09-14T22:36:42.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451321+0000", 2024-09-14T22:36:42.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451322+0000", 2024-09-14T22:36:42.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0529999999999999e-06 2024-09-14T22:36:42.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451328+0000", 2024-09-14T22:36:42.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8300000000000001e-06 2024-09-14T22:36:42.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451344+0000", 2024-09-14T22:36:42.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6750000000000001e-05 2024-09-14T22:36:42.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451358+0000", 2024-09-14T22:36:42.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3200000000000001e-05 2024-09-14T22:36:42.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510968+0000", 2024-09-14T22:36:42.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059610567000000003 2024-09-14T22:36:42.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.819 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.819 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.511094+0000", 2024-09-14T22:36:42.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012629199999999999 2024-09-14T22:36:42.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.819 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.819 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.511260+0000", 2024-09-14T22:36:42.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016513500000000001 2024-09-14T22:36:42.819 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.819 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.819 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.819 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2095 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11532288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:42.820 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.451375+0000", 2024-09-14T22:36:42.820 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.290720728, 2024-09-14T22:36:42.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060228067000000003, 2024-09-14T22:36:42.820 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.820 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.820 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:42.820 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.820 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.820 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.820 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2095 2024-09-14T22:36:42.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.820 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451375+0000", 2024-09-14T22:36:42.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451375+0000", 2024-09-14T22:36:42.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451376+0000", 2024-09-14T22:36:42.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0470000000000001e-06 2024-09-14T22:36:42.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451381+0000", 2024-09-14T22:36:42.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.536e-06 2024-09-14T22:36:42.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451399+0000", 2024-09-14T22:36:42.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7200000000000001e-05 2024-09-14T22:36:42.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451412+0000", 2024-09-14T22:36:42.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3726999999999999e-05 2024-09-14T22:36:42.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.511308+0000", 2024-09-14T22:36:42.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059895176000000001 2024-09-14T22:36:42.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.511444+0000", 2024-09-14T22:36:42.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013621600000000001 2024-09-14T22:36:42.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.511603+0000", 2024-09-14T22:36:42.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015916500000000001 2024-09-14T22:36:42.823 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.823 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.824 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.824 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:45459 3.11 3:8ce265cb:::rbd_data.113cc4e35b3.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:36:42.824 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:36.190430+0000", 2024-09-14T22:36:42.824 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 6.5516659839999996, 2024-09-14T22:36:42.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.06125162, 2024-09-14T22:36:42.824 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.824 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.824 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:42.824 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.824 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.824 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.824 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 45459 2024-09-14T22:36:42.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.825 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.825 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.825 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190430+0000", 2024-09-14T22:36:42.825 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.825 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.825 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190430+0000", 2024-09-14T22:36:42.825 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.825 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.826 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190432+0000", 2024-09-14T22:36:42.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3039999999999999e-06 2024-09-14T22:36:42.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.826 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.826 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190443+0000", 2024-09-14T22:36:42.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1178999999999999e-05 2024-09-14T22:36:42.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.826 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.826 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190474+0000", 2024-09-14T22:36:42.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0803000000000003e-05 2024-09-14T22:36:42.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.826 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190489+0000", 2024-09-14T22:36:42.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.517e-05 2024-09-14T22:36:42.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190623+0000", 2024-09-14T22:36:42.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013405599999999999 2024-09-14T22:36:42.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190752+0000", 2024-09-14T22:36:42.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012870899999999999 2024-09-14T22:36:42.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:36:42.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.203210+0000", 2024-09-14T22:36:42.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012458484000000001 2024-09-14T22:36:42.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:42.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.251517+0000", 2024-09-14T22:36:42.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.048306453999999999 2024-09-14T22:36:42.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:42.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.251571+0000", 2024-09-14T22:36:42.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4228999999999999e-05 2024-09-14T22:36:42.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:42.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.251628+0000", 2024-09-14T22:36:42.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6922000000000002e-05 2024-09-14T22:36:42.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.251681+0000", 2024-09-14T22:36:42.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3310000000000003e-05 2024-09-14T22:36:42.829 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.829 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.829 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.829 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:45462 3.11 3:8ce265cb:::rbd_data.113cc4e35b3.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:36:42.830 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:36.258125+0000", 2024-09-14T22:36:42.830 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 6.4839704060000001, 2024-09-14T22:36:42.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059222599000000001, 2024-09-14T22:36:42.830 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:42.830 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:42.830 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:42.830 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:42.830 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:42.830 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:42.830 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 45462 2024-09-14T22:36:42.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.830 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:42.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:42.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258125+0000", 2024-09-14T22:36:42.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:42.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258125+0000", 2024-09-14T22:36:42.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:42.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:42.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258127+0000", 2024-09-14T22:36:42.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3159999999999999e-06 2024-09-14T22:36:42.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:42.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258139+0000", 2024-09-14T22:36:42.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1667e-05 2024-09-14T22:36:42.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.832 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:42.832 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258170+0000", 2024-09-14T22:36:42.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0929999999999997e-05 2024-09-14T22:36:42.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.832 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:42.832 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258185+0000", 2024-09-14T22:36:42.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4423000000000001e-05 2024-09-14T22:36:42.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.832 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:42.832 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258291+0000", 2024-09-14T22:36:42.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106285 2024-09-14T22:36:42.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:42.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258446+0000", 2024-09-14T22:36:42.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015486499999999999 2024-09-14T22:36:42.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:36:42.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.269615+0000", 2024-09-14T22:36:42.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011169095 2024-09-14T22:36:42.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:42.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.317113+0000", 2024-09-14T22:36:42.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.047498382999999998 2024-09-14T22:36:42.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.834 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:42.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.317144+0000", 2024-09-14T22:36:42.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0970000000000003e-05 2024-09-14T22:36:42.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.834 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:42.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.317233+0000", 2024-09-14T22:36:42.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9205999999999999e-05 2024-09-14T22:36:42.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:42.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:42.834 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:42.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.317348+0000", 2024-09-14T22:36:42.835 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000114459 2024-09-14T22:36:42.835 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.835 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.835 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.835 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:42.835 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:42.835 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:42.837 INFO:teuthology.orchestra.run.smithi001.stdout:1244 writesame 0x8514979 thru 0x8523cc9 data_size 0x1b(0xf351 bytes) 2024-09-14T22:36:42.837 INFO:teuthology.orchestra.run.smithi001.stdout:1246 write 0xc23f203 thru 0xc2441d0 (0x4fce bytes) 2024-09-14T22:36:42.837 INFO:teuthology.orchestra.run.smithi001.stdout:1248 write 0x80918e1 thru 0x80a0121 (0xe841 bytes) 2024-09-14T22:36:42.837 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:42.837 INFO:teuthology.orchestra.run.smithi001.stdout:1250 writesame 0x50321c9 thru 0x5035c97 data_size 0x1(0x3acf bytes) 2024-09-14T22:36:42.837 INFO:teuthology.orchestra.run.smithi001.stdout:1251 read 0x31d015d thru 0x31d952b (0x93cf bytes) 2024-09-14T22:36:42.837 INFO:teuthology.orchestra.run.smithi001.stdout:1252 writesame 0xd5d544e thru 0xd5d5d47 data_size 0x3(0x8fa bytes) 2024-09-14T22:36:42.838 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:36:42.838 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:42.830+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:42.847 INFO:teuthology.orchestra.run.smithi001.stdout:1254 trunc from 0xd5d5d48 to 0x54878d 2024-09-14T22:36:42.900 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:42.900 INFO:teuthology.orchestra.run.smithi001.stdout:1256 read 0x39a493 thru 0x3a1ba7 (0x7715 bytes) 2024-09-14T22:36:42.901 INFO:teuthology.orchestra.run.smithi001.stdout:1257 read 0x500ae6 thru 0x5035e2 (0x2afd bytes) 2024-09-14T22:36:42.907 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:36:42.907 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:36:42.911 INFO:teuthology.orchestra.run.smithi001.stdout:1259 trunc from 0x54878d to 0x6b5f86c 2024-09-14T22:36:42.924 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:42.929 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:36:42.929 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:36:42.929 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-14T22:36:42.937 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:42.931+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:42.939 INFO:teuthology.orchestra.run.smithi001.stdout:1262 write 0x8e177e8 thru 0x8e187a9 (0xfc2 bytes) 2024-09-14T22:36:42.945 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:42.948 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:36:42.948 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:36:42.965 INFO:teuthology.orchestra.run.smithi001.stdout:1266 writesame 0xe016166 thru 0xe01f301 data_size 0x2(0x919c bytes) 2024-09-14T22:36:42.965 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:42.965 INFO:teuthology.orchestra.run.smithi001.stdout:1269 punch from 0xb7c54ea to 0xb7d4516, (0xf02c bytes) 2024-09-14T22:36:42.965 INFO:teuthology.orchestra.run.smithi001.stdout:1270 read 0xc2a9dd thru 0xc330d0 (0x86f4 bytes) 2024-09-14T22:36:42.965 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:42.965 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:42.965 INFO:teuthology.orchestra.run.smithi001.stdout:1273 read 0x4f4433f thru 0x4f4e066 (0x9d28 bytes) 2024-09-14T22:36:42.972 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:36:42.972 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:36:42.978 INFO:teuthology.orchestra.run.smithi001.stdout:1274 write 0xe86af10 thru 0xe86daed (0x2bde bytes) 2024-09-14T22:36:42.984 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:42.984 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:42.985 INFO:teuthology.orchestra.run.smithi001.stdout:1275 writesame 0x4d012a4 thru 0x4d0f260 data_size 0xb(0xdfbd bytes) 2024-09-14T22:36:42.993 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:36:42.994 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:36:42.997 INFO:teuthology.orchestra.run.smithi001.stdout:1276 read 0x9352ce4 thru 0x935945a (0x6777 bytes) 2024-09-14T22:36:42.997 INFO:teuthology.orchestra.run.smithi001.stdout:1277 read 0xe31e644 thru 0xe32dcd6 (0xf693 bytes) 2024-09-14T22:36:42.997 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:42.997 INFO:teuthology.orchestra.run.smithi001.stdout:1279 write 0xbf73cd7 thru 0xbf75b16 (0x1e40 bytes) 2024-09-14T22:36:42.999 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:36:43.004 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:43.004 INFO:teuthology.orchestra.run.smithi001.stdout:1281 read 0x6343aab thru 0x6350709 (0xcc5f bytes) 2024-09-14T22:36:43.004 INFO:teuthology.orchestra.run.smithi001.stdout:1284 read 0x9c9f439 thru 0x9ca30a8 (0x3c70 bytes) 2024-09-14T22:36:43.004 INFO:teuthology.orchestra.run.smithi001.stdout:1285 punch from 0xc2e51ff to 0xc2eeb1e, (0x991f bytes) 2024-09-14T22:36:43.004 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:43.005 INFO:teuthology.orchestra.run.smithi001.stdout:1287 write 0x593e261 thru 0x594135d (0x30fd bytes) 2024-09-14T22:36:43.013 INFO:teuthology.orchestra.run.smithi001.stdout:1289 trunc from 0xe86daee to 0xcf9047c 2024-09-14T22:36:43.017 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:36:43.018 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:36:43.034 INFO:teuthology.orchestra.run.smithi001.stdout:1290 write 0x9a5e0f7 thru 0x9a5ffd9 (0x1ee3 bytes) 2024-09-14T22:36:43.038 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:43.031+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:43.039 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:36:43.039 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:36:43.040 INFO:teuthology.orchestra.run.smithi001.stdout:1291 write 0x5d71e76 thru 0x5d7624a (0x43d5 bytes) 2024-09-14T22:36:43.043 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:36:43.044 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:36:43.046 INFO:teuthology.orchestra.run.smithi001.stdout:1295 read 0x9ac3dc6 thru 0x9ac4499 (0x6d4 bytes) 2024-09-14T22:36:43.046 INFO:teuthology.orchestra.run.smithi001.stdout:1296 punch from 0x364aed0 to 0x364c574, (0x16a4 bytes) 2024-09-14T22:36:43.046 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:43.046 INFO:teuthology.orchestra.run.smithi001.stdout:1298 read 0x6e977c2 thru 0x6e9f060 (0x789f bytes) 2024-09-14T22:36:43.046 INFO:teuthology.orchestra.run.smithi001.stdout:1299 write 0x3af3e thru 0x483d1 (0xd494 bytes) 2024-09-14T22:36:43.062 INFO:teuthology.orchestra.run.smithi001.stdout:1300 writesame 0xa257332 thru 0xa25bbec data_size 0x1(0x48bb bytes) 2024-09-14T22:36:43.087 INFO:teuthology.orchestra.run.smithi001.stdout:1301 write 0x224c1cb thru 0x224c4f4 (0x32a bytes) 2024-09-14T22:36:43.093 INFO:teuthology.orchestra.run.smithi001.stdout:1302 read 0x60290c6 thru 0x6032a63 (0x999e bytes) 2024-09-14T22:36:43.093 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:43.094 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:43.094 INFO:teuthology.orchestra.run.smithi001.stdout:1306 write 0x669dd69 thru 0x66acdb1 (0xf049 bytes) 2024-09-14T22:36:43.102 INFO:teuthology.orchestra.run.smithi001.stdout:1307 writesame 0x9d56561 thru 0x9d64800 data_size 0x62(0xe2a0 bytes) 2024-09-14T22:36:43.123 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:43.123 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:43.139 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:43.135+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:43.139 INFO:teuthology.orchestra.run.smithi001.stdout:1308 trunc from 0xcf9047c to 0x21a7a4 2024-09-14T22:36:43.146 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:36:43.207 INFO:teuthology.orchestra.run.smithi001.stdout:1309 writesame 0x13d327c thru 0x13d4b4b data_size 0x8(0x18d0 bytes) 2024-09-14T22:36:43.222 INFO:teuthology.orchestra.run.smithi001.stdout:1310 write 0x16fec6d thru 0x17026d4 (0x3a68 bytes) 2024-09-14T22:36:43.239 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:43.234+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:43.276 INFO:teuthology.orchestra.run.smithi001.stdout:1311 writesame 0xe267cdc thru 0xe26a6ad data_size 0x35(0x29d2 bytes) 2024-09-14T22:36:43.276 INFO:teuthology.orchestra.run.smithi001.stdout:1313 punch from 0x1b11b80 to 0x1b1713e, (0x55be bytes) 2024-09-14T22:36:43.276 INFO:teuthology.orchestra.run.smithi001.stdout:1314 write 0x50267f5 thru 0x502fd05 (0x9511 bytes) 2024-09-14T22:36:43.284 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:43.284 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:43.284 INFO:teuthology.orchestra.run.smithi001.stdout:1317 write 0x2a0f162 thru 0x2a196b8 (0xa557 bytes) 2024-09-14T22:36:43.292 INFO:teuthology.orchestra.run.smithi001.stdout:1318 punch from 0x4a608d1 to 0x4a6902e, (0x875d bytes) 2024-09-14T22:36:43.292 INFO:teuthology.orchestra.run.smithi001.stdout:1319 writesame 0xc14e1e2 thru 0xc150917 data_size 0x15(0x2736 bytes) 2024-09-14T22:36:43.301 INFO:teuthology.orchestra.run.smithi001.stdout:1320 flatten 2024-09-14T22:36:43.315 INFO:teuthology.orchestra.run.smithi001.stdout:1321 write 0x521579f thru 0x5219d8a (0x45ec bytes) 2024-09-14T22:36:43.323 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:43.323 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:43.341 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:43.334+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:43.345 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:36:43.345 INFO:teuthology.orchestra.run.smithi001.stdout:1323 trunc from 0xe26a6ae to 0x2a85fc1 2024-09-14T22:36:43.392 INFO:teuthology.orchestra.run.smithi001.stdout:1326 trunc from 0x2a85fc1 to 0xc7313f5 2024-09-14T22:36:43.406 INFO:teuthology.orchestra.run.smithi001.stdout:1327 writesame 0xbabb9f1 thru 0xbac5d27 data_size 0x2f(0xa337 bytes) 2024-09-14T22:36:43.414 INFO:teuthology.orchestra.run.smithi001.stdout:1328 read 0x85408e9 thru 0x8545d97 (0x54af bytes) 2024-09-14T22:36:43.414 INFO:teuthology.orchestra.run.smithi001.stdout:1329 read 0x4983e thru 0x49c12 (0x3d5 bytes) 2024-09-14T22:36:43.415 INFO:teuthology.orchestra.run.smithi001.stdout:1331 writesame 0x5f8fcd9 thru 0x5f9c98f data_size 0x1b(0xccb7 bytes) 2024-09-14T22:36:43.423 INFO:teuthology.orchestra.run.smithi001.stdout:1333 read 0x344fc3f thru 0x3457ca8 (0x806a bytes) 2024-09-14T22:36:43.423 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:43.423 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:43.423 INFO:teuthology.orchestra.run.smithi001.stdout:1338 read 0x7fc16dc thru 0x7fc349a (0x1dbf bytes) 2024-09-14T22:36:43.424 INFO:teuthology.orchestra.run.smithi001.stdout:1339 writesame 0x3c58f4e thru 0x3c5b655 data_size 0x4(0x2708 bytes) 2024-09-14T22:36:43.432 INFO:teuthology.orchestra.run.smithi001.stdout:1342 read 0x839e3c2 thru 0x83a6511 (0x8150 bytes) 2024-09-14T22:36:43.432 INFO:teuthology.orchestra.run.smithi001.stdout:1344 writesame 0xa114933 thru 0xa121104 data_size 0x1(0xc7d2 bytes) 2024-09-14T22:36:43.441 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:43.435+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:43.469 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:43.469 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:43.482 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:36:43.507 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:43.507 INFO:teuthology.orchestra.run.smithi001.stdout:1346 clone 10 order 24 su 4194304 sc 11 2024-09-14T22:36:43.542 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:43.535+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:43.609 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:43.609 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:43.624 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:36:43.643 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:43.639+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:43.743 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:43.738+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:43.792 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:43.792 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:43.807 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:36:43.843 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:43.839+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:43.878 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T22:36:43.944 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:43.939+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:43.984 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:43.984 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:44.001 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:36:44.045 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:44.039+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:44.124 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:44.124 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:44.142 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:36:44.186 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:44.182+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:44.247 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:44.243+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:44.272 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-14T22:36:44.276 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:44.277 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:44.296 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:36:44.349 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:44.342+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:44.449 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:44.443+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:44.469 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:44.469 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:44.483 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:36:44.513 INFO:teuthology.orchestra.run.smithi001.stdout:1347 write 0x13afdea thru 0x13b6b40 (0x6d57 bytes) 2024-09-14T22:36:44.529 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:44.529 INFO:teuthology.orchestra.run.smithi001.stdout:1349 read 0xd67f67 thru 0xd75a10 (0xdaaa bytes) 2024-09-14T22:36:44.536 INFO:teuthology.orchestra.run.smithi001.stdout:1351 trunc from 0xc7313f5 to 0x78e9815 2024-09-14T22:36:44.574 INFO:teuthology.orchestra.run.smithi001.stdout:1352 write 0x7054394 thru 0x705c53e (0x81ab bytes) 2024-09-14T22:36:44.576 INFO:teuthology.orchestra.run.smithi001.stdout:1353 read 0x2efd5c6 thru 0x2f077b5 (0xa1f0 bytes) 2024-09-14T22:36:44.577 INFO:teuthology.orchestra.run.smithi001.stdout:1355 write 0x339c8d4 thru 0x33a654f (0x9c7c bytes) 2024-09-14T22:36:44.579 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:44.575+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:44.579 INFO:teuthology.orchestra.run.smithi001.stdout:1357 write 0x38e4a2b thru 0x38f0f4f (0xc525 bytes) 2024-09-14T22:36:44.581 INFO:teuthology.orchestra.run.smithi001.stdout:1359 write 0x693f88c thru 0x6949e38 (0xa5ad bytes) 2024-09-14T22:36:44.584 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:44.598 INFO:teuthology.orchestra.run.smithi001.stdout:1362 writesame 0xa2c0b45 thru 0xa2c1be8 data_size 0x4(0x10a4 bytes) 2024-09-14T22:36:44.599 INFO:teuthology.orchestra.run.smithi001.stdout:1364 punch from 0xa09f2de to 0xa0a8630, (0x9352 bytes) 2024-09-14T22:36:44.599 INFO:teuthology.orchestra.run.smithi001.stdout:1368 read 0x29b6ca0 thru 0x29c1aab (0xae0c bytes) 2024-09-14T22:36:44.600 INFO:teuthology.orchestra.run.smithi001.stdout:1370 write 0x13d6be0 thru 0x13ddcd5 (0x70f6 bytes) 2024-09-14T22:36:44.603 INFO:teuthology.orchestra.run.smithi001.stdout:1371 read 0x523b8a4 thru 0x52423d0 (0x6b2d bytes) 2024-09-14T22:36:44.603 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:44.608 INFO:teuthology.orchestra.run.smithi001.stdout:1373 trunc from 0xa2c1be9 to 0x829c82b 2024-09-14T22:36:44.620 INFO:teuthology.orchestra.run.smithi001.stdout:1374 punch from 0x396e2c to 0x397ee1, (0x10b5 bytes) 2024-09-14T22:36:44.620 INFO:teuthology.orchestra.run.smithi001.stdout:1375 read 0xe8a97a thru 0xe8ccd1 (0x2358 bytes) 2024-09-14T22:36:44.620 INFO:teuthology.orchestra.run.smithi001.stdout:1376 punch from 0x40423cb to 0x4044aea, (0x271f bytes) 2024-09-14T22:36:44.621 INFO:teuthology.orchestra.run.smithi001.stdout:1377 writesame 0x7b46742 thru 0x7b4d078 data_size 0x1(0x6937 bytes) 2024-09-14T22:36:44.642 INFO:teuthology.orchestra.run.smithi001.stdout:1379 writesame 0x35a8684 thru 0x35abf8f data_size 0x6(0x390c bytes) 2024-09-14T22:36:44.646 INFO:teuthology.orchestra.run.smithi001.stdout:1380 writesame 0x71028c2 thru 0x710cfcb data_size 0x3(0xa70a bytes) 2024-09-14T22:36:44.651 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:44.646+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:44.656 INFO:teuthology.orchestra.run.smithi001.stdout:1381 read 0x5922280 thru 0x5928027 (0x5da8 bytes) 2024-09-14T22:36:44.660 INFO:teuthology.orchestra.run.smithi001.stdout:1382 trunc from 0x829c82b to 0x9329bad 2024-09-14T22:36:44.665 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:44.665 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:44.673 INFO:teuthology.orchestra.run.smithi001.stdout:1383 punch from 0x140716f to 0x1415ce4, (0xeb75 bytes) 2024-09-14T22:36:44.673 INFO:teuthology.orchestra.run.smithi001.stdout:1384 read 0x32d6808 thru 0x32e11fc (0xa9f5 bytes) 2024-09-14T22:36:44.674 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:44.674 INFO:teuthology.orchestra.run.smithi001.stdout:1386 write 0x720ceed thru 0x7219a8d (0xcba1 bytes) 2024-09-14T22:36:44.680 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:36:44.691 INFO:teuthology.orchestra.run.smithi001.stdout:1389 trunc from 0x9329bad to 0xe27f148 2024-09-14T22:36:44.714 INFO:teuthology.orchestra.run.smithi001.stdout:1391 trunc from 0xe27f148 to 0x9a9ae4f 2024-09-14T22:36:44.728 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:44.739 INFO:teuthology.orchestra.run.smithi001.stdout:1394 trunc from 0x9a9ae4f to 0x4f4813f 2024-09-14T22:36:44.751 INFO:teuthology.orchestra.run.smithi001.stdout:1395 writesame 0x3e2056e thru 0x3e2f957 data_size 0x9(0xf3ea bytes) 2024-09-14T22:36:44.756 INFO:teuthology.orchestra.run.smithi001.stdout:1396 punch from 0x2abb434 to 0x2ac562e, (0xa1fa bytes) 2024-09-14T22:36:44.756 INFO:teuthology.orchestra.run.smithi001.stdout:1397 read 0x2f6e483 thru 0x2f7c033 (0xdbb1 bytes) 2024-09-14T22:36:44.757 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:44.757 INFO:teuthology.orchestra.run.smithi001.stdout:1400 punch from 0x1c8d46e to 0x1c96ab4, (0x9646 bytes) 2024-09-14T22:36:44.771 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:44.767+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:44.785 INFO:teuthology.orchestra.run.smithi001.stdout:1401 write 0xb781225 thru 0xb7846c8 (0x34a4 bytes) 2024-09-14T22:36:44.812 INFO:teuthology.orchestra.run.smithi001.stdout:1402 write 0xde972ce thru 0xdea0fd5 (0x9d08 bytes) 2024-09-14T22:36:44.849 INFO:teuthology.orchestra.run.smithi001.stdout:1404 trunc from 0xdea0fd6 to 0x3121659 2024-09-14T22:36:44.852 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:44.852 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:44.853 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:44.846+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:44.869 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:36:44.897 INFO:teuthology.orchestra.run.smithi001.stdout:1406 write 0x60503ff thru 0x605419a (0x3d9c bytes) 2024-09-14T22:36:44.925 INFO:teuthology.orchestra.run.smithi001.stdout:1407 write 0xb1de394 thru 0xb1e86bd (0xa32a bytes) 2024-09-14T22:36:44.944 INFO:teuthology.orchestra.run.smithi001.stdout:1408 write 0xc414c48 thru 0xc41b3d1 (0x678a bytes) 2024-09-14T22:36:44.952 INFO:teuthology.orchestra.run.smithi001.stdout:1409 punch from 0x6872524 to 0x6874a83, (0x255f bytes) 2024-09-14T22:36:44.952 INFO:teuthology.orchestra.run.smithi001.stdout:1410 write 0x459826d thru 0x459be91 (0x3c25 bytes) 2024-09-14T22:36:44.953 INFO:teuthology.orchestra.run.smithi001.stdout:1412 clone 11 order 23 su 65536 sc 10 2024-09-14T22:36:44.955 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:44.951+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:44.958 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2024-09-14T22:36:45.037 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:45.037 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:45.045 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:36:45.051 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:36:45.096 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:45.091+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:45.155 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:45.151+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:45.199 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:36:45.199 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:36:45.215 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:36:45.216 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:36:45.216 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-14T22:36:45.220 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:45.220 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:45.234 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:36:45.235 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:36:45.235 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:36:45.256 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:36:45.256 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:36:45.275 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:45.271+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:45.280 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:36:45.280 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:36:45.301 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:36:45.301 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:36:45.322 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:36:45.322 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:36:45.326 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:36:45.327 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:36:45.357 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:45.350+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:45.401 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:45.401 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:36:45.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:36:45.402 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:36:45.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.402 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19590 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7272448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:45.402 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446391+0000", 2024-09-14T22:36:45.402 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.952106294, 2024-09-14T22:36:45.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027116533000000002, 2024-09-14T22:36:45.402 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:45.402 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:45.402 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:45.402 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:45.402 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:45.402 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:45.402 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19590 2024-09-14T22:36:45.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.403 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:45.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:45.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446391+0000", 2024-09-14T22:36:45.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:45.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446391+0000", 2024-09-14T22:36:45.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:45.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446392+0000", 2024-09-14T22:36:45.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4299999999999999e-07 2024-09-14T22:36:45.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:45.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446393+0000", 2024-09-14T22:36:45.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8059999999999999e-06 2024-09-14T22:36:45.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:45.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446399+0000", 2024-09-14T22:36:45.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6740000000000002e-06 2024-09-14T22:36:45.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:45.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446404+0000", 2024-09-14T22:36:45.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6240000000000001e-06 2024-09-14T22:36:45.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:45.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473382+0000", 2024-09-14T22:36:45.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026978716999999999 2024-09-14T22:36:45.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:45.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473446+0000", 2024-09-14T22:36:45.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3971000000000001e-05 2024-09-14T22:36:45.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:45.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473508+0000", 2024-09-14T22:36:45.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1397999999999997e-05 2024-09-14T22:36:45.406 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.406 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:45.406 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.406 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19591 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7301120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:45.406 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446415+0000", 2024-09-14T22:36:45.406 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.95208294, 2024-09-14T22:36:45.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027255773, 2024-09-14T22:36:45.406 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:45.407 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:45.407 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:45.407 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:45.407 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:45.407 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:45.407 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19591 2024-09-14T22:36:45.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.407 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:45.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:45.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446415+0000", 2024-09-14T22:36:45.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:45.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446415+0000", 2024-09-14T22:36:45.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:45.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446415+0000", 2024-09-14T22:36:45.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8800000000000002e-07 2024-09-14T22:36:45.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:45.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446417+0000", 2024-09-14T22:36:45.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7990000000000001e-06 2024-09-14T22:36:45.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:45.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446423+0000", 2024-09-14T22:36:45.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.066e-06 2024-09-14T22:36:45.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:45.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446428+0000", 2024-09-14T22:36:45.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5889999999999996e-06 2024-09-14T22:36:45.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:45.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473535+0000", 2024-09-14T22:36:45.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027107220000000001 2024-09-14T22:36:45.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:45.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473602+0000", 2024-09-14T22:36:45.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6482999999999993e-05 2024-09-14T22:36:45.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:45.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473670+0000", 2024-09-14T22:36:45.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8928000000000001e-05 2024-09-14T22:36:45.410 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.410 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:45.410 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.411 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19592 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7329792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:45.411 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446433+0000", 2024-09-14T22:36:45.411 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.952064279, 2024-09-14T22:36:45.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027394960999999999, 2024-09-14T22:36:45.411 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:45.411 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:45.411 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:45.411 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:45.411 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:45.411 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:45.411 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19592 2024-09-14T22:36:45.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.411 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:45.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:45.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446433+0000", 2024-09-14T22:36:45.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:45.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446433+0000", 2024-09-14T22:36:45.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:45.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446434+0000", 2024-09-14T22:36:45.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3600000000000004e-07 2024-09-14T22:36:45.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:45.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446436+0000", 2024-09-14T22:36:45.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.796e-06 2024-09-14T22:36:45.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:45.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446441+0000", 2024-09-14T22:36:45.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7209999999999999e-06 2024-09-14T22:36:45.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:45.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446450+0000", 2024-09-14T22:36:45.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0780000000000002e-06 2024-09-14T22:36:45.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:45.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473705+0000", 2024-09-14T22:36:45.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027254246999999999 2024-09-14T22:36:45.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:45.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473769+0000", 2024-09-14T22:36:45.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4101000000000002e-05 2024-09-14T22:36:45.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:45.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473828+0000", 2024-09-14T22:36:45.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9481999999999997e-05 2024-09-14T22:36:45.414 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.414 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:45.415 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.415 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20100 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7219200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:45.415 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462167+0000", 2024-09-14T22:36:45.415 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.936330556, 2024-09-14T22:36:45.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027082189999999999, 2024-09-14T22:36:45.415 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:45.415 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:45.415 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:45.415 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:45.415 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:45.415 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:45.416 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20100 2024-09-14T22:36:45.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.416 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:45.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:45.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462167+0000", 2024-09-14T22:36:45.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:45.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462167+0000", 2024-09-14T22:36:45.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:45.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462168+0000", 2024-09-14T22:36:45.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8800000000000002e-07 2024-09-14T22:36:45.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:45.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462177+0000", 2024-09-14T22:36:45.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9330000000000007e-06 2024-09-14T22:36:45.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:45.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462183+0000", 2024-09-14T22:36:45.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6200000000000001e-06 2024-09-14T22:36:45.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:45.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462188+0000", 2024-09-14T22:36:45.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1590000000000001e-06 2024-09-14T22:36:45.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:45.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489109+0000", 2024-09-14T22:36:45.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026920881000000001 2024-09-14T22:36:45.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:45.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489181+0000", 2024-09-14T22:36:45.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2035999999999999e-05 2024-09-14T22:36:45.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:45.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489249+0000", 2024-09-14T22:36:45.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7972999999999994e-05 2024-09-14T22:36:45.419 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.419 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:45.419 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.419 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20101 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7247872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:45.419 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462195+0000", 2024-09-14T22:36:45.419 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.936302273000001, 2024-09-14T22:36:45.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027205994000000001, 2024-09-14T22:36:45.420 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:45.420 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:45.420 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:45.420 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:45.420 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:45.420 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:45.420 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20101 2024-09-14T22:36:45.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.420 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:45.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:45.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462195+0000", 2024-09-14T22:36:45.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:45.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462195+0000", 2024-09-14T22:36:45.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:45.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462196+0000", 2024-09-14T22:36:45.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8000000000000001e-07 2024-09-14T22:36:45.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:45.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462198+0000", 2024-09-14T22:36:45.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0439999999999998e-06 2024-09-14T22:36:45.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:45.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462205+0000", 2024-09-14T22:36:45.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3359999999999997e-06 2024-09-14T22:36:45.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:45.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462210+0000", 2024-09-14T22:36:45.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0649999999999998e-06 2024-09-14T22:36:45.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:45.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489279+0000", 2024-09-14T22:36:45.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027069123 2024-09-14T22:36:45.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:45.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489342+0000", 2024-09-14T22:36:45.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2759e-05 2024-09-14T22:36:45.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:45.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489401+0000", 2024-09-14T22:36:45.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9286999999999997e-05 2024-09-14T22:36:45.423 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.423 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:45.423 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.424 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20102 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7276544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:45.424 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462216+0000", 2024-09-14T22:36:45.424 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.936281324999999, 2024-09-14T22:36:45.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027352785000000001, 2024-09-14T22:36:45.424 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:45.424 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:45.424 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:45.424 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:45.424 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:45.424 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:45.424 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20102 2024-09-14T22:36:45.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.424 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:45.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:45.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462216+0000", 2024-09-14T22:36:45.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:45.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462216+0000", 2024-09-14T22:36:45.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:45.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462217+0000", 2024-09-14T22:36:45.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9100000000000003e-07 2024-09-14T22:36:45.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:45.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462219+0000", 2024-09-14T22:36:45.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9740000000000001e-06 2024-09-14T22:36:45.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:45.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462235+0000", 2024-09-14T22:36:45.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6050999999999999e-05 2024-09-14T22:36:45.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:45.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462247+0000", 2024-09-14T22:36:45.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2245999999999999e-05 2024-09-14T22:36:45.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:45.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489431+0000", 2024-09-14T22:36:45.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027183624999999999 2024-09-14T22:36:45.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:45.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489504+0000", 2024-09-14T22:36:45.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3498999999999995e-05 2024-09-14T22:36:45.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:45.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489569+0000", 2024-09-14T22:36:45.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4698999999999998e-05 2024-09-14T22:36:45.428 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.428 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:45.428 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.428 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.428 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20103 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7305216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:45.428 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462256+0000", 2024-09-14T22:36:45.428 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.936242118999999, 2024-09-14T22:36:45.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027473543, 2024-09-14T22:36:45.428 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:45.428 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:45.428 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:45.428 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:45.428 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:45.428 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:45.429 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20103 2024-09-14T22:36:45.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.429 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:45.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:45.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462256+0000", 2024-09-14T22:36:45.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:45.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462256+0000", 2024-09-14T22:36:45.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:45.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462256+0000", 2024-09-14T22:36:45.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1399999999999997e-07 2024-09-14T22:36:45.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:45.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462259+0000", 2024-09-14T22:36:45.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.694e-06 2024-09-14T22:36:45.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:45.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462266+0000", 2024-09-14T22:36:45.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8940000000000004e-06 2024-09-14T22:36:45.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:45.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462271+0000", 2024-09-14T22:36:45.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0459999999999996e-06 2024-09-14T22:36:45.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:45.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489601+0000", 2024-09-14T22:36:45.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027330027 2024-09-14T22:36:45.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:45.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489663+0000", 2024-09-14T22:36:45.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2807999999999999e-05 2024-09-14T22:36:45.432 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.432 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.432 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:45.432 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489729+0000", 2024-09-14T22:36:45.432 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5560000000000002e-05 2024-09-14T22:36:45.432 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.432 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:45.432 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.432 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.432 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.432 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20104 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7333888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:45.432 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462277+0000", 2024-09-14T22:36:45.432 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.936220583000001, 2024-09-14T22:36:45.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027652614999999998, 2024-09-14T22:36:45.433 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:45.433 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:45.433 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:45.433 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:45.433 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:45.433 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:45.433 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20104 2024-09-14T22:36:45.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.433 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:45.433 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.433 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:45.433 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462277+0000", 2024-09-14T22:36:45.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:45.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462277+0000", 2024-09-14T22:36:45.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:45.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462277+0000", 2024-09-14T22:36:45.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2800000000000002e-07 2024-09-14T22:36:45.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:45.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462280+0000", 2024-09-14T22:36:45.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2570000000000002e-06 2024-09-14T22:36:45.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.435 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:45.435 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462286+0000", 2024-09-14T22:36:45.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2199999999999997e-06 2024-09-14T22:36:45.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.435 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:45.435 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462291+0000", 2024-09-14T22:36:45.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5689999999999998e-06 2024-09-14T22:36:45.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.435 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:45.436 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489760+0000", 2024-09-14T22:36:45.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027469345999999999 2024-09-14T22:36:45.436 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.436 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.436 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:45.436 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489852+0000", 2024-09-14T22:36:45.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1941999999999995e-05 2024-09-14T22:36:45.436 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.436 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.436 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:45.436 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489930+0000", 2024-09-14T22:36:45.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7853000000000002e-05 2024-09-14T22:36:45.436 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.436 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:45.437 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.437 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.437 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20869 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7168000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:45.437 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483633+0000", 2024-09-14T22:36:45.437 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.914864228000001, 2024-09-14T22:36:45.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028181490999999999, 2024-09-14T22:36:45.437 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:45.437 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:45.437 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:45.437 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:45.437 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:45.437 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:45.437 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20869 2024-09-14T22:36:45.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.438 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:45.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:45.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483633+0000", 2024-09-14T22:36:45.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:45.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483633+0000", 2024-09-14T22:36:45.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:45.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483634+0000", 2024-09-14T22:36:45.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2599999999999998e-07 2024-09-14T22:36:45.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:45.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483636+0000", 2024-09-14T22:36:45.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1679999999999998e-06 2024-09-14T22:36:45.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:45.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483642+0000", 2024-09-14T22:36:45.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2149999999999998e-06 2024-09-14T22:36:45.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:45.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483647+0000", 2024-09-14T22:36:45.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7679999999999999e-06 2024-09-14T22:36:45.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:45.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.510694+0000", 2024-09-14T22:36:45.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027047073000000001 2024-09-14T22:36:45.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:45.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.510771+0000", 2024-09-14T22:36:45.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6814e-05 2024-09-14T22:36:45.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:45.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511815+0000", 2024-09-14T22:36:45.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0010440270000000001 2024-09-14T22:36:45.441 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.441 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:45.441 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.441 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20870 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7196672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:45.441 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483652+0000", 2024-09-14T22:36:45.441 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.914845226000001, 2024-09-14T22:36:45.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028301248000000001, 2024-09-14T22:36:45.441 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:45.441 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:45.441 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:45.441 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:45.442 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:45.442 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:45.442 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20870 2024-09-14T22:36:45.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.442 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:45.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:45.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483652+0000", 2024-09-14T22:36:45.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:45.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483652+0000", 2024-09-14T22:36:45.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.443 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:45.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483653+0000", 2024-09-14T22:36:45.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3099999999999997e-07 2024-09-14T22:36:45.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.443 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:45.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483655+0000", 2024-09-14T22:36:45.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.877e-06 2024-09-14T22:36:45.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.443 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:45.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483661+0000", 2024-09-14T22:36:45.444 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6049999999999997e-06 2024-09-14T22:36:45.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.444 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:45.444 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483666+0000", 2024-09-14T22:36:45.444 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3410000000000002e-06 2024-09-14T22:36:45.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.444 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:45.444 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511854+0000", 2024-09-14T22:36:45.444 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028187773999999999 2024-09-14T22:36:45.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.444 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:45.444 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511906+0000", 2024-09-14T22:36:45.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2012000000000003e-05 2024-09-14T22:36:45.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:45.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511954+0000", 2024-09-14T22:36:45.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8007999999999998e-05 2024-09-14T22:36:45.445 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.445 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:45.445 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.445 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20871 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7225344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:45.445 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483671+0000", 2024-09-14T22:36:45.445 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.914826303, 2024-09-14T22:36:45.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028405392000000002, 2024-09-14T22:36:45.446 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:45.446 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:45.446 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:45.446 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:45.446 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:45.446 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:45.446 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20871 2024-09-14T22:36:45.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.446 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:45.446 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.446 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:45.446 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483671+0000", 2024-09-14T22:36:45.446 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.446 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.447 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:45.447 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483671+0000", 2024-09-14T22:36:45.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.447 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:45.447 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483672+0000", 2024-09-14T22:36:45.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:36:45.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.447 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:45.447 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483674+0000", 2024-09-14T22:36:45.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3190000000000002e-06 2024-09-14T22:36:45.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.448 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:45.448 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483680+0000", 2024-09-14T22:36:45.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6860000000000003e-06 2024-09-14T22:36:45.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.448 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:45.448 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483684+0000", 2024-09-14T22:36:45.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3769999999999998e-06 2024-09-14T22:36:45.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.448 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:45.448 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511982+0000", 2024-09-14T22:36:45.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028297478000000001 2024-09-14T22:36:45.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.449 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:45.449 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512027+0000", 2024-09-14T22:36:45.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5290000000000002e-05 2024-09-14T22:36:45.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.449 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:45.449 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512077+0000", 2024-09-14T22:36:45.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9876000000000002e-05 2024-09-14T22:36:45.449 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.449 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:45.449 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.450 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.450 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20872 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7254016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:45.450 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483689+0000", 2024-09-14T22:36:45.450 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.914808233, 2024-09-14T22:36:45.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028494049, 2024-09-14T22:36:45.450 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:45.450 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:45.450 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:45.450 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:45.450 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:45.450 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:45.450 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20872 2024-09-14T22:36:45.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.451 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:45.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:45.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483689+0000", 2024-09-14T22:36:45.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:45.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483689+0000", 2024-09-14T22:36:45.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:45.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483690+0000", 2024-09-14T22:36:45.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:36:45.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:45.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483692+0000", 2024-09-14T22:36:45.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.778e-06 2024-09-14T22:36:45.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:45.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483697+0000", 2024-09-14T22:36:45.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7069999999999997e-06 2024-09-14T22:36:45.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:45.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483702+0000", 2024-09-14T22:36:45.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4689999999999999e-06 2024-09-14T22:36:45.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:45.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512104+0000", 2024-09-14T22:36:45.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028401948999999999 2024-09-14T22:36:45.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:45.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512142+0000", 2024-09-14T22:36:45.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8114999999999997e-05 2024-09-14T22:36:45.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:45.454 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512183+0000", 2024-09-14T22:36:45.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1496999999999999e-05 2024-09-14T22:36:45.454 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.454 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:45.454 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.454 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.454 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.454 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20873 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7282688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:45.454 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483707+0000", 2024-09-14T22:36:45.454 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.914790519, 2024-09-14T22:36:45.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028581662000000001, 2024-09-14T22:36:45.454 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:45.454 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:45.454 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:45.455 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:45.455 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:45.455 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:45.455 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20873 2024-09-14T22:36:45.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.455 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:45.455 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.455 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:45.455 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483707+0000", 2024-09-14T22:36:45.455 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.455 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.455 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:45.455 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483707+0000", 2024-09-14T22:36:45.455 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:45.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483707+0000", 2024-09-14T22:36:45.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2500000000000001e-07 2024-09-14T22:36:45.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:45.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483709+0000", 2024-09-14T22:36:45.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.891e-06 2024-09-14T22:36:45.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:45.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483715+0000", 2024-09-14T22:36:45.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7019999999999998e-06 2024-09-14T22:36:45.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:45.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483720+0000", 2024-09-14T22:36:45.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4900000000000002e-06 2024-09-14T22:36:45.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:45.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512213+0000", 2024-09-14T22:36:45.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028493622 2024-09-14T22:36:45.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:45.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512248+0000", 2024-09-14T22:36:45.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5228000000000002e-05 2024-09-14T22:36:45.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:45.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512289+0000", 2024-09-14T22:36:45.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0404000000000001e-05 2024-09-14T22:36:45.458 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.458 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:45.458 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.458 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20874 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7311360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:45.458 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483725+0000", 2024-09-14T22:36:45.458 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.914772879999999, 2024-09-14T22:36:45.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028777193999999999, 2024-09-14T22:36:45.459 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:45.459 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:45.459 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:45.459 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:45.459 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:45.459 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:45.459 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20874 2024-09-14T22:36:45.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.459 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:45.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.459 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:45.459 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483725+0000", 2024-09-14T22:36:45.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.460 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.460 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:45.460 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483725+0000", 2024-09-14T22:36:45.460 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.460 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.460 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:45.460 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483725+0000", 2024-09-14T22:36:45.460 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8000000000000001e-07 2024-09-14T22:36:45.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.460 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.460 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:45.460 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483727+0000", 2024-09-14T22:36:45.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8920000000000001e-06 2024-09-14T22:36:45.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:45.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483733+0000", 2024-09-14T22:36:45.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6840000000000001e-06 2024-09-14T22:36:45.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:45.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483737+0000", 2024-09-14T22:36:45.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4440000000000001e-06 2024-09-14T22:36:45.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:45.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512326+0000", 2024-09-14T22:36:45.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028588612999999999 2024-09-14T22:36:45.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:45.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512399+0000", 2024-09-14T22:36:45.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2782999999999998e-05 2024-09-14T22:36:45.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:45.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512502+0000", 2024-09-14T22:36:45.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000103398 2024-09-14T22:36:45.462 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.462 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:45.462 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.463 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20912 3.1c 3:3b9f022e:::rbd_data.113c4558bb1.000000000000009a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 497664~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:45.463 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.484800+0000", 2024-09-14T22:36:45.463 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.913697352, 2024-09-14T22:36:45.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045787688999999999, 2024-09-14T22:36:45.463 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:45.463 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:45.463 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:45.463 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:45.463 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:45.463 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:45.463 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20912 2024-09-14T22:36:45.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.463 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:45.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:45.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484800+0000", 2024-09-14T22:36:45.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:45.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484800+0000", 2024-09-14T22:36:45.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:45.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484801+0000", 2024-09-14T22:36:45.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1939999999999999e-06 2024-09-14T22:36:45.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.465 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:45.465 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484809+0000", 2024-09-14T22:36:45.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2989999999999999e-06 2024-09-14T22:36:45.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.465 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:45.465 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484823+0000", 2024-09-14T22:36:45.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3986e-05 2024-09-14T22:36:45.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.465 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:45.466 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484831+0000", 2024-09-14T22:36:45.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4009999999999997e-06 2024-09-14T22:36:45.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:45.466 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512318+0000", 2024-09-14T22:36:45.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027487208999999999 2024-09-14T22:36:45.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:45.466 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512450+0000", 2024-09-14T22:36:45.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000131681 2024-09-14T22:36:45.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:36:45.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512666+0000", 2024-09-14T22:36:45.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021547 2024-09-14T22:36:45.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:45.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.513788+0000", 2024-09-14T22:36:45.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001122975 2024-09-14T22:36:45.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:45.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.530473+0000", 2024-09-14T22:36:45.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.016684984999999999 2024-09-14T22:36:45.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:45.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.530533+0000", 2024-09-14T22:36:45.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9296999999999998e-05 2024-09-14T22:36:45.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:45.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.530588+0000", 2024-09-14T22:36:45.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5192000000000003e-05 2024-09-14T22:36:45.468 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.468 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:45.468 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.468 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:30512 3.1d 3:b9e08d52:::rbd_data.113cda0183a0.0000000000000008:6 [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:45.469 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.331867+0000", 2024-09-14T22:36:45.469 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.066630397999999, 2024-09-14T22:36:45.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027901088000000001, 2024-09-14T22:36:45.469 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:45.469 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:45.469 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:45.469 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:45.469 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:45.469 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:45.469 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 30512 2024-09-14T22:36:45.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.469 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:45.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:45.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331867+0000", 2024-09-14T22:36:45.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.470 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.470 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.470 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:45.470 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331867+0000", 2024-09-14T22:36:45.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.470 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.470 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.470 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:45.470 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331868+0000", 2024-09-14T22:36:45.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8999999999999996e-07 2024-09-14T22:36:45.470 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.470 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.470 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:45.470 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331871+0000", 2024-09-14T22:36:45.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.163e-06 2024-09-14T22:36:45.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:45.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331880+0000", 2024-09-14T22:36:45.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3449999999999995e-06 2024-09-14T22:36:45.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:45.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331888+0000", 2024-09-14T22:36:45.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5320000000000001e-06 2024-09-14T22:36:45.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:45.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.358443+0000", 2024-09-14T22:36:45.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026554845000000001 2024-09-14T22:36:45.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:45.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.359649+0000", 2024-09-14T22:36:45.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001205991 2024-09-14T22:36:45.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:45.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.359768+0000", 2024-09-14T22:36:45.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119522 2024-09-14T22:36:45.472 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.472 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:45.473 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.473 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:30513 3.1d 3:b9e08d52:::rbd_data.113cda0183a0.0000000000000008:6 [read 456704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:45.473 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.331897+0000", 2024-09-14T22:36:45.473 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.066600171999999, 2024-09-14T22:36:45.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028098122999999999, 2024-09-14T22:36:45.473 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:45.473 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:45.473 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:45.473 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:45.473 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:45.473 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:45.473 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 30513 2024-09-14T22:36:45.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.474 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:45.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:45.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331897+0000", 2024-09-14T22:36:45.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:45.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331897+0000", 2024-09-14T22:36:45.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:45.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331898+0000", 2024-09-14T22:36:45.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:36:45.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.475 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:45.475 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331901+0000", 2024-09-14T22:36:45.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7889999999999999e-06 2024-09-14T22:36:45.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.475 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:45.475 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331910+0000", 2024-09-14T22:36:45.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.346e-06 2024-09-14T22:36:45.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.475 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:45.476 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331917+0000", 2024-09-14T22:36:45.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2239999999999998e-06 2024-09-14T22:36:45.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:45.476 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.359807+0000", 2024-09-14T22:36:45.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027889225 2024-09-14T22:36:45.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:45.476 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.359895+0000", 2024-09-14T22:36:45.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8721000000000007e-05 2024-09-14T22:36:45.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:45.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.359996+0000", 2024-09-14T22:36:45.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010022300000000001 2024-09-14T22:36:45.477 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.477 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:45.477 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.477 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:30514 3.1d 3:b9e08d52:::rbd_data.113cda0183a0.0000000000000008:6 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:45.477 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.331927+0000", 2024-09-14T22:36:45.477 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.066570359, 2024-09-14T22:36:45.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028476330000000001, 2024-09-14T22:36:45.477 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:45.477 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:45.478 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:45.478 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:45.478 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:45.478 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:45.478 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 30514 2024-09-14T22:36:45.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.478 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:45.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:45.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331927+0000", 2024-09-14T22:36:45.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:45.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331927+0000", 2024-09-14T22:36:45.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:45.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331928+0000", 2024-09-14T22:36:45.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5700000000000002e-07 2024-09-14T22:36:45.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:45.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331931+0000", 2024-09-14T22:36:45.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9909999999999998e-06 2024-09-14T22:36:45.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:45.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331940+0000", 2024-09-14T22:36:45.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.312e-06 2024-09-14T22:36:45.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.480 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:45.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331948+0000", 2024-09-14T22:36:45.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5630000000000003e-06 2024-09-14T22:36:45.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.480 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:45.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360032+0000", 2024-09-14T22:36:45.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028083877 2024-09-14T22:36:45.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:45.481 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360282+0000", 2024-09-14T22:36:45.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00025062100000000003 2024-09-14T22:36:45.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:45.481 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360404+0000", 2024-09-14T22:36:45.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000121409 2024-09-14T22:36:45.481 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.481 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:45.481 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.481 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:30515 3.1d 3:b9e08d52:::rbd_data.113cda0183a0.0000000000000008:6 [read 514048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:45.482 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.331957+0000", 2024-09-14T22:36:45.482 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.066540534, 2024-09-14T22:36:45.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028684729999999999, 2024-09-14T22:36:45.482 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:45.482 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:45.482 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:45.482 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:45.482 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:45.482 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:45.482 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 30515 2024-09-14T22:36:45.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.482 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:45.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:45.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331957+0000", 2024-09-14T22:36:45.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:45.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331957+0000", 2024-09-14T22:36:45.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:45.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331958+0000", 2024-09-14T22:36:45.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1200000000000003e-07 2024-09-14T22:36:45.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:45.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331960+0000", 2024-09-14T22:36:45.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6929999999999999e-06 2024-09-14T22:36:45.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.484 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:45.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331970+0000", 2024-09-14T22:36:45.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4020000000000008e-06 2024-09-14T22:36:45.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.484 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:45.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331977+0000", 2024-09-14T22:36:45.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7660000000000001e-06 2024-09-14T22:36:45.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.485 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:45.485 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360444+0000", 2024-09-14T22:36:45.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028467399000000001 2024-09-14T22:36:45.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.485 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:45.485 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360536+0000", 2024-09-14T22:36:45.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1618000000000003e-05 2024-09-14T22:36:45.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.485 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:45.485 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360642+0000", 2024-09-14T22:36:45.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010624 2024-09-14T22:36:45.485 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.486 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:45.486 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.486 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:46576 3.1d 3:b9b1f07f:::rbd_data.113c8642facc.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T22:36:45.486 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:37.379223+0000", 2024-09-14T22:36:45.486 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.0192750089999993, 2024-09-14T22:36:45.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072706044999999997, 2024-09-14T22:36:45.486 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:45.486 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:45.486 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:45.486 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:45.486 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:45.486 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:45.487 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 46576 2024-09-14T22:36:45.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.487 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:45.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:45.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379223+0000", 2024-09-14T22:36:45.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:45.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379223+0000", 2024-09-14T22:36:45.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:45.488 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379226+0000", 2024-09-14T22:36:45.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0929999999999999e-06 2024-09-14T22:36:45.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.488 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:45.488 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379240+0000", 2024-09-14T22:36:45.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4606999999999999e-05 2024-09-14T22:36:45.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.488 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:45.488 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379272+0000", 2024-09-14T22:36:45.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2073000000000003e-05 2024-09-14T22:36:45.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:45.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379288+0000", 2024-09-14T22:36:45.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6036999999999999e-05 2024-09-14T22:36:45.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:45.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379451+0000", 2024-09-14T22:36:45.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000162795 2024-09-14T22:36:45.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:45.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379702+0000", 2024-09-14T22:36:45.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00025106500000000002 2024-09-14T22:36:45.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:36:45.490 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.394067+0000", 2024-09-14T22:36:45.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014364564999999999 2024-09-14T22:36:45.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:45.490 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.451737+0000", 2024-09-14T22:36:45.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057670185999999998 2024-09-14T22:36:45.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:45.490 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.451823+0000", 2024-09-14T22:36:45.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6075000000000005e-05 2024-09-14T22:36:45.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:45.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.451881+0000", 2024-09-14T22:36:45.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7454999999999998e-05 2024-09-14T22:36:45.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:45.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.451929+0000", 2024-09-14T22:36:45.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8093999999999997e-05 2024-09-14T22:36:45.491 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.491 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:45.491 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.491 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.491 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:45.491 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:45.493 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:36:45.493 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:45.486+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:45.594 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:45.587+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:45.621 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:45.621 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:45.622 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:36:45.622 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:45.635 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:36:45.723 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:45.719+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:45.795 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:45.790+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:45.798 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:45.798 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:45.798 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:36:45.798 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:36:45.798 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:45.812 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:36:45.899 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:45.895+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:45.959 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [2] live_osds: [0, 1, 3, 4, 5, 6, 7] 2024-09-14T22:36:45.959 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:36:45.959 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T22:36:45.959 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T22:36:45.986 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:45.986 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:36:45.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:36:45.986 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:36:45.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.987 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2151 3.a 3:5478be22:::rbd_data.113cde4a8b6b.0000000000000143:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:45.987 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.514710+0000", 2024-09-14T22:36:45.987 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 22.465898764999999, 2024-09-14T22:36:45.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012174763, 2024-09-14T22:36:45.987 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:45.987 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:45.987 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:45.987 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:45.987 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:45.987 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:45.987 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2151 2024-09-14T22:36:45.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.987 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:45.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.988 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:45.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.514710+0000", 2024-09-14T22:36:45.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.988 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:45.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.514710+0000", 2024-09-14T22:36:45.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.988 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:45.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.514712+0000", 2024-09-14T22:36:45.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3649999999999998e-06 2024-09-14T22:36:45.989 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.989 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.989 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:45.989 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.514723+0000", 2024-09-14T22:36:45.989 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0936e-05 2024-09-14T22:36:45.989 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.989 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.989 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:45.989 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.514747+0000", 2024-09-14T22:36:45.989 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3288000000000001e-05 2024-09-14T22:36:45.989 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.989 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.989 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:45.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.514756+0000", 2024-09-14T22:36:45.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6949999999999995e-06 2024-09-14T22:36:45.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.990 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:45.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.514873+0000", 2024-09-14T22:36:45.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011657700000000001 2024-09-14T22:36:45.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.990 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:45.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.515139+0000", 2024-09-14T22:36:45.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026613100000000002 2024-09-14T22:36:45.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.991 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:36:45.991 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.517580+0000", 2024-09-14T22:36:45.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002441365 2024-09-14T22:36:45.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.991 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.991 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:45.991 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.526706+0000", 2024-09-14T22:36:45.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0091256459999999994 2024-09-14T22:36:45.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.991 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.991 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:45.991 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.526762+0000", 2024-09-14T22:36:45.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5489999999999999e-05 2024-09-14T22:36:45.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:45.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.526815+0000", 2024-09-14T22:36:45.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3458000000000002e-05 2024-09-14T22:36:45.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:45.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.526885+0000", 2024-09-14T22:36:45.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9812000000000002e-05 2024-09-14T22:36:45.992 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.992 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:45.992 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.993 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.993 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:5363 3.10 3:0bb60ae6:::rbd_data.113cde4a8b6b.0000000000000094:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:45.993 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.907462+0000", 2024-09-14T22:36:45.993 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 22.073147164000002, 2024-09-14T22:36:45.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0080656949999999995, 2024-09-14T22:36:45.993 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:45.993 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:45.993 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:45.993 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:45.993 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:45.993 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:45.993 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 5363 2024-09-14T22:36:45.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.994 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:45.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:45.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.907462+0000", 2024-09-14T22:36:45.994 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:45.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.907462+0000", 2024-09-14T22:36:45.994 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:45.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:45.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.907465+0000", 2024-09-14T22:36:45.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.393e-06 2024-09-14T22:36:45.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.995 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:45.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.907479+0000", 2024-09-14T22:36:45.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3920000000000001e-05 2024-09-14T22:36:45.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.995 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:45.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.907521+0000", 2024-09-14T22:36:45.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2525999999999999e-05 2024-09-14T22:36:45.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.996 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:45.996 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.907546+0000", 2024-09-14T22:36:45.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4783e-05 2024-09-14T22:36:45.996 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.996 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:45.996 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.907769+0000", 2024-09-14T22:36:45.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022249499999999999 2024-09-14T22:36:45.996 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.996 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op must wait for map", 2024-09-14T22:36:45.996 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.907792+0000", 2024-09-14T22:36:45.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2912999999999999e-05 2024-09-14T22:36:45.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.997 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:45.997 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.913820+0000", 2024-09-14T22:36:45.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.006028828 2024-09-14T22:36:45.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.997 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:45.997 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.913959+0000", 2024-09-14T22:36:45.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013830900000000001 2024-09-14T22:36:45.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.997 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:45.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.914169+0000", 2024-09-14T22:36:45.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000209898 2024-09-14T22:36:45.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:45.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.915324+0000", 2024-09-14T22:36:45.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0011553640000000001 2024-09-14T22:36:45.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:45.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.915381+0000", 2024-09-14T22:36:45.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7086999999999997e-05 2024-09-14T22:36:45.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:45.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.915456+0000", 2024-09-14T22:36:45.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5038999999999998e-05 2024-09-14T22:36:45.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:45.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:45.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.915527+0000", 2024-09-14T22:36:45.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.114e-05 2024-09-14T22:36:45.999 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.999 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:45.999 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:45.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:45.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.000 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:63537 3.10 3:0c425a9a:::rbd_data.113cce440cee.000000000000001d:a [read 655360~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:36:46.000 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:41.993297+0000", 2024-09-14T22:36:46.000 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.987311563, 2024-09-14T22:36:46.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0067916579999999999, 2024-09-14T22:36:46.000 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.000 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.000 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.000 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.000 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.000 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.000 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 63537 2024-09-14T22:36:46.000 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.000 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.001 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993297+0000", 2024-09-14T22:36:46.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.001 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993297+0000", 2024-09-14T22:36:46.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.001 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993298+0000", 2024-09-14T22:36:46.002 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7000000000000003e-07 2024-09-14T22:36:46.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.002 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.002 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.002 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993302+0000", 2024-09-14T22:36:46.002 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2039999999999999e-06 2024-09-14T22:36:46.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.002 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.002 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.002 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993316+0000", 2024-09-14T22:36:46.002 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3609e-05 2024-09-14T22:36:46.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.002 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.002 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993323+0000", 2024-09-14T22:36:46.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4599999999999997e-06 2024-09-14T22:36:46.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.999810+0000", 2024-09-14T22:36:46.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0064868039999999997 2024-09-14T22:36:46.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.999940+0000", 2024-09-14T22:36:46.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000129849 2024-09-14T22:36:46.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.004 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.000089+0000", 2024-09-14T22:36:46.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014886200000000001 2024-09-14T22:36:46.004 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.004 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.004 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.004 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:63538 3.10 3:0c425a9a:::rbd_data.113cce440cee.000000000000001d:a [read 679936~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:36:46.004 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:41.993334+0000", 2024-09-14T22:36:46.004 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.987274846, 2024-09-14T22:36:46.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0068890929999999998, 2024-09-14T22:36:46.005 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.005 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.005 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.005 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.005 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.005 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.005 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 63538 2024-09-14T22:36:46.005 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.005 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.005 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.005 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.005 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993334+0000", 2024-09-14T22:36:46.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.006 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.006 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.006 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993334+0000", 2024-09-14T22:36:46.006 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.006 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.006 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.006 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993335+0000", 2024-09-14T22:36:46.006 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5300000000000004e-07 2024-09-14T22:36:46.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.006 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.006 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.007 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993338+0000", 2024-09-14T22:36:46.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4630000000000001e-06 2024-09-14T22:36:46.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.007 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.007 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993350+0000", 2024-09-14T22:36:46.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2037e-05 2024-09-14T22:36:46.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.007 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.007 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993359+0000", 2024-09-14T22:36:46.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4400000000000005e-06 2024-09-14T22:36:46.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.000106+0000", 2024-09-14T22:36:46.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0067476530000000002 2024-09-14T22:36:46.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.000158+0000", 2024-09-14T22:36:46.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2280000000000001e-05 2024-09-14T22:36:46.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.000223+0000", 2024-09-14T22:36:46.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4566999999999994e-05 2024-09-14T22:36:46.008 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.008 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.009 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.009 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:63553 3.10 3:0f1b8cf8:::rbd_data.113c8642facc.000000000000000a:9 [read 4849664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:36:46.009 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:41.993368+0000", 2024-09-14T22:36:46.009 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.98724049, 2024-09-14T22:36:46.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0071530079999999998, 2024-09-14T22:36:46.009 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.009 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.009 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.009 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.009 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.009 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.010 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 63553 2024-09-14T22:36:46.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.010 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.010 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.010 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993368+0000", 2024-09-14T22:36:46.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.010 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.010 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993368+0000", 2024-09-14T22:36:46.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.011 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.011 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.011 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993369+0000", 2024-09-14T22:36:46.011 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T22:36:46.011 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.011 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.011 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.011 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993373+0000", 2024-09-14T22:36:46.011 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5640000000000001e-06 2024-09-14T22:36:46.011 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.011 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.011 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.011 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993392+0000", 2024-09-14T22:36:46.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9079000000000001e-05 2024-09-14T22:36:46.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993408+0000", 2024-09-14T22:36:46.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5913000000000002e-05 2024-09-14T22:36:46.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.000271+0000", 2024-09-14T22:36:46.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0068634710000000003 2024-09-14T22:36:46.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.000376+0000", 2024-09-14T22:36:46.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000104999 2024-09-14T22:36:46.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.000521+0000", 2024-09-14T22:36:46.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000145322 2024-09-14T22:36:46.013 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.013 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.013 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.014 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:63555 3.10 3:0f1b8cf8:::rbd_data.113c8642facc.000000000000000a:9 [read 4890624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:36:46.014 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:41.993428+0000", 2024-09-14T22:36:46.014 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.987180924, 2024-09-14T22:36:46.014 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0072193999999999999, 2024-09-14T22:36:46.014 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.014 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.014 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.014 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.014 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.014 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.014 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 63555 2024-09-14T22:36:46.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.015 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.015 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.015 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993428+0000", 2024-09-14T22:36:46.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.015 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.015 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.015 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993428+0000", 2024-09-14T22:36:46.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.015 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.015 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.015 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993429+0000", 2024-09-14T22:36:46.016 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.13e-06 2024-09-14T22:36:46.016 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.016 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.016 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.016 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993435+0000", 2024-09-14T22:36:46.016 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7529999999999998e-06 2024-09-14T22:36:46.016 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.016 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.016 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.016 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993455+0000", 2024-09-14T22:36:46.016 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0604000000000001e-05 2024-09-14T22:36:46.016 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.016 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993472+0000", 2024-09-14T22:36:46.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.66e-05 2024-09-14T22:36:46.017 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.000545+0000", 2024-09-14T22:36:46.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0070728969999999999 2024-09-14T22:36:46.017 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.000589+0000", 2024-09-14T22:36:46.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3758999999999999e-05 2024-09-14T22:36:46.017 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.018 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.018 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.000647+0000", 2024-09-14T22:36:46.018 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8656999999999998e-05 2024-09-14T22:36:46.018 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.018 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.018 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.018 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.018 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:63557 3.10 3:0f1b8cf8:::rbd_data.113c8642facc.000000000000000a:9 [read 4931584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:36:46.018 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:41.993494+0000", 2024-09-14T22:36:46.018 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.9871147040000001, 2024-09-14T22:36:46.018 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.007277576, 2024-09-14T22:36:46.019 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.019 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.019 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.019 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.019 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.019 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.019 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 63557 2024-09-14T22:36:46.019 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.019 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.019 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.019 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993494+0000", 2024-09-14T22:36:46.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.019 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.020 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.020 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.020 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993494+0000", 2024-09-14T22:36:46.020 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.020 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.020 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.020 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.020 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993495+0000", 2024-09-14T22:36:46.020 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.083e-06 2024-09-14T22:36:46.020 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.020 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.020 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.020 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993501+0000", 2024-09-14T22:36:46.021 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1240000000000002e-06 2024-09-14T22:36:46.021 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.021 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.021 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.021 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993522+0000", 2024-09-14T22:36:46.021 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1046000000000001e-05 2024-09-14T22:36:46.021 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.021 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.021 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.021 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993533+0000", 2024-09-14T22:36:46.021 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0305999999999999e-05 2024-09-14T22:36:46.021 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.021 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.022 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.000679+0000", 2024-09-14T22:36:46.022 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0071462280000000001 2024-09-14T22:36:46.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.022 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.022 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.000719+0000", 2024-09-14T22:36:46.022 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0201999999999999e-05 2024-09-14T22:36:46.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.022 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.022 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.000772+0000", 2024-09-14T22:36:46.022 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2587000000000003e-05 2024-09-14T22:36:46.022 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.022 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.023 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.023 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:63559 3.10 3:0f1b8cf8:::rbd_data.113c8642facc.000000000000000a:9 [read 4972544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:36:46.023 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:41.993547+0000", 2024-09-14T22:36:46.023 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.9870622510000002, 2024-09-14T22:36:46.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0073356990000000002, 2024-09-14T22:36:46.023 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.023 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.023 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.023 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.023 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.023 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.024 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 63559 2024-09-14T22:36:46.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.024 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.024 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.024 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993547+0000", 2024-09-14T22:36:46.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.024 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.024 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993547+0000", 2024-09-14T22:36:46.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.025 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.025 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993547+0000", 2024-09-14T22:36:46.025 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4899999999999995e-07 2024-09-14T22:36:46.025 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.025 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.025 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.025 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993550+0000", 2024-09-14T22:36:46.025 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2200000000000001e-06 2024-09-14T22:36:46.025 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.025 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.025 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.025 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993562+0000", 2024-09-14T22:36:46.025 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1164e-05 2024-09-14T22:36:46.025 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.026 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.026 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.026 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:41.993568+0000", 2024-09-14T22:36:46.026 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7129999999999999e-06 2024-09-14T22:36:46.026 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.026 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.026 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.026 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.000797+0000", 2024-09-14T22:36:46.026 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0072290130000000003 2024-09-14T22:36:46.026 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.026 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.026 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.026 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.000834+0000", 2024-09-14T22:36:46.027 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6365999999999997e-05 2024-09-14T22:36:46.027 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.027 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.027 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.000882+0000", 2024-09-14T22:36:46.027 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8573999999999999e-05 2024-09-14T22:36:46.027 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.027 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.027 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.027 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.027 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:64489 3.10 3:0eea6554:::rbd_data.113cce440cee.0000000000000002:a [read 1392640~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:36:46.027 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:42.866429+0000", 2024-09-14T22:36:46.027 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.1141795000000001, 2024-09-14T22:36:46.028 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.006992056, 2024-09-14T22:36:46.028 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.028 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.028 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.028 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.028 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.028 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.028 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 64489 2024-09-14T22:36:46.028 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.028 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.028 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.028 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.029 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866429+0000", 2024-09-14T22:36:46.029 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.029 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.029 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.029 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.029 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866429+0000", 2024-09-14T22:36:46.029 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.029 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.029 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.029 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.029 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866430+0000", 2024-09-14T22:36:46.029 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1289999999999999e-06 2024-09-14T22:36:46.029 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.029 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.030 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.030 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866437+0000", 2024-09-14T22:36:46.030 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4679999999999997e-06 2024-09-14T22:36:46.030 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.030 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.030 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.030 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866453+0000", 2024-09-14T22:36:46.030 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6466000000000001e-05 2024-09-14T22:36:46.030 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.030 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.030 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.030 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866465+0000", 2024-09-14T22:36:46.030 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.172e-05 2024-09-14T22:36:46.031 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.031 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.031 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.031 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.873038+0000", 2024-09-14T22:36:46.031 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0065726029999999998 2024-09-14T22:36:46.031 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.031 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.031 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.031 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.873213+0000", 2024-09-14T22:36:46.031 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017484799999999999 2024-09-14T22:36:46.031 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.031 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.031 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.032 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.873421+0000", 2024-09-14T22:36:46.032 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000208822 2024-09-14T22:36:46.032 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.032 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.032 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.032 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.032 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.032 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:64494 3.10 3:0eea6554:::rbd_data.113cce440cee.0000000000000002:a [read 1417216~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:36:46.032 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:42.866481+0000", 2024-09-14T22:36:46.032 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.1141281030000001, 2024-09-14T22:36:46.032 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.007509478, 2024-09-14T22:36:46.032 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.032 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.032 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.033 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.033 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.033 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.033 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 64494 2024-09-14T22:36:46.033 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.033 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.033 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.033 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.033 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866481+0000", 2024-09-14T22:36:46.033 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.033 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.033 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.033 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.034 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866481+0000", 2024-09-14T22:36:46.034 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.034 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.034 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.034 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.034 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866482+0000", 2024-09-14T22:36:46.034 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.274e-06 2024-09-14T22:36:46.034 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.034 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.034 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.034 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866488+0000", 2024-09-14T22:36:46.034 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7370000000000003e-06 2024-09-14T22:36:46.034 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.034 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.035 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.035 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866529+0000", 2024-09-14T22:36:46.035 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0893999999999998e-05 2024-09-14T22:36:46.035 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.035 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.035 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.035 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866543+0000", 2024-09-14T22:36:46.035 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4486e-05 2024-09-14T22:36:46.035 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.035 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.035 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.035 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.873524+0000", 2024-09-14T22:36:46.035 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0069812880000000004 2024-09-14T22:36:46.035 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.036 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.036 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.036 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.873778+0000", 2024-09-14T22:36:46.036 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00025345800000000003 2024-09-14T22:36:46.036 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.036 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.036 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.036 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.873990+0000", 2024-09-14T22:36:46.036 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021234100000000001 2024-09-14T22:36:46.036 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.036 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.036 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.036 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.037 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.037 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:64499 3.10 3:0eea6554:::rbd_data.113cce440cee.0000000000000002:a [read 1441792~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:36:46.037 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:42.866562+0000", 2024-09-14T22:36:46.037 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.114046332, 2024-09-14T22:36:46.037 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0081299249999999997, 2024-09-14T22:36:46.037 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.037 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.037 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.037 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.037 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.037 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.037 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 64499 2024-09-14T22:36:46.037 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.038 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.038 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.038 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.038 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866562+0000", 2024-09-14T22:36:46.038 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.038 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.038 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.038 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.038 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866562+0000", 2024-09-14T22:36:46.038 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.038 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.038 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.038 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.038 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866563+0000", 2024-09-14T22:36:46.039 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0249999999999999e-06 2024-09-14T22:36:46.039 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.039 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.039 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.039 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866570+0000", 2024-09-14T22:36:46.039 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0920000000000003e-06 2024-09-14T22:36:46.039 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.039 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.039 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.039 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866588+0000", 2024-09-14T22:36:46.039 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8448e-05 2024-09-14T22:36:46.039 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.039 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.040 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.040 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866601+0000", 2024-09-14T22:36:46.040 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.342e-05 2024-09-14T22:36:46.040 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.040 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.040 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.040 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.874073+0000", 2024-09-14T22:36:46.040 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0074719670000000004 2024-09-14T22:36:46.040 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.040 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.040 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.040 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.874248+0000", 2024-09-14T22:36:46.040 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000174983 2024-09-14T22:36:46.040 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.040 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.041 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.041 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.874692+0000", 2024-09-14T22:36:46.041 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00044399000000000001 2024-09-14T22:36:46.041 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.041 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.041 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.041 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.041 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.041 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:64504 3.10 3:0eea6554:::rbd_data.113cce440cee.0000000000000002:a [read 1048576~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:36:46.041 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:42.866621+0000", 2024-09-14T22:36:46.041 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.1139878269999999, 2024-09-14T22:36:46.041 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0088726299999999994, 2024-09-14T22:36:46.041 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.041 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.042 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.042 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.042 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.042 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.042 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 64504 2024-09-14T22:36:46.042 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.042 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.042 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.042 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.042 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866621+0000", 2024-09-14T22:36:46.042 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.042 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.042 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.042 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.043 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866621+0000", 2024-09-14T22:36:46.043 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.043 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.043 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.043 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.043 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866622+0000", 2024-09-14T22:36:46.043 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0279999999999999e-06 2024-09-14T22:36:46.043 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.043 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.043 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.043 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866627+0000", 2024-09-14T22:36:46.043 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0799999999999996e-06 2024-09-14T22:36:46.043 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.043 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.044 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.044 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866643+0000", 2024-09-14T22:36:46.044 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6246e-05 2024-09-14T22:36:46.044 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.044 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.044 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.044 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866657+0000", 2024-09-14T22:36:46.044 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.33e-05 2024-09-14T22:36:46.044 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.044 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.044 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.044 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.874793+0000", 2024-09-14T22:36:46.044 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0081367820000000004 2024-09-14T22:36:46.045 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.045 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.045 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.045 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.875006+0000", 2024-09-14T22:36:46.045 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021308899999999999 2024-09-14T22:36:46.045 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.045 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.045 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.045 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.875494+0000", 2024-09-14T22:36:46.045 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00048710499999999999 2024-09-14T22:36:46.045 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.045 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.045 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.046 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.046 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.046 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:64509 3.10 3:0eea6554:::rbd_data.113cce440cee.0000000000000002:a [read 1073152~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:36:46.046 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:42.866685+0000", 2024-09-14T22:36:46.046 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.1139235529999998, 2024-09-14T22:36:46.046 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0092649140000000008, 2024-09-14T22:36:46.046 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.046 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.046 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.046 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.046 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.046 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.046 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 64509 2024-09-14T22:36:46.046 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.046 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.047 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.047 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.047 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866685+0000", 2024-09-14T22:36:46.047 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.047 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.047 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.047 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.047 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866685+0000", 2024-09-14T22:36:46.047 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.047 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.047 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.047 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.047 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866686+0000", 2024-09-14T22:36:46.047 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.088e-06 2024-09-14T22:36:46.047 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.048 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.048 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.048 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866694+0000", 2024-09-14T22:36:46.048 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3830000000000003e-06 2024-09-14T22:36:46.048 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.048 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.048 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.048 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866712+0000", 2024-09-14T22:36:46.048 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7863000000000002e-05 2024-09-14T22:36:46.048 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.048 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.048 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.048 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866725+0000", 2024-09-14T22:36:46.048 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3191e-05 2024-09-14T22:36:46.048 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.049 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.049 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.049 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.875568+0000", 2024-09-14T22:36:46.049 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0088428229999999997 2024-09-14T22:36:46.049 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.049 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.049 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.049 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.875742+0000", 2024-09-14T22:36:46.049 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000174055 2024-09-14T22:36:46.049 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.049 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.049 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.049 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.875950+0000", 2024-09-14T22:36:46.049 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020851100000000001 2024-09-14T22:36:46.049 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.050 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.050 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.050 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.050 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.050 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:64514 3.10 3:0eea6554:::rbd_data.113cce440cee.0000000000000002:a [read 1097728~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:36:46.050 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:42.866743+0000", 2024-09-14T22:36:46.050 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.1138662419999998, 2024-09-14T22:36:46.050 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010083414000000001, 2024-09-14T22:36:46.050 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.050 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.050 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.050 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.051 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.051 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.051 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 64514 2024-09-14T22:36:46.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.051 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866743+0000", 2024-09-14T22:36:46.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866743+0000", 2024-09-14T22:36:46.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.052 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.052 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.052 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866744+0000", 2024-09-14T22:36:46.052 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.054e-06 2024-09-14T22:36:46.052 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.052 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.052 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866750+0000", 2024-09-14T22:36:46.052 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0800000000000002e-06 2024-09-14T22:36:46.052 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866767+0000", 2024-09-14T22:36:46.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7510999999999999e-05 2024-09-14T22:36:46.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866780+0000", 2024-09-14T22:36:46.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.254e-05 2024-09-14T22:36:46.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.876031+0000", 2024-09-14T22:36:46.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0092516040000000001 2024-09-14T22:36:46.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.054 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.054 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.876311+0000", 2024-09-14T22:36:46.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00028006400000000001 2024-09-14T22:36:46.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.054 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.054 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.876826+0000", 2024-09-14T22:36:46.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00051456099999999997 2024-09-14T22:36:46.054 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.054 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.054 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.054 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:64519 3.10 3:0eea6554:::rbd_data.113cce440cee.0000000000000002:a [read 1130496~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:36:46.055 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:42.866797+0000", 2024-09-14T22:36:46.055 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.113811809, 2024-09-14T22:36:46.055 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010484596000000001, 2024-09-14T22:36:46.055 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.055 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.055 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.055 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.055 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.055 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.055 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 64519 2024-09-14T22:36:46.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.055 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866797+0000", 2024-09-14T22:36:46.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.056 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866797+0000", 2024-09-14T22:36:46.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.056 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866798+0000", 2024-09-14T22:36:46.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.78e-07 2024-09-14T22:36:46.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.056 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866804+0000", 2024-09-14T22:36:46.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7679999999999997e-06 2024-09-14T22:36:46.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.057 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.057 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866820+0000", 2024-09-14T22:36:46.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6671e-05 2024-09-14T22:36:46.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.057 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.057 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866834+0000", 2024-09-14T22:36:46.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3129000000000001e-05 2024-09-14T22:36:46.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.057 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.057 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.876901+0000", 2024-09-14T22:36:46.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010067026 2024-09-14T22:36:46.058 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.058 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.058 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.877064+0000", 2024-09-14T22:36:46.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016383899999999999 2024-09-14T22:36:46.058 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.058 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.058 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.877282+0000", 2024-09-14T22:36:46.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000217185 2024-09-14T22:36:46.058 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.058 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.059 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.059 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:64524 3.10 3:0eea6554:::rbd_data.113cce440cee.0000000000000002:a [read 1466368~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:36:46.059 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:42.866890+0000", 2024-09-14T22:36:46.059 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.113718644, 2024-09-14T22:36:46.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01082909, 2024-09-14T22:36:46.059 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.059 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.059 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.059 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.059 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.059 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.059 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 64524 2024-09-14T22:36:46.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.060 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866890+0000", 2024-09-14T22:36:46.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866890+0000", 2024-09-14T22:36:46.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866892+0000", 2024-09-14T22:36:46.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.42e-06 2024-09-14T22:36:46.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866898+0000", 2024-09-14T22:36:46.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9460000000000003e-06 2024-09-14T22:36:46.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866918+0000", 2024-09-14T22:36:46.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0234e-05 2024-09-14T22:36:46.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866932+0000", 2024-09-14T22:36:46.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4609e-05 2024-09-14T22:36:46.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.877345+0000", 2024-09-14T22:36:46.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010412744 2024-09-14T22:36:46.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.877509+0000", 2024-09-14T22:36:46.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016388199999999999 2024-09-14T22:36:46.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.063 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.063 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.063 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.877719+0000", 2024-09-14T22:36:46.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021025500000000001 2024-09-14T22:36:46.063 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.063 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.063 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.063 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.063 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.063 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:64529 3.10 3:0eea6554:::rbd_data.113cce440cee.0000000000000002:a [read 1490944~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:36:46.063 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:42.866951+0000", 2024-09-14T22:36:46.063 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.1136581639999998, 2024-09-14T22:36:46.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01154114, 2024-09-14T22:36:46.063 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.064 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.064 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.064 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.064 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.064 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.064 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 64529 2024-09-14T22:36:46.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.064 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.064 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.064 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866951+0000", 2024-09-14T22:36:46.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866951+0000", 2024-09-14T22:36:46.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866952+0000", 2024-09-14T22:36:46.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7060000000000001e-06 2024-09-14T22:36:46.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866958+0000", 2024-09-14T22:36:46.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3940000000000004e-06 2024-09-14T22:36:46.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866975+0000", 2024-09-14T22:36:46.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7139999999999999e-05 2024-09-14T22:36:46.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.866988+0000", 2024-09-14T22:36:46.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3325e-05 2024-09-14T22:36:46.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.877822+0000", 2024-09-14T22:36:46.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010833799 2024-09-14T22:36:46.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.067 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.067 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.877956+0000", 2024-09-14T22:36:46.067 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013437 2024-09-14T22:36:46.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.067 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.067 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.878492+0000", 2024-09-14T22:36:46.067 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00053540599999999999 2024-09-14T22:36:46.067 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.067 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.067 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.068 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:64534 3.10 3:0eea6554:::rbd_data.113cce440cee.0000000000000002:a [read 1523712~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:36:46.068 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:42.867005+0000", 2024-09-14T22:36:46.068 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.1136039360000001, 2024-09-14T22:36:46.068 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011926878, 2024-09-14T22:36:46.068 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.068 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.068 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.068 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.068 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.068 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.068 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 64534 2024-09-14T22:36:46.068 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.068 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.068 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.068 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.069 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.867005+0000", 2024-09-14T22:36:46.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.069 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.867005+0000", 2024-09-14T22:36:46.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.069 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.867006+0000", 2024-09-14T22:36:46.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0900000000000001e-07 2024-09-14T22:36:46.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.867010+0000", 2024-09-14T22:36:46.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7600000000000002e-06 2024-09-14T22:36:46.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.867030+0000", 2024-09-14T22:36:46.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9408000000000001e-05 2024-09-14T22:36:46.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.867043+0000", 2024-09-14T22:36:46.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3608000000000001e-05 2024-09-14T22:36:46.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.071 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.071 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.878581+0000", 2024-09-14T22:36:46.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011537558999999999 2024-09-14T22:36:46.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.071 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.071 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.878762+0000", 2024-09-14T22:36:46.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018091 2024-09-14T22:36:46.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.071 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.071 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.878932+0000", 2024-09-14T22:36:46.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016992400000000001 2024-09-14T22:36:46.071 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.072 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.072 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.072 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.072 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.072 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:64539 3.10 3:0eea6554:::rbd_data.113cce440cee.0000000000000002:a [read 1548288~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:36:46.072 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:42.867061+0000", 2024-09-14T22:36:46.072 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.1135477909999998, 2024-09-14T22:36:46.072 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012362163000000001, 2024-09-14T22:36:46.072 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.072 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.072 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.072 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.072 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.072 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.072 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 64539 2024-09-14T22:36:46.073 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.073 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.073 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.073 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.867061+0000", 2024-09-14T22:36:46.073 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.073 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.073 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.073 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.867061+0000", 2024-09-14T22:36:46.073 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.073 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.073 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.074 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.867062+0000", 2024-09-14T22:36:46.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8299999999999995e-07 2024-09-14T22:36:46.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.074 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.074 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.867067+0000", 2024-09-14T22:36:46.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1030000000000001e-06 2024-09-14T22:36:46.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.074 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.074 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.867084+0000", 2024-09-14T22:36:46.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6790999999999999e-05 2024-09-14T22:36:46.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.075 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.867096+0000", 2024-09-14T22:36:46.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1972e-05 2024-09-14T22:36:46.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.075 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.075 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.879026+0000", 2024-09-14T22:36:46.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011930488 2024-09-14T22:36:46.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.075 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.075 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.879192+0000", 2024-09-14T22:36:46.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016615400000000001 2024-09-14T22:36:46.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.076 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.076 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.076 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.879423+0000", 2024-09-14T22:36:46.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00023067199999999999 2024-09-14T22:36:46.076 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.076 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.076 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.076 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.076 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.076 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:64543 3.10 3:0eea6554:::rbd_data.113cce440cee.0000000000000002:a [read 1310720~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:36:46.076 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:42.867109+0000", 2024-09-14T22:36:46.076 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.1134995399999998, 2024-09-14T22:36:46.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012812624, 2024-09-14T22:36:46.076 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.077 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.077 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.077 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.077 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.077 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.077 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 64543 2024-09-14T22:36:46.077 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.077 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.077 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.077 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.867109+0000", 2024-09-14T22:36:46.077 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.077 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.078 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.078 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.867109+0000", 2024-09-14T22:36:46.078 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.078 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.078 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.078 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.867111+0000", 2024-09-14T22:36:46.078 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5799999999999999e-06 2024-09-14T22:36:46.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.078 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.078 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.078 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.867116+0000", 2024-09-14T22:36:46.078 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3149999999999999e-06 2024-09-14T22:36:46.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.079 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.079 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.079 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.867134+0000", 2024-09-14T22:36:46.079 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7917000000000001e-05 2024-09-14T22:36:46.079 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.079 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.079 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.079 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.867147+0000", 2024-09-14T22:36:46.079 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3134e-05 2024-09-14T22:36:46.079 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.079 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.079 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.079 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.879507+0000", 2024-09-14T22:36:46.079 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012360236 2024-09-14T22:36:46.079 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.080 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.080 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.080 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.879664+0000", 2024-09-14T22:36:46.080 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015612699999999999 2024-09-14T22:36:46.080 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.080 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.080 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.080 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:42.879922+0000", 2024-09-14T22:36:46.080 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00025831500000000001 2024-09-14T22:36:46.080 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.080 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.080 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.080 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.080 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.081 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:46.082 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:36:46.083 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:46.075+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:46.091 INFO:teuthology.orchestra.run.smithi001.stdout:leaving image image_client.0-clone10 intact 2024-09-14T22:36:46.182 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:46.178+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:46.256 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:46.256 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:46.256 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:36:46.256 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:46.271 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:36:46.284 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:46.278+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:46.298 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:36:46.299 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:46.299 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 46, 2024-09-14T22:36:46.299 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "f07f407a-b17d-42c4-bf73-92dd80114a4f", 2024-09-14T22:36:46.299 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-09-14T22:35:45.983055+0000", 2024-09-14T22:36:46.299 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-09-14T22:36:45.957433+0000", 2024-09-14T22:36:46.299 INFO:teuthology.orchestra.run.smithi001.stdout: "last_up_change": "2024-09-14T22:36:26.755008+0000", 2024-09-14T22:36:46.299 INFO:teuthology.orchestra.run.smithi001.stdout: "last_in_change": "2024-09-14T22:35:48.283160+0000", 2024-09-14T22:36:46.299 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T22:36:46.299 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_num": 5804032, 2024-09-14T22:36:46.299 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_set": [ 2024-09-14T22:36:46.300 INFO:teuthology.orchestra.run.smithi001.stdout: "nodeep-scrub", 2024-09-14T22:36:46.300 INFO:teuthology.orchestra.run.smithi001.stdout: "pglog_hardlimit", 2024-09-14T22:36:46.300 INFO:teuthology.orchestra.run.smithi001.stdout: "purged_snapdirs", 2024-09-14T22:36:46.300 INFO:teuthology.orchestra.run.smithi001.stdout: "recovery_deletes", 2024-09-14T22:36:46.300 INFO:teuthology.orchestra.run.smithi001.stdout: "sortbitwise" 2024-09-14T22:36:46.300 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:36:46.300 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_version": 5, 2024-09-14T22:36:46.300 INFO:teuthology.orchestra.run.smithi001.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T22:36:46.300 INFO:teuthology.orchestra.run.smithi001.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T22:36:46.300 INFO:teuthology.orchestra.run.smithi001.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T22:36:46.300 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_snapshot": "", 2024-09-14T22:36:46.300 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_max": 3, 2024-09-14T22:36:46.301 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd": 8, 2024-09-14T22:36:46.301 INFO:teuthology.orchestra.run.smithi001.stdout: "require_min_compat_client": "luminous", 2024-09-14T22:36:46.301 INFO:teuthology.orchestra.run.smithi001.stdout: "min_compat_client": "jewel", 2024-09-14T22:36:46.301 INFO:teuthology.orchestra.run.smithi001.stdout: "require_osd_release": "squid", 2024-09-14T22:36:46.301 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_crimson": false, 2024-09-14T22:36:46.301 INFO:teuthology.orchestra.run.smithi001.stdout: "pools": [ 2024-09-14T22:36:46.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.301 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 1, 2024-09-14T22:36:46.301 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": ".mgr", 2024-09-14T22:36:46.301 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-14T22:35:58.938784+0000", 2024-09-14T22:36:46.301 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-14T22:36:46.301 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool", 2024-09-14T22:36:46.302 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-14T22:36:46.302 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-14T22:36:46.302 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-14T22:36:46.302 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-14T22:36:46.302 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:36:46.302 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:36:46.302 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:36:46.302 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:36:46.302 INFO:teuthology.orchestra.run.smithi001.stdout: "is_stretch_pool": false, 2024-09-14T22:36:46.302 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-14T22:36:46.302 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:36:46.302 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 1, 2024-09-14T22:36:46.303 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 1, 2024-09-14T22:36:46.303 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 1, 2024-09-14T22:36:46.303 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 1, 2024-09-14T22:36:46.303 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 1, 2024-09-14T22:36:46.303 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-14T22:36:46.303 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-14T22:36:46.303 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-14T22:36:46.303 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-14T22:36:46.303 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-14T22:36:46.303 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-14T22:36:46.303 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-14T22:36:46.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.303 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "18", 2024-09-14T22:36:46.304 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-14T22:36:46.304 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:36:46.304 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:36:46.304 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-14T22:36:46.304 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:36:46.304 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 0, 2024-09-14T22:36:46.304 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 0, 2024-09-14T22:36:46.304 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-14T22:36:46.304 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-14T22:36:46.304 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-14T22:36:46.304 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-14T22:36:46.304 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-14T22:36:46.304 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-14T22:36:46.304 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-14T22:36:46.305 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-14T22:36:46.305 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-14T22:36:46.305 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-14T22:36:46.305 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-14T22:36:46.305 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:36:46.305 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:36:46.305 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:36:46.305 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-14T22:36:46.305 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-14T22:36:46.305 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-14T22:36:46.305 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-14T22:36:46.305 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-14T22:36:46.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.305 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-14T22:36:46.306 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-14T22:36:46.306 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-14T22:36:46.306 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:36:46.306 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:36:46.306 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:36:46.306 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:36:46.306 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-14T22:36:46.306 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-14T22:36:46.306 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-14T22:36:46.306 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-14T22:36:46.306 INFO:teuthology.orchestra.run.smithi001.stdout: "options": { 2024-09-14T22:36:46.306 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_max": 32, 2024-09-14T22:36:46.306 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_min": 1 2024-09-14T22:36:46.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.307 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-14T22:36:46.307 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": {} 2024-09-14T22:36:46.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.307 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-14T22:36:46.307 INFO:teuthology.orchestra.run.smithi001.stdout: "score_type": "Fair distribution", 2024-09-14T22:36:46.307 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 8, 2024-09-14T22:36:46.307 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 8, 2024-09-14T22:36:46.307 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 0.25, 2024-09-14T22:36:46.307 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 2, 2024-09-14T22:36:46.307 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 2, 2024-09-14T22:36:46.307 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 0.87667274475097656, 2024-09-14T22:36:46.307 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T22:36:46.307 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T22:36:46.307 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.308 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 2, 2024-09-14T22:36:46.308 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "rbd", 2024-09-14T22:36:46.308 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-14T22:36:04.304242+0000", 2024-09-14T22:36:46.308 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 8193, 2024-09-14T22:36:46.308 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T22:36:46.308 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-14T22:36:46.308 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-14T22:36:46.308 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-14T22:36:46.308 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-14T22:36:46.308 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:36:46.308 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:36:46.309 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:36:46.309 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:36:46.309 INFO:teuthology.orchestra.run.smithi001.stdout: "is_stretch_pool": false, 2024-09-14T22:36:46.309 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-14T22:36:46.309 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:36:46.309 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 8, 2024-09-14T22:36:46.309 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 8, 2024-09-14T22:36:46.309 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 8, 2024-09-14T22:36:46.309 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 8, 2024-09-14T22:36:46.309 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 8, 2024-09-14T22:36:46.309 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-14T22:36:46.309 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-14T22:36:46.309 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-14T22:36:46.310 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-14T22:36:46.310 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-14T22:36:46.310 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-14T22:36:46.310 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-14T22:36:46.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.310 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "22", 2024-09-14T22:36:46.310 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-14T22:36:46.310 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:36:46.310 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:36:46.310 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-14T22:36:46.310 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:36:46.310 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 2, 2024-09-14T22:36:46.310 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 22, 2024-09-14T22:36:46.311 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-14T22:36:46.311 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-14T22:36:46.311 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-14T22:36:46.311 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-14T22:36:46.311 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-14T22:36:46.311 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-14T22:36:46.311 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-14T22:36:46.311 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-14T22:36:46.311 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-14T22:36:46.311 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-14T22:36:46.311 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-14T22:36:46.311 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:36:46.311 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:36:46.312 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:36:46.312 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-14T22:36:46.312 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-14T22:36:46.312 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-14T22:36:46.312 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-14T22:36:46.312 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-14T22:36:46.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.312 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-14T22:36:46.312 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-14T22:36:46.312 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-14T22:36:46.312 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:36:46.312 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:36:46.312 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:36:46.313 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:36:46.313 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-14T22:36:46.313 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-14T22:36:46.313 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-14T22:36:46.313 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-14T22:36:46.313 INFO:teuthology.orchestra.run.smithi001.stdout: "options": {}, 2024-09-14T22:36:46.313 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-14T22:36:46.313 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd": {} 2024-09-14T22:36:46.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.313 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-14T22:36:46.313 INFO:teuthology.orchestra.run.smithi001.stdout: "score_type": "Fair distribution", 2024-09-14T22:36:46.313 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 2, 2024-09-14T22:36:46.313 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 2, 2024-09-14T22:36:46.313 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 0.75, 2024-09-14T22:36:46.314 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 1.5, 2024-09-14T22:36:46.314 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 1.5, 2024-09-14T22:36:46.314 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 0.87667274475097656, 2024-09-14T22:36:46.314 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T22:36:46.314 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T22:36:46.314 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.314 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 3, 2024-09-14T22:36:46.314 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "pool_client.0", 2024-09-14T22:36:46.314 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-14T22:36:14.374249+0000", 2024-09-14T22:36:46.314 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 8193, 2024-09-14T22:36:46.314 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T22:36:46.314 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-14T22:36:46.315 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-14T22:36:46.315 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-14T22:36:46.315 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-14T22:36:46.315 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:36:46.315 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:36:46.315 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:36:46.315 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:36:46.315 INFO:teuthology.orchestra.run.smithi001.stdout: "is_stretch_pool": false, 2024-09-14T22:36:46.315 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-14T22:36:46.315 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:36:46.315 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 32, 2024-09-14T22:36:46.315 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 32, 2024-09-14T22:36:46.315 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 32, 2024-09-14T22:36:46.315 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 32, 2024-09-14T22:36:46.316 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 32, 2024-09-14T22:36:46.316 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-14T22:36:46.316 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-14T22:36:46.316 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-14T22:36:46.316 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-14T22:36:46.316 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-14T22:36:46.316 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-14T22:36:46.316 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-14T22:36:46.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.316 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "46", 2024-09-14T22:36:46.316 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-14T22:36:46.316 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:36:46.316 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:36:46.317 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-14T22:36:46.317 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:36:46.317 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 13, 2024-09-14T22:36:46.317 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 46, 2024-09-14T22:36:46.317 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-14T22:36:46.317 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-14T22:36:46.317 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-14T22:36:46.317 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-14T22:36:46.317 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-14T22:36:46.317 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-14T22:36:46.317 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-14T22:36:46.317 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-14T22:36:46.317 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-14T22:36:46.318 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-14T22:36:46.318 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-14T22:36:46.318 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:36:46.318 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:36:46.318 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:36:46.318 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-14T22:36:46.318 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-14T22:36:46.318 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-14T22:36:46.318 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-14T22:36:46.318 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-14T22:36:46.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.318 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-14T22:36:46.318 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-14T22:36:46.318 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-14T22:36:46.318 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:36:46.319 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:36:46.319 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:36:46.319 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:36:46.319 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-14T22:36:46.319 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-14T22:36:46.319 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-14T22:36:46.319 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-14T22:36:46.319 INFO:teuthology.orchestra.run.smithi001.stdout: "options": {}, 2024-09-14T22:36:46.319 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-14T22:36:46.319 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd": {} 2024-09-14T22:36:46.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.319 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-14T22:36:46.320 INFO:teuthology.orchestra.run.smithi001.stdout: "score_type": "Fair distribution", 2024-09-14T22:36:46.320 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 1.7400000095367432, 2024-09-14T22:36:46.320 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 1.7400000095367432, 2024-09-14T22:36:46.320 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 0.87999999523162842, 2024-09-14T22:36:46.320 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 1.5299999713897705, 2024-09-14T22:36:46.320 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 1.5299999713897705, 2024-09-14T22:36:46.320 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 0.87667274475097656, 2024-09-14T22:36:46.320 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T22:36:46.320 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T22:36:46.320 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.320 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.320 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:36:46.320 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2024-09-14T22:36:46.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.321 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 0, 2024-09-14T22:36:46.321 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "9831fcc5-9040-4a80-99d4-838225e65139", 2024-09-14T22:36:46.321 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:36:46.321 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:36:46.321 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:36:46.321 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:36:46.321 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:36:46.321 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:36:46.321 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:36:46.321 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 42, 2024-09-14T22:36:46.322 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:36:46.322 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:36:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:36:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6824", 2024-09-14T22:36:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:36:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6825", 2024-09-14T22:36:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:36:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:36:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6826", 2024-09-14T22:36:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:36:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6827", 2024-09-14T22:36:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:36:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:36:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6830", 2024-09-14T22:36:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:36:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:46.326 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6831", 2024-09-14T22:36:46.326 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:36:46.326 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.326 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.326 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:36:46.326 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:46.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.326 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:46.326 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6828", 2024-09-14T22:36:46.326 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:36:46.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.327 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:46.327 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6829", 2024-09-14T22:36:46.327 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:36:46.327 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.327 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.327 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6825/1702929600", 2024-09-14T22:36:46.327 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6827/1702929600", 2024-09-14T22:36:46.327 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6831/1702929600", 2024-09-14T22:36:46.327 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6829/1702929600", 2024-09-14T22:36:46.327 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:36:46.327 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:36:46.327 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:36:46.327 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.328 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 1, 2024-09-14T22:36:46.328 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "7f79abe8-115d-46b2-8677-72fcb0c8ab7d", 2024-09-14T22:36:46.328 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:36:46.328 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:36:46.328 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:36:46.328 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 0.1606292724609375, 2024-09-14T22:36:46.328 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:36:46.328 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:36:46.328 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2024-09-14T22:36:46.328 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 31, 2024-09-14T22:36:46.328 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:36:46.329 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:36:46.329 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:36:46.329 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:46.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.329 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:46.329 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6808", 2024-09-14T22:36:46.329 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:36:46.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.329 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:46.329 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6809", 2024-09-14T22:36:46.329 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:36:46.329 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.330 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.330 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:36:46.330 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:46.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.330 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:46.330 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6810", 2024-09-14T22:36:46.330 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:36:46.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.330 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:46.330 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6811", 2024-09-14T22:36:46.330 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:36:46.330 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.331 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.331 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:36:46.331 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:46.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.331 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:46.331 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6814", 2024-09-14T22:36:46.331 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:36:46.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.331 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:46.331 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6815", 2024-09-14T22:36:46.331 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:36:46.331 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.332 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.332 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:36:46.332 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:46.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.332 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:46.332 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6812", 2024-09-14T22:36:46.332 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:36:46.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.332 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:46.332 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6813", 2024-09-14T22:36:46.332 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:36:46.332 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.333 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.333 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6809/719990653", 2024-09-14T22:36:46.333 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6811/719990653", 2024-09-14T22:36:46.333 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6815/719990653", 2024-09-14T22:36:46.333 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6813/719990653", 2024-09-14T22:36:46.333 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:36:46.333 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:36:46.333 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:36:46.333 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.333 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 2, 2024-09-14T22:36:46.334 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "1f5768b2-4db4-4b2e-8d5e-5206f8bde28d", 2024-09-14T22:36:46.334 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 0, 2024-09-14T22:36:46.334 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:36:46.334 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:36:46.334 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:36:46.334 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:36:46.334 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:36:46.334 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:36:46.334 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 19, 2024-09-14T22:36:46.334 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 31, 2024-09-14T22:36:46.334 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:36:46.334 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:36:46.334 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:46.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.335 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:46.335 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6816", 2024-09-14T22:36:46.335 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2536940607 2024-09-14T22:36:46.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.335 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:46.335 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6817", 2024-09-14T22:36:46.335 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2536940607 2024-09-14T22:36:46.335 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.335 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.335 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:36:46.335 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:46.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.336 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:46.336 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6818", 2024-09-14T22:36:46.336 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2536940607 2024-09-14T22:36:46.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.336 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:46.336 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6819", 2024-09-14T22:36:46.336 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2536940607 2024-09-14T22:36:46.336 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.336 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.336 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:36:46.336 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:46.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.337 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:46.337 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6822", 2024-09-14T22:36:46.337 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2536940607 2024-09-14T22:36:46.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.337 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:46.337 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6823", 2024-09-14T22:36:46.337 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2536940607 2024-09-14T22:36:46.337 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.337 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.337 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:36:46.337 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:46.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.338 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:46.338 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6820", 2024-09-14T22:36:46.338 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2536940607 2024-09-14T22:36:46.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.338 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:46.338 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6821", 2024-09-14T22:36:46.338 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2536940607 2024-09-14T22:36:46.338 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.338 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.338 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6817/2536940607", 2024-09-14T22:36:46.338 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6819/2536940607", 2024-09-14T22:36:46.339 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6823/2536940607", 2024-09-14T22:36:46.339 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6821/2536940607", 2024-09-14T22:36:46.339 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:36:46.339 INFO:teuthology.orchestra.run.smithi001.stdout: "exists" 2024-09-14T22:36:46.339 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.339 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 3, 2024-09-14T22:36:46.339 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "8103f3c7-7899-4fc3-bb02-dd2273120138", 2024-09-14T22:36:46.339 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:36:46.339 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:36:46.339 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:36:46.339 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 0.852752685546875, 2024-09-14T22:36:46.339 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:36:46.340 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:36:46.340 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:36:46.340 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 31, 2024-09-14T22:36:46.340 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:36:46.340 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:36:46.340 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:36:46.340 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:46.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.340 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:46.340 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6800", 2024-09-14T22:36:46.340 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:36:46.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.340 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:46.341 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6801", 2024-09-14T22:36:46.341 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:36:46.341 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.341 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.341 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:36:46.341 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:46.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.341 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:46.341 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6802", 2024-09-14T22:36:46.341 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:36:46.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.342 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:46.342 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6803", 2024-09-14T22:36:46.342 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:36:46.342 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.342 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.342 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:36:46.342 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:46.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.342 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:46.342 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6806", 2024-09-14T22:36:46.342 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:36:46.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.343 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:46.343 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6807", 2024-09-14T22:36:46.343 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:36:46.343 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.343 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.343 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:36:46.343 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:46.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.343 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:46.343 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6804", 2024-09-14T22:36:46.343 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:36:46.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.343 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:46.344 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6805", 2024-09-14T22:36:46.344 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:36:46.344 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.344 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.344 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6801/750403671", 2024-09-14T22:36:46.344 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6803/750403671", 2024-09-14T22:36:46.344 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6807/750403671", 2024-09-14T22:36:46.344 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6805/750403671", 2024-09-14T22:36:46.344 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:36:46.344 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:36:46.344 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:36:46.344 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.345 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 4, 2024-09-14T22:36:46.345 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "23cce5cb-726e-44c2-8edb-c2567a89dfbc", 2024-09-14T22:36:46.345 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:36:46.345 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:36:46.345 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:36:46.345 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:36:46.345 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:36:46.345 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:36:46.345 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:36:46.345 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 31, 2024-09-14T22:36:46.345 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:36:46.345 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:36:46.345 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:36:46.346 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:46.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.346 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:46.346 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6816", 2024-09-14T22:36:46.346 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:36:46.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.346 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:46.346 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6817", 2024-09-14T22:36:46.346 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:36:46.346 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.346 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.347 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:36:46.347 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:46.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.347 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:46.347 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6818", 2024-09-14T22:36:46.347 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:36:46.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.347 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:46.347 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6819", 2024-09-14T22:36:46.347 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:36:46.347 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.347 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.348 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:36:46.348 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:46.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.348 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:46.348 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6822", 2024-09-14T22:36:46.348 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:36:46.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.348 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:46.348 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6823", 2024-09-14T22:36:46.349 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:36:46.349 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.349 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.349 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:36:46.349 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:46.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.349 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:46.349 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6820", 2024-09-14T22:36:46.349 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:36:46.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.349 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:46.349 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6821", 2024-09-14T22:36:46.349 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:36:46.350 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.350 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.350 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.176:6817/1408471829", 2024-09-14T22:36:46.350 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.176:6819/1408471829", 2024-09-14T22:36:46.350 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.176:6823/1408471829", 2024-09-14T22:36:46.350 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.176:6821/1408471829", 2024-09-14T22:36:46.350 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:36:46.350 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:36:46.350 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:36:46.350 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.351 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 5, 2024-09-14T22:36:46.351 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "eb2f187b-d2f1-49ab-ad5e-ea6dc441a28e", 2024-09-14T22:36:46.351 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:36:46.351 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:36:46.351 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:36:46.351 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:36:46.351 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:36:46.351 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:36:46.351 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:36:46.351 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 31, 2024-09-14T22:36:46.351 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:36:46.351 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:36:46.352 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:36:46.352 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:46.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.352 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:46.352 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6805", 2024-09-14T22:36:46.352 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:36:46.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.352 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:46.353 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6808", 2024-09-14T22:36:46.353 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:36:46.353 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.353 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.353 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:36:46.353 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:46.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.353 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:46.353 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6810", 2024-09-14T22:36:46.353 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:36:46.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.354 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:46.354 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6811", 2024-09-14T22:36:46.354 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:36:46.354 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.354 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.354 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:36:46.354 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:46.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.354 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:46.354 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6814", 2024-09-14T22:36:46.354 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:36:46.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.355 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:46.355 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6815", 2024-09-14T22:36:46.355 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:36:46.355 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.355 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.355 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:36:46.355 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:46.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.355 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:46.355 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6812", 2024-09-14T22:36:46.355 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:36:46.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.356 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:46.356 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6813", 2024-09-14T22:36:46.356 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:36:46.356 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.356 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.356 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.176:6808/1945902326", 2024-09-14T22:36:46.356 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.176:6811/1945902326", 2024-09-14T22:36:46.356 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.176:6815/1945902326", 2024-09-14T22:36:46.356 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.176:6813/1945902326", 2024-09-14T22:36:46.356 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:36:46.357 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:36:46.357 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:36:46.357 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.357 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 6, 2024-09-14T22:36:46.357 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "d371aed8-2ad2-49a5-8ece-4b7fd37b4822", 2024-09-14T22:36:46.357 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:36:46.357 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:36:46.357 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:36:46.357 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:36:46.357 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:36:46.357 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:36:46.358 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:36:46.358 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 42, 2024-09-14T22:36:46.358 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:36:46.358 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:36:46.358 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:36:46.358 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:46.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.358 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:46.358 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6800", 2024-09-14T22:36:46.358 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:36:46.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.358 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:46.359 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6801", 2024-09-14T22:36:46.359 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:36:46.359 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.359 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.359 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:36:46.359 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:46.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.359 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:46.359 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6802", 2024-09-14T22:36:46.359 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:36:46.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.360 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:46.360 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6803", 2024-09-14T22:36:46.360 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:36:46.360 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.360 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.360 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:36:46.360 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:46.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.361 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:46.361 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6807", 2024-09-14T22:36:46.361 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:36:46.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.361 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:46.361 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6809", 2024-09-14T22:36:46.361 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:36:46.361 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.361 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.361 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:36:46.361 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:46.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.362 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:46.362 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6804", 2024-09-14T22:36:46.362 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:36:46.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.362 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:46.362 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6806", 2024-09-14T22:36:46.362 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:36:46.362 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.362 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.362 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.176:6801/4116085586", 2024-09-14T22:36:46.363 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.176:6803/4116085586", 2024-09-14T22:36:46.363 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.176:6809/4116085586", 2024-09-14T22:36:46.363 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.176:6806/4116085586", 2024-09-14T22:36:46.363 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:36:46.363 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:36:46.363 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:36:46.363 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.363 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 7, 2024-09-14T22:36:46.363 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "a012c819-8d49-4310-a840-c2752d9314ab", 2024-09-14T22:36:46.363 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:36:46.363 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:36:46.363 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:36:46.364 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:36:46.364 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:36:46.364 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:36:46.364 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:36:46.364 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 27, 2024-09-14T22:36:46.364 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:36:46.364 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:36:46.364 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:36:46.364 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:46.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.364 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:46.364 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6824", 2024-09-14T22:36:46.364 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:36:46.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.365 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:46.365 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6825", 2024-09-14T22:36:46.365 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:36:46.365 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.365 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.365 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:36:46.365 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:46.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.365 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:46.365 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6826", 2024-09-14T22:36:46.365 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:36:46.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.366 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:46.366 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6827", 2024-09-14T22:36:46.366 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:36:46.366 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.366 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.366 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:36:46.366 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:46.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.366 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:46.366 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6830", 2024-09-14T22:36:46.366 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:36:46.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.367 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:46.367 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6831", 2024-09-14T22:36:46.367 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:36:46.367 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.367 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.367 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:36:46.367 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:46.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.367 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:46.367 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6828", 2024-09-14T22:36:46.367 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:36:46.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.368 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:46.368 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6829", 2024-09-14T22:36:46.368 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:36:46.368 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.368 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.368 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.176:6825/877760244", 2024-09-14T22:36:46.368 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.176:6827/877760244", 2024-09-14T22:36:46.368 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.176:6831/877760244", 2024-09-14T22:36:46.368 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.176:6829/877760244", 2024-09-14T22:36:46.368 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:36:46.368 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:36:46.369 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:36:46.369 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.369 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.369 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:36:46.369 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_xinfo": [ 2024-09-14T22:36:46.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.369 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 0, 2024-09-14T22:36:46.369 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:36:46.369 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:36:46.369 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:36:46.369 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:36:46.369 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:36:46.369 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:55.511688+0000", 2024-09-14T22:36:46.370 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:36:46.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.370 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 1, 2024-09-14T22:36:46.370 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:36:46.370 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:36:46.370 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:36:46.370 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:36:46.370 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:36:46.370 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.742414+0000", 2024-09-14T22:36:46.370 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:36:46.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.370 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 2, 2024-09-14T22:36:46.371 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:36:46.371 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:36:46.371 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:36:46.371 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:36:46.371 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:36:46.371 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.871355+0000", 2024-09-14T22:36:46.371 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 31 2024-09-14T22:36:46.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.371 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 3, 2024-09-14T22:36:46.371 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:36:46.371 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:36:46.371 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:36:46.371 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:36:46.372 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:36:46.372 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.504553+0000", 2024-09-14T22:36:46.372 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:36:46.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.372 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 4, 2024-09-14T22:36:46.372 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:36:46.372 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:36:46.372 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:36:46.372 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:36:46.372 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:36:46.372 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.589640+0000", 2024-09-14T22:36:46.372 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:36:46.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.373 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 5, 2024-09-14T22:36:46.373 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:36:46.373 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:36:46.373 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:36:46.373 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:36:46.373 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:36:46.373 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.518845+0000", 2024-09-14T22:36:46.373 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:36:46.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.373 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 6, 2024-09-14T22:36:46.373 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:36:46.373 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:36:46.374 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:36:46.374 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:36:46.374 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:36:46.374 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.731632+0000", 2024-09-14T22:36:46.374 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:36:46.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.374 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 7, 2024-09-14T22:36:46.374 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:36:46.374 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:36:46.374 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:36:46.374 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:36:46.374 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:36:46.374 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.952634+0000", 2024-09-14T22:36:46.374 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:36:46.375 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.375 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:36:46.375 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap": [], 2024-09-14T22:36:46.375 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap_items": [], 2024-09-14T22:36:46.375 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap_primaries": [], 2024-09-14T22:36:46.375 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_temp": [], 2024-09-14T22:36:46.375 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_temp": [], 2024-09-14T22:36:46.375 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklist": {}, 2024-09-14T22:36:46.375 INFO:teuthology.orchestra.run.smithi001.stdout: "range_blocklist": {}, 2024-09-14T22:36:46.375 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profiles": { 2024-09-14T22:36:46.375 INFO:teuthology.orchestra.run.smithi001.stdout: "default": { 2024-09-14T22:36:46.375 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-failure-domain": "osd", 2024-09-14T22:36:46.375 INFO:teuthology.orchestra.run.smithi001.stdout: "k": "2", 2024-09-14T22:36:46.375 INFO:teuthology.orchestra.run.smithi001.stdout: "m": "1", 2024-09-14T22:36:46.376 INFO:teuthology.orchestra.run.smithi001.stdout: "plugin": "jerasure", 2024-09-14T22:36:46.376 INFO:teuthology.orchestra.run.smithi001.stdout: "technique": "reed_sol_van" 2024-09-14T22:36:46.376 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.376 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps_queue": [], 2024-09-14T22:36:46.376 INFO:teuthology.orchestra.run.smithi001.stdout: "new_removed_snaps": [], 2024-09-14T22:36:46.376 INFO:teuthology.orchestra.run.smithi001.stdout: "new_purged_snaps": [], 2024-09-14T22:36:46.376 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_node_flags": {}, 2024-09-14T22:36:46.376 INFO:teuthology.orchestra.run.smithi001.stdout: "device_class_flags": {}, 2024-09-14T22:36:46.376 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": { 2024-09-14T22:36:46.376 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode_enabled": false, 2024-09-14T22:36:46.376 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_bucket_count": 0, 2024-09-14T22:36:46.376 INFO:teuthology.orchestra.run.smithi001.stdout: "degraded_stretch_mode": 0, 2024-09-14T22:36:46.376 INFO:teuthology.orchestra.run.smithi001.stdout: "recovering_stretch_mode": 0, 2024-09-14T22:36:46.377 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode_bucket": 0 2024-09-14T22:36:46.377 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.377 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:46.377 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-14T22:36:46.379 INFO:tasks.thrashosds.thrasher:j is {'epoch': 46, 'fsid': 'f07f407a-b17d-42c4-bf73-92dd80114a4f', 'created': '2024-09-14T22:35:45.983055+0000', 'modified': '2024-09-14T22:36:45.957433+0000', 'last_up_change': '2024-09-14T22:36:26.755008+0000', 'last_in_change': '2024-09-14T22:35:48.283160+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T22:35:58.938784+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8766727447509766, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T22:36:04.304242+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 0.75, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.8766727447509766, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T22:36:14.374249+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 13, 'snap_epoch': 46, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7400000095367432, 'score_stable': 1.7400000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 0.8766727447509766, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': '9831fcc5-9040-4a80-99d4-838225e65139', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 42, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6824', 'nonce': 1702929600}, {'type': 'v1', 'addr': '172.21.15.1:6825', 'nonce': 1702929600}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6826', 'nonce': 1702929600}, {'type': 'v1', 'addr': '172.21.15.1:6827', 'nonce': 1702929600}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6830', 'nonce': 1702929600}, {'type': 'v1', 'addr': '172.21.15.1:6831', 'nonce': 1702929600}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6828', 'nonce': 1702929600}, {'type': 'v1', 'addr': '172.21.15.1:6829', 'nonce': 1702929600}]}, 'public_addr': '172.21.15.1:6825/1702929600', 'cluster_addr': '172.21.15.1:6827/1702929600', 'heartbeat_back_addr': '172.21.15.1:6831/1702929600', 'heartbeat_front_addr': '172.21.15.1:6829/1702929600', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '7f79abe8-115d-46b2-8677-72fcb0c8ab7d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.1606292724609375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6808', 'nonce': 719990653}, {'type': 'v1', 'addr': '172.21.15.1:6809', 'nonce': 719990653}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6810', 'nonce': 719990653}, {'type': 'v1', 'addr': '172.21.15.1:6811', 'nonce': 719990653}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6814', 'nonce': 719990653}, {'type': 'v1', 'addr': '172.21.15.1:6815', 'nonce': 719990653}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6812', 'nonce': 719990653}, {'type': 'v1', 'addr': '172.21.15.1:6813', 'nonce': 719990653}]}, 'public_addr': '172.21.15.1:6809/719990653', 'cluster_addr': '172.21.15.1:6811/719990653', 'heartbeat_back_addr': '172.21.15.1:6815/719990653', 'heartbeat_front_addr': '172.21.15.1:6813/719990653', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '1f5768b2-4db4-4b2e-8d5e-5206f8bde28d', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 19, 'down_at': 31, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6816', 'nonce': 2536940607}, {'type': 'v1', 'addr': '172.21.15.1:6817', 'nonce': 2536940607}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6818', 'nonce': 2536940607}, {'type': 'v1', 'addr': '172.21.15.1:6819', 'nonce': 2536940607}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6822', 'nonce': 2536940607}, {'type': 'v1', 'addr': '172.21.15.1:6823', 'nonce': 2536940607}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6820', 'nonce': 2536940607}, {'type': 'v1', 'addr': '172.21.15.1:6821', 'nonce': 2536940607}]}, 'public_addr': '172.21.15.1:6817/2536940607', 'cluster_addr': '172.21.15.1:6819/2536940607', 'heartbeat_back_addr': '172.21.15.1:6823/2536940607', 'heartbeat_front_addr': '172.21.15.1:6821/2536940607', 'state': ['exists']}, {'osd': 3, 'uuid': '8103f3c7-7899-4fc3-bb02-dd2273120138', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.852752685546875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6800', 'nonce': 750403671}, {'type': 'v1', 'addr': '172.21.15.1:6801', 'nonce': 750403671}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6802', 'nonce': 750403671}, {'type': 'v1', 'addr': '172.21.15.1:6803', 'nonce': 750403671}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6806', 'nonce': 750403671}, {'type': 'v1', 'addr': '172.21.15.1:6807', 'nonce': 750403671}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6804', 'nonce': 750403671}, {'type': 'v1', 'addr': '172.21.15.1:6805', 'nonce': 750403671}]}, 'public_addr': '172.21.15.1:6801/750403671', 'cluster_addr': '172.21.15.1:6803/750403671', 'heartbeat_back_addr': '172.21.15.1:6807/750403671', 'heartbeat_front_addr': '172.21.15.1:6805/750403671', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '23cce5cb-726e-44c2-8edb-c2567a89dfbc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6816', 'nonce': 1408471829}, {'type': 'v1', 'addr': '172.21.15.176:6817', 'nonce': 1408471829}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6818', 'nonce': 1408471829}, {'type': 'v1', 'addr': '172.21.15.176:6819', 'nonce': 1408471829}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6822', 'nonce': 1408471829}, {'type': 'v1', 'addr': '172.21.15.176:6823', 'nonce': 1408471829}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6820', 'nonce': 1408471829}, {'type': 'v1', 'addr': '172.21.15.176:6821', 'nonce': 1408471829}]}, 'public_addr': '172.21.15.176:6817/1408471829', 'cluster_addr': '172.21.15.176:6819/1408471829', 'heartbeat_back_addr': '172.21.15.176:6823/1408471829', 'heartbeat_front_addr': '172.21.15.176:6821/1408471829', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'eb2f187b-d2f1-49ab-ad5e-ea6dc441a28e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6805', 'nonce': 1945902326}, {'type': 'v1', 'addr': '172.21.15.176:6808', 'nonce': 1945902326}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6810', 'nonce': 1945902326}, {'type': 'v1', 'addr': '172.21.15.176:6811', 'nonce': 1945902326}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6814', 'nonce': 1945902326}, {'type': 'v1', 'addr': '172.21.15.176:6815', 'nonce': 1945902326}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6812', 'nonce': 1945902326}, {'type': 'v1', 'addr': '172.21.15.176:6813', 'nonce': 1945902326}]}, 'public_addr': '172.21.15.176:6808/1945902326', 'cluster_addr': '172.21.15.176:6811/1945902326', 'heartbeat_back_addr': '172.21.15.176:6815/1945902326', 'heartbeat_front_addr': '172.21.15.176:6813/1945902326', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'd371aed8-2ad2-49a5-8ece-4b7fd37b4822', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 42, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6800', 'nonce': 4116085586}, {'type': 'v1', 'addr': '172.21.15.176:6801', 'nonce': 4116085586}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6802', 'nonce': 4116085586}, {'type': 'v1', 'addr': '172.21.15.176:6803', 'nonce': 4116085586}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6807', 'nonce': 4116085586}, {'type': 'v1', 'addr': '172.21.15.176:6809', 'nonce': 4116085586}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6804', 'nonce': 4116085586}, {'type': 'v1', 'addr': '172.21.15.176:6806', 'nonce': 4116085586}]}, 'public_addr': '172.21.15.176:6801/4116085586', 'cluster_addr': '172.21.15.176:6803/4116085586', 'heartbeat_back_addr': '172.21.15.176:6809/4116085586', 'heartbeat_front_addr': '172.21.15.176:6806/4116085586', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'a012c819-8d49-4310-a840-c2752d9314ab', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6824', 'nonce': 877760244}, {'type': 'v1', 'addr': '172.21.15.176:6825', 'nonce': 877760244}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6826', 'nonce': 877760244}, {'type': 'v1', 'addr': '172.21.15.176:6827', 'nonce': 877760244}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6830', 'nonce': 877760244}, {'type': 'v1', 'addr': '172.21.15.176:6831', 'nonce': 877760244}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6828', 'nonce': 877760244}, {'type': 'v1', 'addr': '172.21.15.176:6829', 'nonce': 877760244}]}, 'public_addr': '172.21.15.176:6825/877760244', 'cluster_addr': '172.21.15.176:6827/877760244', 'heartbeat_back_addr': '172.21.15.176:6831/877760244', 'heartbeat_front_addr': '172.21.15.176:6829/877760244', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:55.511688+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.742414+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.871355+0000', 'dead_epoch': 31}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.504553+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.589640+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.518845+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.731632+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.952634+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T22:36:46.379 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:36:46.384 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:46.379+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:46.389 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:46.389 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:46.389 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:36:46.389 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:36:46.389 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:46.407 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:36:46.485 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:46.478+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:46.574 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:46.574 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:36:46.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:36:46.574 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:36:46.574 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.574 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:36:46.574 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:36:46.574 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.562734528, 2024-09-14T22:36:46.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.398029152, 2024-09-14T22:36:46.574 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.574 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.575 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:46.575 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.575 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:36:46.575 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:36:46.575 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 1 2024-09-14T22:36:46.575 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.575 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.575 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.575 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.575 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:36:46.575 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.575 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.576 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.576 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.576 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:36:46.576 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.576 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.576 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.576 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.576 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006183+0000", 2024-09-14T22:36:46.576 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9059999999999998e-06 2024-09-14T22:36:46.576 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.576 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.576 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.577 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006198+0000", 2024-09-14T22:36:46.577 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.523e-05 2024-09-14T22:36:46.577 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.577 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.577 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.577 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006219+0000", 2024-09-14T22:36:46.577 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1342999999999999e-05 2024-09-14T22:36:46.577 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.577 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.577 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.577 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006240+0000", 2024-09-14T22:36:46.577 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1186999999999999e-05 2024-09-14T22:36:46.577 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.577 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.578 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.578 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006363+0000", 2024-09-14T22:36:46.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012258200000000001 2024-09-14T22:36:46.578 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.578 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:36:46.578 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006392+0000", 2024-09-14T22:36:46.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9607e-05 2024-09-14T22:36:46.578 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.578 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.578 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401084+0000", 2024-09-14T22:36:46.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.39469142699999998 2024-09-14T22:36:46.578 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.579 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.579 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401394+0000", 2024-09-14T22:36:46.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030987799999999998 2024-09-14T22:36:46.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.579 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:46.579 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401860+0000", 2024-09-14T22:36:46.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00046654299999999998 2024-09-14T22:36:46.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.580 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:46.580 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.403386+0000", 2024-09-14T22:36:46.580 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001525322 2024-09-14T22:36:46.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.580 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:46.580 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404042+0000", 2024-09-14T22:36:46.580 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065676099999999996 2024-09-14T22:36:46.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.580 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:46.580 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404136+0000", 2024-09-14T22:36:46.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4032999999999999e-05 2024-09-14T22:36:46.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.581 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404209+0000", 2024-09-14T22:36:46.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2333e-05 2024-09-14T22:36:46.581 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.581 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.581 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.581 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2079 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11401216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:46.582 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450429+0000", 2024-09-14T22:36:46.582 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.11848474, 2024-09-14T22:36:46.582 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055279107000000001, 2024-09-14T22:36:46.582 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.582 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.582 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.582 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.582 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.582 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.582 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2079 2024-09-14T22:36:46.582 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.582 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.582 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.582 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450429+0000", 2024-09-14T22:36:46.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.583 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.583 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450429+0000", 2024-09-14T22:36:46.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.583 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.583 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450431+0000", 2024-09-14T22:36:46.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1039999999999999e-06 2024-09-14T22:36:46.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450435+0000", 2024-09-14T22:36:46.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9300000000000002e-06 2024-09-14T22:36:46.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450455+0000", 2024-09-14T22:36:46.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9242999999999999e-05 2024-09-14T22:36:46.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450470+0000", 2024-09-14T22:36:46.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5523e-05 2024-09-14T22:36:46.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.585 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.505413+0000", 2024-09-14T22:36:46.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054943193000000001 2024-09-14T22:36:46.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.585 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.505538+0000", 2024-09-14T22:36:46.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012467199999999999 2024-09-14T22:36:46.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.586 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.505709+0000", 2024-09-14T22:36:46.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017044200000000001 2024-09-14T22:36:46.586 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.586 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.586 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.586 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2080 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11409408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:46.586 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450489+0000", 2024-09-14T22:36:46.587 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.118424993000001, 2024-09-14T22:36:46.587 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055572178, 2024-09-14T22:36:46.587 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.587 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.587 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.587 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.587 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.587 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.587 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2080 2024-09-14T22:36:46.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.587 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.587 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.587 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450489+0000", 2024-09-14T22:36:46.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450489+0000", 2024-09-14T22:36:46.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450490+0000", 2024-09-14T22:36:46.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.023e-06 2024-09-14T22:36:46.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450495+0000", 2024-09-14T22:36:46.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1399999999999999e-06 2024-09-14T22:36:46.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450512+0000", 2024-09-14T22:36:46.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6892999999999999e-05 2024-09-14T22:36:46.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450525+0000", 2024-09-14T22:36:46.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2877000000000001e-05 2024-09-14T22:36:46.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.590 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.505787+0000", 2024-09-14T22:36:46.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055261432999999999 2024-09-14T22:36:46.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.590 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.505899+0000", 2024-09-14T22:36:46.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112456 2024-09-14T22:36:46.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.590 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506061+0000", 2024-09-14T22:36:46.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000162356 2024-09-14T22:36:46.591 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.591 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.591 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.591 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2081 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11417600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:46.591 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450544+0000", 2024-09-14T22:36:46.591 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.118369674, 2024-09-14T22:36:46.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.05598185, 2024-09-14T22:36:46.591 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.591 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.591 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.591 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.591 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.592 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.592 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2081 2024-09-14T22:36:46.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.592 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.592 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.592 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450544+0000", 2024-09-14T22:36:46.592 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.592 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450544+0000", 2024-09-14T22:36:46.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.593 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450546+0000", 2024-09-14T22:36:46.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5370000000000001e-06 2024-09-14T22:36:46.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.593 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450553+0000", 2024-09-14T22:36:46.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8630000000000002e-06 2024-09-14T22:36:46.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450571+0000", 2024-09-14T22:36:46.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7657000000000001e-05 2024-09-14T22:36:46.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450585+0000", 2024-09-14T22:36:46.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4056000000000001e-05 2024-09-14T22:36:46.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506185+0000", 2024-09-14T22:36:46.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055599920999999997 2024-09-14T22:36:46.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.595 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.595 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506334+0000", 2024-09-14T22:36:46.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014919900000000001 2024-09-14T22:36:46.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.595 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.595 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506526+0000", 2024-09-14T22:36:46.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019261699999999999 2024-09-14T22:36:46.596 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.596 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.596 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.596 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.596 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.596 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2082 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11425792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:46.596 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450602+0000", 2024-09-14T22:36:46.596 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.118311946999999, 2024-09-14T22:36:46.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056238654999999999, 2024-09-14T22:36:46.596 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.596 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.597 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.597 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.597 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.597 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.597 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2082 2024-09-14T22:36:46.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.597 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.597 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.597 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450602+0000", 2024-09-14T22:36:46.597 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450602+0000", 2024-09-14T22:36:46.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450603+0000", 2024-09-14T22:36:46.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1060000000000001e-06 2024-09-14T22:36:46.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450608+0000", 2024-09-14T22:36:46.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1499999999999998e-06 2024-09-14T22:36:46.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450626+0000", 2024-09-14T22:36:46.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7484000000000001e-05 2024-09-14T22:36:46.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450639+0000", 2024-09-14T22:36:46.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3485999999999999e-05 2024-09-14T22:36:46.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506599+0000", 2024-09-14T22:36:46.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055959395000000002 2024-09-14T22:36:46.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.600 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506706+0000", 2024-09-14T22:36:46.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010703499999999999 2024-09-14T22:36:46.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.600 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506841+0000", 2024-09-14T22:36:46.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013499900000000001 2024-09-14T22:36:46.601 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.601 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.601 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.601 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2083 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11433984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:46.601 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450658+0000", 2024-09-14T22:36:46.601 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.118256520999999, 2024-09-14T22:36:46.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056459267, 2024-09-14T22:36:46.601 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.601 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.601 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.601 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.602 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.602 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.602 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2083 2024-09-14T22:36:46.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.602 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.602 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450658+0000", 2024-09-14T22:36:46.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.602 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450658+0000", 2024-09-14T22:36:46.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450659+0000", 2024-09-14T22:36:46.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T22:36:46.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450664+0000", 2024-09-14T22:36:46.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4700000000000001e-06 2024-09-14T22:36:46.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450684+0000", 2024-09-14T22:36:46.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.916e-05 2024-09-14T22:36:46.604 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.604 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.604 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450698+0000", 2024-09-14T22:36:46.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4100000000000001e-05 2024-09-14T22:36:46.604 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.604 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.604 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506881+0000", 2024-09-14T22:36:46.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056183250999999997 2024-09-14T22:36:46.604 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.605 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.605 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506982+0000", 2024-09-14T22:36:46.605 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010090900000000001 2024-09-14T22:36:46.605 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.605 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.605 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.605 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507117+0000", 2024-09-14T22:36:46.605 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013510400000000001 2024-09-14T22:36:46.605 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.605 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.605 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.605 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.606 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2084 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11442176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:46.606 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450716+0000", 2024-09-14T22:36:46.606 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.118198662000001, 2024-09-14T22:36:46.606 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056802328999999999, 2024-09-14T22:36:46.606 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.606 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.606 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.606 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.606 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.606 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.606 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2084 2024-09-14T22:36:46.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.606 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450716+0000", 2024-09-14T22:36:46.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450716+0000", 2024-09-14T22:36:46.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450717+0000", 2024-09-14T22:36:46.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.08e-06 2024-09-14T22:36:46.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.608 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450722+0000", 2024-09-14T22:36:46.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5099999999999998e-06 2024-09-14T22:36:46.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.608 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450740+0000", 2024-09-14T22:36:46.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7836999999999999e-05 2024-09-14T22:36:46.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.608 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450757+0000", 2024-09-14T22:36:46.609 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7513e-05 2024-09-14T22:36:46.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.609 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.609 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.609 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507224+0000", 2024-09-14T22:36:46.609 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056466954 2024-09-14T22:36:46.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.609 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.609 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.609 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507355+0000", 2024-09-14T22:36:46.609 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000130207 2024-09-14T22:36:46.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.609 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.610 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.610 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507518+0000", 2024-09-14T22:36:46.610 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016322799999999999 2024-09-14T22:36:46.610 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.610 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.610 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.610 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.610 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.610 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2085 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11450368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:46.610 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450774+0000", 2024-09-14T22:36:46.610 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.118139858999999, 2024-09-14T22:36:46.610 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057099952000000002, 2024-09-14T22:36:46.610 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.611 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.611 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.611 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.611 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.611 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.611 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2085 2024-09-14T22:36:46.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.611 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.611 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.611 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450774+0000", 2024-09-14T22:36:46.611 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.612 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450774+0000", 2024-09-14T22:36:46.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.612 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450775+0000", 2024-09-14T22:36:46.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1030000000000001e-06 2024-09-14T22:36:46.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450781+0000", 2024-09-14T22:36:46.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.22e-06 2024-09-14T22:36:46.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.613 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450798+0000", 2024-09-14T22:36:46.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7350000000000002e-05 2024-09-14T22:36:46.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.613 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450812+0000", 2024-09-14T22:36:46.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.358e-05 2024-09-14T22:36:46.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507576+0000", 2024-09-14T22:36:46.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056764468999999998 2024-09-14T22:36:46.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.614 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507714+0000", 2024-09-14T22:36:46.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013826800000000001 2024-09-14T22:36:46.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.614 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.615 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507874+0000", 2024-09-14T22:36:46.615 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015996200000000001 2024-09-14T22:36:46.615 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.615 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.615 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.615 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.615 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2086 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11458560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:46.615 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450830+0000", 2024-09-14T22:36:46.615 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.118084569000001, 2024-09-14T22:36:46.616 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057480495999999999, 2024-09-14T22:36:46.616 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.616 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.616 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.616 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.616 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.616 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.616 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2086 2024-09-14T22:36:46.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.616 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.616 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.616 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.617 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450830+0000", 2024-09-14T22:36:46.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.617 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.617 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450830+0000", 2024-09-14T22:36:46.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.617 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.617 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450831+0000", 2024-09-14T22:36:46.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0860000000000001e-06 2024-09-14T22:36:46.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.618 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.618 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450836+0000", 2024-09-14T22:36:46.618 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4999999999999999e-06 2024-09-14T22:36:46.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.618 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.618 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450853+0000", 2024-09-14T22:36:46.618 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7184e-05 2024-09-14T22:36:46.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.618 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.618 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450869+0000", 2024-09-14T22:36:46.618 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5965999999999999e-05 2024-09-14T22:36:46.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507972+0000", 2024-09-14T22:36:46.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057103095999999999 2024-09-14T22:36:46.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508119+0000", 2024-09-14T22:36:46.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014651200000000001 2024-09-14T22:36:46.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508310+0000", 2024-09-14T22:36:46.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019115199999999999 2024-09-14T22:36:46.619 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.619 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.620 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.620 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2087 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11466752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:46.620 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450888+0000", 2024-09-14T22:36:46.620 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.118026523000001, 2024-09-14T22:36:46.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057792806000000002, 2024-09-14T22:36:46.620 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.620 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.620 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.620 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.620 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.620 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.620 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2087 2024-09-14T22:36:46.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.621 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.621 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.621 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450888+0000", 2024-09-14T22:36:46.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.621 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.621 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450888+0000", 2024-09-14T22:36:46.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450889+0000", 2024-09-14T22:36:46.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4270000000000001e-06 2024-09-14T22:36:46.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450895+0000", 2024-09-14T22:36:46.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4700000000000001e-06 2024-09-14T22:36:46.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450913+0000", 2024-09-14T22:36:46.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7958999999999999e-05 2024-09-14T22:36:46.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.623 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.623 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450926+0000", 2024-09-14T22:36:46.623 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3842999999999999e-05 2024-09-14T22:36:46.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.623 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.623 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508394+0000", 2024-09-14T22:36:46.623 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057467795000000002 2024-09-14T22:36:46.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.623 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.624 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508516+0000", 2024-09-14T22:36:46.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000121822 2024-09-14T22:36:46.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.624 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.624 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.624 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508680+0000", 2024-09-14T22:36:46.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016448999999999999 2024-09-14T22:36:46.624 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.624 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.624 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.625 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.625 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2088 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11474944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:46.625 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450943+0000", 2024-09-14T22:36:46.625 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.11797091, 2024-09-14T22:36:46.625 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058110368000000003, 2024-09-14T22:36:46.625 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.625 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.625 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.625 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.625 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.625 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.625 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2088 2024-09-14T22:36:46.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.626 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.626 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450943+0000", 2024-09-14T22:36:46.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.626 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450943+0000", 2024-09-14T22:36:46.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.627 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.627 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450944+0000", 2024-09-14T22:36:46.627 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2100000000000001e-06 2024-09-14T22:36:46.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.627 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.627 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450950+0000", 2024-09-14T22:36:46.627 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3329999999999996e-06 2024-09-14T22:36:46.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.627 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.627 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450968+0000", 2024-09-14T22:36:46.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7802999999999999e-05 2024-09-14T22:36:46.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.628 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450981+0000", 2024-09-14T22:36:46.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.349e-05 2024-09-14T22:36:46.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.628 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508752+0000", 2024-09-14T22:36:46.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057770957999999997 2024-09-14T22:36:46.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.629 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.629 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508884+0000", 2024-09-14T22:36:46.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013186900000000001 2024-09-14T22:36:46.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.629 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.629 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.629 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509054+0000", 2024-09-14T22:36:46.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000169705 2024-09-14T22:36:46.629 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.629 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.629 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.629 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.630 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2089 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11483136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:46.630 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450998+0000", 2024-09-14T22:36:46.630 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.117916016999999, 2024-09-14T22:36:46.630 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058422566000000002, 2024-09-14T22:36:46.630 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.630 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.630 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.630 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.630 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.630 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.630 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2089 2024-09-14T22:36:46.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.630 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.630 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450998+0000", 2024-09-14T22:36:46.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450998+0000", 2024-09-14T22:36:46.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450999+0000", 2024-09-14T22:36:46.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0100000000000001e-06 2024-09-14T22:36:46.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.632 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.632 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451014+0000", 2024-09-14T22:36:46.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4486e-05 2024-09-14T22:36:46.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.632 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.632 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451032+0000", 2024-09-14T22:36:46.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7893e-05 2024-09-14T22:36:46.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.633 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.633 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451046+0000", 2024-09-14T22:36:46.633 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4783999999999999e-05 2024-09-14T22:36:46.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.633 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.633 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509131+0000", 2024-09-14T22:36:46.633 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058085070000000003 2024-09-14T22:36:46.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.633 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.633 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509253+0000", 2024-09-14T22:36:46.634 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012173299999999999 2024-09-14T22:36:46.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.634 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.634 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.634 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509421+0000", 2024-09-14T22:36:46.634 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016759000000000001 2024-09-14T22:36:46.634 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.634 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.634 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.634 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.634 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2090 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11491328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:46.634 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.451064+0000", 2024-09-14T22:36:46.635 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.117850240999999, 2024-09-14T22:36:46.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058723076999999999, 2024-09-14T22:36:46.635 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.635 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.635 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.635 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.635 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.635 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.635 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2090 2024-09-14T22:36:46.635 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.635 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.635 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.635 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.635 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451064+0000", 2024-09-14T22:36:46.636 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.636 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.636 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.636 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451064+0000", 2024-09-14T22:36:46.636 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.636 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.636 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.636 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451065+0000", 2024-09-14T22:36:46.636 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2160000000000001e-06 2024-09-14T22:36:46.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.636 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.637 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.637 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451070+0000", 2024-09-14T22:36:46.637 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1340000000000003e-06 2024-09-14T22:36:46.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.637 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.637 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.637 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451088+0000", 2024-09-14T22:36:46.637 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7476e-05 2024-09-14T22:36:46.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.637 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.637 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.637 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451101+0000", 2024-09-14T22:36:46.637 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3723999999999999e-05 2024-09-14T22:36:46.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509492+0000", 2024-09-14T22:36:46.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058390899000000003 2024-09-14T22:36:46.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509618+0000", 2024-09-14T22:36:46.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012593899999999999 2024-09-14T22:36:46.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509787+0000", 2024-09-14T22:36:46.639 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000168689 2024-09-14T22:36:46.639 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.639 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.639 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.639 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.639 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2091 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11499520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:46.639 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.451119+0000", 2024-09-14T22:36:46.639 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.117795187999999, 2024-09-14T22:36:46.639 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059045274000000002, 2024-09-14T22:36:46.639 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.639 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.639 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.639 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.640 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.640 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.640 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2091 2024-09-14T22:36:46.640 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.640 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.640 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.640 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.640 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451119+0000", 2024-09-14T22:36:46.640 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.640 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.640 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.640 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.641 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451119+0000", 2024-09-14T22:36:46.641 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.641 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.641 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.641 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.641 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451120+0000", 2024-09-14T22:36:46.641 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0669999999999999e-06 2024-09-14T22:36:46.641 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.641 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.641 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.641 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451125+0000", 2024-09-14T22:36:46.641 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3329999999999996e-06 2024-09-14T22:36:46.641 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.642 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.642 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.642 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451143+0000", 2024-09-14T22:36:46.642 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7309999999999999e-05 2024-09-14T22:36:46.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.642 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.642 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.642 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451157+0000", 2024-09-14T22:36:46.642 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3835999999999999e-05 2024-09-14T22:36:46.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.642 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.642 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.642 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509863+0000", 2024-09-14T22:36:46.642 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058706345 2024-09-14T22:36:46.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.643 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.643 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509985+0000", 2024-09-14T22:36:46.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000121736 2024-09-14T22:36:46.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.643 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.643 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510164+0000", 2024-09-14T22:36:46.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017964700000000001 2024-09-14T22:36:46.643 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.643 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.643 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.644 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2092 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11507712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:46.644 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.451202+0000", 2024-09-14T22:36:46.644 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.117712516000001, 2024-09-14T22:36:46.644 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059345031999999999, 2024-09-14T22:36:46.644 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.644 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.644 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.644 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.644 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.644 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.644 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2092 2024-09-14T22:36:46.644 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.644 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.644 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.644 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.644 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451202+0000", 2024-09-14T22:36:46.645 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.645 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.645 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.645 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.645 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451202+0000", 2024-09-14T22:36:46.645 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.645 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.645 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.645 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.645 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451203+0000", 2024-09-14T22:36:46.645 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6199999999999999e-06 2024-09-14T22:36:46.645 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.646 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.646 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.646 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451209+0000", 2024-09-14T22:36:46.646 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1999999999999999e-06 2024-09-14T22:36:46.646 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.646 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.646 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.646 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451230+0000", 2024-09-14T22:36:46.646 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0247000000000001e-05 2024-09-14T22:36:46.646 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.646 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.646 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.646 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451245+0000", 2024-09-14T22:36:46.647 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4993e-05 2024-09-14T22:36:46.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.647 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.647 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510239+0000", 2024-09-14T22:36:46.647 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058994321000000002 2024-09-14T22:36:46.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.647 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.647 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510371+0000", 2024-09-14T22:36:46.647 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013195899999999999 2024-09-14T22:36:46.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.647 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.647 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510547+0000", 2024-09-14T22:36:46.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000175692 2024-09-14T22:36:46.648 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.648 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.648 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.648 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.648 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.648 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2093 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11515904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:46.648 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.451264+0000", 2024-09-14T22:36:46.648 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.117650479000002, 2024-09-14T22:36:46.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059636293999999999, 2024-09-14T22:36:46.648 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.648 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.648 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.648 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.649 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.649 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.649 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2093 2024-09-14T22:36:46.649 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.649 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.649 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.649 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.649 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451264+0000", 2024-09-14T22:36:46.649 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.649 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.649 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.649 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.650 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451264+0000", 2024-09-14T22:36:46.650 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.650 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.650 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.650 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.650 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451265+0000", 2024-09-14T22:36:46.650 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3400000000000001e-06 2024-09-14T22:36:46.650 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.650 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.650 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.650 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451270+0000", 2024-09-14T22:36:46.650 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4360000000000001e-06 2024-09-14T22:36:46.651 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.651 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.651 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.651 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451289+0000", 2024-09-14T22:36:46.651 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8383e-05 2024-09-14T22:36:46.651 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.651 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.651 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.651 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451303+0000", 2024-09-14T22:36:46.651 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.455e-05 2024-09-14T22:36:46.651 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.651 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.651 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.652 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510627+0000", 2024-09-14T22:36:46.652 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059323280999999999 2024-09-14T22:36:46.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.652 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.652 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510745+0000", 2024-09-14T22:36:46.652 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011867900000000001 2024-09-14T22:36:46.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.652 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.652 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510900+0000", 2024-09-14T22:36:46.652 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000154625 2024-09-14T22:36:46.652 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.652 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.653 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.653 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.653 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.653 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2094 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11524096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:46.653 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.451321+0000", 2024-09-14T22:36:46.653 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.117593493000001, 2024-09-14T22:36:46.653 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059938827, 2024-09-14T22:36:46.653 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.654 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.654 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.654 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.654 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.654 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.654 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2094 2024-09-14T22:36:46.654 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.654 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.654 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.654 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.654 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451321+0000", 2024-09-14T22:36:46.654 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.655 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.655 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.655 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.655 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451321+0000", 2024-09-14T22:36:46.655 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.655 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.655 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.655 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.655 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451322+0000", 2024-09-14T22:36:46.655 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0529999999999999e-06 2024-09-14T22:36:46.655 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.655 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.656 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.656 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451328+0000", 2024-09-14T22:36:46.656 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8300000000000001e-06 2024-09-14T22:36:46.656 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.656 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.656 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.656 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451344+0000", 2024-09-14T22:36:46.656 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6750000000000001e-05 2024-09-14T22:36:46.656 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.656 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.656 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.656 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451358+0000", 2024-09-14T22:36:46.656 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3200000000000001e-05 2024-09-14T22:36:46.657 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.657 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.657 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.657 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510968+0000", 2024-09-14T22:36:46.657 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059610567000000003 2024-09-14T22:36:46.657 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.657 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.657 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.657 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.511094+0000", 2024-09-14T22:36:46.657 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012629199999999999 2024-09-14T22:36:46.657 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.657 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.658 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.658 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.511260+0000", 2024-09-14T22:36:46.658 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016513500000000001 2024-09-14T22:36:46.658 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.658 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.658 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.658 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.658 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.658 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2095 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11532288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:46.658 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.451375+0000", 2024-09-14T22:36:46.658 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.117539319999999, 2024-09-14T22:36:46.658 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060228067000000003, 2024-09-14T22:36:46.658 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.659 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.659 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:46.659 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.659 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.659 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.659 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2095 2024-09-14T22:36:46.659 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.659 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.659 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.659 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.659 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451375+0000", 2024-09-14T22:36:46.659 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.659 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.660 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.660 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451375+0000", 2024-09-14T22:36:46.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.660 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.660 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451376+0000", 2024-09-14T22:36:46.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0470000000000001e-06 2024-09-14T22:36:46.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.660 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.660 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451381+0000", 2024-09-14T22:36:46.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.536e-06 2024-09-14T22:36:46.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.661 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.661 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451399+0000", 2024-09-14T22:36:46.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7200000000000001e-05 2024-09-14T22:36:46.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.661 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.661 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451412+0000", 2024-09-14T22:36:46.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3726999999999999e-05 2024-09-14T22:36:46.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.511308+0000", 2024-09-14T22:36:46.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059895176000000001 2024-09-14T22:36:46.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.511444+0000", 2024-09-14T22:36:46.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013621600000000001 2024-09-14T22:36:46.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.511603+0000", 2024-09-14T22:36:46.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015916500000000001 2024-09-14T22:36:46.663 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.663 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.663 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.663 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:45459 3.11 3:8ce265cb:::rbd_data.113cc4e35b3.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:36:46.663 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:36.190430+0000", 2024-09-14T22:36:46.663 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.378484576, 2024-09-14T22:36:46.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.06125162, 2024-09-14T22:36:46.663 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.663 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.663 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:46.663 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.663 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.664 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.664 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 45459 2024-09-14T22:36:46.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.664 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.664 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.664 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190430+0000", 2024-09-14T22:36:46.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.664 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.664 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190430+0000", 2024-09-14T22:36:46.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190432+0000", 2024-09-14T22:36:46.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3039999999999999e-06 2024-09-14T22:36:46.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190443+0000", 2024-09-14T22:36:46.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1178999999999999e-05 2024-09-14T22:36:46.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190474+0000", 2024-09-14T22:36:46.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0803000000000003e-05 2024-09-14T22:36:46.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190489+0000", 2024-09-14T22:36:46.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.517e-05 2024-09-14T22:36:46.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190623+0000", 2024-09-14T22:36:46.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013405599999999999 2024-09-14T22:36:46.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190752+0000", 2024-09-14T22:36:46.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012870899999999999 2024-09-14T22:36:46.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:36:46.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.203210+0000", 2024-09-14T22:36:46.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012458484000000001 2024-09-14T22:36:46.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:46.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.251517+0000", 2024-09-14T22:36:46.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.048306453999999999 2024-09-14T22:36:46.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:46.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.251571+0000", 2024-09-14T22:36:46.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4228999999999999e-05 2024-09-14T22:36:46.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:46.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.251628+0000", 2024-09-14T22:36:46.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6922000000000002e-05 2024-09-14T22:36:46.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.251681+0000", 2024-09-14T22:36:46.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3310000000000003e-05 2024-09-14T22:36:46.669 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.669 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.669 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.669 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:45462 3.11 3:8ce265cb:::rbd_data.113cc4e35b3.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:36:46.669 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:36.258125+0000", 2024-09-14T22:36:46.669 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.310788998, 2024-09-14T22:36:46.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059222599000000001, 2024-09-14T22:36:46.669 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:46.669 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:46.669 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:46.669 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:46.670 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:46.670 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.670 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 45462 2024-09-14T22:36:46.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.670 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:46.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:46.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258125+0000", 2024-09-14T22:36:46.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:46.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258125+0000", 2024-09-14T22:36:46.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:46.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:46.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258127+0000", 2024-09-14T22:36:46.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3159999999999999e-06 2024-09-14T22:36:46.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:46.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258139+0000", 2024-09-14T22:36:46.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1667e-05 2024-09-14T22:36:46.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:46.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258170+0000", 2024-09-14T22:36:46.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0929999999999997e-05 2024-09-14T22:36:46.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258185+0000", 2024-09-14T22:36:46.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4423000000000001e-05 2024-09-14T22:36:46.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.673 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:46.673 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258291+0000", 2024-09-14T22:36:46.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106285 2024-09-14T22:36:46.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.673 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:46.673 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258446+0000", 2024-09-14T22:36:46.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015486499999999999 2024-09-14T22:36:46.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.673 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:36:46.673 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.269615+0000", 2024-09-14T22:36:46.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011169095 2024-09-14T22:36:46.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:46.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.317113+0000", 2024-09-14T22:36:46.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.047498382999999998 2024-09-14T22:36:46.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:46.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.317144+0000", 2024-09-14T22:36:46.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0970000000000003e-05 2024-09-14T22:36:46.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:46.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.317233+0000", 2024-09-14T22:36:46.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9205999999999999e-05 2024-09-14T22:36:46.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:46.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:46.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:46.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.317348+0000", 2024-09-14T22:36:46.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000114459 2024-09-14T22:36:46.675 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.675 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.675 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.675 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:46.675 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:46.675 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:46.676 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:46.676 INFO:teuthology.orchestra.run.smithi001.stdout:1417 writesame 0x97db760 thru 0x97e67e7 data_size 0x1c(0xb088 bytes) 2024-09-14T22:36:46.677 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:46.671+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:46.678 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:36:46.715 INFO:teuthology.orchestra.run.smithi001.stdout:1418 read 0xb38f1db thru 0xb398e53 (0x9c79 bytes) 2024-09-14T22:36:46.716 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:46.717 INFO:teuthology.orchestra.run.smithi001.stdout:1424 read 0x93b0bb2 thru 0x93b8de4 (0x8233 bytes) 2024-09-14T22:36:46.717 INFO:teuthology.orchestra.run.smithi001.stdout:1425 writesame 0x7dcdfad thru 0x7ddbd20 data_size 0x2(0xdd74 bytes) 2024-09-14T22:36:46.754 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:36:46.755 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-14T22:36:46.756 INFO:teuthology.orchestra.run.smithi001.stdout:1426 punch from 0x7878aa0 to 0x787e2ba, (0x581a bytes) 2024-09-14T22:36:46.756 INFO:teuthology.orchestra.run.smithi001.stdout:1427 write 0x24c5f5 thru 0x252819 (0x6225 bytes) 2024-09-14T22:36:46.777 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2024-09-14T22:36:45.959814+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":437684683,"num_objects":500,"num_object_clones":172,"num_object_copies":1000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":52,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":500,"num_whiteouts":215,"num_read":76594,"num_read_kb":239481,"num_write":6354,"num_write_kb":112610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4943,"ondisk_log_size":4943,"up":77,"acting":77,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":290552,"kb_used_data":16088,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754684168,"statfs":{"total":773094113280,"available":772796588032,"internally_reserved":0,"allocated":16474112,"data_stored":145823822,"data_compressed":1375134,"data_compressed_allocated":10354688,"data_compressed_original":143102522,"omap_allocated":12750,"internal_metadata":280874546},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":42862472,"num_objects":155,"num_object_clones":46,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":89,"num_read":20707,"num_read_kb":154608,"num_write":683,"num_write_kb":62594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.091038"},"pg_stats":[{"pgid":"3.1f","version":"43'116","reported_seq":2066,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:42.879596+0000","last_change":"2024-09-14T22:36:17.721881+0000","last_active":"2024-09-14T22:36:42.879596+0000","last_peered":"2024-09-14T22:36:42.879596+0000","last_clean":"2024-09-14T22:36:42.879596+0000","last_became_active":"2024-09-14T22:36:15.718313+0000","last_became_peered":"2024-09-14T22:36:15.718313+0000","last_unstale":"2024-09-14T22:36:42.879596+0000","last_undegraded":"2024-09-14T22:36:42.879596+0000","last_fullsized":"2024-09-14T22:36:42.879596+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:44:34.007496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017190879999999999,"stat_sum":{"num_bytes":18260901,"num_objects":19,"num_object_clones":6,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":7,"num_read":2027,"num_read_kb":3603,"num_write":129,"num_write_kb":2473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"43'662","reported_seq":3013,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:42.593583+0000","last_change":"2024-09-14T22:36:17.733296+0000","last_active":"2024-09-14T22:36:42.593583+0000","last_peered":"2024-09-14T22:36:42.593583+0000","last_clean":"2024-09-14T22:36:42.593583+0000","last_became_active":"2024-09-14T22:36:15.709272+0000","last_became_peered":"2024-09-14T22:36:15.709272+0000","last_unstale":"2024-09-14T22:36:42.593583+0000","last_undegraded":"2024-09-14T22:36:42.593583+0000","last_fullsized":"2024-09-14T22:36:42.593583+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:18.516925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001723619,"stat_sum":{"num_bytes":39067737,"num_objects":14,"num_object_clones":4,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":5,"num_read":5576,"num_read_kb":18118,"num_write":1223,"num_write_kb":6879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"43'158","reported_seq":5417,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:42.508844+0000","last_change":"2024-09-14T22:36:17.702313+0000","last_active":"2024-09-14T22:36:42.508844+0000","last_peered":"2024-09-14T22:36:42.508844+0000","last_clean":"2024-09-14T22:36:42.508844+0000","last_became_active":"2024-09-14T22:36:15.719488+0000","last_became_peered":"2024-09-14T22:36:15.719488+0000","last_unstale":"2024-09-14T22:36:42.508844+0000","last_undegraded":"2024-09-14T22:36:42.508844+0000","last_fullsized":"2024-09-14T22:36:42.508844+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:27:43.420521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053914200000000005,"stat_sum":{"num_bytes":32802358,"num_objects":27,"num_object_clones":11,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":12,"num_read":5240,"num_read_kb":29303,"num_write":148,"num_write_kb":4832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"43'118","reported_seq":702,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:42.858206+0000","last_change":"2024-09-14T22:36:17.702413+0000","last_active":"2024-09-14T22:36:42.858206+0000","last_peered":"2024-09-14T22:36:42.858206+0000","last_clean":"2024-09-14T22:36:42.858206+0000","last_became_active":"2024-09-14T22:36:15.719470+0000","last_became_peered":"2024-09-14T22:36:15.719470+0000","last_unstale":"2024-09-14T22:36:42.858206+0000","last_undegraded":"2024-09-14T22:36:42.858206+0000","last_fullsized":"2024-09-14T22:36:42.858206+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:03:33.315266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055079499999999997,"stat_sum":{"num_bytes":13523292,"num_objects":28,"num_object_clones":10,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":14,"num_read":574,"num_read_kb":1312,"num_write":107,"num_write_kb":4932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"43'101","reported_seq":3474,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:42.876278+0000","last_change":"2024-09-14T22:36:17.720920+0000","last_active":"2024-09-14T22:36:42.876278+0000","last_peered":"2024-09-14T22:36:42.876278+0000","last_clean":"2024-09-14T22:36:42.876278+0000","last_became_active":"2024-09-14T22:36:15.709684+0000","last_became_peered":"2024-09-14T22:36:15.709684+0000","last_unstale":"2024-09-14T22:36:42.876278+0000","last_undegraded":"2024-09-14T22:36:42.876278+0000","last_fullsized":"2024-09-14T22:36:42.876278+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:35:23.446759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092291499999999996,"stat_sum":{"num_bytes":20138135,"num_objects":18,"num_object_clones":7,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":8,"num_read":3349,"num_read_kb":15441,"num_write":90,"num_write_kb":6741,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"43'72","reported_seq":711,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:42.821040+0000","last_change":"2024-09-14T22:36:17.711364+0000","last_active":"2024-09-14T22:36:42.821040+0000","last_peered":"2024-09-14T22:36:42.821040+0000","last_clean":"2024-09-14T22:36:42.821040+0000","last_became_active":"2024-09-14T22:36:15.701852+0000","last_became_peered":"2024-09-14T22:36:15.701852+0000","last_unstale":"2024-09-14T22:36:42.821040+0000","last_undegraded":"2024-09-14T22:36:42.821040+0000","last_fullsized":"2024-09-14T22:36:42.821040+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:45:21.647003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012502780000000001,"stat_sum":{"num_bytes":1046547,"num_objects":15,"num_object_clones":5,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":8,"num_read":593,"num_read_kb":816,"num_write":64,"num_write_kb":214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"42'129","reported_seq":2666,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-14T22:36:41.243240+0000","last_change":"2024-09-14T22:36:17.705423+0000","last_active":"2024-09-14T22:36:41.243240+0000","last_peered":"2024-09-14T22:36:41.243240+0000","last_clean":"2024-09-14T22:36:41.243240+0000","last_became_active":"2024-09-14T22:36:15.700990+0000","last_became_peered":"2024-09-14T22:36:15.700990+0000","last_unstale":"2024-09-14T22:36:41.243240+0000","last_undegraded":"2024-09-14T22:36:41.243240+0000","last_fullsized":"2024-09-14T22:36:41.243240+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:07:47.408064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010520620000000001,"stat_sum":{"num_bytes":3109851,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":5,"num_read":2879,"num_read_kb":2835,"num_write":122,"num_write_kb":282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"43'136","reported_seq":1866,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:42.594887+0000","last_change":"2024-09-14T22:36:17.733672+0000","last_active":"2024-09-14T22:36:42.594887+0000","last_peered":"2024-09-14T22:36:42.594887+0000","last_clean":"2024-09-14T22:36:42.594887+0000","last_became_active":"2024-09-14T22:36:15.701583+0000","last_became_peered":"2024-09-14T22:36:15.701583+0000","last_unstale":"2024-09-14T22:36:42.594887+0000","last_undegraded":"2024-09-14T22:36:42.594887+0000","last_fullsized":"2024-09-14T22:36:42.594887+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:34:00.871489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002063484,"stat_sum":{"num_bytes":12191790,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":3,"num_read":1944,"num_read_kb":13482,"num_write":209,"num_write_kb":3657,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"43'91","reported_seq":2325,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:42.879828+0000","last_change":"2024-09-14T22:36:17.721195+0000","last_active":"2024-09-14T22:36:42.879828+0000","last_peered":"2024-09-14T22:36:42.879828+0000","last_clean":"2024-09-14T22:36:42.879828+0000","last_became_active":"2024-09-14T22:36:15.715869+0000","last_became_peered":"2024-09-14T22:36:15.715869+0000","last_unstale":"2024-09-14T22:36:42.879828+0000","last_undegraded":"2024-09-14T22:36:42.879828+0000","last_fullsized":"2024-09-14T22:36:42.879828+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:40:46.988203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001160672,"stat_sum":{"num_bytes":10016984,"num_objects":15,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":6,"num_read":2199,"num_read_kb":3359,"num_write":124,"num_write_kb":7569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"43'88","reported_seq":2308,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:42.857323+0000","last_change":"2024-09-14T22:36:17.703048+0000","last_active":"2024-09-14T22:36:42.857323+0000","last_peered":"2024-09-14T22:36:42.857323+0000","last_clean":"2024-09-14T22:36:42.857323+0000","last_became_active":"2024-09-14T22:36:15.725078+0000","last_became_peered":"2024-09-14T22:36:15.725078+0000","last_unstale":"2024-09-14T22:36:42.857323+0000","last_undegraded":"2024-09-14T22:36:42.857323+0000","last_fullsized":"2024-09-14T22:36:42.857323+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:31:49.044631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010500290000000001,"stat_sum":{"num_bytes":16797696,"num_objects":19,"num_object_clones":8,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":10,"num_read":2196,"num_read_kb":16190,"num_write":83,"num_write_kb":5789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"43'131","reported_seq":2621,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:42.487115+0000","last_change":"2024-09-14T22:36:18.009371+0000","last_active":"2024-09-14T22:36:42.487115+0000","last_peered":"2024-09-14T22:36:42.487115+0000","last_clean":"2024-09-14T22:36:42.487115+0000","last_became_active":"2024-09-14T22:36:15.717341+0000","last_became_peered":"2024-09-14T22:36:15.717341+0000","last_unstale":"2024-09-14T22:36:42.487115+0000","last_undegraded":"2024-09-14T22:36:42.487115+0000","last_fullsized":"2024-09-14T22:36:42.487115+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:23:12.445207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054571999999999997,"stat_sum":{"num_bytes":3708965,"num_objects":11,"num_object_clones":4,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":5,"num_read":2649,"num_read_kb":3157,"num_write":149,"num_write_kb":442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"43'73","reported_seq":2462,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:42.807973+0000","last_change":"2024-09-14T22:36:17.711384+0000","last_active":"2024-09-14T22:36:42.807973+0000","last_peered":"2024-09-14T22:36:42.807973+0000","last_clean":"2024-09-14T22:36:42.807973+0000","last_became_active":"2024-09-14T22:36:15.723075+0000","last_became_peered":"2024-09-14T22:36:15.723075+0000","last_unstale":"2024-09-14T22:36:42.807973+0000","last_undegraded":"2024-09-14T22:36:42.807973+0000","last_fullsized":"2024-09-14T22:36:42.807973+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:54:34.687593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010804009999999999,"stat_sum":{"num_bytes":6649871,"num_objects":17,"num_object_clones":7,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":8,"num_read":2356,"num_read_kb":2956,"num_write":64,"num_write_kb":4461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:41.849853+0000","last_change":"2024-09-14T22:36:08.042860+0000","last_active":"2024-09-14T22:36:41.849853+0000","last_peered":"2024-09-14T22:36:41.849853+0000","last_clean":"2024-09-14T22:36:41.849853+0000","last_became_active":"2024-09-14T22:36:06.040961+0000","last_became_peered":"2024-09-14T22:36:06.040961+0000","last_unstale":"2024-09-14T22:36:41.849853+0000","last_undegraded":"2024-09-14T22:36:41.849853+0000","last_fullsized":"2024-09-14T22:36:41.849853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:03:48.547834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010780950000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"43'90","reported_seq":549,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:42.719025+0000","last_change":"2024-09-14T22:36:17.720897+0000","last_active":"2024-09-14T22:36:42.719025+0000","last_peered":"2024-09-14T22:36:42.719025+0000","last_clean":"2024-09-14T22:36:42.719025+0000","last_became_active":"2024-09-14T22:36:15.700982+0000","last_became_peered":"2024-09-14T22:36:15.700982+0000","last_unstale":"2024-09-14T22:36:42.719025+0000","last_undegraded":"2024-09-14T22:36:42.719025+0000","last_fullsized":"2024-09-14T22:36:42.719025+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:08:58.323614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010495459999999999,"stat_sum":{"num_bytes":7676446,"num_objects":16,"num_object_clones":6,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":731,"num_read_kb":1903,"num_write":85,"num_write_kb":2854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:41.930865+0000","last_change":"2024-09-14T22:36:41.930087+0000","last_active":"2024-09-14T22:36:41.930865+0000","last_peered":"2024-09-14T22:36:41.930865+0000","last_clean":"2024-09-14T22:36:41.930865+0000","last_became_active":"2024-09-14T22:36:41.929522+0000","last_became_peered":"2024-09-14T22:36:41.929522+0000","last_unstale":"2024-09-14T22:36:41.930865+0000","last_undegraded":"2024-09-14T22:36:41.930865+0000","last_fullsized":"2024-09-14T22:36:41.930865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:15:10.589042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"42'186","reported_seq":2907,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:42.670001+0000","last_change":"2024-09-14T22:36:21.722517+0000","last_active":"2024-09-14T22:36:42.670001+0000","last_peered":"2024-09-14T22:36:42.670001+0000","last_clean":"2024-09-14T22:36:42.670001+0000","last_became_active":"2024-09-14T22:36:21.722085+0000","last_became_peered":"2024-09-14T22:36:21.722085+0000","last_unstale":"2024-09-14T22:36:42.670001+0000","last_undegraded":"2024-09-14T22:36:42.670001+0000","last_fullsized":"2024-09-14T22:36:42.670001+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:27:11.564605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020774109999999999,"stat_sum":{"num_bytes":41242153,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":3,"num_read":3478,"num_read_kb":16469,"num_write":251,"num_write_kb":5625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:41.921159+0000","last_change":"2024-09-14T22:36:08.054606+0000","last_active":"2024-09-14T22:36:41.921159+0000","last_peered":"2024-09-14T22:36:41.921159+0000","last_clean":"2024-09-14T22:36:41.921159+0000","last_became_active":"2024-09-14T22:36:06.266908+0000","last_became_peered":"2024-09-14T22:36:06.266908+0000","last_unstale":"2024-09-14T22:36:41.921159+0000","last_undegraded":"2024-09-14T22:36:41.921159+0000","last_fullsized":"2024-09-14T22:36:41.921159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:36.738643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098249300000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"41'119","reported_seq":2317,"reported_epoch":42,"state":"active+undersized+degraded","last_fresh":"2024-09-14T22:36:40.845240+0000","last_change":"2024-09-14T22:36:27.775362+0000","last_active":"2024-09-14T22:36:40.845240+0000","last_peered":"2024-09-14T22:36:40.845240+0000","last_clean":"2024-09-14T22:36:25.819901+0000","last_became_active":"2024-09-14T22:36:27.775362+0000","last_became_peered":"2024-09-14T22:36:27.775362+0000","last_unstale":"2024-09-14T22:36:40.845240+0000","last_undegraded":"2024-09-14T22:36:27.774765+0000","last_fullsized":"2024-09-14T22:36:27.774675+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:42:28.215564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012564830000000001,"stat_sum":{"num_bytes":13031461,"num_objects":20,"num_object_clones":8,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":9,"num_read":1923,"num_read_kb":3898,"num_write":150,"num_write_kb":5633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":20}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:41.931886+0000","last_change":"2024-09-14T22:36:41.930961+0000","last_active":"2024-09-14T22:36:41.931886+0000","last_peered":"2024-09-14T22:36:41.931886+0000","last_clean":"2024-09-14T22:36:41.931886+0000","last_became_active":"2024-09-14T22:36:41.930619+0000","last_became_peered":"2024-09-14T22:36:41.930619+0000","last_unstale":"2024-09-14T22:36:41.931886+0000","last_undegraded":"2024-09-14T22:36:41.931886+0000","last_fullsized":"2024-09-14T22:36:41.931886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:04:40.582999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"43'125","reported_seq":736,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:42.291786+0000","last_change":"2024-09-14T22:36:17.702597+0000","last_active":"2024-09-14T22:36:42.291786+0000","last_peered":"2024-09-14T22:36:42.291786+0000","last_clean":"2024-09-14T22:36:42.291786+0000","last_became_active":"2024-09-14T22:36:15.717186+0000","last_became_peered":"2024-09-14T22:36:15.717186+0000","last_unstale":"2024-09-14T22:36:42.291786+0000","last_undegraded":"2024-09-14T22:36:42.291786+0000","last_fullsized":"2024-09-14T22:36:42.291786+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:28:44.715910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071220099999999998,"stat_sum":{"num_bytes":3168537,"num_objects":14,"num_object_clones":4,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":5,"num_read":1083,"num_read_kb":1567,"num_write":108,"num_write_kb":326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"43'70","reported_seq":354,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:42.316701+0000","last_change":"2024-09-14T22:36:17.711068+0000","last_active":"2024-09-14T22:36:42.316701+0000","last_peered":"2024-09-14T22:36:42.316701+0000","last_clean":"2024-09-14T22:36:42.316701+0000","last_became_active":"2024-09-14T22:36:15.706147+0000","last_became_peered":"2024-09-14T22:36:15.706147+0000","last_unstale":"2024-09-14T22:36:42.316701+0000","last_undegraded":"2024-09-14T22:36:42.316701+0000","last_fullsized":"2024-09-14T22:36:42.316701+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:38.273830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082625500000000002,"stat_sum":{"num_bytes":34816,"num_objects":8,"num_object_clones":4,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":4,"num_read":231,"num_read_kb":34,"num_write":66,"num_write_kb":218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":63,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:41.854070+0000","last_change":"2024-09-14T22:36:08.055938+0000","last_active":"2024-09-14T22:36:41.854070+0000","last_peered":"2024-09-14T22:36:41.854070+0000","last_clean":"2024-09-14T22:36:41.854070+0000","last_became_active":"2024-09-14T22:36:06.039600+0000","last_became_peered":"2024-09-14T22:36:06.039600+0000","last_unstale":"2024-09-14T22:36:41.854070+0000","last_undegraded":"2024-09-14T22:36:41.854070+0000","last_fullsized":"2024-09-14T22:36:41.854070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:49:09.245563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021617870000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"42'189","reported_seq":5914,"reported_epoch":42,"state":"active+undersized+degraded","last_fresh":"2024-09-14T22:36:41.201284+0000","last_change":"2024-09-14T22:36:27.775190+0000","last_active":"2024-09-14T22:36:41.201284+0000","last_peered":"2024-09-14T22:36:41.201284+0000","last_clean":"2024-09-14T22:36:25.848645+0000","last_became_active":"2024-09-14T22:36:27.775190+0000","last_became_peered":"2024-09-14T22:36:27.775190+0000","last_unstale":"2024-09-14T22:36:41.201284+0000","last_undegraded":"2024-09-14T22:36:27.774713+0000","last_fullsized":"2024-09-14T22:36:27.774588+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:25:47.631452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021500769999999998,"stat_sum":{"num_bytes":24707127,"num_objects":16,"num_object_clones":5,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":5,"num_read":6003,"num_read_kb":7569,"num_write":211,"num_write_kb":3010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":16}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":30,"reported_epoch":43,"state":"active+undersized","last_fresh":"2024-09-14T22:36:41.849278+0000","last_change":"2024-09-14T22:36:27.773450+0000","last_active":"2024-09-14T22:36:41.849278+0000","last_peered":"2024-09-14T22:36:41.849278+0000","last_clean":"2024-09-14T22:36:05.010064+0000","last_became_active":"2024-09-14T22:36:27.773450+0000","last_became_peered":"2024-09-14T22:36:27.773450+0000","last_unstale":"2024-09-14T22:36:41.849278+0000","last_undegraded":"2024-09-14T22:36:41.849278+0000","last_fullsized":"2024-09-14T22:36:27.772739+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:05:20.367237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"43'90","reported_seq":275,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:42.584064+0000","last_change":"2024-09-14T22:36:17.720438+0000","last_active":"2024-09-14T22:36:42.584064+0000","last_peered":"2024-09-14T22:36:42.584064+0000","last_clean":"2024-09-14T22:36:42.584064+0000","last_became_active":"2024-09-14T22:36:15.711105+0000","last_became_peered":"2024-09-14T22:36:15.711105+0000","last_unstale":"2024-09-14T22:36:42.584064+0000","last_undegraded":"2024-09-14T22:36:42.584064+0000","last_fullsized":"2024-09-14T22:36:42.584064+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:04:08.730892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085549400000000005,"stat_sum":{"num_bytes":983309,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":144,"num_read_kb":501,"num_write":87,"num_write_kb":288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:41.920813+0000","last_change":"2024-09-14T22:36:08.054759+0000","last_active":"2024-09-14T22:36:41.920813+0000","last_peered":"2024-09-14T22:36:41.920813+0000","last_clean":"2024-09-14T22:36:41.920813+0000","last_became_active":"2024-09-14T22:36:06.039969+0000","last_became_peered":"2024-09-14T22:36:06.039969+0000","last_unstale":"2024-09-14T22:36:41.920813+0000","last_undegraded":"2024-09-14T22:36:41.920813+0000","last_fullsized":"2024-09-14T22:36:41.920813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:03:45.369532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010817520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"43'750","reported_seq":1642,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:42.789573+0000","last_change":"2024-09-14T22:36:21.724315+0000","last_active":"2024-09-14T22:36:42.789573+0000","last_peered":"2024-09-14T22:36:42.789573+0000","last_clean":"2024-09-14T22:36:42.789573+0000","last_became_active":"2024-09-14T22:36:21.723138+0000","last_became_peered":"2024-09-14T22:36:21.723138+0000","last_unstale":"2024-09-14T22:36:42.789573+0000","last_undegraded":"2024-09-14T22:36:42.789573+0000","last_fullsized":"2024-09-14T22:36:42.789573+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:56:02.113482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2347122,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":5,"num_read":5216,"num_read_kb":5326,"num_write":1209,"num_write_kb":1388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":64,"reported_epoch":43,"state":"active+undersized+degraded","last_fresh":"2024-09-14T22:36:41.853906+0000","last_change":"2024-09-14T22:36:27.781594+0000","last_active":"2024-09-14T22:36:41.853906+0000","last_peered":"2024-09-14T22:36:41.853906+0000","last_clean":"2024-09-14T22:36:23.739926+0000","last_became_active":"2024-09-14T22:36:27.781594+0000","last_became_peered":"2024-09-14T22:36:27.781594+0000","last_unstale":"2024-09-14T22:36:41.853906+0000","last_undegraded":"2024-09-14T22:36:27.780369+0000","last_fullsized":"2024-09-14T22:36:27.780115+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:49:52.683148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011401880000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":1}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":212,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:41.920961+0000","last_change":"2024-09-14T22:36:00.402862+0000","last_active":"2024-09-14T22:36:41.920961+0000","last_peered":"2024-09-14T22:36:41.920961+0000","last_clean":"2024-09-14T22:36:41.920961+0000","last_became_active":"2024-09-14T22:36:00.400529+0000","last_became_peered":"2024-09-14T22:36:00.400529+0000","last_unstale":"2024-09-14T22:36:41.920961+0000","last_undegraded":"2024-09-14T22:36:41.920961+0000","last_fullsized":"2024-09-14T22:36:41.920961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_clean_scrub_stamp":"2024-09-14T22:35:58.989025+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:07:12.951728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"43'95","reported_seq":2325,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:42.487547+0000","last_change":"2024-09-14T22:36:17.732784+0000","last_active":"2024-09-14T22:36:42.487547+0000","last_peered":"2024-09-14T22:36:42.487547+0000","last_clean":"2024-09-14T22:36:42.487547+0000","last_became_active":"2024-09-14T22:36:15.704885+0000","last_became_peered":"2024-09-14T22:36:15.704885+0000","last_unstale":"2024-09-14T22:36:42.487547+0000","last_undegraded":"2024-09-14T22:36:42.487547+0000","last_fullsized":"2024-09-14T22:36:42.487547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:30:53.542817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001381511,"stat_sum":{"num_bytes":11814259,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":9,"num_read":2198,"num_read_kb":3693,"num_write":130,"num_write_kb":3427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"43'94","reported_seq":4085,"reported_epoch":43,"state":"active+undersized+degraded","last_fresh":"2024-09-14T22:36:42.820318+0000","last_change":"2024-09-14T22:36:27.777838+0000","last_active":"2024-09-14T22:36:42.820318+0000","last_peered":"2024-09-14T22:36:42.820318+0000","last_clean":"2024-09-14T22:36:25.764517+0000","last_became_active":"2024-09-14T22:36:27.777838+0000","last_became_peered":"2024-09-14T22:36:27.777838+0000","last_unstale":"2024-09-14T22:36:42.820318+0000","last_undegraded":"2024-09-14T22:36:27.776696+0000","last_fullsized":"2024-09-14T22:36:27.776472+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:08:11.969813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060272699999999995,"stat_sum":{"num_bytes":15957391,"num_objects":15,"num_object_clones":5,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":5,"num_read":4146,"num_read_kb":4578,"num_write":91,"num_write_kb":2276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":15}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"43'155","reported_seq":576,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:42.344592+0000","last_change":"2024-09-14T22:36:17.776247+0000","last_active":"2024-09-14T22:36:42.344592+0000","last_peered":"2024-09-14T22:36:42.344592+0000","last_clean":"2024-09-14T22:36:42.344592+0000","last_became_active":"2024-09-14T22:36:15.718794+0000","last_became_peered":"2024-09-14T22:36:15.718794+0000","last_unstale":"2024-09-14T22:36:42.344592+0000","last_undegraded":"2024-09-14T22:36:42.344592+0000","last_fullsized":"2024-09-14T22:36:42.344592+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:55.557047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088662699999999997,"stat_sum":{"num_bytes":5142528,"num_objects":11,"num_object_clones":4,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":5,"num_read":894,"num_read_kb":1612,"num_write":149,"num_write_kb":4975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"43'99","reported_seq":590,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:42.346300+0000","last_change":"2024-09-14T22:36:17.733002+0000","last_active":"2024-09-14T22:36:42.346300+0000","last_peered":"2024-09-14T22:36:42.346300+0000","last_clean":"2024-09-14T22:36:42.346300+0000","last_became_active":"2024-09-14T22:36:15.702885+0000","last_became_peered":"2024-09-14T22:36:15.702885+0000","last_unstale":"2024-09-14T22:36:42.346300+0000","last_undegraded":"2024-09-14T22:36:42.346300+0000","last_fullsized":"2024-09-14T22:36:42.346300+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:10:04.525471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015180580000000001,"stat_sum":{"num_bytes":7629492,"num_objects":15,"num_object_clones":5,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":7,"num_read":786,"num_read_kb":1922,"num_write":87,"num_write_kb":2752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"43'141","reported_seq":2603,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:42.880320+0000","last_change":"2024-09-14T22:36:17.702738+0000","last_active":"2024-09-14T22:36:42.880320+0000","last_peered":"2024-09-14T22:36:42.880320+0000","last_clean":"2024-09-14T22:36:42.880320+0000","last_became_active":"2024-09-14T22:36:15.713452+0000","last_became_peered":"2024-09-14T22:36:15.713452+0000","last_unstale":"2024-09-14T22:36:42.880320+0000","last_undegraded":"2024-09-14T22:36:42.880320+0000","last_fullsized":"2024-09-14T22:36:42.880320+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:33:17.714532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082782099999999996,"stat_sum":{"num_bytes":10258432,"num_objects":25,"num_object_clones":10,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":11,"num_read":2621,"num_read_kb":4445,"num_write":157,"num_write_kb":2966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"43'81","reported_seq":493,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:42.858672+0000","last_change":"2024-09-14T22:36:18.009776+0000","last_active":"2024-09-14T22:36:42.858672+0000","last_peered":"2024-09-14T22:36:42.858672+0000","last_clean":"2024-09-14T22:36:42.858672+0000","last_became_active":"2024-09-14T22:36:15.723205+0000","last_became_peered":"2024-09-14T22:36:15.723205+0000","last_unstale":"2024-09-14T22:36:42.858672+0000","last_undegraded":"2024-09-14T22:36:42.858672+0000","last_fullsized":"2024-09-14T22:36:42.858672+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:18:44.947677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087783999999999996,"stat_sum":{"num_bytes":1704925,"num_objects":17,"num_object_clones":6,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":10,"num_read":367,"num_read_kb":1647,"num_write":71,"num_write_kb":807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"43'143","reported_seq":445,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:42.887078+0000","last_change":"2024-09-14T22:36:18.009666+0000","last_active":"2024-09-14T22:36:42.887078+0000","last_peered":"2024-09-14T22:36:42.887078+0000","last_clean":"2024-09-14T22:36:42.887078+0000","last_became_active":"2024-09-14T22:36:15.715659+0000","last_became_peered":"2024-09-14T22:36:15.715659+0000","last_unstale":"2024-09-14T22:36:42.887078+0000","last_undegraded":"2024-09-14T22:36:42.887078+0000","last_fullsized":"2024-09-14T22:36:42.887078+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:30:36.817492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000776448,"stat_sum":{"num_bytes":2461732,"num_objects":15,"num_object_clones":5,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":8,"num_read":594,"num_read_kb":1434,"num_write":182,"num_write_kb":362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"43'95","reported_seq":4878,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:42.861107+0000","last_change":"2024-09-14T22:36:18.010182+0000","last_active":"2024-09-14T22:36:42.861107+0000","last_peered":"2024-09-14T22:36:42.861107+0000","last_clean":"2024-09-14T22:36:42.861107+0000","last_became_active":"2024-09-14T22:36:15.719693+0000","last_became_peered":"2024-09-14T22:36:15.719693+0000","last_unstale":"2024-09-14T22:36:42.861107+0000","last_undegraded":"2024-09-14T22:36:42.861107+0000","last_fullsized":"2024-09-14T22:36:42.861107+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:45:49.116089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001045011,"stat_sum":{"num_bytes":12026994,"num_objects":13,"num_object_clones":4,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":4,"num_read":4845,"num_read_kb":8124,"num_write":100,"num_write_kb":1283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"43'112","reported_seq":2083,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:42.879347+0000","last_change":"2024-09-14T22:36:17.776422+0000","last_active":"2024-09-14T22:36:42.879347+0000","last_peered":"2024-09-14T22:36:42.879347+0000","last_clean":"2024-09-14T22:36:42.879347+0000","last_became_active":"2024-09-14T22:36:15.707650+0000","last_became_peered":"2024-09-14T22:36:15.707650+0000","last_unstale":"2024-09-14T22:36:42.879347+0000","last_undegraded":"2024-09-14T22:36:42.879347+0000","last_fullsized":"2024-09-14T22:36:42.879347+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:34:25.986030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094227299999999998,"stat_sum":{"num_bytes":19175723,"num_objects":18,"num_object_clones":7,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":8,"num_read":1949,"num_read_kb":16683,"num_write":148,"num_write_kb":8382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"43'83","reported_seq":527,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:42.808320+0000","last_change":"2024-09-14T22:36:18.009347+0000","last_active":"2024-09-14T22:36:42.808320+0000","last_peered":"2024-09-14T22:36:42.808320+0000","last_clean":"2024-09-14T22:36:42.808320+0000","last_became_active":"2024-09-14T22:36:15.715569+0000","last_became_peered":"2024-09-14T22:36:15.715569+0000","last_unstale":"2024-09-14T22:36:42.808320+0000","last_undegraded":"2024-09-14T22:36:42.808320+0000","last_fullsized":"2024-09-14T22:36:42.808320+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:00:49.651038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054921600000000005,"stat_sum":{"num_bytes":19284114,"num_objects":12,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":5,"num_read":397,"num_read_kb":3485,"num_write":118,"num_write_kb":7719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"42'85","reported_seq":3480,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:42.601516+0000","last_change":"2024-09-14T22:36:17.720448+0000","last_active":"2024-09-14T22:36:42.601516+0000","last_peered":"2024-09-14T22:36:42.601516+0000","last_clean":"2024-09-14T22:36:42.601516+0000","last_became_active":"2024-09-14T22:36:15.716696+0000","last_became_peered":"2024-09-14T22:36:15.716696+0000","last_unstale":"2024-09-14T22:36:42.601516+0000","last_undegraded":"2024-09-14T22:36:42.601516+0000","last_fullsized":"2024-09-14T22:36:42.601516+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:02:27.517516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086547600000000001,"stat_sum":{"num_bytes":27060302,"num_objects":13,"num_object_clones":5,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":5,"num_read":3368,"num_read_kb":26069,"num_write":79,"num_write_kb":1059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"43'146","reported_seq":3938,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T22:36:42.508302+0000","last_change":"2024-09-14T22:36:18.010088+0000","last_active":"2024-09-14T22:36:42.508302+0000","last_peered":"2024-09-14T22:36:42.508302+0000","last_clean":"2024-09-14T22:36:42.508302+0000","last_became_active":"2024-09-14T22:36:15.715866+0000","last_became_peered":"2024-09-14T22:36:15.715866+0000","last_unstale":"2024-09-14T22:36:42.508302+0000","last_undegraded":"2024-09-14T22:36:42.508302+0000","last_fullsized":"2024-09-14T22:36:42.508302+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:11:28.023749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010286189999999999,"stat_sum":{"num_bytes":34073306,"num_objects":17,"num_object_clones":6,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":7,"num_read":3922,"num_read_kb":17850,"num_write":202,"num_write_kb":6571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":437094296,"num_objects":496,"num_object_clones":172,"num_object_copies":992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":51,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":496,"num_whiteouts":215,"num_read":76478,"num_read_kb":239381,"num_write":6193,"num_write_kb":110192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12120064,"data_stored":140449557,"data_compressed":1093253,"data_compressed_allocated":9834496,"data_compressed_original":139342394,"omap_allocated":0,"internal_metadata":0},"log_size":4823,"ondisk_log_size":4823,"up":61,"acting":61,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":14,"acting":14,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542156,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33764,"kb_used_data":2580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338076,"statfs":{"total":96636764160,"available":96602189824,"internally_reserved":0,"allocated":2641920,"data_stored":24163963,"data_compressed":252028,"data_compressed_allocated":1744896,"data_compressed_original":23762135,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542157,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32628,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1474560,"data_stored":12813124,"data_compressed":106312,"data_compressed_allocated":884736,"data_compressed_original":12555331,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542157,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37984,"kb_used_data":2636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333856,"statfs":{"total":96636764160,"available":96597868544,"internally_reserved":0,"allocated":2699264,"data_stored":25509371,"data_compressed":201650,"data_compressed_allocated":1785856,"data_compressed_original":25096796,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542156,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34596,"kb_used_data":3412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337244,"statfs":{"total":96636764160,"available":96601337856,"internally_reserved":0,"allocated":3493888,"data_stored":36772472,"data_compressed":234914,"data_compressed_allocated":2519040,"data_compressed_original":36331198,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542157,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45444,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326396,"statfs":{"total":96636764160,"available":96590229504,"internally_reserved":0,"allocated":1822720,"data_stored":11601494,"data_compressed":234744,"data_compressed_allocated":954368,"data_compressed_original":11208707,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31496,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340344,"statfs":{"total":96636764160,"available":96604512256,"internally_reserved":0,"allocated":315392,"data_stored":122731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36628,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335212,"statfs":{"total":96636764160,"available":96599257088,"internally_reserved":0,"allocated":1302528,"data_stored":8470368,"data_compressed":88724,"data_compressed_allocated":598016,"data_compressed_original":8171520,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542156,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38012,"kb_used_data":2660,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333828,"statfs":{"total":96636764160,"available":96597839872,"internally_reserved":0,"allocated":2723840,"data_stored":26370299,"data_compressed":256762,"data_compressed_allocated":1867776,"data_compressed_original":25976835,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":25561254,"data_compressed":250890,"data_compressed_allocated":1826816,"data_compressed_original":25391107,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":839680,"data_stored":8279198,"data_compressed":88724,"data_compressed_allocated":598016,"data_compressed_original":8171520,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":29424,"data_compressed":10286,"data_compressed_allocated":12288,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1286144,"data_stored":11331551,"data_compressed":196436,"data_compressed_allocated":913408,"data_compressed_original":11147267,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2949120,"data_stored":36497502,"data_compressed":194007,"data_compressed_allocated":2473984,"data_compressed_original":36265662,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2007040,"data_stored":24082046,"data_compressed":136739,"data_compressed_allocated":1638400,"data_compressed_original":23896668,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":991232,"data_stored":12609073,"data_compressed":106312,"data_compressed_allocated":884736,"data_compressed_original":12555331,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1830912,"data_stored":22059509,"data_compressed":109859,"data_compressed_allocated":1486848,"data_compressed_original":21898455,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:36:46.780 INFO:tasks.thrashosds.thrasher:Setting 3.2 to [1, 0, 3, 6] 2024-09-14T22:36:46.780 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.2', '1', '0', '3', '6'] 2024-09-14T22:36:46.780 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.2 1 0 3 6 2024-09-14T22:36:46.780 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:46.770+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:46.786 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:46.786 INFO:teuthology.orchestra.run.smithi001.stdout:1431 read 0x77eb001 thru 0x77f98e5 (0xe8e5 bytes) 2024-09-14T22:36:46.787 INFO:teuthology.orchestra.run.smithi001.stdout:1432 read 0x8a0bfe7 thru 0x8a170d5 (0xb0ef bytes) 2024-09-14T22:36:46.788 INFO:teuthology.orchestra.run.smithi001.stdout:1433 write 0x7564eac thru 0x756a55d (0x56b2 bytes) 2024-09-14T22:36:46.803 INFO:teuthology.orchestra.run.smithi001.stdout:1434 read 0x67470cb thru 0x675214a (0xb080 bytes) 2024-09-14T22:36:46.803 INFO:teuthology.orchestra.run.smithi001.stdout:1435 read 0xb0e36fe thru 0xb0f0721 (0xd024 bytes) 2024-09-14T22:36:46.804 INFO:teuthology.orchestra.run.smithi001.stdout:1436 read 0x1943e15 thru 0x1946b34 (0x2d20 bytes) 2024-09-14T22:36:46.805 INFO:teuthology.orchestra.run.smithi001.stdout:1440 read 0x81e6fd8 thru 0x81ebd96 (0x4dbf bytes) 2024-09-14T22:36:46.805 INFO:teuthology.orchestra.run.smithi001.stdout:1441 write 0x8a7cbc1 thru 0x8a7e292 (0x16d2 bytes) 2024-09-14T22:36:46.819 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:46.820 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:36:46.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.820 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:69714 3.1b 3:de4f1c2b:::rbd_object_map.113cb4c83968:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T22:36:46.820 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:46.817930+0000", 2024-09-14T22:36:46.820 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0016168459999999999, 2024-09-14T22:36:46.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0016375020000000001, 2024-09-14T22:36:46.820 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:46.820 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:46.820 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "waiting for sub ops", 2024-09-14T22:36:46.820 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:46.820 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:46.820 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:46.820 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 69714 2024-09-14T22:36:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:46.817930+0000", 2024-09-14T22:36:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:46.817930+0000", 2024-09-14T22:36:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:46.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:46.817931+0000", 2024-09-14T22:36:46.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1480000000000001e-06 2024-09-14T22:36:46.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:46.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:46.817938+0000", 2024-09-14T22:36:46.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9720000000000003e-06 2024-09-14T22:36:46.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:46.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:46.817950+0000", 2024-09-14T22:36:46.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2556e-05 2024-09-14T22:36:46.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:46.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:46.817958+0000", 2024-09-14T22:36:46.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2110000000000001e-06 2024-09-14T22:36:46.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:46.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:46.818014+0000", 2024-09-14T22:36:46.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6171000000000001e-05 2024-09-14T22:36:46.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:46.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:46.818078+0000", 2024-09-14T22:36:46.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4416000000000001e-05 2024-09-14T22:36:46.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 4", 2024-09-14T22:36:46.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:46.818329+0000", 2024-09-14T22:36:46.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025125400000000002 2024-09-14T22:36:46.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:46.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:46.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:36:46.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:46.819264+0000", 2024-09-14T22:36:46.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00093484999999999998 2024-09-14T22:36:46.824 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.824 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:46.824 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.825 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:46.825 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:36:46.825 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 1 2024-09-14T22:36:46.825 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:46.829 INFO:teuthology.orchestra.run.smithi001.stdout:1442 trunc from 0xc41b3d2 to 0x82b009d 2024-09-14T22:36:46.844 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:36:46.877 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:46.877 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:46.878 INFO:teuthology.orchestra.run.smithi001.stdout:1451 read 0x1e1816c thru 0x1e23e54 (0xbce9 bytes) 2024-09-14T22:36:46.878 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:46.874+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:46.878 INFO:teuthology.orchestra.run.smithi001.stdout:1452 punch from 0x1ca0e75 to 0x1ca2859, (0x19e4 bytes) 2024-09-14T22:36:46.879 INFO:teuthology.orchestra.run.smithi001.stdout:1454 punch from 0x4a774e6 to 0x4a8288a, (0xb3a4 bytes) 2024-09-14T22:36:46.879 INFO:teuthology.orchestra.run.smithi001.stdout:1455 write 0x918d37 thru 0x919300 (0x5ca bytes) 2024-09-14T22:36:46.891 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:46.891 INFO:teuthology.orchestra.run.smithi001.stdout:1457 writesame 0x69d357e thru 0x69d5291 data_size 0x2(0x1d14 bytes) 2024-09-14T22:36:46.894 INFO:teuthology.orchestra.run.smithi001.stdout:1458 punch from 0x6021c3e to 0x602fc81, (0xe043 bytes) 2024-09-14T22:36:46.894 INFO:teuthology.orchestra.run.smithi001.stdout:1459 writesame 0xe3a52f thru 0xe421b2 data_size 0x1a(0x7c84 bytes) 2024-09-14T22:36:46.905 INFO:teuthology.orchestra.run.smithi001.stdout:1460 punch from 0x3e6c98a to 0x3e6f7ac, (0x2e22 bytes) 2024-09-14T22:36:46.905 INFO:teuthology.orchestra.run.smithi001.stdout:1462 write 0x6fca0fe thru 0x6fd3d7b (0x9c7e bytes) 2024-09-14T22:36:46.907 INFO:teuthology.orchestra.run.smithi001.stdout:1465 write 0x25adeee thru 0x25b54e8 (0x75fb bytes) 2024-09-14T22:36:46.929 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:46.942 INFO:teuthology.orchestra.run.smithi001.stdout:1467 write 0xcaaa9f5 thru 0xcab6bf5 (0xc201 bytes) 2024-09-14T22:36:46.950 INFO:teuthology.orchestra.run.smithi001.stdout:1468 read 0xc9a4818 thru 0xc9acd87 (0x8570 bytes) 2024-09-14T22:36:46.955 INFO:teuthology.orchestra.run.smithi001.stdout:1469 trunc from 0xcab6bf6 to 0x9d3e5c3 2024-09-14T22:36:46.972 INFO:teuthology.orchestra.run.smithi001.stdout:1472 read 0x4e04a06 thru 0x4e088dc (0x3ed7 bytes) 2024-09-14T22:36:46.972 INFO:teuthology.orchestra.run.smithi001.stdout:1473 write 0x8f46d69 thru 0x8f4be8e (0x5126 bytes) 2024-09-14T22:36:46.974 INFO:teuthology.orchestra.run.smithi001.stdout:1474 write 0x6e9cf56 thru 0x6eab144 (0xe1ef bytes) 2024-09-14T22:36:46.978 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:46.974+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:46.979 INFO:teuthology.orchestra.run.smithi001.stdout:1475 trunc from 0x9d3e5c3 to 0xc06caae 2024-09-14T22:36:46.982 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T22:36:46.986 INFO:teuthology.orchestra.run.smithi001.stdout:1476 writesame 0x19f3c26 thru 0x19fa2cf data_size 0x11(0x66aa bytes) 2024-09-14T22:36:46.989 INFO:teuthology.orchestra.run.smithi001.stdout:1478 write 0x6b31ada thru 0x6b3ae36 (0x935d bytes) 2024-09-14T22:36:46.991 INFO:teuthology.orchestra.run.smithi001.stdout:1479 write 0x498994d thru 0x49984e4 (0xeb98 bytes) 2024-09-14T22:36:46.993 INFO:teuthology.orchestra.run.smithi001.stdout:1480 punch from 0x9c3bc2f to 0x9c3e371, (0x2742 bytes) 2024-09-14T22:36:46.993 INFO:teuthology.orchestra.run.smithi001.stdout:1481 read 0x9aaba66 thru 0x9aaddc9 (0x2364 bytes) 2024-09-14T22:36:46.993 INFO:teuthology.orchestra.run.smithi001.stdout:1482 punch from 0x66f5be2 to 0x66fb356, (0x5774 bytes) 2024-09-14T22:36:46.993 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:46.993 INFO:teuthology.orchestra.run.smithi001.stdout:1487 punch from 0x27c7904 to 0x27d56dc, (0xddd8 bytes) 2024-09-14T22:36:46.994 INFO:teuthology.orchestra.run.smithi001.stdout:1488 read 0x7fbd5c8 thru 0x7fc3d12 (0x674b bytes) 2024-09-14T22:36:46.994 INFO:teuthology.orchestra.run.smithi001.stdout:1489 punch from 0x68e423c to 0x68ee0a5, (0x9e69 bytes) 2024-09-14T22:36:46.994 INFO:teuthology.orchestra.run.smithi001.stdout:1491 writesame 0x796226e thru 0x796c934 data_size 0x1(0xa6c7 bytes) 2024-09-14T22:36:47.009 INFO:teuthology.orchestra.run.smithi001.stdout:1494 trunc from 0xc06caae to 0xbb63aa5 2024-09-14T22:36:47.026 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:47.026 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:36:47.026 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:36:47.026 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:36:47.026 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:47.031 INFO:teuthology.orchestra.run.smithi001.stdout:1495 write 0xed786c5 thru 0xed7957f (0xebb bytes) 2024-09-14T22:36:47.035 INFO:teuthology.orchestra.run.smithi001.stdout:1496 punch from 0xd5384b0 to 0xd53b412, (0x2f62 bytes) 2024-09-14T22:36:47.046 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:36:47.047 INFO:teuthology.orchestra.run.smithi001.stdout:1497 trunc from 0xed79580 to 0x7be8ad4 2024-09-14T22:36:47.079 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:47.075+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:47.081 INFO:teuthology.orchestra.run.smithi001.stdout:1498 writesame 0x9fa881a thru 0x9fa9d71 data_size 0x4(0x1558 bytes) 2024-09-14T22:36:47.094 INFO:teuthology.orchestra.run.smithi001.stdout:1499 write 0xb3b1cae thru 0xb3b1fe4 (0x337 bytes) 2024-09-14T22:36:47.101 INFO:teuthology.orchestra.run.smithi001.stdout:1500 read 0x1c4acc6 thru 0x1c54271 (0x95ac bytes) 2024-09-14T22:36:47.105 INFO:teuthology.orchestra.run.smithi001.stdout:1501 trunc from 0xb3b1fe5 to 0x85f7736 2024-09-14T22:36:47.140 INFO:teuthology.orchestra.run.smithi001.stdout:1502 write 0xad823f8 thru 0xad91e60 (0xfa69 bytes) 2024-09-14T22:36:47.159 INFO:teuthology.orchestra.run.smithi001.stdout:1503 write 0xd9fad36 thru 0xd9fcf8f (0x225a bytes) 2024-09-14T22:36:47.162 INFO:teuthology.orchestra.run.smithi001.stdout:1505 read 0x8fd5095 thru 0x8fd97ac (0x4718 bytes) 2024-09-14T22:36:47.163 INFO:teuthology.orchestra.run.smithi001.stdout:1509 read 0x5308d06 thru 0x531376c (0xaa67 bytes) 2024-09-14T22:36:47.164 INFO:teuthology.orchestra.run.smithi001.stdout:1510 write 0xb18be48 thru 0xb19be33 (0xffec bytes) 2024-09-14T22:36:47.173 INFO:teuthology.orchestra.run.smithi001.stdout:1512 trunc from 0xd9fcf90 to 0xd2237cf 2024-09-14T22:36:47.181 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:47.175+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:47.183 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:47.183 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:36:47.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:36:47.183 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:36:47.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.183 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2770 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14331904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:47.183 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.551455+0000", 2024-09-14T22:36:47.183 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.629990844000002, 2024-09-14T22:36:47.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028737275, 2024-09-14T22:36:47.183 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:47.183 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:47.183 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:47.183 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:47.184 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:47.184 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:47.184 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2770 2024-09-14T22:36:47.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.184 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:47.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.184 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:47.184 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.551455+0000", 2024-09-14T22:36:47.184 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.184 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:47.184 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.551455+0000", 2024-09-14T22:36:47.184 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.185 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:47.185 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.551455+0000", 2024-09-14T22:36:47.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.39e-07 2024-09-14T22:36:47.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.185 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:47.185 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.551457+0000", 2024-09-14T22:36:47.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2000000000000001e-06 2024-09-14T22:36:47.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.186 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:47.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.551464+0000", 2024-09-14T22:36:47.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3180000000000003e-06 2024-09-14T22:36:47.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.186 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:47.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.551476+0000", 2024-09-14T22:36:47.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2345e-05 2024-09-14T22:36:47.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:47.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.579963+0000", 2024-09-14T22:36:47.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028487545999999999 2024-09-14T22:36:47.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:47.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.580075+0000", 2024-09-14T22:36:47.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111292 2024-09-14T22:36:47.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:47.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.580192+0000", 2024-09-14T22:36:47.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000117135 2024-09-14T22:36:47.188 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.188 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:47.188 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.188 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3021 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14292992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:47.188 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559557+0000", 2024-09-14T22:36:47.188 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.621888238, 2024-09-14T22:36:47.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028720181000000001, 2024-09-14T22:36:47.189 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:47.189 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:47.189 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:47.189 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:47.189 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:47.189 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:47.189 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3021 2024-09-14T22:36:47.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.189 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:47.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:47.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559557+0000", 2024-09-14T22:36:47.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:47.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559557+0000", 2024-09-14T22:36:47.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:47.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559558+0000", 2024-09-14T22:36:47.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4499999999999996e-07 2024-09-14T22:36:47.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:47.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559562+0000", 2024-09-14T22:36:47.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.065e-06 2024-09-14T22:36:47.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.191 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:47.191 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559573+0000", 2024-09-14T22:36:47.191 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1398e-05 2024-09-14T22:36:47.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.191 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:47.191 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559582+0000", 2024-09-14T22:36:47.191 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6100000000000006e-06 2024-09-14T22:36:47.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.191 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:47.191 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588042+0000", 2024-09-14T22:36:47.191 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028460388999999999 2024-09-14T22:36:47.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:47.192 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588147+0000", 2024-09-14T22:36:47.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000104727 2024-09-14T22:36:47.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:47.192 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588277+0000", 2024-09-14T22:36:47.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000130147 2024-09-14T22:36:47.192 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.192 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:47.192 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.193 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3022 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14301184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:47.193 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559594+0000", 2024-09-14T22:36:47.193 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.621851847999999, 2024-09-14T22:36:47.193 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028916481000000001, 2024-09-14T22:36:47.193 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:47.193 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:47.193 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:47.193 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:47.193 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:47.193 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:47.193 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3022 2024-09-14T22:36:47.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.194 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:47.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:47.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559594+0000", 2024-09-14T22:36:47.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:47.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559594+0000", 2024-09-14T22:36:47.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:47.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559594+0000", 2024-09-14T22:36:47.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7000000000000004e-07 2024-09-14T22:36:47.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:47.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559604+0000", 2024-09-14T22:36:47.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6020000000000006e-06 2024-09-14T22:36:47.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:47.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559615+0000", 2024-09-14T22:36:47.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1032999999999999e-05 2024-09-14T22:36:47.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:47.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559638+0000", 2024-09-14T22:36:47.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3255999999999999e-05 2024-09-14T22:36:47.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:47.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588323+0000", 2024-09-14T22:36:47.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.02868447 2024-09-14T22:36:47.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:47.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588443+0000", 2024-09-14T22:36:47.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120056 2024-09-14T22:36:47.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:47.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588510+0000", 2024-09-14T22:36:47.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7393999999999999e-05 2024-09-14T22:36:47.197 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.197 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:47.197 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.197 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3023 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14309376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:47.197 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559651+0000", 2024-09-14T22:36:47.197 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.621794361999999, 2024-09-14T22:36:47.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029146231000000002, 2024-09-14T22:36:47.197 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:47.197 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:47.197 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:47.197 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:47.198 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:47.198 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:47.198 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3023 2024-09-14T22:36:47.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.198 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:47.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:47.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559651+0000", 2024-09-14T22:36:47.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:47.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559651+0000", 2024-09-14T22:36:47.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:47.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559652+0000", 2024-09-14T22:36:47.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2699999999999998e-07 2024-09-14T22:36:47.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:47.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559656+0000", 2024-09-14T22:36:47.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7019999999999999e-06 2024-09-14T22:36:47.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:47.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559666+0000", 2024-09-14T22:36:47.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0400999999999999e-05 2024-09-14T22:36:47.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:47.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559674+0000", 2024-09-14T22:36:47.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8890000000000002e-06 2024-09-14T22:36:47.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:47.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588565+0000", 2024-09-14T22:36:47.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028891217 2024-09-14T22:36:47.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:47.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588679+0000", 2024-09-14T22:36:47.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114058 2024-09-14T22:36:47.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:47.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588797+0000", 2024-09-14T22:36:47.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118137 2024-09-14T22:36:47.201 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.201 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:47.201 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.202 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3024 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14317568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:47.202 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559685+0000", 2024-09-14T22:36:47.202 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.621760251000001, 2024-09-14T22:36:47.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029326498999999999, 2024-09-14T22:36:47.202 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:47.202 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:47.202 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:47.202 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:47.202 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:47.202 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:47.202 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3024 2024-09-14T22:36:47.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.202 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559685+0000", 2024-09-14T22:36:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559685+0000", 2024-09-14T22:36:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559686+0000", 2024-09-14T22:36:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0999999999999998e-07 2024-09-14T22:36:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:47.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559697+0000", 2024-09-14T22:36:47.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.147e-05 2024-09-14T22:36:47.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:47.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559708+0000", 2024-09-14T22:36:47.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0546e-05 2024-09-14T22:36:47.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:47.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559716+0000", 2024-09-14T22:36:47.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.704e-06 2024-09-14T22:36:47.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:47.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588833+0000", 2024-09-14T22:36:47.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029116442999999999 2024-09-14T22:36:47.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:47.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588920+0000", 2024-09-14T22:36:47.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6916999999999995e-05 2024-09-14T22:36:47.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.206 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:47.206 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589012+0000", 2024-09-14T22:36:47.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1809000000000001e-05 2024-09-14T22:36:47.206 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.206 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:47.206 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.206 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3025 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14325760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:47.206 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559729+0000", 2024-09-14T22:36:47.206 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.621716352, 2024-09-14T22:36:47.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029422501, 2024-09-14T22:36:47.206 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:47.207 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:47.207 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:47.207 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:47.207 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:47.207 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:47.207 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3025 2024-09-14T22:36:47.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.207 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:47.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:47.207 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559729+0000", 2024-09-14T22:36:47.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:47.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559729+0000", 2024-09-14T22:36:47.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:47.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559730+0000", 2024-09-14T22:36:47.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8100000000000002e-07 2024-09-14T22:36:47.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:47.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559733+0000", 2024-09-14T22:36:47.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4740000000000001e-06 2024-09-14T22:36:47.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:47.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559744+0000", 2024-09-14T22:36:47.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0489000000000001e-05 2024-09-14T22:36:47.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:47.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559760+0000", 2024-09-14T22:36:47.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5817000000000001e-05 2024-09-14T22:36:47.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:47.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589050+0000", 2024-09-14T22:36:47.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.02929056 2024-09-14T22:36:47.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:47.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589101+0000", 2024-09-14T22:36:47.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0992000000000002e-05 2024-09-14T22:36:47.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:47.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589152+0000", 2024-09-14T22:36:47.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0387999999999999e-05 2024-09-14T22:36:47.210 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.210 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:47.210 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.210 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3026 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14333952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:47.211 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559795+0000", 2024-09-14T22:36:47.211 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.621650528, 2024-09-14T22:36:47.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029483812000000002, 2024-09-14T22:36:47.211 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:47.211 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:47.211 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:47.211 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:47.211 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:47.211 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:47.211 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3026 2024-09-14T22:36:47.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.211 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:47.211 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.211 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:47.211 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559795+0000", 2024-09-14T22:36:47.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.212 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:47.212 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559795+0000", 2024-09-14T22:36:47.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.212 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:47.212 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559796+0000", 2024-09-14T22:36:47.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3200000000000003e-07 2024-09-14T22:36:47.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.212 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:47.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559800+0000", 2024-09-14T22:36:47.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2930000000000002e-06 2024-09-14T22:36:47.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:47.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559812+0000", 2024-09-14T22:36:47.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2053e-05 2024-09-14T22:36:47.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:47.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559821+0000", 2024-09-14T22:36:47.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8869999999999998e-06 2024-09-14T22:36:47.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:47.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589177+0000", 2024-09-14T22:36:47.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029356308000000001 2024-09-14T22:36:47.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:47.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589229+0000", 2024-09-14T22:36:47.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1326000000000003e-05 2024-09-14T22:36:47.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:47.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589279+0000", 2024-09-14T22:36:47.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0012999999999997e-05 2024-09-14T22:36:47.214 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.214 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:47.215 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.215 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3027 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14342144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:47.215 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559833+0000", 2024-09-14T22:36:47.215 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.621612800000001, 2024-09-14T22:36:47.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029619187000000002, 2024-09-14T22:36:47.215 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:47.215 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:47.215 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:47.215 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:47.215 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:47.215 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:47.215 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3027 2024-09-14T22:36:47.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.216 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:47.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:47.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559833+0000", 2024-09-14T22:36:47.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:47.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559833+0000", 2024-09-14T22:36:47.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:47.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559833+0000", 2024-09-14T22:36:47.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9400000000000005e-07 2024-09-14T22:36:47.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.217 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:47.217 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559845+0000", 2024-09-14T22:36:47.217 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1623e-05 2024-09-14T22:36:47.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.217 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:47.217 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559856+0000", 2024-09-14T22:36:47.217 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0832999999999999e-05 2024-09-14T22:36:47.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.217 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:47.217 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559864+0000", 2024-09-14T22:36:47.217 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3119999999999994e-06 2024-09-14T22:36:47.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:47.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589299+0000", 2024-09-14T22:36:47.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029434699000000002 2024-09-14T22:36:47.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:47.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589354+0000", 2024-09-14T22:36:47.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5467999999999998e-05 2024-09-14T22:36:47.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:47.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589452+0000", 2024-09-14T22:36:47.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7558000000000003e-05 2024-09-14T22:36:47.219 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.219 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:47.219 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.219 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3028 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14350336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:47.219 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559875+0000", 2024-09-14T22:36:47.219 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.621569911999998, 2024-09-14T22:36:47.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029775586999999999, 2024-09-14T22:36:47.219 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:47.219 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:47.219 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:47.220 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:47.220 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:47.220 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:47.220 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3028 2024-09-14T22:36:47.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.220 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:47.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:47.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559875+0000", 2024-09-14T22:36:47.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:47.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559875+0000", 2024-09-14T22:36:47.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.221 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:47.221 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559876+0000", 2024-09-14T22:36:47.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5199999999999996e-07 2024-09-14T22:36:47.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.221 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:47.221 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559879+0000", 2024-09-14T22:36:47.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.095e-06 2024-09-14T22:36:47.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.221 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:47.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559889+0000", 2024-09-14T22:36:47.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0019e-05 2024-09-14T22:36:47.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:47.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559897+0000", 2024-09-14T22:36:47.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9189999999999999e-06 2024-09-14T22:36:47.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:47.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589486+0000", 2024-09-14T22:36:47.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029588482999999999 2024-09-14T22:36:47.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:47.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589578+0000", 2024-09-14T22:36:47.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2329000000000003e-05 2024-09-14T22:36:47.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:47.223 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589651+0000", 2024-09-14T22:36:47.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2990000000000004e-05 2024-09-14T22:36:47.223 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.223 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:47.223 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.223 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4068 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13252608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:47.223 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680102+0000", 2024-09-14T22:36:47.223 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.501343092999999, 2024-09-14T22:36:47.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028907656, 2024-09-14T22:36:47.224 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:47.224 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:47.224 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:47.224 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:47.224 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:47.224 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:47.224 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4068 2024-09-14T22:36:47.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.224 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:47.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:47.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680102+0000", 2024-09-14T22:36:47.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:47.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680102+0000", 2024-09-14T22:36:47.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:47.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680103+0000", 2024-09-14T22:36:47.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6600000000000002e-07 2024-09-14T22:36:47.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:47.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680105+0000", 2024-09-14T22:36:47.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.616e-06 2024-09-14T22:36:47.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:47.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680114+0000", 2024-09-14T22:36:47.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4120000000000001e-06 2024-09-14T22:36:47.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:47.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680119+0000", 2024-09-14T22:36:47.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6210000000000001e-06 2024-09-14T22:36:47.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:47.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.708813+0000", 2024-09-14T22:36:47.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028693330999999999 2024-09-14T22:36:47.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:47.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.708903+0000", 2024-09-14T22:36:47.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0297000000000006e-05 2024-09-14T22:36:47.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:47.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709010+0000", 2024-09-14T22:36:47.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106913 2024-09-14T22:36:47.228 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.228 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:47.228 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.228 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4069 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13260800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:47.228 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680128+0000", 2024-09-14T22:36:47.228 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.501317818, 2024-09-14T22:36:47.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029145476, 2024-09-14T22:36:47.228 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:47.228 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:47.229 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:47.229 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:47.229 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:47.229 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:47.229 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4069 2024-09-14T22:36:47.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.229 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:47.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:47.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680128+0000", 2024-09-14T22:36:47.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:47.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680128+0000", 2024-09-14T22:36:47.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:47.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680128+0000", 2024-09-14T22:36:47.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7599999999999998e-07 2024-09-14T22:36:47.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:47.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680131+0000", 2024-09-14T22:36:47.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9050000000000001e-06 2024-09-14T22:36:47.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:47.231 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680138+0000", 2024-09-14T22:36:47.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2559999999999996e-06 2024-09-14T22:36:47.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:47.231 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680144+0000", 2024-09-14T22:36:47.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4779999999999999e-06 2024-09-14T22:36:47.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:47.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709054+0000", 2024-09-14T22:36:47.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028910207 2024-09-14T22:36:47.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:47.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709180+0000", 2024-09-14T22:36:47.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012608699999999999 2024-09-14T22:36:47.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:47.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709273+0000", 2024-09-14T22:36:47.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3166999999999998e-05 2024-09-14T22:36:47.232 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.232 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:47.233 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.233 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4070 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13268992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:47.233 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680151+0000", 2024-09-14T22:36:47.233 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.501294536, 2024-09-14T22:36:47.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029327986, 2024-09-14T22:36:47.233 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:47.233 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:47.233 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:47.233 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:47.233 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:47.233 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:47.233 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4070 2024-09-14T22:36:47.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.234 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:47.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:47.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680151+0000", 2024-09-14T22:36:47.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:47.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680151+0000", 2024-09-14T22:36:47.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:47.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680151+0000", 2024-09-14T22:36:47.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:36:47.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:47.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680154+0000", 2024-09-14T22:36:47.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3350000000000001e-06 2024-09-14T22:36:47.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:47.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680162+0000", 2024-09-14T22:36:47.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.473e-06 2024-09-14T22:36:47.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:47.236 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680177+0000", 2024-09-14T22:36:47.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.465e-05 2024-09-14T22:36:47.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:47.236 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709356+0000", 2024-09-14T22:36:47.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029178744 2024-09-14T22:36:47.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:47.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709428+0000", 2024-09-14T22:36:47.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2428000000000005e-05 2024-09-14T22:36:47.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:47.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709479+0000", 2024-09-14T22:36:47.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0915999999999998e-05 2024-09-14T22:36:47.237 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.237 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:47.237 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.238 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4071 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13277184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:47.238 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680185+0000", 2024-09-14T22:36:47.238 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.501260036000001, 2024-09-14T22:36:47.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029452392000000001, 2024-09-14T22:36:47.238 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:47.238 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:47.238 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:47.238 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:47.238 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:47.238 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:47.238 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4071 2024-09-14T22:36:47.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.238 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:47.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:47.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680185+0000", 2024-09-14T22:36:47.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:47.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680185+0000", 2024-09-14T22:36:47.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:47.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680186+0000", 2024-09-14T22:36:47.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6100000000000001e-07 2024-09-14T22:36:47.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:47.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680188+0000", 2024-09-14T22:36:47.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.509e-06 2024-09-14T22:36:47.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:47.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680197+0000", 2024-09-14T22:36:47.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8550000000000008e-06 2024-09-14T22:36:47.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:47.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680203+0000", 2024-09-14T22:36:47.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5849999999999999e-06 2024-09-14T22:36:47.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:47.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709514+0000", 2024-09-14T22:36:47.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029311054 2024-09-14T22:36:47.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:47.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709585+0000", 2024-09-14T22:36:47.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0875000000000002e-05 2024-09-14T22:36:47.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:47.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709638+0000", 2024-09-14T22:36:47.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3053000000000002e-05 2024-09-14T22:36:47.242 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.242 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:47.242 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.242 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4072 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13285376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:47.242 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680211+0000", 2024-09-14T22:36:47.242 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.501234754999999, 2024-09-14T22:36:47.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029591177999999999, 2024-09-14T22:36:47.242 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:47.242 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:47.242 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:47.243 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:47.243 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:47.243 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:47.243 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4072 2024-09-14T22:36:47.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.243 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:47.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.243 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:47.243 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680211+0000", 2024-09-14T22:36:47.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:47.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680211+0000", 2024-09-14T22:36:47.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:47.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680211+0000", 2024-09-14T22:36:47.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6699999999999999e-07 2024-09-14T22:36:47.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:47.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680214+0000", 2024-09-14T22:36:47.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0149999999999999e-06 2024-09-14T22:36:47.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:47.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680222+0000", 2024-09-14T22:36:47.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.371e-06 2024-09-14T22:36:47.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:47.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680228+0000", 2024-09-14T22:36:47.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6230000000000002e-06 2024-09-14T22:36:47.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:47.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709670+0000", 2024-09-14T22:36:47.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029441694000000001 2024-09-14T22:36:47.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.246 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:47.246 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709741+0000", 2024-09-14T22:36:47.246 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1668999999999994e-05 2024-09-14T22:36:47.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.246 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:47.246 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709802+0000", 2024-09-14T22:36:47.246 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0338999999999999e-05 2024-09-14T22:36:47.246 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.246 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:47.246 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.246 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4073 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13293568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:47.247 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680236+0000", 2024-09-14T22:36:47.247 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.501209396, 2024-09-14T22:36:47.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029755238999999999, 2024-09-14T22:36:47.247 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:47.247 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:47.247 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:47.247 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:47.247 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:47.247 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:47.247 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4073 2024-09-14T22:36:47.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.247 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:47.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:47.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680236+0000", 2024-09-14T22:36:47.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:47.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680236+0000", 2024-09-14T22:36:47.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:47.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680236+0000", 2024-09-14T22:36:47.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5499999999999998e-07 2024-09-14T22:36:47.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:47.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680239+0000", 2024-09-14T22:36:47.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4380000000000002e-06 2024-09-14T22:36:47.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:47.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680248+0000", 2024-09-14T22:36:47.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6319999999999997e-06 2024-09-14T22:36:47.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:47.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680261+0000", 2024-09-14T22:36:47.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3193000000000001e-05 2024-09-14T22:36:47.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.250 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:47.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709831+0000", 2024-09-14T22:36:47.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029570486 2024-09-14T22:36:47.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.250 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:47.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709883+0000", 2024-09-14T22:36:47.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2290999999999998e-05 2024-09-14T22:36:47.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.251 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:47.251 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709991+0000", 2024-09-14T22:36:47.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107744 2024-09-14T22:36:47.251 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.251 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:47.251 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.251 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4074 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13301760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:47.251 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680269+0000", 2024-09-14T22:36:47.251 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.501176498, 2024-09-14T22:36:47.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030007875999999999, 2024-09-14T22:36:47.251 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:47.251 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:47.252 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:47.252 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:47.252 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:47.252 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:47.252 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4074 2024-09-14T22:36:47.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.252 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:47.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.252 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:47.252 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680269+0000", 2024-09-14T22:36:47.252 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:47.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680269+0000", 2024-09-14T22:36:47.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:47.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680269+0000", 2024-09-14T22:36:47.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7300000000000001e-07 2024-09-14T22:36:47.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:47.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680277+0000", 2024-09-14T22:36:47.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8480000000000001e-06 2024-09-14T22:36:47.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:47.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680286+0000", 2024-09-14T22:36:47.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0529999999999996e-06 2024-09-14T22:36:47.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:47.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680292+0000", 2024-09-14T22:36:47.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.728e-06 2024-09-14T22:36:47.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:47.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.710024+0000", 2024-09-14T22:36:47.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029732176999999999 2024-09-14T22:36:47.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:47.255 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.710109+0000", 2024-09-14T22:36:47.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4715000000000005e-05 2024-09-14T22:36:47.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:47.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.710277+0000", 2024-09-14T22:36:47.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000167882 2024-09-14T22:36:47.256 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.256 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:47.256 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.256 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:12663 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 374784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T22:36:47.256 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:28.325542+0000", 2024-09-14T22:36:47.256 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.855903487999999, 2024-09-14T22:36:47.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028843810000000001, 2024-09-14T22:36:47.257 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:47.257 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:47.257 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:47.257 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:47.257 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:47.257 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:47.257 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12663 2024-09-14T22:36:47.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.257 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:47.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:47.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325542+0000", 2024-09-14T22:36:47.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:47.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325542+0000", 2024-09-14T22:36:47.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:47.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325542+0000", 2024-09-14T22:36:47.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.44e-07 2024-09-14T22:36:47.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:47.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325545+0000", 2024-09-14T22:36:47.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.796e-06 2024-09-14T22:36:47.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.259 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:47.259 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325553+0000", 2024-09-14T22:36:47.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8569999999999995e-06 2024-09-14T22:36:47.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.259 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:47.259 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325560+0000", 2024-09-14T22:36:47.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5379999999999999e-06 2024-09-14T22:36:47.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.259 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:47.259 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354196+0000", 2024-09-14T22:36:47.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028635864 2024-09-14T22:36:47.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.260 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:47.260 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354280+0000", 2024-09-14T22:36:47.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4324000000000001e-05 2024-09-14T22:36:47.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.260 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:47.260 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354386+0000", 2024-09-14T22:36:47.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105887 2024-09-14T22:36:47.260 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.260 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:47.261 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.261 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:12664 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T22:36:47.261 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:28.325596+0000", 2024-09-14T22:36:47.261 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.855848984000001, 2024-09-14T22:36:47.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028964581, 2024-09-14T22:36:47.261 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:47.261 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:47.261 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:47.261 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:47.261 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:47.261 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:47.261 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12664 2024-09-14T22:36:47.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.262 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:47.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:47.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325596+0000", 2024-09-14T22:36:47.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:47.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325596+0000", 2024-09-14T22:36:47.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:47.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325597+0000", 2024-09-14T22:36:47.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4300000000000002e-07 2024-09-14T22:36:47.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:47.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325600+0000", 2024-09-14T22:36:47.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2219999999999998e-06 2024-09-14T22:36:47.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:47.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325610+0000", 2024-09-14T22:36:47.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2650000000000002e-06 2024-09-14T22:36:47.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:47.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325617+0000", 2024-09-14T22:36:47.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1199999999999996e-06 2024-09-14T22:36:47.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:47.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354438+0000", 2024-09-14T22:36:47.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028821205999999999 2024-09-14T22:36:47.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:47.265 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354495+0000", 2024-09-14T22:36:47.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6752999999999997e-05 2024-09-14T22:36:47.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:47.265 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354561+0000", 2024-09-14T22:36:47.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6272000000000005e-05 2024-09-14T22:36:47.265 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.265 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:47.265 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.266 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:12665 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 391168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T22:36:47.266 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:28.325626+0000", 2024-09-14T22:36:47.266 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.855819662000002, 2024-09-14T22:36:47.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029080366, 2024-09-14T22:36:47.266 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:47.266 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:47.266 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:47.266 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:47.266 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:47.266 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:47.266 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12665 2024-09-14T22:36:47.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.266 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:47.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:47.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325626+0000", 2024-09-14T22:36:47.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:47.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325626+0000", 2024-09-14T22:36:47.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:47.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325626+0000", 2024-09-14T22:36:47.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.06e-07 2024-09-14T22:36:47.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:47.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325629+0000", 2024-09-14T22:36:47.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7149999999999998e-06 2024-09-14T22:36:47.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:47.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325637+0000", 2024-09-14T22:36:47.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7360000000000001e-06 2024-09-14T22:36:47.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:47.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325643+0000", 2024-09-14T22:36:47.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2480000000000001e-06 2024-09-14T22:36:47.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.269 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:47.269 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354589+0000", 2024-09-14T22:36:47.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028945538 2024-09-14T22:36:47.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.269 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:47.269 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354635+0000", 2024-09-14T22:36:47.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6374e-05 2024-09-14T22:36:47.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:47.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354706+0000", 2024-09-14T22:36:47.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1049000000000004e-05 2024-09-14T22:36:47.270 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.270 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:47.270 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.270 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:13927 3.1b 3:ddfd6241:::rbd_data.113cda0183a0.0000000000000009:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-14T22:36:47.270 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:31.537348+0000", 2024-09-14T22:36:47.270 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.644097062, 2024-09-14T22:36:47.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032156573000000001, 2024-09-14T22:36:47.270 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:47.270 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:47.270 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:47.271 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:47.271 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:47.271 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:47.271 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 13927 2024-09-14T22:36:47.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.271 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:47.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:47.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537348+0000", 2024-09-14T22:36:47.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:47.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537348+0000", 2024-09-14T22:36:47.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:47.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:47.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537349+0000", 2024-09-14T22:36:47.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.068e-06 2024-09-14T22:36:47.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:47.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537358+0000", 2024-09-14T22:36:47.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.653e-06 2024-09-14T22:36:47.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:47.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537374+0000", 2024-09-14T22:36:47.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5594e-05 2024-09-14T22:36:47.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.273 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:47.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537383+0000", 2024-09-14T22:36:47.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0929999999999992e-06 2024-09-14T22:36:47.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.273 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:47.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537429+0000", 2024-09-14T22:36:47.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5948000000000001e-05 2024-09-14T22:36:47.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:47.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537490+0000", 2024-09-14T22:36:47.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1002000000000003e-05 2024-09-14T22:36:47.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 4", 2024-09-14T22:36:47.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.542781+0000", 2024-09-14T22:36:47.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.005290861 2024-09-14T22:36:47.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:47.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.569350+0000", 2024-09-14T22:36:47.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026569299000000001 2024-09-14T22:36:47.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:36:47.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.569375+0000", 2024-09-14T22:36:47.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5448000000000001e-05 2024-09-14T22:36:47.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:47.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.569444+0000", 2024-09-14T22:36:47.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9184999999999996e-05 2024-09-14T22:36:47.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:47.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:47.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:47.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.569505+0000", 2024-09-14T22:36:47.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0421999999999998e-05 2024-09-14T22:36:47.276 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.276 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:47.276 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.276 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:47.276 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:47.276 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:47.277 INFO:teuthology.orchestra.run.smithi001.stdout:1513 write 0xaf3338c thru 0xaf4218d (0xee02 bytes) 2024-09-14T22:36:47.278 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:36:47.279 INFO:teuthology.orchestra.run.smithi001.stdout:1514 trunc from 0xd2237cf to 0x949473a 2024-09-14T22:36:47.279 INFO:teuthology.orchestra.run.smithi001.stdout:1515 writesame 0xb756bad thru 0xb75f916 data_size 0x17(0x8d6a bytes) 2024-09-14T22:36:47.279 INFO:teuthology.orchestra.run.smithi001.stdout:1516 read 0x32b67d6 thru 0x32b7196 (0x9c1 bytes) 2024-09-14T22:36:47.279 INFO:teuthology.orchestra.run.smithi001.stdout:1517 read 0x3f2b731 thru 0x3f3713c (0xba0c bytes) 2024-09-14T22:36:47.279 INFO:teuthology.orchestra.run.smithi001.stdout:1520 trunc from 0xb75f917 to 0x870e410 2024-09-14T22:36:47.279 INFO:teuthology.orchestra.run.smithi001.stdout:1522 write 0x6b0a161 thru 0x6b134f7 (0x9397 bytes) 2024-09-14T22:36:47.279 INFO:teuthology.orchestra.run.smithi001.stdout:1523 read 0xc16cc6 thru 0xc1b739 (0x4a74 bytes) 2024-09-14T22:36:47.279 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:47.279 INFO:teuthology.orchestra.run.smithi001.stdout:1525 writesame 0xb09e832 thru 0xb0a9f45 data_size 0x2(0xb714 bytes) 2024-09-14T22:36:47.281 INFO:teuthology.orchestra.run.smithi001.stdout:1527 trunc from 0xb0a9f46 to 0x2b1f629 2024-09-14T22:36:47.282 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:47.275+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:47.328 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:36:47.382 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:47.378+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:47.386 INFO:teuthology.orchestra.run.smithi001.stdout:1528 write 0xdeb3119 thru 0xdeb3213 (0xfb bytes) 2024-09-14T22:36:47.391 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:47.391 INFO:teuthology.orchestra.run.smithi001.stdout:1532 read 0xd687ef6 thru 0xd6965ca (0xe6d5 bytes) 2024-09-14T22:36:47.392 INFO:teuthology.orchestra.run.smithi001.stdout:1534 read 0x24466d thru 0x24555b (0xeef bytes) 2024-09-14T22:36:47.393 INFO:teuthology.orchestra.run.smithi001.stdout:1535 read 0x5fb2c84 thru 0x5fc095d (0xdcda bytes) 2024-09-14T22:36:47.393 INFO:teuthology.orchestra.run.smithi001.stdout:1536 punch from 0x50e2294 to 0x50eb5e0, (0x934c bytes) 2024-09-14T22:36:47.393 INFO:teuthology.orchestra.run.smithi001.stdout:1537 punch from 0x3346e31 to 0x33560f2, (0xf2c1 bytes) 2024-09-14T22:36:47.406 INFO:teuthology.orchestra.run.smithi001.stdout:1538 trunc from 0xdeb3214 to 0x2041c33 2024-09-14T22:36:47.411 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:47.411 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:36:47.411 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:36:47.411 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:47.427 INFO:teuthology.orchestra.run.smithi001.stdout:1539 read 0x1f84a70 thru 0x1f87af7 (0x3088 bytes) 2024-09-14T22:36:47.428 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:36:47.436 INFO:teuthology.orchestra.run.smithi001.stdout:1540 write 0x27dd7a9 thru 0x27e5f82 (0x87da bytes) 2024-09-14T22:36:47.455 INFO:teuthology.orchestra.run.smithi001.stdout:1542 write 0x89314d0 thru 0x8935f9b (0x4acc bytes) 2024-09-14T22:36:47.462 INFO:teuthology.orchestra.run.smithi001.stdout:1543 read 0x524c2e2 thru 0x52504e3 (0x4202 bytes) 2024-09-14T22:36:47.477 INFO:teuthology.orchestra.run.smithi001.stdout:1544 write 0xe51d450 thru 0xe524996 (0x7547 bytes) 2024-09-14T22:36:47.480 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:36:47.480 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:36:47.483 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:47.479+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:47.485 INFO:teuthology.orchestra.run.smithi001.stdout:1545 write 0xa38f430 thru 0xa39e165 (0xed36 bytes) 2024-09-14T22:36:47.491 INFO:teuthology.orchestra.run.smithi001.stdout:1546 read 0x40f5ad4 thru 0x40fd471 (0x799e bytes) 2024-09-14T22:36:47.502 INFO:teuthology.orchestra.run.smithi001.stdout:1547 trunc from 0xe524997 to 0x52fa47d 2024-09-14T22:36:47.525 INFO:teuthology.orchestra.run.smithi001.stdout:1548 writesame 0x34e2727 thru 0x34edba3 data_size 0x1(0xb47d bytes) 2024-09-14T22:36:47.555 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:47.555 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:36:47.556 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:36:47.556 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:36:47.556 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:47.569 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:36:47.585 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:47.579+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:47.597 INFO:teuthology.orchestra.run.smithi001.stdout:1549 writesame 0xe4684fe thru 0xe4756db data_size 0x1(0xd1de bytes) 2024-09-14T22:36:47.597 INFO:teuthology.orchestra.run.smithi001.stdout:1551 read 0xcfc8870 thru 0xcfcbf17 (0x36a8 bytes) 2024-09-14T22:36:47.598 INFO:teuthology.orchestra.run.smithi001.stdout:1555 read 0xe00d6fa thru 0xe0149d0 (0x72d7 bytes) 2024-09-14T22:36:47.598 INFO:teuthology.orchestra.run.smithi001.stdout:1556 write 0x9ebc2e7 thru 0x9ebde1e (0x1b38 bytes) 2024-09-14T22:36:47.607 INFO:teuthology.orchestra.run.smithi001.stdout:1558 trunc from 0xe4756dc to 0xd154b71 2024-09-14T22:36:47.618 INFO:teuthology.orchestra.run.smithi001.stdout:1560 writesame 0x91e072 thru 0x923d39 data_size 0x4(0x5cc8 bytes) 2024-09-14T22:36:47.624 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:47.627 INFO:teuthology.orchestra.run.smithi001.stdout:1563 trunc from 0xd154b71 to 0xa8cc527 2024-09-14T22:36:47.637 INFO:teuthology.orchestra.run.smithi001.stdout:1564 write 0x6c7707b thru 0x6c7ddef (0x6d75 bytes) 2024-09-14T22:36:47.640 INFO:teuthology.orchestra.run.smithi001.stdout:1565 writesame 0x2a45d72 thru 0x2a461d5 data_size 0x2(0x464 bytes) 2024-09-14T22:36:47.642 INFO:teuthology.orchestra.run.smithi001.stdout:1567 writesame 0x14cac4 thru 0x15a6da data_size 0x7(0xdc17 bytes) 2024-09-14T22:36:47.649 INFO:teuthology.orchestra.run.smithi001.stdout:1568 write 0x7e3f31a thru 0x7e47572 (0x8259 bytes) 2024-09-14T22:36:47.668 INFO:teuthology.orchestra.run.smithi001.stdout:1570 write 0xd234ccf thru 0xd2436a5 (0xe9d7 bytes) 2024-09-14T22:36:47.675 INFO:teuthology.orchestra.run.smithi001.stdout:1571 read 0x6a316e4 thru 0x6a326eb (0x1008 bytes) 2024-09-14T22:36:47.676 INFO:teuthology.orchestra.run.smithi001.stdout:1573 write 0x35d26bc thru 0x35dd051 (0xa996 bytes) 2024-09-14T22:36:47.679 INFO:teuthology.orchestra.run.smithi001.stdout:1577 write 0xef0c73 thru 0xeff2be (0xe64c bytes) 2024-09-14T22:36:47.682 INFO:teuthology.orchestra.run.smithi001.stdout:1578 writesame 0x1117965 thru 0x111ff1f data_size 0x29(0x85bb bytes) 2024-09-14T22:36:47.685 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:47.685 INFO:teuthology.orchestra.run.smithi001.stdout:1580 write 0x72a9e9 thru 0x73493e (0x9f56 bytes) 2024-09-14T22:36:47.686 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:47.682+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:47.689 INFO:teuthology.orchestra.run.smithi001.stdout:1581 write 0x4a58fa3 thru 0x4a61c1e (0x8c7c bytes) 2024-09-14T22:36:47.692 INFO:teuthology.orchestra.run.smithi001.stdout:1582 read 0x7dc43fb thru 0x7dd16ff (0xd305 bytes) 2024-09-14T22:36:47.692 INFO:teuthology.orchestra.run.smithi001.stdout:1583 write 0xbab9a70 thru 0xbabe567 (0x4af8 bytes) 2024-09-14T22:36:47.699 INFO:teuthology.orchestra.run.smithi001.stdout:1584 read 0x5e4de99 thru 0x5e4eb77 (0xcdf bytes) 2024-09-14T22:36:47.699 INFO:teuthology.orchestra.run.smithi001.stdout:1585 read 0xe45e5f thru 0xe52781 (0xc923 bytes) 2024-09-14T22:36:47.700 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:47.700 INFO:teuthology.orchestra.run.smithi001.stdout:1589 punch from 0x59adb4e to 0x59b36ec, (0x5b9e bytes) 2024-09-14T22:36:47.700 INFO:teuthology.orchestra.run.smithi001.stdout:1590 write 0x79fbcd2 thru 0x7a0b93a (0xfc69 bytes) 2024-09-14T22:36:47.706 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:47.706 INFO:teuthology.orchestra.run.smithi001.stdout:1592 write 0x1663172 thru 0x1672c54 (0xfae3 bytes) 2024-09-14T22:36:47.710 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:47.711 INFO:teuthology.orchestra.run.smithi001.stdout:1594 punch from 0xc4bccfb to 0xc4be133, (0x1438 bytes) 2024-09-14T22:36:47.711 INFO:teuthology.orchestra.run.smithi001.stdout:1595 read 0xc4f01c2 thru 0xc4fca2c (0xc86b bytes) 2024-09-14T22:36:47.716 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:36:47.716 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:36:47.716 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-14T22:36:47.721 INFO:teuthology.orchestra.run.smithi001.stdout:1596 trunc from 0xd2436a6 to 0x6e06150 2024-09-14T22:36:47.741 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:36:47.741 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:36:47.741 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:47.742 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:47.745 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:47.749 INFO:teuthology.orchestra.run.smithi001.stdout:1598 trunc from 0x6e06150 to 0x351d66a 2024-09-14T22:36:47.760 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:36:47.769 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:36:47.769 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:36:47.781 INFO:teuthology.orchestra.run.smithi001.stdout:1599 trunc from 0x351d66a to 0x969e333 2024-09-14T22:36:47.786 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:47.782+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:47.790 INFO:teuthology.orchestra.run.smithi001.stdout:1600 punch from 0x2139bab to 0x21471e7, (0xd63c bytes) 2024-09-14T22:36:47.790 INFO:teuthology.orchestra.run.smithi001.stdout:1601 punch from 0x158070 to 0x1590d3, (0x1063 bytes) 2024-09-14T22:36:47.790 INFO:teuthology.orchestra.run.smithi001.stdout:1602 read 0x421ac81 thru 0x42240d3 (0x9453 bytes) 2024-09-14T22:36:47.794 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:36:47.795 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:36:47.801 INFO:teuthology.orchestra.run.smithi001.stdout:1603 writesame 0xb151408 thru 0xb1591d6 data_size 0x2b(0x7dcf bytes) 2024-09-14T22:36:47.810 INFO:teuthology.orchestra.run.smithi001.stdout:1605 write 0xc41f1d3 thru 0xc42c63e (0xd46c bytes) 2024-09-14T22:36:47.819 INFO:teuthology.orchestra.run.smithi001.stdout:1608 write 0x66b69ac thru 0x66c5359 (0xe9ae bytes) 2024-09-14T22:36:47.821 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:36:47.821 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:36:47.827 INFO:teuthology.orchestra.run.smithi001.stdout:1610 writesame 0x9f12dca thru 0x9f1ffda data_size 0x1(0xd211 bytes) 2024-09-14T22:36:47.846 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:36:47.847 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:36:47.849 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:36:47.850 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:36:47.887 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:47.883+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:47.888 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:47.888 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:47.889 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:47.889 INFO:teuthology.orchestra.run.smithi001.stdout:1612 read 0x6f1d886 thru 0x6f2aa44 (0xd1bf bytes) 2024-09-14T22:36:47.890 INFO:teuthology.orchestra.run.smithi001.stdout:1614 writesame 0x6360888 thru 0x6362838 data_size 0x13(0x1fb1 bytes) 2024-09-14T22:36:47.896 INFO:teuthology.orchestra.run.smithi001.stdout:1615 punch from 0x6b4863 to 0x6bf3aa, (0xab47 bytes) 2024-09-14T22:36:47.896 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:47.898 INFO:teuthology.orchestra.run.smithi001.stdout:1617 trunc from 0xc42c63f to 0xa3b150f 2024-09-14T22:36:47.903 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:36:47.911 INFO:teuthology.orchestra.run.smithi001.stdout:1619 punch from 0x9e8e827 to 0x9e9a899, (0xc072 bytes) 2024-09-14T22:36:47.911 INFO:teuthology.orchestra.run.smithi001.stdout:1620 read 0x4bdd8d9 thru 0x4be129c (0x39c4 bytes) 2024-09-14T22:36:47.912 INFO:teuthology.orchestra.run.smithi001.stdout:1621 write 0x12ccfd3 thru 0x12dcc42 (0xfc70 bytes) 2024-09-14T22:36:47.915 INFO:teuthology.orchestra.run.smithi001.stdout:1622 punch from 0x21ff1c3 to 0x2201cf1, (0x2b2e bytes) 2024-09-14T22:36:47.926 INFO:teuthology.orchestra.run.smithi001.stdout:1623 write 0xc3cea24 thru 0xc3d339b (0x4978 bytes) 2024-09-14T22:36:47.934 INFO:teuthology.orchestra.run.smithi001.stdout:1624 writesame 0x118c441 thru 0x1194b05 data_size 0x1(0x86c5 bytes) 2024-09-14T22:36:47.959 INFO:teuthology.orchestra.run.smithi001.stdout:1625 read 0x1b311 thru 0x247fc (0x94ec bytes) 2024-09-14T22:36:47.969 INFO:teuthology.orchestra.run.smithi001.stderr:set 3.2 pg_upmap_items mapping to [1->0,3->6] 2024-09-14T22:36:47.973 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2024-09-14T22:36:47.975 INFO:teuthology.orchestra.run.smithi001.stdout:1626 trunc from 0xc3d339c to 0x2792646 2024-09-14T22:36:47.988 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:47.983+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:48.040 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:48.040 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:48.054 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:36:48.089 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:48.083+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:48.191 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:48.186+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:48.221 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:48.221 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:48.235 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:36:48.292 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:48.287+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:48.368 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:48.368 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:48.382 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:36:48.424 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:48.419+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:48.494 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:48.487+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:48.540 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:48.541 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:48.555 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:36:48.599 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:48.595+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:48.695 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:48.691+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:48.720 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:48.721 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:48.734 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:36:48.827 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:48.823+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:48.898 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:48.891+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:48.903 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:48.903 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:48.917 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:36:48.999 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:48.994+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:49.084 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:49.085 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:49.099 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:36:49.100 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:49.095+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:49.202 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:49.195+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:49.221 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:49.221 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:49.236 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:36:49.302 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:49.298+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:49.404 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:49.398+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:49.405 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:49.405 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:49.419 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:36:49.504 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:49.499+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:49.542 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:49.542 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:49.556 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:36:49.647 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:49.643+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:49.706 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:49.702+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:49.728 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:49.728 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:49.742 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:36:49.827 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:49.823+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:49.851 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:36:49.903 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:49.903 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:49.908 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:49.903+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:49.917 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:36:50.002 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T22:36:50.013 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:36:50.014 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:36:50.014 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:50.011+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:50.030 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:36:50.031 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:36:50.031 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-14T22:36:50.047 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:36:50.048 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:36:50.071 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:36:50.072 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:36:50.081 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:50.082 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:50.097 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:50.097 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:50.097 INFO:teuthology.orchestra.run.smithi001.stdout:1629 read 0xe68948 thru 0xe697ce (0xe87 bytes) 2024-09-14T22:36:50.098 INFO:teuthology.orchestra.run.smithi001.stdout:1630 read 0x122679f thru 0x12327a7 (0xc009 bytes) 2024-09-14T22:36:50.098 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:36:50.099 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:36:50.099 INFO:teuthology.orchestra.run.smithi001.stdout:1631 read 0x132b324 thru 0x132b88a (0x567 bytes) 2024-09-14T22:36:50.100 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:36:50.116 INFO:teuthology.orchestra.run.smithi001.stdout:1632 write 0x88636ff thru 0x8865a24 (0x2326 bytes) 2024-09-14T22:36:50.121 INFO:teuthology.orchestra.run.smithi001.stdout:1634 read 0x79dd907 thru 0x79df37e (0x1a78 bytes) 2024-09-14T22:36:50.127 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:36:50.127 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:36:50.137 INFO:teuthology.orchestra.run.smithi001.stdout:1635 writesame 0xc8ad510 thru 0xc8b46dd data_size 0x7(0x71ce bytes) 2024-09-14T22:36:50.138 INFO:teuthology.orchestra.run.smithi001.stdout:1636 writesame 0xa20465b thru 0xa20a30d data_size 0x1(0x5cb3 bytes) 2024-09-14T22:36:50.143 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:50.139+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:50.148 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:36:50.149 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:36:50.152 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:36:50.152 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:36:50.159 INFO:teuthology.orchestra.run.smithi001.stdout:1638 write 0xd064cc6 thru 0xd070921 (0xbc5c bytes) 2024-09-14T22:36:50.163 INFO:teuthology.orchestra.run.smithi001.stdout:1639 write 0x9c986b thru 0x9cb7c9 (0x1f5f bytes) 2024-09-14T22:36:50.164 INFO:teuthology.orchestra.run.smithi001.stdout:1640 write 0x3b1937c thru 0x3b1e939 (0x55be bytes) 2024-09-14T22:36:50.166 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:50.173 INFO:teuthology.orchestra.run.smithi001.stdout:1642 write 0xda329b0 thru 0xda4212b (0xf77c bytes) 2024-09-14T22:36:50.177 INFO:teuthology.orchestra.run.smithi001.stdout:1643 writesame 0xba983b9 thru 0xba9aa13 data_size 0x3(0x265b bytes) 2024-09-14T22:36:50.179 INFO:teuthology.orchestra.run.smithi001.stdout:1644 read 0x3946c93 thru 0x3952a1f (0xbd8d bytes) 2024-09-14T22:36:50.181 INFO:teuthology.orchestra.run.smithi001.stdout:1645 read 0x906b7ce thru 0x90709da (0x520d bytes) 2024-09-14T22:36:50.181 INFO:teuthology.orchestra.run.smithi001.stdout:1646 write 0x4f6979c thru 0x4f6b743 (0x1fa8 bytes) 2024-09-14T22:36:50.183 INFO:teuthology.orchestra.run.smithi001.stdout:1647 read 0x817e914 thru 0x81871db (0x88c8 bytes) 2024-09-14T22:36:50.183 INFO:teuthology.orchestra.run.smithi001.stdout:1648 write 0x9f74e91 thru 0x9f7a501 (0x5671 bytes) 2024-09-14T22:36:50.186 INFO:teuthology.orchestra.run.smithi001.stdout:1649 write 0x795be3 thru 0x7989c3 (0x2de1 bytes) 2024-09-14T22:36:50.189 INFO:teuthology.orchestra.run.smithi001.stdout:1650 trunc from 0xda4212c to 0xe7a2853 2024-09-14T22:36:50.195 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:50.196 INFO:teuthology.orchestra.run.smithi001.stdout:1652 write 0x60c6ec5 thru 0x60d6735 (0xf871 bytes) 2024-09-14T22:36:50.199 INFO:teuthology.orchestra.run.smithi001.stdout:1653 writesame 0x492f906 thru 0x4931961 data_size 0x13(0x205c bytes) 2024-09-14T22:36:50.201 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:50.201 INFO:teuthology.orchestra.run.smithi001.stdout:1657 write 0xcf33785 thru 0xcf3cdea (0x9666 bytes) 2024-09-14T22:36:50.203 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:50.203 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:50.203 INFO:teuthology.orchestra.run.smithi001.stdout:1660 punch from 0x63acaf4 to 0x63b7fbe, (0xb4ca bytes) 2024-09-14T22:36:50.203 INFO:teuthology.orchestra.run.smithi001.stdout:1661 write 0xba24b3e thru 0xba29462 (0x4925 bytes) 2024-09-14T22:36:50.208 INFO:teuthology.orchestra.run.smithi001.stdout:1662 trunc from 0xe7a2853 to 0xd2d5542 2024-09-14T22:36:50.211 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:50.207+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:50.216 INFO:teuthology.orchestra.run.smithi001.stdout:1663 write 0xbcb73d6 thru 0xbcbeb92 (0x77bd bytes) 2024-09-14T22:36:50.220 INFO:teuthology.orchestra.run.smithi001.stdout:1665 write 0xc3c50d1 thru 0xc3d2ee1 (0xde11 bytes) 2024-09-14T22:36:50.224 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:50.224 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:50.224 INFO:teuthology.orchestra.run.smithi001.stdout:1670 writesame 0x23be1 thru 0x31376 data_size 0x5(0xd796 bytes) 2024-09-14T22:36:50.230 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:50.230 INFO:teuthology.orchestra.run.smithi001.stdout:1673 write 0xb3be327 thru 0xb3c6c81 (0x895b bytes) 2024-09-14T22:36:50.234 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:50.234 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:36:50.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:36:50.234 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:36:50.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.234 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:36:50.234 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:36:50.234 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 50.224623059000002, 2024-09-14T22:36:50.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.398029152, 2024-09-14T22:36:50.235 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:50.235 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:50.235 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:50.235 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:50.235 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:36:50.235 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:36:50.235 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 1 2024-09-14T22:36:50.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.236 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:50.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:50.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:36:50.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:50.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:36:50.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:50.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006183+0000", 2024-09-14T22:36:50.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9059999999999998e-06 2024-09-14T22:36:50.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:50.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006198+0000", 2024-09-14T22:36:50.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.523e-05 2024-09-14T22:36:50.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:50.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006219+0000", 2024-09-14T22:36:50.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1342999999999999e-05 2024-09-14T22:36:50.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006240+0000", 2024-09-14T22:36:50.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1186999999999999e-05 2024-09-14T22:36:50.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:50.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006363+0000", 2024-09-14T22:36:50.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012258200000000001 2024-09-14T22:36:50.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:36:50.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006392+0000", 2024-09-14T22:36:50.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9607e-05 2024-09-14T22:36:50.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:50.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401084+0000", 2024-09-14T22:36:50.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.39469142699999998 2024-09-14T22:36:50.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:50.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401394+0000", 2024-09-14T22:36:50.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030987799999999998 2024-09-14T22:36:50.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:36:50.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401860+0000", 2024-09-14T22:36:50.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00046654299999999998 2024-09-14T22:36:50.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:50.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.403386+0000", 2024-09-14T22:36:50.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001525322 2024-09-14T22:36:50.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:50.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404042+0000", 2024-09-14T22:36:50.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065676099999999996 2024-09-14T22:36:50.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:50.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404136+0000", 2024-09-14T22:36:50.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4032999999999999e-05 2024-09-14T22:36:50.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:50.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404209+0000", 2024-09-14T22:36:50.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2333e-05 2024-09-14T22:36:50.241 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.241 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:50.242 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.242 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2079 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11401216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:50.242 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450429+0000", 2024-09-14T22:36:50.242 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.780373270999998, 2024-09-14T22:36:50.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055279107000000001, 2024-09-14T22:36:50.242 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:50.242 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:50.242 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:50.242 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:50.242 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:50.242 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.243 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2079 2024-09-14T22:36:50.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.243 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:50.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:50.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450429+0000", 2024-09-14T22:36:50.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:50.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450429+0000", 2024-09-14T22:36:50.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:50.244 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450431+0000", 2024-09-14T22:36:50.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1039999999999999e-06 2024-09-14T22:36:50.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:50.244 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450435+0000", 2024-09-14T22:36:50.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9300000000000002e-06 2024-09-14T22:36:50.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:50.244 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450455+0000", 2024-09-14T22:36:50.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9242999999999999e-05 2024-09-14T22:36:50.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450470+0000", 2024-09-14T22:36:50.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5523e-05 2024-09-14T22:36:50.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:50.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.505413+0000", 2024-09-14T22:36:50.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054943193000000001 2024-09-14T22:36:50.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:50.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.505538+0000", 2024-09-14T22:36:50.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012467199999999999 2024-09-14T22:36:50.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:50.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.505709+0000", 2024-09-14T22:36:50.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017044200000000001 2024-09-14T22:36:50.246 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.246 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:50.246 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.247 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2080 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11409408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:50.247 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450489+0000", 2024-09-14T22:36:50.247 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.780313524, 2024-09-14T22:36:50.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055572178, 2024-09-14T22:36:50.247 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:50.247 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:50.247 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:50.247 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:50.247 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:50.247 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.247 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2080 2024-09-14T22:36:50.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.247 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:50.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.248 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:50.248 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450489+0000", 2024-09-14T22:36:50.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.248 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:50.248 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450489+0000", 2024-09-14T22:36:50.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.248 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:50.249 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450490+0000", 2024-09-14T22:36:50.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.023e-06 2024-09-14T22:36:50.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.249 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:50.249 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450495+0000", 2024-09-14T22:36:50.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1399999999999999e-06 2024-09-14T22:36:50.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.249 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:50.249 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450512+0000", 2024-09-14T22:36:50.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6892999999999999e-05 2024-09-14T22:36:50.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450525+0000", 2024-09-14T22:36:50.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2877000000000001e-05 2024-09-14T22:36:50.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:50.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.505787+0000", 2024-09-14T22:36:50.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055261432999999999 2024-09-14T22:36:50.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:50.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.505899+0000", 2024-09-14T22:36:50.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112456 2024-09-14T22:36:50.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:50.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506061+0000", 2024-09-14T22:36:50.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000162356 2024-09-14T22:36:50.251 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.251 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:50.251 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.251 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2081 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11417600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:50.252 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450544+0000", 2024-09-14T22:36:50.252 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.780258204999999, 2024-09-14T22:36:50.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.05598185, 2024-09-14T22:36:50.252 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:50.252 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:50.252 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:50.252 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:50.252 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:50.252 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.252 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2081 2024-09-14T22:36:50.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.252 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:50.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:50.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450544+0000", 2024-09-14T22:36:50.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:50.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450544+0000", 2024-09-14T22:36:50.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:50.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450546+0000", 2024-09-14T22:36:50.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5370000000000001e-06 2024-09-14T22:36:50.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.254 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:50.254 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450553+0000", 2024-09-14T22:36:50.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8630000000000002e-06 2024-09-14T22:36:50.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.254 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:50.254 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450571+0000", 2024-09-14T22:36:50.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7657000000000001e-05 2024-09-14T22:36:50.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.254 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.254 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450585+0000", 2024-09-14T22:36:50.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4056000000000001e-05 2024-09-14T22:36:50.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:50.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506185+0000", 2024-09-14T22:36:50.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055599920999999997 2024-09-14T22:36:50.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:50.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506334+0000", 2024-09-14T22:36:50.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014919900000000001 2024-09-14T22:36:50.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:50.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506526+0000", 2024-09-14T22:36:50.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019261699999999999 2024-09-14T22:36:50.256 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.256 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:50.256 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.256 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2082 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11425792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:50.256 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450602+0000", 2024-09-14T22:36:50.256 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.780200478000001, 2024-09-14T22:36:50.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056238654999999999, 2024-09-14T22:36:50.256 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:50.256 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:50.257 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:50.257 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:50.257 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:50.257 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.257 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2082 2024-09-14T22:36:50.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.257 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:50.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:50.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450602+0000", 2024-09-14T22:36:50.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:50.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450602+0000", 2024-09-14T22:36:50.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:50.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450603+0000", 2024-09-14T22:36:50.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1060000000000001e-06 2024-09-14T22:36:50.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:50.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450608+0000", 2024-09-14T22:36:50.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1499999999999998e-06 2024-09-14T22:36:50.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:50.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450626+0000", 2024-09-14T22:36:50.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7484000000000001e-05 2024-09-14T22:36:50.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450639+0000", 2024-09-14T22:36:50.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3485999999999999e-05 2024-09-14T22:36:50.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:50.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506599+0000", 2024-09-14T22:36:50.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055959395000000002 2024-09-14T22:36:50.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:50.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506706+0000", 2024-09-14T22:36:50.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010703499999999999 2024-09-14T22:36:50.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:50.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506841+0000", 2024-09-14T22:36:50.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013499900000000001 2024-09-14T22:36:50.260 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.261 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:50.261 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.261 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2083 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11433984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:50.261 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450658+0000", 2024-09-14T22:36:50.261 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.780145052000002, 2024-09-14T22:36:50.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056459267, 2024-09-14T22:36:50.261 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:50.262 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:50.262 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:50.262 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:50.262 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:50.262 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.262 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2083 2024-09-14T22:36:50.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.262 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:50.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:50.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450658+0000", 2024-09-14T22:36:50.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.263 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:50.263 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450658+0000", 2024-09-14T22:36:50.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.263 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.263 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.263 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:50.263 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450659+0000", 2024-09-14T22:36:50.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T22:36:50.263 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.263 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.263 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:50.263 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450664+0000", 2024-09-14T22:36:50.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4700000000000001e-06 2024-09-14T22:36:50.263 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:50.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450684+0000", 2024-09-14T22:36:50.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.916e-05 2024-09-14T22:36:50.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450698+0000", 2024-09-14T22:36:50.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4100000000000001e-05 2024-09-14T22:36:50.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:50.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506881+0000", 2024-09-14T22:36:50.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056183250999999997 2024-09-14T22:36:50.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:50.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506982+0000", 2024-09-14T22:36:50.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010090900000000001 2024-09-14T22:36:50.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:50.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507117+0000", 2024-09-14T22:36:50.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013510400000000001 2024-09-14T22:36:50.265 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.265 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:50.265 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.266 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2084 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11442176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:50.266 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450716+0000", 2024-09-14T22:36:50.266 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.780087193, 2024-09-14T22:36:50.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056802328999999999, 2024-09-14T22:36:50.266 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:50.266 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:50.266 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:50.266 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:50.266 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:50.266 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.266 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2084 2024-09-14T22:36:50.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.267 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:50.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:50.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450716+0000", 2024-09-14T22:36:50.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:50.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450716+0000", 2024-09-14T22:36:50.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:50.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450717+0000", 2024-09-14T22:36:50.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.08e-06 2024-09-14T22:36:50.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.268 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:50.268 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450722+0000", 2024-09-14T22:36:50.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5099999999999998e-06 2024-09-14T22:36:50.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.268 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:50.268 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450740+0000", 2024-09-14T22:36:50.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7836999999999999e-05 2024-09-14T22:36:50.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450757+0000", 2024-09-14T22:36:50.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7513e-05 2024-09-14T22:36:50.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:50.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507224+0000", 2024-09-14T22:36:50.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056466954 2024-09-14T22:36:50.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:50.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507355+0000", 2024-09-14T22:36:50.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000130207 2024-09-14T22:36:50.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:50.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507518+0000", 2024-09-14T22:36:50.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016322799999999999 2024-09-14T22:36:50.270 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.270 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:50.270 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.270 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2085 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11450368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:50.270 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450774+0000", 2024-09-14T22:36:50.270 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.780028389999998, 2024-09-14T22:36:50.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057099952000000002, 2024-09-14T22:36:50.270 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:50.271 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:50.271 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:50.271 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:50.271 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:50.271 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.271 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2085 2024-09-14T22:36:50.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.271 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:50.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:50.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450774+0000", 2024-09-14T22:36:50.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.272 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:50.272 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450774+0000", 2024-09-14T22:36:50.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.272 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.272 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:50.272 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450775+0000", 2024-09-14T22:36:50.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1030000000000001e-06 2024-09-14T22:36:50.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.272 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.272 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:50.272 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450781+0000", 2024-09-14T22:36:50.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.22e-06 2024-09-14T22:36:50.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:50.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450798+0000", 2024-09-14T22:36:50.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7350000000000002e-05 2024-09-14T22:36:50.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450812+0000", 2024-09-14T22:36:50.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.358e-05 2024-09-14T22:36:50.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:50.274 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507576+0000", 2024-09-14T22:36:50.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056764468999999998 2024-09-14T22:36:50.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:50.274 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507714+0000", 2024-09-14T22:36:50.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013826800000000001 2024-09-14T22:36:50.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:50.274 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507874+0000", 2024-09-14T22:36:50.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015996200000000001 2024-09-14T22:36:50.274 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.274 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:50.275 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.275 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2086 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11458560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:50.275 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450830+0000", 2024-09-14T22:36:50.275 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.779973099999999, 2024-09-14T22:36:50.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057480495999999999, 2024-09-14T22:36:50.275 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:50.275 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:50.275 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:50.275 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:50.275 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:50.275 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.276 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2086 2024-09-14T22:36:50.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.276 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:50.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:50.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450830+0000", 2024-09-14T22:36:50.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:50.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450830+0000", 2024-09-14T22:36:50.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:50.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450831+0000", 2024-09-14T22:36:50.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0860000000000001e-06 2024-09-14T22:36:50.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:50.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450836+0000", 2024-09-14T22:36:50.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4999999999999999e-06 2024-09-14T22:36:50.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:50.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450853+0000", 2024-09-14T22:36:50.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7184e-05 2024-09-14T22:36:50.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450869+0000", 2024-09-14T22:36:50.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5965999999999999e-05 2024-09-14T22:36:50.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:50.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507972+0000", 2024-09-14T22:36:50.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057103095999999999 2024-09-14T22:36:50.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:50.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508119+0000", 2024-09-14T22:36:50.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014651200000000001 2024-09-14T22:36:50.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:50.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508310+0000", 2024-09-14T22:36:50.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019115199999999999 2024-09-14T22:36:50.279 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.279 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:50.279 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.279 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2087 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11466752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:50.280 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450888+0000", 2024-09-14T22:36:50.280 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.779915054, 2024-09-14T22:36:50.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057792806000000002, 2024-09-14T22:36:50.280 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:50.280 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:50.280 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:50.280 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:50.280 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:50.280 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.280 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2087 2024-09-14T22:36:50.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.280 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:50.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:50.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450888+0000", 2024-09-14T22:36:50.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:50.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450888+0000", 2024-09-14T22:36:50.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:50.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450889+0000", 2024-09-14T22:36:50.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4270000000000001e-06 2024-09-14T22:36:50.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:50.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450895+0000", 2024-09-14T22:36:50.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4700000000000001e-06 2024-09-14T22:36:50.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:50.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450913+0000", 2024-09-14T22:36:50.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7958999999999999e-05 2024-09-14T22:36:50.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450926+0000", 2024-09-14T22:36:50.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3842999999999999e-05 2024-09-14T22:36:50.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:50.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508394+0000", 2024-09-14T22:36:50.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057467795000000002 2024-09-14T22:36:50.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:50.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508516+0000", 2024-09-14T22:36:50.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000121822 2024-09-14T22:36:50.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:50.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508680+0000", 2024-09-14T22:36:50.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016448999999999999 2024-09-14T22:36:50.284 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.284 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:50.284 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.284 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2088 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11474944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:50.284 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450943+0000", 2024-09-14T22:36:50.284 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.779859440999999, 2024-09-14T22:36:50.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058110368000000003, 2024-09-14T22:36:50.284 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:50.285 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:50.285 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:50.285 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:50.285 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:50.285 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.285 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2088 2024-09-14T22:36:50.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.285 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:50.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.285 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:50.285 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450943+0000", 2024-09-14T22:36:50.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:50.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450943+0000", 2024-09-14T22:36:50.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:50.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450944+0000", 2024-09-14T22:36:50.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2100000000000001e-06 2024-09-14T22:36:50.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:50.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450950+0000", 2024-09-14T22:36:50.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3329999999999996e-06 2024-09-14T22:36:50.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.287 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:50.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450968+0000", 2024-09-14T22:36:50.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7802999999999999e-05 2024-09-14T22:36:50.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.287 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450981+0000", 2024-09-14T22:36:50.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.349e-05 2024-09-14T22:36:50.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.287 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:50.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508752+0000", 2024-09-14T22:36:50.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057770957999999997 2024-09-14T22:36:50.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:50.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508884+0000", 2024-09-14T22:36:50.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013186900000000001 2024-09-14T22:36:50.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:50.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509054+0000", 2024-09-14T22:36:50.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000169705 2024-09-14T22:36:50.288 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.288 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:50.288 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.289 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2089 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11483136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:50.289 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450998+0000", 2024-09-14T22:36:50.289 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.779804548000001, 2024-09-14T22:36:50.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058422566000000002, 2024-09-14T22:36:50.289 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:50.289 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:50.289 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:50.289 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:50.289 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:50.289 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.289 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2089 2024-09-14T22:36:50.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.289 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:50.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:50.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450998+0000", 2024-09-14T22:36:50.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:50.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450998+0000", 2024-09-14T22:36:50.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:50.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450999+0000", 2024-09-14T22:36:50.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0100000000000001e-06 2024-09-14T22:36:50.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:50.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451014+0000", 2024-09-14T22:36:50.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4486e-05 2024-09-14T22:36:50.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:50.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451032+0000", 2024-09-14T22:36:50.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7893e-05 2024-09-14T22:36:50.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451046+0000", 2024-09-14T22:36:50.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4783999999999999e-05 2024-09-14T22:36:50.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:50.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509131+0000", 2024-09-14T22:36:50.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058085070000000003 2024-09-14T22:36:50.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:50.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509253+0000", 2024-09-14T22:36:50.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012173299999999999 2024-09-14T22:36:50.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:50.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509421+0000", 2024-09-14T22:36:50.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016759000000000001 2024-09-14T22:36:50.293 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.293 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:50.293 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.293 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2090 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11491328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:50.293 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.451064+0000", 2024-09-14T22:36:50.293 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.779738772000002, 2024-09-14T22:36:50.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058723076999999999, 2024-09-14T22:36:50.294 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:50.294 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:50.294 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:50.294 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:50.294 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:50.294 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.294 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2090 2024-09-14T22:36:50.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.294 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:50.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:50.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451064+0000", 2024-09-14T22:36:50.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:50.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451064+0000", 2024-09-14T22:36:50.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:50.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451065+0000", 2024-09-14T22:36:50.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2160000000000001e-06 2024-09-14T22:36:50.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:50.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451070+0000", 2024-09-14T22:36:50.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1340000000000003e-06 2024-09-14T22:36:50.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:50.296 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451088+0000", 2024-09-14T22:36:50.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7476e-05 2024-09-14T22:36:50.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.296 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451101+0000", 2024-09-14T22:36:50.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3723999999999999e-05 2024-09-14T22:36:50.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:50.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509492+0000", 2024-09-14T22:36:50.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058390899000000003 2024-09-14T22:36:50.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:50.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509618+0000", 2024-09-14T22:36:50.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012593899999999999 2024-09-14T22:36:50.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:50.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509787+0000", 2024-09-14T22:36:50.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000168689 2024-09-14T22:36:50.297 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.298 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:50.298 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.298 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2091 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11499520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:50.298 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.451119+0000", 2024-09-14T22:36:50.298 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.779683719000001, 2024-09-14T22:36:50.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059045274000000002, 2024-09-14T22:36:50.298 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:50.298 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:50.299 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:50.299 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:50.299 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:50.299 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.299 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2091 2024-09-14T22:36:50.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.299 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:50.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.299 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:50.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451119+0000", 2024-09-14T22:36:50.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.300 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:50.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451119+0000", 2024-09-14T22:36:50.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.300 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:50.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451120+0000", 2024-09-14T22:36:50.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0669999999999999e-06 2024-09-14T22:36:50.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.300 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:50.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451125+0000", 2024-09-14T22:36:50.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3329999999999996e-06 2024-09-14T22:36:50.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:50.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451143+0000", 2024-09-14T22:36:50.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7309999999999999e-05 2024-09-14T22:36:50.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451157+0000", 2024-09-14T22:36:50.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3835999999999999e-05 2024-09-14T22:36:50.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:50.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509863+0000", 2024-09-14T22:36:50.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058706345 2024-09-14T22:36:50.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:50.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509985+0000", 2024-09-14T22:36:50.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000121736 2024-09-14T22:36:50.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:50.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510164+0000", 2024-09-14T22:36:50.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017964700000000001 2024-09-14T22:36:50.302 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.303 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:50.303 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.303 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2092 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11507712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:50.303 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.451202+0000", 2024-09-14T22:36:50.303 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.779601047, 2024-09-14T22:36:50.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059345031999999999, 2024-09-14T22:36:50.303 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:50.303 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:50.303 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:50.303 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:50.303 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:50.303 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.304 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2092 2024-09-14T22:36:50.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.304 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:50.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:50.304 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451202+0000", 2024-09-14T22:36:50.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:50.304 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451202+0000", 2024-09-14T22:36:50.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:50.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451203+0000", 2024-09-14T22:36:50.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6199999999999999e-06 2024-09-14T22:36:50.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:50.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451209+0000", 2024-09-14T22:36:50.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1999999999999999e-06 2024-09-14T22:36:50.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:50.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451230+0000", 2024-09-14T22:36:50.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0247000000000001e-05 2024-09-14T22:36:50.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451245+0000", 2024-09-14T22:36:50.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4993e-05 2024-09-14T22:36:50.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:50.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510239+0000", 2024-09-14T22:36:50.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058994321000000002 2024-09-14T22:36:50.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:50.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510371+0000", 2024-09-14T22:36:50.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013195899999999999 2024-09-14T22:36:50.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:50.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510547+0000", 2024-09-14T22:36:50.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000175692 2024-09-14T22:36:50.307 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.307 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:50.307 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.307 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2093 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11515904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:50.307 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.451264+0000", 2024-09-14T22:36:50.308 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.779539010000001, 2024-09-14T22:36:50.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059636293999999999, 2024-09-14T22:36:50.308 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:50.308 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:50.308 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:50.308 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:50.308 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:50.308 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.308 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2093 2024-09-14T22:36:50.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.308 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:50.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.308 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:50.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451264+0000", 2024-09-14T22:36:50.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:50.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451264+0000", 2024-09-14T22:36:50.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:50.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451265+0000", 2024-09-14T22:36:50.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3400000000000001e-06 2024-09-14T22:36:50.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:50.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451270+0000", 2024-09-14T22:36:50.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4360000000000001e-06 2024-09-14T22:36:50.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:50.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451289+0000", 2024-09-14T22:36:50.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8383e-05 2024-09-14T22:36:50.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451303+0000", 2024-09-14T22:36:50.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.455e-05 2024-09-14T22:36:50.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:50.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510627+0000", 2024-09-14T22:36:50.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059323280999999999 2024-09-14T22:36:50.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:50.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510745+0000", 2024-09-14T22:36:50.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011867900000000001 2024-09-14T22:36:50.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:50.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510900+0000", 2024-09-14T22:36:50.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000154625 2024-09-14T22:36:50.312 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.312 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:50.312 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.312 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2094 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11524096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:50.312 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.451321+0000", 2024-09-14T22:36:50.312 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.779482024, 2024-09-14T22:36:50.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059938827, 2024-09-14T22:36:50.312 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:50.312 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:50.313 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:50.313 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:50.313 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:50.313 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.313 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2094 2024-09-14T22:36:50.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.313 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:50.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:50.313 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451321+0000", 2024-09-14T22:36:50.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:50.314 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451321+0000", 2024-09-14T22:36:50.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:50.314 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451322+0000", 2024-09-14T22:36:50.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0529999999999999e-06 2024-09-14T22:36:50.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:50.314 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451328+0000", 2024-09-14T22:36:50.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8300000000000001e-06 2024-09-14T22:36:50.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:50.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451344+0000", 2024-09-14T22:36:50.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6750000000000001e-05 2024-09-14T22:36:50.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451358+0000", 2024-09-14T22:36:50.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3200000000000001e-05 2024-09-14T22:36:50.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:50.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510968+0000", 2024-09-14T22:36:50.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059610567000000003 2024-09-14T22:36:50.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:50.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.511094+0000", 2024-09-14T22:36:50.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012629199999999999 2024-09-14T22:36:50.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:50.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.511260+0000", 2024-09-14T22:36:50.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016513500000000001 2024-09-14T22:36:50.316 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.316 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:50.316 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.317 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2095 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11532288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:50.317 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.451375+0000", 2024-09-14T22:36:50.317 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.779427851000001, 2024-09-14T22:36:50.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060228067000000003, 2024-09-14T22:36:50.317 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:50.317 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:50.317 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:50.317 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:50.317 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:50.317 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.317 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2095 2024-09-14T22:36:50.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.317 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:50.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:50.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451375+0000", 2024-09-14T22:36:50.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:50.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451375+0000", 2024-09-14T22:36:50.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:50.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451376+0000", 2024-09-14T22:36:50.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0470000000000001e-06 2024-09-14T22:36:50.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:50.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451381+0000", 2024-09-14T22:36:50.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.536e-06 2024-09-14T22:36:50.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:50.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451399+0000", 2024-09-14T22:36:50.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7200000000000001e-05 2024-09-14T22:36:50.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451412+0000", 2024-09-14T22:36:50.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3726999999999999e-05 2024-09-14T22:36:50.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:50.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.511308+0000", 2024-09-14T22:36:50.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059895176000000001 2024-09-14T22:36:50.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:50.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.511444+0000", 2024-09-14T22:36:50.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013621600000000001 2024-09-14T22:36:50.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:50.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.511603+0000", 2024-09-14T22:36:50.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015916500000000001 2024-09-14T22:36:50.321 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.321 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:50.321 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.321 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:45459 3.11 3:8ce265cb:::rbd_data.113cc4e35b3.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:36:50.321 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:36.190430+0000", 2024-09-14T22:36:50.321 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.040373107000001, 2024-09-14T22:36:50.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.06125162, 2024-09-14T22:36:50.321 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:50.321 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:50.321 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:50.322 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:50.322 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:50.322 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.322 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 45459 2024-09-14T22:36:50.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.322 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:50.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:50.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190430+0000", 2024-09-14T22:36:50.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:50.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190430+0000", 2024-09-14T22:36:50.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:50.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190432+0000", 2024-09-14T22:36:50.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3039999999999999e-06 2024-09-14T22:36:50.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:50.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190443+0000", 2024-09-14T22:36:50.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1178999999999999e-05 2024-09-14T22:36:50.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:50.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190474+0000", 2024-09-14T22:36:50.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0803000000000003e-05 2024-09-14T22:36:50.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190489+0000", 2024-09-14T22:36:50.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.517e-05 2024-09-14T22:36:50.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:50.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190623+0000", 2024-09-14T22:36:50.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013405599999999999 2024-09-14T22:36:50.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:50.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190752+0000", 2024-09-14T22:36:50.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012870899999999999 2024-09-14T22:36:50.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:36:50.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.203210+0000", 2024-09-14T22:36:50.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012458484000000001 2024-09-14T22:36:50.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:50.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.251517+0000", 2024-09-14T22:36:50.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.048306453999999999 2024-09-14T22:36:50.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:50.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.251571+0000", 2024-09-14T22:36:50.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4228999999999999e-05 2024-09-14T22:36:50.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:50.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.251628+0000", 2024-09-14T22:36:50.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6922000000000002e-05 2024-09-14T22:36:50.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:50.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.251681+0000", 2024-09-14T22:36:50.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3310000000000003e-05 2024-09-14T22:36:50.327 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.327 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:50.327 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.327 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:45462 3.11 3:8ce265cb:::rbd_data.113cc4e35b3.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:36:50.327 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:36.258125+0000", 2024-09-14T22:36:50.327 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 13.972677529, 2024-09-14T22:36:50.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059222599000000001, 2024-09-14T22:36:50.327 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:50.327 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:50.327 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:50.328 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:50.328 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:50.328 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.328 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 45462 2024-09-14T22:36:50.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.328 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:50.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:50.328 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258125+0000", 2024-09-14T22:36:50.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:50.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258125+0000", 2024-09-14T22:36:50.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:50.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:50.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258127+0000", 2024-09-14T22:36:50.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3159999999999999e-06 2024-09-14T22:36:50.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:50.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258139+0000", 2024-09-14T22:36:50.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1667e-05 2024-09-14T22:36:50.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:50.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258170+0000", 2024-09-14T22:36:50.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0929999999999997e-05 2024-09-14T22:36:50.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258185+0000", 2024-09-14T22:36:50.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4423000000000001e-05 2024-09-14T22:36:50.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:50.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258291+0000", 2024-09-14T22:36:50.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106285 2024-09-14T22:36:50.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:50.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258446+0000", 2024-09-14T22:36:50.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015486499999999999 2024-09-14T22:36:50.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:36:50.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.269615+0000", 2024-09-14T22:36:50.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011169095 2024-09-14T22:36:50.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:50.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.317113+0000", 2024-09-14T22:36:50.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.047498382999999998 2024-09-14T22:36:50.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:50.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.317144+0000", 2024-09-14T22:36:50.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0970000000000003e-05 2024-09-14T22:36:50.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:50.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.317233+0000", 2024-09-14T22:36:50.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9205999999999999e-05 2024-09-14T22:36:50.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:50.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:50.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:50.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.317348+0000", 2024-09-14T22:36:50.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000114459 2024-09-14T22:36:50.333 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.333 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:50.333 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.333 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:50.333 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:50.333 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:50.335 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:50.327+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:50.336 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:36:50.338 INFO:teuthology.orchestra.run.smithi001.stdout:1674 writesame 0x2cbb67e thru 0x2cbce24 data_size 0x1(0x17a7 bytes) 2024-09-14T22:36:50.338 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:50.338 INFO:teuthology.orchestra.run.smithi001.stdout:1679 punch from 0x5311da9 to 0x53163ec, (0x4643 bytes) 2024-09-14T22:36:50.338 INFO:teuthology.orchestra.run.smithi001.stdout:1680 write 0x5dd6c0d thru 0x5dd725f (0x653 bytes) 2024-09-14T22:36:50.338 INFO:teuthology.orchestra.run.smithi001.stdout:1681 read 0x76559d1 thru 0x7659f31 (0x4561 bytes) 2024-09-14T22:36:50.338 INFO:teuthology.orchestra.run.smithi001.stdout:1682 writesame 0x2ae8bc9 thru 0x2aeba04 data_size 0x16(0x2e3c bytes) 2024-09-14T22:36:50.338 INFO:teuthology.orchestra.run.smithi001.stdout:1683 write 0x9f8675f thru 0x9f941eb (0xda8d bytes) 2024-09-14T22:36:50.338 INFO:teuthology.orchestra.run.smithi001.stdout:1684 read 0xec2357 thru 0xed0ce5 (0xe98f bytes) 2024-09-14T22:36:50.338 INFO:teuthology.orchestra.run.smithi001.stdout:1685 read 0x9421004 thru 0x942eb60 (0xdb5d bytes) 2024-09-14T22:36:50.338 INFO:teuthology.orchestra.run.smithi001.stdout:1686 write 0xd2a345e thru 0xd2a49f2 (0x1595 bytes) 2024-09-14T22:36:50.338 INFO:teuthology.orchestra.run.smithi001.stdout:1687 punch from 0x2255813 to 0x225b8af, (0x609c bytes) 2024-09-14T22:36:50.338 INFO:teuthology.orchestra.run.smithi001.stdout:1688 read 0xd16c604 thru 0xd179342 (0xcd3f bytes) 2024-09-14T22:36:50.339 INFO:teuthology.orchestra.run.smithi001.stdout:1689 read 0xbf56606 thru 0xbf5b8c5 (0x52c0 bytes) 2024-09-14T22:36:50.339 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:50.339 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:50.339 INFO:teuthology.orchestra.run.smithi001.stdout:1692 read 0x342b1c5 thru 0x3434b4a (0x9986 bytes) 2024-09-14T22:36:50.339 INFO:teuthology.orchestra.run.smithi001.stdout:1694 punch from 0xbe9e88f to 0xbea7d19, (0x948a bytes) 2024-09-14T22:36:50.339 INFO:teuthology.orchestra.run.smithi001.stdout:1695 read 0x47689da thru 0x476ea14 (0x603b bytes) 2024-09-14T22:36:50.339 INFO:teuthology.orchestra.run.smithi001.stdout:1696 read 0x9d713c3 thru 0x9d7c183 (0xadc1 bytes) 2024-09-14T22:36:50.339 INFO:teuthology.orchestra.run.smithi001.stdout:1697 write 0x80c9408 thru 0x80c9478 (0x71 bytes) 2024-09-14T22:36:50.339 INFO:teuthology.orchestra.run.smithi001.stdout:1698 punch from 0x50ed410 to 0x50f1987, (0x4577 bytes) 2024-09-14T22:36:50.339 INFO:teuthology.orchestra.run.smithi001.stdout:1699 trunc from 0xd2d5542 to 0x1079292 2024-09-14T22:36:50.339 INFO:teuthology.orchestra.run.smithi001.stdout:1700 punch from 0x94473c to 0x946ec5, (0x2789 bytes) 2024-09-14T22:36:50.339 INFO:teuthology.orchestra.run.smithi001.stdout:1701 read 0xa65622 thru 0xa72d6e (0xd74d bytes) 2024-09-14T22:36:50.339 INFO:teuthology.orchestra.run.smithi001.stdout:1702 trunc from 0x1079292 to 0x60cc9bf 2024-09-14T22:36:50.339 INFO:teuthology.orchestra.run.smithi001.stdout:1704 write 0xa0514d5 thru 0xa05a223 (0x8d4f bytes) 2024-09-14T22:36:50.350 INFO:teuthology.orchestra.run.smithi001.stdout:1705 write 0xa54f891 thru 0xa5585be (0x8d2e bytes) 2024-09-14T22:36:50.355 INFO:teuthology.orchestra.run.smithi001.stdout:1706 write 0x35955 thru 0x3c2d3 (0x697f bytes) 2024-09-14T22:36:50.358 INFO:teuthology.orchestra.run.smithi001.stdout:1707 read 0x1955aab thru 0x195e1ac (0x8702 bytes) 2024-09-14T22:36:50.359 INFO:teuthology.orchestra.run.smithi001.stdout:1708 punch from 0x2a7b01 to 0x2b594a, (0xde49 bytes) 2024-09-14T22:36:50.359 INFO:teuthology.orchestra.run.smithi001.stdout:1709 punch from 0x2fbec99 to 0x2fc6912, (0x7c79 bytes) 2024-09-14T22:36:50.359 INFO:teuthology.orchestra.run.smithi001.stdout:1710 read 0x7eb04d5 thru 0x7eb9fa9 (0x9ad5 bytes) 2024-09-14T22:36:50.359 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:50.359 INFO:teuthology.orchestra.run.smithi001.stdout:1714 punch from 0x9f65218 to 0x9f6ba71, (0x6859 bytes) 2024-09-14T22:36:50.359 INFO:teuthology.orchestra.run.smithi001.stdout:1715 read 0x9c429b thru 0x9ca4a0 (0x6206 bytes) 2024-09-14T22:36:50.360 INFO:teuthology.orchestra.run.smithi001.stdout:1716 punch from 0x2e83875 to 0x2e88470, (0x4bfb bytes) 2024-09-14T22:36:50.360 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:50.360 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:50.360 INFO:teuthology.orchestra.run.smithi001.stdout:1719 read 0x1cde234 thru 0x1ce349a (0x5267 bytes) 2024-09-14T22:36:50.361 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:50.361 INFO:teuthology.orchestra.run.smithi001.stdout:1721 read 0x2c08f9 thru 0x2c4aa9 (0x41b1 bytes) 2024-09-14T22:36:50.361 INFO:teuthology.orchestra.run.smithi001.stdout:1722 write 0x17b3dbe thru 0x17be52d (0xa770 bytes) 2024-09-14T22:36:50.364 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:50.364 INFO:teuthology.orchestra.run.smithi001.stdout:1725 writesame 0x7afb0c6 thru 0x7b0396d data_size 0x4(0x88a8 bytes) 2024-09-14T22:36:50.372 INFO:teuthology.orchestra.run.smithi001.stdout:1727 read 0xa12732 thru 0xa14850 (0x211f bytes) 2024-09-14T22:36:50.389 INFO:teuthology.orchestra.run.smithi001.stdout:1728 write 0xda2e62b thru 0xda3880e (0xa1e4 bytes) 2024-09-14T22:36:50.399 INFO:teuthology.orchestra.run.smithi001.stdout:1731 read 0x9c26091 thru 0x9c2e2d6 (0x8246 bytes) 2024-09-14T22:36:50.400 INFO:teuthology.orchestra.run.smithi001.stdout:1732 writesame 0x67be8d8 thru 0x67c1f69 data_size 0x37(0x3692 bytes) 2024-09-14T22:36:50.408 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:50.408 INFO:teuthology.orchestra.run.smithi001.stdout:1734 write 0x427e21c thru 0x4289b04 (0xb8e9 bytes) 2024-09-14T22:36:50.411 INFO:teuthology.orchestra.run.smithi001.stdout:1735 write 0x9a2f34a thru 0x9a32808 (0x34bf bytes) 2024-09-14T22:36:50.418 INFO:teuthology.orchestra.run.smithi001.stdout:1736 read 0x3d9fb0b thru 0x3da40b7 (0x45ad bytes) 2024-09-14T22:36:50.419 INFO:teuthology.orchestra.run.smithi001.stdout:1737 write 0x92b15c1 thru 0x92be9c6 (0xd406 bytes) 2024-09-14T22:36:50.436 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:50.431+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:50.437 INFO:teuthology.orchestra.run.smithi001.stdout:1738 trunc from 0xda3880f to 0x6607f57 2024-09-14T22:36:50.467 INFO:teuthology.orchestra.run.smithi001.stdout:1739 read 0x55b8a08 thru 0x55b9d6e (0x1367 bytes) 2024-09-14T22:36:50.479 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:50.479 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:36:50.480 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:36:50.480 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:50.485 INFO:teuthology.orchestra.run.smithi001.stdout:1740 writesame 0x8f1f227 thru 0x8f20240 data_size 0x3(0x101a bytes) 2024-09-14T22:36:50.487 INFO:teuthology.orchestra.run.smithi001.stdout:1741 read 0x58ed88f thru 0x58f8f69 (0xb6db bytes) 2024-09-14T22:36:50.488 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:50.496 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:36:50.498 INFO:teuthology.orchestra.run.smithi001.stdout:1743 writesame 0x8f5642b thru 0x8f61841 data_size 0x1(0xb417 bytes) 2024-09-14T22:36:50.498 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:50.498 INFO:teuthology.orchestra.run.smithi001.stdout:1745 read 0x85797ce thru 0x858503b (0xb86e bytes) 2024-09-14T22:36:50.498 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:50.499 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:50.499 INFO:teuthology.orchestra.run.smithi001.stdout:1749 read 0x27f591 thru 0x285e41 (0x68b1 bytes) 2024-09-14T22:36:50.500 INFO:teuthology.orchestra.run.smithi001.stdout:1750 write 0x3a4114b thru 0x3a45dc6 (0x4c7c bytes) 2024-09-14T22:36:50.503 INFO:teuthology.orchestra.run.smithi001.stdout:1751 punch from 0x6c840dd to 0x6c8a935, (0x6858 bytes) 2024-09-14T22:36:50.503 INFO:teuthology.orchestra.run.smithi001.stdout:1752 write 0x45d20bb thru 0x45e0f43 (0xee89 bytes) 2024-09-14T22:36:50.523 INFO:teuthology.orchestra.run.smithi001.stdout:1753 write 0xe35d55c thru 0xe363558 (0x5ffd bytes) 2024-09-14T22:36:50.531 INFO:teuthology.orchestra.run.smithi001.stdout:1754 punch from 0xa9399b2 to 0xa93cfdf, (0x362d bytes) 2024-09-14T22:36:50.531 INFO:teuthology.orchestra.run.smithi001.stdout:1755 write 0x27c72d5 thru 0x27ce6d5 (0x7401 bytes) 2024-09-14T22:36:50.533 INFO:teuthology.orchestra.run.smithi001.stdout:1756 write 0x42e7a9 thru 0x4325a8 (0x3e00 bytes) 2024-09-14T22:36:50.535 INFO:teuthology.orchestra.run.smithi001.stdout:1757 writesame 0xd059921 thru 0xd05cf76 data_size 0x41(0x3656 bytes) 2024-09-14T22:36:50.536 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:50.531+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:50.542 INFO:teuthology.orchestra.run.smithi001.stdout:1758 read 0x1ff57ed thru 0x20007a3 (0xafb7 bytes) 2024-09-14T22:36:50.543 INFO:teuthology.orchestra.run.smithi001.stdout:1759 punch from 0x11bbff6 to 0x11c0a3f, (0x4a49 bytes) 2024-09-14T22:36:50.553 INFO:teuthology.orchestra.run.smithi001.stdout:1763 write 0xed39fc5 thru 0xed3d73e (0x377a bytes) 2024-09-14T22:36:50.559 INFO:teuthology.orchestra.run.smithi001.stdout:1766 write 0xe3a8343 thru 0xe3b6671 (0xe32f bytes) 2024-09-14T22:36:50.566 INFO:teuthology.orchestra.run.smithi001.stdout:1768 punch from 0x58fc3be to 0x58fd02d, (0xc6f bytes) 2024-09-14T22:36:50.566 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:50.566 INFO:teuthology.orchestra.run.smithi001.stdout:1772 punch from 0x345d50f to 0x346c7eb, (0xf2dc bytes) 2024-09-14T22:36:50.567 INFO:teuthology.orchestra.run.smithi001.stdout:1773 writesame 0x8ea1270 thru 0x8ea73e6 data_size 0x1(0x6177 bytes) 2024-09-14T22:36:50.592 INFO:teuthology.orchestra.run.smithi001.stdout:1774 write 0xc3d41bb thru 0xc3dc0b8 (0x7efe bytes) 2024-09-14T22:36:50.596 INFO:teuthology.orchestra.run.smithi001.stdout:1775 read 0x85eced3 thru 0x85efb88 (0x2cb6 bytes) 2024-09-14T22:36:50.618 INFO:teuthology.orchestra.run.smithi001.stdout:1776 trunc from 0xed3d73f to 0x56cff6c 2024-09-14T22:36:50.626 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:50.628 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:36:50.628 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:36:50.629 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:36:50.629 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:50.637 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:50.631+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:50.642 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:36:50.673 INFO:teuthology.orchestra.run.smithi001.stdout:1777 writesame 0xb450af9 thru 0xb45a47d data_size 0x1(0x9985 bytes) 2024-09-14T22:36:50.673 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:50.676 INFO:teuthology.orchestra.run.smithi001.stdout:1781 trunc from 0xb45a47e to 0xce0199b 2024-09-14T22:36:50.688 INFO:teuthology.orchestra.run.smithi001.stdout:1783 write 0xcdb3a76 thru 0xcdc3799 (0xfd24 bytes) 2024-09-14T22:36:50.697 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2024-09-14T22:36:50.697 INFO:teuthology.orchestra.run.smithi001.stdout:1784 trunc from 0xce0199b to 0xca6c0f1 2024-09-14T22:36:50.738 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:50.731+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:50.799 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:50.799 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:36:50.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:36:50.799 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:36:50.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.799 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:2770 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14331904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:50.799 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.551455+0000", 2024-09-14T22:36:50.800 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.245867818000001, 2024-09-14T22:36:50.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028737275, 2024-09-14T22:36:50.800 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:50.800 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:50.800 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:50.800 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:50.800 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:50.800 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.800 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2770 2024-09-14T22:36:50.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.800 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:50.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:50.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.551455+0000", 2024-09-14T22:36:50.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:50.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.551455+0000", 2024-09-14T22:36:50.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:50.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.551455+0000", 2024-09-14T22:36:50.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.39e-07 2024-09-14T22:36:50.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:50.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.551457+0000", 2024-09-14T22:36:50.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2000000000000001e-06 2024-09-14T22:36:50.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:50.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.551464+0000", 2024-09-14T22:36:50.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3180000000000003e-06 2024-09-14T22:36:50.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.551476+0000", 2024-09-14T22:36:50.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2345e-05 2024-09-14T22:36:50.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:50.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.579963+0000", 2024-09-14T22:36:50.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028487545999999999 2024-09-14T22:36:50.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:50.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.580075+0000", 2024-09-14T22:36:50.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111292 2024-09-14T22:36:50.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:50.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.580192+0000", 2024-09-14T22:36:50.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000117135 2024-09-14T22:36:50.804 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.804 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:50.804 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.804 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3021 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14292992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:50.804 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559557+0000", 2024-09-14T22:36:50.804 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.237765211999999, 2024-09-14T22:36:50.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028720181000000001, 2024-09-14T22:36:50.804 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:50.804 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:50.804 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:50.805 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:50.805 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:50.805 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.805 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3021 2024-09-14T22:36:50.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.805 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:50.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:50.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559557+0000", 2024-09-14T22:36:50.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:50.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559557+0000", 2024-09-14T22:36:50.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:50.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559558+0000", 2024-09-14T22:36:50.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4499999999999996e-07 2024-09-14T22:36:50.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:50.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559562+0000", 2024-09-14T22:36:50.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.065e-06 2024-09-14T22:36:50.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:50.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559573+0000", 2024-09-14T22:36:50.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1398e-05 2024-09-14T22:36:50.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559582+0000", 2024-09-14T22:36:50.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6100000000000006e-06 2024-09-14T22:36:50.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:50.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588042+0000", 2024-09-14T22:36:50.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028460388999999999 2024-09-14T22:36:50.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:50.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588147+0000", 2024-09-14T22:36:50.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000104727 2024-09-14T22:36:50.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:50.809 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588277+0000", 2024-09-14T22:36:50.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000130147 2024-09-14T22:36:50.809 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.809 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:50.809 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.809 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3022 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14301184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:50.809 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559594+0000", 2024-09-14T22:36:50.809 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.237728822000001, 2024-09-14T22:36:50.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028916481000000001, 2024-09-14T22:36:50.810 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:50.810 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:50.810 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:50.810 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:50.810 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:50.810 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.810 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3022 2024-09-14T22:36:50.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.810 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:50.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:50.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559594+0000", 2024-09-14T22:36:50.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:50.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559594+0000", 2024-09-14T22:36:50.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:50.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559594+0000", 2024-09-14T22:36:50.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7000000000000004e-07 2024-09-14T22:36:50.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:50.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559604+0000", 2024-09-14T22:36:50.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6020000000000006e-06 2024-09-14T22:36:50.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:50.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559615+0000", 2024-09-14T22:36:50.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1032999999999999e-05 2024-09-14T22:36:50.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559638+0000", 2024-09-14T22:36:50.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3255999999999999e-05 2024-09-14T22:36:50.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:50.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588323+0000", 2024-09-14T22:36:50.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.02868447 2024-09-14T22:36:50.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:50.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588443+0000", 2024-09-14T22:36:50.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120056 2024-09-14T22:36:50.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:50.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588510+0000", 2024-09-14T22:36:50.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7393999999999999e-05 2024-09-14T22:36:50.813 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.814 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:50.814 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.814 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3023 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14309376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:50.814 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559651+0000", 2024-09-14T22:36:50.814 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.237671335999998, 2024-09-14T22:36:50.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029146231000000002, 2024-09-14T22:36:50.814 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:50.814 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:50.814 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:50.814 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:50.814 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:50.815 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.815 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3023 2024-09-14T22:36:50.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.815 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:50.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:50.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559651+0000", 2024-09-14T22:36:50.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:50.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559651+0000", 2024-09-14T22:36:50.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:50.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559652+0000", 2024-09-14T22:36:50.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2699999999999998e-07 2024-09-14T22:36:50.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:50.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559656+0000", 2024-09-14T22:36:50.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7019999999999999e-06 2024-09-14T22:36:50.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:50.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559666+0000", 2024-09-14T22:36:50.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0400999999999999e-05 2024-09-14T22:36:50.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559674+0000", 2024-09-14T22:36:50.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8890000000000002e-06 2024-09-14T22:36:50.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:50.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588565+0000", 2024-09-14T22:36:50.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028891217 2024-09-14T22:36:50.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:50.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588679+0000", 2024-09-14T22:36:50.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114058 2024-09-14T22:36:50.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:50.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588797+0000", 2024-09-14T22:36:50.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118137 2024-09-14T22:36:50.819 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.819 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:50.819 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.820 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3024 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14317568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:50.820 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559685+0000", 2024-09-14T22:36:50.820 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.237637225, 2024-09-14T22:36:50.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029326498999999999, 2024-09-14T22:36:50.820 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:50.820 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:50.820 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:50.820 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:50.820 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:50.820 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.820 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3024 2024-09-14T22:36:50.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.821 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:50.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:50.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559685+0000", 2024-09-14T22:36:50.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:50.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559685+0000", 2024-09-14T22:36:50.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:50.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559686+0000", 2024-09-14T22:36:50.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0999999999999998e-07 2024-09-14T22:36:50.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:50.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559697+0000", 2024-09-14T22:36:50.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.147e-05 2024-09-14T22:36:50.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:50.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559708+0000", 2024-09-14T22:36:50.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0546e-05 2024-09-14T22:36:50.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559716+0000", 2024-09-14T22:36:50.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.704e-06 2024-09-14T22:36:50.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:50.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588833+0000", 2024-09-14T22:36:50.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029116442999999999 2024-09-14T22:36:50.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:50.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588920+0000", 2024-09-14T22:36:50.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6916999999999995e-05 2024-09-14T22:36:50.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:50.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589012+0000", 2024-09-14T22:36:50.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1809000000000001e-05 2024-09-14T22:36:50.824 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.824 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:50.824 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.825 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3025 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14325760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:50.825 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559729+0000", 2024-09-14T22:36:50.825 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.237593325999999, 2024-09-14T22:36:50.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029422501, 2024-09-14T22:36:50.825 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:50.825 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:50.825 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:50.825 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:50.826 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:50.826 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.826 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3025 2024-09-14T22:36:50.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.826 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:50.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:50.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559729+0000", 2024-09-14T22:36:50.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:50.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559729+0000", 2024-09-14T22:36:50.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:50.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559730+0000", 2024-09-14T22:36:50.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8100000000000002e-07 2024-09-14T22:36:50.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:50.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559733+0000", 2024-09-14T22:36:50.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4740000000000001e-06 2024-09-14T22:36:50.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:50.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559744+0000", 2024-09-14T22:36:50.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0489000000000001e-05 2024-09-14T22:36:50.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559760+0000", 2024-09-14T22:36:50.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5817000000000001e-05 2024-09-14T22:36:50.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:50.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589050+0000", 2024-09-14T22:36:50.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.02929056 2024-09-14T22:36:50.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:50.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589101+0000", 2024-09-14T22:36:50.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0992000000000002e-05 2024-09-14T22:36:50.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:50.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589152+0000", 2024-09-14T22:36:50.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0387999999999999e-05 2024-09-14T22:36:50.829 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.829 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:50.829 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.830 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3026 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14333952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:50.830 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559795+0000", 2024-09-14T22:36:50.830 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.237527501999999, 2024-09-14T22:36:50.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029483812000000002, 2024-09-14T22:36:50.830 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:50.830 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:50.830 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:50.830 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:50.830 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:50.830 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.830 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3026 2024-09-14T22:36:50.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.830 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:50.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:50.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559795+0000", 2024-09-14T22:36:50.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:50.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559795+0000", 2024-09-14T22:36:50.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:50.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559796+0000", 2024-09-14T22:36:50.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3200000000000003e-07 2024-09-14T22:36:50.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:50.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559800+0000", 2024-09-14T22:36:50.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2930000000000002e-06 2024-09-14T22:36:50.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:50.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559812+0000", 2024-09-14T22:36:50.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2053e-05 2024-09-14T22:36:50.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559821+0000", 2024-09-14T22:36:50.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8869999999999998e-06 2024-09-14T22:36:50.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:50.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589177+0000", 2024-09-14T22:36:50.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029356308000000001 2024-09-14T22:36:50.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:50.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589229+0000", 2024-09-14T22:36:50.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1326000000000003e-05 2024-09-14T22:36:50.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:50.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589279+0000", 2024-09-14T22:36:50.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0012999999999997e-05 2024-09-14T22:36:50.834 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.834 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:50.834 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.834 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3027 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14342144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:50.834 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559833+0000", 2024-09-14T22:36:50.834 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.237489774, 2024-09-14T22:36:50.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029619187000000002, 2024-09-14T22:36:50.835 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:50.835 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:50.835 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:50.835 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:50.835 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:50.835 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.835 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3027 2024-09-14T22:36:50.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.835 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:50.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:50.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559833+0000", 2024-09-14T22:36:50.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:50.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559833+0000", 2024-09-14T22:36:50.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:50.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559833+0000", 2024-09-14T22:36:50.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9400000000000005e-07 2024-09-14T22:36:50.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:50.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559845+0000", 2024-09-14T22:36:50.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1623e-05 2024-09-14T22:36:50.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:50.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559856+0000", 2024-09-14T22:36:50.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0832999999999999e-05 2024-09-14T22:36:50.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559864+0000", 2024-09-14T22:36:50.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3119999999999994e-06 2024-09-14T22:36:50.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:50.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589299+0000", 2024-09-14T22:36:50.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029434699000000002 2024-09-14T22:36:50.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:50.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589354+0000", 2024-09-14T22:36:50.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5467999999999998e-05 2024-09-14T22:36:50.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.840 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:50.840 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589452+0000", 2024-09-14T22:36:50.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7558000000000003e-05 2024-09-14T22:36:50.840 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.840 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:50.840 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.840 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3028 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14350336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:50.840 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559875+0000", 2024-09-14T22:36:50.840 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.237446886000001, 2024-09-14T22:36:50.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029775586999999999, 2024-09-14T22:36:50.841 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:50.841 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:50.841 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:50.841 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:50.841 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:50.841 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.841 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3028 2024-09-14T22:36:50.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.841 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:50.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:50.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559875+0000", 2024-09-14T22:36:50.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:50.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559875+0000", 2024-09-14T22:36:50.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:50.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559876+0000", 2024-09-14T22:36:50.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5199999999999996e-07 2024-09-14T22:36:50.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:50.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559879+0000", 2024-09-14T22:36:50.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.095e-06 2024-09-14T22:36:50.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:50.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559889+0000", 2024-09-14T22:36:50.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0019e-05 2024-09-14T22:36:50.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559897+0000", 2024-09-14T22:36:50.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9189999999999999e-06 2024-09-14T22:36:50.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:50.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589486+0000", 2024-09-14T22:36:50.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029588482999999999 2024-09-14T22:36:50.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:50.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589578+0000", 2024-09-14T22:36:50.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2329000000000003e-05 2024-09-14T22:36:50.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:50.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589651+0000", 2024-09-14T22:36:50.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2990000000000004e-05 2024-09-14T22:36:50.845 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.845 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:50.845 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.845 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4068 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13252608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:50.845 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680102+0000", 2024-09-14T22:36:50.845 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.117220067000002, 2024-09-14T22:36:50.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028907656, 2024-09-14T22:36:50.845 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:50.846 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:50.846 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:50.846 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:50.846 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:50.846 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.846 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4068 2024-09-14T22:36:50.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.846 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:50.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:50.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680102+0000", 2024-09-14T22:36:50.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:50.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680102+0000", 2024-09-14T22:36:50.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:50.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680103+0000", 2024-09-14T22:36:50.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6600000000000002e-07 2024-09-14T22:36:50.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:50.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680105+0000", 2024-09-14T22:36:50.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.616e-06 2024-09-14T22:36:50.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:50.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680114+0000", 2024-09-14T22:36:50.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4120000000000001e-06 2024-09-14T22:36:50.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680119+0000", 2024-09-14T22:36:50.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6210000000000001e-06 2024-09-14T22:36:50.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:50.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.708813+0000", 2024-09-14T22:36:50.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028693330999999999 2024-09-14T22:36:50.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:50.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.708903+0000", 2024-09-14T22:36:50.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0297000000000006e-05 2024-09-14T22:36:50.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:50.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709010+0000", 2024-09-14T22:36:50.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106913 2024-09-14T22:36:50.850 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.850 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:50.850 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.850 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4069 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13260800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:50.850 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680128+0000", 2024-09-14T22:36:50.850 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.117194791999999, 2024-09-14T22:36:50.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029145476, 2024-09-14T22:36:50.850 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:50.851 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:50.851 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:50.851 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:50.851 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:50.851 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.851 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4069 2024-09-14T22:36:50.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.851 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:50.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:50.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680128+0000", 2024-09-14T22:36:50.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:50.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680128+0000", 2024-09-14T22:36:50.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:50.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680128+0000", 2024-09-14T22:36:50.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7599999999999998e-07 2024-09-14T22:36:50.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:50.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680131+0000", 2024-09-14T22:36:50.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9050000000000001e-06 2024-09-14T22:36:50.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:50.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680138+0000", 2024-09-14T22:36:50.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2559999999999996e-06 2024-09-14T22:36:50.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680144+0000", 2024-09-14T22:36:50.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4779999999999999e-06 2024-09-14T22:36:50.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:50.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709054+0000", 2024-09-14T22:36:50.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028910207 2024-09-14T22:36:50.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:50.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709180+0000", 2024-09-14T22:36:50.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012608699999999999 2024-09-14T22:36:50.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:50.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709273+0000", 2024-09-14T22:36:50.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3166999999999998e-05 2024-09-14T22:36:50.855 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.855 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:50.855 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.855 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4070 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13268992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:50.855 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680151+0000", 2024-09-14T22:36:50.855 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.117171509999999, 2024-09-14T22:36:50.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029327986, 2024-09-14T22:36:50.856 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:50.856 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:50.856 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:50.856 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:50.856 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:50.856 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.856 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4070 2024-09-14T22:36:50.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.856 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:50.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:50.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680151+0000", 2024-09-14T22:36:50.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:50.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680151+0000", 2024-09-14T22:36:50.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:50.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680151+0000", 2024-09-14T22:36:50.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:36:50.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:50.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680154+0000", 2024-09-14T22:36:50.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3350000000000001e-06 2024-09-14T22:36:50.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:50.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680162+0000", 2024-09-14T22:36:50.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.473e-06 2024-09-14T22:36:50.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680177+0000", 2024-09-14T22:36:50.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.465e-05 2024-09-14T22:36:50.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:50.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709356+0000", 2024-09-14T22:36:50.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029178744 2024-09-14T22:36:50.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:50.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709428+0000", 2024-09-14T22:36:50.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2428000000000005e-05 2024-09-14T22:36:50.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:50.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709479+0000", 2024-09-14T22:36:50.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0915999999999998e-05 2024-09-14T22:36:50.860 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.860 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:50.860 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.861 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4071 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13277184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:50.861 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680185+0000", 2024-09-14T22:36:50.861 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.11713701, 2024-09-14T22:36:50.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029452392000000001, 2024-09-14T22:36:50.861 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:50.861 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:50.861 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:50.861 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:50.861 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:50.861 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.861 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4071 2024-09-14T22:36:50.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.861 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:50.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:50.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680185+0000", 2024-09-14T22:36:50.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:50.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680185+0000", 2024-09-14T22:36:50.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:50.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680186+0000", 2024-09-14T22:36:50.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6100000000000001e-07 2024-09-14T22:36:50.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:50.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680188+0000", 2024-09-14T22:36:50.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.509e-06 2024-09-14T22:36:50.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:50.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680197+0000", 2024-09-14T22:36:50.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8550000000000008e-06 2024-09-14T22:36:50.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680203+0000", 2024-09-14T22:36:50.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5849999999999999e-06 2024-09-14T22:36:50.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:50.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709514+0000", 2024-09-14T22:36:50.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029311054 2024-09-14T22:36:50.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:50.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709585+0000", 2024-09-14T22:36:50.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0875000000000002e-05 2024-09-14T22:36:50.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:50.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709638+0000", 2024-09-14T22:36:50.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3053000000000002e-05 2024-09-14T22:36:50.865 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.865 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:50.865 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.866 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4072 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13285376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:50.866 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680211+0000", 2024-09-14T22:36:50.866 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.117111729000001, 2024-09-14T22:36:50.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029591177999999999, 2024-09-14T22:36:50.866 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:50.866 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:50.866 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:50.866 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:50.866 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:50.866 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.866 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4072 2024-09-14T22:36:50.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.867 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:50.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:50.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680211+0000", 2024-09-14T22:36:50.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:50.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680211+0000", 2024-09-14T22:36:50.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:50.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680211+0000", 2024-09-14T22:36:50.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6699999999999999e-07 2024-09-14T22:36:50.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:50.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680214+0000", 2024-09-14T22:36:50.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0149999999999999e-06 2024-09-14T22:36:50.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:50.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680222+0000", 2024-09-14T22:36:50.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.371e-06 2024-09-14T22:36:50.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680228+0000", 2024-09-14T22:36:50.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6230000000000002e-06 2024-09-14T22:36:50.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:50.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709670+0000", 2024-09-14T22:36:50.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029441694000000001 2024-09-14T22:36:50.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:50.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709741+0000", 2024-09-14T22:36:50.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1668999999999994e-05 2024-09-14T22:36:50.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:50.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709802+0000", 2024-09-14T22:36:50.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0338999999999999e-05 2024-09-14T22:36:50.871 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.871 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:50.871 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.871 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4073 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13293568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:50.871 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680236+0000", 2024-09-14T22:36:50.871 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.117086369999999, 2024-09-14T22:36:50.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029755238999999999, 2024-09-14T22:36:50.872 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:50.872 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:50.872 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:50.872 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:50.872 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:50.872 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.872 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4073 2024-09-14T22:36:50.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.872 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:50.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:50.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680236+0000", 2024-09-14T22:36:50.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:50.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680236+0000", 2024-09-14T22:36:50.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:50.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680236+0000", 2024-09-14T22:36:50.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5499999999999998e-07 2024-09-14T22:36:50.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:50.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680239+0000", 2024-09-14T22:36:50.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4380000000000002e-06 2024-09-14T22:36:50.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:50.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680248+0000", 2024-09-14T22:36:50.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6319999999999997e-06 2024-09-14T22:36:50.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680261+0000", 2024-09-14T22:36:50.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3193000000000001e-05 2024-09-14T22:36:50.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:50.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709831+0000", 2024-09-14T22:36:50.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029570486 2024-09-14T22:36:50.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:50.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709883+0000", 2024-09-14T22:36:50.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2290999999999998e-05 2024-09-14T22:36:50.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:50.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709991+0000", 2024-09-14T22:36:50.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107744 2024-09-14T22:36:50.876 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.876 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:50.876 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.876 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4074 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13301760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:50.876 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680269+0000", 2024-09-14T22:36:50.876 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.117053471999998, 2024-09-14T22:36:50.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030007875999999999, 2024-09-14T22:36:50.876 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:50.876 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:50.877 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:50.877 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:50.877 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:50.877 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.877 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4074 2024-09-14T22:36:50.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.877 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:50.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:50.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680269+0000", 2024-09-14T22:36:50.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:50.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680269+0000", 2024-09-14T22:36:50.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:50.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680269+0000", 2024-09-14T22:36:50.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7300000000000001e-07 2024-09-14T22:36:50.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:50.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680277+0000", 2024-09-14T22:36:50.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8480000000000001e-06 2024-09-14T22:36:50.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:50.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680286+0000", 2024-09-14T22:36:50.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0529999999999996e-06 2024-09-14T22:36:50.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680292+0000", 2024-09-14T22:36:50.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.728e-06 2024-09-14T22:36:50.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:50.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.710024+0000", 2024-09-14T22:36:50.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029732176999999999 2024-09-14T22:36:50.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:50.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.710109+0000", 2024-09-14T22:36:50.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4715000000000005e-05 2024-09-14T22:36:50.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:50.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.710277+0000", 2024-09-14T22:36:50.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000167882 2024-09-14T22:36:50.881 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.881 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:50.881 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.882 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:12663 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 374784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T22:36:50.882 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:28.325542+0000", 2024-09-14T22:36:50.882 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.471780462000002, 2024-09-14T22:36:50.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028843810000000001, 2024-09-14T22:36:50.882 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:50.882 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:50.882 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:50.882 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:50.882 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:50.882 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.882 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12663 2024-09-14T22:36:50.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.882 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:50.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:50.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325542+0000", 2024-09-14T22:36:50.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:50.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325542+0000", 2024-09-14T22:36:50.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:50.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325542+0000", 2024-09-14T22:36:50.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.44e-07 2024-09-14T22:36:50.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:50.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325545+0000", 2024-09-14T22:36:50.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.796e-06 2024-09-14T22:36:50.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:50.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325553+0000", 2024-09-14T22:36:50.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8569999999999995e-06 2024-09-14T22:36:50.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325560+0000", 2024-09-14T22:36:50.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5379999999999999e-06 2024-09-14T22:36:50.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:50.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354196+0000", 2024-09-14T22:36:50.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028635864 2024-09-14T22:36:50.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:50.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354280+0000", 2024-09-14T22:36:50.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4324000000000001e-05 2024-09-14T22:36:50.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.886 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:50.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354386+0000", 2024-09-14T22:36:50.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105887 2024-09-14T22:36:50.886 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.886 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:50.886 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.886 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:12664 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T22:36:50.886 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:28.325596+0000", 2024-09-14T22:36:50.886 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.471725958, 2024-09-14T22:36:50.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028964581, 2024-09-14T22:36:50.887 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:50.887 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:50.887 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:50.887 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:50.887 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:50.887 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.887 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12664 2024-09-14T22:36:50.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.887 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:50.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:50.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325596+0000", 2024-09-14T22:36:50.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:50.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325596+0000", 2024-09-14T22:36:50.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:50.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325597+0000", 2024-09-14T22:36:50.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4300000000000002e-07 2024-09-14T22:36:50.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:50.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325600+0000", 2024-09-14T22:36:50.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2219999999999998e-06 2024-09-14T22:36:50.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:50.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325610+0000", 2024-09-14T22:36:50.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2650000000000002e-06 2024-09-14T22:36:50.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325617+0000", 2024-09-14T22:36:50.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1199999999999996e-06 2024-09-14T22:36:50.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:50.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354438+0000", 2024-09-14T22:36:50.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028821205999999999 2024-09-14T22:36:50.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:50.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354495+0000", 2024-09-14T22:36:50.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6752999999999997e-05 2024-09-14T22:36:50.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:50.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354561+0000", 2024-09-14T22:36:50.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6272000000000005e-05 2024-09-14T22:36:50.892 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.892 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:50.892 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.892 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:12665 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 391168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T22:36:50.892 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:28.325626+0000", 2024-09-14T22:36:50.892 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.471696636000001, 2024-09-14T22:36:50.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029080366, 2024-09-14T22:36:50.892 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:50.892 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:50.892 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:50.892 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:50.892 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:50.893 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.893 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12665 2024-09-14T22:36:50.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.893 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:50.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:50.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325626+0000", 2024-09-14T22:36:50.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:50.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325626+0000", 2024-09-14T22:36:50.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:50.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325626+0000", 2024-09-14T22:36:50.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.06e-07 2024-09-14T22:36:50.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:50.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325629+0000", 2024-09-14T22:36:50.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7149999999999998e-06 2024-09-14T22:36:50.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:50.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325637+0000", 2024-09-14T22:36:50.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7360000000000001e-06 2024-09-14T22:36:50.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325643+0000", 2024-09-14T22:36:50.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2480000000000001e-06 2024-09-14T22:36:50.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:50.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354589+0000", 2024-09-14T22:36:50.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028945538 2024-09-14T22:36:50.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:50.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354635+0000", 2024-09-14T22:36:50.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6374e-05 2024-09-14T22:36:50.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:50.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354706+0000", 2024-09-14T22:36:50.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1049000000000004e-05 2024-09-14T22:36:50.896 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.896 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:50.896 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.897 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:13927 3.1b 3:ddfd6241:::rbd_data.113cda0183a0.0000000000000009:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-14T22:36:50.897 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:31.537348+0000", 2024-09-14T22:36:50.897 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.259974035999999, 2024-09-14T22:36:50.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032156573000000001, 2024-09-14T22:36:50.897 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:50.897 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:50.897 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:50.897 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:50.897 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:50.897 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:50.897 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 13927 2024-09-14T22:36:50.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.897 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:50.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:50.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537348+0000", 2024-09-14T22:36:50.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:50.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537348+0000", 2024-09-14T22:36:50.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:50.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:50.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537349+0000", 2024-09-14T22:36:50.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.068e-06 2024-09-14T22:36:50.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:50.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537358+0000", 2024-09-14T22:36:50.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.653e-06 2024-09-14T22:36:50.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:50.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537374+0000", 2024-09-14T22:36:50.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5594e-05 2024-09-14T22:36:50.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:50.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537383+0000", 2024-09-14T22:36:50.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0929999999999992e-06 2024-09-14T22:36:50.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:50.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537429+0000", 2024-09-14T22:36:50.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5948000000000001e-05 2024-09-14T22:36:50.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:50.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537490+0000", 2024-09-14T22:36:50.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1002000000000003e-05 2024-09-14T22:36:50.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 4", 2024-09-14T22:36:50.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.542781+0000", 2024-09-14T22:36:50.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.005290861 2024-09-14T22:36:50.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:50.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.569350+0000", 2024-09-14T22:36:50.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026569299000000001 2024-09-14T22:36:50.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:36:50.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.569375+0000", 2024-09-14T22:36:50.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5448000000000001e-05 2024-09-14T22:36:50.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:36:50.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.569444+0000", 2024-09-14T22:36:50.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9184999999999996e-05 2024-09-14T22:36:50.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:50.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:50.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:50.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.569505+0000", 2024-09-14T22:36:50.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0421999999999998e-05 2024-09-14T22:36:50.903 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.903 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:50.903 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.903 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:50.903 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:50.903 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:50.905 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:36:50.905 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:50.899+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:51.006 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:51.003+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:51.037 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:51.040 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:36:51.040 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:36:51.041 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:51.051 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:36:51.141 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:51.139+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:51.207 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:51.203+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:51.217 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:51.218 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:36:51.218 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:36:51.218 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:36:51.218 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:51.232 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:36:51.308 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:51.303+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:51.401 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:51.401 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:36:51.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:36:51.401 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:36:51.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.401 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4149 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13129728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:51.401 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703281+0000", 2024-09-14T22:36:51.401 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.696175484000001, 2024-09-14T22:36:51.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045957114, 2024-09-14T22:36:51.402 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:51.402 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:51.402 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:51.402 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:51.402 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:51.402 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:51.402 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4149 2024-09-14T22:36:51.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.402 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:51.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:51.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703281+0000", 2024-09-14T22:36:51.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:51.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703281+0000", 2024-09-14T22:36:51.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:51.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703282+0000", 2024-09-14T22:36:51.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4199999999999996e-07 2024-09-14T22:36:51.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:51.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703289+0000", 2024-09-14T22:36:51.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0770000000000002e-06 2024-09-14T22:36:51.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:51.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703296+0000", 2024-09-14T22:36:51.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8129999999999998e-06 2024-09-14T22:36:51.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:51.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703307+0000", 2024-09-14T22:36:51.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1663e-05 2024-09-14T22:36:51.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:51.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749073+0000", 2024-09-14T22:36:51.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.04576537 2024-09-14T22:36:51.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:51.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749156+0000", 2024-09-14T22:36:51.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3399999999999994e-05 2024-09-14T22:36:51.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:51.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749239+0000", 2024-09-14T22:36:51.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2249000000000005e-05 2024-09-14T22:36:51.405 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.405 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:51.405 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.406 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4150 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13137920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:51.406 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703318+0000", 2024-09-14T22:36:51.406 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.696139290000001, 2024-09-14T22:36:51.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046170585, 2024-09-14T22:36:51.406 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:51.406 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:51.406 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:51.406 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:51.406 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:51.406 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:51.406 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4150 2024-09-14T22:36:51.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.406 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:51.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:51.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703318+0000", 2024-09-14T22:36:51.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:51.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703318+0000", 2024-09-14T22:36:51.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:51.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703318+0000", 2024-09-14T22:36:51.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8800000000000003e-07 2024-09-14T22:36:51.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:51.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703321+0000", 2024-09-14T22:36:51.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7860000000000001e-06 2024-09-14T22:36:51.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:51.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703328+0000", 2024-09-14T22:36:51.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2810000000000003e-06 2024-09-14T22:36:51.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:51.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703340+0000", 2024-09-14T22:36:51.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1745e-05 2024-09-14T22:36:51.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:51.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749286+0000", 2024-09-14T22:36:51.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045946222000000002 2024-09-14T22:36:51.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:51.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749390+0000", 2024-09-14T22:36:51.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000103614 2024-09-14T22:36:51.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:51.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749488+0000", 2024-09-14T22:36:51.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8449000000000006e-05 2024-09-14T22:36:51.409 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.409 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:51.409 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.410 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4151 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13146112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:51.410 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703348+0000", 2024-09-14T22:36:51.410 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.696109127, 2024-09-14T22:36:51.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046349509999999997, 2024-09-14T22:36:51.410 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:51.410 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:51.410 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:51.410 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:51.410 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:51.410 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:51.410 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4151 2024-09-14T22:36:51.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.410 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:51.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:51.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703348+0000", 2024-09-14T22:36:51.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:51.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703348+0000", 2024-09-14T22:36:51.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:51.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703348+0000", 2024-09-14T22:36:51.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6899999999999998e-07 2024-09-14T22:36:51.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:51.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703351+0000", 2024-09-14T22:36:51.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5950000000000001e-06 2024-09-14T22:36:51.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:51.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703357+0000", 2024-09-14T22:36:51.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4150000000000004e-06 2024-09-14T22:36:51.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:51.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703364+0000", 2024-09-14T22:36:51.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4069999999999998e-06 2024-09-14T22:36:51.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:51.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749530+0000", 2024-09-14T22:36:51.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046166281000000003 2024-09-14T22:36:51.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:51.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749615+0000", 2024-09-14T22:36:51.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5409999999999997e-05 2024-09-14T22:36:51.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:51.414 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749697+0000", 2024-09-14T22:36:51.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1933000000000003e-05 2024-09-14T22:36:51.414 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.414 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:51.414 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.414 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4152 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13154304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:51.414 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703371+0000", 2024-09-14T22:36:51.414 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.696085725, 2024-09-14T22:36:51.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046552233999999998, 2024-09-14T22:36:51.414 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:51.414 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:51.415 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:51.415 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:51.415 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:51.415 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:51.415 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4152 2024-09-14T22:36:51.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.415 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:51.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:51.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703371+0000", 2024-09-14T22:36:51.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:51.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703371+0000", 2024-09-14T22:36:51.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:51.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703371+0000", 2024-09-14T22:36:51.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.34e-07 2024-09-14T22:36:51.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:51.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703374+0000", 2024-09-14T22:36:51.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.227e-06 2024-09-14T22:36:51.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:51.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703380+0000", 2024-09-14T22:36:51.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4699999999999999e-06 2024-09-14T22:36:51.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:51.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703385+0000", 2024-09-14T22:36:51.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.259e-06 2024-09-14T22:36:51.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:51.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749742+0000", 2024-09-14T22:36:51.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046357031 2024-09-14T22:36:51.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:51.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749825+0000", 2024-09-14T22:36:51.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2674000000000002e-05 2024-09-14T22:36:51.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:51.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749923+0000", 2024-09-14T22:36:51.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8239000000000006e-05 2024-09-14T22:36:51.418 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.418 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:51.418 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.419 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4153 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13162496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:51.419 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703392+0000", 2024-09-14T22:36:51.419 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.696064429, 2024-09-14T22:36:51.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046675756999999998, 2024-09-14T22:36:51.419 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:51.419 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:51.419 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:51.419 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:51.419 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:51.419 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:51.419 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4153 2024-09-14T22:36:51.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.419 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:51.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:51.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703392+0000", 2024-09-14T22:36:51.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:51.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703392+0000", 2024-09-14T22:36:51.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:51.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.705934+0000", 2024-09-14T22:36:51.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025418540000000001 2024-09-14T22:36:51.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:51.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.705940+0000", 2024-09-14T22:36:51.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0950000000000001e-06 2024-09-14T22:36:51.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:51.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.705952+0000", 2024-09-14T22:36:51.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1973e-05 2024-09-14T22:36:51.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:51.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.705958+0000", 2024-09-14T22:36:51.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.541e-06 2024-09-14T22:36:51.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:51.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749969+0000", 2024-09-14T22:36:51.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044011300000000003 2024-09-14T22:36:51.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:51.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.750020+0000", 2024-09-14T22:36:51.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1054000000000003e-05 2024-09-14T22:36:51.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:51.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.750068+0000", 2024-09-14T22:36:51.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7939999999999998e-05 2024-09-14T22:36:51.423 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.423 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:51.423 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.423 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4165 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13260800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:51.423 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706178+0000", 2024-09-14T22:36:51.423 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.693278517, 2024-09-14T22:36:51.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045915594999999997, 2024-09-14T22:36:51.423 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:51.423 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:51.424 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:51.424 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:51.424 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:51.424 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:51.424 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4165 2024-09-14T22:36:51.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.424 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:51.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:51.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706178+0000", 2024-09-14T22:36:51.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:51.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706178+0000", 2024-09-14T22:36:51.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:51.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706179+0000", 2024-09-14T22:36:51.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4999999999999998e-07 2024-09-14T22:36:51.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:51.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706181+0000", 2024-09-14T22:36:51.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4679999999999999e-06 2024-09-14T22:36:51.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:51.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706189+0000", 2024-09-14T22:36:51.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.994e-06 2024-09-14T22:36:51.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:51.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706195+0000", 2024-09-14T22:36:51.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6119999999999998e-06 2024-09-14T22:36:51.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:51.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752003+0000", 2024-09-14T22:36:51.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045807618000000001 2024-09-14T22:36:51.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:51.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752050+0000", 2024-09-14T22:36:51.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7301e-05 2024-09-14T22:36:51.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:51.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752094+0000", 2024-09-14T22:36:51.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4152e-05 2024-09-14T22:36:51.427 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.427 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:51.427 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.428 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4166 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13268992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:51.428 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706205+0000", 2024-09-14T22:36:51.428 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.693252328, 2024-09-14T22:36:51.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046092665999999997, 2024-09-14T22:36:51.428 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:51.428 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:51.428 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:51.428 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:51.428 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:51.428 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:51.428 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4166 2024-09-14T22:36:51.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.429 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:51.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:51.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706205+0000", 2024-09-14T22:36:51.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:51.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706205+0000", 2024-09-14T22:36:51.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:51.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706205+0000", 2024-09-14T22:36:51.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:36:51.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:51.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706208+0000", 2024-09-14T22:36:51.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8049999999999998e-06 2024-09-14T22:36:51.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:51.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706215+0000", 2024-09-14T22:36:51.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2010000000000002e-06 2024-09-14T22:36:51.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:51.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706223+0000", 2024-09-14T22:36:51.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7260000000000002e-06 2024-09-14T22:36:51.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:51.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752129+0000", 2024-09-14T22:36:51.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045906532 2024-09-14T22:36:51.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:51.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752213+0000", 2024-09-14T22:36:51.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4060000000000005e-05 2024-09-14T22:36:51.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:51.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752297+0000", 2024-09-14T22:36:51.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3807999999999994e-05 2024-09-14T22:36:51.432 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.432 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:51.432 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.432 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4167 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13277184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:51.432 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706233+0000", 2024-09-14T22:36:51.432 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.693223716999999, 2024-09-14T22:36:51.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046190500000000002, 2024-09-14T22:36:51.433 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:51.433 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:51.433 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:51.433 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:51.433 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:51.433 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:51.433 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4167 2024-09-14T22:36:51.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.433 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:51.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:51.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706233+0000", 2024-09-14T22:36:51.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:51.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706233+0000", 2024-09-14T22:36:51.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:51.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706234+0000", 2024-09-14T22:36:51.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.75e-07 2024-09-14T22:36:51.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:51.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706237+0000", 2024-09-14T22:36:51.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9890000000000001e-06 2024-09-14T22:36:51.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:51.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706244+0000", 2024-09-14T22:36:51.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6240000000000002e-06 2024-09-14T22:36:51.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:51.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706255+0000", 2024-09-14T22:36:51.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0328e-05 2024-09-14T22:36:51.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:51.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752338+0000", 2024-09-14T22:36:51.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046083172999999998 2024-09-14T22:36:51.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:51.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752381+0000", 2024-09-14T22:36:51.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2973999999999999e-05 2024-09-14T22:36:51.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:51.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752424+0000", 2024-09-14T22:36:51.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2936999999999999e-05 2024-09-14T22:36:51.437 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.437 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:51.437 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.437 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4168 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13285376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:51.437 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706261+0000", 2024-09-14T22:36:51.437 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.693196372999999, 2024-09-14T22:36:51.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046378635000000001, 2024-09-14T22:36:51.437 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:51.437 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:51.437 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:51.437 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:51.438 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:51.438 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:51.438 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4168 2024-09-14T22:36:51.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.438 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:51.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:51.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706261+0000", 2024-09-14T22:36:51.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:51.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706261+0000", 2024-09-14T22:36:51.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:51.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706261+0000", 2024-09-14T22:36:51.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2899999999999999e-07 2024-09-14T22:36:51.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:51.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706264+0000", 2024-09-14T22:36:51.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.774e-06 2024-09-14T22:36:51.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:51.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706267+0000", 2024-09-14T22:36:51.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7179999999999998e-06 2024-09-14T22:36:51.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:51.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706274+0000", 2024-09-14T22:36:51.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7599999999999997e-06 2024-09-14T22:36:51.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:51.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752458+0000", 2024-09-14T22:36:51.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046183825999999997 2024-09-14T22:36:51.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:51.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752545+0000", 2024-09-14T22:36:51.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7272000000000001e-05 2024-09-14T22:36:51.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:51.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752639+0000", 2024-09-14T22:36:51.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3956000000000006e-05 2024-09-14T22:36:51.441 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.441 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:51.441 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.441 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4169 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13293568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:51.441 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706278+0000", 2024-09-14T22:36:51.442 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.693178646, 2024-09-14T22:36:51.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046489181999999997, 2024-09-14T22:36:51.442 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:51.442 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:51.442 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:51.442 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:51.442 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:51.442 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:51.442 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4169 2024-09-14T22:36:51.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.442 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:51.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:51.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706278+0000", 2024-09-14T22:36:51.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:51.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706278+0000", 2024-09-14T22:36:51.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:51.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706279+0000", 2024-09-14T22:36:51.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6100000000000002e-07 2024-09-14T22:36:51.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:51.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706280+0000", 2024-09-14T22:36:51.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2589999999999999e-06 2024-09-14T22:36:51.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:51.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706283+0000", 2024-09-14T22:36:51.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.501e-06 2024-09-14T22:36:51.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:51.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706286+0000", 2024-09-14T22:36:51.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5409999999999999e-06 2024-09-14T22:36:51.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:51.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752680+0000", 2024-09-14T22:36:51.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046393886000000002 2024-09-14T22:36:51.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:51.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752723+0000", 2024-09-14T22:36:51.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2870000000000001e-05 2024-09-14T22:36:51.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:51.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752767+0000", 2024-09-14T22:36:51.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4864000000000003e-05 2024-09-14T22:36:51.446 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.446 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:51.446 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.446 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4170 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13301760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:51.446 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706289+0000", 2024-09-14T22:36:51.446 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.693167748, 2024-09-14T22:36:51.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046691115999999998, 2024-09-14T22:36:51.446 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:51.446 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:51.446 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:51.446 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:51.447 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:51.447 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:51.447 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4170 2024-09-14T22:36:51.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.447 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:51.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:51.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706289+0000", 2024-09-14T22:36:51.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:51.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706289+0000", 2024-09-14T22:36:51.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:51.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706289+0000", 2024-09-14T22:36:51.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3200000000000001e-07 2024-09-14T22:36:51.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:51.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706296+0000", 2024-09-14T22:36:51.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.922e-06 2024-09-14T22:36:51.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:51.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706303+0000", 2024-09-14T22:36:51.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3980000000000004e-06 2024-09-14T22:36:51.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:51.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706307+0000", 2024-09-14T22:36:51.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6750000000000001e-06 2024-09-14T22:36:51.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:51.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752799+0000", 2024-09-14T22:36:51.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046491609000000003 2024-09-14T22:36:51.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:51.450 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752884+0000", 2024-09-14T22:36:51.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4876999999999994e-05 2024-09-14T22:36:51.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.450 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:51.450 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752980+0000", 2024-09-14T22:36:51.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6403000000000005e-05 2024-09-14T22:36:51.450 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.450 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:51.450 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.450 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6292 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8329216~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:51.450 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966066+0000", 2024-09-14T22:36:51.450 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.433390809999999, 2024-09-14T22:36:51.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045967367000000002, 2024-09-14T22:36:51.451 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:51.451 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:51.451 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:51.451 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:51.451 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:51.451 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:51.451 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6292 2024-09-14T22:36:51.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.451 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:51.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:51.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966066+0000", 2024-09-14T22:36:51.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:51.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966066+0000", 2024-09-14T22:36:51.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:51.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966067+0000", 2024-09-14T22:36:51.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6499999999999999e-07 2024-09-14T22:36:51.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:51.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966069+0000", 2024-09-14T22:36:51.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4059999999999999e-06 2024-09-14T22:36:51.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:51.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966077+0000", 2024-09-14T22:36:51.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5120000000000002e-06 2024-09-14T22:36:51.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:51.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966083+0000", 2024-09-14T22:36:51.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5590000000000002e-06 2024-09-14T22:36:51.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:51.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.011820+0000", 2024-09-14T22:36:51.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045737281999999997 2024-09-14T22:36:51.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:51.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.011911+0000", 2024-09-14T22:36:51.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0153000000000002e-05 2024-09-14T22:36:51.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:51.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012033+0000", 2024-09-14T22:36:51.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012289000000000001 2024-09-14T22:36:51.455 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.455 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:51.455 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.455 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6293 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8337408~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:51.455 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966091+0000", 2024-09-14T22:36:51.455 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.433365716000001, 2024-09-14T22:36:51.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046181672, 2024-09-14T22:36:51.455 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:51.456 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:51.456 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:51.456 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:51.456 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:51.456 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:51.456 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6293 2024-09-14T22:36:51.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.456 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:51.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:51.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966091+0000", 2024-09-14T22:36:51.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:51.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966091+0000", 2024-09-14T22:36:51.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:51.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966092+0000", 2024-09-14T22:36:51.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:36:51.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:51.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966094+0000", 2024-09-14T22:36:51.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5830000000000001e-06 2024-09-14T22:36:51.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:51.458 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966102+0000", 2024-09-14T22:36:51.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7470000000000005e-06 2024-09-14T22:36:51.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:51.458 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966107+0000", 2024-09-14T22:36:51.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1209999999999998e-06 2024-09-14T22:36:51.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:51.459 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012079+0000", 2024-09-14T22:36:51.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045971953000000003 2024-09-14T22:36:51.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.459 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:51.459 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012168+0000", 2024-09-14T22:36:51.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9398e-05 2024-09-14T22:36:51.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.459 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:51.459 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012273+0000", 2024-09-14T22:36:51.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000104483 2024-09-14T22:36:51.459 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.459 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:51.460 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.460 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6294 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8345600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:51.460 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966115+0000", 2024-09-14T22:36:51.460 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.433342353, 2024-09-14T22:36:51.460 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046297036999999999, 2024-09-14T22:36:51.460 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:51.460 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:51.460 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:51.460 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:51.460 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:51.460 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:51.460 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6294 2024-09-14T22:36:51.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.461 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:51.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:51.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966115+0000", 2024-09-14T22:36:51.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:51.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966115+0000", 2024-09-14T22:36:51.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:51.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966115+0000", 2024-09-14T22:36:51.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7e-07 2024-09-14T22:36:51.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:51.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966118+0000", 2024-09-14T22:36:51.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.571e-06 2024-09-14T22:36:51.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:51.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966125+0000", 2024-09-14T22:36:51.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1690000000000003e-06 2024-09-14T22:36:51.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:51.463 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966130+0000", 2024-09-14T22:36:51.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3290000000000001e-06 2024-09-14T22:36:51.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.463 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:51.463 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012298+0000", 2024-09-14T22:36:51.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046168168000000002 2024-09-14T22:36:51.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.463 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:51.463 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012348+0000", 2024-09-14T22:36:51.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9663999999999997e-05 2024-09-14T22:36:51.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.464 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:51.464 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012412+0000", 2024-09-14T22:36:51.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3665999999999997e-05 2024-09-14T22:36:51.464 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.464 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:51.464 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.464 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6295 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8353792~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:51.464 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966138+0000", 2024-09-14T22:36:51.464 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.433319375, 2024-09-14T22:36:51.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046505112000000001, 2024-09-14T22:36:51.464 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:51.464 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:51.465 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:51.465 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:51.465 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:51.465 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:51.465 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6295 2024-09-14T22:36:51.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.465 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:51.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.465 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:51.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966138+0000", 2024-09-14T22:36:51.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:51.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966138+0000", 2024-09-14T22:36:51.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:51.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966138+0000", 2024-09-14T22:36:51.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5900000000000002e-07 2024-09-14T22:36:51.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:51.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966140+0000", 2024-09-14T22:36:51.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4710000000000001e-06 2024-09-14T22:36:51.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:51.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966147+0000", 2024-09-14T22:36:51.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6760000000000001e-06 2024-09-14T22:36:51.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:51.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966153+0000", 2024-09-14T22:36:51.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4779999999999999e-06 2024-09-14T22:36:51.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:51.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012452+0000", 2024-09-14T22:36:51.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046299298000000003 2024-09-14T22:36:51.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:51.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012537+0000", 2024-09-14T22:36:51.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5439999999999995e-05 2024-09-14T22:36:51.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:51.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012643+0000", 2024-09-14T22:36:51.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010529 2024-09-14T22:36:51.468 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.468 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:51.468 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.469 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6296 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8361984~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:51.469 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966160+0000", 2024-09-14T22:36:51.469 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.433297321000001, 2024-09-14T22:36:51.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046601423000000003, 2024-09-14T22:36:51.469 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:51.469 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:51.469 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:51.469 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:51.469 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:51.469 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:51.469 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6296 2024-09-14T22:36:51.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.469 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:51.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:51.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966160+0000", 2024-09-14T22:36:51.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:51.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966160+0000", 2024-09-14T22:36:51.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:51.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966160+0000", 2024-09-14T22:36:51.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7999999999999995e-07 2024-09-14T22:36:51.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:51.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966163+0000", 2024-09-14T22:36:51.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.514e-06 2024-09-14T22:36:51.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:51.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966169+0000", 2024-09-14T22:36:51.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8880000000000002e-06 2024-09-14T22:36:51.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:51.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966173+0000", 2024-09-14T22:36:51.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9189999999999998e-06 2024-09-14T22:36:51.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:51.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012660+0000", 2024-09-14T22:36:51.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046486522000000002 2024-09-14T22:36:51.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:51.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012703+0000", 2024-09-14T22:36:51.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2715e-05 2024-09-14T22:36:51.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:51.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012761+0000", 2024-09-14T22:36:51.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8285000000000002e-05 2024-09-14T22:36:51.473 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.473 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:51.473 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.473 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6297 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8370176~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:51.473 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966180+0000", 2024-09-14T22:36:51.473 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.433276516999999, 2024-09-14T22:36:51.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046808482999999998, 2024-09-14T22:36:51.473 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:51.474 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:51.474 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:51.474 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:51.474 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:51.474 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:51.474 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6297 2024-09-14T22:36:51.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.474 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:51.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:51.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966180+0000", 2024-09-14T22:36:51.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:51.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966180+0000", 2024-09-14T22:36:51.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:51.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966181+0000", 2024-09-14T22:36:51.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0600000000000001e-07 2024-09-14T22:36:51.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:51.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966183+0000", 2024-09-14T22:36:51.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3539999999999998e-06 2024-09-14T22:36:51.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:51.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966190+0000", 2024-09-14T22:36:51.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.489e-06 2024-09-14T22:36:51.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:51.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966195+0000", 2024-09-14T22:36:51.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3890000000000004e-06 2024-09-14T22:36:51.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:51.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012799+0000", 2024-09-14T22:36:51.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046604200999999998 2024-09-14T22:36:51.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:51.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012883+0000", 2024-09-14T22:36:51.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3714000000000005e-05 2024-09-14T22:36:51.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:51.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012989+0000", 2024-09-14T22:36:51.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010593 2024-09-14T22:36:51.477 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.477 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:51.477 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.478 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6298 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8378368~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:51.478 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966203+0000", 2024-09-14T22:36:51.478 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.433253455999999, 2024-09-14T22:36:51.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046904649999999999, 2024-09-14T22:36:51.478 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:51.478 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:51.478 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:51.478 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:51.478 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:51.478 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:51.478 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6298 2024-09-14T22:36:51.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.478 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:51.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:51.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966203+0000", 2024-09-14T22:36:51.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:51.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966203+0000", 2024-09-14T22:36:51.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:51.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966205+0000", 2024-09-14T22:36:51.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5009999999999999e-06 2024-09-14T22:36:51.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:51.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966207+0000", 2024-09-14T22:36:51.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2019999999999998e-06 2024-09-14T22:36:51.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:51.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966214+0000", 2024-09-14T22:36:51.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.815e-06 2024-09-14T22:36:51.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:51.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966219+0000", 2024-09-14T22:36:51.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0590000000000002e-06 2024-09-14T22:36:51.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:51.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013005+0000", 2024-09-14T22:36:51.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046785852000000003 2024-09-14T22:36:51.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:51.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013050+0000", 2024-09-14T22:36:51.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.511e-05 2024-09-14T22:36:51.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:51.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013108+0000", 2024-09-14T22:36:51.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8111e-05 2024-09-14T22:36:51.482 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.482 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:51.482 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.482 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6299 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8386560~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:51.482 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966238+0000", 2024-09-14T22:36:51.482 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.433219135000002, 2024-09-14T22:36:51.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047091099999999997, 2024-09-14T22:36:51.482 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:51.482 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:51.482 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:51.483 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:51.483 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:51.483 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:51.483 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6299 2024-09-14T22:36:51.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.483 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:51.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:51.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966238+0000", 2024-09-14T22:36:51.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:51.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966238+0000", 2024-09-14T22:36:51.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.484 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.484 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:51.484 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966238+0000", 2024-09-14T22:36:51.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1500000000000005e-07 2024-09-14T22:36:51.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.484 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.484 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:51.484 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966241+0000", 2024-09-14T22:36:51.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8270000000000002e-06 2024-09-14T22:36:51.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.484 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.484 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:51.484 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966249+0000", 2024-09-14T22:36:51.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0029999999999994e-06 2024-09-14T22:36:51.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:51.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966255+0000", 2024-09-14T22:36:51.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7760000000000003e-06 2024-09-14T22:36:51.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:51.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013136+0000", 2024-09-14T22:36:51.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046881379000000001 2024-09-14T22:36:51.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:51.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013223+0000", 2024-09-14T22:36:51.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6681000000000006e-05 2024-09-14T22:36:51.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:51.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013329+0000", 2024-09-14T22:36:51.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105919 2024-09-14T22:36:51.486 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.486 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:51.486 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.486 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:31370 3.0 3:02f48f58:::rbd_data.113cda0183a0.0000000000000000:6 [read 813056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:51.486 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:36:51.486 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.037059041999999, 2024-09-14T22:36:51.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047519796000000003, 2024-09-14T22:36:51.486 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:51.486 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:51.487 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:51.487 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:51.487 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:51.487 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:51.487 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 31370 2024-09-14T22:36:51.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.487 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:51.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:51.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:36:51.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:51.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:36:51.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.488 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:51.488 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:36:51.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0999999999999999e-07 2024-09-14T22:36:51.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.488 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:51.488 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362401+0000", 2024-09-14T22:36:51.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2900000000000001e-06 2024-09-14T22:36:51.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.488 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:51.488 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362406+0000", 2024-09-14T22:36:51.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6960000000000002e-06 2024-09-14T22:36:51.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:51.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362412+0000", 2024-09-14T22:36:51.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.823e-06 2024-09-14T22:36:51.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:51.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.400270+0000", 2024-09-14T22:36:51.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037857737000000002 2024-09-14T22:36:51.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:51.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.400342+0000", 2024-09-14T22:36:51.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1797000000000004e-05 2024-09-14T22:36:51.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:51.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.409918+0000", 2024-09-14T22:36:51.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0095760429999999994 2024-09-14T22:36:51.490 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.490 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:51.490 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.490 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.490 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:51.490 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:51.491 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:36:51.492 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:51.487+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:51.592 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:51.587+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:51.622 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:51.622 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:36:51.623 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:36:51.623 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:51.639 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:36:51.693 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:51.687+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:51.741 INFO:teuthology.orchestra.run.smithi001.stdout:1786 write 0xd912e97 thru 0xd9182ec (0x5456 bytes) 2024-09-14T22:36:51.748 INFO:teuthology.orchestra.run.smithi001.stdout:1787 clone 12 order 21 su 524288 sc 8 2024-09-14T22:36:51.795 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:51.787+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:51.815 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:51.815 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:36:51.815 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:36:51.815 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:36:51.815 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:51.829 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:36:51.895 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:51.891+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:51.960 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:51.960 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:36:51.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:36:51.960 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:36:51.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.960 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35300 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1253376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:51.960 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.506930+0000", 2024-09-14T22:36:51.961 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.450269881000001, 2024-09-14T22:36:51.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044627325000000002, 2024-09-14T22:36:51.961 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:51.961 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:51.961 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:51.961 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:51.961 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:51.961 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:51.961 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35300 2024-09-14T22:36:51.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.961 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:51.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:51.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506930+0000", 2024-09-14T22:36:51.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:51.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506930+0000", 2024-09-14T22:36:51.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:51.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506930+0000", 2024-09-14T22:36:51.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2500000000000001e-07 2024-09-14T22:36:51.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:51.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506932+0000", 2024-09-14T22:36:51.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8300000000000001e-06 2024-09-14T22:36:51.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:51.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506940+0000", 2024-09-14T22:36:51.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2810000000000003e-06 2024-09-14T22:36:51.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:51.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506944+0000", 2024-09-14T22:36:51.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8049999999999997e-06 2024-09-14T22:36:51.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:51.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.551359+0000", 2024-09-14T22:36:51.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044414370000000002 2024-09-14T22:36:51.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:51.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.551432+0000", 2024-09-14T22:36:51.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3338999999999997e-05 2024-09-14T22:36:51.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:51.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.551557+0000", 2024-09-14T22:36:51.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000125275 2024-09-14T22:36:51.964 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.964 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:51.964 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.965 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35301 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1282048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:51.965 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.506950+0000", 2024-09-14T22:36:51.965 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.450249679999999, 2024-09-14T22:36:51.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044873612, 2024-09-14T22:36:51.965 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:51.965 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:51.965 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:51.965 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:51.965 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:51.965 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:51.965 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35301 2024-09-14T22:36:51.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.965 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:51.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:51.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506950+0000", 2024-09-14T22:36:51.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:51.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506950+0000", 2024-09-14T22:36:51.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:51.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506951+0000", 2024-09-14T22:36:51.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0400000000000002e-07 2024-09-14T22:36:51.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:51.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506953+0000", 2024-09-14T22:36:51.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9470000000000002e-06 2024-09-14T22:36:51.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:51.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506958+0000", 2024-09-14T22:36:51.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8930000000000002e-06 2024-09-14T22:36:51.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:51.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506963+0000", 2024-09-14T22:36:51.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.459e-06 2024-09-14T22:36:51.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:51.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.551642+0000", 2024-09-14T22:36:51.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044678766000000002 2024-09-14T22:36:51.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:51.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.551715+0000", 2024-09-14T22:36:51.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3603999999999995e-05 2024-09-14T22:36:51.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:51.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.551824+0000", 2024-09-14T22:36:51.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108539 2024-09-14T22:36:51.969 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.969 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:51.969 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.969 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35302 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1310720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:51.969 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.506970+0000", 2024-09-14T22:36:51.969 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.450229969999999, 2024-09-14T22:36:51.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045057898999999998, 2024-09-14T22:36:51.969 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:51.969 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:51.969 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:51.969 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:51.970 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:51.970 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:51.970 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35302 2024-09-14T22:36:51.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.970 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:51.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:51.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506970+0000", 2024-09-14T22:36:51.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:51.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506970+0000", 2024-09-14T22:36:51.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:51.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506970+0000", 2024-09-14T22:36:51.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0999999999999999e-07 2024-09-14T22:36:51.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:51.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506973+0000", 2024-09-14T22:36:51.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4310000000000001e-06 2024-09-14T22:36:51.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:51.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506981+0000", 2024-09-14T22:36:51.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2830000000000001e-06 2024-09-14T22:36:51.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:51.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506986+0000", 2024-09-14T22:36:51.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3959999999999997e-06 2024-09-14T22:36:51.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:51.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.551856+0000", 2024-09-14T22:36:51.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044869603000000001 2024-09-14T22:36:51.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.973 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:51.973 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.551931+0000", 2024-09-14T22:36:51.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5253999999999995e-05 2024-09-14T22:36:51.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.973 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:51.973 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552028+0000", 2024-09-14T22:36:51.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6521999999999995e-05 2024-09-14T22:36:51.973 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.973 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:51.973 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.974 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35303 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1339392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:51.974 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.506994+0000", 2024-09-14T22:36:51.974 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.450206061999999, 2024-09-14T22:36:51.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045224116000000002, 2024-09-14T22:36:51.974 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:51.974 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:51.974 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:51.974 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:51.974 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:51.974 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:51.974 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35303 2024-09-14T22:36:51.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.975 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:51.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:51.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506994+0000", 2024-09-14T22:36:51.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:51.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506994+0000", 2024-09-14T22:36:51.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:51.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506994+0000", 2024-09-14T22:36:51.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1699999999999999e-07 2024-09-14T22:36:51.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:51.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506997+0000", 2024-09-14T22:36:51.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.357e-06 2024-09-14T22:36:51.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:51.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507005+0000", 2024-09-14T22:36:51.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1939999999999998e-06 2024-09-14T22:36:51.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:51.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507013+0000", 2024-09-14T22:36:51.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3930000000000008e-06 2024-09-14T22:36:51.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:51.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552058+0000", 2024-09-14T22:36:51.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045044645000000001 2024-09-14T22:36:51.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:51.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552132+0000", 2024-09-14T22:36:51.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3831000000000005e-05 2024-09-14T22:36:51.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:51.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552218+0000", 2024-09-14T22:36:51.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6279000000000005e-05 2024-09-14T22:36:51.978 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.978 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:51.978 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.978 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35304 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1368064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:51.978 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507021+0000", 2024-09-14T22:36:51.978 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.450178502, 2024-09-14T22:36:51.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045388246, 2024-09-14T22:36:51.978 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:51.978 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:51.978 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:51.978 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:51.979 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:51.979 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:51.979 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35304 2024-09-14T22:36:51.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.979 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:51.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:51.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507021+0000", 2024-09-14T22:36:51.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:51.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507021+0000", 2024-09-14T22:36:51.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:51.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507022+0000", 2024-09-14T22:36:51.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.82e-07 2024-09-14T22:36:51.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:51.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507024+0000", 2024-09-14T22:36:51.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4949999999999998e-06 2024-09-14T22:36:51.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:51.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507032+0000", 2024-09-14T22:36:51.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9449999999999993e-06 2024-09-14T22:36:51.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:51.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507039+0000", 2024-09-14T22:36:51.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2129999999999996e-06 2024-09-14T22:36:51.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:51.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552250+0000", 2024-09-14T22:36:51.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045211150999999998 2024-09-14T22:36:51.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:51.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552326+0000", 2024-09-14T22:36:51.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6008999999999997e-05 2024-09-14T22:36:51.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:51.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552410+0000", 2024-09-14T22:36:51.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3950999999999996e-05 2024-09-14T22:36:51.982 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.982 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:51.982 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.982 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35305 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1396736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:51.982 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507048+0000", 2024-09-14T22:36:51.982 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.450151964, 2024-09-14T22:36:51.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045554728000000003, 2024-09-14T22:36:51.983 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:51.983 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:51.983 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:51.983 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:51.983 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:51.983 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:51.983 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35305 2024-09-14T22:36:51.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.983 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:51.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:51.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507048+0000", 2024-09-14T22:36:51.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:51.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507048+0000", 2024-09-14T22:36:51.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:51.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507048+0000", 2024-09-14T22:36:51.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7800000000000002e-07 2024-09-14T22:36:51.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:51.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507050+0000", 2024-09-14T22:36:51.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.024e-06 2024-09-14T22:36:51.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:51.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507057+0000", 2024-09-14T22:36:51.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1800000000000001e-06 2024-09-14T22:36:51.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:51.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507062+0000", 2024-09-14T22:36:51.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0930000000000002e-06 2024-09-14T22:36:51.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:51.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552439+0000", 2024-09-14T22:36:51.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045377460000000001 2024-09-14T22:36:51.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:51.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552512+0000", 2024-09-14T22:36:51.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2633999999999996e-05 2024-09-14T22:36:51.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:51.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552603+0000", 2024-09-14T22:36:51.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0958999999999994e-05 2024-09-14T22:36:51.986 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.986 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:51.987 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.987 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35306 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1425408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:51.987 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507069+0000", 2024-09-14T22:36:51.987 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.450130421000001, 2024-09-14T22:36:51.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045725259999999997, 2024-09-14T22:36:51.987 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:51.987 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:51.987 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:51.987 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:51.987 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:51.987 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:51.987 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35306 2024-09-14T22:36:51.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.988 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:51.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:51.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507069+0000", 2024-09-14T22:36:51.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:51.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507069+0000", 2024-09-14T22:36:51.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:51.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507070+0000", 2024-09-14T22:36:51.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0400000000000002e-07 2024-09-14T22:36:51.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:51.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507072+0000", 2024-09-14T22:36:51.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.891e-06 2024-09-14T22:36:51.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:51.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507078+0000", 2024-09-14T22:36:51.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0279999999999997e-06 2024-09-14T22:36:51.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:51.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507082+0000", 2024-09-14T22:36:51.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4680000000000003e-06 2024-09-14T22:36:51.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:51.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552632+0000", 2024-09-14T22:36:51.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045550144000000001 2024-09-14T22:36:51.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:51.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552707+0000", 2024-09-14T22:36:51.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4148999999999998e-05 2024-09-14T22:36:51.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:51.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552795+0000", 2024-09-14T22:36:51.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8176000000000004e-05 2024-09-14T22:36:51.991 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.991 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:51.991 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.991 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35307 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1454080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:51.991 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:36:51.991 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.450111870000001, 2024-09-14T22:36:51.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045891358, 2024-09-14T22:36:51.991 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:51.991 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:51.991 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:51.991 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:51.991 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:51.991 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:51.992 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35307 2024-09-14T22:36:51.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.992 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:51.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:51.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:36:51.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:51.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:36:51.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:51.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:36:51.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4700000000000002e-07 2024-09-14T22:36:51.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:51.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507093+0000", 2024-09-14T22:36:51.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9520000000000002e-06 2024-09-14T22:36:51.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:51.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507099+0000", 2024-09-14T22:36:51.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0240000000000003e-06 2024-09-14T22:36:51.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:51.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507105+0000", 2024-09-14T22:36:51.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5520000000000004e-06 2024-09-14T22:36:51.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:51.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552826+0000", 2024-09-14T22:36:51.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045721494000000001 2024-09-14T22:36:51.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:51.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552896+0000", 2024-09-14T22:36:51.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9519999999999998e-05 2024-09-14T22:36:51.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:51.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552979+0000", 2024-09-14T22:36:51.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3468999999999997e-05 2024-09-14T22:36:51.995 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.995 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:51.995 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.995 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35308 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1482752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:51.995 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:36:51.995 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.450088873999999, 2024-09-14T22:36:51.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046076533000000003, 2024-09-14T22:36:51.995 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:51.996 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:51.996 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:51.996 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:51.996 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:51.996 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:51.996 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35308 2024-09-14T22:36:51.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.996 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:51.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:51.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:36:51.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:51.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:36:51.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:51.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:51.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:36:51.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:36:51.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:51.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507113+0000", 2024-09-14T22:36:51.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9099999999999999e-06 2024-09-14T22:36:51.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:51.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507119+0000", 2024-09-14T22:36:51.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7440000000000004e-06 2024-09-14T22:36:51.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:51.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507124+0000", 2024-09-14T22:36:51.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2129999999999999e-06 2024-09-14T22:36:51.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:51.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553007+0000", 2024-09-14T22:36:51.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045882488999999999 2024-09-14T22:36:51.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:51.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553076+0000", 2024-09-14T22:36:51.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9233000000000006e-05 2024-09-14T22:36:51.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:51.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553188+0000", 2024-09-14T22:36:51.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111557 2024-09-14T22:36:51.999 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.999 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:51.999 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:51.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:51.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:51.999 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35309 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1511424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:51.999 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:36:51.999 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.450069969000001, 2024-09-14T22:36:51.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046231869000000002, 2024-09-14T22:36:51.999 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:52.000 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:52.000 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:52.000 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:52.000 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:52.000 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:52.000 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35309 2024-09-14T22:36:52.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.000 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:52.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:52.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:36:52.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:52.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:52.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:36:52.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:52.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:52.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:36:52.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9099999999999999e-07 2024-09-14T22:36:52.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:52.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507132+0000", 2024-09-14T22:36:52.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8190000000000001e-06 2024-09-14T22:36:52.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.002 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:52.002 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507139+0000", 2024-09-14T22:36:52.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9419999999999998e-06 2024-09-14T22:36:52.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.002 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:52.002 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507145+0000", 2024-09-14T22:36:52.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2600000000000002e-06 2024-09-14T22:36:52.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.002 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:52.002 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553225+0000", 2024-09-14T22:36:52.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046079991000000001 2024-09-14T22:36:52.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:52.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553284+0000", 2024-09-14T22:36:52.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8638999999999999e-05 2024-09-14T22:36:52.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:52.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553362+0000", 2024-09-14T22:36:52.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7826999999999999e-05 2024-09-14T22:36:52.003 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:52.003 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:52.003 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:52.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.003 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35310 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1540096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:52.003 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507151+0000", 2024-09-14T22:36:52.004 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.450048512999999, 2024-09-14T22:36:52.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046444541999999998, 2024-09-14T22:36:52.004 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:52.004 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:52.004 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:52.004 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:52.004 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:52.004 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:52.004 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35310 2024-09-14T22:36:52.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.004 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:52.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:52.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507151+0000", 2024-09-14T22:36:52.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:52.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:52.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507151+0000", 2024-09-14T22:36:52.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:52.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:52.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507152+0000", 2024-09-14T22:36:52.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1699999999999999e-07 2024-09-14T22:36:52.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:52.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507154+0000", 2024-09-14T22:36:52.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.902e-06 2024-09-14T22:36:52.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:52.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507159+0000", 2024-09-14T22:36:52.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.823e-06 2024-09-14T22:36:52.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:52.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507165+0000", 2024-09-14T22:36:52.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.587e-06 2024-09-14T22:36:52.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:52.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553395+0000", 2024-09-14T22:36:52.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046230318999999999 2024-09-14T22:36:52.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:52.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553434+0000", 2024-09-14T22:36:52.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8680999999999998e-05 2024-09-14T22:36:52.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:52.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553596+0000", 2024-09-14T22:36:52.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016191299999999999 2024-09-14T22:36:52.007 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:52.007 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:52.007 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:52.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.008 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35311 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1568768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:52.008 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:36:52.008 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.450029260000001, 2024-09-14T22:36:52.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046623592999999998, 2024-09-14T22:36:52.008 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:52.008 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:52.008 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:52.008 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:52.008 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:52.008 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:52.008 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35311 2024-09-14T22:36:52.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.008 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:52.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:52.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:36:52.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:52.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:52.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:36:52.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:52.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:52.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:36:52.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:36:52.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:52.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507176+0000", 2024-09-14T22:36:52.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3349999999999997e-06 2024-09-14T22:36:52.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:52.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507200+0000", 2024-09-14T22:36:52.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4003e-05 2024-09-14T22:36:52.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:52.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507206+0000", 2024-09-14T22:36:52.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.976e-06 2024-09-14T22:36:52.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:52.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553627+0000", 2024-09-14T22:36:52.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046420284999999999 2024-09-14T22:36:52.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:52.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553696+0000", 2024-09-14T22:36:52.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9567999999999995e-05 2024-09-14T22:36:52.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:52.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553794+0000", 2024-09-14T22:36:52.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8060000000000006e-05 2024-09-14T22:36:52.011 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:52.012 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:52.012 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:52.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.012 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35562 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1370112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:52.012 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514584+0000", 2024-09-14T22:36:52.012 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.442615873000001, 2024-09-14T22:36:52.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044805208999999999, 2024-09-14T22:36:52.012 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:52.012 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:52.012 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:52.012 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:52.012 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:52.012 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:52.012 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35562 2024-09-14T22:36:52.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.013 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:52.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:52.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514584+0000", 2024-09-14T22:36:52.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:52.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:52.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514584+0000", 2024-09-14T22:36:52.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:52.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:52.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514584+0000", 2024-09-14T22:36:52.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0900000000000002e-07 2024-09-14T22:36:52.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:52.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514587+0000", 2024-09-14T22:36:52.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1739999999999999e-06 2024-09-14T22:36:52.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:52.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514593+0000", 2024-09-14T22:36:52.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3080000000000004e-06 2024-09-14T22:36:52.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:52.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514600+0000", 2024-09-14T22:36:52.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6420000000000001e-06 2024-09-14T22:36:52.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:52.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559211+0000", 2024-09-14T22:36:52.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044611454000000002 2024-09-14T22:36:52.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:52.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559296+0000", 2024-09-14T22:36:52.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4676e-05 2024-09-14T22:36:52.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:52.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559389+0000", 2024-09-14T22:36:52.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3546000000000002e-05 2024-09-14T22:36:52.016 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:52.016 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:52.016 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:52.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.016 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35563 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1398784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:52.016 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514606+0000", 2024-09-14T22:36:52.016 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.442593544000001, 2024-09-14T22:36:52.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045029859999999998, 2024-09-14T22:36:52.016 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:52.016 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:52.016 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:52.017 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:52.017 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:52.017 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:52.017 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35563 2024-09-14T22:36:52.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.017 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:52.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:52.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514606+0000", 2024-09-14T22:36:52.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:52.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:52.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514606+0000", 2024-09-14T22:36:52.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:52.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:52.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514607+0000", 2024-09-14T22:36:52.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1199999999999998e-07 2024-09-14T22:36:52.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:52.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514609+0000", 2024-09-14T22:36:52.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.199e-06 2024-09-14T22:36:52.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:52.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514615+0000", 2024-09-14T22:36:52.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1260000000000003e-06 2024-09-14T22:36:52.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:52.019 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514620+0000", 2024-09-14T22:36:52.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.045e-06 2024-09-14T22:36:52.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:52.019 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559440+0000", 2024-09-14T22:36:52.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044820169999999999 2024-09-14T22:36:52.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:52.019 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559524+0000", 2024-09-14T22:36:52.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4124999999999998e-05 2024-09-14T22:36:52.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.020 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:52.020 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559636+0000", 2024-09-14T22:36:52.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111783 2024-09-14T22:36:52.020 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:52.020 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:52.020 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:52.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.020 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35564 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1427456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:52.020 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514627+0000", 2024-09-14T22:36:52.020 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.442572805000001, 2024-09-14T22:36:52.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045232025000000002, 2024-09-14T22:36:52.020 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:52.021 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:52.021 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:52.021 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:52.021 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:52.021 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:52.021 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35564 2024-09-14T22:36:52.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.021 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:52.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:52.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514627+0000", 2024-09-14T22:36:52.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:52.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:52.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514627+0000", 2024-09-14T22:36:52.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:52.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:52.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514628+0000", 2024-09-14T22:36:52.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4299999999999998e-07 2024-09-14T22:36:52.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:52.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514630+0000", 2024-09-14T22:36:52.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1579999999999999e-06 2024-09-14T22:36:52.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:52.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514636+0000", 2024-09-14T22:36:52.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5520000000000001e-06 2024-09-14T22:36:52.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.023 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:52.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514641+0000", 2024-09-14T22:36:52.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6199999999999998e-06 2024-09-14T22:36:52.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.023 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:52.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559679+0000", 2024-09-14T22:36:52.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045038474000000002 2024-09-14T22:36:52.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.023 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:52.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559763+0000", 2024-09-14T22:36:52.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3357999999999997e-05 2024-09-14T22:36:52.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.024 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:52.024 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559859+0000", 2024-09-14T22:36:52.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6420000000000002e-05 2024-09-14T22:36:52.024 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:52.024 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:52.024 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:52.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.024 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35565 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1456128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:52.024 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514648+0000", 2024-09-14T22:36:52.024 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.442551906999999, 2024-09-14T22:36:52.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045442455999999999, 2024-09-14T22:36:52.025 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:52.025 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:52.025 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:52.025 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:52.025 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:52.025 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:52.025 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35565 2024-09-14T22:36:52.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.025 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:52.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:52.025 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514648+0000", 2024-09-14T22:36:52.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:52.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:52.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514648+0000", 2024-09-14T22:36:52.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:52.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:52.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514648+0000", 2024-09-14T22:36:52.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1399999999999997e-07 2024-09-14T22:36:52.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:52.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514651+0000", 2024-09-14T22:36:52.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.136e-06 2024-09-14T22:36:52.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:52.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514657+0000", 2024-09-14T22:36:52.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.241e-06 2024-09-14T22:36:52.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:52.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514662+0000", 2024-09-14T22:36:52.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9690000000000002e-06 2024-09-14T22:36:52.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:52.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559903+0000", 2024-09-14T22:36:52.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045241393999999997 2024-09-14T22:36:52.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:52.028 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559985+0000", 2024-09-14T22:36:52.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1460999999999998e-05 2024-09-14T22:36:52.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:52.028 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560090+0000", 2024-09-14T22:36:52.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010584100000000001 2024-09-14T22:36:52.028 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:52.028 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:52.028 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:52.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.028 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35566 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1484800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:52.029 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514668+0000", 2024-09-14T22:36:52.029 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.442531483, 2024-09-14T22:36:52.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045644974999999997, 2024-09-14T22:36:52.029 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:52.029 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:52.029 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:52.029 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:52.029 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:52.029 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:52.029 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35566 2024-09-14T22:36:52.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.029 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:52.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:52.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514668+0000", 2024-09-14T22:36:52.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:52.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:52.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514668+0000", 2024-09-14T22:36:52.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:52.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:52.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514669+0000", 2024-09-14T22:36:52.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9099999999999999e-07 2024-09-14T22:36:52.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:52.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514671+0000", 2024-09-14T22:36:52.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3800000000000001e-06 2024-09-14T22:36:52.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:52.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514677+0000", 2024-09-14T22:36:52.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9660000000000001e-06 2024-09-14T22:36:52.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:52.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514682+0000", 2024-09-14T22:36:52.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0939999999999998e-06 2024-09-14T22:36:52.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:52.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560133+0000", 2024-09-14T22:36:52.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045450398000000003 2024-09-14T22:36:52.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:52.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560207+0000", 2024-09-14T22:36:52.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4196999999999995e-05 2024-09-14T22:36:52.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:52.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560313+0000", 2024-09-14T22:36:52.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106549 2024-09-14T22:36:52.032 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:52.032 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:52.033 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:52.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.033 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35567 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1513472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:52.033 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:36:52.033 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.442510943999999, 2024-09-14T22:36:52.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045793215999999998, 2024-09-14T22:36:52.033 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:52.033 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:52.033 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:52.033 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:52.033 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:52.033 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:52.033 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35567 2024-09-14T22:36:52.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.033 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:52.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:52.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:36:52.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:52.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:52.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:36:52.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:52.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:52.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:36:52.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1899999999999998e-07 2024-09-14T22:36:52.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:52.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514692+0000", 2024-09-14T22:36:52.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2699999999999999e-06 2024-09-14T22:36:52.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:52.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514698+0000", 2024-09-14T22:36:52.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0939999999999996e-06 2024-09-14T22:36:52.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:52.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514703+0000", 2024-09-14T22:36:52.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3870000000000003e-06 2024-09-14T22:36:52.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:52.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560353+0000", 2024-09-14T22:36:52.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045649914 2024-09-14T22:36:52.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:52.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560416+0000", 2024-09-14T22:36:52.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3096999999999996e-05 2024-09-14T22:36:52.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:52.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560482+0000", 2024-09-14T22:36:52.037 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6035e-05 2024-09-14T22:36:52.037 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:52.037 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:52.037 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:52.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.037 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35568 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1542144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:52.037 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:36:52.037 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.442486146, 2024-09-14T22:36:52.037 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045995921000000002, 2024-09-14T22:36:52.037 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:52.037 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:52.037 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:52.037 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:52.037 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:52.037 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:52.038 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35568 2024-09-14T22:36:52.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.038 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:52.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.038 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:52.038 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:36:52.038 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:52.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.038 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:52.038 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:36:52.038 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:52.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.038 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:52.038 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:36:52.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4099999999999999e-07 2024-09-14T22:36:52.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.039 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:52.039 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514720+0000", 2024-09-14T22:36:52.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6869999999999999e-06 2024-09-14T22:36:52.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.039 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:52.039 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514727+0000", 2024-09-14T22:36:52.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3540000000000002e-06 2024-09-14T22:36:52.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.039 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:52.039 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514732+0000", 2024-09-14T22:36:52.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7690000000000004e-06 2024-09-14T22:36:52.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:52.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560523+0000", 2024-09-14T22:36:52.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045791373000000003 2024-09-14T22:36:52.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:52.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560606+0000", 2024-09-14T22:36:52.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2690999999999998e-05 2024-09-14T22:36:52.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:52.041 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560710+0000", 2024-09-14T22:36:52.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000103606 2024-09-14T22:36:52.041 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:52.041 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:52.041 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:52.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.041 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35569 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1570816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:52.041 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:36:52.041 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.442460337, 2024-09-14T22:36:52.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046110630999999999, 2024-09-14T22:36:52.041 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:52.041 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:52.041 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:52.041 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:52.042 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:52.042 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:52.042 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35569 2024-09-14T22:36:52.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.042 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:52.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.042 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:52.042 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:36:52.042 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:52.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.042 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:52.042 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:36:52.042 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:52.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:52.043 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:36:52.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.82e-07 2024-09-14T22:36:52.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:52.043 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514743+0000", 2024-09-14T22:36:52.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.269e-06 2024-09-14T22:36:52.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:52.043 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514750+0000", 2024-09-14T22:36:52.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7340000000000002e-06 2024-09-14T22:36:52.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:52.044 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514755+0000", 2024-09-14T22:36:52.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7260000000000002e-06 2024-09-14T22:36:52.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.044 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:52.044 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560731+0000", 2024-09-14T22:36:52.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045976377999999998 2024-09-14T22:36:52.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.044 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:52.044 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560777+0000", 2024-09-14T22:36:52.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5868999999999997e-05 2024-09-14T22:36:52.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:52.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:52.045 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:52.045 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560850+0000", 2024-09-14T22:36:52.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3172999999999999e-05 2024-09-14T22:36:52.045 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:52.045 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:52.045 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:52.045 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:52.045 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:52.045 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:52.046 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:36:52.047 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:52.039+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:52.147 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:52.143+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:52.153 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:36:52.178 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:52.178 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:52.178 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:36:52.178 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:52.192 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:36:52.248 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:52.243+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:52.314 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:36:52.314 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:36:52.315 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:52.315 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:52.315 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:36:52.315 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:36:52.315 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:52.328 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:36:52.331 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:36:52.331 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:36:52.331 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-14T22:36:52.348 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:36:52.348 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:36:52.367 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:36:52.368 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:36:52.371 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:52.367+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:52.387 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:36:52.387 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:36:52.407 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:36:52.407 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:36:52.427 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:36:52.427 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:36:52.431 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:36:52.431 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:36:52.449 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:52.443+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:52.496 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:52.496 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:52.510 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:36:52.552 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:52.547+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:52.651 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:52.651+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:52.669 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:52.669 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:52.683 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:36:52.721 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T22:36:52.752 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:52.747+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:52.812 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:52.813 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:52.820 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone11 from 0xd9182ed (overlap 0x1079292) to 0x0 2024-09-14T22:36:52.829 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:36:52.852 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:52.847+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:52.915 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-14T22:36:52.954 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:52.947+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:53.003 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [2] live_osds: [0, 1, 3, 4, 5, 6, 7] 2024-09-14T22:36:53.003 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:36:53.003 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T22:36:53.004 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T22:36:53.004 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:53.005 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:53.019 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:36:53.054 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:53.051+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:53.155 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:53.151+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:53.185 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:53.185 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:53.199 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:36:53.256 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:53.251+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:53.298 INFO:teuthology.orchestra.run.smithi001.stdout:1788 write 0x7ce59f5 thru 0x7cf2162 (0xc76e bytes) 2024-09-14T22:36:53.309 INFO:teuthology.orchestra.run.smithi001.stdout:1789 read 0x78d0a59 thru 0x78de4fd (0xdaa5 bytes) 2024-09-14T22:36:53.310 INFO:teuthology.orchestra.run.smithi001.stdout:1790 read 0xd8268d1 thru 0xd828399 (0x1ac9 bytes) 2024-09-14T22:36:53.310 INFO:teuthology.orchestra.run.smithi001.stdout:1793 read 0x18efe3c thru 0x18fbc13 (0xbdd8 bytes) 2024-09-14T22:36:53.311 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:36:53.311 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:53.311 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 52, 2024-09-14T22:36:53.312 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "f07f407a-b17d-42c4-bf73-92dd80114a4f", 2024-09-14T22:36:53.312 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-09-14T22:35:45.983055+0000", 2024-09-14T22:36:53.312 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-09-14T22:36:52.704621+0000", 2024-09-14T22:36:53.312 INFO:teuthology.orchestra.run.smithi001.stdout: "last_up_change": "2024-09-14T22:36:26.755008+0000", 2024-09-14T22:36:53.312 INFO:teuthology.orchestra.run.smithi001.stdout: "last_in_change": "2024-09-14T22:35:48.283160+0000", 2024-09-14T22:36:53.312 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T22:36:53.312 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_num": 5801984, 2024-09-14T22:36:53.312 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_set": [ 2024-09-14T22:36:53.312 INFO:teuthology.orchestra.run.smithi001.stdout: "noscrub", 2024-09-14T22:36:53.312 INFO:teuthology.orchestra.run.smithi001.stdout: "pglog_hardlimit", 2024-09-14T22:36:53.312 INFO:teuthology.orchestra.run.smithi001.stdout: "purged_snapdirs", 2024-09-14T22:36:53.312 INFO:teuthology.orchestra.run.smithi001.stdout: "recovery_deletes", 2024-09-14T22:36:53.312 INFO:teuthology.orchestra.run.smithi001.stdout: "sortbitwise" 2024-09-14T22:36:53.313 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:36:53.313 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_version": 5, 2024-09-14T22:36:53.313 INFO:teuthology.orchestra.run.smithi001.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T22:36:53.313 INFO:teuthology.orchestra.run.smithi001.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T22:36:53.313 INFO:teuthology.orchestra.run.smithi001.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T22:36:53.313 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_snapshot": "", 2024-09-14T22:36:53.313 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_max": 3, 2024-09-14T22:36:53.313 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd": 8, 2024-09-14T22:36:53.313 INFO:teuthology.orchestra.run.smithi001.stdout: "require_min_compat_client": "luminous", 2024-09-14T22:36:53.313 INFO:teuthology.orchestra.run.smithi001.stdout: "min_compat_client": "luminous", 2024-09-14T22:36:53.313 INFO:teuthology.orchestra.run.smithi001.stdout: "require_osd_release": "squid", 2024-09-14T22:36:53.313 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_crimson": false, 2024-09-14T22:36:53.313 INFO:teuthology.orchestra.run.smithi001.stdout: "pools": [ 2024-09-14T22:36:53.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.313 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 1, 2024-09-14T22:36:53.314 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": ".mgr", 2024-09-14T22:36:53.314 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-14T22:35:58.938784+0000", 2024-09-14T22:36:53.314 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-14T22:36:53.314 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool", 2024-09-14T22:36:53.314 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-14T22:36:53.314 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-14T22:36:53.314 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-14T22:36:53.314 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-14T22:36:53.314 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:36:53.314 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:36:53.314 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:36:53.314 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:36:53.314 INFO:teuthology.orchestra.run.smithi001.stdout: "is_stretch_pool": false, 2024-09-14T22:36:53.314 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-14T22:36:53.314 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:36:53.315 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 1, 2024-09-14T22:36:53.315 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 1, 2024-09-14T22:36:53.315 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 1, 2024-09-14T22:36:53.315 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 1, 2024-09-14T22:36:53.315 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 1, 2024-09-14T22:36:53.315 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-14T22:36:53.315 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-14T22:36:53.315 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-14T22:36:53.315 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-14T22:36:53.315 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-14T22:36:53.316 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-14T22:36:53.316 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-14T22:36:53.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.316 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "18", 2024-09-14T22:36:53.316 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-14T22:36:53.316 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:36:53.316 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:36:53.316 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-14T22:36:53.316 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:36:53.316 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 0, 2024-09-14T22:36:53.316 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 0, 2024-09-14T22:36:53.316 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-14T22:36:53.316 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-14T22:36:53.317 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-14T22:36:53.317 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-14T22:36:53.317 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-14T22:36:53.317 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-14T22:36:53.317 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-14T22:36:53.317 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-14T22:36:53.317 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-14T22:36:53.317 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-14T22:36:53.317 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-14T22:36:53.317 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:36:53.317 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:36:53.317 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:36:53.317 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-14T22:36:53.317 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-14T22:36:53.318 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-14T22:36:53.318 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-14T22:36:53.318 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-14T22:36:53.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.318 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-14T22:36:53.318 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-14T22:36:53.318 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-14T22:36:53.318 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:36:53.318 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:36:53.318 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:36:53.318 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:36:53.318 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-14T22:36:53.318 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-14T22:36:53.318 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-14T22:36:53.319 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-14T22:36:53.319 INFO:teuthology.orchestra.run.smithi001.stdout: "options": { 2024-09-14T22:36:53.319 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_max": 32, 2024-09-14T22:36:53.319 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_min": 1 2024-09-14T22:36:53.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.319 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-14T22:36:53.319 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": {} 2024-09-14T22:36:53.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.319 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-14T22:36:53.319 INFO:teuthology.orchestra.run.smithi001.stdout: "score_type": "Fair distribution", 2024-09-14T22:36:53.319 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 8, 2024-09-14T22:36:53.319 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 8, 2024-09-14T22:36:53.319 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 0.25, 2024-09-14T22:36:53.319 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 2, 2024-09-14T22:36:53.319 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 2, 2024-09-14T22:36:53.320 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 0.87667274475097656, 2024-09-14T22:36:53.320 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T22:36:53.320 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T22:36:53.320 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.320 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 2, 2024-09-14T22:36:53.320 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "rbd", 2024-09-14T22:36:53.320 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-14T22:36:04.304242+0000", 2024-09-14T22:36:53.320 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 8193, 2024-09-14T22:36:53.320 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T22:36:53.320 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-14T22:36:53.320 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-14T22:36:53.321 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-14T22:36:53.321 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-14T22:36:53.321 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:36:53.321 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:36:53.321 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:36:53.321 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:36:53.321 INFO:teuthology.orchestra.run.smithi001.stdout: "is_stretch_pool": false, 2024-09-14T22:36:53.321 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-14T22:36:53.321 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:36:53.321 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 8, 2024-09-14T22:36:53.321 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 8, 2024-09-14T22:36:53.321 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 8, 2024-09-14T22:36:53.321 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 8, 2024-09-14T22:36:53.321 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 8, 2024-09-14T22:36:53.322 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-14T22:36:53.322 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-14T22:36:53.322 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-14T22:36:53.322 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-14T22:36:53.322 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-14T22:36:53.322 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-14T22:36:53.322 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-14T22:36:53.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.322 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "22", 2024-09-14T22:36:53.322 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-14T22:36:53.322 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:36:53.322 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:36:53.322 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-14T22:36:53.322 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:36:53.322 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 2, 2024-09-14T22:36:53.323 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 22, 2024-09-14T22:36:53.323 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-14T22:36:53.323 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-14T22:36:53.323 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-14T22:36:53.323 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-14T22:36:53.323 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-14T22:36:53.323 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-14T22:36:53.323 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-14T22:36:53.323 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-14T22:36:53.323 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-14T22:36:53.323 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-14T22:36:53.323 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-14T22:36:53.323 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:36:53.324 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:36:53.324 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:36:53.324 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-14T22:36:53.324 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-14T22:36:53.324 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-14T22:36:53.324 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-14T22:36:53.324 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-14T22:36:53.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.324 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-14T22:36:53.324 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-14T22:36:53.324 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-14T22:36:53.324 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:36:53.324 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:36:53.324 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:36:53.325 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:36:53.325 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-14T22:36:53.325 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-14T22:36:53.325 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-14T22:36:53.325 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-14T22:36:53.325 INFO:teuthology.orchestra.run.smithi001.stdout: "options": {}, 2024-09-14T22:36:53.325 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-14T22:36:53.325 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd": {} 2024-09-14T22:36:53.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.325 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-14T22:36:53.325 INFO:teuthology.orchestra.run.smithi001.stdout: "score_type": "Fair distribution", 2024-09-14T22:36:53.325 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 2, 2024-09-14T22:36:53.325 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 2, 2024-09-14T22:36:53.325 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 0.75, 2024-09-14T22:36:53.325 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 1.5, 2024-09-14T22:36:53.326 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 1.5, 2024-09-14T22:36:53.326 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 0.87667274475097656, 2024-09-14T22:36:53.326 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T22:36:53.326 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T22:36:53.326 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.326 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 3, 2024-09-14T22:36:53.326 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "pool_client.0", 2024-09-14T22:36:53.326 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-14T22:36:14.374249+0000", 2024-09-14T22:36:53.326 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 8193, 2024-09-14T22:36:53.326 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T22:36:53.327 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-14T22:36:53.327 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-14T22:36:53.327 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-14T22:36:53.327 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-14T22:36:53.327 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:36:53.327 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:36:53.327 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:36:53.327 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:36:53.327 INFO:teuthology.orchestra.run.smithi001.stdout: "is_stretch_pool": false, 2024-09-14T22:36:53.327 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-14T22:36:53.327 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:36:53.327 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 32, 2024-09-14T22:36:53.327 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 32, 2024-09-14T22:36:53.327 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 32, 2024-09-14T22:36:53.327 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 32, 2024-09-14T22:36:53.328 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 32, 2024-09-14T22:36:53.328 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-14T22:36:53.328 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-14T22:36:53.328 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-14T22:36:53.328 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-14T22:36:53.328 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-14T22:36:53.328 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-14T22:36:53.328 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-14T22:36:53.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.328 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "52", 2024-09-14T22:36:53.328 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-14T22:36:53.328 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:36:53.328 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:36:53.328 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-14T22:36:53.329 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:36:53.329 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 14, 2024-09-14T22:36:53.329 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 52, 2024-09-14T22:36:53.329 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-14T22:36:53.329 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-14T22:36:53.329 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-14T22:36:53.329 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-14T22:36:53.329 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-14T22:36:53.329 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-14T22:36:53.329 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-14T22:36:53.329 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-14T22:36:53.329 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-14T22:36:53.329 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-14T22:36:53.329 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-14T22:36:53.330 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:36:53.330 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:36:53.330 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:36:53.330 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-14T22:36:53.330 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-14T22:36:53.330 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-14T22:36:53.330 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-14T22:36:53.330 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-14T22:36:53.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.330 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-14T22:36:53.330 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-14T22:36:53.330 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-14T22:36:53.330 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:36:53.330 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:36:53.330 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:36:53.331 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:36:53.331 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-14T22:36:53.331 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-14T22:36:53.331 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-14T22:36:53.331 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-14T22:36:53.331 INFO:teuthology.orchestra.run.smithi001.stdout: "options": {}, 2024-09-14T22:36:53.331 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-14T22:36:53.331 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd": {} 2024-09-14T22:36:53.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.331 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-14T22:36:53.331 INFO:teuthology.orchestra.run.smithi001.stdout: "score_type": "Fair distribution", 2024-09-14T22:36:53.331 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 1.7400000095367432, 2024-09-14T22:36:53.331 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 1.7400000095367432, 2024-09-14T22:36:53.331 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 0.87999999523162842, 2024-09-14T22:36:53.332 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 1.5299999713897705, 2024-09-14T22:36:53.332 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 1.5299999713897705, 2024-09-14T22:36:53.332 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 0.87667274475097656, 2024-09-14T22:36:53.332 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T22:36:53.332 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T22:36:53.332 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.332 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.332 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:36:53.332 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2024-09-14T22:36:53.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.332 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 0, 2024-09-14T22:36:53.332 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "9831fcc5-9040-4a80-99d4-838225e65139", 2024-09-14T22:36:53.332 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:36:53.332 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:36:53.333 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:36:53.333 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:36:53.333 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:36:53.333 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:36:53.333 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:36:53.333 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 42, 2024-09-14T22:36:53.333 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:36:53.333 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:36:53.333 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:36:53.333 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:53.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.333 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:53.333 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6824", 2024-09-14T22:36:53.333 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:36:53.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.334 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:53.334 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6825", 2024-09-14T22:36:53.334 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:36:53.334 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.334 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.334 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:36:53.334 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:53.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.334 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:53.334 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6826", 2024-09-14T22:36:53.334 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:36:53.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.335 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:53.335 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6827", 2024-09-14T22:36:53.335 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:36:53.335 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.335 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.335 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:36:53.335 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:53.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.335 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:53.335 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6830", 2024-09-14T22:36:53.335 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:36:53.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.336 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:53.336 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6831", 2024-09-14T22:36:53.336 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:36:53.336 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.336 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.336 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:36:53.336 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:53.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.336 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:53.336 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6828", 2024-09-14T22:36:53.336 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:36:53.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.336 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:53.336 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6829", 2024-09-14T22:36:53.337 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:36:53.337 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.337 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.337 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6825/1702929600", 2024-09-14T22:36:53.337 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6827/1702929600", 2024-09-14T22:36:53.337 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6831/1702929600", 2024-09-14T22:36:53.337 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6829/1702929600", 2024-09-14T22:36:53.337 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:36:53.337 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:36:53.337 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:36:53.337 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.338 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 1, 2024-09-14T22:36:53.338 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "7f79abe8-115d-46b2-8677-72fcb0c8ab7d", 2024-09-14T22:36:53.338 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:36:53.338 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:36:53.338 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:36:53.338 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 0.1606292724609375, 2024-09-14T22:36:53.338 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:36:53.338 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:36:53.338 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2024-09-14T22:36:53.338 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 31, 2024-09-14T22:36:53.338 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:36:53.338 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:36:53.338 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:36:53.338 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:53.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.339 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:53.339 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6808", 2024-09-14T22:36:53.339 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:36:53.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.339 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:53.339 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6809", 2024-09-14T22:36:53.339 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:36:53.339 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.339 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.339 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:36:53.340 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:53.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.340 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:53.340 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6810", 2024-09-14T22:36:53.340 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:36:53.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.340 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:53.340 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6811", 2024-09-14T22:36:53.340 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:36:53.340 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.340 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.340 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:36:53.340 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:53.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.341 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:53.341 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6814", 2024-09-14T22:36:53.341 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:36:53.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.341 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:53.341 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6815", 2024-09-14T22:36:53.341 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:36:53.341 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.341 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.341 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:36:53.341 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:53.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.342 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:53.342 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6812", 2024-09-14T22:36:53.342 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:36:53.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.342 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:53.342 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6813", 2024-09-14T22:36:53.342 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:36:53.342 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.342 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.342 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6809/719990653", 2024-09-14T22:36:53.342 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6811/719990653", 2024-09-14T22:36:53.342 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6815/719990653", 2024-09-14T22:36:53.343 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6813/719990653", 2024-09-14T22:36:53.343 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:36:53.343 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:36:53.343 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:36:53.343 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.343 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 2, 2024-09-14T22:36:53.343 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "1f5768b2-4db4-4b2e-8d5e-5206f8bde28d", 2024-09-14T22:36:53.343 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 0, 2024-09-14T22:36:53.343 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:36:53.343 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:36:53.343 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:36:53.343 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:36:53.344 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:36:53.344 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:36:53.344 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 19, 2024-09-14T22:36:53.344 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 31, 2024-09-14T22:36:53.344 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:36:53.344 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:36:53.344 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:53.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.344 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:53.344 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6816", 2024-09-14T22:36:53.344 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2536940607 2024-09-14T22:36:53.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.344 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:53.345 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6817", 2024-09-14T22:36:53.345 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2536940607 2024-09-14T22:36:53.345 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.345 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.345 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:36:53.345 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:53.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.345 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:53.345 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6818", 2024-09-14T22:36:53.345 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2536940607 2024-09-14T22:36:53.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.346 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:53.346 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6819", 2024-09-14T22:36:53.346 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2536940607 2024-09-14T22:36:53.346 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.346 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.346 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:36:53.346 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:53.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.346 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:53.346 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6822", 2024-09-14T22:36:53.346 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2536940607 2024-09-14T22:36:53.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.347 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:53.347 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6823", 2024-09-14T22:36:53.347 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2536940607 2024-09-14T22:36:53.347 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.347 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.347 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:36:53.347 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:53.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.347 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:53.347 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6820", 2024-09-14T22:36:53.347 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2536940607 2024-09-14T22:36:53.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.348 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:53.348 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6821", 2024-09-14T22:36:53.348 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2536940607 2024-09-14T22:36:53.348 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.348 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.348 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6817/2536940607", 2024-09-14T22:36:53.348 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6819/2536940607", 2024-09-14T22:36:53.348 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6823/2536940607", 2024-09-14T22:36:53.348 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6821/2536940607", 2024-09-14T22:36:53.348 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:36:53.348 INFO:teuthology.orchestra.run.smithi001.stdout: "exists" 2024-09-14T22:36:53.348 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.349 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 3, 2024-09-14T22:36:53.349 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "8103f3c7-7899-4fc3-bb02-dd2273120138", 2024-09-14T22:36:53.349 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:36:53.349 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:36:53.349 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:36:53.349 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 0.852752685546875, 2024-09-14T22:36:53.349 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:36:53.349 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:36:53.349 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:36:53.349 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 49, 2024-09-14T22:36:53.349 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:36:53.349 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:36:53.349 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:36:53.350 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:53.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.350 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:53.350 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6800", 2024-09-14T22:36:53.350 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:36:53.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.350 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:53.350 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6801", 2024-09-14T22:36:53.350 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:36:53.350 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.350 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.350 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:36:53.350 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:53.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.351 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:53.351 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6802", 2024-09-14T22:36:53.351 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:36:53.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.351 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:53.351 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6803", 2024-09-14T22:36:53.351 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:36:53.351 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.351 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.351 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:36:53.351 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:53.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.351 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:53.352 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6806", 2024-09-14T22:36:53.352 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:36:53.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.352 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:53.352 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6807", 2024-09-14T22:36:53.352 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:36:53.352 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.352 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.352 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:36:53.352 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:53.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.352 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:53.353 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6804", 2024-09-14T22:36:53.353 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:36:53.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.353 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:53.353 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6805", 2024-09-14T22:36:53.353 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:36:53.353 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.353 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.353 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6801/750403671", 2024-09-14T22:36:53.353 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6803/750403671", 2024-09-14T22:36:53.353 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6807/750403671", 2024-09-14T22:36:53.353 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6805/750403671", 2024-09-14T22:36:53.354 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:36:53.354 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:36:53.354 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:36:53.354 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.354 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 4, 2024-09-14T22:36:53.354 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "23cce5cb-726e-44c2-8edb-c2567a89dfbc", 2024-09-14T22:36:53.354 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:36:53.354 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:36:53.354 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:36:53.354 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:36:53.354 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:36:53.354 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:36:53.355 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:36:53.355 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 31, 2024-09-14T22:36:53.355 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:36:53.355 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:36:53.355 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:36:53.355 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:53.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.355 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:53.355 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6816", 2024-09-14T22:36:53.355 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:36:53.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.355 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:53.355 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6817", 2024-09-14T22:36:53.356 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:36:53.356 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.356 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.356 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:36:53.356 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:53.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.356 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:53.356 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6818", 2024-09-14T22:36:53.356 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:36:53.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.356 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:53.357 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6819", 2024-09-14T22:36:53.357 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:36:53.357 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.357 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.357 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:36:53.357 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:53.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.357 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:53.357 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6822", 2024-09-14T22:36:53.357 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:36:53.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.357 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:53.357 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6823", 2024-09-14T22:36:53.358 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:36:53.358 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.358 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.358 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:36:53.358 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:53.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.358 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:53.358 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6820", 2024-09-14T22:36:53.358 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:36:53.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.358 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:53.358 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6821", 2024-09-14T22:36:53.358 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:36:53.358 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.359 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.359 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.176:6817/1408471829", 2024-09-14T22:36:53.359 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.176:6819/1408471829", 2024-09-14T22:36:53.359 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.176:6823/1408471829", 2024-09-14T22:36:53.359 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.176:6821/1408471829", 2024-09-14T22:36:53.359 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:36:53.359 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:36:53.359 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:36:53.360 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.360 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 5, 2024-09-14T22:36:53.360 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "eb2f187b-d2f1-49ab-ad5e-ea6dc441a28e", 2024-09-14T22:36:53.360 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:36:53.360 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:36:53.360 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:36:53.360 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:36:53.360 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:36:53.360 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:36:53.360 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:36:53.360 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 31, 2024-09-14T22:36:53.360 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:36:53.361 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:36:53.361 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:36:53.361 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:53.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.361 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:53.361 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6805", 2024-09-14T22:36:53.361 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:36:53.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.361 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:53.361 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6808", 2024-09-14T22:36:53.361 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:36:53.361 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.361 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.362 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:36:53.362 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:53.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.362 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:53.362 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6810", 2024-09-14T22:36:53.362 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:36:53.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.362 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:53.362 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6811", 2024-09-14T22:36:53.362 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:36:53.362 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.362 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.363 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:36:53.363 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:53.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.363 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:53.363 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6814", 2024-09-14T22:36:53.363 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:36:53.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.363 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:53.363 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6815", 2024-09-14T22:36:53.363 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:36:53.363 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.363 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.364 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:36:53.364 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:53.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.364 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:53.364 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6812", 2024-09-14T22:36:53.364 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:36:53.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.364 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:53.364 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6813", 2024-09-14T22:36:53.364 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:36:53.364 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.364 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.365 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.176:6808/1945902326", 2024-09-14T22:36:53.365 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.176:6811/1945902326", 2024-09-14T22:36:53.365 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.176:6815/1945902326", 2024-09-14T22:36:53.365 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.176:6813/1945902326", 2024-09-14T22:36:53.365 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:36:53.365 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:36:53.365 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:36:53.365 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.365 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 6, 2024-09-14T22:36:53.365 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "d371aed8-2ad2-49a5-8ece-4b7fd37b4822", 2024-09-14T22:36:53.365 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:36:53.365 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:36:53.366 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:36:53.366 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:36:53.366 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:36:53.366 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:36:53.366 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:36:53.366 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 50, 2024-09-14T22:36:53.366 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:36:53.366 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:36:53.366 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:36:53.366 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:53.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.366 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:53.366 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6800", 2024-09-14T22:36:53.366 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:36:53.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.367 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:53.367 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6801", 2024-09-14T22:36:53.367 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:36:53.367 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.367 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.367 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:36:53.367 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:53.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.367 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:53.367 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6802", 2024-09-14T22:36:53.367 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:36:53.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.367 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:53.368 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6803", 2024-09-14T22:36:53.368 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:36:53.368 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.368 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.368 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:36:53.368 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:53.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.368 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:53.368 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6807", 2024-09-14T22:36:53.368 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:36:53.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.368 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:53.369 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6809", 2024-09-14T22:36:53.369 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:36:53.369 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.369 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.369 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:36:53.369 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:53.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.369 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:53.369 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6804", 2024-09-14T22:36:53.369 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:36:53.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.369 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:53.369 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6806", 2024-09-14T22:36:53.370 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:36:53.370 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.370 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.370 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.176:6801/4116085586", 2024-09-14T22:36:53.370 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.176:6803/4116085586", 2024-09-14T22:36:53.370 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.176:6809/4116085586", 2024-09-14T22:36:53.370 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.176:6806/4116085586", 2024-09-14T22:36:53.370 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:36:53.370 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:36:53.370 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:36:53.370 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.370 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 7, 2024-09-14T22:36:53.370 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "a012c819-8d49-4310-a840-c2752d9314ab", 2024-09-14T22:36:53.371 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:36:53.371 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:36:53.371 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:36:53.371 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:36:53.371 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:36:53.371 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:36:53.371 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:36:53.371 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 27, 2024-09-14T22:36:53.371 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:36:53.371 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:36:53.371 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:36:53.371 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:53.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.371 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:53.371 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6824", 2024-09-14T22:36:53.372 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:36:53.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.372 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:53.372 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6825", 2024-09-14T22:36:53.372 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:36:53.372 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.372 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.372 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:36:53.372 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:53.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.372 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:53.372 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6826", 2024-09-14T22:36:53.372 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:36:53.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.373 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:53.373 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6827", 2024-09-14T22:36:53.373 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:36:53.373 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.373 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.373 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:36:53.373 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:53.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.373 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:53.373 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6830", 2024-09-14T22:36:53.373 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:36:53.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.374 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:53.374 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6831", 2024-09-14T22:36:53.374 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:36:53.374 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.374 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.374 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:36:53.374 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:36:53.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.374 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:36:53.374 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6828", 2024-09-14T22:36:53.374 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:36:53.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.375 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:36:53.375 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6829", 2024-09-14T22:36:53.375 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:36:53.375 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.375 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.375 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.176:6825/877760244", 2024-09-14T22:36:53.375 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.176:6827/877760244", 2024-09-14T22:36:53.375 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.176:6831/877760244", 2024-09-14T22:36:53.375 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.176:6829/877760244", 2024-09-14T22:36:53.375 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:36:53.375 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:36:53.375 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:36:53.375 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.375 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.376 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:36:53.376 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_xinfo": [ 2024-09-14T22:36:53.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.376 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 0, 2024-09-14T22:36:53.376 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:36:53.376 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:36:53.376 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:36:53.376 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:36:53.376 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:36:53.376 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:55.511688+0000", 2024-09-14T22:36:53.376 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:36:53.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.376 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 1, 2024-09-14T22:36:53.377 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:36:53.377 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:36:53.377 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:36:53.377 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:36:53.377 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:36:53.377 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.742414+0000", 2024-09-14T22:36:53.377 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:36:53.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.377 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 2, 2024-09-14T22:36:53.377 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:36:53.377 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:36:53.377 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:36:53.377 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:36:53.378 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:36:53.378 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.871355+0000", 2024-09-14T22:36:53.378 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 31 2024-09-14T22:36:53.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.378 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 3, 2024-09-14T22:36:53.378 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:36:53.378 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:36:53.378 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:36:53.378 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:36:53.378 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:36:53.378 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.504553+0000", 2024-09-14T22:36:53.378 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:36:53.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.379 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 4, 2024-09-14T22:36:53.379 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:36:53.379 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:36:53.379 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:36:53.379 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:36:53.379 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:36:53.379 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.589640+0000", 2024-09-14T22:36:53.379 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:36:53.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.379 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 5, 2024-09-14T22:36:53.379 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:36:53.379 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:36:53.379 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:36:53.379 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:36:53.380 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:36:53.380 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.518845+0000", 2024-09-14T22:36:53.380 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:36:53.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.380 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 6, 2024-09-14T22:36:53.380 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:36:53.380 INFO:teuthology.orchestra.run.smithi001.stdout:1795 read 0x5559276 thru 0x556773c (0xe4c7 bytes) 2024-09-14T22:36:53.381 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:36:53.381 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:36:53.381 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:36:53.381 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:36:53.381 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.731632+0000", 2024-09-14T22:36:53.381 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:36:53.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.381 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 7, 2024-09-14T22:36:53.381 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:36:53.381 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:36:53.381 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:36:53.381 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:36:53.382 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:36:53.382 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.952634+0000", 2024-09-14T22:36:53.382 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:36:53.382 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.382 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:36:53.382 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap": [], 2024-09-14T22:36:53.382 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap_items": [ 2024-09-14T22:36:53.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.382 INFO:teuthology.orchestra.run.smithi001.stdout: "pgid": "3.2", 2024-09-14T22:36:53.382 INFO:teuthology.orchestra.run.smithi001.stdout: "mappings": [ 2024-09-14T22:36:53.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:53.382 INFO:teuthology.orchestra.run.smithi001.stdout: "from": 3, 2024-09-14T22:36:53.382 INFO:teuthology.orchestra.run.smithi001.stdout: "to": 6 2024-09-14T22:36:53.382 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.382 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:53.382 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.383 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:36:53.383 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap_primaries": [], 2024-09-14T22:36:53.383 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_temp": [], 2024-09-14T22:36:53.383 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_temp": [], 2024-09-14T22:36:53.383 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklist": {}, 2024-09-14T22:36:53.383 INFO:teuthology.orchestra.run.smithi001.stdout: "range_blocklist": {}, 2024-09-14T22:36:53.383 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profiles": { 2024-09-14T22:36:53.383 INFO:teuthology.orchestra.run.smithi001.stdout: "default": { 2024-09-14T22:36:53.383 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-failure-domain": "osd", 2024-09-14T22:36:53.383 INFO:teuthology.orchestra.run.smithi001.stdout: "k": "2", 2024-09-14T22:36:53.383 INFO:teuthology.orchestra.run.smithi001.stdout: "m": "1", 2024-09-14T22:36:53.383 INFO:teuthology.orchestra.run.smithi001.stdout: "plugin": "jerasure", 2024-09-14T22:36:53.383 INFO:teuthology.orchestra.run.smithi001.stdout: "technique": "reed_sol_van" 2024-09-14T22:36:53.383 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:53.384 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps_queue": [], 2024-09-14T22:36:53.384 INFO:teuthology.orchestra.run.smithi001.stdout: "new_removed_snaps": [], 2024-09-14T22:36:53.384 INFO:teuthology.orchestra.run.smithi001.stdout: "new_purged_snaps": [], 2024-09-14T22:36:53.384 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_node_flags": {}, 2024-09-14T22:36:53.384 INFO:teuthology.orchestra.run.smithi001.stdout: "device_class_flags": {}, 2024-09-14T22:36:53.384 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": { 2024-09-14T22:36:53.384 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode_enabled": false, 2024-09-14T22:36:53.384 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_bucket_count": 0, 2024-09-14T22:36:53.384 INFO:teuthology.orchestra.run.smithi001.stdout: "degraded_stretch_mode": 0, 2024-09-14T22:36:53.384 INFO:teuthology.orchestra.run.smithi001.stdout: "recovering_stretch_mode": 0, 2024-09-14T22:36:53.384 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode_bucket": 0 2024-09-14T22:36:53.384 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:53.384 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:53.385 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:53.385 INFO:teuthology.orchestra.run.smithi001.stdout:1798 write 0xd38acde thru 0xd38e204 (0x3527 bytes) 2024-09-14T22:36:53.385 INFO:teuthology.orchestra.run.smithi001.stdout:1801 punch from 0xcea825b to 0xceb5241, (0xcfe6 bytes) 2024-09-14T22:36:53.385 INFO:teuthology.orchestra.run.smithi001.stdout:1802 write 0x89a6874 thru 0x89b2597 (0xbd24 bytes) 2024-09-14T22:36:53.385 INFO:teuthology.orchestra.run.smithi001.stdout:1803 write 0xabb23f2 thru 0xabbfa31 (0xd640 bytes) 2024-09-14T22:36:53.387 INFO:tasks.thrashosds.thrasher:j is {'epoch': 52, 'fsid': 'f07f407a-b17d-42c4-bf73-92dd80114a4f', 'created': '2024-09-14T22:35:45.983055+0000', 'modified': '2024-09-14T22:36:52.704621+0000', 'last_up_change': '2024-09-14T22:36:26.755008+0000', 'last_in_change': '2024-09-14T22:35:48.283160+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T22:35:58.938784+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8766727447509766, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T22:36:04.304242+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 0.75, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.8766727447509766, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T22:36:14.374249+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 14, 'snap_epoch': 52, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7400000095367432, 'score_stable': 1.7400000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 0.8766727447509766, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': '9831fcc5-9040-4a80-99d4-838225e65139', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 42, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6824', 'nonce': 1702929600}, {'type': 'v1', 'addr': '172.21.15.1:6825', 'nonce': 1702929600}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6826', 'nonce': 1702929600}, {'type': 'v1', 'addr': '172.21.15.1:6827', 'nonce': 1702929600}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6830', 'nonce': 1702929600}, {'type': 'v1', 'addr': '172.21.15.1:6831', 'nonce': 1702929600}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6828', 'nonce': 1702929600}, {'type': 'v1', 'addr': '172.21.15.1:6829', 'nonce': 1702929600}]}, 'public_addr': '172.21.15.1:6825/1702929600', 'cluster_addr': '172.21.15.1:6827/1702929600', 'heartbeat_back_addr': '172.21.15.1:6831/1702929600', 'heartbeat_front_addr': '172.21.15.1:6829/1702929600', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '7f79abe8-115d-46b2-8677-72fcb0c8ab7d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.1606292724609375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6808', 'nonce': 719990653}, {'type': 'v1', 'addr': '172.21.15.1:6809', 'nonce': 719990653}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6810', 'nonce': 719990653}, {'type': 'v1', 'addr': '172.21.15.1:6811', 'nonce': 719990653}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6814', 'nonce': 719990653}, {'type': 'v1', 'addr': '172.21.15.1:6815', 'nonce': 719990653}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6812', 'nonce': 719990653}, {'type': 'v1', 'addr': '172.21.15.1:6813', 'nonce': 719990653}]}, 'public_addr': '172.21.15.1:6809/719990653', 'cluster_addr': '172.21.15.1:6811/719990653', 'heartbeat_back_addr': '172.21.15.1:6815/719990653', 'heartbeat_front_addr': '172.21.15.1:6813/719990653', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '1f5768b2-4db4-4b2e-8d5e-5206f8bde28d', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 19, 'down_at': 31, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6816', 'nonce': 2536940607}, {'type': 'v1', 'addr': '172.21.15.1:6817', 'nonce': 2536940607}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6818', 'nonce': 2536940607}, {'type': 'v1', 'addr': '172.21.15.1:6819', 'nonce': 2536940607}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6822', 'nonce': 2536940607}, {'type': 'v1', 'addr': '172.21.15.1:6823', 'nonce': 2536940607}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6820', 'nonce': 2536940607}, {'type': 'v1', 'addr': '172.21.15.1:6821', 'nonce': 2536940607}]}, 'public_addr': '172.21.15.1:6817/2536940607', 'cluster_addr': '172.21.15.1:6819/2536940607', 'heartbeat_back_addr': '172.21.15.1:6823/2536940607', 'heartbeat_front_addr': '172.21.15.1:6821/2536940607', 'state': ['exists']}, {'osd': 3, 'uuid': '8103f3c7-7899-4fc3-bb02-dd2273120138', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.852752685546875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6800', 'nonce': 750403671}, {'type': 'v1', 'addr': '172.21.15.1:6801', 'nonce': 750403671}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6802', 'nonce': 750403671}, {'type': 'v1', 'addr': '172.21.15.1:6803', 'nonce': 750403671}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6806', 'nonce': 750403671}, {'type': 'v1', 'addr': '172.21.15.1:6807', 'nonce': 750403671}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6804', 'nonce': 750403671}, {'type': 'v1', 'addr': '172.21.15.1:6805', 'nonce': 750403671}]}, 'public_addr': '172.21.15.1:6801/750403671', 'cluster_addr': '172.21.15.1:6803/750403671', 'heartbeat_back_addr': '172.21.15.1:6807/750403671', 'heartbeat_front_addr': '172.21.15.1:6805/750403671', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '23cce5cb-726e-44c2-8edb-c2567a89dfbc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6816', 'nonce': 1408471829}, {'type': 'v1', 'addr': '172.21.15.176:6817', 'nonce': 1408471829}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6818', 'nonce': 1408471829}, {'type': 'v1', 'addr': '172.21.15.176:6819', 'nonce': 1408471829}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6822', 'nonce': 1408471829}, {'type': 'v1', 'addr': '172.21.15.176:6823', 'nonce': 1408471829}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6820', 'nonce': 1408471829}, {'type': 'v1', 'addr': '172.21.15.176:6821', 'nonce': 1408471829}]}, 'public_addr': '172.21.15.176:6817/1408471829', 'cluster_addr': '172.21.15.176:6819/1408471829', 'heartbeat_back_addr': '172.21.15.176:6823/1408471829', 'heartbeat_front_addr': '172.21.15.176:6821/1408471829', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'eb2f187b-d2f1-49ab-ad5e-ea6dc441a28e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6805', 'nonce': 1945902326}, {'type': 'v1', 'addr': '172.21.15.176:6808', 'nonce': 1945902326}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6810', 'nonce': 1945902326}, {'type': 'v1', 'addr': '172.21.15.176:6811', 'nonce': 1945902326}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6814', 'nonce': 1945902326}, {'type': 'v1', 'addr': '172.21.15.176:6815', 'nonce': 1945902326}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6812', 'nonce': 1945902326}, {'type': 'v1', 'addr': '172.21.15.176:6813', 'nonce': 1945902326}]}, 'public_addr': '172.21.15.176:6808/1945902326', 'cluster_addr': '172.21.15.176:6811/1945902326', 'heartbeat_back_addr': '172.21.15.176:6815/1945902326', 'heartbeat_front_addr': '172.21.15.176:6813/1945902326', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'd371aed8-2ad2-49a5-8ece-4b7fd37b4822', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6800', 'nonce': 4116085586}, {'type': 'v1', 'addr': '172.21.15.176:6801', 'nonce': 4116085586}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6802', 'nonce': 4116085586}, {'type': 'v1', 'addr': '172.21.15.176:6803', 'nonce': 4116085586}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6807', 'nonce': 4116085586}, {'type': 'v1', 'addr': '172.21.15.176:6809', 'nonce': 4116085586}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6804', 'nonce': 4116085586}, {'type': 'v1', 'addr': '172.21.15.176:6806', 'nonce': 4116085586}]}, 'public_addr': '172.21.15.176:6801/4116085586', 'cluster_addr': '172.21.15.176:6803/4116085586', 'heartbeat_back_addr': '172.21.15.176:6809/4116085586', 'heartbeat_front_addr': '172.21.15.176:6806/4116085586', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'a012c819-8d49-4310-a840-c2752d9314ab', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6824', 'nonce': 877760244}, {'type': 'v1', 'addr': '172.21.15.176:6825', 'nonce': 877760244}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6826', 'nonce': 877760244}, {'type': 'v1', 'addr': '172.21.15.176:6827', 'nonce': 877760244}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6830', 'nonce': 877760244}, {'type': 'v1', 'addr': '172.21.15.176:6831', 'nonce': 877760244}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6828', 'nonce': 877760244}, {'type': 'v1', 'addr': '172.21.15.176:6829', 'nonce': 877760244}]}, 'public_addr': '172.21.15.176:6825/877760244', 'cluster_addr': '172.21.15.176:6827/877760244', 'heartbeat_back_addr': '172.21.15.176:6831/877760244', 'heartbeat_front_addr': '172.21.15.176:6829/877760244', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:55.511688+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.742414+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.871355+0000', 'dead_epoch': 31}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.504553+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.589640+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.518845+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.731632+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.952634+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.2', 'mappings': [{'from': 3, 'to': 6}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T22:36:53.387 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:36:53.388 INFO:teuthology.orchestra.run.smithi001.stdout:1804 write 0x6890b1c thru 0x6890ea6 (0x38b bytes) 2024-09-14T22:36:53.388 INFO:teuthology.orchestra.run.smithi001.stdout:1805 write 0xeb9692b thru 0xeb970ce (0x7a4 bytes) 2024-09-14T22:36:53.388 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:53.388 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:53.388 INFO:teuthology.orchestra.run.smithi001.stdout:1809 write 0x71e8d43 thru 0x71eca7b (0x3d39 bytes) 2024-09-14T22:36:53.388 INFO:teuthology.orchestra.run.smithi001.stdout:1811 read 0xe51e4c5 thru 0xe52693b (0x8477 bytes) 2024-09-14T22:36:53.388 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:53.388 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:53.388 INFO:teuthology.orchestra.run.smithi001.stdout:1814 trunc from 0xeb970cf to 0xbb4ab5d 2024-09-14T22:36:53.389 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:53.389 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:53.390 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:36:53.394 INFO:teuthology.orchestra.run.smithi001.stdout:1816 read 0x1653ca8 thru 0x165fb8a (0xbee3 bytes) 2024-09-14T22:36:53.394 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:53.391+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:53.394 INFO:teuthology.orchestra.run.smithi001.stdout:1817 writesame 0x70a2817 thru 0x70a92a3 data_size 0x1(0x6a8d bytes) 2024-09-14T22:36:53.406 INFO:teuthology.orchestra.run.smithi001.stdout:1819 punch from 0x873c2d9 to 0x873e4b2, (0x21d9 bytes) 2024-09-14T22:36:53.406 INFO:teuthology.orchestra.run.smithi001.stdout:1821 punch from 0xba67741 to 0xba6d785, (0x6044 bytes) 2024-09-14T22:36:53.418 INFO:teuthology.orchestra.run.smithi001.stdout:1823 write 0xe19441a thru 0xe1990c0 (0x4ca7 bytes) 2024-09-14T22:36:53.421 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:53.422 INFO:teuthology.orchestra.run.smithi001.stdout:1825 read 0xc523adc thru 0xc52932f (0x5854 bytes) 2024-09-14T22:36:53.422 INFO:teuthology.orchestra.run.smithi001.stdout:1826 read 0x36056d8 thru 0x3606703 (0x102c bytes) 2024-09-14T22:36:53.422 INFO:teuthology.orchestra.run.smithi001.stdout:1827 writesame 0x2df2bb3 thru 0x2dfeccb data_size 0x1(0xc119 bytes) 2024-09-14T22:36:53.479 INFO:teuthology.orchestra.run.smithi001.stdout:1829 trunc from 0xe1990c1 to 0x30a6864 2024-09-14T22:36:53.492 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:53.487+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:53.515 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:53.516 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:53.529 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:36:53.536 INFO:teuthology.orchestra.run.smithi001.stdout:1830 write 0x4f80134 thru 0x4f8511e (0x4feb bytes) 2024-09-14T22:36:53.541 INFO:teuthology.orchestra.run.smithi001.stdout:1831 writesame 0x5d640e thru 0x5da395 data_size 0x4c(0x3f88 bytes) 2024-09-14T22:36:53.548 INFO:teuthology.orchestra.run.smithi001.stdout:1832 punch from 0x21897a2 to 0x2199089, (0xf8e7 bytes) 2024-09-14T22:36:53.567 INFO:teuthology.orchestra.run.smithi001.stdout:1833 write 0xd97f0db thru 0xd98769d (0x85c3 bytes) 2024-09-14T22:36:53.580 INFO:teuthology.orchestra.run.smithi001.stdout:1834 punch from 0x1c953b8 to 0x1c9c325, (0x6f6d bytes) 2024-09-14T22:36:53.580 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:53.580 INFO:teuthology.orchestra.run.smithi001.stdout:1837 writesame 0x362826a thru 0x3629041 data_size 0x4(0xdd8 bytes) 2024-09-14T22:36:53.593 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:53.591+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:53.608 INFO:teuthology.orchestra.run.smithi001.stdout:1838 trunc from 0xd98769e to 0x24d393e 2024-09-14T22:36:53.662 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:53.662 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:53.662 INFO:teuthology.orchestra.run.smithi001.stdout:1839 trunc from 0x24d393e to 0x987f290 2024-09-14T22:36:53.674 INFO:teuthology.orchestra.run.smithi001.stdout:1841 trunc from 0x987f290 to 0x9ae09ce 2024-09-14T22:36:53.676 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:36:53.686 INFO:teuthology.orchestra.run.smithi001.stdout:1842 punch from 0x66202d6 to 0x662e479, (0xe1a3 bytes) 2024-09-14T22:36:53.703 INFO:teuthology.orchestra.run.smithi001.stdout:1843 trunc from 0x9ae09ce to 0x17e076f 2024-09-14T22:36:53.723 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:53.719+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:53.756 INFO:teuthology.orchestra.run.smithi001.stdout:1844 punch from 0x124f634 to 0x125364c, (0x4018 bytes) 2024-09-14T22:36:53.759 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:36:53.759 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-14T22:36:53.785 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2024-09-14T22:36:52.872601+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":571527376,"num_objects":547,"num_object_clones":172,"num_object_copies":1094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":60,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":547,"num_whiteouts":215,"num_read":85390,"num_read_kb":309101,"num_write":7098,"num_write_kb":119844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5561,"ondisk_log_size":5561,"up":77,"acting":77,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":299980,"kb_used_data":17188,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754674740,"statfs":{"total":773094113280,"available":772786933760,"internally_reserved":0,"allocated":17600512,"data_stored":149584291,"data_compressed":1634276,"data_compressed_allocated":10854400,"data_compressed_original":146551170,"omap_allocated":12750,"internal_metadata":289394226},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4278443,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":121,"num_read_kb":915,"num_write":48,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.891364"},"pg_stats":[{"pgid":"3.1f","version":"46'127","reported_seq":2464,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:47.685646+0000","last_change":"2024-09-14T22:36:17.721881+0000","last_active":"2024-09-14T22:36:47.685646+0000","last_peered":"2024-09-14T22:36:47.685646+0000","last_clean":"2024-09-14T22:36:47.685646+0000","last_became_active":"2024-09-14T22:36:15.718313+0000","last_became_peered":"2024-09-14T22:36:15.718313+0000","last_unstale":"2024-09-14T22:36:47.685646+0000","last_undegraded":"2024-09-14T22:36:47.685646+0000","last_fullsized":"2024-09-14T22:36:47.685646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:26:30.647359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017190879999999999,"stat_sum":{"num_bytes":27728920,"num_objects":20,"num_object_clones":6,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":7,"num_read":2417,"num_read_kb":8252,"num_write":144,"num_write_kb":2795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"46'708","reported_seq":3400,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:47.809412+0000","last_change":"2024-09-14T22:36:17.733296+0000","last_active":"2024-09-14T22:36:47.809412+0000","last_peered":"2024-09-14T22:36:47.809412+0000","last_clean":"2024-09-14T22:36:47.809412+0000","last_became_active":"2024-09-14T22:36:15.709272+0000","last_became_peered":"2024-09-14T22:36:15.709272+0000","last_unstale":"2024-09-14T22:36:47.809412+0000","last_undegraded":"2024-09-14T22:36:47.809412+0000","last_fullsized":"2024-09-14T22:36:47.809412+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":708,"log_dups_size":0,"ondisk_log_size":708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:13:12.577725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001723619,"stat_sum":{"num_bytes":44638297,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":5,"num_read":6302,"num_read_kb":19813,"num_write":1273,"num_write_kb":7085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"43'158","reported_seq":5802,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:45.995827+0000","last_change":"2024-09-14T22:36:17.702313+0000","last_active":"2024-09-14T22:36:45.995827+0000","last_peered":"2024-09-14T22:36:45.995827+0000","last_clean":"2024-09-14T22:36:45.995827+0000","last_became_active":"2024-09-14T22:36:15.719488+0000","last_became_peered":"2024-09-14T22:36:15.719488+0000","last_unstale":"2024-09-14T22:36:45.995827+0000","last_undegraded":"2024-09-14T22:36:45.995827+0000","last_fullsized":"2024-09-14T22:36:45.995827+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:38:50.424482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053914200000000005,"stat_sum":{"num_bytes":32802358,"num_objects":27,"num_object_clones":11,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":12,"num_read":5617,"num_read_kb":35135,"num_write":148,"num_write_kb":4832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"46'137","reported_seq":904,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:47.734748+0000","last_change":"2024-09-14T22:36:17.702413+0000","last_active":"2024-09-14T22:36:47.734748+0000","last_peered":"2024-09-14T22:36:47.734748+0000","last_clean":"2024-09-14T22:36:47.734748+0000","last_became_active":"2024-09-14T22:36:15.719470+0000","last_became_peered":"2024-09-14T22:36:15.719470+0000","last_unstale":"2024-09-14T22:36:47.734748+0000","last_undegraded":"2024-09-14T22:36:47.734748+0000","last_fullsized":"2024-09-14T22:36:47.734748+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:05:40.586959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055079499999999997,"stat_sum":{"num_bytes":13523292,"num_objects":28,"num_object_clones":10,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":14,"num_read":762,"num_read_kb":5509,"num_write":126,"num_write_kb":5049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"46'209","reported_seq":4085,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:47.829845+0000","last_change":"2024-09-14T22:36:17.720920+0000","last_active":"2024-09-14T22:36:47.829845+0000","last_peered":"2024-09-14T22:36:47.829845+0000","last_clean":"2024-09-14T22:36:47.829845+0000","last_became_active":"2024-09-14T22:36:15.709684+0000","last_became_peered":"2024-09-14T22:36:15.709684+0000","last_unstale":"2024-09-14T22:36:47.829845+0000","last_undegraded":"2024-09-14T22:36:47.829845+0000","last_fullsized":"2024-09-14T22:36:47.829845+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:20:04.926087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092291499999999996,"stat_sum":{"num_bytes":25698130,"num_objects":21,"num_object_clones":7,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":8,"num_read":4351,"num_read_kb":26026,"num_write":264,"num_write_kb":7007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"46'75","reported_seq":812,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:47.799043+0000","last_change":"2024-09-14T22:36:17.711364+0000","last_active":"2024-09-14T22:36:47.799043+0000","last_peered":"2024-09-14T22:36:47.799043+0000","last_clean":"2024-09-14T22:36:47.799043+0000","last_became_active":"2024-09-14T22:36:15.701852+0000","last_became_peered":"2024-09-14T22:36:15.701852+0000","last_unstale":"2024-09-14T22:36:47.799043+0000","last_undegraded":"2024-09-14T22:36:47.799043+0000","last_fullsized":"2024-09-14T22:36:47.799043+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:47:34.173003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012502780000000001,"stat_sum":{"num_bytes":1679303,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":8,"num_read":689,"num_read_kb":2138,"num_write":69,"num_write_kb":1241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"49'154","reported_seq":2717,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T22:36:50.802465+0000","last_change":"2024-09-14T22:36:17.705423+0000","last_active":"2024-09-14T22:36:50.802465+0000","last_peered":"2024-09-14T22:36:50.802465+0000","last_clean":"2024-09-14T22:36:50.802465+0000","last_became_active":"2024-09-14T22:36:15.700990+0000","last_became_peered":"2024-09-14T22:36:15.700990+0000","last_unstale":"2024-09-14T22:36:50.802465+0000","last_undegraded":"2024-09-14T22:36:50.802465+0000","last_fullsized":"2024-09-14T22:36:50.802465+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:47:15.624780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010520620000000001,"stat_sum":{"num_bytes":5534535,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":5,"num_read":2905,"num_read_kb":2890,"num_write":149,"num_write_kb":618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"43'136","reported_seq":1957,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:46.046457+0000","last_change":"2024-09-14T22:36:17.733672+0000","last_active":"2024-09-14T22:36:46.046457+0000","last_peered":"2024-09-14T22:36:46.046457+0000","last_clean":"2024-09-14T22:36:46.046457+0000","last_became_active":"2024-09-14T22:36:15.701583+0000","last_became_peered":"2024-09-14T22:36:15.701583+0000","last_unstale":"2024-09-14T22:36:46.046457+0000","last_undegraded":"2024-09-14T22:36:46.046457+0000","last_fullsized":"2024-09-14T22:36:46.046457+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:51:55.495895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002063484,"stat_sum":{"num_bytes":12191790,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":3,"num_read":2027,"num_read_kb":14146,"num_write":209,"num_write_kb":3657,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"43'93","reported_seq":2518,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:45.976659+0000","last_change":"2024-09-14T22:36:17.721195+0000","last_active":"2024-09-14T22:36:45.976659+0000","last_peered":"2024-09-14T22:36:45.976659+0000","last_clean":"2024-09-14T22:36:45.976659+0000","last_became_active":"2024-09-14T22:36:15.715869+0000","last_became_peered":"2024-09-14T22:36:15.715869+0000","last_unstale":"2024-09-14T22:36:45.976659+0000","last_undegraded":"2024-09-14T22:36:45.976659+0000","last_fullsized":"2024-09-14T22:36:45.976659+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:57:57.786788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001160672,"stat_sum":{"num_bytes":10121105,"num_objects":15,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":6,"num_read":2382,"num_read_kb":8059,"num_write":126,"num_write_kb":7611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"46'104","reported_seq":2701,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:47.674338+0000","last_change":"2024-09-14T22:36:17.703048+0000","last_active":"2024-09-14T22:36:47.674338+0000","last_peered":"2024-09-14T22:36:47.674338+0000","last_clean":"2024-09-14T22:36:47.674338+0000","last_became_active":"2024-09-14T22:36:15.725078+0000","last_became_peered":"2024-09-14T22:36:15.725078+0000","last_unstale":"2024-09-14T22:36:47.674338+0000","last_undegraded":"2024-09-14T22:36:47.674338+0000","last_fullsized":"2024-09-14T22:36:47.674338+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:49:52.208600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010500290000000001,"stat_sum":{"num_bytes":20992000,"num_objects":20,"num_object_clones":8,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":10,"num_read":2573,"num_read_kb":17401,"num_write":100,"num_write_kb":6084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"43'133","reported_seq":2631,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:46.035395+0000","last_change":"2024-09-14T22:36:18.009371+0000","last_active":"2024-09-14T22:36:46.035395+0000","last_peered":"2024-09-14T22:36:46.035395+0000","last_clean":"2024-09-14T22:36:46.035395+0000","last_became_active":"2024-09-14T22:36:15.717341+0000","last_became_peered":"2024-09-14T22:36:15.717341+0000","last_unstale":"2024-09-14T22:36:46.035395+0000","last_undegraded":"2024-09-14T22:36:46.035395+0000","last_fullsized":"2024-09-14T22:36:46.035395+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:54:33.253465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054571999999999997,"stat_sum":{"num_bytes":3708965,"num_objects":11,"num_object_clones":4,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":5,"num_read":2649,"num_read_kb":3157,"num_write":151,"num_write_kb":498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"44'86","reported_seq":3098,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:47.352589+0000","last_change":"2024-09-14T22:36:17.711384+0000","last_active":"2024-09-14T22:36:47.352589+0000","last_peered":"2024-09-14T22:36:47.352589+0000","last_clean":"2024-09-14T22:36:47.352589+0000","last_became_active":"2024-09-14T22:36:15.723075+0000","last_became_peered":"2024-09-14T22:36:15.723075+0000","last_unstale":"2024-09-14T22:36:47.352589+0000","last_undegraded":"2024-09-14T22:36:47.352589+0000","last_fullsized":"2024-09-14T22:36:47.352589+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:24:40.055092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010804009999999999,"stat_sum":{"num_bytes":15572708,"num_objects":20,"num_object_clones":7,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":8,"num_read":2981,"num_read_kb":7105,"num_write":77,"num_write_kb":4561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":67,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:45.968182+0000","last_change":"2024-09-14T22:36:08.042860+0000","last_active":"2024-09-14T22:36:45.968182+0000","last_peered":"2024-09-14T22:36:45.968182+0000","last_clean":"2024-09-14T22:36:45.968182+0000","last_became_active":"2024-09-14T22:36:06.040961+0000","last_became_peered":"2024-09-14T22:36:06.040961+0000","last_unstale":"2024-09-14T22:36:45.968182+0000","last_undegraded":"2024-09-14T22:36:45.968182+0000","last_fullsized":"2024-09-14T22:36:45.968182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:03:48.547834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010780950000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"46'101","reported_seq":570,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:47.286606+0000","last_change":"2024-09-14T22:36:17.720897+0000","last_active":"2024-09-14T22:36:47.286606+0000","last_peered":"2024-09-14T22:36:47.286606+0000","last_clean":"2024-09-14T22:36:47.286606+0000","last_became_active":"2024-09-14T22:36:15.700982+0000","last_became_peered":"2024-09-14T22:36:15.700982+0000","last_unstale":"2024-09-14T22:36:47.286606+0000","last_undegraded":"2024-09-14T22:36:47.286606+0000","last_fullsized":"2024-09-14T22:36:47.286606+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:14:14.704431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010495459999999999,"stat_sum":{"num_bytes":10101278,"num_objects":17,"num_object_clones":6,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":6,"num_read":742,"num_read_kb":1923,"num_write":96,"num_write_kb":2962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:45.968383+0000","last_change":"2024-09-14T22:36:41.930087+0000","last_active":"2024-09-14T22:36:45.968383+0000","last_peered":"2024-09-14T22:36:45.968383+0000","last_clean":"2024-09-14T22:36:45.968383+0000","last_became_active":"2024-09-14T22:36:41.929522+0000","last_became_peered":"2024-09-14T22:36:41.929522+0000","last_unstale":"2024-09-14T22:36:45.968383+0000","last_undegraded":"2024-09-14T22:36:45.968383+0000","last_fullsized":"2024-09-14T22:36:45.968383+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:15:10.589042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"42'186","reported_seq":3033,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:46.035596+0000","last_change":"2024-09-14T22:36:21.722517+0000","last_active":"2024-09-14T22:36:46.035596+0000","last_peered":"2024-09-14T22:36:46.035596+0000","last_clean":"2024-09-14T22:36:46.035596+0000","last_became_active":"2024-09-14T22:36:21.722085+0000","last_became_peered":"2024-09-14T22:36:21.722085+0000","last_unstale":"2024-09-14T22:36:46.035596+0000","last_undegraded":"2024-09-14T22:36:46.035596+0000","last_fullsized":"2024-09-14T22:36:46.035596+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:16:57.001622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020774109999999999,"stat_sum":{"num_bytes":41242153,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":3,"num_read":3596,"num_read_kb":17413,"num_write":251,"num_write_kb":5625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":67,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:46.035475+0000","last_change":"2024-09-14T22:36:08.054606+0000","last_active":"2024-09-14T22:36:46.035475+0000","last_peered":"2024-09-14T22:36:46.035475+0000","last_clean":"2024-09-14T22:36:46.035475+0000","last_became_active":"2024-09-14T22:36:06.266908+0000","last_became_peered":"2024-09-14T22:36:06.266908+0000","last_unstale":"2024-09-14T22:36:46.035475+0000","last_undegraded":"2024-09-14T22:36:46.035475+0000","last_fullsized":"2024-09-14T22:36:46.035475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:36.738643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098249300000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"41'119","reported_seq":2430,"reported_epoch":50,"state":"active+undersized+degraded","last_fresh":"2024-09-14T22:36:50.802624+0000","last_change":"2024-09-14T22:36:27.775362+0000","last_active":"2024-09-14T22:36:50.802624+0000","last_peered":"2024-09-14T22:36:50.802624+0000","last_clean":"2024-09-14T22:36:25.819901+0000","last_became_active":"2024-09-14T22:36:27.775362+0000","last_became_peered":"2024-09-14T22:36:27.775362+0000","last_unstale":"2024-09-14T22:36:50.802624+0000","last_undegraded":"2024-09-14T22:36:27.774765+0000","last_fullsized":"2024-09-14T22:36:27.774675+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:27:07.768315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012564830000000001,"stat_sum":{"num_bytes":13031461,"num_objects":20,"num_object_clones":8,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":9,"num_read":2017,"num_read_kb":6272,"num_write":150,"num_write_kb":5633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":20}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:45.976109+0000","last_change":"2024-09-14T22:36:41.930961+0000","last_active":"2024-09-14T22:36:45.976109+0000","last_peered":"2024-09-14T22:36:45.976109+0000","last_clean":"2024-09-14T22:36:45.976109+0000","last_became_active":"2024-09-14T22:36:41.930619+0000","last_became_peered":"2024-09-14T22:36:41.930619+0000","last_unstale":"2024-09-14T22:36:45.976109+0000","last_undegraded":"2024-09-14T22:36:45.976109+0000","last_fullsized":"2024-09-14T22:36:45.976109+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:04:40.582999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"46'133","reported_seq":775,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:47.678101+0000","last_change":"2024-09-14T22:36:17.702597+0000","last_active":"2024-09-14T22:36:47.678101+0000","last_peered":"2024-09-14T22:36:47.678101+0000","last_clean":"2024-09-14T22:36:47.678101+0000","last_became_active":"2024-09-14T22:36:15.717186+0000","last_became_peered":"2024-09-14T22:36:15.717186+0000","last_unstale":"2024-09-14T22:36:47.678101+0000","last_undegraded":"2024-09-14T22:36:47.678101+0000","last_fullsized":"2024-09-14T22:36:47.678101+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:52:22.611789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071220099999999998,"stat_sum":{"num_bytes":9263385,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":5,"num_read":1159,"num_read_kb":2649,"num_write":116,"num_write_kb":1051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"46'81","reported_seq":635,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:47.674193+0000","last_change":"2024-09-14T22:36:17.711068+0000","last_active":"2024-09-14T22:36:47.674193+0000","last_peered":"2024-09-14T22:36:47.674193+0000","last_clean":"2024-09-14T22:36:47.674193+0000","last_became_active":"2024-09-14T22:36:15.706147+0000","last_became_peered":"2024-09-14T22:36:15.706147+0000","last_unstale":"2024-09-14T22:36:47.674193+0000","last_undegraded":"2024-09-14T22:36:47.674193+0000","last_fullsized":"2024-09-14T22:36:47.674193+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:12:05.941543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082625500000000002,"stat_sum":{"num_bytes":5414489,"num_objects":9,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":4,"num_read":498,"num_read_kb":77,"num_write":78,"num_write_kb":352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:45.995393+0000","last_change":"2024-09-14T22:36:08.055938+0000","last_active":"2024-09-14T22:36:45.995393+0000","last_peered":"2024-09-14T22:36:45.995393+0000","last_clean":"2024-09-14T22:36:45.995393+0000","last_became_active":"2024-09-14T22:36:06.039600+0000","last_became_peered":"2024-09-14T22:36:06.039600+0000","last_unstale":"2024-09-14T22:36:45.995393+0000","last_undegraded":"2024-09-14T22:36:45.995393+0000","last_fullsized":"2024-09-14T22:36:45.995393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:49:09.245563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021617870000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"49'231","reported_seq":6345,"reported_epoch":50,"state":"active+undersized+degraded","last_fresh":"2024-09-14T22:36:50.802455+0000","last_change":"2024-09-14T22:36:27.775190+0000","last_active":"2024-09-14T22:36:50.802455+0000","last_peered":"2024-09-14T22:36:50.802455+0000","last_clean":"2024-09-14T22:36:25.848645+0000","last_became_active":"2024-09-14T22:36:27.775190+0000","last_became_peered":"2024-09-14T22:36:27.775190+0000","last_unstale":"2024-09-14T22:36:50.802455+0000","last_undegraded":"2024-09-14T22:36:27.774713+0000","last_fullsized":"2024-09-14T22:36:27.774588+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:28:31.412755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021500769999999998,"stat_sum":{"num_bytes":42322391,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":5,"num_read":6404,"num_read_kb":11387,"num_write":257,"num_write_kb":4755,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":22}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":36,"reported_epoch":46,"state":"active+undersized","last_fresh":"2024-09-14T22:36:46.045817+0000","last_change":"2024-09-14T22:36:27.773450+0000","last_active":"2024-09-14T22:36:46.045817+0000","last_peered":"2024-09-14T22:36:46.045817+0000","last_clean":"2024-09-14T22:36:05.010064+0000","last_became_active":"2024-09-14T22:36:27.773450+0000","last_became_peered":"2024-09-14T22:36:27.773450+0000","last_unstale":"2024-09-14T22:36:46.045817+0000","last_undegraded":"2024-09-14T22:36:46.045817+0000","last_fullsized":"2024-09-14T22:36:27.772739+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:05:20.367237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"46'130","reported_seq":588,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:47.311494+0000","last_change":"2024-09-14T22:36:17.720438+0000","last_active":"2024-09-14T22:36:47.311494+0000","last_peered":"2024-09-14T22:36:47.311494+0000","last_clean":"2024-09-14T22:36:47.311494+0000","last_became_active":"2024-09-14T22:36:15.711105+0000","last_became_peered":"2024-09-14T22:36:15.711105+0000","last_unstale":"2024-09-14T22:36:47.311494+0000","last_undegraded":"2024-09-14T22:36:47.311494+0000","last_fullsized":"2024-09-14T22:36:47.311494+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:37:31.942594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085549400000000005,"stat_sum":{"num_bytes":9716431,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":3,"num_read":564,"num_read_kb":774,"num_write":144,"num_write_kb":368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":67,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:46.035262+0000","last_change":"2024-09-14T22:36:08.054759+0000","last_active":"2024-09-14T22:36:46.035262+0000","last_peered":"2024-09-14T22:36:46.035262+0000","last_clean":"2024-09-14T22:36:46.035262+0000","last_became_active":"2024-09-14T22:36:06.039969+0000","last_became_peered":"2024-09-14T22:36:06.039969+0000","last_unstale":"2024-09-14T22:36:46.035262+0000","last_undegraded":"2024-09-14T22:36:46.035262+0000","last_fullsized":"2024-09-14T22:36:46.035262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:03:45.369532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010817520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"46'766","reported_seq":1866,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:47.762197+0000","last_change":"2024-09-14T22:36:21.724315+0000","last_active":"2024-09-14T22:36:47.762197+0000","last_peered":"2024-09-14T22:36:47.762197+0000","last_clean":"2024-09-14T22:36:47.762197+0000","last_became_active":"2024-09-14T22:36:21.723138+0000","last_became_peered":"2024-09-14T22:36:21.723138+0000","last_unstale":"2024-09-14T22:36:47.762197+0000","last_undegraded":"2024-09-14T22:36:47.762197+0000","last_fullsized":"2024-09-14T22:36:47.762197+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":766,"log_dups_size":0,"ondisk_log_size":766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:24:51.791803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13407561,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":5,"num_read":5426,"num_read_kb":5504,"num_write":1224,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":70,"reported_epoch":46,"state":"active+undersized+degraded","last_fresh":"2024-09-14T22:36:45.995373+0000","last_change":"2024-09-14T22:36:27.781594+0000","last_active":"2024-09-14T22:36:45.995373+0000","last_peered":"2024-09-14T22:36:45.995373+0000","last_clean":"2024-09-14T22:36:23.739926+0000","last_became_active":"2024-09-14T22:36:27.781594+0000","last_became_peered":"2024-09-14T22:36:27.781594+0000","last_unstale":"2024-09-14T22:36:45.995373+0000","last_undegraded":"2024-09-14T22:36:27.780369+0000","last_fullsized":"2024-09-14T22:36:27.780115+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:49:52.683148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011401880000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":1}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":218,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:46.035255+0000","last_change":"2024-09-14T22:36:00.402862+0000","last_active":"2024-09-14T22:36:46.035255+0000","last_peered":"2024-09-14T22:36:46.035255+0000","last_clean":"2024-09-14T22:36:46.035255+0000","last_became_active":"2024-09-14T22:36:00.400529+0000","last_became_peered":"2024-09-14T22:36:00.400529+0000","last_unstale":"2024-09-14T22:36:46.035255+0000","last_undegraded":"2024-09-14T22:36:46.035255+0000","last_fullsized":"2024-09-14T22:36:46.035255+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_clean_scrub_stamp":"2024-09-14T22:35:58.989025+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:07:12.951728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"46'98","reported_seq":2336,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:47.175517+0000","last_change":"2024-09-14T22:36:17.732784+0000","last_active":"2024-09-14T22:36:47.175517+0000","last_peered":"2024-09-14T22:36:47.175517+0000","last_clean":"2024-09-14T22:36:47.175517+0000","last_became_active":"2024-09-14T22:36:15.704885+0000","last_became_peered":"2024-09-14T22:36:15.704885+0000","last_unstale":"2024-09-14T22:36:47.175517+0000","last_undegraded":"2024-09-14T22:36:47.175517+0000","last_fullsized":"2024-09-14T22:36:47.175517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:16:39.754371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001381511,"stat_sum":{"num_bytes":11814259,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":9,"num_read":2198,"num_read_kb":3693,"num_write":132,"num_write_kb":3444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"46'136","reported_seq":4631,"reported_epoch":46,"state":"active+undersized+degraded","last_fresh":"2024-09-14T22:36:47.641586+0000","last_change":"2024-09-14T22:36:27.777838+0000","last_active":"2024-09-14T22:36:47.641586+0000","last_peered":"2024-09-14T22:36:47.641586+0000","last_clean":"2024-09-14T22:36:25.764517+0000","last_became_active":"2024-09-14T22:36:27.777838+0000","last_became_peered":"2024-09-14T22:36:27.777838+0000","last_unstale":"2024-09-14T22:36:47.641586+0000","last_undegraded":"2024-09-14T22:36:27.776696+0000","last_fullsized":"2024-09-14T22:36:27.776472+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:43:54.295544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060272699999999995,"stat_sum":{"num_bytes":27409441,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":4980,"num_read_kb":7126,"num_write":131,"num_write_kb":2545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":17}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"46'217","reported_seq":1088,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:47.755883+0000","last_change":"2024-09-14T22:36:17.776247+0000","last_active":"2024-09-14T22:36:47.755883+0000","last_peered":"2024-09-14T22:36:47.755883+0000","last_clean":"2024-09-14T22:36:47.755883+0000","last_became_active":"2024-09-14T22:36:15.718794+0000","last_became_peered":"2024-09-14T22:36:15.718794+0000","last_unstale":"2024-09-14T22:36:47.755883+0000","last_undegraded":"2024-09-14T22:36:47.755883+0000","last_fullsized":"2024-09-14T22:36:47.755883+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:16:38.308523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088662699999999997,"stat_sum":{"num_bytes":11717466,"num_objects":14,"num_object_clones":4,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":5,"num_read":1665,"num_read_kb":4129,"num_write":208,"num_write_kb":5095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"46'106","reported_seq":685,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:47.691174+0000","last_change":"2024-09-14T22:36:17.733002+0000","last_active":"2024-09-14T22:36:47.691174+0000","last_peered":"2024-09-14T22:36:47.691174+0000","last_clean":"2024-09-14T22:36:47.691174+0000","last_became_active":"2024-09-14T22:36:15.702885+0000","last_became_peered":"2024-09-14T22:36:15.702885+0000","last_unstale":"2024-09-14T22:36:47.691174+0000","last_undegraded":"2024-09-14T22:36:47.691174+0000","last_fullsized":"2024-09-14T22:36:47.691174+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:09:26.069881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015180580000000001,"stat_sum":{"num_bytes":13200052,"num_objects":16,"num_object_clones":5,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":7,"num_read":871,"num_read_kb":3970,"num_write":94,"num_write_kb":2791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"46'153","reported_seq":2736,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:47.732946+0000","last_change":"2024-09-14T22:36:17.702738+0000","last_active":"2024-09-14T22:36:47.732946+0000","last_peered":"2024-09-14T22:36:47.732946+0000","last_clean":"2024-09-14T22:36:47.732946+0000","last_became_active":"2024-09-14T22:36:15.713452+0000","last_became_peered":"2024-09-14T22:36:15.713452+0000","last_unstale":"2024-09-14T22:36:47.732946+0000","last_undegraded":"2024-09-14T22:36:47.732946+0000","last_fullsized":"2024-09-14T22:36:47.732946+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:52:49.401049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082782099999999996,"stat_sum":{"num_bytes":10258432,"num_objects":25,"num_object_clones":10,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":11,"num_read":2742,"num_read_kb":7330,"num_write":169,"num_write_kb":3067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"46'87","reported_seq":507,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:47.486937+0000","last_change":"2024-09-14T22:36:18.009776+0000","last_active":"2024-09-14T22:36:47.486937+0000","last_peered":"2024-09-14T22:36:47.486937+0000","last_clean":"2024-09-14T22:36:47.486937+0000","last_became_active":"2024-09-14T22:36:15.723205+0000","last_became_peered":"2024-09-14T22:36:15.723205+0000","last_unstale":"2024-09-14T22:36:47.486937+0000","last_undegraded":"2024-09-14T22:36:47.486937+0000","last_fullsized":"2024-09-14T22:36:47.486937+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:31.954413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087783999999999996,"stat_sum":{"num_bytes":1704925,"num_objects":17,"num_object_clones":6,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":10,"num_read":367,"num_read_kb":1647,"num_write":76,"num_write_kb":885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"46'192","reported_seq":503,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:47.483524+0000","last_change":"2024-09-14T22:36:18.009666+0000","last_active":"2024-09-14T22:36:47.483524+0000","last_peered":"2024-09-14T22:36:47.483524+0000","last_clean":"2024-09-14T22:36:47.483524+0000","last_became_active":"2024-09-14T22:36:15.715659+0000","last_became_peered":"2024-09-14T22:36:15.715659+0000","last_unstale":"2024-09-14T22:36:47.483524+0000","last_undegraded":"2024-09-14T22:36:47.483524+0000","last_fullsized":"2024-09-14T22:36:47.483524+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:32:27.180584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000776448,"stat_sum":{"num_bytes":2461780,"num_objects":15,"num_object_clones":5,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":8,"num_read":812,"num_read_kb":1610,"num_write":261,"num_write_kb":468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"46'106","reported_seq":4899,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:47.243615+0000","last_change":"2024-09-14T22:36:18.010182+0000","last_active":"2024-09-14T22:36:47.243615+0000","last_peered":"2024-09-14T22:36:47.243615+0000","last_clean":"2024-09-14T22:36:47.243615+0000","last_became_active":"2024-09-14T22:36:15.719693+0000","last_became_peered":"2024-09-14T22:36:15.719693+0000","last_unstale":"2024-09-14T22:36:47.243615+0000","last_undegraded":"2024-09-14T22:36:47.243615+0000","last_fullsized":"2024-09-14T22:36:47.243615+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:17:50.916340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001045011,"stat_sum":{"num_bytes":12026994,"num_objects":13,"num_object_clones":4,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":4,"num_read":4854,"num_read_kb":8125,"num_write":111,"num_write_kb":1352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"44'123","reported_seq":2597,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:47.321498+0000","last_change":"2024-09-14T22:36:17.776422+0000","last_active":"2024-09-14T22:36:47.321498+0000","last_peered":"2024-09-14T22:36:47.321498+0000","last_clean":"2024-09-14T22:36:47.321498+0000","last_became_active":"2024-09-14T22:36:15.707650+0000","last_became_peered":"2024-09-14T22:36:15.707650+0000","last_unstale":"2024-09-14T22:36:47.321498+0000","last_undegraded":"2024-09-14T22:36:47.321498+0000","last_fullsized":"2024-09-14T22:36:47.321498+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:27:56.534384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094227299999999998,"stat_sum":{"num_bytes":25894889,"num_objects":21,"num_object_clones":7,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":8,"num_read":2450,"num_read_kb":20760,"num_write":159,"num_write_kb":8467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"46'89","reported_seq":637,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:47.817994+0000","last_change":"2024-09-14T22:36:18.009347+0000","last_active":"2024-09-14T22:36:47.817994+0000","last_peered":"2024-09-14T22:36:47.817994+0000","last_clean":"2024-09-14T22:36:47.817994+0000","last_became_active":"2024-09-14T22:36:15.715569+0000","last_became_peered":"2024-09-14T22:36:15.715569+0000","last_unstale":"2024-09-14T22:36:47.817994+0000","last_undegraded":"2024-09-14T22:36:47.817994+0000","last_fullsized":"2024-09-14T22:36:47.817994+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:56:43.479659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054921600000000005,"stat_sum":{"num_bytes":19527400,"num_objects":13,"num_object_clones":4,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":5,"num_read":493,"num_read_kb":5913,"num_write":124,"num_write_kb":7790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"44'89","reported_seq":3762,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:46.286757+0000","last_change":"2024-09-14T22:36:17.720448+0000","last_active":"2024-09-14T22:36:46.286757+0000","last_peered":"2024-09-14T22:36:46.286757+0000","last_clean":"2024-09-14T22:36:46.286757+0000","last_became_active":"2024-09-14T22:36:15.716696+0000","last_became_peered":"2024-09-14T22:36:15.716696+0000","last_unstale":"2024-09-14T22:36:46.286757+0000","last_undegraded":"2024-09-14T22:36:46.286757+0000","last_fullsized":"2024-09-14T22:36:46.286757+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:57:33.599338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086547600000000001,"stat_sum":{"num_bytes":27870666,"num_objects":15,"num_object_clones":5,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":5,"num_read":3643,"num_read_kb":28227,"num_write":83,"num_write_kb":1111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"46'178","reported_seq":4178,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T22:36:47.888462+0000","last_change":"2024-09-14T22:36:18.010088+0000","last_active":"2024-09-14T22:36:47.888462+0000","last_peered":"2024-09-14T22:36:47.888462+0000","last_clean":"2024-09-14T22:36:47.888462+0000","last_became_active":"2024-09-14T22:36:15.715866+0000","last_became_peered":"2024-09-14T22:36:15.715866+0000","last_unstale":"2024-09-14T22:36:47.888462+0000","last_undegraded":"2024-09-14T22:36:47.888462+0000","last_fullsized":"2024-09-14T22:36:47.888462+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:34:47.039304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010286189999999999,"stat_sum":{"num_bytes":48360133,"num_objects":21,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":7,"num_read":4138,"num_read_kb":20841,"num_write":237,"num_write_kb":7025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":570936989,"num_objects":543,"num_object_clones":172,"num_object_copies":1086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":59,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":543,"num_whiteouts":215,"num_read":85274,"num_read_kb":309001,"num_write":6937,"num_write_kb":117426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":13811712,"data_stored":147347688,"data_compressed":1761821,"data_compressed_allocated":10973184,"data_compressed_original":145969538,"omap_allocated":0,"internal_metadata":0},"log_size":5441,"ondisk_log_size":5441,"up":61,"acting":61,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":14,"acting":14,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542157,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37812,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334028,"statfs":{"total":96636764160,"available":96598044672,"internally_reserved":0,"allocated":2523136,"data_stored":23241780,"data_compressed":146670,"data_compressed_allocated":1585152,"data_compressed_original":22820055,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542158,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32680,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339160,"statfs":{"total":96636764160,"available":96603299840,"internally_reserved":0,"allocated":1523712,"data_stored":12872985,"data_compressed":106348,"data_compressed_allocated":888832,"data_compressed_original":12592195,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542158,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38196,"kb_used_data":2844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333644,"statfs":{"total":96636764160,"available":96597651456,"internally_reserved":0,"allocated":2912256,"data_stored":25604446,"data_compressed":324115,"data_compressed_allocated":1908736,"data_compressed_original":25137756,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542157,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38896,"kb_used_data":3552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332944,"statfs":{"total":96636764160,"available":96596934656,"internally_reserved":0,"allocated":3637248,"data_stored":37615096,"data_compressed":236261,"data_compressed_allocated":2580480,"data_compressed_original":37135970,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542158,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45620,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326220,"statfs":{"total":96636764160,"available":96590049280,"internally_reserved":0,"allocated":1994752,"data_stored":12642701,"data_compressed":256444,"data_compressed_allocated":1044480,"data_compressed_original":12216323,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31496,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340344,"statfs":{"total":96636764160,"available":96604512256,"internally_reserved":0,"allocated":315392,"data_stored":122731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37056,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334784,"statfs":{"total":96636764160,"available":96598818816,"internally_reserved":0,"allocated":1744896,"data_stored":10692295,"data_compressed":212761,"data_compressed_allocated":856064,"data_compressed_original":10295204,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542157,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38224,"kb_used_data":2880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333616,"statfs":{"total":96636764160,"available":96597622784,"internally_reserved":0,"allocated":2949120,"data_stored":26792257,"data_compressed":351677,"data_compressed_allocated":1990656,"data_compressed_original":26353667,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2428928,"data_stored":26076077,"data_compressed":368952,"data_compressed_allocated":1982464,"data_compressed_original":25858051,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1171456,"data_stored":10446637,"data_compressed":212761,"data_compressed_allocated":856064,"data_compressed_original":10295204,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":29424,"data_compressed":10286,"data_compressed_allocated":12288,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1585152,"data_stored":12586761,"data_compressed":310270,"data_compressed_allocated":1114112,"data_compressed_original":12355587,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3248128,"data_stored":37655762,"data_compressed":284972,"data_compressed_allocated":2650112,"data_compressed_original":37365346,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2314240,"data_stored":25298534,"data_compressed":280642,"data_compressed_allocated":1859584,"data_compressed_original":25068124,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1077248,"data_stored":12825826,"data_compressed":153077,"data_compressed_allocated":950272,"data_compressed_original":12764227,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1945600,"data_stored":22428667,"data_compressed":140861,"data_compressed_allocated":1548288,"data_compressed_original":22246615,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:36:53.787 INFO:tasks.thrashosds.thrasher:Setting 3.9 to [7, 4, 1, 2] 2024-09-14T22:36:53.787 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.9', '7', '4', '1', '2'] 2024-09-14T22:36:53.787 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.9 7 4 1 2 2024-09-14T22:36:53.789 INFO:teuthology.orchestra.run.smithi001.stdout:1845 write 0xde3008f thru 0xde31bdf (0x1b51 bytes) 2024-09-14T22:36:53.800 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:53.801 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:53.802 INFO:teuthology.orchestra.run.smithi001.stdout:1846 trunc from 0xde31be0 to 0x9f2ca07 2024-09-14T22:36:53.802 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:53.799+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:53.815 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:36:53.817 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:53.832 INFO:teuthology.orchestra.run.smithi001.stdout:1848 writesame 0xe10753e thru 0xe1103ec data_size 0x1(0x8eaf bytes) 2024-09-14T22:36:53.833 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:53.833 INFO:teuthology.orchestra.run.smithi001.stdout:1850 read 0x75a9ab9 thru 0x75b02f7 (0x683f bytes) 2024-09-14T22:36:53.833 INFO:teuthology.orchestra.run.smithi001.stdout:1852 write 0x74ed7b1 thru 0x74f5a52 (0x82a2 bytes) 2024-09-14T22:36:53.838 INFO:teuthology.orchestra.run.smithi001.stdout:truncating to largest ever: 0xedb6f81 2024-09-14T22:36:53.839 INFO:teuthology.orchestra.run.smithi001.stdout:1853 trunc from 0xe1103ed to 0xedb6f81 2024-09-14T22:36:53.847 INFO:teuthology.orchestra.run.smithi001.stdout:1855 read 0x3c4008e thru 0x3c4d2e2 (0xd255 bytes) 2024-09-14T22:36:53.847 INFO:teuthology.orchestra.run.smithi001.stdout:1856 punch from 0x845da50 to 0x8462bfa, (0x51aa bytes) 2024-09-14T22:36:53.847 INFO:teuthology.orchestra.run.smithi001.stdout:1857 read 0xa6a2bfb thru 0xa6abba3 (0x8fa9 bytes) 2024-09-14T22:36:53.847 INFO:teuthology.orchestra.run.smithi001.stdout:1858 write 0x60fc010 thru 0x61067b3 (0xa7a4 bytes) 2024-09-14T22:36:53.855 INFO:teuthology.orchestra.run.smithi001.stdout:1859 write 0xa839b73 thru 0xa849579 (0xfa07 bytes) 2024-09-14T22:36:53.861 INFO:teuthology.orchestra.run.smithi001.stdout:1860 read 0x1ad05ff thru 0x1adb3ea (0xadec bytes) 2024-09-14T22:36:53.861 INFO:teuthology.orchestra.run.smithi001.stdout:1862 read 0xe2ee94 thru 0xe32444 (0x35b1 bytes) 2024-09-14T22:36:53.862 INFO:teuthology.orchestra.run.smithi001.stdout:1863 read 0xc941c86 thru 0xc946711 (0x4a8c bytes) 2024-09-14T22:36:53.862 INFO:teuthology.orchestra.run.smithi001.stdout:1864 punch from 0x176757b to 0x1772fd2, (0xba57 bytes) 2024-09-14T22:36:53.862 INFO:teuthology.orchestra.run.smithi001.stdout:1865 writesame 0x3b35bbc thru 0x3b39fab data_size 0x8(0x43f0 bytes) 2024-09-14T22:36:53.868 INFO:teuthology.orchestra.run.smithi001.stdout:1866 write 0x1d4a755 thru 0x1d4b889 (0x1135 bytes) 2024-09-14T22:36:53.871 INFO:teuthology.orchestra.run.smithi001.stdout:1867 punch from 0xf3744e to 0xf3feaf, (0x8a61 bytes) 2024-09-14T22:36:53.871 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:53.871 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:53.871 INFO:teuthology.orchestra.run.smithi001.stdout:1870 writesame 0x4ebaa0d thru 0x4ec41c6 data_size 0x1(0x97ba bytes) 2024-09-14T22:36:53.903 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:53.899+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:53.908 INFO:teuthology.orchestra.run.smithi001.stdout:1871 read 0x66fb5d4 thru 0x6705677 (0xa0a4 bytes) 2024-09-14T22:36:53.908 INFO:teuthology.orchestra.run.smithi001.stdout:1872 read 0x9e4a314 thru 0x9e4d25e (0x2f4b bytes) 2024-09-14T22:36:53.908 INFO:teuthology.orchestra.run.smithi001.stdout:1873 write 0xda91929 thru 0xda9a84b (0x8f23 bytes) 2024-09-14T22:36:53.911 INFO:teuthology.orchestra.run.smithi001.stdout:1874 read 0x834593 thru 0x841f88 (0xd9f6 bytes) 2024-09-14T22:36:53.913 INFO:teuthology.orchestra.run.smithi001.stdout:1875 writesame 0x32d9279 thru 0x32e3272 data_size 0x1(0x9ffa bytes) 2024-09-14T22:36:53.939 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:53.939 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:53.939 INFO:teuthology.orchestra.run.smithi001.stdout:1878 read 0xd132ff2 thru 0xd13c476 (0x9485 bytes) 2024-09-14T22:36:53.939 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:53.939 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:53.939 INFO:teuthology.orchestra.run.smithi001.stdout:1882 read 0xc21eaa9 thru 0xc22720c (0x8764 bytes) 2024-09-14T22:36:53.939 INFO:teuthology.orchestra.run.smithi001.stdout:1883 read 0x8167431 thru 0x816c493 (0x5063 bytes) 2024-09-14T22:36:53.939 INFO:teuthology.orchestra.run.smithi001.stdout:1884 writesame 0x54b8bbc thru 0x54bc9fb data_size 0x30(0x3e40 bytes) 2024-09-14T22:36:53.943 INFO:teuthology.orchestra.run.smithi001.stdout:1887 write 0xa24165b thru 0xa24512a (0x3ad0 bytes) 2024-09-14T22:36:53.947 INFO:teuthology.orchestra.run.smithi001.stdout:1891 read 0x7440cdc thru 0x74469c4 (0x5ce9 bytes) 2024-09-14T22:36:53.947 INFO:teuthology.orchestra.run.smithi001.stdout:1892 write 0xec49561 thru 0xec4f942 (0x63e2 bytes) 2024-09-14T22:36:53.951 INFO:teuthology.orchestra.run.smithi001.stdout:1893 read 0xd44925d thru 0xd458070 (0xee14 bytes) 2024-09-14T22:36:53.951 INFO:teuthology.orchestra.run.smithi001.stdout:1894 writesame 0xaec9cff thru 0xaecd01b data_size 0x1(0x331d bytes) 2024-09-14T22:36:53.961 INFO:teuthology.orchestra.run.smithi001.stdout:1895 read 0x77b2a53 thru 0x77b38d2 (0xe80 bytes) 2024-09-14T22:36:53.961 INFO:teuthology.orchestra.run.smithi001.stdout:1896 read 0xe9e6c9 thru 0xea234a (0x3c82 bytes) 2024-09-14T22:36:53.961 INFO:teuthology.orchestra.run.smithi001.stdout:1897 write 0xcff921f thru 0xcffe100 (0x4ee2 bytes) 2024-09-14T22:36:53.967 INFO:teuthology.orchestra.run.smithi001.stdout:1899 write 0xa720496 thru 0xa72b13e (0xaca9 bytes) 2024-09-14T22:36:53.983 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:53.983 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:53.996 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:53.991+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:53.996 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:36:53.997 INFO:teuthology.orchestra.run.smithi001.stdout:1900 trunc from 0xedb6f81 to 0x11cf167 2024-09-14T22:36:54.009 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2024-09-14T22:36:54.036 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:54.036 INFO:teuthology.orchestra.run.smithi001.stdout:1905 read 0xf54be thru 0xfd6ce (0x8211 bytes) 2024-09-14T22:36:54.036 INFO:teuthology.orchestra.run.smithi001.stdout:1907 clone 13 order 21 su 1048576 sc 12 2024-09-14T22:36:54.096 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:54.091+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:54.123 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:54.123 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:54.137 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:36:54.198 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:54.191+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:54.299 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:54.295+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:54.308 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:54.308 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:54.323 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:36:54.399 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:54.395+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:54.432 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:36:54.447 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:54.447 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:54.462 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:36:54.500 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:54.495+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:54.597 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:36:54.597 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:36:54.601 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:54.599+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:54.615 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:36:54.615 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:36:54.616 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-14T22:36:54.634 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:36:54.634 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:36:54.643 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:54.644 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:36:54.644 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:36:54.644 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:54.655 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:36:54.655 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:36:54.658 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:36:54.677 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:36:54.677 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:36:54.701 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:36:54.702 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:36:54.702 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:54.699+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:54.724 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:36:54.724 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:36:54.728 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:36:54.729 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:36:54.788 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:54.788 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:36:54.788 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:36:54.788 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:36:54.788 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:54.802 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:36:54.803 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:54.799+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:54.903 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:54.899+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:54.928 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:54.928 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:36:54.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:36:54.928 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:36:54.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.928 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4149 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13129728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:54.928 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703281+0000", 2024-09-14T22:36:54.928 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.220881951999999, 2024-09-14T22:36:54.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045957114, 2024-09-14T22:36:54.929 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:54.929 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:54.929 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:54.929 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:54.929 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:54.929 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:54.929 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4149 2024-09-14T22:36:54.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.929 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:54.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:54.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703281+0000", 2024-09-14T22:36:54.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:54.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:54.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703281+0000", 2024-09-14T22:36:54.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:54.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:54.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703282+0000", 2024-09-14T22:36:54.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4199999999999996e-07 2024-09-14T22:36:54.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:54.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703289+0000", 2024-09-14T22:36:54.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0770000000000002e-06 2024-09-14T22:36:54.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:54.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703296+0000", 2024-09-14T22:36:54.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8129999999999998e-06 2024-09-14T22:36:54.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:54.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703307+0000", 2024-09-14T22:36:54.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1663e-05 2024-09-14T22:36:54.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:54.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749073+0000", 2024-09-14T22:36:54.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.04576537 2024-09-14T22:36:54.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:54.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749156+0000", 2024-09-14T22:36:54.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3399999999999994e-05 2024-09-14T22:36:54.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:54.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749239+0000", 2024-09-14T22:36:54.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2249000000000005e-05 2024-09-14T22:36:54.932 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:54.933 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:54.933 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:54.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.933 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4150 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13137920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:54.933 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703318+0000", 2024-09-14T22:36:54.933 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.220845757999999, 2024-09-14T22:36:54.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046170585, 2024-09-14T22:36:54.933 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:54.933 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:54.933 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:54.933 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:54.933 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:54.934 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:54.934 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4150 2024-09-14T22:36:54.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.934 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:54.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:54.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703318+0000", 2024-09-14T22:36:54.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:54.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:54.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703318+0000", 2024-09-14T22:36:54.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:54.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:54.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703318+0000", 2024-09-14T22:36:54.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8800000000000003e-07 2024-09-14T22:36:54.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:54.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703321+0000", 2024-09-14T22:36:54.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7860000000000001e-06 2024-09-14T22:36:54.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:54.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703328+0000", 2024-09-14T22:36:54.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2810000000000003e-06 2024-09-14T22:36:54.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:54.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703340+0000", 2024-09-14T22:36:54.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1745e-05 2024-09-14T22:36:54.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:54.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749286+0000", 2024-09-14T22:36:54.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045946222000000002 2024-09-14T22:36:54.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:54.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749390+0000", 2024-09-14T22:36:54.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000103614 2024-09-14T22:36:54.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:54.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749488+0000", 2024-09-14T22:36:54.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8449000000000006e-05 2024-09-14T22:36:54.937 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:54.937 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:54.938 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:54.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.938 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4151 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13146112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:54.938 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703348+0000", 2024-09-14T22:36:54.938 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.220815595000001, 2024-09-14T22:36:54.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046349509999999997, 2024-09-14T22:36:54.938 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:54.938 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:54.938 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:54.938 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:54.938 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:54.938 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:54.938 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4151 2024-09-14T22:36:54.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.939 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:54.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:54.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703348+0000", 2024-09-14T22:36:54.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:54.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:54.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703348+0000", 2024-09-14T22:36:54.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:54.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:54.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703348+0000", 2024-09-14T22:36:54.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6899999999999998e-07 2024-09-14T22:36:54.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:54.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703351+0000", 2024-09-14T22:36:54.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5950000000000001e-06 2024-09-14T22:36:54.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:54.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703357+0000", 2024-09-14T22:36:54.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4150000000000004e-06 2024-09-14T22:36:54.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:54.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703364+0000", 2024-09-14T22:36:54.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4069999999999998e-06 2024-09-14T22:36:54.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:54.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749530+0000", 2024-09-14T22:36:54.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046166281000000003 2024-09-14T22:36:54.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:54.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749615+0000", 2024-09-14T22:36:54.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5409999999999997e-05 2024-09-14T22:36:54.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:54.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749697+0000", 2024-09-14T22:36:54.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1933000000000003e-05 2024-09-14T22:36:54.942 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:54.942 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:54.942 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:54.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.942 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4152 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13154304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:54.942 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703371+0000", 2024-09-14T22:36:54.942 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.220792193000001, 2024-09-14T22:36:54.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046552233999999998, 2024-09-14T22:36:54.942 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:54.943 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:54.943 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:54.943 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:54.943 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:54.943 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:54.943 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4152 2024-09-14T22:36:54.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.943 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:54.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:54.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703371+0000", 2024-09-14T22:36:54.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:54.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:54.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703371+0000", 2024-09-14T22:36:54.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:54.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:54.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703371+0000", 2024-09-14T22:36:54.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.34e-07 2024-09-14T22:36:54.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:54.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703374+0000", 2024-09-14T22:36:54.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.227e-06 2024-09-14T22:36:54.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:54.945 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703380+0000", 2024-09-14T22:36:54.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4699999999999999e-06 2024-09-14T22:36:54.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:54.945 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703385+0000", 2024-09-14T22:36:54.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.259e-06 2024-09-14T22:36:54.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:54.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749742+0000", 2024-09-14T22:36:54.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046357031 2024-09-14T22:36:54.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:54.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749825+0000", 2024-09-14T22:36:54.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2674000000000002e-05 2024-09-14T22:36:54.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:54.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749923+0000", 2024-09-14T22:36:54.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8239000000000006e-05 2024-09-14T22:36:54.946 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:54.946 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:54.946 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:54.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.947 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4153 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13162496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:54.947 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703392+0000", 2024-09-14T22:36:54.947 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.220770897000001, 2024-09-14T22:36:54.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046675756999999998, 2024-09-14T22:36:54.947 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:54.947 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:54.947 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:54.947 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:54.947 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:54.947 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:54.947 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4153 2024-09-14T22:36:54.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.947 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:54.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:54.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703392+0000", 2024-09-14T22:36:54.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:54.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:54.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703392+0000", 2024-09-14T22:36:54.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:54.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:54.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.705934+0000", 2024-09-14T22:36:54.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025418540000000001 2024-09-14T22:36:54.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:54.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.705940+0000", 2024-09-14T22:36:54.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0950000000000001e-06 2024-09-14T22:36:54.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:54.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.705952+0000", 2024-09-14T22:36:54.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1973e-05 2024-09-14T22:36:54.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:54.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.705958+0000", 2024-09-14T22:36:54.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.541e-06 2024-09-14T22:36:54.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:54.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749969+0000", 2024-09-14T22:36:54.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044011300000000003 2024-09-14T22:36:54.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:54.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.750020+0000", 2024-09-14T22:36:54.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1054000000000003e-05 2024-09-14T22:36:54.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:54.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.750068+0000", 2024-09-14T22:36:54.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7939999999999998e-05 2024-09-14T22:36:54.951 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:54.951 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:54.951 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:54.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.951 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4165 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13260800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:54.951 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706178+0000", 2024-09-14T22:36:54.951 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.217984985000001, 2024-09-14T22:36:54.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045915594999999997, 2024-09-14T22:36:54.951 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:54.951 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:54.951 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:54.951 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:54.952 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:54.952 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:54.952 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4165 2024-09-14T22:36:54.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.952 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:54.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:54.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706178+0000", 2024-09-14T22:36:54.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:54.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:54.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706178+0000", 2024-09-14T22:36:54.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:54.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:54.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706179+0000", 2024-09-14T22:36:54.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4999999999999998e-07 2024-09-14T22:36:54.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:54.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706181+0000", 2024-09-14T22:36:54.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4679999999999999e-06 2024-09-14T22:36:54.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:54.954 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706189+0000", 2024-09-14T22:36:54.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.994e-06 2024-09-14T22:36:54.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.954 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:54.954 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706195+0000", 2024-09-14T22:36:54.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6119999999999998e-06 2024-09-14T22:36:54.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.954 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:54.954 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752003+0000", 2024-09-14T22:36:54.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045807618000000001 2024-09-14T22:36:54.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:54.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752050+0000", 2024-09-14T22:36:54.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7301e-05 2024-09-14T22:36:54.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:54.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752094+0000", 2024-09-14T22:36:54.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4152e-05 2024-09-14T22:36:54.955 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:54.955 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:54.955 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:54.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.956 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4166 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13268992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:54.956 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706205+0000", 2024-09-14T22:36:54.956 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.217958796000001, 2024-09-14T22:36:54.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046092665999999997, 2024-09-14T22:36:54.956 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:54.956 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:54.956 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:54.956 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:54.956 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:54.956 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:54.956 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4166 2024-09-14T22:36:54.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.957 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:54.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:54.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706205+0000", 2024-09-14T22:36:54.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:54.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:54.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706205+0000", 2024-09-14T22:36:54.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:54.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:54.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706205+0000", 2024-09-14T22:36:54.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:36:54.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:54.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706208+0000", 2024-09-14T22:36:54.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8049999999999998e-06 2024-09-14T22:36:54.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:54.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706215+0000", 2024-09-14T22:36:54.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2010000000000002e-06 2024-09-14T22:36:54.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:54.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706223+0000", 2024-09-14T22:36:54.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7260000000000002e-06 2024-09-14T22:36:54.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:54.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752129+0000", 2024-09-14T22:36:54.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045906532 2024-09-14T22:36:54.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:54.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752213+0000", 2024-09-14T22:36:54.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4060000000000005e-05 2024-09-14T22:36:54.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:54.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752297+0000", 2024-09-14T22:36:54.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3807999999999994e-05 2024-09-14T22:36:54.960 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:54.960 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:54.960 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:54.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.961 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4167 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13277184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:54.961 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706233+0000", 2024-09-14T22:36:54.961 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.217930185, 2024-09-14T22:36:54.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046190500000000002, 2024-09-14T22:36:54.961 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:54.961 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:54.961 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:54.961 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:54.961 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:54.961 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:54.961 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4167 2024-09-14T22:36:54.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.961 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:54.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:54.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706233+0000", 2024-09-14T22:36:54.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:54.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:54.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706233+0000", 2024-09-14T22:36:54.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:54.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:54.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706234+0000", 2024-09-14T22:36:54.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.75e-07 2024-09-14T22:36:54.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:54.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706237+0000", 2024-09-14T22:36:54.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9890000000000001e-06 2024-09-14T22:36:54.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:54.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706244+0000", 2024-09-14T22:36:54.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6240000000000002e-06 2024-09-14T22:36:54.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:54.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706255+0000", 2024-09-14T22:36:54.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0328e-05 2024-09-14T22:36:54.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:54.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752338+0000", 2024-09-14T22:36:54.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046083172999999998 2024-09-14T22:36:54.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:54.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752381+0000", 2024-09-14T22:36:54.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2973999999999999e-05 2024-09-14T22:36:54.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:54.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752424+0000", 2024-09-14T22:36:54.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2936999999999999e-05 2024-09-14T22:36:54.964 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:54.965 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:54.965 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:54.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.965 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4168 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13285376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:54.965 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706261+0000", 2024-09-14T22:36:54.965 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.217902841000001, 2024-09-14T22:36:54.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046378635000000001, 2024-09-14T22:36:54.965 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:54.965 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:54.965 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:54.965 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:54.965 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:54.966 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:54.966 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4168 2024-09-14T22:36:54.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.966 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:54.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:54.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706261+0000", 2024-09-14T22:36:54.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:54.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:54.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706261+0000", 2024-09-14T22:36:54.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:54.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:54.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706261+0000", 2024-09-14T22:36:54.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2899999999999999e-07 2024-09-14T22:36:54.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:54.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706264+0000", 2024-09-14T22:36:54.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.774e-06 2024-09-14T22:36:54.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:54.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706267+0000", 2024-09-14T22:36:54.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7179999999999998e-06 2024-09-14T22:36:54.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:54.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706274+0000", 2024-09-14T22:36:54.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7599999999999997e-06 2024-09-14T22:36:54.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:54.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752458+0000", 2024-09-14T22:36:54.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046183825999999997 2024-09-14T22:36:54.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:54.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752545+0000", 2024-09-14T22:36:54.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7272000000000001e-05 2024-09-14T22:36:54.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:54.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752639+0000", 2024-09-14T22:36:54.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3956000000000006e-05 2024-09-14T22:36:54.969 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:54.969 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:54.969 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:54.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.969 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4169 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13293568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:54.969 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706278+0000", 2024-09-14T22:36:54.969 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.217885114000001, 2024-09-14T22:36:54.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046489181999999997, 2024-09-14T22:36:54.969 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:54.970 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:54.970 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:54.970 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:54.970 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:54.970 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:54.970 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4169 2024-09-14T22:36:54.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.970 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:54.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:54.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706278+0000", 2024-09-14T22:36:54.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:54.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:54.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706278+0000", 2024-09-14T22:36:54.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:54.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:54.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706279+0000", 2024-09-14T22:36:54.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6100000000000002e-07 2024-09-14T22:36:54.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:54.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706280+0000", 2024-09-14T22:36:54.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2589999999999999e-06 2024-09-14T22:36:54.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:54.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706283+0000", 2024-09-14T22:36:54.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.501e-06 2024-09-14T22:36:54.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.973 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:54.973 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706286+0000", 2024-09-14T22:36:54.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5409999999999999e-06 2024-09-14T22:36:54.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.973 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:54.973 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752680+0000", 2024-09-14T22:36:54.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046393886000000002 2024-09-14T22:36:54.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.973 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:54.973 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752723+0000", 2024-09-14T22:36:54.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2870000000000001e-05 2024-09-14T22:36:54.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:54.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752767+0000", 2024-09-14T22:36:54.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4864000000000003e-05 2024-09-14T22:36:54.974 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:54.974 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:54.974 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:54.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.974 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4170 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13301760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:36:54.974 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706289+0000", 2024-09-14T22:36:54.974 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.217874215999998, 2024-09-14T22:36:54.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046691115999999998, 2024-09-14T22:36:54.975 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:54.975 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:54.975 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:54.975 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:54.975 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:54.975 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:54.975 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4170 2024-09-14T22:36:54.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.975 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:54.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:54.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706289+0000", 2024-09-14T22:36:54.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:54.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:54.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706289+0000", 2024-09-14T22:36:54.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:54.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:54.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706289+0000", 2024-09-14T22:36:54.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3200000000000001e-07 2024-09-14T22:36:54.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:54.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706296+0000", 2024-09-14T22:36:54.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.922e-06 2024-09-14T22:36:54.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:54.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706303+0000", 2024-09-14T22:36:54.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3980000000000004e-06 2024-09-14T22:36:54.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:54.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706307+0000", 2024-09-14T22:36:54.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6750000000000001e-06 2024-09-14T22:36:54.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:54.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752799+0000", 2024-09-14T22:36:54.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046491609000000003 2024-09-14T22:36:54.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:54.978 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752884+0000", 2024-09-14T22:36:54.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4876999999999994e-05 2024-09-14T22:36:54.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:54.978 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752980+0000", 2024-09-14T22:36:54.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6403000000000005e-05 2024-09-14T22:36:54.978 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:54.978 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:54.978 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:54.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.979 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6292 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8329216~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:54.979 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966066+0000", 2024-09-14T22:36:54.979 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 30.958097278, 2024-09-14T22:36:54.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045967367000000002, 2024-09-14T22:36:54.979 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:54.979 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:54.979 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:54.979 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:54.979 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:54.979 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:54.979 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6292 2024-09-14T22:36:54.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.979 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:54.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:54.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966066+0000", 2024-09-14T22:36:54.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:54.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:54.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966066+0000", 2024-09-14T22:36:54.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:54.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:54.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966067+0000", 2024-09-14T22:36:54.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6499999999999999e-07 2024-09-14T22:36:54.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:54.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966069+0000", 2024-09-14T22:36:54.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4059999999999999e-06 2024-09-14T22:36:54.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:54.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966077+0000", 2024-09-14T22:36:54.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5120000000000002e-06 2024-09-14T22:36:54.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:54.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966083+0000", 2024-09-14T22:36:54.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5590000000000002e-06 2024-09-14T22:36:54.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:54.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.011820+0000", 2024-09-14T22:36:54.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045737281999999997 2024-09-14T22:36:54.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:54.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.011911+0000", 2024-09-14T22:36:54.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0153000000000002e-05 2024-09-14T22:36:54.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:54.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012033+0000", 2024-09-14T22:36:54.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012289000000000001 2024-09-14T22:36:54.983 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:54.983 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:54.983 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:54.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.983 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6293 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8337408~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:54.983 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966091+0000", 2024-09-14T22:36:54.983 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 30.958072183999999, 2024-09-14T22:36:54.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046181672, 2024-09-14T22:36:54.984 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:54.984 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:54.984 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:54.984 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:54.984 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:54.984 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:54.984 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6293 2024-09-14T22:36:54.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.984 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:54.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:54.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966091+0000", 2024-09-14T22:36:54.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:54.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:54.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966091+0000", 2024-09-14T22:36:54.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:54.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:54.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966092+0000", 2024-09-14T22:36:54.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:36:54.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:54.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966094+0000", 2024-09-14T22:36:54.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5830000000000001e-06 2024-09-14T22:36:54.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:54.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966102+0000", 2024-09-14T22:36:54.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7470000000000005e-06 2024-09-14T22:36:54.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:54.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966107+0000", 2024-09-14T22:36:54.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1209999999999998e-06 2024-09-14T22:36:54.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:54.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012079+0000", 2024-09-14T22:36:54.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045971953000000003 2024-09-14T22:36:54.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:54.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012168+0000", 2024-09-14T22:36:54.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9398e-05 2024-09-14T22:36:54.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:54.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012273+0000", 2024-09-14T22:36:54.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000104483 2024-09-14T22:36:54.988 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:54.988 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:54.988 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:54.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.988 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6294 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8345600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:54.988 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966115+0000", 2024-09-14T22:36:54.988 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 30.958048820999998, 2024-09-14T22:36:54.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046297036999999999, 2024-09-14T22:36:54.988 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:54.988 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:54.988 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:54.988 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:54.989 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:54.989 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:54.989 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6294 2024-09-14T22:36:54.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.989 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:54.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:54.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966115+0000", 2024-09-14T22:36:54.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:54.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:54.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966115+0000", 2024-09-14T22:36:54.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:54.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:54.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966115+0000", 2024-09-14T22:36:54.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7e-07 2024-09-14T22:36:54.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:54.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966118+0000", 2024-09-14T22:36:54.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.571e-06 2024-09-14T22:36:54.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:54.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966125+0000", 2024-09-14T22:36:54.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1690000000000003e-06 2024-09-14T22:36:54.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:54.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966130+0000", 2024-09-14T22:36:54.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3290000000000001e-06 2024-09-14T22:36:54.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:54.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012298+0000", 2024-09-14T22:36:54.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046168168000000002 2024-09-14T22:36:54.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:54.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012348+0000", 2024-09-14T22:36:54.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9663999999999997e-05 2024-09-14T22:36:54.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:54.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012412+0000", 2024-09-14T22:36:54.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3665999999999997e-05 2024-09-14T22:36:54.992 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:54.992 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:54.992 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:54.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.993 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6295 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8353792~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:54.993 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966138+0000", 2024-09-14T22:36:54.993 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 30.958025843000001, 2024-09-14T22:36:54.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046505112000000001, 2024-09-14T22:36:54.993 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:54.993 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:54.993 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:54.993 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:54.993 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:54.993 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:54.993 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6295 2024-09-14T22:36:54.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.993 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:54.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:54.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966138+0000", 2024-09-14T22:36:54.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:54.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:54.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966138+0000", 2024-09-14T22:36:54.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:54.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:54.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966138+0000", 2024-09-14T22:36:54.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5900000000000002e-07 2024-09-14T22:36:54.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:54.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966140+0000", 2024-09-14T22:36:54.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4710000000000001e-06 2024-09-14T22:36:54.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:54.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966147+0000", 2024-09-14T22:36:54.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6760000000000001e-06 2024-09-14T22:36:54.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:54.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966153+0000", 2024-09-14T22:36:54.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4779999999999999e-06 2024-09-14T22:36:54.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:54.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012452+0000", 2024-09-14T22:36:54.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046299298000000003 2024-09-14T22:36:54.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:54.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012537+0000", 2024-09-14T22:36:54.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5439999999999995e-05 2024-09-14T22:36:54.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:54.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012643+0000", 2024-09-14T22:36:54.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010529 2024-09-14T22:36:54.997 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:54.997 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:54.997 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:54.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.997 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6296 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8361984~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:54.997 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966160+0000", 2024-09-14T22:36:54.997 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 30.958003788999999, 2024-09-14T22:36:54.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046601423000000003, 2024-09-14T22:36:54.997 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:54.997 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:54.998 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:54.998 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:54.998 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:54.998 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:54.998 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6296 2024-09-14T22:36:54.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.998 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:54.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:54.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966160+0000", 2024-09-14T22:36:54.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:54.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:54.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966160+0000", 2024-09-14T22:36:54.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:54.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:54.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966160+0000", 2024-09-14T22:36:54.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7999999999999995e-07 2024-09-14T22:36:54.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:54.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966163+0000", 2024-09-14T22:36:54.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.514e-06 2024-09-14T22:36:54.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:54.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:54.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:54.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966169+0000", 2024-09-14T22:36:54.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8880000000000002e-06 2024-09-14T22:36:55.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:55.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966173+0000", 2024-09-14T22:36:55.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9189999999999998e-06 2024-09-14T22:36:55.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:55.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012660+0000", 2024-09-14T22:36:55.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046486522000000002 2024-09-14T22:36:55.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:55.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012703+0000", 2024-09-14T22:36:55.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2715e-05 2024-09-14T22:36:55.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:55.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012761+0000", 2024-09-14T22:36:55.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8285000000000002e-05 2024-09-14T22:36:55.001 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.001 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:55.001 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.001 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6297 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8370176~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:55.002 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966180+0000", 2024-09-14T22:36:55.002 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 30.957982985000001, 2024-09-14T22:36:55.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046808482999999998, 2024-09-14T22:36:55.002 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:55.002 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:55.002 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:55.002 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:55.002 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:55.002 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:55.002 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6297 2024-09-14T22:36:55.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.002 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:55.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:55.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966180+0000", 2024-09-14T22:36:55.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:55.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966180+0000", 2024-09-14T22:36:55.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:55.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966181+0000", 2024-09-14T22:36:55.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0600000000000001e-07 2024-09-14T22:36:55.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:55.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966183+0000", 2024-09-14T22:36:55.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3539999999999998e-06 2024-09-14T22:36:55.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:55.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966190+0000", 2024-09-14T22:36:55.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.489e-06 2024-09-14T22:36:55.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:55.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966195+0000", 2024-09-14T22:36:55.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3890000000000004e-06 2024-09-14T22:36:55.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:55.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012799+0000", 2024-09-14T22:36:55.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046604200999999998 2024-09-14T22:36:55.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:55.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012883+0000", 2024-09-14T22:36:55.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3714000000000005e-05 2024-09-14T22:36:55.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:55.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012989+0000", 2024-09-14T22:36:55.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010593 2024-09-14T22:36:55.006 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.006 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:55.006 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.006 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6298 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8378368~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:55.006 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966203+0000", 2024-09-14T22:36:55.006 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 30.957959924000001, 2024-09-14T22:36:55.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046904649999999999, 2024-09-14T22:36:55.007 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:55.007 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:55.007 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:55.007 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:55.007 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:55.007 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:55.007 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6298 2024-09-14T22:36:55.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.007 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:55.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:55.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966203+0000", 2024-09-14T22:36:55.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:55.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966203+0000", 2024-09-14T22:36:55.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:55.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966205+0000", 2024-09-14T22:36:55.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5009999999999999e-06 2024-09-14T22:36:55.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:55.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966207+0000", 2024-09-14T22:36:55.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2019999999999998e-06 2024-09-14T22:36:55.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:55.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966214+0000", 2024-09-14T22:36:55.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.815e-06 2024-09-14T22:36:55.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:55.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966219+0000", 2024-09-14T22:36:55.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0590000000000002e-06 2024-09-14T22:36:55.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:55.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013005+0000", 2024-09-14T22:36:55.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046785852000000003 2024-09-14T22:36:55.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:55.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013050+0000", 2024-09-14T22:36:55.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.511e-05 2024-09-14T22:36:55.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:55.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013108+0000", 2024-09-14T22:36:55.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8111e-05 2024-09-14T22:36:55.011 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.011 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:55.011 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.011 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6299 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8386560~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:55.011 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966238+0000", 2024-09-14T22:36:55.011 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 30.957925603, 2024-09-14T22:36:55.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047091099999999997, 2024-09-14T22:36:55.011 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:55.011 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:55.011 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:55.011 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:55.012 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:55.012 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:55.012 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6299 2024-09-14T22:36:55.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.012 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:55.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:55.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966238+0000", 2024-09-14T22:36:55.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:55.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966238+0000", 2024-09-14T22:36:55.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:55.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966238+0000", 2024-09-14T22:36:55.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1500000000000005e-07 2024-09-14T22:36:55.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:55.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966241+0000", 2024-09-14T22:36:55.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8270000000000002e-06 2024-09-14T22:36:55.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:55.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966249+0000", 2024-09-14T22:36:55.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0029999999999994e-06 2024-09-14T22:36:55.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:55.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966255+0000", 2024-09-14T22:36:55.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7760000000000003e-06 2024-09-14T22:36:55.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:55.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013136+0000", 2024-09-14T22:36:55.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046881379000000001 2024-09-14T22:36:55.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:55.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013223+0000", 2024-09-14T22:36:55.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6681000000000006e-05 2024-09-14T22:36:55.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:55.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013329+0000", 2024-09-14T22:36:55.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105919 2024-09-14T22:36:55.015 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.015 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:55.015 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.015 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:31370 3.0 3:02f48f58:::rbd_data.113cda0183a0.0000000000000000:6 [read 813056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:55.016 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:36:55.016 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 21.561765510000001, 2024-09-14T22:36:55.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047519796000000003, 2024-09-14T22:36:55.016 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:55.016 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:55.016 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:55.016 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:55.016 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:55.016 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:55.016 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 31370 2024-09-14T22:36:55.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.016 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:55.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:55.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:36:55.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:55.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:36:55.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:55.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:36:55.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0999999999999999e-07 2024-09-14T22:36:55.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:55.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362401+0000", 2024-09-14T22:36:55.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2900000000000001e-06 2024-09-14T22:36:55.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:55.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362406+0000", 2024-09-14T22:36:55.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6960000000000002e-06 2024-09-14T22:36:55.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:55.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362412+0000", 2024-09-14T22:36:55.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.823e-06 2024-09-14T22:36:55.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:55.019 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.400270+0000", 2024-09-14T22:36:55.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037857737000000002 2024-09-14T22:36:55.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:55.019 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.400342+0000", 2024-09-14T22:36:55.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1797000000000004e-05 2024-09-14T22:36:55.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:55.019 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.409918+0000", 2024-09-14T22:36:55.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0095760429999999994 2024-09-14T22:36:55.020 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.020 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:55.020 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.020 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.020 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:55.020 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:55.021 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:36:55.022 INFO:teuthology.orchestra.run.smithi001.stderr:set 3.9 pg_upmap_items mapping to [7->4,1->2] 2024-09-14T22:36:55.022 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:55.015+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:55.125 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:55.123+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:55.187 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone12 from 0x11cf167 (overlap 0x11cf167) to 0x557b9e 2024-09-14T22:36:55.224 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:55.224 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:36:55.224 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:36:55.224 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:55.224 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:55.219+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:55.240 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:36:55.324 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:55.319+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:55.362 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-14T22:36:55.396 INFO:teuthology.orchestra.run.smithi001.stdout:1908 write 0x61492b5 thru 0x6156296 (0xcfe2 bytes) 2024-09-14T22:36:55.417 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:55.418 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:36:55.418 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:36:55.418 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:36:55.418 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:55.424 INFO:teuthology.orchestra.run.smithi001.stdout:1909 write 0x915fd75 thru 0x916fbae (0xfe3a bytes) 2024-09-14T22:36:55.424 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:55.419+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:55.432 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:36:55.438 INFO:teuthology.orchestra.run.smithi001.stdout:1911 trunc from 0x916fbaf to 0xdc62615 2024-09-14T22:36:55.451 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:55.452 INFO:teuthology.orchestra.run.smithi001.stdout:1913 read 0xa8f17fe thru 0xa8f2cf2 (0x14f5 bytes) 2024-09-14T22:36:55.452 INFO:teuthology.orchestra.run.smithi001.stdout:1914 read 0x2ce417c thru 0x2cee9ca (0xa84f bytes) 2024-09-14T22:36:55.452 INFO:teuthology.orchestra.run.smithi001.stdout:1916 writesame 0x7ca01d2 thru 0x7ca1fdc data_size 0x1(0x1e0b bytes) 2024-09-14T22:36:55.463 INFO:teuthology.orchestra.run.smithi001.stdout:1917 write 0xd9584e5 thru 0xd95b545 (0x3061 bytes) 2024-09-14T22:36:55.470 INFO:teuthology.orchestra.run.smithi001.stdout:1918 punch from 0x935c599 to 0x93661ad, (0x9c14 bytes) 2024-09-14T22:36:55.470 INFO:teuthology.orchestra.run.smithi001.stdout:1919 write 0x85dfea9 thru 0x85ec330 (0xc488 bytes) 2024-09-14T22:36:55.478 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:55.485 INFO:teuthology.orchestra.run.smithi001.stdout:1921 trunc from 0xdc62615 to 0x9a3b4ac 2024-09-14T22:36:55.510 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:55.510 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:55.510 INFO:teuthology.orchestra.run.smithi001.stdout:1926 writesame 0x113b499 thru 0x1149383 data_size 0x1(0xdeeb bytes) 2024-09-14T22:36:55.526 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:55.519+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:55.572 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:55.572 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:36:55.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:36:55.572 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:36:55.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.573 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35300 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1253376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:55.573 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.506930+0000", 2024-09-14T22:36:55.573 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.062365065000002, 2024-09-14T22:36:55.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044627325000000002, 2024-09-14T22:36:55.573 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:55.573 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:55.573 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:55.573 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:55.573 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:55.573 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:55.573 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35300 2024-09-14T22:36:55.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.574 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:55.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:55.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506930+0000", 2024-09-14T22:36:55.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:55.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506930+0000", 2024-09-14T22:36:55.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:55.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506930+0000", 2024-09-14T22:36:55.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2500000000000001e-07 2024-09-14T22:36:55.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:55.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506932+0000", 2024-09-14T22:36:55.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8300000000000001e-06 2024-09-14T22:36:55.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:55.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506940+0000", 2024-09-14T22:36:55.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2810000000000003e-06 2024-09-14T22:36:55.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:55.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506944+0000", 2024-09-14T22:36:55.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8049999999999997e-06 2024-09-14T22:36:55.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:55.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.551359+0000", 2024-09-14T22:36:55.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044414370000000002 2024-09-14T22:36:55.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:55.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.551432+0000", 2024-09-14T22:36:55.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3338999999999997e-05 2024-09-14T22:36:55.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:55.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.551557+0000", 2024-09-14T22:36:55.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000125275 2024-09-14T22:36:55.577 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.577 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:55.577 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.577 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35301 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1282048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:55.577 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.506950+0000", 2024-09-14T22:36:55.577 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.062344864, 2024-09-14T22:36:55.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044873612, 2024-09-14T22:36:55.577 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:55.577 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:55.578 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:55.578 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:55.578 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:55.578 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:55.578 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35301 2024-09-14T22:36:55.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.578 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:55.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:55.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506950+0000", 2024-09-14T22:36:55.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:55.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506950+0000", 2024-09-14T22:36:55.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:55.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506951+0000", 2024-09-14T22:36:55.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0400000000000002e-07 2024-09-14T22:36:55.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:55.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506953+0000", 2024-09-14T22:36:55.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9470000000000002e-06 2024-09-14T22:36:55.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:55.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506958+0000", 2024-09-14T22:36:55.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8930000000000002e-06 2024-09-14T22:36:55.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:55.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506963+0000", 2024-09-14T22:36:55.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.459e-06 2024-09-14T22:36:55.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:55.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.551642+0000", 2024-09-14T22:36:55.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044678766000000002 2024-09-14T22:36:55.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:55.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.551715+0000", 2024-09-14T22:36:55.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3603999999999995e-05 2024-09-14T22:36:55.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:55.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.551824+0000", 2024-09-14T22:36:55.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108539 2024-09-14T22:36:55.581 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.581 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:55.582 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.582 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35302 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1310720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:55.582 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.506970+0000", 2024-09-14T22:36:55.582 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.062325154, 2024-09-14T22:36:55.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045057898999999998, 2024-09-14T22:36:55.582 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:55.582 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:55.582 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:55.582 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:55.582 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:55.582 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:55.582 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35302 2024-09-14T22:36:55.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.583 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:55.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:55.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506970+0000", 2024-09-14T22:36:55.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:55.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506970+0000", 2024-09-14T22:36:55.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:55.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506970+0000", 2024-09-14T22:36:55.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0999999999999999e-07 2024-09-14T22:36:55.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:55.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506973+0000", 2024-09-14T22:36:55.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4310000000000001e-06 2024-09-14T22:36:55.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:55.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506981+0000", 2024-09-14T22:36:55.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2830000000000001e-06 2024-09-14T22:36:55.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:55.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506986+0000", 2024-09-14T22:36:55.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3959999999999997e-06 2024-09-14T22:36:55.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:55.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.551856+0000", 2024-09-14T22:36:55.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044869603000000001 2024-09-14T22:36:55.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:55.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.551931+0000", 2024-09-14T22:36:55.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5253999999999995e-05 2024-09-14T22:36:55.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:55.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552028+0000", 2024-09-14T22:36:55.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6521999999999995e-05 2024-09-14T22:36:55.586 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.586 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:55.586 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.586 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35303 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1339392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:55.587 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.506994+0000", 2024-09-14T22:36:55.587 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.062301246000001, 2024-09-14T22:36:55.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045224116000000002, 2024-09-14T22:36:55.587 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:55.587 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:55.587 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:55.587 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:55.587 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:55.587 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:55.587 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35303 2024-09-14T22:36:55.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.587 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:55.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:55.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506994+0000", 2024-09-14T22:36:55.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:55.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506994+0000", 2024-09-14T22:36:55.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:55.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506994+0000", 2024-09-14T22:36:55.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1699999999999999e-07 2024-09-14T22:36:55.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:55.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506997+0000", 2024-09-14T22:36:55.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.357e-06 2024-09-14T22:36:55.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:55.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507005+0000", 2024-09-14T22:36:55.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1939999999999998e-06 2024-09-14T22:36:55.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:55.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507013+0000", 2024-09-14T22:36:55.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3930000000000008e-06 2024-09-14T22:36:55.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:55.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552058+0000", 2024-09-14T22:36:55.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045044645000000001 2024-09-14T22:36:55.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:55.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552132+0000", 2024-09-14T22:36:55.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3831000000000005e-05 2024-09-14T22:36:55.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:55.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552218+0000", 2024-09-14T22:36:55.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6279000000000005e-05 2024-09-14T22:36:55.591 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.591 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:55.591 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.591 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35304 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1368064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:55.591 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507021+0000", 2024-09-14T22:36:55.591 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.062273686000001, 2024-09-14T22:36:55.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045388246, 2024-09-14T22:36:55.591 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:55.591 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:55.591 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:55.591 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:55.591 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:55.592 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:55.592 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35304 2024-09-14T22:36:55.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.592 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:55.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:55.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507021+0000", 2024-09-14T22:36:55.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:55.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507021+0000", 2024-09-14T22:36:55.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:55.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507022+0000", 2024-09-14T22:36:55.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.82e-07 2024-09-14T22:36:55.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:55.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507024+0000", 2024-09-14T22:36:55.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4949999999999998e-06 2024-09-14T22:36:55.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:55.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507032+0000", 2024-09-14T22:36:55.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9449999999999993e-06 2024-09-14T22:36:55.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:55.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507039+0000", 2024-09-14T22:36:55.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2129999999999996e-06 2024-09-14T22:36:55.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:55.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552250+0000", 2024-09-14T22:36:55.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045211150999999998 2024-09-14T22:36:55.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:55.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552326+0000", 2024-09-14T22:36:55.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6008999999999997e-05 2024-09-14T22:36:55.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:55.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552410+0000", 2024-09-14T22:36:55.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3950999999999996e-05 2024-09-14T22:36:55.595 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.596 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:55.596 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.596 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35305 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1396736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:55.596 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507048+0000", 2024-09-14T22:36:55.596 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.062247148000001, 2024-09-14T22:36:55.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045554728000000003, 2024-09-14T22:36:55.596 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:55.596 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:55.596 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:55.596 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:55.596 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:55.597 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:55.597 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35305 2024-09-14T22:36:55.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.597 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:55.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:55.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507048+0000", 2024-09-14T22:36:55.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:55.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507048+0000", 2024-09-14T22:36:55.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:55.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507048+0000", 2024-09-14T22:36:55.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7800000000000002e-07 2024-09-14T22:36:55.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:55.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507050+0000", 2024-09-14T22:36:55.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.024e-06 2024-09-14T22:36:55.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:55.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507057+0000", 2024-09-14T22:36:55.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1800000000000001e-06 2024-09-14T22:36:55.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:55.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507062+0000", 2024-09-14T22:36:55.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0930000000000002e-06 2024-09-14T22:36:55.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:55.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552439+0000", 2024-09-14T22:36:55.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045377460000000001 2024-09-14T22:36:55.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:55.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552512+0000", 2024-09-14T22:36:55.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2633999999999996e-05 2024-09-14T22:36:55.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:55.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552603+0000", 2024-09-14T22:36:55.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0958999999999994e-05 2024-09-14T22:36:55.600 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.600 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:55.600 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.601 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35306 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1425408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:55.601 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507069+0000", 2024-09-14T22:36:55.601 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.062225604999998, 2024-09-14T22:36:55.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045725259999999997, 2024-09-14T22:36:55.601 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:55.601 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:55.601 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:55.601 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:55.601 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:55.601 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:55.601 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35306 2024-09-14T22:36:55.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.601 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:55.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:55.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507069+0000", 2024-09-14T22:36:55.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:55.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507069+0000", 2024-09-14T22:36:55.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:55.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507070+0000", 2024-09-14T22:36:55.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0400000000000002e-07 2024-09-14T22:36:55.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:55.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507072+0000", 2024-09-14T22:36:55.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.891e-06 2024-09-14T22:36:55.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:55.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507078+0000", 2024-09-14T22:36:55.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0279999999999997e-06 2024-09-14T22:36:55.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:55.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507082+0000", 2024-09-14T22:36:55.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4680000000000003e-06 2024-09-14T22:36:55.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:55.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552632+0000", 2024-09-14T22:36:55.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045550144000000001 2024-09-14T22:36:55.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:55.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552707+0000", 2024-09-14T22:36:55.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4148999999999998e-05 2024-09-14T22:36:55.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:55.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552795+0000", 2024-09-14T22:36:55.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8176000000000004e-05 2024-09-14T22:36:55.605 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.605 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:55.605 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.605 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35307 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1454080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:55.605 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:36:55.605 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.062207054000002, 2024-09-14T22:36:55.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045891358, 2024-09-14T22:36:55.605 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:55.605 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:55.605 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:55.605 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:55.606 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:55.606 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:55.606 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35307 2024-09-14T22:36:55.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.606 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:55.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:55.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:36:55.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:55.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:36:55.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:55.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:36:55.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4700000000000002e-07 2024-09-14T22:36:55.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:55.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507093+0000", 2024-09-14T22:36:55.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9520000000000002e-06 2024-09-14T22:36:55.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:55.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507099+0000", 2024-09-14T22:36:55.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0240000000000003e-06 2024-09-14T22:36:55.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:55.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507105+0000", 2024-09-14T22:36:55.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5520000000000004e-06 2024-09-14T22:36:55.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:55.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552826+0000", 2024-09-14T22:36:55.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045721494000000001 2024-09-14T22:36:55.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:55.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552896+0000", 2024-09-14T22:36:55.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9519999999999998e-05 2024-09-14T22:36:55.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:55.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552979+0000", 2024-09-14T22:36:55.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3468999999999997e-05 2024-09-14T22:36:55.609 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.609 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:55.609 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.609 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35308 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1482752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:55.609 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:36:55.610 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.062184058, 2024-09-14T22:36:55.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046076533000000003, 2024-09-14T22:36:55.610 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:55.610 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:55.610 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:55.610 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:55.610 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:55.610 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:55.610 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35308 2024-09-14T22:36:55.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.610 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:55.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:55.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:36:55.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:55.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:36:55.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:55.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:36:55.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:36:55.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:55.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507113+0000", 2024-09-14T22:36:55.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9099999999999999e-06 2024-09-14T22:36:55.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:55.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507119+0000", 2024-09-14T22:36:55.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7440000000000004e-06 2024-09-14T22:36:55.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:55.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507124+0000", 2024-09-14T22:36:55.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2129999999999999e-06 2024-09-14T22:36:55.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:55.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553007+0000", 2024-09-14T22:36:55.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045882488999999999 2024-09-14T22:36:55.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:55.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553076+0000", 2024-09-14T22:36:55.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9233000000000006e-05 2024-09-14T22:36:55.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:55.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553188+0000", 2024-09-14T22:36:55.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111557 2024-09-14T22:36:55.613 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.613 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:55.613 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.614 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35309 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1511424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:55.614 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:36:55.614 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.062165152999999, 2024-09-14T22:36:55.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046231869000000002, 2024-09-14T22:36:55.614 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:55.614 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:55.614 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:55.614 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:55.614 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:55.614 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:55.614 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35309 2024-09-14T22:36:55.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.614 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:55.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:55.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:36:55.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:55.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:36:55.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:55.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:36:55.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9099999999999999e-07 2024-09-14T22:36:55.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.616 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:55.616 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507132+0000", 2024-09-14T22:36:55.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8190000000000001e-06 2024-09-14T22:36:55.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.616 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:55.616 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507139+0000", 2024-09-14T22:36:55.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9419999999999998e-06 2024-09-14T22:36:55.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.616 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:55.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507145+0000", 2024-09-14T22:36:55.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2600000000000002e-06 2024-09-14T22:36:55.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:55.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553225+0000", 2024-09-14T22:36:55.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046079991000000001 2024-09-14T22:36:55.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:55.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553284+0000", 2024-09-14T22:36:55.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8638999999999999e-05 2024-09-14T22:36:55.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:55.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553362+0000", 2024-09-14T22:36:55.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7826999999999999e-05 2024-09-14T22:36:55.618 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.618 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:55.618 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.618 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35310 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1540096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:55.618 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507151+0000", 2024-09-14T22:36:55.618 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.062143697, 2024-09-14T22:36:55.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046444541999999998, 2024-09-14T22:36:55.618 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:55.619 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:55.619 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:55.619 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:55.619 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:55.619 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:55.619 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35310 2024-09-14T22:36:55.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.619 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:55.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:55.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507151+0000", 2024-09-14T22:36:55.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:55.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507151+0000", 2024-09-14T22:36:55.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:55.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507152+0000", 2024-09-14T22:36:55.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1699999999999999e-07 2024-09-14T22:36:55.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:55.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507154+0000", 2024-09-14T22:36:55.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.902e-06 2024-09-14T22:36:55.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:55.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507159+0000", 2024-09-14T22:36:55.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.823e-06 2024-09-14T22:36:55.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:55.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507165+0000", 2024-09-14T22:36:55.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.587e-06 2024-09-14T22:36:55.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:55.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553395+0000", 2024-09-14T22:36:55.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046230318999999999 2024-09-14T22:36:55.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:55.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553434+0000", 2024-09-14T22:36:55.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8680999999999998e-05 2024-09-14T22:36:55.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:55.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553596+0000", 2024-09-14T22:36:55.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016191299999999999 2024-09-14T22:36:55.623 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.623 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:55.623 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.623 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35311 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1568768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:55.623 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:36:55.623 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.062124443999998, 2024-09-14T22:36:55.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046623592999999998, 2024-09-14T22:36:55.623 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:55.623 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:55.623 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:55.623 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:55.623 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:55.624 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:55.624 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35311 2024-09-14T22:36:55.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.624 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:55.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:55.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:36:55.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:55.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:36:55.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.625 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:55.625 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:36:55.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:36:55.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.625 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:55.625 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507176+0000", 2024-09-14T22:36:55.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3349999999999997e-06 2024-09-14T22:36:55.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.625 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:55.625 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507200+0000", 2024-09-14T22:36:55.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4003e-05 2024-09-14T22:36:55.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:55.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507206+0000", 2024-09-14T22:36:55.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.976e-06 2024-09-14T22:36:55.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:55.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553627+0000", 2024-09-14T22:36:55.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046420284999999999 2024-09-14T22:36:55.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:55.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553696+0000", 2024-09-14T22:36:55.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9567999999999995e-05 2024-09-14T22:36:55.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:55.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553794+0000", 2024-09-14T22:36:55.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8060000000000006e-05 2024-09-14T22:36:55.627 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.627 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:55.627 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.628 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35562 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1370112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:55.628 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514584+0000", 2024-09-14T22:36:55.628 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.054711056999999, 2024-09-14T22:36:55.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044805208999999999, 2024-09-14T22:36:55.628 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:55.628 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:55.628 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:55.628 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:55.628 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:55.628 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:55.628 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35562 2024-09-14T22:36:55.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.628 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:55.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.629 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:55.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514584+0000", 2024-09-14T22:36:55.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.629 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:55.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514584+0000", 2024-09-14T22:36:55.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.629 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:55.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514584+0000", 2024-09-14T22:36:55.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0900000000000002e-07 2024-09-14T22:36:55.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:55.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514587+0000", 2024-09-14T22:36:55.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1739999999999999e-06 2024-09-14T22:36:55.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:55.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514593+0000", 2024-09-14T22:36:55.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3080000000000004e-06 2024-09-14T22:36:55.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:55.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514600+0000", 2024-09-14T22:36:55.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6420000000000001e-06 2024-09-14T22:36:55.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:55.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559211+0000", 2024-09-14T22:36:55.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044611454000000002 2024-09-14T22:36:55.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:55.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559296+0000", 2024-09-14T22:36:55.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4676e-05 2024-09-14T22:36:55.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:55.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559389+0000", 2024-09-14T22:36:55.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3546000000000002e-05 2024-09-14T22:36:55.632 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.632 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:55.632 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.632 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35563 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1398784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:55.632 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514606+0000", 2024-09-14T22:36:55.632 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.054688727999999, 2024-09-14T22:36:55.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045029859999999998, 2024-09-14T22:36:55.632 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:55.632 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:55.633 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:55.633 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:55.633 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:55.633 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:55.633 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35563 2024-09-14T22:36:55.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.633 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:55.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:55.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514606+0000", 2024-09-14T22:36:55.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:55.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514606+0000", 2024-09-14T22:36:55.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:55.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514607+0000", 2024-09-14T22:36:55.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1199999999999998e-07 2024-09-14T22:36:55.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:55.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514609+0000", 2024-09-14T22:36:55.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.199e-06 2024-09-14T22:36:55.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:55.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514615+0000", 2024-09-14T22:36:55.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1260000000000003e-06 2024-09-14T22:36:55.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:55.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514620+0000", 2024-09-14T22:36:55.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.045e-06 2024-09-14T22:36:55.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:55.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559440+0000", 2024-09-14T22:36:55.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044820169999999999 2024-09-14T22:36:55.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:55.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559524+0000", 2024-09-14T22:36:55.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4124999999999998e-05 2024-09-14T22:36:55.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:55.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559636+0000", 2024-09-14T22:36:55.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111783 2024-09-14T22:36:55.636 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.636 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:55.636 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.637 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35564 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1427456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:55.637 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514627+0000", 2024-09-14T22:36:55.637 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.054667988999999, 2024-09-14T22:36:55.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045232025000000002, 2024-09-14T22:36:55.637 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:55.637 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:55.637 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:55.637 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:55.637 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:55.637 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:55.637 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35564 2024-09-14T22:36:55.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.637 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:55.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:55.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514627+0000", 2024-09-14T22:36:55.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:55.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514627+0000", 2024-09-14T22:36:55.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:55.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514628+0000", 2024-09-14T22:36:55.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4299999999999998e-07 2024-09-14T22:36:55.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:55.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514630+0000", 2024-09-14T22:36:55.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1579999999999999e-06 2024-09-14T22:36:55.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:55.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514636+0000", 2024-09-14T22:36:55.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5520000000000001e-06 2024-09-14T22:36:55.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:55.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514641+0000", 2024-09-14T22:36:55.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6199999999999998e-06 2024-09-14T22:36:55.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:55.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559679+0000", 2024-09-14T22:36:55.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045038474000000002 2024-09-14T22:36:55.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:55.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559763+0000", 2024-09-14T22:36:55.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3357999999999997e-05 2024-09-14T22:36:55.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:55.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559859+0000", 2024-09-14T22:36:55.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6420000000000002e-05 2024-09-14T22:36:55.641 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.641 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:55.641 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.641 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35565 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1456128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:55.641 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514648+0000", 2024-09-14T22:36:55.641 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.054647091, 2024-09-14T22:36:55.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045442455999999999, 2024-09-14T22:36:55.641 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:55.641 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:55.641 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:55.641 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:55.641 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:55.642 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:55.642 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35565 2024-09-14T22:36:55.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.642 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:55.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:55.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514648+0000", 2024-09-14T22:36:55.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:55.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514648+0000", 2024-09-14T22:36:55.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:55.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514648+0000", 2024-09-14T22:36:55.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1399999999999997e-07 2024-09-14T22:36:55.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:55.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514651+0000", 2024-09-14T22:36:55.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.136e-06 2024-09-14T22:36:55.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:55.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514657+0000", 2024-09-14T22:36:55.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.241e-06 2024-09-14T22:36:55.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.644 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:55.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514662+0000", 2024-09-14T22:36:55.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9690000000000002e-06 2024-09-14T22:36:55.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.644 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:55.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559903+0000", 2024-09-14T22:36:55.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045241393999999997 2024-09-14T22:36:55.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.644 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:55.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559985+0000", 2024-09-14T22:36:55.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1460999999999998e-05 2024-09-14T22:36:55.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:55.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560090+0000", 2024-09-14T22:36:55.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010584100000000001 2024-09-14T22:36:55.645 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.645 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:55.645 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.645 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35566 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1484800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:55.645 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514668+0000", 2024-09-14T22:36:55.645 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.054626667000001, 2024-09-14T22:36:55.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045644974999999997, 2024-09-14T22:36:55.645 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:55.645 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:55.646 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:55.646 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:55.646 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:55.646 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:55.646 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35566 2024-09-14T22:36:55.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.646 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:55.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:55.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514668+0000", 2024-09-14T22:36:55.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:55.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514668+0000", 2024-09-14T22:36:55.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:55.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514669+0000", 2024-09-14T22:36:55.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9099999999999999e-07 2024-09-14T22:36:55.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:55.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514671+0000", 2024-09-14T22:36:55.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3800000000000001e-06 2024-09-14T22:36:55.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:55.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514677+0000", 2024-09-14T22:36:55.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9660000000000001e-06 2024-09-14T22:36:55.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:55.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514682+0000", 2024-09-14T22:36:55.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0939999999999998e-06 2024-09-14T22:36:55.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:55.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560133+0000", 2024-09-14T22:36:55.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045450398000000003 2024-09-14T22:36:55.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:55.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560207+0000", 2024-09-14T22:36:55.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4196999999999995e-05 2024-09-14T22:36:55.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:55.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560313+0000", 2024-09-14T22:36:55.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106549 2024-09-14T22:36:55.649 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.649 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:55.649 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.649 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35567 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1513472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:55.649 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:36:55.650 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.054606128, 2024-09-14T22:36:55.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045793215999999998, 2024-09-14T22:36:55.650 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:55.650 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:55.650 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:55.650 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:55.650 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:55.650 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:55.650 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35567 2024-09-14T22:36:55.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.650 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:55.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:55.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:36:55.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:55.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:36:55.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:55.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:36:55.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1899999999999998e-07 2024-09-14T22:36:55.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:55.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514692+0000", 2024-09-14T22:36:55.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2699999999999999e-06 2024-09-14T22:36:55.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:55.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514698+0000", 2024-09-14T22:36:55.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0939999999999996e-06 2024-09-14T22:36:55.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:55.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514703+0000", 2024-09-14T22:36:55.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3870000000000003e-06 2024-09-14T22:36:55.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:55.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560353+0000", 2024-09-14T22:36:55.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045649914 2024-09-14T22:36:55.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:55.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560416+0000", 2024-09-14T22:36:55.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3096999999999996e-05 2024-09-14T22:36:55.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:55.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560482+0000", 2024-09-14T22:36:55.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6035e-05 2024-09-14T22:36:55.654 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.654 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:55.654 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.654 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35568 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1542144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:55.654 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:36:55.654 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.054581330000001, 2024-09-14T22:36:55.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045995921000000002, 2024-09-14T22:36:55.654 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:55.654 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:55.654 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:55.654 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:55.654 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:55.654 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:55.654 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35568 2024-09-14T22:36:55.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.655 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:55.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:55.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:36:55.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:55.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:36:55.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:55.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:36:55.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4099999999999999e-07 2024-09-14T22:36:55.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:55.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514720+0000", 2024-09-14T22:36:55.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6869999999999999e-06 2024-09-14T22:36:55.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:55.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514727+0000", 2024-09-14T22:36:55.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3540000000000002e-06 2024-09-14T22:36:55.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:55.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514732+0000", 2024-09-14T22:36:55.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7690000000000004e-06 2024-09-14T22:36:55.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:55.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560523+0000", 2024-09-14T22:36:55.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045791373000000003 2024-09-14T22:36:55.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:55.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560606+0000", 2024-09-14T22:36:55.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2690999999999998e-05 2024-09-14T22:36:55.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:55.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560710+0000", 2024-09-14T22:36:55.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000103606 2024-09-14T22:36:55.658 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.658 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:55.658 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.658 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35569 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1570816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:55.658 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:36:55.658 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.054555521000001, 2024-09-14T22:36:55.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046110630999999999, 2024-09-14T22:36:55.658 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:55.658 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:55.658 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:55.658 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:55.659 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:55.659 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:55.659 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35569 2024-09-14T22:36:55.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.659 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:55.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:55.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:36:55.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:55.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:36:55.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:55.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:55.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:36:55.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.82e-07 2024-09-14T22:36:55.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:55.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514743+0000", 2024-09-14T22:36:55.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.269e-06 2024-09-14T22:36:55.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:55.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514750+0000", 2024-09-14T22:36:55.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7340000000000002e-06 2024-09-14T22:36:55.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:55.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514755+0000", 2024-09-14T22:36:55.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7260000000000002e-06 2024-09-14T22:36:55.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:55.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560731+0000", 2024-09-14T22:36:55.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045976377999999998 2024-09-14T22:36:55.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:55.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560777+0000", 2024-09-14T22:36:55.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5868999999999997e-05 2024-09-14T22:36:55.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:55.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:55.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:55.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560850+0000", 2024-09-14T22:36:55.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3172999999999999e-05 2024-09-14T22:36:55.662 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.662 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:55.662 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.662 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:55.662 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:55.662 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:55.664 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:36:55.665 INFO:teuthology.orchestra.run.smithi001.stdout:1927 write 0xe5b4878 thru 0xe5c3da8 (0xf531 bytes) 2024-09-14T22:36:55.665 INFO:teuthology.orchestra.run.smithi001.stdout:1928 punch from 0xcc00c93 to 0xcc04a9d, (0x3e0a bytes) 2024-09-14T22:36:55.665 INFO:teuthology.orchestra.run.smithi001.stdout:1929 writesame 0xc7956cc thru 0xc79c131 data_size 0x1(0x6a66 bytes) 2024-09-14T22:36:55.665 INFO:teuthology.orchestra.run.smithi001.stdout:1931 write 0x1d418bc thru 0x1d4efdc (0xd721 bytes) 2024-09-14T22:36:55.665 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:55.659+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:55.695 INFO:teuthology.orchestra.run.smithi001.stdout:1934 trunc from 0xe5c3da9 to 0xd92fe9 2024-09-14T22:36:55.746 INFO:teuthology.orchestra.run.smithi001.stdout:1935 read 0x336262 thru 0x336623 (0x3c2 bytes) 2024-09-14T22:36:55.759 INFO:teuthology.orchestra.run.smithi001.stdout:1937 write 0x147d771 thru 0x1484773 (0x7003 bytes) 2024-09-14T22:36:55.764 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:55.759+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:55.794 INFO:teuthology.orchestra.run.smithi001.stdout:1938 trunc from 0x1484774 to 0xcf3cf15 2024-09-14T22:36:55.807 INFO:teuthology.orchestra.run.smithi001.stdout:1941 trunc from 0xcf3cf15 to 0xc365638 2024-09-14T22:36:55.810 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:55.810 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:36:55.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:55.810 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4412.0:74259 3.13 e54/23 3:c9f7ee03:::rbd_header.113cce28790:head v 54'214, mlcod=54'212)", 2024-09-14T22:36:55.810 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:55.807483+0000", 2024-09-14T22:36:55.810 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.00032327499999999998, 2024-09-14T22:36:55.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000350152, 2024-09-14T22:36:55.810 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:55.810 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:55.810 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:55.810 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:55.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:55.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:55.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.807483+0000", 2024-09-14T22:36:55.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:55.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:55.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:55.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:55.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.807483+0000", 2024-09-14T22:36:55.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:55.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:55.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:55.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:55.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.807484+0000", 2024-09-14T22:36:55.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2399999999999996e-07 2024-09-14T22:36:55.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:55.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:55.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:55.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.807491+0000", 2024-09-14T22:36:55.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4529999999999996e-06 2024-09-14T22:36:55.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:55.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:55.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:55.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.807505+0000", 2024-09-14T22:36:55.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4212999999999999e-05 2024-09-14T22:36:55.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:55.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:55.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:55.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.807513+0000", 2024-09-14T22:36:55.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8120000000000008e-06 2024-09-14T22:36:55.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:55.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:55.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:55.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.807586+0000", 2024-09-14T22:36:55.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3406999999999997e-05 2024-09-14T22:36:55.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:55.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:55.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:55.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.807603+0000", 2024-09-14T22:36:55.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6719999999999999e-05 2024-09-14T22:36:55.813 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:55.813 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:55.814 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:55.814 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:55.814 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T22:36:55.814 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 1 2024-09-14T22:36:55.814 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:55.819 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:55.819 INFO:teuthology.orchestra.run.smithi001.stdout:1944 write 0x627279f thru 0x6272e00 (0x662 bytes) 2024-09-14T22:36:55.825 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:36:55.825 INFO:teuthology.orchestra.run.smithi001.stdout:1945 write 0x100def7 thru 0x1017321 (0x942b bytes) 2024-09-14T22:36:55.829 INFO:teuthology.orchestra.run.smithi001.stdout:1948 write 0x775dafb thru 0x7769947 (0xbe4d bytes) 2024-09-14T22:36:55.837 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:55.837 INFO:teuthology.orchestra.run.smithi001.stdout:1950 read 0x7487c53 thru 0x7495fcf (0xe37d bytes) 2024-09-14T22:36:55.838 INFO:teuthology.orchestra.run.smithi001.stdout:1951 read 0x5db81a9 thru 0x5dc1798 (0x95f0 bytes) 2024-09-14T22:36:55.838 INFO:teuthology.orchestra.run.smithi001.stdout:1952 write 0x1699102 thru 0x1699823 (0x722 bytes) 2024-09-14T22:36:55.841 INFO:teuthology.orchestra.run.smithi001.stdout:1953 writesame 0xb1e8154 thru 0xb1f3a09 data_size 0x6(0xb8b6 bytes) 2024-09-14T22:36:55.855 INFO:teuthology.orchestra.run.smithi001.stdout:1954 punch from 0x83668a7 to 0x83726af, (0xbe08 bytes) 2024-09-14T22:36:55.855 INFO:teuthology.orchestra.run.smithi001.stdout:1955 punch from 0x16249c2 to 0x1634679, (0xfcb7 bytes) 2024-09-14T22:36:55.855 INFO:teuthology.orchestra.run.smithi001.stdout:1956 write 0xa137481 thru 0xa138226 (0xda6 bytes) 2024-09-14T22:36:55.862 INFO:teuthology.orchestra.run.smithi001.stdout:1957 write 0x6c27edc thru 0x6c283c9 (0x4ee bytes) 2024-09-14T22:36:55.865 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:55.859+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:55.883 INFO:teuthology.orchestra.run.smithi001.stdout:1959 write 0xcc7e023 thru 0xcc85342 (0x7320 bytes) 2024-09-14T22:36:55.889 INFO:teuthology.orchestra.run.smithi001.stdout:1961 clone 14 order 18 su 16384 sc 9 2024-09-14T22:36:55.966 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:55.963+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:55.996 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:55.996 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:55.996 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:36:55.996 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:36:55.996 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:56.010 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:36:56.034 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T22:36:56.067 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:56.063+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:56.142 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:56.142 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:36:56.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:36:56.143 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:36:56.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.143 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6376 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8230912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:56.143 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932578+0000", 2024-09-14T22:36:56.143 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.206965038, 2024-09-14T22:36:56.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051558771000000003, 2024-09-14T22:36:56.143 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.143 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.143 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.143 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.143 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.143 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.143 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6376 2024-09-14T22:36:56.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.144 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.144 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932578+0000", 2024-09-14T22:36:56.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.144 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932578+0000", 2024-09-14T22:36:56.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.144 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932579+0000", 2024-09-14T22:36:56.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8599999999999996e-07 2024-09-14T22:36:56.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932597+0000", 2024-09-14T22:36:56.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7771e-05 2024-09-14T22:36:56.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932618+0000", 2024-09-14T22:36:56.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1026000000000002e-05 2024-09-14T22:36:56.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932635+0000", 2024-09-14T22:36:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.702e-05 2024-09-14T22:36:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.983951+0000", 2024-09-14T22:36:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051316639999999997 2024-09-14T22:36:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984031+0000", 2024-09-14T22:36:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0085000000000003e-05 2024-09-14T22:36:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984137+0000", 2024-09-14T22:36:56.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105243 2024-09-14T22:36:56.147 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.147 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.147 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.147 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6377 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8239104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:56.147 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932688+0000", 2024-09-14T22:36:56.147 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.206855007000001, 2024-09-14T22:36:56.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051650696000000003, 2024-09-14T22:36:56.147 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.147 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.147 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.147 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.148 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.148 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.148 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6377 2024-09-14T22:36:56.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.148 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.148 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.148 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932688+0000", 2024-09-14T22:36:56.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.148 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.148 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932688+0000", 2024-09-14T22:36:56.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.149 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932689+0000", 2024-09-14T22:36:56.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4309999999999999e-06 2024-09-14T22:36:56.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.149 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932695+0000", 2024-09-14T22:36:56.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8889999999999999e-06 2024-09-14T22:36:56.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.149 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932713+0000", 2024-09-14T22:36:56.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7416999999999999e-05 2024-09-14T22:36:56.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932726+0000", 2024-09-14T22:36:56.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3145e-05 2024-09-14T22:36:56.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984181+0000", 2024-09-14T22:36:56.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051455630000000002 2024-09-14T22:36:56.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984247+0000", 2024-09-14T22:36:56.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5350000000000003e-05 2024-09-14T22:36:56.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984339+0000", 2024-09-14T22:36:56.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1834000000000002e-05 2024-09-14T22:36:56.151 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.151 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.151 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.151 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6378 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8247296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:56.151 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932743+0000", 2024-09-14T22:36:56.151 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.206799773, 2024-09-14T22:36:56.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051819373000000002, 2024-09-14T22:36:56.151 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.151 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.151 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.151 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.152 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.152 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.152 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6378 2024-09-14T22:36:56.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.152 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.152 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.152 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932743+0000", 2024-09-14T22:36:56.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.152 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.152 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932743+0000", 2024-09-14T22:36:56.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932745+0000", 2024-09-14T22:36:56.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.637e-06 2024-09-14T22:36:56.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932750+0000", 2024-09-14T22:36:56.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7859999999999996e-06 2024-09-14T22:36:56.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932767+0000", 2024-09-14T22:36:56.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7621999999999998e-05 2024-09-14T22:36:56.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932781+0000", 2024-09-14T22:36:56.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3403e-05 2024-09-14T22:36:56.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984383+0000", 2024-09-14T22:36:56.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051602249000000003 2024-09-14T22:36:56.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984461+0000", 2024-09-14T22:36:56.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8467000000000006e-05 2024-09-14T22:36:56.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984562+0000", 2024-09-14T22:36:56.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101209 2024-09-14T22:36:56.156 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.156 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.156 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.157 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6379 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8255488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:56.157 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932796+0000", 2024-09-14T22:36:56.157 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.206746381999999, 2024-09-14T22:36:56.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051961245000000003, 2024-09-14T22:36:56.157 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.157 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.157 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.157 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.157 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.157 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.157 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6379 2024-09-14T22:36:56.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.157 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932796+0000", 2024-09-14T22:36:56.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932796+0000", 2024-09-14T22:36:56.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932797+0000", 2024-09-14T22:36:56.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9700000000000005e-07 2024-09-14T22:36:56.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932823+0000", 2024-09-14T22:36:56.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5273999999999999e-05 2024-09-14T22:36:56.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932840+0000", 2024-09-14T22:36:56.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7774e-05 2024-09-14T22:36:56.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932862+0000", 2024-09-14T22:36:56.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1957e-05 2024-09-14T22:36:56.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984603+0000", 2024-09-14T22:36:56.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051740543 2024-09-14T22:36:56.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984669+0000", 2024-09-14T22:36:56.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.567e-05 2024-09-14T22:36:56.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984758+0000", 2024-09-14T22:36:56.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9129999999999995e-05 2024-09-14T22:36:56.161 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.161 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.161 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.161 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6380 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8263680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:56.161 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932879+0000", 2024-09-14T22:36:56.161 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.206664302999997, 2024-09-14T22:36:56.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052102860000000001, 2024-09-14T22:36:56.161 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.161 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.161 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.161 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.162 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.162 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.162 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6380 2024-09-14T22:36:56.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.162 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932879+0000", 2024-09-14T22:36:56.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932879+0000", 2024-09-14T22:36:56.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932880+0000", 2024-09-14T22:36:56.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.054e-06 2024-09-14T22:36:56.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932885+0000", 2024-09-14T22:36:56.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1290000000000004e-06 2024-09-14T22:36:56.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932901+0000", 2024-09-14T22:36:56.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5908000000000001e-05 2024-09-14T22:36:56.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932913+0000", 2024-09-14T22:36:56.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2408999999999999e-05 2024-09-14T22:36:56.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984800+0000", 2024-09-14T22:36:56.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051886671000000002 2024-09-14T22:36:56.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984879+0000", 2024-09-14T22:36:56.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9200000000000001e-05 2024-09-14T22:36:56.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984981+0000", 2024-09-14T22:36:56.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102489 2024-09-14T22:36:56.165 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.165 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.165 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.165 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6381 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8271872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:56.165 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932929+0000", 2024-09-14T22:36:56.165 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.206613972, 2024-09-14T22:36:56.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052251484000000001, 2024-09-14T22:36:56.165 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.166 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.166 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.166 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.166 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.166 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.166 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6381 2024-09-14T22:36:56.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.166 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932929+0000", 2024-09-14T22:36:56.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932929+0000", 2024-09-14T22:36:56.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932930+0000", 2024-09-14T22:36:56.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3399999999999998e-07 2024-09-14T22:36:56.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932934+0000", 2024-09-14T22:36:56.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5719999999999996e-06 2024-09-14T22:36:56.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932961+0000", 2024-09-14T22:36:56.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6437000000000001e-05 2024-09-14T22:36:56.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932973+0000", 2024-09-14T22:36:56.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2097e-05 2024-09-14T22:36:56.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985021+0000", 2024-09-14T22:36:56.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052047955999999999 2024-09-14T22:36:56.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985089+0000", 2024-09-14T22:36:56.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7725999999999995e-05 2024-09-14T22:36:56.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985180+0000", 2024-09-14T22:36:56.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1861999999999996e-05 2024-09-14T22:36:56.169 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.169 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.169 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.170 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6382 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8280064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:56.170 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933013+0000", 2024-09-14T22:36:56.170 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.206529809999999, 2024-09-14T22:36:56.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052391118, 2024-09-14T22:36:56.170 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.170 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.170 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.170 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.170 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.170 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.170 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6382 2024-09-14T22:36:56.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.170 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933013+0000", 2024-09-14T22:36:56.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933013+0000", 2024-09-14T22:36:56.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933014+0000", 2024-09-14T22:36:56.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.375e-06 2024-09-14T22:36:56.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933020+0000", 2024-09-14T22:36:56.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0220000000000001e-06 2024-09-14T22:36:56.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933038+0000", 2024-09-14T22:36:56.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7694999999999999e-05 2024-09-14T22:36:56.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933051+0000", 2024-09-14T22:36:56.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2721999999999999e-05 2024-09-14T22:36:56.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985224+0000", 2024-09-14T22:36:56.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052172757 2024-09-14T22:36:56.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985303+0000", 2024-09-14T22:36:56.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9426999999999997e-05 2024-09-14T22:36:56.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985404+0000", 2024-09-14T22:36:56.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010111999999999999 2024-09-14T22:36:56.173 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.173 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.173 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.174 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6383 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8288256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:56.174 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933082+0000", 2024-09-14T22:36:56.174 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.206460829999997, 2024-09-14T22:36:56.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052525761999999997, 2024-09-14T22:36:56.174 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.174 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.174 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.174 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.174 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.174 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.174 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6383 2024-09-14T22:36:56.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.174 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933082+0000", 2024-09-14T22:36:56.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933082+0000", 2024-09-14T22:36:56.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933084+0000", 2024-09-14T22:36:56.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5659999999999999e-06 2024-09-14T22:36:56.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933089+0000", 2024-09-14T22:36:56.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.271e-06 2024-09-14T22:36:56.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933105+0000", 2024-09-14T22:36:56.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5685999999999999e-05 2024-09-14T22:36:56.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933118+0000", 2024-09-14T22:36:56.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3188e-05 2024-09-14T22:36:56.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985448+0000", 2024-09-14T22:36:56.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052330176999999999 2024-09-14T22:36:56.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985514+0000", 2024-09-14T22:36:56.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6029000000000001e-05 2024-09-14T22:36:56.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985608+0000", 2024-09-14T22:36:56.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3844999999999993e-05 2024-09-14T22:36:56.178 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.178 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.178 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.178 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6384 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8296448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:56.178 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933134+0000", 2024-09-14T22:36:56.178 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.206408471000003, 2024-09-14T22:36:56.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052688835000000003, 2024-09-14T22:36:56.178 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.178 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.178 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.178 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.178 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.179 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.179 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6384 2024-09-14T22:36:56.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.179 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933134+0000", 2024-09-14T22:36:56.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933134+0000", 2024-09-14T22:36:56.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933135+0000", 2024-09-14T22:36:56.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.78e-07 2024-09-14T22:36:56.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933141+0000", 2024-09-14T22:36:56.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1540000000000001e-06 2024-09-14T22:36:56.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933156+0000", 2024-09-14T22:36:56.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5488000000000001e-05 2024-09-14T22:36:56.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933169+0000", 2024-09-14T22:36:56.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2649e-05 2024-09-14T22:36:56.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985650+0000", 2024-09-14T22:36:56.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052481112000000003 2024-09-14T22:36:56.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985724+0000", 2024-09-14T22:36:56.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3953000000000002e-05 2024-09-14T22:36:56.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985823+0000", 2024-09-14T22:36:56.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9500999999999995e-05 2024-09-14T22:36:56.182 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.182 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.182 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.182 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6385 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8304640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:56.182 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933185+0000", 2024-09-14T22:36:56.182 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.206357767999997, 2024-09-14T22:36:56.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052892844000000001, 2024-09-14T22:36:56.182 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.182 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.183 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.183 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.183 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.183 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.183 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6385 2024-09-14T22:36:56.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.183 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933185+0000", 2024-09-14T22:36:56.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933185+0000", 2024-09-14T22:36:56.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933186+0000", 2024-09-14T22:36:56.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0139999999999999e-06 2024-09-14T22:36:56.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933191+0000", 2024-09-14T22:36:56.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7230000000000004e-06 2024-09-14T22:36:56.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933207+0000", 2024-09-14T22:36:56.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.658e-05 2024-09-14T22:36:56.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933220+0000", 2024-09-14T22:36:56.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2583e-05 2024-09-14T22:36:56.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985880+0000", 2024-09-14T22:36:56.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052660140000000001 2024-09-14T22:36:56.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985965+0000", 2024-09-14T22:36:56.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4472e-05 2024-09-14T22:36:56.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986078+0000", 2024-09-14T22:36:56.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011333200000000001 2024-09-14T22:36:56.186 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.186 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.186 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.186 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6386 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8312832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:56.186 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933236+0000", 2024-09-14T22:36:56.186 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.206306873999999, 2024-09-14T22:36:56.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053075342999999997, 2024-09-14T22:36:56.187 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.187 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.187 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.187 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.187 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.187 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.187 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6386 2024-09-14T22:36:56.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.187 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933236+0000", 2024-09-14T22:36:56.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933236+0000", 2024-09-14T22:36:56.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933237+0000", 2024-09-14T22:36:56.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6500000000000008e-07 2024-09-14T22:36:56.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933242+0000", 2024-09-14T22:36:56.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.938e-06 2024-09-14T22:36:56.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933258+0000", 2024-09-14T22:36:56.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6334e-05 2024-09-14T22:36:56.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933270+0000", 2024-09-14T22:36:56.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1973999999999999e-05 2024-09-14T22:36:56.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986123+0000", 2024-09-14T22:36:56.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052852769000000001 2024-09-14T22:36:56.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986204+0000", 2024-09-14T22:36:56.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1458999999999994e-05 2024-09-14T22:36:56.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986311+0000", 2024-09-14T22:36:56.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010690400000000001 2024-09-14T22:36:56.190 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.190 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.190 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.191 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6387 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8321024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:56.191 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933335+0000", 2024-09-14T22:36:56.191 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.206207603999999, 2024-09-14T22:36:56.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053182363000000003, 2024-09-14T22:36:56.191 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.191 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.191 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.191 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.191 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.191 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.191 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6387 2024-09-14T22:36:56.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.191 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933335+0000", 2024-09-14T22:36:56.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933335+0000", 2024-09-14T22:36:56.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933337+0000", 2024-09-14T22:36:56.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3829999999999999e-06 2024-09-14T22:36:56.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933343+0000", 2024-09-14T22:36:56.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1489999999999999e-06 2024-09-14T22:36:56.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933361+0000", 2024-09-14T22:36:56.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8105e-05 2024-09-14T22:36:56.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933374+0000", 2024-09-14T22:36:56.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.33e-05 2024-09-14T22:36:56.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986355+0000", 2024-09-14T22:36:56.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052981053 2024-09-14T22:36:56.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986424+0000", 2024-09-14T22:36:56.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8864999999999999e-05 2024-09-14T22:36:56.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986518+0000", 2024-09-14T22:36:56.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3508e-05 2024-09-14T22:36:56.195 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.195 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.195 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.195 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6388 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8329216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:56.195 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933390+0000", 2024-09-14T22:36:56.195 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.206153018000002, 2024-09-14T22:36:56.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053371605000000003, 2024-09-14T22:36:56.195 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.195 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.195 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.195 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.195 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.196 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.196 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6388 2024-09-14T22:36:56.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.196 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933390+0000", 2024-09-14T22:36:56.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933390+0000", 2024-09-14T22:36:56.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933391+0000", 2024-09-14T22:36:56.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.443e-06 2024-09-14T22:36:56.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933396+0000", 2024-09-14T22:36:56.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1680000000000003e-06 2024-09-14T22:36:56.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933412+0000", 2024-09-14T22:36:56.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5302999999999999e-05 2024-09-14T22:36:56.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933439+0000", 2024-09-14T22:36:56.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6747999999999999e-05 2024-09-14T22:36:56.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986562+0000", 2024-09-14T22:36:56.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053123161000000002 2024-09-14T22:36:56.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986641+0000", 2024-09-14T22:36:56.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9086999999999997e-05 2024-09-14T22:36:56.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986761+0000", 2024-09-14T22:36:56.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120695 2024-09-14T22:36:56.199 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.199 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.199 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.199 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6389 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8337408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:56.199 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933456+0000", 2024-09-14T22:36:56.199 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.206087167, 2024-09-14T22:36:56.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053514182, 2024-09-14T22:36:56.199 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.199 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.199 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.200 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.200 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.200 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.200 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6389 2024-09-14T22:36:56.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.200 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933456+0000", 2024-09-14T22:36:56.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933456+0000", 2024-09-14T22:36:56.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933457+0000", 2024-09-14T22:36:56.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0020000000000001e-06 2024-09-14T22:36:56.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933462+0000", 2024-09-14T22:36:56.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8400000000000002e-06 2024-09-14T22:36:56.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933477+0000", 2024-09-14T22:36:56.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.592e-05 2024-09-14T22:36:56.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933489+0000", 2024-09-14T22:36:56.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1955e-05 2024-09-14T22:36:56.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986798+0000", 2024-09-14T22:36:56.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053308263000000002 2024-09-14T22:36:56.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986871+0000", 2024-09-14T22:36:56.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3518999999999999e-05 2024-09-14T22:36:56.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986970+0000", 2024-09-14T22:36:56.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8683000000000004e-05 2024-09-14T22:36:56.203 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.203 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.203 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.203 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6390 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8345600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:56.203 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933504+0000", 2024-09-14T22:36:56.203 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.206038382000003, 2024-09-14T22:36:56.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053719579000000003, 2024-09-14T22:36:56.203 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.204 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.204 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.204 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.204 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.204 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.204 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6390 2024-09-14T22:36:56.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.204 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933504+0000", 2024-09-14T22:36:56.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933504+0000", 2024-09-14T22:36:56.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933505+0000", 2024-09-14T22:36:56.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8299999999999995e-07 2024-09-14T22:36:56.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933520+0000", 2024-09-14T22:36:56.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4948e-05 2024-09-14T22:36:56.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933538+0000", 2024-09-14T22:36:56.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7286e-05 2024-09-14T22:36:56.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933550+0000", 2024-09-14T22:36:56.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2223e-05 2024-09-14T22:36:56.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987028+0000", 2024-09-14T22:36:56.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053478500999999998 2024-09-14T22:36:56.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987111+0000", 2024-09-14T22:36:56.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3158999999999994e-05 2024-09-14T22:36:56.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987224+0000", 2024-09-14T22:36:56.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011257899999999999 2024-09-14T22:36:56.207 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.207 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.207 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.208 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6391 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8353792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:56.208 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933576+0000", 2024-09-14T22:36:56.208 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.205966936999999, 2024-09-14T22:36:56.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053858196999999997, 2024-09-14T22:36:56.208 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.208 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.208 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.208 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.208 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.208 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.208 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6391 2024-09-14T22:36:56.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.208 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933576+0000", 2024-09-14T22:36:56.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933576+0000", 2024-09-14T22:36:56.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933578+0000", 2024-09-14T22:36:56.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9539999999999998e-06 2024-09-14T22:36:56.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933584+0000", 2024-09-14T22:36:56.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9030000000000001e-06 2024-09-14T22:36:56.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933601+0000", 2024-09-14T22:36:56.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7703e-05 2024-09-14T22:36:56.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933614+0000", 2024-09-14T22:36:56.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2330999999999999e-05 2024-09-14T22:36:56.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987275+0000", 2024-09-14T22:36:56.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053661470000000003 2024-09-14T22:36:56.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987342+0000", 2024-09-14T22:36:56.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6266000000000006e-05 2024-09-14T22:36:56.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987434+0000", 2024-09-14T22:36:56.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2570000000000003e-05 2024-09-14T22:36:56.211 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.212 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.212 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.212 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6392 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8361984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:56.212 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933648+0000", 2024-09-14T22:36:56.212 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.205894549, 2024-09-14T22:36:56.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054032430999999999, 2024-09-14T22:36:56.212 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.212 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.212 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.212 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.212 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.213 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.213 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6392 2024-09-14T22:36:56.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.213 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933648+0000", 2024-09-14T22:36:56.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933648+0000", 2024-09-14T22:36:56.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933650+0000", 2024-09-14T22:36:56.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3969999999999999e-06 2024-09-14T22:36:56.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933655+0000", 2024-09-14T22:36:56.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7690000000000002e-06 2024-09-14T22:36:56.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933672+0000", 2024-09-14T22:36:56.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6359999999999999e-05 2024-09-14T22:36:56.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933696+0000", 2024-09-14T22:36:56.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4630999999999998e-05 2024-09-14T22:36:56.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987490+0000", 2024-09-14T22:36:56.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053793256999999997 2024-09-14T22:36:56.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987572+0000", 2024-09-14T22:36:56.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1897999999999994e-05 2024-09-14T22:36:56.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987681+0000", 2024-09-14T22:36:56.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109119 2024-09-14T22:36:56.216 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.216 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.217 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.217 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6393 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8370176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:56.217 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933713+0000", 2024-09-14T22:36:56.217 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.205829594999997, 2024-09-14T22:36:56.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054194111000000003, 2024-09-14T22:36:56.217 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.217 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.217 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.217 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.218 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.218 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.218 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6393 2024-09-14T22:36:56.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.218 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933713+0000", 2024-09-14T22:36:56.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933713+0000", 2024-09-14T22:36:56.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933714+0000", 2024-09-14T22:36:56.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2109999999999999e-06 2024-09-14T22:36:56.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933721+0000", 2024-09-14T22:36:56.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2489999999999998e-06 2024-09-14T22:36:56.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933751+0000", 2024-09-14T22:36:56.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0239999999999998e-05 2024-09-14T22:36:56.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933764+0000", 2024-09-14T22:36:56.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.322e-05 2024-09-14T22:36:56.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987737+0000", 2024-09-14T22:36:56.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053972539 2024-09-14T22:36:56.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987813+0000", 2024-09-14T22:36:56.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6635000000000001e-05 2024-09-14T22:36:56.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987907+0000", 2024-09-14T22:36:56.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4017000000000005e-05 2024-09-14T22:36:56.221 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.221 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.221 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.221 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6394 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8378368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:56.221 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933781+0000", 2024-09-14T22:36:56.222 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.205761836000001, 2024-09-14T22:36:56.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054378276000000003, 2024-09-14T22:36:56.222 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.222 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.222 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.222 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.222 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.222 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.222 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6394 2024-09-14T22:36:56.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.222 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933781+0000", 2024-09-14T22:36:56.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933781+0000", 2024-09-14T22:36:56.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933782+0000", 2024-09-14T22:36:56.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9199999999999999e-07 2024-09-14T22:36:56.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933788+0000", 2024-09-14T22:36:56.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9340000000000003e-06 2024-09-14T22:36:56.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933803+0000", 2024-09-14T22:36:56.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5362999999999998e-05 2024-09-14T22:36:56.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933816+0000", 2024-09-14T22:36:56.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2456999999999999e-05 2024-09-14T22:36:56.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987961+0000", 2024-09-14T22:36:56.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054145581999999998 2024-09-14T22:36:56.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988042+0000", 2024-09-14T22:36:56.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1075000000000006e-05 2024-09-14T22:36:56.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988159+0000", 2024-09-14T22:36:56.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116873 2024-09-14T22:36:56.226 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.226 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.226 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.226 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6395 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8386560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:36:56.226 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:36:56.226 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.205710942000003, 2024-09-14T22:36:56.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054534379000000001, 2024-09-14T22:36:56.226 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.226 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.226 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.226 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.226 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.226 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.226 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6395 2024-09-14T22:36:56.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.227 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:36:56.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:36:56.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933833+0000", 2024-09-14T22:36:56.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3859999999999999e-06 2024-09-14T22:36:56.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933838+0000", 2024-09-14T22:36:56.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8709999999999996e-06 2024-09-14T22:36:56.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933854+0000", 2024-09-14T22:36:56.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6189e-05 2024-09-14T22:36:56.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933871+0000", 2024-09-14T22:36:56.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.626e-05 2024-09-14T22:36:56.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988198+0000", 2024-09-14T22:36:56.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054327581999999999 2024-09-14T22:36:56.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988274+0000", 2024-09-14T22:36:56.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5759000000000004e-05 2024-09-14T22:36:56.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988366+0000", 2024-09-14T22:36:56.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2331999999999996e-05 2024-09-14T22:36:56.229 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.230 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.230 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.230 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.230 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.230 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:56.230 INFO:teuthology.orchestra.run.smithi001.stdout:flattening image image_client.0-clone13 2024-09-14T22:36:56.231 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:36:56.232 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:56.227+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:56.332 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:56.327+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:56.391 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-14T22:36:56.412 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:56.412 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:56.412 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:36:56.412 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:56.414 INFO:teuthology.orchestra.run.smithi001.stdout:1963 trunc from 0xcc85343 to 0xa00988f 2024-09-14T22:36:56.427 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:36:56.455 INFO:teuthology.orchestra.run.smithi001.stdout:1966 write 0xee7a49 thru 0xef6cb2 (0xf26a bytes) 2024-09-14T22:36:56.469 INFO:teuthology.orchestra.run.smithi001.stdout:1967 read 0x4c31b34 thru 0x4c3b043 (0x9510 bytes) 2024-09-14T22:36:56.469 INFO:teuthology.orchestra.run.smithi001.stdout:1968 clone 15 order 19 su 4096 sc 10 2024-09-14T22:36:56.471 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:56.467+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:56.533 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:56.527+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:56.587 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:56.587 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:56.587 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:36:56.587 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:36:56.587 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:56.601 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:36:56.643 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:56.639+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:56.730 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:36:56.735 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:56.731+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:56.767 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:56.767 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:36:56.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:36:56.767 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:36:56.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.768 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19590 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7272448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:56.768 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446391+0000", 2024-09-14T22:36:56.768 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.316104011, 2024-09-14T22:36:56.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027116533000000002, 2024-09-14T22:36:56.768 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.768 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.768 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.768 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.768 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.768 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.768 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19590 2024-09-14T22:36:56.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.768 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446391+0000", 2024-09-14T22:36:56.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446391+0000", 2024-09-14T22:36:56.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446392+0000", 2024-09-14T22:36:56.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4299999999999999e-07 2024-09-14T22:36:56.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446393+0000", 2024-09-14T22:36:56.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8059999999999999e-06 2024-09-14T22:36:56.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446399+0000", 2024-09-14T22:36:56.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6740000000000002e-06 2024-09-14T22:36:56.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446404+0000", 2024-09-14T22:36:56.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6240000000000001e-06 2024-09-14T22:36:56.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473382+0000", 2024-09-14T22:36:56.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026978716999999999 2024-09-14T22:36:56.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473446+0000", 2024-09-14T22:36:56.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3971000000000001e-05 2024-09-14T22:36:56.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473508+0000", 2024-09-14T22:36:56.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1397999999999997e-05 2024-09-14T22:36:56.772 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.772 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.772 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.773 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19591 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7301120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:56.773 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446415+0000", 2024-09-14T22:36:56.773 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.316080657000001, 2024-09-14T22:36:56.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027255773, 2024-09-14T22:36:56.773 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.773 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.773 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.773 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.773 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.773 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.773 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19591 2024-09-14T22:36:56.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.774 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446415+0000", 2024-09-14T22:36:56.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446415+0000", 2024-09-14T22:36:56.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446415+0000", 2024-09-14T22:36:56.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8800000000000002e-07 2024-09-14T22:36:56.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446417+0000", 2024-09-14T22:36:56.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7990000000000001e-06 2024-09-14T22:36:56.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446423+0000", 2024-09-14T22:36:56.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.066e-06 2024-09-14T22:36:56.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446428+0000", 2024-09-14T22:36:56.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5889999999999996e-06 2024-09-14T22:36:56.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.776 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473535+0000", 2024-09-14T22:36:56.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027107220000000001 2024-09-14T22:36:56.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.776 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473602+0000", 2024-09-14T22:36:56.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6482999999999993e-05 2024-09-14T22:36:56.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473670+0000", 2024-09-14T22:36:56.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8928000000000001e-05 2024-09-14T22:36:56.777 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.777 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.777 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.777 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:19592 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7329792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:56.777 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.446433+0000", 2024-09-14T22:36:56.777 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.316061995999998, 2024-09-14T22:36:56.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027394960999999999, 2024-09-14T22:36:56.777 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.778 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.778 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.778 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.778 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.778 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.778 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19592 2024-09-14T22:36:56.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.778 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446433+0000", 2024-09-14T22:36:56.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446433+0000", 2024-09-14T22:36:56.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446434+0000", 2024-09-14T22:36:56.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3600000000000004e-07 2024-09-14T22:36:56.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446436+0000", 2024-09-14T22:36:56.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.796e-06 2024-09-14T22:36:56.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446441+0000", 2024-09-14T22:36:56.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7209999999999999e-06 2024-09-14T22:36:56.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.446450+0000", 2024-09-14T22:36:56.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0780000000000002e-06 2024-09-14T22:36:56.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473705+0000", 2024-09-14T22:36:56.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027254246999999999 2024-09-14T22:36:56.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473769+0000", 2024-09-14T22:36:56.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4101000000000002e-05 2024-09-14T22:36:56.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.473828+0000", 2024-09-14T22:36:56.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9481999999999997e-05 2024-09-14T22:36:56.781 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.782 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.782 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.782 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20101 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7247872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:56.782 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462195+0000", 2024-09-14T22:36:56.782 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.300299989999999, 2024-09-14T22:36:56.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027205994000000001, 2024-09-14T22:36:56.782 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.782 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.782 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.782 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.782 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.782 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.782 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20101 2024-09-14T22:36:56.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.783 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462195+0000", 2024-09-14T22:36:56.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462195+0000", 2024-09-14T22:36:56.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462196+0000", 2024-09-14T22:36:56.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8000000000000001e-07 2024-09-14T22:36:56.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462198+0000", 2024-09-14T22:36:56.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0439999999999998e-06 2024-09-14T22:36:56.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462205+0000", 2024-09-14T22:36:56.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3359999999999997e-06 2024-09-14T22:36:56.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462210+0000", 2024-09-14T22:36:56.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0649999999999998e-06 2024-09-14T22:36:56.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489279+0000", 2024-09-14T22:36:56.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027069123 2024-09-14T22:36:56.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489342+0000", 2024-09-14T22:36:56.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2759e-05 2024-09-14T22:36:56.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489401+0000", 2024-09-14T22:36:56.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9286999999999997e-05 2024-09-14T22:36:56.786 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.786 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.786 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.786 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20102 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7276544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:56.786 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462216+0000", 2024-09-14T22:36:56.786 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.300279042, 2024-09-14T22:36:56.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027352785000000001, 2024-09-14T22:36:56.787 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.787 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.787 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.787 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.787 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.787 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.787 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20102 2024-09-14T22:36:56.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.787 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462216+0000", 2024-09-14T22:36:56.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462216+0000", 2024-09-14T22:36:56.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462217+0000", 2024-09-14T22:36:56.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9100000000000003e-07 2024-09-14T22:36:56.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462219+0000", 2024-09-14T22:36:56.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9740000000000001e-06 2024-09-14T22:36:56.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462235+0000", 2024-09-14T22:36:56.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6050999999999999e-05 2024-09-14T22:36:56.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462247+0000", 2024-09-14T22:36:56.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2245999999999999e-05 2024-09-14T22:36:56.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489431+0000", 2024-09-14T22:36:56.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027183624999999999 2024-09-14T22:36:56.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489504+0000", 2024-09-14T22:36:56.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3498999999999995e-05 2024-09-14T22:36:56.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489569+0000", 2024-09-14T22:36:56.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4698999999999998e-05 2024-09-14T22:36:56.790 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.790 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.791 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.791 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20103 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7305216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:56.791 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462256+0000", 2024-09-14T22:36:56.791 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.300239835999999, 2024-09-14T22:36:56.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027473543, 2024-09-14T22:36:56.791 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.791 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.791 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.791 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.791 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.791 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.792 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20103 2024-09-14T22:36:56.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.792 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462256+0000", 2024-09-14T22:36:56.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462256+0000", 2024-09-14T22:36:56.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462256+0000", 2024-09-14T22:36:56.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1399999999999997e-07 2024-09-14T22:36:56.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462259+0000", 2024-09-14T22:36:56.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.694e-06 2024-09-14T22:36:56.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462266+0000", 2024-09-14T22:36:56.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8940000000000004e-06 2024-09-14T22:36:56.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462271+0000", 2024-09-14T22:36:56.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0459999999999996e-06 2024-09-14T22:36:56.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489601+0000", 2024-09-14T22:36:56.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027330027 2024-09-14T22:36:56.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489663+0000", 2024-09-14T22:36:56.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2807999999999999e-05 2024-09-14T22:36:56.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489729+0000", 2024-09-14T22:36:56.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5560000000000002e-05 2024-09-14T22:36:56.795 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.795 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.796 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.796 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20104 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7333888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:56.796 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.462277+0000", 2024-09-14T22:36:56.796 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.300218300000001, 2024-09-14T22:36:56.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027652614999999998, 2024-09-14T22:36:56.796 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.796 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.796 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.796 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.797 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.797 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.797 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20104 2024-09-14T22:36:56.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.797 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462277+0000", 2024-09-14T22:36:56.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462277+0000", 2024-09-14T22:36:56.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462277+0000", 2024-09-14T22:36:56.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2800000000000002e-07 2024-09-14T22:36:56.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462280+0000", 2024-09-14T22:36:56.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2570000000000002e-06 2024-09-14T22:36:56.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462286+0000", 2024-09-14T22:36:56.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2199999999999997e-06 2024-09-14T22:36:56.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.462291+0000", 2024-09-14T22:36:56.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5689999999999998e-06 2024-09-14T22:36:56.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489760+0000", 2024-09-14T22:36:56.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027469345999999999 2024-09-14T22:36:56.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489852+0000", 2024-09-14T22:36:56.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1941999999999995e-05 2024-09-14T22:36:56.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.489930+0000", 2024-09-14T22:36:56.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7853000000000002e-05 2024-09-14T22:36:56.800 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.800 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.801 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.801 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20869 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7168000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:56.801 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483633+0000", 2024-09-14T22:36:56.801 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.278861944999999, 2024-09-14T22:36:56.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028181490999999999, 2024-09-14T22:36:56.801 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.801 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.801 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.801 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.801 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.801 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.802 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20869 2024-09-14T22:36:56.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.802 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483633+0000", 2024-09-14T22:36:56.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483633+0000", 2024-09-14T22:36:56.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483634+0000", 2024-09-14T22:36:56.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2599999999999998e-07 2024-09-14T22:36:56.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483636+0000", 2024-09-14T22:36:56.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1679999999999998e-06 2024-09-14T22:36:56.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483642+0000", 2024-09-14T22:36:56.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2149999999999998e-06 2024-09-14T22:36:56.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483647+0000", 2024-09-14T22:36:56.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7679999999999999e-06 2024-09-14T22:36:56.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.510694+0000", 2024-09-14T22:36:56.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027047073000000001 2024-09-14T22:36:56.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.510771+0000", 2024-09-14T22:36:56.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6814e-05 2024-09-14T22:36:56.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511815+0000", 2024-09-14T22:36:56.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0010440270000000001 2024-09-14T22:36:56.805 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.805 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.805 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.805 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20870 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7196672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:56.805 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483652+0000", 2024-09-14T22:36:56.805 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.278842943000001, 2024-09-14T22:36:56.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028301248000000001, 2024-09-14T22:36:56.805 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.806 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.806 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.806 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.806 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.806 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.806 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20870 2024-09-14T22:36:56.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.806 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483652+0000", 2024-09-14T22:36:56.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483652+0000", 2024-09-14T22:36:56.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483653+0000", 2024-09-14T22:36:56.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3099999999999997e-07 2024-09-14T22:36:56.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483655+0000", 2024-09-14T22:36:56.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.877e-06 2024-09-14T22:36:56.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483661+0000", 2024-09-14T22:36:56.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6049999999999997e-06 2024-09-14T22:36:56.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483666+0000", 2024-09-14T22:36:56.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3410000000000002e-06 2024-09-14T22:36:56.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511854+0000", 2024-09-14T22:36:56.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028187773999999999 2024-09-14T22:36:56.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511906+0000", 2024-09-14T22:36:56.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2012000000000003e-05 2024-09-14T22:36:56.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511954+0000", 2024-09-14T22:36:56.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8007999999999998e-05 2024-09-14T22:36:56.809 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.809 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.809 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.810 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20871 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7225344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:56.810 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483671+0000", 2024-09-14T22:36:56.810 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.278824019999998, 2024-09-14T22:36:56.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028405392000000002, 2024-09-14T22:36:56.810 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.810 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.810 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.810 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.810 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.810 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.810 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20871 2024-09-14T22:36:56.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.810 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483671+0000", 2024-09-14T22:36:56.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483671+0000", 2024-09-14T22:36:56.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483672+0000", 2024-09-14T22:36:56.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:36:56.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483674+0000", 2024-09-14T22:36:56.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3190000000000002e-06 2024-09-14T22:36:56.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483680+0000", 2024-09-14T22:36:56.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6860000000000003e-06 2024-09-14T22:36:56.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483684+0000", 2024-09-14T22:36:56.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3769999999999998e-06 2024-09-14T22:36:56.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511982+0000", 2024-09-14T22:36:56.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028297478000000001 2024-09-14T22:36:56.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512027+0000", 2024-09-14T22:36:56.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5290000000000002e-05 2024-09-14T22:36:56.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512077+0000", 2024-09-14T22:36:56.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9876000000000002e-05 2024-09-14T22:36:56.814 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.814 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.814 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.814 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20872 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7254016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:56.814 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483689+0000", 2024-09-14T22:36:56.814 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.278805949999999, 2024-09-14T22:36:56.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028494049, 2024-09-14T22:36:56.814 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.814 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.814 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.814 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.814 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.815 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.815 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20872 2024-09-14T22:36:56.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.815 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483689+0000", 2024-09-14T22:36:56.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483689+0000", 2024-09-14T22:36:56.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483690+0000", 2024-09-14T22:36:56.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:36:56.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483692+0000", 2024-09-14T22:36:56.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.778e-06 2024-09-14T22:36:56.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483697+0000", 2024-09-14T22:36:56.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7069999999999997e-06 2024-09-14T22:36:56.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483702+0000", 2024-09-14T22:36:56.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4689999999999999e-06 2024-09-14T22:36:56.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512104+0000", 2024-09-14T22:36:56.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028401948999999999 2024-09-14T22:36:56.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512142+0000", 2024-09-14T22:36:56.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8114999999999997e-05 2024-09-14T22:36:56.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512183+0000", 2024-09-14T22:36:56.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1496999999999999e-05 2024-09-14T22:36:56.818 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.818 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.818 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.818 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20873 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7282688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:56.819 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483707+0000", 2024-09-14T22:36:56.819 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.278788236, 2024-09-14T22:36:56.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028581662000000001, 2024-09-14T22:36:56.819 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.819 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.819 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.819 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.819 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.819 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.819 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20873 2024-09-14T22:36:56.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.819 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483707+0000", 2024-09-14T22:36:56.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483707+0000", 2024-09-14T22:36:56.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483707+0000", 2024-09-14T22:36:56.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2500000000000001e-07 2024-09-14T22:36:56.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483709+0000", 2024-09-14T22:36:56.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.891e-06 2024-09-14T22:36:56.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483715+0000", 2024-09-14T22:36:56.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7019999999999998e-06 2024-09-14T22:36:56.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483720+0000", 2024-09-14T22:36:56.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4900000000000002e-06 2024-09-14T22:36:56.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512213+0000", 2024-09-14T22:36:56.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028493622 2024-09-14T22:36:56.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512248+0000", 2024-09-14T22:36:56.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5228000000000002e-05 2024-09-14T22:36:56.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512289+0000", 2024-09-14T22:36:56.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0404000000000001e-05 2024-09-14T22:36:56.823 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.823 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.823 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.824 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20874 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7311360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:56.824 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483725+0000", 2024-09-14T22:36:56.824 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.278770597000001, 2024-09-14T22:36:56.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028777193999999999, 2024-09-14T22:36:56.824 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.824 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.824 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.824 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.824 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.824 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.824 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20874 2024-09-14T22:36:56.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.825 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.825 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.825 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483725+0000", 2024-09-14T22:36:56.825 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.825 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.825 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483725+0000", 2024-09-14T22:36:56.825 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.825 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.826 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483725+0000", 2024-09-14T22:36:56.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8000000000000001e-07 2024-09-14T22:36:56.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.826 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.826 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483727+0000", 2024-09-14T22:36:56.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8920000000000001e-06 2024-09-14T22:36:56.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.826 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.826 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483733+0000", 2024-09-14T22:36:56.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6840000000000001e-06 2024-09-14T22:36:56.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483737+0000", 2024-09-14T22:36:56.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4440000000000001e-06 2024-09-14T22:36:56.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512326+0000", 2024-09-14T22:36:56.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028588612999999999 2024-09-14T22:36:56.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512399+0000", 2024-09-14T22:36:56.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2782999999999998e-05 2024-09-14T22:36:56.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512502+0000", 2024-09-14T22:36:56.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000103398 2024-09-14T22:36:56.828 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.828 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.828 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.828 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20912 3.1c 3:3b9f022e:::rbd_data.113c4558bb1.000000000000009a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 497664~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:36:56.828 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.484800+0000", 2024-09-14T22:36:56.828 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.277695069, 2024-09-14T22:36:56.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045787688999999999, 2024-09-14T22:36:56.828 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.828 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.829 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:56.829 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.829 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.829 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.829 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20912 2024-09-14T22:36:56.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.829 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484800+0000", 2024-09-14T22:36:56.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484800+0000", 2024-09-14T22:36:56.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484801+0000", 2024-09-14T22:36:56.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1939999999999999e-06 2024-09-14T22:36:56.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484809+0000", 2024-09-14T22:36:56.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2989999999999999e-06 2024-09-14T22:36:56.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484823+0000", 2024-09-14T22:36:56.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3986e-05 2024-09-14T22:36:56.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484831+0000", 2024-09-14T22:36:56.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4009999999999997e-06 2024-09-14T22:36:56.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512318+0000", 2024-09-14T22:36:56.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027487208999999999 2024-09-14T22:36:56.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.832 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.832 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512450+0000", 2024-09-14T22:36:56.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000131681 2024-09-14T22:36:56.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.832 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:36:56.832 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512666+0000", 2024-09-14T22:36:56.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021547 2024-09-14T22:36:56.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.832 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:56.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.513788+0000", 2024-09-14T22:36:56.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001122975 2024-09-14T22:36:56.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:56.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.530473+0000", 2024-09-14T22:36:56.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.016684984999999999 2024-09-14T22:36:56.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:56.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.530533+0000", 2024-09-14T22:36:56.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9296999999999998e-05 2024-09-14T22:36:56.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.834 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.530588+0000", 2024-09-14T22:36:56.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5192000000000003e-05 2024-09-14T22:36:56.834 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.834 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.834 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.834 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:30512 3.1d 3:b9e08d52:::rbd_data.113cda0183a0.0000000000000008:6 [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:56.834 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.331867+0000", 2024-09-14T22:36:56.834 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.430628115000001, 2024-09-14T22:36:56.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027901088000000001, 2024-09-14T22:36:56.835 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.835 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.835 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.835 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.835 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.835 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.835 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 30512 2024-09-14T22:36:56.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.835 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.835 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.835 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331867+0000", 2024-09-14T22:36:56.835 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331867+0000", 2024-09-14T22:36:56.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331868+0000", 2024-09-14T22:36:56.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8999999999999996e-07 2024-09-14T22:36:56.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331871+0000", 2024-09-14T22:36:56.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.163e-06 2024-09-14T22:36:56.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.837 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.837 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331880+0000", 2024-09-14T22:36:56.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3449999999999995e-06 2024-09-14T22:36:56.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.837 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.837 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331888+0000", 2024-09-14T22:36:56.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5320000000000001e-06 2024-09-14T22:36:56.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.358443+0000", 2024-09-14T22:36:56.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026554845000000001 2024-09-14T22:36:56.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.359649+0000", 2024-09-14T22:36:56.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001205991 2024-09-14T22:36:56.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.359768+0000", 2024-09-14T22:36:56.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119522 2024-09-14T22:36:56.839 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.839 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.839 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.839 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:30513 3.1d 3:b9e08d52:::rbd_data.113cda0183a0.0000000000000008:6 [read 456704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:56.839 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.331897+0000", 2024-09-14T22:36:56.839 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.430597889000001, 2024-09-14T22:36:56.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028098122999999999, 2024-09-14T22:36:56.839 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.839 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.839 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.839 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.839 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.839 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.839 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 30513 2024-09-14T22:36:56.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.840 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331897+0000", 2024-09-14T22:36:56.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331897+0000", 2024-09-14T22:36:56.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331898+0000", 2024-09-14T22:36:56.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:36:56.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331901+0000", 2024-09-14T22:36:56.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7889999999999999e-06 2024-09-14T22:36:56.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331910+0000", 2024-09-14T22:36:56.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.346e-06 2024-09-14T22:36:56.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331917+0000", 2024-09-14T22:36:56.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2239999999999998e-06 2024-09-14T22:36:56.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.359807+0000", 2024-09-14T22:36:56.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027889225 2024-09-14T22:36:56.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.359895+0000", 2024-09-14T22:36:56.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8721000000000007e-05 2024-09-14T22:36:56.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.359996+0000", 2024-09-14T22:36:56.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010022300000000001 2024-09-14T22:36:56.843 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.843 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.843 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.843 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:30514 3.1d 3:b9e08d52:::rbd_data.113cda0183a0.0000000000000008:6 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:56.843 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.331927+0000", 2024-09-14T22:36:56.844 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.430568076, 2024-09-14T22:36:56.844 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028476330000000001, 2024-09-14T22:36:56.844 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.844 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.844 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.844 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.844 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.844 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.844 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 30514 2024-09-14T22:36:56.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.844 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.844 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.844 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331927+0000", 2024-09-14T22:36:56.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331927+0000", 2024-09-14T22:36:56.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331928+0000", 2024-09-14T22:36:56.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5700000000000002e-07 2024-09-14T22:36:56.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331931+0000", 2024-09-14T22:36:56.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9909999999999998e-06 2024-09-14T22:36:56.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331940+0000", 2024-09-14T22:36:56.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.312e-06 2024-09-14T22:36:56.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331948+0000", 2024-09-14T22:36:56.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5630000000000003e-06 2024-09-14T22:36:56.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360032+0000", 2024-09-14T22:36:56.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028083877 2024-09-14T22:36:56.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360282+0000", 2024-09-14T22:36:56.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00025062100000000003 2024-09-14T22:36:56.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360404+0000", 2024-09-14T22:36:56.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000121409 2024-09-14T22:36:56.847 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.847 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.848 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.848 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:30515 3.1d 3:b9e08d52:::rbd_data.113cda0183a0.0000000000000008:6 [read 514048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:56.848 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.331957+0000", 2024-09-14T22:36:56.848 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.430538251000002, 2024-09-14T22:36:56.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028684729999999999, 2024-09-14T22:36:56.848 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.848 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.848 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:36:56.848 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.849 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.849 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.849 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 30515 2024-09-14T22:36:56.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.849 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331957+0000", 2024-09-14T22:36:56.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331957+0000", 2024-09-14T22:36:56.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.850 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331958+0000", 2024-09-14T22:36:56.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1200000000000003e-07 2024-09-14T22:36:56.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.850 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331960+0000", 2024-09-14T22:36:56.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6929999999999999e-06 2024-09-14T22:36:56.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.850 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331970+0000", 2024-09-14T22:36:56.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4020000000000008e-06 2024-09-14T22:36:56.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331977+0000", 2024-09-14T22:36:56.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7660000000000001e-06 2024-09-14T22:36:56.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360444+0000", 2024-09-14T22:36:56.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028467399000000001 2024-09-14T22:36:56.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360536+0000", 2024-09-14T22:36:56.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1618000000000003e-05 2024-09-14T22:36:56.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.852 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360642+0000", 2024-09-14T22:36:56.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010624 2024-09-14T22:36:56.852 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.852 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.852 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.852 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:46576 3.1d 3:b9b1f07f:::rbd_data.113c8642facc.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T22:36:56.852 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:37.379223+0000", 2024-09-14T22:36:56.852 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.383272726000001, 2024-09-14T22:36:56.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072706044999999997, 2024-09-14T22:36:56.853 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.853 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.853 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:56.853 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.853 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.853 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.853 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 46576 2024-09-14T22:36:56.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.853 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379223+0000", 2024-09-14T22:36:56.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379223+0000", 2024-09-14T22:36:56.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379226+0000", 2024-09-14T22:36:56.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0929999999999999e-06 2024-09-14T22:36:56.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379240+0000", 2024-09-14T22:36:56.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4606999999999999e-05 2024-09-14T22:36:56.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379272+0000", 2024-09-14T22:36:56.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2073000000000003e-05 2024-09-14T22:36:56.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379288+0000", 2024-09-14T22:36:56.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6036999999999999e-05 2024-09-14T22:36:56.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.856 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379451+0000", 2024-09-14T22:36:56.856 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000162795 2024-09-14T22:36:56.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.856 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.856 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379702+0000", 2024-09-14T22:36:56.856 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00025106500000000002 2024-09-14T22:36:56.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.856 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:36:56.856 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.394067+0000", 2024-09-14T22:36:56.856 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014364564999999999 2024-09-14T22:36:56.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:56.857 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.451737+0000", 2024-09-14T22:36:56.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057670185999999998 2024-09-14T22:36:56.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:56.857 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.451823+0000", 2024-09-14T22:36:56.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6075000000000005e-05 2024-09-14T22:36:56.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:56.857 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.451881+0000", 2024-09-14T22:36:56.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7454999999999998e-05 2024-09-14T22:36:56.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.451929+0000", 2024-09-14T22:36:56.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8093999999999997e-05 2024-09-14T22:36:56.858 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.858 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.858 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.858 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:74169 3.16 3:69acee67:::rbd_data.113cce28790.0000000000000067:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T22:36:56.858 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:55.614872+0000", 2024-09-14T22:36:56.858 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.1476235400000001, 2024-09-14T22:36:56.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036045858, 2024-09-14T22:36:56.859 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:36:56.859 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:36:56.859 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:36:56.859 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:36:56.859 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:36:56.859 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:56.859 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74169 2024-09-14T22:36:56.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.859 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:36:56.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.859 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:36:56.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.614872+0000", 2024-09-14T22:36:56.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:36:56.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.614872+0000", 2024-09-14T22:36:56.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:36:56.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:36:56.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.614875+0000", 2024-09-14T22:36:56.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2799999999999999e-06 2024-09-14T22:36:56.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:36:56.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.614891+0000", 2024-09-14T22:36:56.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5613000000000001e-05 2024-09-14T22:36:56.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:36:56.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.614931+0000", 2024-09-14T22:36:56.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0540000000000001e-05 2024-09-14T22:36:56.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:36:56.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.614956+0000", 2024-09-14T22:36:56.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4939e-05 2024-09-14T22:36:56.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:36:56.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.615096+0000", 2024-09-14T22:36:56.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013952600000000001 2024-09-14T22:36:56.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:36:56.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.615345+0000", 2024-09-14T22:36:56.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024922099999999999 2024-09-14T22:36:56.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T22:36:56.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.621370+0000", 2024-09-14T22:36:56.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0060251239999999998 2024-09-14T22:36:56.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:36:56.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.650734+0000", 2024-09-14T22:36:56.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029364155999999999 2024-09-14T22:36:56.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:36:56.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.650760+0000", 2024-09-14T22:36:56.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5446999999999999e-05 2024-09-14T22:36:56.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:36:56.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.650842+0000", 2024-09-14T22:36:56.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2268999999999994e-05 2024-09-14T22:36:56.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:36:56.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:36:56.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:36:56.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.650918+0000", 2024-09-14T22:36:56.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5742999999999996e-05 2024-09-14T22:36:56.864 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.864 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.864 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.864 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:36:56.864 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:36:56.864 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:56.865 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:36:56.866 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:56.863+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:56.891 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:36:56.891 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:36:56.908 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:36:56.909 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:36:56.909 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-14T22:36:56.928 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:36:56.928 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:36:56.947 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:36:56.947 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:36:56.967 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:56.963+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:56.968 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:36:56.968 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:36:56.988 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:36:56.988 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:36:57.009 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:36:57.009 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:36:57.013 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:36:57.014 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:36:57.027 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2024-09-14T22:36:57.041 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:57.041 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:57.057 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:36:57.103 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:57.099+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:57.131 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone14 from 0xa00988f (overlap 0xa00988f) to 0x900894d 2024-09-14T22:36:57.169 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:57.163+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:57.204 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:57.205 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:57.220 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:36:57.270 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:57.267+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:57.371 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:57.363+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:57.385 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:57.385 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:57.390 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-14T22:36:57.400 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:36:57.487 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:57.483+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:57.568 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:57.568 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:57.573 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:57.567+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:57.582 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:36:57.674 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:57.671+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:57.744 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:57.744 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:57.758 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:36:57.761 INFO:teuthology.orchestra.run.smithi001.stdout:1970 write 0x2a27582 thru 0x2a2baf0 (0x456f bytes) 2024-09-14T22:36:57.774 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:57.771+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:57.777 INFO:teuthology.orchestra.run.smithi001.stdout:1971 writesame 0x999b083 thru 0x999f27c data_size 0xf(0x41fa bytes) 2024-09-14T22:36:57.789 INFO:teuthology.orchestra.run.smithi001.stdout:1972 read 0x1e6515c thru 0x1e656b7 (0x55c bytes) 2024-09-14T22:36:57.796 INFO:teuthology.orchestra.run.smithi001.stdout:1973 trunc from 0xa00988f to 0x4cbd77a 2024-09-14T22:36:57.858 INFO:teuthology.orchestra.run.smithi001.stdout:1974 writesame 0xd824586 thru 0xd82921e data_size 0x1(0x4c99 bytes) 2024-09-14T22:36:57.858 INFO:teuthology.orchestra.run.smithi001.stdout:1975 write 0x2c2e48c thru 0x2c32625 (0x419a bytes) 2024-09-14T22:36:57.865 INFO:teuthology.orchestra.run.smithi001.stdout:1976 read 0x1f0c992 thru 0x1f10ba7 (0x4216 bytes) 2024-09-14T22:36:57.866 INFO:teuthology.orchestra.run.smithi001.stdout:1977 write 0xc01003d thru 0xc017964 (0x7928 bytes) 2024-09-14T22:36:57.875 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:57.871+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:57.881 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:57.881 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:57.886 INFO:teuthology.orchestra.run.smithi001.stdout:1980 read 0x9287ad4 thru 0x9288d6e (0x129b bytes) 2024-09-14T22:36:57.886 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:57.890 INFO:teuthology.orchestra.run.smithi001.stdout:1982 trunc from 0xd82921f to 0xaed9616 2024-09-14T22:36:57.897 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:36:57.909 INFO:teuthology.orchestra.run.smithi001.stdout:1983 punch from 0x8909215 to 0x8914cb8, (0xbaa3 bytes) 2024-09-14T22:36:57.909 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:57.909 INFO:teuthology.orchestra.run.smithi001.stdout:1986 read 0xa4b0636 thru 0xa4bc9e4 (0xc3af bytes) 2024-09-14T22:36:57.917 INFO:teuthology.orchestra.run.smithi001.stdout:1988 trunc from 0xaed9616 to 0x737917f 2024-09-14T22:36:57.935 INFO:teuthology.orchestra.run.smithi001.stdout:1989 writesame 0x7d971f4 thru 0x7d9d0b9 data_size 0x7(0x5ec6 bytes) 2024-09-14T22:36:57.935 INFO:teuthology.orchestra.run.smithi001.stdout:1990 read 0x20e7b90 thru 0x20f3437 (0xb8a8 bytes) 2024-09-14T22:36:57.935 INFO:teuthology.orchestra.run.smithi001.stdout:1991 punch from 0x59578af to 0x5959c4b, (0x239c bytes) 2024-09-14T22:36:57.935 INFO:teuthology.orchestra.run.smithi001.stdout:1992 read 0x166c492 thru 0x1671a46 (0x55b5 bytes) 2024-09-14T22:36:57.941 INFO:teuthology.orchestra.run.smithi001.stdout:1993 trunc from 0x7d9d0ba to 0x5b0a729 2024-09-14T22:36:57.968 INFO:teuthology.orchestra.run.smithi001.stdout:1994 write 0xb0ba806 thru 0xb0c7b04 (0xd2ff bytes) 2024-09-14T22:36:57.976 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:57.971+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:57.987 INFO:teuthology.orchestra.run.smithi001.stdout:1995 write 0x8bf0a72 thru 0x8bf29bd (0x1f4c bytes) 2024-09-14T22:36:57.994 INFO:teuthology.orchestra.run.smithi001.stdout:1996 write 0x228e17a thru 0x229d920 (0xf7a7 bytes) 2024-09-14T22:36:58.011 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:58.028 INFO:teuthology.orchestra.run.smithi001.stdout:1999 trunc from 0xb0c7b05 to 0x10e142d 2024-09-14T22:36:58.046 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:58.046 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:58.061 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:36:58.109 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:58.107+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:58.178 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:58.175+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:58.235 INFO:teuthology.orchestra.run.smithi001.stdout:2000 trunc from 0x10e142d to 0x645ffe2 2024-09-14T22:36:58.243 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:58.252 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:58.252 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:58.254 INFO:teuthology.orchestra.run.smithi001.stdout:2004 writesame 0x7324f49 thru 0x732ab9b data_size 0x1d(0x5c53 bytes) 2024-09-14T22:36:58.254 INFO:teuthology.orchestra.run.smithi001.stdout:2005 write 0x1a510a5 thru 0x1a5c3ca (0xb326 bytes) 2024-09-14T22:36:58.266 INFO:teuthology.orchestra.run.smithi001.stdout:2007 write 0x697019f thru 0x697e803 (0xe665 bytes) 2024-09-14T22:36:58.270 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:36:58.290 INFO:teuthology.orchestra.run.smithi001.stdout:2009 trunc from 0x732ab9c to 0x51779a3 2024-09-14T22:36:58.291 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:58.287+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:58.321 INFO:teuthology.orchestra.run.smithi001.stdout:2013 write 0x8c1b58f thru 0x8c1cfa2 (0x1a14 bytes) 2024-09-14T22:36:58.342 INFO:teuthology.orchestra.run.smithi001.stdout:2014 writesame 0xdd3619d thru 0xdd43942 data_size 0x9(0xd7a6 bytes) 2024-09-14T22:36:58.342 INFO:teuthology.orchestra.run.smithi001.stdout:2015 read 0x9c7a735 thru 0x9c8263d (0x7f09 bytes) 2024-09-14T22:36:58.343 INFO:teuthology.orchestra.run.smithi001.stdout:2016 write 0x3a4025b thru 0x3a4439a (0x4140 bytes) 2024-09-14T22:36:58.362 INFO:teuthology.orchestra.run.smithi001.stdout:2017 trunc from 0xdd43943 to 0x623c4c4 2024-09-14T22:36:58.378 INFO:teuthology.orchestra.run.smithi001.stdout:2020 punch from 0x149dc32 to 0x149ea3f, (0xe0d bytes) 2024-09-14T22:36:58.379 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:58.375+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:58.383 INFO:teuthology.orchestra.run.smithi001.stdout:2021 trunc from 0x623c4c4 to 0x33f434e 2024-09-14T22:36:58.419 INFO:teuthology.orchestra.run.smithi001.stdout:2022 write 0x843fb1a thru 0x844bf2f (0xc416 bytes) 2024-09-14T22:36:58.437 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:58.437 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:58.442 INFO:teuthology.orchestra.run.smithi001.stdout:2023 read 0xfa330a thru 0xfaead9 (0xb7d0 bytes) 2024-09-14T22:36:58.444 INFO:teuthology.orchestra.run.smithi001.stdout:2025 flatten 2024-09-14T22:36:58.458 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:36:58.501 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:58.499+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:58.581 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:58.575+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:58.682 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:36:58.679+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:36:58.688 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:58.688 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:58.706 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:36:58.708 INFO:teuthology.orchestra.run.smithi001.stdout:2027 read 0x33caf26 thru 0x33d97db (0xe8b6 bytes) 2024-09-14T22:36:58.708 INFO:teuthology.orchestra.run.smithi001.stdout:2028 punch from 0x74290fc to 0x742e336, (0x523a bytes) 2024-09-14T22:36:58.708 INFO:teuthology.orchestra.run.smithi001.stdout:2029 read 0x35ad25d thru 0x35ae101 (0xea5 bytes) 2024-09-14T22:36:58.708 INFO:teuthology.orchestra.run.smithi001.stdout:2030 punch from 0x41abc2b to 0x41adfe5, (0x23ba bytes) 2024-09-14T22:36:58.723 INFO:teuthology.orchestra.run.smithi001.stdout:2031 write 0xebb5429 thru 0xebc3db9 (0xe991 bytes) 2024-09-14T22:36:58.735 INFO:teuthology.orchestra.run.smithi001.stdout:2032 read 0xa85ddbf thru 0xa867b65 (0x9da7 bytes) 2024-09-14T22:36:58.735 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:58.735 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:58.736 INFO:teuthology.orchestra.run.smithi001.stdout:2037 punch from 0x86c9700 to 0x86cf82c, (0x612c bytes) 2024-09-14T22:36:58.736 INFO:teuthology.orchestra.run.smithi001.stdout:2038 read 0x1e58bc4 thru 0x1e6896c (0xfda9 bytes) 2024-09-14T22:36:58.736 INFO:teuthology.orchestra.run.smithi001.stdout:2039 read 0x3bab818 thru 0x3bae70e (0x2ef7 bytes) 2024-09-14T22:36:58.736 INFO:teuthology.orchestra.run.smithi001.stdout:2044 write 0xcc7e923 thru 0xcc81c92 (0x3370 bytes) 2024-09-14T22:36:58.740 INFO:teuthology.orchestra.run.smithi001.stdout:2046 punch from 0xaf11e14 to 0xaf1bf1c, (0xa108 bytes) 2024-09-14T22:36:58.741 INFO:teuthology.orchestra.run.smithi001.stdout:2047 writesame 0x425c775 thru 0x4269b24 data_size 0x18(0xd3b0 bytes) 2024-09-14T22:36:58.753 INFO:teuthology.orchestra.run.smithi001.stdout:2048 read 0x8c24db4 thru 0x8c27b50 (0x2d9d bytes) 2024-09-14T22:36:58.758 INFO:teuthology.orchestra.run.smithi001.stdout:2049 trunc from 0xebc3dba to 0xb376c39 2024-09-14T22:36:58.771 INFO:teuthology.orchestra.run.smithi001.stdout:2051 read 0xb344622 thru 0xb3522ff (0xdcde bytes) 2024-09-14T22:36:58.779 INFO:teuthology.orchestra.run.smithi001.stdout:2052 write 0xbfac080 thru 0xbfb1547 (0x54c8 bytes) 2024-09-14T22:36:58.782 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:58.779+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:58.789 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:58.789 INFO:teuthology.orchestra.run.smithi001.stdout:2054 read 0x6afbea6 thru 0x6b06254 (0xa3af bytes) 2024-09-14T22:36:58.789 INFO:teuthology.orchestra.run.smithi001.stdout:2057 read 0x38abb2b thru 0x38b305f (0x7535 bytes) 2024-09-14T22:36:58.789 INFO:teuthology.orchestra.run.smithi001.stdout:2058 write 0x1ae2d12 thru 0x1af1802 (0xeaf1 bytes) 2024-09-14T22:36:58.793 INFO:teuthology.orchestra.run.smithi001.stdout:2060 read 0x98f133d thru 0x98f54bc (0x4180 bytes) 2024-09-14T22:36:58.793 INFO:teuthology.orchestra.run.smithi001.stdout:2061 writesame 0xa99acdf thru 0xa9a7e5e data_size 0x40(0xd180 bytes) 2024-09-14T22:36:58.803 INFO:teuthology.orchestra.run.smithi001.stdout:2062 read 0x3c5986a thru 0x3c65079 (0xb810 bytes) 2024-09-14T22:36:58.803 INFO:teuthology.orchestra.run.smithi001.stdout:2063 read 0x33c252b thru 0x33c70a0 (0x4b76 bytes) 2024-09-14T22:36:58.803 INFO:teuthology.orchestra.run.smithi001.stdout:2069 writesame 0x64bc3e8 thru 0x64ca57c data_size 0x11(0xe195 bytes) 2024-09-14T22:36:58.814 INFO:teuthology.orchestra.run.smithi001.stdout:2070 write 0x18fd503 thru 0x18fe16a (0xc68 bytes) 2024-09-14T22:36:58.820 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:58.820 INFO:teuthology.orchestra.run.smithi001.stdout:2072 read 0xbcd4eab thru 0xbcd8f5f (0x40b5 bytes) 2024-09-14T22:36:58.820 INFO:teuthology.orchestra.run.smithi001.stdout:2073 punch from 0x31b36de to 0x31c2c15, (0xf537 bytes) 2024-09-14T22:36:58.820 INFO:teuthology.orchestra.run.smithi001.stdout:2076 clone 16 order 21 su 131072 sc 7 2024-09-14T22:36:58.844 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:58.844 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:58.858 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:36:58.884 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:58.879+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:58.985 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:36:58.979+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:36:59.015 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:36:59.025 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:59.025 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:59.046 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:36:59.049 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T22:36:59.086 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:59.079+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:59.187 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:36:59.183+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:36:59.211 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:36:59.212 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:36:59.241 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:36:59.242 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:36:59.246 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:36:59.246 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:36:59.246 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-14T22:36:59.266 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:36:59.272 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:36:59.272 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:36:59.287 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:36:59.283+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:36:59.296 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:36:59.297 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:36:59.318 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:36:59.318 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:36:59.327 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-14T22:36:59.340 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:36:59.340 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:36:59.366 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:36:59.367 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:36:59.371 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:36:59.371 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:36:59.388 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:59.383+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:59.422 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:59.422 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:36:59.422 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:36:59.422 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:36:59.422 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:59.445 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:36:59.489 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:36:59.483+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:36:59.585 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:36:59.585 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:36:59.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:36:59.585 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:36:59.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.585 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35300 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1253376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:59.585 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.506930+0000", 2024-09-14T22:36:59.585 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.077091745000001, 2024-09-14T22:36:59.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044627325000000002, 2024-09-14T22:36:59.586 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:59.586 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:59.586 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:59.586 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:59.586 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:59.586 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:59.586 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35300 2024-09-14T22:36:59.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.586 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:59.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:59.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506930+0000", 2024-09-14T22:36:59.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:59.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506930+0000", 2024-09-14T22:36:59.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:59.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506930+0000", 2024-09-14T22:36:59.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2500000000000001e-07 2024-09-14T22:36:59.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:59.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506932+0000", 2024-09-14T22:36:59.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8300000000000001e-06 2024-09-14T22:36:59.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:59.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506940+0000", 2024-09-14T22:36:59.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2810000000000003e-06 2024-09-14T22:36:59.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:59.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506944+0000", 2024-09-14T22:36:59.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8049999999999997e-06 2024-09-14T22:36:59.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:59.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.551359+0000", 2024-09-14T22:36:59.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044414370000000002 2024-09-14T22:36:59.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:59.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.551432+0000", 2024-09-14T22:36:59.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3338999999999997e-05 2024-09-14T22:36:59.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:59.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.551557+0000", 2024-09-14T22:36:59.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000125275 2024-09-14T22:36:59.590 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.590 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:59.590 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.590 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35301 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1282048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:59.590 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.506950+0000", 2024-09-14T22:36:59.590 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.077071543999999, 2024-09-14T22:36:59.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044873612, 2024-09-14T22:36:59.590 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:59.590 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:59.591 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:59.591 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:59.591 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:59.591 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:59.591 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35301 2024-09-14T22:36:59.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.591 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:59.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:59.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506950+0000", 2024-09-14T22:36:59.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:59.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506950+0000", 2024-09-14T22:36:59.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:59.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506951+0000", 2024-09-14T22:36:59.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0400000000000002e-07 2024-09-14T22:36:59.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:59.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506953+0000", 2024-09-14T22:36:59.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9470000000000002e-06 2024-09-14T22:36:59.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:59.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506958+0000", 2024-09-14T22:36:59.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8930000000000002e-06 2024-09-14T22:36:59.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:59.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506963+0000", 2024-09-14T22:36:59.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.459e-06 2024-09-14T22:36:59.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:59.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.551642+0000", 2024-09-14T22:36:59.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044678766000000002 2024-09-14T22:36:59.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:59.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.551715+0000", 2024-09-14T22:36:59.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3603999999999995e-05 2024-09-14T22:36:59.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:59.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.551824+0000", 2024-09-14T22:36:59.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108539 2024-09-14T22:36:59.595 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.595 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:59.595 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.595 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35302 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1310720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:59.595 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.506970+0000", 2024-09-14T22:36:59.595 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.077051833999999, 2024-09-14T22:36:59.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045057898999999998, 2024-09-14T22:36:59.595 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:59.595 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:59.595 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:59.595 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:59.596 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:59.596 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:59.596 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35302 2024-09-14T22:36:59.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.596 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:59.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:59.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506970+0000", 2024-09-14T22:36:59.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:59.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506970+0000", 2024-09-14T22:36:59.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:59.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506970+0000", 2024-09-14T22:36:59.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0999999999999999e-07 2024-09-14T22:36:59.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:59.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506973+0000", 2024-09-14T22:36:59.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4310000000000001e-06 2024-09-14T22:36:59.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:59.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506981+0000", 2024-09-14T22:36:59.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2830000000000001e-06 2024-09-14T22:36:59.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:59.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506986+0000", 2024-09-14T22:36:59.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3959999999999997e-06 2024-09-14T22:36:59.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:59.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.551856+0000", 2024-09-14T22:36:59.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044869603000000001 2024-09-14T22:36:59.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:59.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.551931+0000", 2024-09-14T22:36:59.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5253999999999995e-05 2024-09-14T22:36:59.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:59.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552028+0000", 2024-09-14T22:36:59.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6521999999999995e-05 2024-09-14T22:36:59.599 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.599 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:59.599 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.600 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35303 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1339392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:59.600 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.506994+0000", 2024-09-14T22:36:59.600 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.077027926, 2024-09-14T22:36:59.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045224116000000002, 2024-09-14T22:36:59.600 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:59.600 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:59.600 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:59.600 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:59.600 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:59.600 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:59.600 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35303 2024-09-14T22:36:59.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.601 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:59.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:59.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506994+0000", 2024-09-14T22:36:59.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:59.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506994+0000", 2024-09-14T22:36:59.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:59.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506994+0000", 2024-09-14T22:36:59.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1699999999999999e-07 2024-09-14T22:36:59.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:59.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.506997+0000", 2024-09-14T22:36:59.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.357e-06 2024-09-14T22:36:59.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:59.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507005+0000", 2024-09-14T22:36:59.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1939999999999998e-06 2024-09-14T22:36:59.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:59.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507013+0000", 2024-09-14T22:36:59.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3930000000000008e-06 2024-09-14T22:36:59.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:59.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552058+0000", 2024-09-14T22:36:59.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045044645000000001 2024-09-14T22:36:59.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:59.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552132+0000", 2024-09-14T22:36:59.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3831000000000005e-05 2024-09-14T22:36:59.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:59.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552218+0000", 2024-09-14T22:36:59.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6279000000000005e-05 2024-09-14T22:36:59.604 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.604 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:59.604 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.604 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35304 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1368064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:59.604 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507021+0000", 2024-09-14T22:36:59.604 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.077000366, 2024-09-14T22:36:59.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045388246, 2024-09-14T22:36:59.605 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:59.605 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:59.605 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:59.605 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:59.605 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:59.605 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:59.605 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35304 2024-09-14T22:36:59.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.605 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:59.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:59.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507021+0000", 2024-09-14T22:36:59.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:59.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507021+0000", 2024-09-14T22:36:59.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:59.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507022+0000", 2024-09-14T22:36:59.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.82e-07 2024-09-14T22:36:59.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:59.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507024+0000", 2024-09-14T22:36:59.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4949999999999998e-06 2024-09-14T22:36:59.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:59.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507032+0000", 2024-09-14T22:36:59.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9449999999999993e-06 2024-09-14T22:36:59.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:59.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507039+0000", 2024-09-14T22:36:59.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2129999999999996e-06 2024-09-14T22:36:59.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:59.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552250+0000", 2024-09-14T22:36:59.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045211150999999998 2024-09-14T22:36:59.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:59.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552326+0000", 2024-09-14T22:36:59.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6008999999999997e-05 2024-09-14T22:36:59.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:59.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552410+0000", 2024-09-14T22:36:59.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3950999999999996e-05 2024-09-14T22:36:59.608 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.608 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:59.608 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.609 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35305 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1396736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:59.609 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507048+0000", 2024-09-14T22:36:59.609 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.076973828, 2024-09-14T22:36:59.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045554728000000003, 2024-09-14T22:36:59.609 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:59.609 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:59.609 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:59.609 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:59.609 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:59.609 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:59.609 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35305 2024-09-14T22:36:59.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.610 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:59.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:59.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507048+0000", 2024-09-14T22:36:59.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:59.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507048+0000", 2024-09-14T22:36:59.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:59.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507048+0000", 2024-09-14T22:36:59.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7800000000000002e-07 2024-09-14T22:36:59.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:59.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507050+0000", 2024-09-14T22:36:59.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.024e-06 2024-09-14T22:36:59.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:59.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507057+0000", 2024-09-14T22:36:59.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1800000000000001e-06 2024-09-14T22:36:59.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:59.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507062+0000", 2024-09-14T22:36:59.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0930000000000002e-06 2024-09-14T22:36:59.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:59.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552439+0000", 2024-09-14T22:36:59.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045377460000000001 2024-09-14T22:36:59.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:59.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552512+0000", 2024-09-14T22:36:59.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2633999999999996e-05 2024-09-14T22:36:59.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:59.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552603+0000", 2024-09-14T22:36:59.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0958999999999994e-05 2024-09-14T22:36:59.613 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.613 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:59.613 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.613 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35306 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1425408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:59.613 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507069+0000", 2024-09-14T22:36:59.613 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.076952285000001, 2024-09-14T22:36:59.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045725259999999997, 2024-09-14T22:36:59.613 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:59.614 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:59.614 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:59.614 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:59.614 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:59.614 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:59.614 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35306 2024-09-14T22:36:59.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.614 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:59.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:59.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507069+0000", 2024-09-14T22:36:59.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:59.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507069+0000", 2024-09-14T22:36:59.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:59.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507070+0000", 2024-09-14T22:36:59.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0400000000000002e-07 2024-09-14T22:36:59.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:59.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507072+0000", 2024-09-14T22:36:59.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.891e-06 2024-09-14T22:36:59.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.616 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:59.616 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507078+0000", 2024-09-14T22:36:59.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0279999999999997e-06 2024-09-14T22:36:59.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.616 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:59.616 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507082+0000", 2024-09-14T22:36:59.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4680000000000003e-06 2024-09-14T22:36:59.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.616 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:59.616 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552632+0000", 2024-09-14T22:36:59.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045550144000000001 2024-09-14T22:36:59.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:59.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552707+0000", 2024-09-14T22:36:59.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4148999999999998e-05 2024-09-14T22:36:59.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:59.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552795+0000", 2024-09-14T22:36:59.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8176000000000004e-05 2024-09-14T22:36:59.617 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.617 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:59.617 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.618 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35307 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1454080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:59.618 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:36:59.618 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.076933734000001, 2024-09-14T22:36:59.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045891358, 2024-09-14T22:36:59.618 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:59.618 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:59.618 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:59.618 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:59.618 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:59.618 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:59.618 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35307 2024-09-14T22:36:59.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.619 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:59.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:59.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:36:59.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:59.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:36:59.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:59.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:36:59.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4700000000000002e-07 2024-09-14T22:36:59.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:59.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507093+0000", 2024-09-14T22:36:59.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9520000000000002e-06 2024-09-14T22:36:59.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:59.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507099+0000", 2024-09-14T22:36:59.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0240000000000003e-06 2024-09-14T22:36:59.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:59.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507105+0000", 2024-09-14T22:36:59.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5520000000000004e-06 2024-09-14T22:36:59.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:59.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552826+0000", 2024-09-14T22:36:59.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045721494000000001 2024-09-14T22:36:59.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:59.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552896+0000", 2024-09-14T22:36:59.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9519999999999998e-05 2024-09-14T22:36:59.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:59.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552979+0000", 2024-09-14T22:36:59.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3468999999999997e-05 2024-09-14T22:36:59.622 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.622 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:59.622 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.622 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35308 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1482752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:59.623 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:36:59.623 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.076910737999999, 2024-09-14T22:36:59.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046076533000000003, 2024-09-14T22:36:59.623 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:59.623 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:59.623 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:59.623 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:59.623 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:59.623 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:59.623 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35308 2024-09-14T22:36:59.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.623 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:59.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:59.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:36:59.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:59.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:36:59.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.625 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:59.625 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:36:59.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:36:59.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.625 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:59.625 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507113+0000", 2024-09-14T22:36:59.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9099999999999999e-06 2024-09-14T22:36:59.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.625 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:59.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507119+0000", 2024-09-14T22:36:59.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7440000000000004e-06 2024-09-14T22:36:59.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:59.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507124+0000", 2024-09-14T22:36:59.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2129999999999999e-06 2024-09-14T22:36:59.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:59.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553007+0000", 2024-09-14T22:36:59.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045882488999999999 2024-09-14T22:36:59.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:59.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553076+0000", 2024-09-14T22:36:59.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9233000000000006e-05 2024-09-14T22:36:59.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:59.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553188+0000", 2024-09-14T22:36:59.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111557 2024-09-14T22:36:59.627 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.628 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:59.628 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.628 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35309 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1511424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:59.628 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:36:59.628 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.076891833000001, 2024-09-14T22:36:59.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046231869000000002, 2024-09-14T22:36:59.628 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:59.628 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:59.628 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:59.628 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:59.629 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:59.629 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:59.629 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35309 2024-09-14T22:36:59.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.629 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:59.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.629 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:59.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:36:59.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:59.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:36:59.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:59.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:36:59.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9099999999999999e-07 2024-09-14T22:36:59.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:59.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507132+0000", 2024-09-14T22:36:59.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8190000000000001e-06 2024-09-14T22:36:59.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:59.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507139+0000", 2024-09-14T22:36:59.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9419999999999998e-06 2024-09-14T22:36:59.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:59.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507145+0000", 2024-09-14T22:36:59.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2600000000000002e-06 2024-09-14T22:36:59.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:59.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553225+0000", 2024-09-14T22:36:59.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046079991000000001 2024-09-14T22:36:59.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:59.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553284+0000", 2024-09-14T22:36:59.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8638999999999999e-05 2024-09-14T22:36:59.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:59.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553362+0000", 2024-09-14T22:36:59.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7826999999999999e-05 2024-09-14T22:36:59.633 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.633 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:59.633 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.633 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35310 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1540096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:59.633 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507151+0000", 2024-09-14T22:36:59.633 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.076870376999999, 2024-09-14T22:36:59.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046444541999999998, 2024-09-14T22:36:59.634 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:59.634 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:59.634 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:59.634 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:59.634 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:59.634 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:59.634 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35310 2024-09-14T22:36:59.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.634 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:59.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:59.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507151+0000", 2024-09-14T22:36:59.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:59.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507151+0000", 2024-09-14T22:36:59.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:59.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507152+0000", 2024-09-14T22:36:59.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1699999999999999e-07 2024-09-14T22:36:59.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:59.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507154+0000", 2024-09-14T22:36:59.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.902e-06 2024-09-14T22:36:59.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:59.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507159+0000", 2024-09-14T22:36:59.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.823e-06 2024-09-14T22:36:59.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:59.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507165+0000", 2024-09-14T22:36:59.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.587e-06 2024-09-14T22:36:59.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:59.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553395+0000", 2024-09-14T22:36:59.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046230318999999999 2024-09-14T22:36:59.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:59.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553434+0000", 2024-09-14T22:36:59.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8680999999999998e-05 2024-09-14T22:36:59.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:59.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553596+0000", 2024-09-14T22:36:59.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016191299999999999 2024-09-14T22:36:59.638 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.638 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:59.638 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.639 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35311 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1568768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:59.639 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:36:59.639 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.076851124000001, 2024-09-14T22:36:59.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046623592999999998, 2024-09-14T22:36:59.639 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:59.639 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:59.639 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:59.640 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:59.640 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:59.640 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:59.640 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35311 2024-09-14T22:36:59.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.640 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:59.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:59.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:36:59.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:59.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:36:59.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:59.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:36:59.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:36:59.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:59.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507176+0000", 2024-09-14T22:36:59.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3349999999999997e-06 2024-09-14T22:36:59.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:59.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507200+0000", 2024-09-14T22:36:59.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4003e-05 2024-09-14T22:36:59.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:59.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507206+0000", 2024-09-14T22:36:59.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.976e-06 2024-09-14T22:36:59.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:59.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553627+0000", 2024-09-14T22:36:59.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046420284999999999 2024-09-14T22:36:59.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:59.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553696+0000", 2024-09-14T22:36:59.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9567999999999995e-05 2024-09-14T22:36:59.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.644 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:59.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553794+0000", 2024-09-14T22:36:59.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8060000000000006e-05 2024-09-14T22:36:59.644 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.644 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:59.644 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.644 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35562 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1370112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:59.644 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514584+0000", 2024-09-14T22:36:59.644 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.069437737000001, 2024-09-14T22:36:59.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044805208999999999, 2024-09-14T22:36:59.645 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:59.645 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:59.645 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:59.645 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:59.645 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:59.645 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:59.645 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35562 2024-09-14T22:36:59.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.645 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:59.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:59.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514584+0000", 2024-09-14T22:36:59.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:59.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514584+0000", 2024-09-14T22:36:59.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:59.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514584+0000", 2024-09-14T22:36:59.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0900000000000002e-07 2024-09-14T22:36:59.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:59.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514587+0000", 2024-09-14T22:36:59.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1739999999999999e-06 2024-09-14T22:36:59.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:59.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514593+0000", 2024-09-14T22:36:59.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3080000000000004e-06 2024-09-14T22:36:59.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:59.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514600+0000", 2024-09-14T22:36:59.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6420000000000001e-06 2024-09-14T22:36:59.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:59.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559211+0000", 2024-09-14T22:36:59.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044611454000000002 2024-09-14T22:36:59.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:59.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559296+0000", 2024-09-14T22:36:59.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4676e-05 2024-09-14T22:36:59.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:59.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559389+0000", 2024-09-14T22:36:59.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3546000000000002e-05 2024-09-14T22:36:59.649 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.649 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:59.649 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.649 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35563 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1398784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:59.649 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514606+0000", 2024-09-14T22:36:59.649 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.069415408000001, 2024-09-14T22:36:59.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045029859999999998, 2024-09-14T22:36:59.649 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:59.649 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:59.650 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:59.650 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:59.650 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:59.650 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:59.650 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35563 2024-09-14T22:36:59.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.650 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:59.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:59.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514606+0000", 2024-09-14T22:36:59.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:59.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514606+0000", 2024-09-14T22:36:59.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:59.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514607+0000", 2024-09-14T22:36:59.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1199999999999998e-07 2024-09-14T22:36:59.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:59.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514609+0000", 2024-09-14T22:36:59.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.199e-06 2024-09-14T22:36:59.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:59.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514615+0000", 2024-09-14T22:36:59.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1260000000000003e-06 2024-09-14T22:36:59.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:59.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514620+0000", 2024-09-14T22:36:59.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.045e-06 2024-09-14T22:36:59.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:59.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559440+0000", 2024-09-14T22:36:59.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044820169999999999 2024-09-14T22:36:59.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:59.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559524+0000", 2024-09-14T22:36:59.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4124999999999998e-05 2024-09-14T22:36:59.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:59.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559636+0000", 2024-09-14T22:36:59.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111783 2024-09-14T22:36:59.654 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.654 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:59.654 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.654 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35564 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1427456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:59.655 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514627+0000", 2024-09-14T22:36:59.655 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.069394669000001, 2024-09-14T22:36:59.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045232025000000002, 2024-09-14T22:36:59.655 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:59.655 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:59.655 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:59.655 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:59.655 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:59.655 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:59.655 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35564 2024-09-14T22:36:59.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.655 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:59.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:59.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514627+0000", 2024-09-14T22:36:59.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:59.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514627+0000", 2024-09-14T22:36:59.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:59.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514628+0000", 2024-09-14T22:36:59.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4299999999999998e-07 2024-09-14T22:36:59.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:59.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514630+0000", 2024-09-14T22:36:59.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1579999999999999e-06 2024-09-14T22:36:59.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:59.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514636+0000", 2024-09-14T22:36:59.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5520000000000001e-06 2024-09-14T22:36:59.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:59.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514641+0000", 2024-09-14T22:36:59.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6199999999999998e-06 2024-09-14T22:36:59.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:59.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559679+0000", 2024-09-14T22:36:59.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045038474000000002 2024-09-14T22:36:59.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:59.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559763+0000", 2024-09-14T22:36:59.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3357999999999997e-05 2024-09-14T22:36:59.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:59.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559859+0000", 2024-09-14T22:36:59.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6420000000000002e-05 2024-09-14T22:36:59.659 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.659 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:59.659 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.660 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35565 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1456128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:59.660 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514648+0000", 2024-09-14T22:36:59.660 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.069373770999999, 2024-09-14T22:36:59.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045442455999999999, 2024-09-14T22:36:59.660 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:59.660 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:59.660 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:59.660 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:59.660 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:59.660 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:59.660 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35565 2024-09-14T22:36:59.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.660 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:59.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:59.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514648+0000", 2024-09-14T22:36:59.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:59.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514648+0000", 2024-09-14T22:36:59.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:59.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514648+0000", 2024-09-14T22:36:59.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1399999999999997e-07 2024-09-14T22:36:59.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:59.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514651+0000", 2024-09-14T22:36:59.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.136e-06 2024-09-14T22:36:59.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:59.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514657+0000", 2024-09-14T22:36:59.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.241e-06 2024-09-14T22:36:59.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:59.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514662+0000", 2024-09-14T22:36:59.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9690000000000002e-06 2024-09-14T22:36:59.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:59.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559903+0000", 2024-09-14T22:36:59.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045241393999999997 2024-09-14T22:36:59.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:59.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.559985+0000", 2024-09-14T22:36:59.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1460999999999998e-05 2024-09-14T22:36:59.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:59.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560090+0000", 2024-09-14T22:36:59.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010584100000000001 2024-09-14T22:36:59.664 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.664 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:59.665 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.665 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35566 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1484800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:59.665 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514668+0000", 2024-09-14T22:36:59.665 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.069353347, 2024-09-14T22:36:59.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045644974999999997, 2024-09-14T22:36:59.665 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:59.665 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:59.665 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:59.665 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:59.665 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:59.665 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:59.666 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35566 2024-09-14T22:36:59.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.666 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:59.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.666 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:59.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514668+0000", 2024-09-14T22:36:59.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.666 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:59.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514668+0000", 2024-09-14T22:36:59.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:59.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514669+0000", 2024-09-14T22:36:59.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9099999999999999e-07 2024-09-14T22:36:59.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:59.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514671+0000", 2024-09-14T22:36:59.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3800000000000001e-06 2024-09-14T22:36:59.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:59.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514677+0000", 2024-09-14T22:36:59.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9660000000000001e-06 2024-09-14T22:36:59.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:59.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514682+0000", 2024-09-14T22:36:59.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0939999999999998e-06 2024-09-14T22:36:59.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:59.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560133+0000", 2024-09-14T22:36:59.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045450398000000003 2024-09-14T22:36:59.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:59.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560207+0000", 2024-09-14T22:36:59.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4196999999999995e-05 2024-09-14T22:36:59.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:59.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560313+0000", 2024-09-14T22:36:59.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106549 2024-09-14T22:36:59.670 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.670 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:59.670 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.670 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35567 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1513472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:59.671 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:36:59.671 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.069332807999999, 2024-09-14T22:36:59.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045793215999999998, 2024-09-14T22:36:59.671 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:59.671 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:59.671 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:59.671 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:59.671 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:59.671 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:59.671 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35567 2024-09-14T22:36:59.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.671 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:59.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:59.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:36:59.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:59.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:36:59.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:59.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:36:59.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1899999999999998e-07 2024-09-14T22:36:59.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:59.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514692+0000", 2024-09-14T22:36:59.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2699999999999999e-06 2024-09-14T22:36:59.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:59.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514698+0000", 2024-09-14T22:36:59.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0939999999999996e-06 2024-09-14T22:36:59.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:59.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514703+0000", 2024-09-14T22:36:59.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3870000000000003e-06 2024-09-14T22:36:59.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:59.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560353+0000", 2024-09-14T22:36:59.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045649914 2024-09-14T22:36:59.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:59.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560416+0000", 2024-09-14T22:36:59.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3096999999999996e-05 2024-09-14T22:36:59.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:59.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560482+0000", 2024-09-14T22:36:59.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6035e-05 2024-09-14T22:36:59.675 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.675 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:59.675 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.675 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35568 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1542144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:59.675 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:36:59.676 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.06930801, 2024-09-14T22:36:59.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045995921000000002, 2024-09-14T22:36:59.676 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:59.676 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:59.676 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:59.676 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:59.676 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:59.676 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:59.676 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35568 2024-09-14T22:36:59.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.676 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:59.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:59.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:36:59.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:59.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:36:59.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:59.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:36:59.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4099999999999999e-07 2024-09-14T22:36:59.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:59.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514720+0000", 2024-09-14T22:36:59.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6869999999999999e-06 2024-09-14T22:36:59.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:59.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514727+0000", 2024-09-14T22:36:59.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3540000000000002e-06 2024-09-14T22:36:59.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:59.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514732+0000", 2024-09-14T22:36:59.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7690000000000004e-06 2024-09-14T22:36:59.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:59.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560523+0000", 2024-09-14T22:36:59.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045791373000000003 2024-09-14T22:36:59.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:59.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560606+0000", 2024-09-14T22:36:59.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2690999999999998e-05 2024-09-14T22:36:59.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:59.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560710+0000", 2024-09-14T22:36:59.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000103606 2024-09-14T22:36:59.680 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.680 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:59.680 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.680 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35569 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1570816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:36:59.680 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:36:59.680 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.069282201, 2024-09-14T22:36:59.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046110630999999999, 2024-09-14T22:36:59.681 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:36:59.681 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:36:59.681 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:36:59.681 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:36:59.681 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:36:59.681 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:36:59.681 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35569 2024-09-14T22:36:59.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.681 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:36:59.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.681 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:36:59.681 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:36:59.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:36:59.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:36:59.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:36:59.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:36:59.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:36:59.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.82e-07 2024-09-14T22:36:59.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:36:59.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514743+0000", 2024-09-14T22:36:59.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.269e-06 2024-09-14T22:36:59.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:36:59.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514750+0000", 2024-09-14T22:36:59.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7340000000000002e-06 2024-09-14T22:36:59.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:36:59.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514755+0000", 2024-09-14T22:36:59.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7260000000000002e-06 2024-09-14T22:36:59.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:36:59.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560731+0000", 2024-09-14T22:36:59.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045976377999999998 2024-09-14T22:36:59.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:36:59.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560777+0000", 2024-09-14T22:36:59.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5868999999999997e-05 2024-09-14T22:36:59.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:36:59.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:36:59.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:36:59.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560850+0000", 2024-09-14T22:36:59.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3172999999999999e-05 2024-09-14T22:36:59.685 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.685 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:59.685 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.685 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:36:59.685 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:36:59.685 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:36:59.687 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:36:59.687 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:59.683+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:59.787 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:36:59.783+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:36:59.813 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:59.813 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:59.813 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:36:59.813 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:59.827 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:36:59.848 INFO:teuthology.orchestra.run.smithi001.stdout:2077 write 0x30b1948 thru 0x30b2280 (0x939 bytes) 2024-09-14T22:36:59.860 INFO:teuthology.orchestra.run.smithi001.stdout:2081 writesame 0xa41cc78 thru 0xa427415 data_size 0x23(0xa79e bytes) 2024-09-14T22:36:59.882 INFO:teuthology.orchestra.run.smithi001.stdout:2083 trunc from 0xbfb1548 to 0x511f642 2024-09-14T22:36:59.888 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:36:59.883+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:36:59.916 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:36:59.916 INFO:teuthology.orchestra.run.smithi001.stdout:2085 punch from 0xfe31a to 0x103f5f, (0x5c45 bytes) 2024-09-14T22:36:59.916 INFO:teuthology.orchestra.run.smithi001.stdout:2086 read 0x12138a9 thru 0x122269d (0xedf5 bytes) 2024-09-14T22:36:59.931 INFO:teuthology.orchestra.run.smithi001.stdout:2087 write 0x694caf1 thru 0x694f088 (0x2598 bytes) 2024-09-14T22:36:59.958 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:36:59.959 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:36:59.959 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:36:59.959 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:36:59.959 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:36:59.960 INFO:teuthology.orchestra.run.smithi001.stdout:2088 write 0xc0206ce thru 0xc022a24 (0x2357 bytes) 2024-09-14T22:36:59.967 INFO:teuthology.orchestra.run.smithi001.stdout:2089 read 0x2a39c39 thru 0x2a3d4c0 (0x3888 bytes) 2024-09-14T22:36:59.967 INFO:teuthology.orchestra.run.smithi001.stdout:2090 write 0x1e15289 thru 0x1e17e95 (0x2c0d bytes) 2024-09-14T22:36:59.974 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:36:59.975 INFO:teuthology.orchestra.run.smithi001.stdout:2091 write 0xa965951 thru 0xa971cf9 (0xc3a9 bytes) 2024-09-14T22:36:59.981 INFO:teuthology.orchestra.run.smithi001.stdout:2092 read 0x33d8843 thru 0x33db751 (0x2f0f bytes) 2024-09-14T22:36:59.981 INFO:teuthology.orchestra.run.smithi001.stdout:2095 read 0x92883f4 thru 0x92891d9 (0xde6 bytes) 2024-09-14T22:36:59.981 INFO:teuthology.orchestra.run.smithi001.stdout:2096 read 0x3cdc04a thru 0x3ce8ac6 (0xca7d bytes) 2024-09-14T22:36:59.981 INFO:teuthology.orchestra.run.smithi001.stdout:2097 writesame 0x80d267b thru 0x80dc2a4 data_size 0x9(0x9c2a bytes) 2024-09-14T22:36:59.988 INFO:teuthology.orchestra.run.smithi001.stdout:2098 writesame 0x90eb511 thru 0x90f210b data_size 0xb(0x6bfb bytes) 2024-09-14T22:36:59.995 INFO:teuthology.orchestra.run.smithi001.stdout:2100 read 0x4bce21c thru 0x4bda0d9 (0xbebe bytes) 2024-09-14T22:37:00.000 INFO:teuthology.orchestra.run.smithi001.stdout:2104 trunc from 0xc022a25 to 0xd541f10 2024-09-14T22:37:00.010 INFO:teuthology.orchestra.run.smithi001.stdout:2105 read 0xcba5999 thru 0xcbad0e5 (0x774d bytes) 2024-09-14T22:37:00.011 INFO:teuthology.orchestra.run.smithi001.stdout:2106 write 0x15836ad thru 0x1584efa (0x184e bytes) 2024-09-14T22:37:00.015 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:00.011+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:00.032 INFO:teuthology.orchestra.run.smithi001.stdout:2108 write 0xa2d0623 thru 0xa2d23c7 (0x1da5 bytes) 2024-09-14T22:37:00.036 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [2] live_osds: [0, 1, 3, 4, 5, 6, 7] 2024-09-14T22:37:00.036 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:37:00.036 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-14T22:37:00.036 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T22:37:00.036 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T22:37:00.036 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-14T22:37:00.037 INFO:teuthology.orchestra.run.smithi001.stdout:2109 read 0xb58cd1e thru 0xb59a67d (0xd960 bytes) 2024-09-14T22:37:00.037 INFO:teuthology.orchestra.run.smithi001.stdout:2110 read 0x39f40bc thru 0x3a03084 (0xefc9 bytes) 2024-09-14T22:37:00.038 INFO:teuthology.orchestra.run.smithi001.stdout:2111 punch from 0x98690a4 to 0x9873474, (0xa3d0 bytes) 2024-09-14T22:37:00.047 INFO:teuthology.orchestra.run.smithi001.stdout:2113 trunc from 0xd541f10 to 0xa8cb3fd 2024-09-14T22:37:00.050 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2024-09-14T22:37:00.075 INFO:teuthology.orchestra.run.smithi001.stdout:2114 write 0xb2e2c64 thru 0xb2eb7a4 (0x8b41 bytes) 2024-09-14T22:37:00.094 INFO:teuthology.orchestra.run.smithi001.stdout:2115 trunc from 0xb2eb7a5 to 0x2da6a87 2024-09-14T22:37:00.094 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:00.091+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:00.123 INFO:teuthology.orchestra.run.smithi001.stdout:2116 read 0x1eb44c thru 0x1f3472 (0x8027 bytes) 2024-09-14T22:37:00.124 INFO:teuthology.orchestra.run.smithi001.stdout:2117 punch from 0xd779f8 to 0xd7b5f8, (0x3c00 bytes) 2024-09-14T22:37:00.124 INFO:teuthology.orchestra.run.smithi001.stdout:2118 read 0x396880 thru 0x3a5c83 (0xf404 bytes) 2024-09-14T22:37:00.126 INFO:teuthology.orchestra.run.smithi001.stdout:2119 read 0x1c98045 thru 0x1c9fe26 (0x7de2 bytes) 2024-09-14T22:37:00.127 INFO:teuthology.orchestra.run.smithi001.stdout:2121 punch from 0x2b4fd07 to 0x2b5b52f, (0xb828 bytes) 2024-09-14T22:37:00.140 INFO:teuthology.orchestra.run.smithi001.stdout:2122 trunc from 0x2da6a87 to 0xc54c09e 2024-09-14T22:37:00.150 INFO:teuthology.orchestra.run.smithi001.stdout:2123 read 0x88e9424 thru 0x88f6e7c (0xda59 bytes) 2024-09-14T22:37:00.152 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:00.152 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:00.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:00.152 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:00.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.152 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6376 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8230912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:00.153 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932578+0000", 2024-09-14T22:37:00.153 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.215079848999999, 2024-09-14T22:37:00.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051558771000000003, 2024-09-14T22:37:00.153 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.153 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.153 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.153 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.153 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.153 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.153 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6376 2024-09-14T22:37:00.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.153 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932578+0000", 2024-09-14T22:37:00.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932578+0000", 2024-09-14T22:37:00.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932579+0000", 2024-09-14T22:37:00.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8599999999999996e-07 2024-09-14T22:37:00.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932597+0000", 2024-09-14T22:37:00.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7771e-05 2024-09-14T22:37:00.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932618+0000", 2024-09-14T22:37:00.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1026000000000002e-05 2024-09-14T22:37:00.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932635+0000", 2024-09-14T22:37:00.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.702e-05 2024-09-14T22:37:00.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.983951+0000", 2024-09-14T22:37:00.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051316639999999997 2024-09-14T22:37:00.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984031+0000", 2024-09-14T22:37:00.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0085000000000003e-05 2024-09-14T22:37:00.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984137+0000", 2024-09-14T22:37:00.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105243 2024-09-14T22:37:00.157 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.157 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.157 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.157 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6377 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8239104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:00.157 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932688+0000", 2024-09-14T22:37:00.157 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.214969818, 2024-09-14T22:37:00.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051650696000000003, 2024-09-14T22:37:00.157 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.157 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.157 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.157 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.158 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.158 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.158 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6377 2024-09-14T22:37:00.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.158 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932688+0000", 2024-09-14T22:37:00.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932688+0000", 2024-09-14T22:37:00.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932689+0000", 2024-09-14T22:37:00.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4309999999999999e-06 2024-09-14T22:37:00.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932695+0000", 2024-09-14T22:37:00.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8889999999999999e-06 2024-09-14T22:37:00.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932713+0000", 2024-09-14T22:37:00.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7416999999999999e-05 2024-09-14T22:37:00.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932726+0000", 2024-09-14T22:37:00.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3145e-05 2024-09-14T22:37:00.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984181+0000", 2024-09-14T22:37:00.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051455630000000002 2024-09-14T22:37:00.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984247+0000", 2024-09-14T22:37:00.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5350000000000003e-05 2024-09-14T22:37:00.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.161 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984339+0000", 2024-09-14T22:37:00.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1834000000000002e-05 2024-09-14T22:37:00.161 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.161 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.161 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.161 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6378 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8247296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:00.161 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932743+0000", 2024-09-14T22:37:00.162 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.214914583999999, 2024-09-14T22:37:00.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051819373000000002, 2024-09-14T22:37:00.162 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.162 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.162 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.162 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.162 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.162 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.162 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6378 2024-09-14T22:37:00.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.162 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932743+0000", 2024-09-14T22:37:00.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932743+0000", 2024-09-14T22:37:00.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932745+0000", 2024-09-14T22:37:00.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.637e-06 2024-09-14T22:37:00.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932750+0000", 2024-09-14T22:37:00.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7859999999999996e-06 2024-09-14T22:37:00.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932767+0000", 2024-09-14T22:37:00.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7621999999999998e-05 2024-09-14T22:37:00.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932781+0000", 2024-09-14T22:37:00.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3403e-05 2024-09-14T22:37:00.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984383+0000", 2024-09-14T22:37:00.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051602249000000003 2024-09-14T22:37:00.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984461+0000", 2024-09-14T22:37:00.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8467000000000006e-05 2024-09-14T22:37:00.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984562+0000", 2024-09-14T22:37:00.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101209 2024-09-14T22:37:00.166 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.166 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.166 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.166 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6379 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8255488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:00.166 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932796+0000", 2024-09-14T22:37:00.166 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.214861192999997, 2024-09-14T22:37:00.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051961245000000003, 2024-09-14T22:37:00.166 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.166 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.166 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.166 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.167 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.167 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.167 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6379 2024-09-14T22:37:00.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.167 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932796+0000", 2024-09-14T22:37:00.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932796+0000", 2024-09-14T22:37:00.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932797+0000", 2024-09-14T22:37:00.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9700000000000005e-07 2024-09-14T22:37:00.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932823+0000", 2024-09-14T22:37:00.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5273999999999999e-05 2024-09-14T22:37:00.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932840+0000", 2024-09-14T22:37:00.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7774e-05 2024-09-14T22:37:00.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932862+0000", 2024-09-14T22:37:00.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1957e-05 2024-09-14T22:37:00.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984603+0000", 2024-09-14T22:37:00.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051740543 2024-09-14T22:37:00.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984669+0000", 2024-09-14T22:37:00.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.567e-05 2024-09-14T22:37:00.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984758+0000", 2024-09-14T22:37:00.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9129999999999995e-05 2024-09-14T22:37:00.170 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.170 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.170 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.170 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6380 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8263680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:00.170 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932879+0000", 2024-09-14T22:37:00.171 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.214779114000002, 2024-09-14T22:37:00.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052102860000000001, 2024-09-14T22:37:00.171 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.171 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.171 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.171 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.171 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.171 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.171 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6380 2024-09-14T22:37:00.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.171 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932879+0000", 2024-09-14T22:37:00.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932879+0000", 2024-09-14T22:37:00.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932880+0000", 2024-09-14T22:37:00.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.054e-06 2024-09-14T22:37:00.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932885+0000", 2024-09-14T22:37:00.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1290000000000004e-06 2024-09-14T22:37:00.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932901+0000", 2024-09-14T22:37:00.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5908000000000001e-05 2024-09-14T22:37:00.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932913+0000", 2024-09-14T22:37:00.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2408999999999999e-05 2024-09-14T22:37:00.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984800+0000", 2024-09-14T22:37:00.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051886671000000002 2024-09-14T22:37:00.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984879+0000", 2024-09-14T22:37:00.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9200000000000001e-05 2024-09-14T22:37:00.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984981+0000", 2024-09-14T22:37:00.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102489 2024-09-14T22:37:00.174 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.174 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.174 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.175 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6381 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8271872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:00.175 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932929+0000", 2024-09-14T22:37:00.175 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.214728782999998, 2024-09-14T22:37:00.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052251484000000001, 2024-09-14T22:37:00.175 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.175 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.175 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.175 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.175 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.175 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.175 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6381 2024-09-14T22:37:00.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.175 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932929+0000", 2024-09-14T22:37:00.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932929+0000", 2024-09-14T22:37:00.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932930+0000", 2024-09-14T22:37:00.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3399999999999998e-07 2024-09-14T22:37:00.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932934+0000", 2024-09-14T22:37:00.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5719999999999996e-06 2024-09-14T22:37:00.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932961+0000", 2024-09-14T22:37:00.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6437000000000001e-05 2024-09-14T22:37:00.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932973+0000", 2024-09-14T22:37:00.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2097e-05 2024-09-14T22:37:00.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985021+0000", 2024-09-14T22:37:00.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052047955999999999 2024-09-14T22:37:00.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985089+0000", 2024-09-14T22:37:00.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7725999999999995e-05 2024-09-14T22:37:00.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985180+0000", 2024-09-14T22:37:00.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1861999999999996e-05 2024-09-14T22:37:00.179 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.179 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.179 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.179 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6382 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8280064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:00.179 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933013+0000", 2024-09-14T22:37:00.179 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.214644620999998, 2024-09-14T22:37:00.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052391118, 2024-09-14T22:37:00.179 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.179 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.180 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.180 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.180 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.180 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.180 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6382 2024-09-14T22:37:00.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.180 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933013+0000", 2024-09-14T22:37:00.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933013+0000", 2024-09-14T22:37:00.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933014+0000", 2024-09-14T22:37:00.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.375e-06 2024-09-14T22:37:00.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933020+0000", 2024-09-14T22:37:00.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0220000000000001e-06 2024-09-14T22:37:00.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933038+0000", 2024-09-14T22:37:00.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7694999999999999e-05 2024-09-14T22:37:00.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933051+0000", 2024-09-14T22:37:00.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2721999999999999e-05 2024-09-14T22:37:00.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985224+0000", 2024-09-14T22:37:00.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052172757 2024-09-14T22:37:00.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985303+0000", 2024-09-14T22:37:00.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9426999999999997e-05 2024-09-14T22:37:00.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985404+0000", 2024-09-14T22:37:00.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010111999999999999 2024-09-14T22:37:00.183 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.183 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.183 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.184 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6383 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8288256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:00.184 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933082+0000", 2024-09-14T22:37:00.184 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.214575641000003, 2024-09-14T22:37:00.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052525761999999997, 2024-09-14T22:37:00.184 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.184 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.184 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.184 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.184 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.184 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.184 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6383 2024-09-14T22:37:00.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.184 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933082+0000", 2024-09-14T22:37:00.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933082+0000", 2024-09-14T22:37:00.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933084+0000", 2024-09-14T22:37:00.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5659999999999999e-06 2024-09-14T22:37:00.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933089+0000", 2024-09-14T22:37:00.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.271e-06 2024-09-14T22:37:00.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933105+0000", 2024-09-14T22:37:00.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5685999999999999e-05 2024-09-14T22:37:00.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933118+0000", 2024-09-14T22:37:00.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3188e-05 2024-09-14T22:37:00.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985448+0000", 2024-09-14T22:37:00.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052330176999999999 2024-09-14T22:37:00.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985514+0000", 2024-09-14T22:37:00.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6029000000000001e-05 2024-09-14T22:37:00.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985608+0000", 2024-09-14T22:37:00.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3844999999999993e-05 2024-09-14T22:37:00.187 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.188 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.188 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.188 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6384 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8296448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:00.188 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933134+0000", 2024-09-14T22:37:00.188 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.214523282000002, 2024-09-14T22:37:00.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052688835000000003, 2024-09-14T22:37:00.188 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.188 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.188 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.188 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.188 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.188 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.188 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6384 2024-09-14T22:37:00.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.189 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933134+0000", 2024-09-14T22:37:00.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933134+0000", 2024-09-14T22:37:00.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933135+0000", 2024-09-14T22:37:00.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.78e-07 2024-09-14T22:37:00.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933141+0000", 2024-09-14T22:37:00.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1540000000000001e-06 2024-09-14T22:37:00.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933156+0000", 2024-09-14T22:37:00.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5488000000000001e-05 2024-09-14T22:37:00.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933169+0000", 2024-09-14T22:37:00.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2649e-05 2024-09-14T22:37:00.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985650+0000", 2024-09-14T22:37:00.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052481112000000003 2024-09-14T22:37:00.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985724+0000", 2024-09-14T22:37:00.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3953000000000002e-05 2024-09-14T22:37:00.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985823+0000", 2024-09-14T22:37:00.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9500999999999995e-05 2024-09-14T22:37:00.192 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.192 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.192 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.192 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6385 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8304640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:00.192 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933185+0000", 2024-09-14T22:37:00.192 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.214472579000002, 2024-09-14T22:37:00.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052892844000000001, 2024-09-14T22:37:00.193 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.193 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.193 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.193 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.193 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.193 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.193 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6385 2024-09-14T22:37:00.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.193 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933185+0000", 2024-09-14T22:37:00.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933185+0000", 2024-09-14T22:37:00.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933186+0000", 2024-09-14T22:37:00.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0139999999999999e-06 2024-09-14T22:37:00.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933191+0000", 2024-09-14T22:37:00.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7230000000000004e-06 2024-09-14T22:37:00.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933207+0000", 2024-09-14T22:37:00.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.658e-05 2024-09-14T22:37:00.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933220+0000", 2024-09-14T22:37:00.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2583e-05 2024-09-14T22:37:00.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985880+0000", 2024-09-14T22:37:00.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052660140000000001 2024-09-14T22:37:00.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985965+0000", 2024-09-14T22:37:00.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4472e-05 2024-09-14T22:37:00.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986078+0000", 2024-09-14T22:37:00.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011333200000000001 2024-09-14T22:37:00.196 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.196 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.196 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.196 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6386 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8312832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:00.196 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933236+0000", 2024-09-14T22:37:00.197 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.214421684999998, 2024-09-14T22:37:00.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053075342999999997, 2024-09-14T22:37:00.197 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.197 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.197 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.197 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.197 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.197 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.197 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6386 2024-09-14T22:37:00.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.197 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933236+0000", 2024-09-14T22:37:00.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933236+0000", 2024-09-14T22:37:00.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933237+0000", 2024-09-14T22:37:00.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6500000000000008e-07 2024-09-14T22:37:00.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933242+0000", 2024-09-14T22:37:00.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.938e-06 2024-09-14T22:37:00.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933258+0000", 2024-09-14T22:37:00.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6334e-05 2024-09-14T22:37:00.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933270+0000", 2024-09-14T22:37:00.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1973999999999999e-05 2024-09-14T22:37:00.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986123+0000", 2024-09-14T22:37:00.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052852769000000001 2024-09-14T22:37:00.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986204+0000", 2024-09-14T22:37:00.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1458999999999994e-05 2024-09-14T22:37:00.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986311+0000", 2024-09-14T22:37:00.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010690400000000001 2024-09-14T22:37:00.200 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.200 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.200 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.201 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6387 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8321024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:00.201 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933335+0000", 2024-09-14T22:37:00.201 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.214322414999998, 2024-09-14T22:37:00.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053182363000000003, 2024-09-14T22:37:00.201 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.201 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.201 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.201 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.201 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.201 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.201 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6387 2024-09-14T22:37:00.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.202 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933335+0000", 2024-09-14T22:37:00.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933335+0000", 2024-09-14T22:37:00.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933337+0000", 2024-09-14T22:37:00.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3829999999999999e-06 2024-09-14T22:37:00.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933343+0000", 2024-09-14T22:37:00.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1489999999999999e-06 2024-09-14T22:37:00.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933361+0000", 2024-09-14T22:37:00.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8105e-05 2024-09-14T22:37:00.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933374+0000", 2024-09-14T22:37:00.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.33e-05 2024-09-14T22:37:00.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986355+0000", 2024-09-14T22:37:00.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052981053 2024-09-14T22:37:00.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986424+0000", 2024-09-14T22:37:00.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8864999999999999e-05 2024-09-14T22:37:00.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986518+0000", 2024-09-14T22:37:00.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3508e-05 2024-09-14T22:37:00.205 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.205 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.205 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.205 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6388 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8329216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:00.206 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933390+0000", 2024-09-14T22:37:00.206 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.214267829000001, 2024-09-14T22:37:00.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053371605000000003, 2024-09-14T22:37:00.206 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.206 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.206 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.206 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.206 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.206 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.206 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6388 2024-09-14T22:37:00.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.206 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933390+0000", 2024-09-14T22:37:00.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933390+0000", 2024-09-14T22:37:00.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933391+0000", 2024-09-14T22:37:00.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.443e-06 2024-09-14T22:37:00.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.208 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933396+0000", 2024-09-14T22:37:00.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1680000000000003e-06 2024-09-14T22:37:00.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.208 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933412+0000", 2024-09-14T22:37:00.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5302999999999999e-05 2024-09-14T22:37:00.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.208 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933439+0000", 2024-09-14T22:37:00.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6747999999999999e-05 2024-09-14T22:37:00.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986562+0000", 2024-09-14T22:37:00.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053123161000000002 2024-09-14T22:37:00.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986641+0000", 2024-09-14T22:37:00.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9086999999999997e-05 2024-09-14T22:37:00.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986761+0000", 2024-09-14T22:37:00.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120695 2024-09-14T22:37:00.209 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.210 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.210 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.210 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6389 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8337408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:00.210 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933456+0000", 2024-09-14T22:37:00.210 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.214201977999998, 2024-09-14T22:37:00.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053514182, 2024-09-14T22:37:00.210 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.210 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.210 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.210 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.210 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.210 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.210 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6389 2024-09-14T22:37:00.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.211 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933456+0000", 2024-09-14T22:37:00.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933456+0000", 2024-09-14T22:37:00.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933457+0000", 2024-09-14T22:37:00.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0020000000000001e-06 2024-09-14T22:37:00.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.212 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.212 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933462+0000", 2024-09-14T22:37:00.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8400000000000002e-06 2024-09-14T22:37:00.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.212 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.212 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933477+0000", 2024-09-14T22:37:00.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.592e-05 2024-09-14T22:37:00.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.212 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.212 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933489+0000", 2024-09-14T22:37:00.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1955e-05 2024-09-14T22:37:00.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986798+0000", 2024-09-14T22:37:00.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053308263000000002 2024-09-14T22:37:00.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986871+0000", 2024-09-14T22:37:00.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3518999999999999e-05 2024-09-14T22:37:00.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986970+0000", 2024-09-14T22:37:00.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8683000000000004e-05 2024-09-14T22:37:00.214 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.214 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.214 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.214 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6390 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8345600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:00.214 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933504+0000", 2024-09-14T22:37:00.214 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.214153193000001, 2024-09-14T22:37:00.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053719579000000003, 2024-09-14T22:37:00.214 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.214 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.214 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.214 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.214 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.215 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.215 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6390 2024-09-14T22:37:00.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.215 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933504+0000", 2024-09-14T22:37:00.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933504+0000", 2024-09-14T22:37:00.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933505+0000", 2024-09-14T22:37:00.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8299999999999995e-07 2024-09-14T22:37:00.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933520+0000", 2024-09-14T22:37:00.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4948e-05 2024-09-14T22:37:00.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933538+0000", 2024-09-14T22:37:00.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7286e-05 2024-09-14T22:37:00.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933550+0000", 2024-09-14T22:37:00.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2223e-05 2024-09-14T22:37:00.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987028+0000", 2024-09-14T22:37:00.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053478500999999998 2024-09-14T22:37:00.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987111+0000", 2024-09-14T22:37:00.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3158999999999994e-05 2024-09-14T22:37:00.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987224+0000", 2024-09-14T22:37:00.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011257899999999999 2024-09-14T22:37:00.218 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.218 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.218 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.218 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6391 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8353792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:00.218 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933576+0000", 2024-09-14T22:37:00.218 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.214081747999998, 2024-09-14T22:37:00.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053858196999999997, 2024-09-14T22:37:00.219 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.219 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.219 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.219 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.219 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.219 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.219 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6391 2024-09-14T22:37:00.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.219 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933576+0000", 2024-09-14T22:37:00.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933576+0000", 2024-09-14T22:37:00.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933578+0000", 2024-09-14T22:37:00.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9539999999999998e-06 2024-09-14T22:37:00.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933584+0000", 2024-09-14T22:37:00.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9030000000000001e-06 2024-09-14T22:37:00.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933601+0000", 2024-09-14T22:37:00.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7703e-05 2024-09-14T22:37:00.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933614+0000", 2024-09-14T22:37:00.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2330999999999999e-05 2024-09-14T22:37:00.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987275+0000", 2024-09-14T22:37:00.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053661470000000003 2024-09-14T22:37:00.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987342+0000", 2024-09-14T22:37:00.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6266000000000006e-05 2024-09-14T22:37:00.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987434+0000", 2024-09-14T22:37:00.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2570000000000003e-05 2024-09-14T22:37:00.222 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.222 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.222 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.222 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6392 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8361984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:00.222 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933648+0000", 2024-09-14T22:37:00.223 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.214009359999999, 2024-09-14T22:37:00.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054032430999999999, 2024-09-14T22:37:00.223 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.223 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.223 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.223 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.223 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.223 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.223 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6392 2024-09-14T22:37:00.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.223 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933648+0000", 2024-09-14T22:37:00.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933648+0000", 2024-09-14T22:37:00.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933650+0000", 2024-09-14T22:37:00.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3969999999999999e-06 2024-09-14T22:37:00.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933655+0000", 2024-09-14T22:37:00.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7690000000000002e-06 2024-09-14T22:37:00.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933672+0000", 2024-09-14T22:37:00.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6359999999999999e-05 2024-09-14T22:37:00.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933696+0000", 2024-09-14T22:37:00.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4630999999999998e-05 2024-09-14T22:37:00.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987490+0000", 2024-09-14T22:37:00.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053793256999999997 2024-09-14T22:37:00.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987572+0000", 2024-09-14T22:37:00.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1897999999999994e-05 2024-09-14T22:37:00.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987681+0000", 2024-09-14T22:37:00.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109119 2024-09-14T22:37:00.226 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.226 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.226 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.227 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6393 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8370176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:00.227 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933713+0000", 2024-09-14T22:37:00.227 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.213944406000003, 2024-09-14T22:37:00.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054194111000000003, 2024-09-14T22:37:00.227 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.227 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.227 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.227 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.227 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.227 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.227 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6393 2024-09-14T22:37:00.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.227 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933713+0000", 2024-09-14T22:37:00.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933713+0000", 2024-09-14T22:37:00.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933714+0000", 2024-09-14T22:37:00.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2109999999999999e-06 2024-09-14T22:37:00.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933721+0000", 2024-09-14T22:37:00.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2489999999999998e-06 2024-09-14T22:37:00.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933751+0000", 2024-09-14T22:37:00.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0239999999999998e-05 2024-09-14T22:37:00.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933764+0000", 2024-09-14T22:37:00.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.322e-05 2024-09-14T22:37:00.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987737+0000", 2024-09-14T22:37:00.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053972539 2024-09-14T22:37:00.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987813+0000", 2024-09-14T22:37:00.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6635000000000001e-05 2024-09-14T22:37:00.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987907+0000", 2024-09-14T22:37:00.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4017000000000005e-05 2024-09-14T22:37:00.230 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.230 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.230 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.231 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6394 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8378368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:00.231 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933781+0000", 2024-09-14T22:37:00.231 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.213876646999999, 2024-09-14T22:37:00.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054378276000000003, 2024-09-14T22:37:00.231 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.231 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.231 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.231 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.231 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.231 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.231 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6394 2024-09-14T22:37:00.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.231 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933781+0000", 2024-09-14T22:37:00.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933781+0000", 2024-09-14T22:37:00.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933782+0000", 2024-09-14T22:37:00.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9199999999999999e-07 2024-09-14T22:37:00.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933788+0000", 2024-09-14T22:37:00.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9340000000000003e-06 2024-09-14T22:37:00.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933803+0000", 2024-09-14T22:37:00.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5362999999999998e-05 2024-09-14T22:37:00.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933816+0000", 2024-09-14T22:37:00.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2456999999999999e-05 2024-09-14T22:37:00.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987961+0000", 2024-09-14T22:37:00.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054145581999999998 2024-09-14T22:37:00.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988042+0000", 2024-09-14T22:37:00.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1075000000000006e-05 2024-09-14T22:37:00.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988159+0000", 2024-09-14T22:37:00.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116873 2024-09-14T22:37:00.235 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.235 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.235 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.235 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6395 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8386560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:00.235 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:37:00.235 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.213825753000002, 2024-09-14T22:37:00.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054534379000000001, 2024-09-14T22:37:00.236 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.236 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.236 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.236 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.236 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.236 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.236 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6395 2024-09-14T22:37:00.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.236 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:37:00.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:37:00.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933833+0000", 2024-09-14T22:37:00.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3859999999999999e-06 2024-09-14T22:37:00.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933838+0000", 2024-09-14T22:37:00.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8709999999999996e-06 2024-09-14T22:37:00.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933854+0000", 2024-09-14T22:37:00.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6189e-05 2024-09-14T22:37:00.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933871+0000", 2024-09-14T22:37:00.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.626e-05 2024-09-14T22:37:00.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988198+0000", 2024-09-14T22:37:00.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054327581999999999 2024-09-14T22:37:00.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988274+0000", 2024-09-14T22:37:00.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5759000000000004e-05 2024-09-14T22:37:00.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988366+0000", 2024-09-14T22:37:00.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2331999999999996e-05 2024-09-14T22:37:00.239 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.239 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.239 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.239 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.239 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.239 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:00.241 INFO:teuthology.orchestra.run.smithi001.stdout:2124 writesame 0xd1ea3f5 thru 0xd1f9544 data_size 0x18(0xf150 bytes) 2024-09-14T22:37:00.241 INFO:teuthology.orchestra.run.smithi001.stdout:2125 trunc from 0xd1f9545 to 0x9e4089e 2024-09-14T22:37:00.241 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:00.241 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T22:37:00.241 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:00.242 INFO:teuthology.orchestra.run.smithi001.stdout:2127 trunc from 0x9e4089e to 0x19c5d22 2024-09-14T22:37:00.242 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:37:00.242 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:00.235+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:00.243 INFO:teuthology.orchestra.run.smithi001.stdout:2128 flatten 2024-09-14T22:37:00.341 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:00.335+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:00.352 INFO:teuthology.orchestra.run.smithi001.stdout:2129 write 0xca966c1 thru 0xcaa3ac6 (0xd406 bytes) 2024-09-14T22:37:00.358 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:00.367 INFO:teuthology.orchestra.run.smithi001.stdout:2131 trunc from 0xcaa3ac7 to 0x77d3c7c 2024-09-14T22:37:00.382 INFO:teuthology.orchestra.run.smithi001.stdout:2132 write 0x29614db thru 0x296e38e (0xceb4 bytes) 2024-09-14T22:37:00.386 INFO:teuthology.orchestra.run.smithi001.stdout:2133 write 0x7449f44 thru 0x74561dc (0xc299 bytes) 2024-09-14T22:37:00.390 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:00.390 INFO:teuthology.orchestra.run.smithi001.stdout:2135 read 0x633680a thru 0x633c991 (0x6188 bytes) 2024-09-14T22:37:00.390 INFO:teuthology.orchestra.run.smithi001.stdout:2137 read 0xea6d4 thru 0xf8778 (0xe0a5 bytes) 2024-09-14T22:37:00.390 INFO:teuthology.orchestra.run.smithi001.stdout:2138 read 0x70ac976 thru 0x70b57de (0x8e69 bytes) 2024-09-14T22:37:00.395 INFO:teuthology.orchestra.run.smithi001.stdout:2139 trunc from 0x77d3c7c to 0x4b433ea 2024-09-14T22:37:00.410 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:00.410 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:00.410 INFO:teuthology.orchestra.run.smithi001.stdout:2142 read 0x3cb7d7 thru 0x3cd843 (0x206d bytes) 2024-09-14T22:37:00.411 INFO:teuthology.orchestra.run.smithi001.stdout:2143 trunc from 0x4b433ea to 0x476c90d 2024-09-14T22:37:00.412 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:00.412 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:00.412 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:37:00.412 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:00.420 INFO:teuthology.orchestra.run.smithi001.stdout:2145 read 0x368274b thru 0x368c68d (0x9f43 bytes) 2024-09-14T22:37:00.427 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:37:00.452 INFO:teuthology.orchestra.run.smithi001.stdout:2146 writesame 0xedc673b thru 0xedccc81 data_size 0x1(0x6547 bytes) 2024-09-14T22:37:00.452 INFO:teuthology.orchestra.run.smithi001.stdout:2147 writesame 0x74b19d8 thru 0x74bd947 data_size 0x18(0xbf70 bytes) 2024-09-14T22:37:00.458 INFO:teuthology.orchestra.run.smithi001.stdout:2151 punch from 0x37b8fd4 to 0x37b9e5e, (0xe8a bytes) 2024-09-14T22:37:00.458 INFO:teuthology.orchestra.run.smithi001.stdout:2152 read 0xda695ec thru 0xda74282 (0xac97 bytes) 2024-09-14T22:37:00.458 INFO:teuthology.orchestra.run.smithi001.stdout:2153 punch from 0x99e19c to 0x9a7cb5, (0x9b19 bytes) 2024-09-14T22:37:00.458 INFO:teuthology.orchestra.run.smithi001.stdout:2154 read 0x925d007 thru 0x925e5ff (0x15f9 bytes) 2024-09-14T22:37:00.458 INFO:teuthology.orchestra.run.smithi001.stdout:2157 writesame 0xda00031 thru 0xda08b43 data_size 0x1(0x8b13 bytes) 2024-09-14T22:37:00.471 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:00.467+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:00.504 INFO:teuthology.orchestra.run.smithi001.stdout:2158 write 0xa954510 thru 0xa9568ca (0x23bb bytes) 2024-09-14T22:37:00.510 INFO:teuthology.orchestra.run.smithi001.stdout:2159 read 0xe680950 thru 0xe690539 (0xfbea bytes) 2024-09-14T22:37:00.511 INFO:teuthology.orchestra.run.smithi001.stdout:2160 writesame 0x1ed25e8 thru 0x1ed513b data_size 0x4(0x2b54 bytes) 2024-09-14T22:37:00.515 INFO:teuthology.orchestra.run.smithi001.stdout:2161 read 0xcad6efc thru 0xcadca88 (0x5b8d bytes) 2024-09-14T22:37:00.515 INFO:teuthology.orchestra.run.smithi001.stdout:2162 write 0xa4e9654 thru 0xa4f0074 (0x6a21 bytes) 2024-09-14T22:37:00.522 INFO:teuthology.orchestra.run.smithi001.stdout:2163 writesame 0x2e60d62 thru 0x2e6c262 data_size 0x1(0xb501 bytes) 2024-09-14T22:37:00.543 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:00.539+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:00.575 INFO:teuthology.orchestra.run.smithi001.stdout:2164 punch from 0x1435ea1 to 0x143b44e, (0x55ad bytes) 2024-09-14T22:37:00.575 INFO:teuthology.orchestra.run.smithi001.stdout:2165 write 0x34fec6c thru 0x3504fad (0x6342 bytes) 2024-09-14T22:37:00.585 INFO:teuthology.orchestra.run.smithi001.stdout:2166 read 0x11843f1 thru 0x1188377 (0x3f87 bytes) 2024-09-14T22:37:00.586 INFO:teuthology.orchestra.run.smithi001.stdout:2167 read 0xb917fbb thru 0xb9199b1 (0x19f7 bytes) 2024-09-14T22:37:00.586 INFO:teuthology.orchestra.run.smithi001.stdout:2168 read 0x5a27fcb thru 0x5a2b226 (0x325c bytes) 2024-09-14T22:37:00.586 INFO:teuthology.orchestra.run.smithi001.stdout:2169 punch from 0xdd32eba to 0xdd4080d, (0xd953 bytes) 2024-09-14T22:37:00.586 INFO:teuthology.orchestra.run.smithi001.stdout:2170 read 0x1389a03 thru 0x1398aff (0xf0fd bytes) 2024-09-14T22:37:00.587 INFO:teuthology.orchestra.run.smithi001.stdout:2172 read 0x621bcf0 thru 0x621c686 (0x997 bytes) 2024-09-14T22:37:00.587 INFO:teuthology.orchestra.run.smithi001.stdout:2174 read 0x52e3672 thru 0x52e368c (0x1b bytes) 2024-09-14T22:37:00.588 INFO:teuthology.orchestra.run.smithi001.stdout:2175 read 0x15d876b thru 0x15e25fa (0x9e90 bytes) 2024-09-14T22:37:00.589 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:00.589 INFO:teuthology.orchestra.run.smithi001.stdout:2177 punch from 0x3b2abb5 to 0x3b35656, (0xaaa1 bytes) 2024-09-14T22:37:00.597 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:00.597 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:00.598 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:37:00.598 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:37:00.598 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:00.612 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:37:00.612 INFO:teuthology.orchestra.run.smithi001.stdout:2178 trunc from 0xedccc82 to 0x6947ed3 2024-09-14T22:37:00.633 INFO:teuthology.orchestra.run.smithi001.stdout:2182 read 0x1bb25bc thru 0x1bc1cfb (0xf740 bytes) 2024-09-14T22:37:00.634 INFO:teuthology.orchestra.run.smithi001.stdout:2183 punch from 0x17b7a71 to 0x17bf5ae, (0x7b3d bytes) 2024-09-14T22:37:00.644 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:00.639+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:00.662 INFO:teuthology.orchestra.run.smithi001.stdout:2184 write 0xc367047 thru 0xc36c6d7 (0x5691 bytes) 2024-09-14T22:37:00.668 INFO:teuthology.orchestra.run.smithi001.stdout:2185 write 0x645b659 thru 0x646164c (0x5ff4 bytes) 2024-09-14T22:37:00.676 INFO:teuthology.orchestra.run.smithi001.stdout:2190 punch from 0xbb6098c to 0xbb6b792, (0xae06 bytes) 2024-09-14T22:37:00.676 INFO:teuthology.orchestra.run.smithi001.stdout:2192 read 0x1e3bf78 thru 0x1e47655 (0xb6de bytes) 2024-09-14T22:37:00.676 INFO:teuthology.orchestra.run.smithi001.stdout:2193 read 0x6bbb1ba thru 0x6bc3519 (0x8360 bytes) 2024-09-14T22:37:00.676 INFO:teuthology.orchestra.run.smithi001.stdout:2195 read 0xc206cb2 thru 0xc20e30c (0x765b bytes) 2024-09-14T22:37:00.680 INFO:teuthology.orchestra.run.smithi001.stdout:2196 trunc from 0xc36c6d8 to 0xdccddac 2024-09-14T22:37:00.722 INFO:teuthology.orchestra.run.smithi001.stdout:2197 trunc from 0xdccddac to 0x1fbf632 2024-09-14T22:37:00.745 INFO:teuthology.orchestra.run.smithi001.stdout:2198 punch from 0x1c08fb4 to 0x1c0bc8a, (0x2cd6 bytes) 2024-09-14T22:37:00.745 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:00.739+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:00.773 INFO:teuthology.orchestra.run.smithi001.stdout:2199 trunc from 0x1fbf632 to 0xc887a8e 2024-09-14T22:37:00.790 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:00.790 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:00.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:00.790 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:00.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.790 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20869 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7168000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:37:00.790 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483633+0000", 2024-09-14T22:37:00.790 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.301746928, 2024-09-14T22:37:00.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028181490999999999, 2024-09-14T22:37:00.791 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.791 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.791 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.791 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.791 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.791 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.791 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20869 2024-09-14T22:37:00.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.791 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483633+0000", 2024-09-14T22:37:00.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483633+0000", 2024-09-14T22:37:00.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483634+0000", 2024-09-14T22:37:00.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2599999999999998e-07 2024-09-14T22:37:00.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483636+0000", 2024-09-14T22:37:00.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1679999999999998e-06 2024-09-14T22:37:00.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483642+0000", 2024-09-14T22:37:00.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2149999999999998e-06 2024-09-14T22:37:00.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483647+0000", 2024-09-14T22:37:00.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7679999999999999e-06 2024-09-14T22:37:00.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.510694+0000", 2024-09-14T22:37:00.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027047073000000001 2024-09-14T22:37:00.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.510771+0000", 2024-09-14T22:37:00.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6814e-05 2024-09-14T22:37:00.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511815+0000", 2024-09-14T22:37:00.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0010440270000000001 2024-09-14T22:37:00.795 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.795 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.795 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.795 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20870 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7196672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:37:00.795 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483652+0000", 2024-09-14T22:37:00.795 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.301727926000002, 2024-09-14T22:37:00.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028301248000000001, 2024-09-14T22:37:00.795 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.795 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.795 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.795 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.796 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.796 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.796 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20870 2024-09-14T22:37:00.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.796 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483652+0000", 2024-09-14T22:37:00.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483652+0000", 2024-09-14T22:37:00.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483653+0000", 2024-09-14T22:37:00.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3099999999999997e-07 2024-09-14T22:37:00.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483655+0000", 2024-09-14T22:37:00.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.877e-06 2024-09-14T22:37:00.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483661+0000", 2024-09-14T22:37:00.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6049999999999997e-06 2024-09-14T22:37:00.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483666+0000", 2024-09-14T22:37:00.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3410000000000002e-06 2024-09-14T22:37:00.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511854+0000", 2024-09-14T22:37:00.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028187773999999999 2024-09-14T22:37:00.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511906+0000", 2024-09-14T22:37:00.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2012000000000003e-05 2024-09-14T22:37:00.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511954+0000", 2024-09-14T22:37:00.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8007999999999998e-05 2024-09-14T22:37:00.799 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.799 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.800 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.800 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20871 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7225344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:37:00.800 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483671+0000", 2024-09-14T22:37:00.800 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.301709002999999, 2024-09-14T22:37:00.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028405392000000002, 2024-09-14T22:37:00.800 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.800 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.800 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.800 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.801 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.801 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.801 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20871 2024-09-14T22:37:00.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.801 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483671+0000", 2024-09-14T22:37:00.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483671+0000", 2024-09-14T22:37:00.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483672+0000", 2024-09-14T22:37:00.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:37:00.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483674+0000", 2024-09-14T22:37:00.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3190000000000002e-06 2024-09-14T22:37:00.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483680+0000", 2024-09-14T22:37:00.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6860000000000003e-06 2024-09-14T22:37:00.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483684+0000", 2024-09-14T22:37:00.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3769999999999998e-06 2024-09-14T22:37:00.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511982+0000", 2024-09-14T22:37:00.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028297478000000001 2024-09-14T22:37:00.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512027+0000", 2024-09-14T22:37:00.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5290000000000002e-05 2024-09-14T22:37:00.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512077+0000", 2024-09-14T22:37:00.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9876000000000002e-05 2024-09-14T22:37:00.804 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.804 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.804 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.805 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20872 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7254016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:37:00.805 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483689+0000", 2024-09-14T22:37:00.805 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.301690933, 2024-09-14T22:37:00.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028494049, 2024-09-14T22:37:00.805 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.805 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.806 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.806 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.806 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.806 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.806 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20872 2024-09-14T22:37:00.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.806 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483689+0000", 2024-09-14T22:37:00.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483689+0000", 2024-09-14T22:37:00.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483690+0000", 2024-09-14T22:37:00.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:37:00.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483692+0000", 2024-09-14T22:37:00.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.778e-06 2024-09-14T22:37:00.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483697+0000", 2024-09-14T22:37:00.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7069999999999997e-06 2024-09-14T22:37:00.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483702+0000", 2024-09-14T22:37:00.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4689999999999999e-06 2024-09-14T22:37:00.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512104+0000", 2024-09-14T22:37:00.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028401948999999999 2024-09-14T22:37:00.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512142+0000", 2024-09-14T22:37:00.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8114999999999997e-05 2024-09-14T22:37:00.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512183+0000", 2024-09-14T22:37:00.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1496999999999999e-05 2024-09-14T22:37:00.809 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.809 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.809 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.810 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20873 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7282688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:37:00.810 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483707+0000", 2024-09-14T22:37:00.810 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.301673219000001, 2024-09-14T22:37:00.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028581662000000001, 2024-09-14T22:37:00.810 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.810 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.810 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.810 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.810 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.810 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.810 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20873 2024-09-14T22:37:00.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.811 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483707+0000", 2024-09-14T22:37:00.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483707+0000", 2024-09-14T22:37:00.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483707+0000", 2024-09-14T22:37:00.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2500000000000001e-07 2024-09-14T22:37:00.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483709+0000", 2024-09-14T22:37:00.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.891e-06 2024-09-14T22:37:00.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483715+0000", 2024-09-14T22:37:00.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7019999999999998e-06 2024-09-14T22:37:00.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483720+0000", 2024-09-14T22:37:00.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4900000000000002e-06 2024-09-14T22:37:00.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512213+0000", 2024-09-14T22:37:00.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028493622 2024-09-14T22:37:00.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512248+0000", 2024-09-14T22:37:00.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5228000000000002e-05 2024-09-14T22:37:00.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512289+0000", 2024-09-14T22:37:00.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0404000000000001e-05 2024-09-14T22:37:00.814 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.814 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.814 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.814 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20874 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7311360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:37:00.814 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483725+0000", 2024-09-14T22:37:00.814 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.301655579999998, 2024-09-14T22:37:00.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028777193999999999, 2024-09-14T22:37:00.815 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.815 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.815 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.815 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.815 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.815 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.815 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20874 2024-09-14T22:37:00.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.815 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483725+0000", 2024-09-14T22:37:00.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483725+0000", 2024-09-14T22:37:00.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483725+0000", 2024-09-14T22:37:00.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8000000000000001e-07 2024-09-14T22:37:00.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483727+0000", 2024-09-14T22:37:00.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8920000000000001e-06 2024-09-14T22:37:00.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483733+0000", 2024-09-14T22:37:00.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6840000000000001e-06 2024-09-14T22:37:00.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483737+0000", 2024-09-14T22:37:00.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4440000000000001e-06 2024-09-14T22:37:00.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512326+0000", 2024-09-14T22:37:00.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028588612999999999 2024-09-14T22:37:00.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512399+0000", 2024-09-14T22:37:00.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2782999999999998e-05 2024-09-14T22:37:00.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512502+0000", 2024-09-14T22:37:00.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000103398 2024-09-14T22:37:00.819 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.819 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.819 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.819 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20912 3.1c 3:3b9f022e:::rbd_data.113c4558bb1.000000000000009a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 497664~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:37:00.819 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.484800+0000", 2024-09-14T22:37:00.819 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.300580052000001, 2024-09-14T22:37:00.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045787688999999999, 2024-09-14T22:37:00.819 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.819 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.819 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:00.819 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.819 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.820 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.820 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20912 2024-09-14T22:37:00.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.820 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484800+0000", 2024-09-14T22:37:00.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484800+0000", 2024-09-14T22:37:00.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484801+0000", 2024-09-14T22:37:00.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1939999999999999e-06 2024-09-14T22:37:00.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484809+0000", 2024-09-14T22:37:00.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2989999999999999e-06 2024-09-14T22:37:00.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484823+0000", 2024-09-14T22:37:00.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3986e-05 2024-09-14T22:37:00.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484831+0000", 2024-09-14T22:37:00.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4009999999999997e-06 2024-09-14T22:37:00.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512318+0000", 2024-09-14T22:37:00.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027487208999999999 2024-09-14T22:37:00.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512450+0000", 2024-09-14T22:37:00.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000131681 2024-09-14T22:37:00.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:37:00.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512666+0000", 2024-09-14T22:37:00.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021547 2024-09-14T22:37:00.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:00.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.513788+0000", 2024-09-14T22:37:00.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001122975 2024-09-14T22:37:00.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.824 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:00.824 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.530473+0000", 2024-09-14T22:37:00.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.016684984999999999 2024-09-14T22:37:00.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.824 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:00.824 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.530533+0000", 2024-09-14T22:37:00.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9296999999999998e-05 2024-09-14T22:37:00.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.824 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.824 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.530588+0000", 2024-09-14T22:37:00.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5192000000000003e-05 2024-09-14T22:37:00.825 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.825 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.825 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.825 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:30512 3.1d 3:b9e08d52:::rbd_data.113cda0183a0.0000000000000008:6 [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:00.825 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.331867+0000", 2024-09-14T22:37:00.825 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.453513097999998, 2024-09-14T22:37:00.825 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027901088000000001, 2024-09-14T22:37:00.825 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.825 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.825 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.825 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.825 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.826 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.826 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 30512 2024-09-14T22:37:00.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.826 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.826 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.826 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331867+0000", 2024-09-14T22:37:00.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.826 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.826 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331867+0000", 2024-09-14T22:37:00.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331868+0000", 2024-09-14T22:37:00.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8999999999999996e-07 2024-09-14T22:37:00.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331871+0000", 2024-09-14T22:37:00.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.163e-06 2024-09-14T22:37:00.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331880+0000", 2024-09-14T22:37:00.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3449999999999995e-06 2024-09-14T22:37:00.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331888+0000", 2024-09-14T22:37:00.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5320000000000001e-06 2024-09-14T22:37:00.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.358443+0000", 2024-09-14T22:37:00.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026554845000000001 2024-09-14T22:37:00.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.359649+0000", 2024-09-14T22:37:00.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001205991 2024-09-14T22:37:00.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.359768+0000", 2024-09-14T22:37:00.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119522 2024-09-14T22:37:00.829 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.829 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.829 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.829 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:30513 3.1d 3:b9e08d52:::rbd_data.113cda0183a0.0000000000000008:6 [read 456704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:00.829 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.331897+0000", 2024-09-14T22:37:00.830 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.453482871999999, 2024-09-14T22:37:00.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028098122999999999, 2024-09-14T22:37:00.830 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.830 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.830 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.830 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.830 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.830 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.830 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 30513 2024-09-14T22:37:00.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.830 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331897+0000", 2024-09-14T22:37:00.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331897+0000", 2024-09-14T22:37:00.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331898+0000", 2024-09-14T22:37:00.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:37:00.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.832 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331901+0000", 2024-09-14T22:37:00.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7889999999999999e-06 2024-09-14T22:37:00.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.832 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.832 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331910+0000", 2024-09-14T22:37:00.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.346e-06 2024-09-14T22:37:00.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.832 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.832 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331917+0000", 2024-09-14T22:37:00.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2239999999999998e-06 2024-09-14T22:37:00.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.359807+0000", 2024-09-14T22:37:00.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027889225 2024-09-14T22:37:00.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.359895+0000", 2024-09-14T22:37:00.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8721000000000007e-05 2024-09-14T22:37:00.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.359996+0000", 2024-09-14T22:37:00.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010022300000000001 2024-09-14T22:37:00.834 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.834 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.834 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.834 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:30514 3.1d 3:b9e08d52:::rbd_data.113cda0183a0.0000000000000008:6 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:00.834 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.331927+0000", 2024-09-14T22:37:00.834 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.453453059000001, 2024-09-14T22:37:00.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028476330000000001, 2024-09-14T22:37:00.834 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.834 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.834 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.834 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.834 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.835 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.835 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 30514 2024-09-14T22:37:00.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.835 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.835 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.835 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331927+0000", 2024-09-14T22:37:00.835 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.835 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.835 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331927+0000", 2024-09-14T22:37:00.835 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331928+0000", 2024-09-14T22:37:00.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5700000000000002e-07 2024-09-14T22:37:00.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331931+0000", 2024-09-14T22:37:00.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9909999999999998e-06 2024-09-14T22:37:00.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331940+0000", 2024-09-14T22:37:00.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.312e-06 2024-09-14T22:37:00.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.837 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.837 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331948+0000", 2024-09-14T22:37:00.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5630000000000003e-06 2024-09-14T22:37:00.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.837 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.837 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360032+0000", 2024-09-14T22:37:00.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028083877 2024-09-14T22:37:00.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.837 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.837 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360282+0000", 2024-09-14T22:37:00.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00025062100000000003 2024-09-14T22:37:00.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360404+0000", 2024-09-14T22:37:00.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000121409 2024-09-14T22:37:00.838 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.838 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.838 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.838 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:30515 3.1d 3:b9e08d52:::rbd_data.113cda0183a0.0000000000000008:6 [read 514048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:00.838 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.331957+0000", 2024-09-14T22:37:00.838 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.453423233999999, 2024-09-14T22:37:00.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028684729999999999, 2024-09-14T22:37:00.838 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.838 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.839 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.839 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.839 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.839 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.839 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 30515 2024-09-14T22:37:00.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.839 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331957+0000", 2024-09-14T22:37:00.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331957+0000", 2024-09-14T22:37:00.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331958+0000", 2024-09-14T22:37:00.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1200000000000003e-07 2024-09-14T22:37:00.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331960+0000", 2024-09-14T22:37:00.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6929999999999999e-06 2024-09-14T22:37:00.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331970+0000", 2024-09-14T22:37:00.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4020000000000008e-06 2024-09-14T22:37:00.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331977+0000", 2024-09-14T22:37:00.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7660000000000001e-06 2024-09-14T22:37:00.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360444+0000", 2024-09-14T22:37:00.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028467399000000001 2024-09-14T22:37:00.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360536+0000", 2024-09-14T22:37:00.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1618000000000003e-05 2024-09-14T22:37:00.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360642+0000", 2024-09-14T22:37:00.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010624 2024-09-14T22:37:00.842 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.842 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.843 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.843 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:46576 3.1d 3:b9b1f07f:::rbd_data.113c8642facc.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T22:37:00.843 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:37.379223+0000", 2024-09-14T22:37:00.843 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.406157708999999, 2024-09-14T22:37:00.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072706044999999997, 2024-09-14T22:37:00.843 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.843 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.843 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:00.843 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.843 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.844 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.844 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 46576 2024-09-14T22:37:00.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.844 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.844 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.844 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379223+0000", 2024-09-14T22:37:00.844 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.844 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379223+0000", 2024-09-14T22:37:00.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379226+0000", 2024-09-14T22:37:00.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0929999999999999e-06 2024-09-14T22:37:00.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379240+0000", 2024-09-14T22:37:00.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4606999999999999e-05 2024-09-14T22:37:00.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379272+0000", 2024-09-14T22:37:00.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2073000000000003e-05 2024-09-14T22:37:00.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379288+0000", 2024-09-14T22:37:00.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6036999999999999e-05 2024-09-14T22:37:00.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379451+0000", 2024-09-14T22:37:00.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000162795 2024-09-14T22:37:00.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379702+0000", 2024-09-14T22:37:00.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00025106500000000002 2024-09-14T22:37:00.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:37:00.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.394067+0000", 2024-09-14T22:37:00.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014364564999999999 2024-09-14T22:37:00.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:00.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.451737+0000", 2024-09-14T22:37:00.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057670185999999998 2024-09-14T22:37:00.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:00.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.451823+0000", 2024-09-14T22:37:00.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6075000000000005e-05 2024-09-14T22:37:00.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:00.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.451881+0000", 2024-09-14T22:37:00.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7454999999999998e-05 2024-09-14T22:37:00.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.451929+0000", 2024-09-14T22:37:00.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8093999999999997e-05 2024-09-14T22:37:00.849 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.849 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.849 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.849 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:74169 3.16 3:69acee67:::rbd_data.113cce28790.0000000000000067:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T22:37:00.850 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:55.614872+0000", 2024-09-14T22:37:00.850 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.1705085229999996, 2024-09-14T22:37:00.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036045858, 2024-09-14T22:37:00.850 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.850 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.850 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:00.850 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.850 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.850 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.850 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74169 2024-09-14T22:37:00.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.850 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.614872+0000", 2024-09-14T22:37:00.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.614872+0000", 2024-09-14T22:37:00.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.614875+0000", 2024-09-14T22:37:00.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2799999999999999e-06 2024-09-14T22:37:00.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.852 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.614891+0000", 2024-09-14T22:37:00.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5613000000000001e-05 2024-09-14T22:37:00.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.852 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.614931+0000", 2024-09-14T22:37:00.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0540000000000001e-05 2024-09-14T22:37:00.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.852 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.614956+0000", 2024-09-14T22:37:00.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4939e-05 2024-09-14T22:37:00.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.615096+0000", 2024-09-14T22:37:00.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013952600000000001 2024-09-14T22:37:00.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.615345+0000", 2024-09-14T22:37:00.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024922099999999999 2024-09-14T22:37:00.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T22:37:00.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.621370+0000", 2024-09-14T22:37:00.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0060251239999999998 2024-09-14T22:37:00.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:00.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.650734+0000", 2024-09-14T22:37:00.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029364155999999999 2024-09-14T22:37:00.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:00.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.650760+0000", 2024-09-14T22:37:00.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5446999999999999e-05 2024-09-14T22:37:00.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:00.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.650842+0000", 2024-09-14T22:37:00.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2268999999999994e-05 2024-09-14T22:37:00.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.650918+0000", 2024-09-14T22:37:00.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5742999999999996e-05 2024-09-14T22:37:00.855 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.855 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.855 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.856 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:85215 3.16 3:688cf0a4:::rbd_data.113cff0d0fb7.000000000000000a:12 [read 344064~12288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:00.856 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:00.251379+0000", 2024-09-14T22:37:00.856 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.53400118699999999, 2024-09-14T22:37:00.856 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027899154999999998, 2024-09-14T22:37:00.856 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.856 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.856 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.856 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.856 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.856 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.856 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 85215 2024-09-14T22:37:00.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.857 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.857 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.251379+0000", 2024-09-14T22:37:00.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.857 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.251379+0000", 2024-09-14T22:37:00.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.251380+0000", 2024-09-14T22:37:00.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.77e-07 2024-09-14T22:37:00.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.251382+0000", 2024-09-14T22:37:00.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1610000000000001e-06 2024-09-14T22:37:00.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.251387+0000", 2024-09-14T22:37:00.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7180000000000001e-06 2024-09-14T22:37:00.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.859 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.251392+0000", 2024-09-14T22:37:00.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.78e-06 2024-09-14T22:37:00.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.859 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.279091+0000", 2024-09-14T22:37:00.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027699242999999998 2024-09-14T22:37:00.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.859 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.279182+0000", 2024-09-14T22:37:00.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0383999999999994e-05 2024-09-14T22:37:00.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.279278+0000", 2024-09-14T22:37:00.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6491999999999997e-05 2024-09-14T22:37:00.860 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.860 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.860 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.860 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:85216 3.16 3:688cf0a4:::rbd_data.113cff0d0fb7.000000000000000a:12 [read 434176~12288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:00.860 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:00.251398+0000", 2024-09-14T22:37:00.860 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.53398225099999996, 2024-09-14T22:37:00.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028095181, 2024-09-14T22:37:00.861 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.861 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.861 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.861 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.861 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.861 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.861 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 85216 2024-09-14T22:37:00.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.861 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.251398+0000", 2024-09-14T22:37:00.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.251398+0000", 2024-09-14T22:37:00.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.251398+0000", 2024-09-14T22:37:00.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9200000000000002e-07 2024-09-14T22:37:00.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.251400+0000", 2024-09-14T22:37:00.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9599999999999999e-06 2024-09-14T22:37:00.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.251406+0000", 2024-09-14T22:37:00.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.643e-06 2024-09-14T22:37:00.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.251410+0000", 2024-09-14T22:37:00.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2459999999999997e-06 2024-09-14T22:37:00.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.279312+0000", 2024-09-14T22:37:00.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027901433 2024-09-14T22:37:00.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.864 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.279393+0000", 2024-09-14T22:37:00.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1384999999999994e-05 2024-09-14T22:37:00.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.864 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.279493+0000", 2024-09-14T22:37:00.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000100122 2024-09-14T22:37:00.865 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.865 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.865 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.865 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.865 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:85373 3.16 3:688cf0a4:::rbd_data.113cff0d0fb7.000000000000000a:12 [read 172032~12288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:00.865 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:00.254699+0000", 2024-09-14T22:37:00.865 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.53068116099999996, 2024-09-14T22:37:00.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028028539000000002, 2024-09-14T22:37:00.865 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.865 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.865 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.865 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.866 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.866 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.866 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 85373 2024-09-14T22:37:00.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.866 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254699+0000", 2024-09-14T22:37:00.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254699+0000", 2024-09-14T22:37:00.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254700+0000", 2024-09-14T22:37:00.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.46e-07 2024-09-14T22:37:00.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254702+0000", 2024-09-14T22:37:00.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1739999999999999e-06 2024-09-14T22:37:00.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254708+0000", 2024-09-14T22:37:00.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.173e-06 2024-09-14T22:37:00.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254713+0000", 2024-09-14T22:37:00.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6040000000000003e-06 2024-09-14T22:37:00.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.282544+0000", 2024-09-14T22:37:00.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027831655 2024-09-14T22:37:00.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.869 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.869 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.282627+0000", 2024-09-14T22:37:00.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2891000000000003e-05 2024-09-14T22:37:00.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.869 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.869 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.282728+0000", 2024-09-14T22:37:00.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000100596 2024-09-14T22:37:00.869 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.869 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.869 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.870 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:85374 3.16 3:688cf0a4:::rbd_data.113cff0d0fb7.000000000000000a:12 [read 262144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:00.870 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:00.254720+0000", 2024-09-14T22:37:00.870 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.53066073000000002, 2024-09-14T22:37:00.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028235064000000001, 2024-09-14T22:37:00.870 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.870 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.870 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.870 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.870 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.870 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.870 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 85374 2024-09-14T22:37:00.870 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.870 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254720+0000", 2024-09-14T22:37:00.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254720+0000", 2024-09-14T22:37:00.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254720+0000", 2024-09-14T22:37:00.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4499999999999997e-07 2024-09-14T22:37:00.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254723+0000", 2024-09-14T22:37:00.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5919999999999999e-06 2024-09-14T22:37:00.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254729+0000", 2024-09-14T22:37:00.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5640000000000002e-06 2024-09-14T22:37:00.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254734+0000", 2024-09-14T22:37:00.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8049999999999997e-06 2024-09-14T22:37:00.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.282761+0000", 2024-09-14T22:37:00.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028026669000000001 2024-09-14T22:37:00.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.282840+0000", 2024-09-14T22:37:00.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9621000000000002e-05 2024-09-14T22:37:00.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.874 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.874 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.282955+0000", 2024-09-14T22:37:00.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000114368 2024-09-14T22:37:00.874 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.874 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.874 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.874 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:85375 3.16 3:688cf0a4:::rbd_data.113cff0d0fb7.000000000000000a:12 [read 356352~12288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:00.874 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:00.254740+0000", 2024-09-14T22:37:00.874 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.53064019399999995, 2024-09-14T22:37:00.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028433526000000001, 2024-09-14T22:37:00.874 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.875 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.875 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.875 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.875 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.875 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.875 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 85375 2024-09-14T22:37:00.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.875 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.875 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.875 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254740+0000", 2024-09-14T22:37:00.875 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254740+0000", 2024-09-14T22:37:00.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254741+0000", 2024-09-14T22:37:00.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8299999999999998e-07 2024-09-14T22:37:00.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254743+0000", 2024-09-14T22:37:00.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.154e-06 2024-09-14T22:37:00.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254748+0000", 2024-09-14T22:37:00.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7459999999999997e-06 2024-09-14T22:37:00.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254753+0000", 2024-09-14T22:37:00.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6600000000000003e-06 2024-09-14T22:37:00.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.282989+0000", 2024-09-14T22:37:00.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028235752999999999 2024-09-14T22:37:00.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.283067+0000", 2024-09-14T22:37:00.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8559000000000005e-05 2024-09-14T22:37:00.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.283174+0000", 2024-09-14T22:37:00.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106271 2024-09-14T22:37:00.879 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.879 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.879 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.879 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:85376 3.16 3:688cf0a4:::rbd_data.113cff0d0fb7.000000000000000a:12 [read 446464~12288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:00.879 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:00.254759+0000", 2024-09-14T22:37:00.879 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.53062110600000001, 2024-09-14T22:37:00.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028655568999999999, 2024-09-14T22:37:00.879 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.879 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.879 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.879 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.879 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.880 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.880 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 85376 2024-09-14T22:37:00.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.880 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254759+0000", 2024-09-14T22:37:00.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254759+0000", 2024-09-14T22:37:00.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.881 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254760+0000", 2024-09-14T22:37:00.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5600000000000001e-07 2024-09-14T22:37:00.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.881 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254762+0000", 2024-09-14T22:37:00.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.142e-06 2024-09-14T22:37:00.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254768+0000", 2024-09-14T22:37:00.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9819999999999996e-06 2024-09-14T22:37:00.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254773+0000", 2024-09-14T22:37:00.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0690000000000001e-06 2024-09-14T22:37:00.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.283207+0000", 2024-09-14T22:37:00.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028434207999999999 2024-09-14T22:37:00.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.883 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.883 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.283296+0000", 2024-09-14T22:37:00.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8636999999999999e-05 2024-09-14T22:37:00.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.883 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.883 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.283415+0000", 2024-09-14T22:37:00.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119175 2024-09-14T22:37:00.883 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.883 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.884 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.884 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:85461 3.16 3:68c13f99:::rbd_data.113cff0d0fb7.000000000000001b:12 [read 380928~12288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:00.884 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:00.261582+0000", 2024-09-14T22:37:00.884 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.52379831399999999, 2024-09-14T22:37:00.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029353918999999999, 2024-09-14T22:37:00.884 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:00.884 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:00.884 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:00.884 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:00.884 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:00.884 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:00.885 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 85461 2024-09-14T22:37:00.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.885 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:00.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:00.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.261582+0000", 2024-09-14T22:37:00.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:00.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.261582+0000", 2024-09-14T22:37:00.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:00.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:00.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.261582+0000", 2024-09-14T22:37:00.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:37:00.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:00.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.261585+0000", 2024-09-14T22:37:00.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.492e-06 2024-09-14T22:37:00.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:00.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.261591+0000", 2024-09-14T22:37:00.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5760000000000002e-06 2024-09-14T22:37:00.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.887 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:00.887 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.261598+0000", 2024-09-14T22:37:00.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4370000000000004e-06 2024-09-14T22:37:00.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.887 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:00.887 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.284316+0000", 2024-09-14T22:37:00.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022718341 2024-09-14T22:37:00.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.887 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:00.887 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.284384+0000", 2024-09-14T22:37:00.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7342999999999995e-05 2024-09-14T22:37:00.888 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:00.888 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:00.888 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:00.888 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.290936+0000", 2024-09-14T22:37:00.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0065523430000000004 2024-09-14T22:37:00.888 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.888 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.888 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.888 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:00.888 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:00.888 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:00.890 INFO:teuthology.orchestra.run.smithi001.stdout:2200 write 0xd3a48b7 thru 0xd3b024b (0xb995 bytes) 2024-09-14T22:37:00.890 INFO:teuthology.orchestra.run.smithi001.stdout:2201 write 0xa822756 thru 0xa82e500 (0xbdab bytes) 2024-09-14T22:37:00.890 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:00.890 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:00.890 INFO:teuthology.orchestra.run.smithi001.stdout:2204 read 0x1743ec7 thru 0x174b767 (0x78a1 bytes) 2024-09-14T22:37:00.891 INFO:teuthology.orchestra.run.smithi001.stdout:2206 punch from 0x48979d5 to 0x489b7bb, (0x3de6 bytes) 2024-09-14T22:37:00.891 INFO:teuthology.orchestra.run.smithi001.stdout:2207 trunc from 0xd3b024c to 0x3718acf 2024-09-14T22:37:00.891 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:37:00.891 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:00.883+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:00.893 INFO:teuthology.orchestra.run.smithi001.stdout:2208 trunc from 0x3718acf to 0xe6fcf49 2024-09-14T22:37:00.907 INFO:teuthology.orchestra.run.smithi001.stdout:2211 write 0xc5c1522 thru 0xc5ce214 (0xccf3 bytes) 2024-09-14T22:37:00.929 INFO:teuthology.orchestra.run.smithi001.stdout:2212 trunc from 0xe6fcf49 to 0x8b688c5 2024-09-14T22:37:00.952 INFO:teuthology.orchestra.run.smithi001.stdout:2213 write 0x5270ab4 thru 0x527c2ed (0xb83a bytes) 2024-09-14T22:37:00.960 INFO:teuthology.orchestra.run.smithi001.stdout:2214 punch from 0x6fcdc0d to 0x6fdcbd2, (0xefc5 bytes) 2024-09-14T22:37:00.960 INFO:teuthology.orchestra.run.smithi001.stdout:2215 read 0x61569d1 thru 0x6160a93 (0xa0c3 bytes) 2024-09-14T22:37:00.960 INFO:teuthology.orchestra.run.smithi001.stdout:2216 read 0x6f3c431 thru 0x6f3d65e (0x122e bytes) 2024-09-14T22:37:00.960 INFO:teuthology.orchestra.run.smithi001.stdout:2217 write 0x397d4e3 thru 0x397e0f1 (0xc0f bytes) 2024-09-14T22:37:00.965 INFO:teuthology.orchestra.run.smithi001.stdout:2218 write 0x585030f thru 0x585efbb (0xecad bytes) 2024-09-14T22:37:00.972 INFO:teuthology.orchestra.run.smithi001.stdout:2219 read 0xd285ff thru 0xd32720 (0xa122 bytes) 2024-09-14T22:37:00.991 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:00.987+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:01.003 INFO:teuthology.orchestra.run.smithi001.stdout:2224 write 0xe89ddee thru 0xe8a3922 (0x5b35 bytes) 2024-09-14T22:37:01.012 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:01.012 INFO:teuthology.orchestra.run.smithi001.stdout:2226 read 0x17e4d11 thru 0x17ebd23 (0x7013 bytes) 2024-09-14T22:37:01.028 INFO:teuthology.orchestra.run.smithi001.stdout:2227 trunc from 0xe8a3923 to 0x9109c13 2024-09-14T22:37:01.048 INFO:teuthology.orchestra.run.smithi001.stdout:2229 punch from 0x222fe7 to 0x228ae3, (0x5afc bytes) 2024-09-14T22:37:01.048 INFO:teuthology.orchestra.run.smithi001.stdout:2230 punch from 0x3f263b9 to 0x3f342f7, (0xdf3e bytes) 2024-09-14T22:37:01.048 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:01.048 INFO:teuthology.orchestra.run.smithi001.stdout:2234 write 0x4480d18 thru 0x448293f (0x1c28 bytes) 2024-09-14T22:37:01.053 INFO:teuthology.orchestra.run.smithi001.stdout:2235 write 0xa5aba thru 0xab9fd (0x5f44 bytes) 2024-09-14T22:37:01.058 INFO:teuthology.orchestra.run.smithi001.stdout:2238 read 0x721bf9e thru 0x721cceb (0xd4e bytes) 2024-09-14T22:37:01.058 INFO:teuthology.orchestra.run.smithi001.stdout:2240 write 0x7afa73a thru 0x7afd523 (0x2dea bytes) 2024-09-14T22:37:01.064 INFO:teuthology.orchestra.run.smithi001.stdout:2242 trunc from 0x9109c13 to 0x893cda5 2024-09-14T22:37:01.070 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:01.070 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:01.071 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:37:01.071 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:01.074 INFO:teuthology.orchestra.run.smithi001.stdout:2243 read 0xe4920b thru 0xe4b945 (0x273b bytes) 2024-09-14T22:37:01.085 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:37:01.092 INFO:teuthology.orchestra.run.smithi001.stdout:2246 write 0xadf524c thru 0xadfde99 (0x8c4e bytes) 2024-09-14T22:37:01.092 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:01.087+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:01.098 INFO:teuthology.orchestra.run.smithi001.stdout:2248 punch from 0x357d90a to 0x358bae0, (0xe1d6 bytes) 2024-09-14T22:37:01.098 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:01.098 INFO:teuthology.orchestra.run.smithi001.stdout:2250 punch from 0x3f6c875 to 0x3f7a5f9, (0xdd84 bytes) 2024-09-14T22:37:01.099 INFO:teuthology.orchestra.run.smithi001.stdout:2252 read 0x2c8f10d thru 0x2c92ea0 (0x3d94 bytes) 2024-09-14T22:37:01.099 INFO:teuthology.orchestra.run.smithi001.stdout:2256 writesame 0x4ead0a0 thru 0x4ebaa76 data_size 0x1d(0xd9d7 bytes) 2024-09-14T22:37:01.114 INFO:teuthology.orchestra.run.smithi001.stdout:2257 writesame 0xb28a5f4 thru 0xb297efd data_size 0xd(0xd90a bytes) 2024-09-14T22:37:01.141 INFO:teuthology.orchestra.run.smithi001.stdout:2259 trunc from 0xb297efe to 0xf35650 2024-09-14T22:37:01.165 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:01.165 INFO:teuthology.orchestra.run.smithi001.stdout:2261 read 0xca23ee thru 0xcaff60 (0xdb73 bytes) 2024-09-14T22:37:01.167 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:01.181 INFO:teuthology.orchestra.run.smithi001.stdout:2263 write 0x2216eac thru 0x221e474 (0x75c9 bytes) 2024-09-14T22:37:01.188 INFO:teuthology.orchestra.run.smithi001.stdout:2264 read 0x1b164af thru 0x1b22652 (0xc1a4 bytes) 2024-09-14T22:37:01.189 INFO:teuthology.orchestra.run.smithi001.stdout:2268 read 0x275731 thru 0x27a1dd (0x4aad bytes) 2024-09-14T22:37:01.193 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:01.191+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:01.217 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:01.217 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:01.217 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:37:01.217 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:37:01.217 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:01.232 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:37:01.235 INFO:teuthology.orchestra.run.smithi001.stdout:2269 write 0xdff0ddc thru 0xdff23ba (0x15df bytes) 2024-09-14T22:37:01.240 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:01.240 INFO:teuthology.orchestra.run.smithi001.stdout:2271 write 0x229bf49 thru 0x22a2f36 (0x6fee bytes) 2024-09-14T22:37:01.248 INFO:teuthology.orchestra.run.smithi001.stdout:2272 read 0x99882d5 thru 0x998df44 (0x5c70 bytes) 2024-09-14T22:37:01.258 INFO:teuthology.orchestra.run.smithi001.stdout:2273 write 0xe8d1e13 thru 0xe8d7886 (0x5a74 bytes) 2024-09-14T22:37:01.264 INFO:teuthology.orchestra.run.smithi001.stdout:2274 write 0x265e603 thru 0x265f05e (0xa5c bytes) 2024-09-14T22:37:01.268 INFO:teuthology.orchestra.run.smithi001.stdout:2275 punch from 0x1e51b82 to 0x1e5cef3, (0xb371 bytes) 2024-09-14T22:37:01.268 INFO:teuthology.orchestra.run.smithi001.stdout:2277 read 0x3d4d4d1 thru 0x3d4fb37 (0x2667 bytes) 2024-09-14T22:37:01.280 INFO:teuthology.orchestra.run.smithi001.stdout:2278 writesame 0xec730e3 thru 0xec78245 data_size 0xf(0x5163 bytes) 2024-09-14T22:37:01.281 INFO:teuthology.orchestra.run.smithi001.stdout:2279 read 0x459bca4 thru 0x45a7f45 (0xc2a2 bytes) 2024-09-14T22:37:01.281 INFO:teuthology.orchestra.run.smithi001.stdout:2280 read 0xd3091a5 thru 0xd30bca8 (0x2b04 bytes) 2024-09-14T22:37:01.281 INFO:teuthology.orchestra.run.smithi001.stdout:2281 read 0xa0a0c1b thru 0xa0ace71 (0xc257 bytes) 2024-09-14T22:37:01.281 INFO:teuthology.orchestra.run.smithi001.stdout:2282 read 0x13aabda thru 0x13b7ad9 (0xcf00 bytes) 2024-09-14T22:37:01.281 INFO:teuthology.orchestra.run.smithi001.stdout:2283 writesame 0x156ad73 thru 0x156aff9 data_size 0x1(0x287 bytes) 2024-09-14T22:37:01.284 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:01.284 INFO:teuthology.orchestra.run.smithi001.stdout:2285 read 0x99592e4 thru 0x995fd5f (0x6a7c bytes) 2024-09-14T22:37:01.284 INFO:teuthology.orchestra.run.smithi001.stdout:2286 read 0x71daba3 thru 0x71e882c (0xdc8a bytes) 2024-09-14T22:37:01.285 INFO:teuthology.orchestra.run.smithi001.stdout:2287 write 0x358ea04 thru 0x3595a14 (0x7011 bytes) 2024-09-14T22:37:01.291 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:01.291 INFO:teuthology.orchestra.run.smithi001.stdout:2289 writesame 0x995da77 thru 0x995f059 data_size 0x1(0x15e3 bytes) 2024-09-14T22:37:01.294 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:01.287+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:01.306 INFO:teuthology.orchestra.run.smithi001.stdout:2290 writesame 0xecd5d48 thru 0xece5472 data_size 0x5(0xf72b bytes) 2024-09-14T22:37:01.306 INFO:teuthology.orchestra.run.smithi001.stdout:2292 write 0xaceccfc thru 0xacfc2e0 (0xf5e5 bytes) 2024-09-14T22:37:01.312 INFO:teuthology.orchestra.run.smithi001.stdout:2293 read 0x7871625 thru 0x787c4cc (0xaea8 bytes) 2024-09-14T22:37:01.312 INFO:teuthology.orchestra.run.smithi001.stdout:2295 writesame 0x51e71aa thru 0x51ef68e data_size 0x1(0x84e5 bytes) 2024-09-14T22:37:01.352 INFO:teuthology.orchestra.run.smithi001.stdout:2296 read 0xe83e3b5 thru 0xe84a19f (0xbdeb bytes) 2024-09-14T22:37:01.352 INFO:teuthology.orchestra.run.smithi001.stdout:2297 writesame 0x91e0ff1 thru 0x91ea0cd data_size 0x1(0x90dd bytes) 2024-09-14T22:37:01.368 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:72252 3.2 3:442c6d72:::rbd_data.113cb4c83968.0000000000000015:head [zero 4456448~393216] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-14T22:37:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:37:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.663664045000001, 2024-09-14T22:37:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0176591500000001, 2024-09-14T22:37:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T22:37:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 72252 2024-09-14T22:37:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:37:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:37:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700101+0000", 2024-09-14T22:37:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4570000000000001e-06 2024-09-14T22:37:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700114+0000", 2024-09-14T22:37:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2799999999999999e-05 2024-09-14T22:37:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700146+0000", 2024-09-14T22:37:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2116000000000002e-05 2024-09-14T22:37:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700170+0000", 2024-09-14T22:37:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3770000000000001e-05 2024-09-14T22:37:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700285+0000", 2024-09-14T22:37:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000115228 2024-09-14T22:37:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op must wait for map", 2024-09-14T22:37:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700297+0000", 2024-09-14T22:37:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1681999999999999e-05 2024-09-14T22:37:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.703312+0000", 2024-09-14T22:37:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003015046 2024-09-14T22:37:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:37:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.703327+0000", 2024-09-14T22:37:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5481999999999998e-05 2024-09-14T22:37:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:51.717566+0000", 2024-09-14T22:37:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0142383509999999 2024-09-14T22:37:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:51.717757+0000", 2024-09-14T22:37:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000191218 2024-09-14T22:37:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:74156 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T22:37:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:37:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.8491953350000001, 2024-09-14T22:37:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062073521, 2024-09-14T22:37:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74156 2024-09-14T22:37:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:37:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:37:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514569+0000", 2024-09-14T22:37:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0229999999999999e-06 2024-09-14T22:37:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514596+0000", 2024-09-14T22:37:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6903e-05 2024-09-14T22:37:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514623+0000", 2024-09-14T22:37:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7206e-05 2024-09-14T22:37:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514637+0000", 2024-09-14T22:37:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:37:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514781+0000", 2024-09-14T22:37:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014394899999999999 2024-09-14T22:37:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.515005+0000", 2024-09-14T22:37:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022458499999999999 2024-09-14T22:37:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:37:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.527785+0000", 2024-09-14T22:37:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012780112 2024-09-14T22:37:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576209+0000", 2024-09-14T22:37:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.048423992999999999 2024-09-14T22:37:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576346+0000", 2024-09-14T22:37:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000136909 2024-09-14T22:37:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576517+0000", 2024-09-14T22:37:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000170848 2024-09-14T22:37:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576640+0000", 2024-09-14T22:37:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000122953 2024-09-14T22:37:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82513 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 557056~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:01.380 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:37:01.380 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.9445635160000001, 2024-09-14T22:37:01.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059345486000000003, 2024-09-14T22:37:01.380 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:01.380 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:01.380 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:01.380 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:01.380 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:01.380 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:01.380 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82513 2024-09-14T22:37:01.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.381 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:01.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:01.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:37:01.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:01.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:37:01.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:01.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419200+0000", 2024-09-14T22:37:01.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.184e-06 2024-09-14T22:37:01.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:01.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419207+0000", 2024-09-14T22:37:01.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2690000000000002e-06 2024-09-14T22:37:01.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:01.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419224+0000", 2024-09-14T22:37:01.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6623e-05 2024-09-14T22:37:01.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:01.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419236+0000", 2024-09-14T22:37:01.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2367e-05 2024-09-14T22:37:01.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:01.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478294+0000", 2024-09-14T22:37:01.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059057927000000003 2024-09-14T22:37:01.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:01.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478402+0000", 2024-09-14T22:37:01.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107921 2024-09-14T22:37:01.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:01.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478544+0000", 2024-09-14T22:37:01.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014219499999999999 2024-09-14T22:37:01.384 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.384 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:01.384 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.384 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82514 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 704512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:01.384 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:37:01.384 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.944504413, 2024-09-14T22:37:01.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059580793, 2024-09-14T22:37:01.384 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:01.384 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:01.384 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:01.384 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:01.385 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:01.385 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:01.385 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82514 2024-09-14T22:37:01.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.385 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:01.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:01.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:37:01.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:01.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:37:01.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:01.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419259+0000", 2024-09-14T22:37:01.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.353e-06 2024-09-14T22:37:01.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:01.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419276+0000", 2024-09-14T22:37:01.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7567e-05 2024-09-14T22:37:01.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:01.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419294+0000", 2024-09-14T22:37:01.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.713e-05 2024-09-14T22:37:01.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:01.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419306+0000", 2024-09-14T22:37:01.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.279e-05 2024-09-14T22:37:01.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:01.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478589+0000", 2024-09-14T22:37:01.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059282978 2024-09-14T22:37:01.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:01.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478694+0000", 2024-09-14T22:37:01.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000104561 2024-09-14T22:37:01.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:01.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478838+0000", 2024-09-14T22:37:01.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014441399999999999 2024-09-14T22:37:01.388 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.388 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:01.388 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.388 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82515 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:01.389 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:37:01.389 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.944439263, 2024-09-14T22:37:01.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059815123999999997, 2024-09-14T22:37:01.389 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:01.389 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:01.389 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:01.389 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:01.389 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:01.389 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:01.389 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82515 2024-09-14T22:37:01.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.389 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:01.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:01.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:37:01.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:01.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:37:01.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:01.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419324+0000", 2024-09-14T22:37:01.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2130000000000001e-06 2024-09-14T22:37:01.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:01.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419331+0000", 2024-09-14T22:37:01.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4730000000000003e-06 2024-09-14T22:37:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419348+0000", 2024-09-14T22:37:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6269999999999998e-05 2024-09-14T22:37:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419360+0000", 2024-09-14T22:37:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2243e-05 2024-09-14T22:37:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478884+0000", 2024-09-14T22:37:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059524369000000001 2024-09-14T22:37:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478990+0000", 2024-09-14T22:37:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010569 2024-09-14T22:37:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479138+0000", 2024-09-14T22:37:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000147866 2024-09-14T22:37:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:01.393 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.393 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82516 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 999424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:01.393 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:37:01.393 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.944386758, 2024-09-14T22:37:01.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061979595999999998, 2024-09-14T22:37:01.393 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:01.393 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:01.393 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:01.393 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:01.393 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:01.393 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:01.393 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82516 2024-09-14T22:37:01.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.393 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:01.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:01.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:37:01.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:01.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:37:01.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:01.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419376+0000", 2024-09-14T22:37:01.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.237e-06 2024-09-14T22:37:01.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:01.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419383+0000", 2024-09-14T22:37:01.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8170000000000001e-06 2024-09-14T22:37:01.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:01.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419400+0000", 2024-09-14T22:37:01.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6636000000000001e-05 2024-09-14T22:37:01.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:01.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419412+0000", 2024-09-14T22:37:01.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2357e-05 2024-09-14T22:37:01.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:01.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481080+0000", 2024-09-14T22:37:01.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061668032999999997 2024-09-14T22:37:01.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:01.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481193+0000", 2024-09-14T22:37:01.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112335 2024-09-14T22:37:01.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:01.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481355+0000", 2024-09-14T22:37:01.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016218100000000001 2024-09-14T22:37:01.396 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.396 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:01.397 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.397 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82538 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1097728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:01.397 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:37:01.397 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.9443346880000001, 2024-09-14T22:37:01.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059971705, 2024-09-14T22:37:01.397 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:01.397 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:01.397 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:01.397 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:01.397 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:01.397 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:01.398 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82538 2024-09-14T22:37:01.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.398 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:01.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:01.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:37:01.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:01.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:37:01.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:01.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419429+0000", 2024-09-14T22:37:01.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.497e-06 2024-09-14T22:37:01.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:01.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419436+0000", 2024-09-14T22:37:01.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2099999999999996e-06 2024-09-14T22:37:01.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:01.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419453+0000", 2024-09-14T22:37:01.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6840000000000001e-05 2024-09-14T22:37:01.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:01.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419465+0000", 2024-09-14T22:37:01.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2283000000000001e-05 2024-09-14T22:37:01.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:01.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479182+0000", 2024-09-14T22:37:01.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059717339000000001 2024-09-14T22:37:01.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:01.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479297+0000", 2024-09-14T22:37:01.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000114259 2024-09-14T22:37:01.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:01.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479399+0000", 2024-09-14T22:37:01.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102277 2024-09-14T22:37:01.401 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.401 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:01.401 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.401 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82539 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:01.401 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:37:01.402 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.944281838, 2024-09-14T22:37:01.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060166935999999997, 2024-09-14T22:37:01.402 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:01.402 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:01.402 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:01.402 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:01.402 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:01.402 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:01.402 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82539 2024-09-14T22:37:01.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.402 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:01.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:01.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:37:01.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:01.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:37:01.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:01.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419482+0000", 2024-09-14T22:37:01.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4130000000000001e-06 2024-09-14T22:37:01.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:01.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419488+0000", 2024-09-14T22:37:01.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8460000000000002e-06 2024-09-14T22:37:01.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:01.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419505+0000", 2024-09-14T22:37:01.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6477000000000002e-05 2024-09-14T22:37:01.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:01.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419518+0000", 2024-09-14T22:37:01.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2797e-05 2024-09-14T22:37:01.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:01.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479438+0000", 2024-09-14T22:37:01.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059919897 2024-09-14T22:37:01.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:01.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479547+0000", 2024-09-14T22:37:01.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109807 2024-09-14T22:37:01.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:01.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479647+0000", 2024-09-14T22:37:01.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9698999999999995e-05 2024-09-14T22:37:01.406 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.406 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:01.406 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.406 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82542 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:01.406 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:37:01.406 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.944229395, 2024-09-14T22:37:01.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060367430999999999, 2024-09-14T22:37:01.406 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:01.406 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:01.406 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:01.406 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:01.406 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:01.406 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:01.407 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82542 2024-09-14T22:37:01.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.407 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:01.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:01.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:37:01.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:01.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:37:01.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:01.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419534+0000", 2024-09-14T22:37:01.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T22:37:01.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:01.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419542+0000", 2024-09-14T22:37:01.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8199999999999997e-06 2024-09-14T22:37:01.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:01.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419559+0000", 2024-09-14T22:37:01.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6852999999999999e-05 2024-09-14T22:37:01.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:01.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419571+0000", 2024-09-14T22:37:01.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2364000000000001e-05 2024-09-14T22:37:01.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:01.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479686+0000", 2024-09-14T22:37:01.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060115522999999997 2024-09-14T22:37:01.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:01.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479795+0000", 2024-09-14T22:37:01.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010831399999999999 2024-09-14T22:37:01.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:01.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479900+0000", 2024-09-14T22:37:01.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105284 2024-09-14T22:37:01.410 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.410 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:01.410 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.410 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82544 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:01.410 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:37:01.410 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.9441756589999999, 2024-09-14T22:37:01.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060590515999999997, 2024-09-14T22:37:01.410 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:01.411 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:01.411 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:01.411 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:01.411 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:01.411 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:01.411 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82544 2024-09-14T22:37:01.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.411 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:01.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:01.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:37:01.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:01.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:37:01.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:01.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419588+0000", 2024-09-14T22:37:01.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.35e-06 2024-09-14T22:37:01.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:01.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419595+0000", 2024-09-14T22:37:01.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4030000000000001e-06 2024-09-14T22:37:01.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:01.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419612+0000", 2024-09-14T22:37:01.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6670000000000001e-05 2024-09-14T22:37:01.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:01.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419624+0000", 2024-09-14T22:37:01.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-05 2024-09-14T22:37:01.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:01.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479957+0000", 2024-09-14T22:37:01.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060333287999999999 2024-09-14T22:37:01.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:01.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480074+0000", 2024-09-14T22:37:01.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116443 2024-09-14T22:37:01.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:01.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480177+0000", 2024-09-14T22:37:01.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000103102 2024-09-14T22:37:01.414 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.414 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:01.414 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.415 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82546 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:01.415 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:37:01.415 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.9441227400000001, 2024-09-14T22:37:01.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060795428999999998, 2024-09-14T22:37:01.415 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:01.415 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:01.415 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:01.415 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:01.415 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:01.415 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:01.415 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82546 2024-09-14T22:37:01.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.415 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:01.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:01.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:37:01.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:01.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:37:01.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:01.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419640+0000", 2024-09-14T22:37:01.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.254e-06 2024-09-14T22:37:01.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:01.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419647+0000", 2024-09-14T22:37:01.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9e-06 2024-09-14T22:37:01.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:01.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419664+0000", 2024-09-14T22:37:01.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6345999999999999e-05 2024-09-14T22:37:01.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:01.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419676+0000", 2024-09-14T22:37:01.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.224e-05 2024-09-14T22:37:01.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:01.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480218+0000", 2024-09-14T22:37:01.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060542133999999997 2024-09-14T22:37:01.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:01.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480336+0000", 2024-09-14T22:37:01.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117472 2024-09-14T22:37:01.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:01.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480435+0000", 2024-09-14T22:37:01.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9083e-05 2024-09-14T22:37:01.419 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.419 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:01.419 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.419 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82548 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:01.419 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:37:01.419 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.94407073, 2024-09-14T22:37:01.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061011636000000001, 2024-09-14T22:37:01.419 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:01.419 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:01.420 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:01.420 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:01.420 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:01.420 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:01.420 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82548 2024-09-14T22:37:01.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.420 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:01.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:01.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:37:01.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:01.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:37:01.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:01.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419693+0000", 2024-09-14T22:37:01.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.387e-06 2024-09-14T22:37:01.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:01.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419699+0000", 2024-09-14T22:37:01.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8730000000000001e-06 2024-09-14T22:37:01.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:01.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419716+0000", 2024-09-14T22:37:01.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6416e-05 2024-09-14T22:37:01.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:01.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419728+0000", 2024-09-14T22:37:01.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2597e-05 2024-09-14T22:37:01.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:01.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480476+0000", 2024-09-14T22:37:01.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060747989000000002 2024-09-14T22:37:01.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:01.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480583+0000", 2024-09-14T22:37:01.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010638500000000001 2024-09-14T22:37:01.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:01.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480703+0000", 2024-09-14T22:37:01.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011998900000000001 2024-09-14T22:37:01.423 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.423 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:01.423 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.423 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82550 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1982464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:01.424 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:37:01.424 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.9440179770000001, 2024-09-14T22:37:01.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061295483999999997, 2024-09-14T22:37:01.424 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:01.424 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:01.424 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:01.424 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:01.424 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:01.424 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:01.424 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82550 2024-09-14T22:37:01.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.424 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:01.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:01.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:37:01.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:01.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:37:01.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:01.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419745+0000", 2024-09-14T22:37:01.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.406e-06 2024-09-14T22:37:01.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:01.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419753+0000", 2024-09-14T22:37:01.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8140000000000001e-06 2024-09-14T22:37:01.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:01.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419770+0000", 2024-09-14T22:37:01.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6719999999999999e-05 2024-09-14T22:37:01.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:01.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419782+0000", 2024-09-14T22:37:01.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2252999999999999e-05 2024-09-14T22:37:01.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:01.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480745+0000", 2024-09-14T22:37:01.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060962385000000001 2024-09-14T22:37:01.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:01.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480900+0000", 2024-09-14T22:37:01.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015513899999999999 2024-09-14T22:37:01.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:01.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481039+0000", 2024-09-14T22:37:01.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013976700000000001 2024-09-14T22:37:01.427 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.428 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:01.428 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.428 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.428 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82588 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1064960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:01.428 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:37:01.428 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.9439640869999999, 2024-09-14T22:37:01.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061863732999999997, 2024-09-14T22:37:01.428 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:01.428 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:01.428 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:01.428 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:01.428 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:01.428 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:01.429 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82588 2024-09-14T22:37:01.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.429 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:01.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:01.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:37:01.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:01.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:37:01.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:01.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419799+0000", 2024-09-14T22:37:01.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4160000000000001e-06 2024-09-14T22:37:01.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:01.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419807+0000", 2024-09-14T22:37:01.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2169999999999994e-06 2024-09-14T22:37:01.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:01.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419824+0000", 2024-09-14T22:37:01.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6776e-05 2024-09-14T22:37:01.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:01.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419837+0000", 2024-09-14T22:37:01.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2680000000000001e-05 2024-09-14T22:37:01.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:01.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481395+0000", 2024-09-14T22:37:01.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061558416999999997 2024-09-14T22:37:01.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:01.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481532+0000", 2024-09-14T22:37:01.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000136861 2024-09-14T22:37:01.432 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.432 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.432 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:01.432 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481662+0000", 2024-09-14T22:37:01.432 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012936600000000001 2024-09-14T22:37:01.432 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.432 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:01.432 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.432 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.432 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.432 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82589 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:01.432 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:37:01.432 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.943909492, 2024-09-14T22:37:01.432 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062064592000000002, 2024-09-14T22:37:01.433 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:01.433 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:01.433 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:01.433 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:01.433 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:01.433 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:01.433 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82589 2024-09-14T22:37:01.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.433 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:01.433 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.433 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:01.433 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:37:01.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:01.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:37:01.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:01.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419854+0000", 2024-09-14T22:37:01.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.367e-06 2024-09-14T22:37:01.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:01.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419862+0000", 2024-09-14T22:37:01.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8930000000000005e-06 2024-09-14T22:37:01.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.435 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:01.435 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419896+0000", 2024-09-14T22:37:01.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4134000000000002e-05 2024-09-14T22:37:01.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.435 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:01.435 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419909+0000", 2024-09-14T22:37:01.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3332999999999999e-05 2024-09-14T22:37:01.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.435 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:01.435 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481702+0000", 2024-09-14T22:37:01.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061792947000000001 2024-09-14T22:37:01.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.436 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.436 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:01.436 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481808+0000", 2024-09-14T22:37:01.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105655 2024-09-14T22:37:01.436 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.436 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.436 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:01.436 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481917+0000", 2024-09-14T22:37:01.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109263 2024-09-14T22:37:01.436 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.436 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:01.436 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.436 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.436 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.437 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82590 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1359872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:01.437 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:37:01.437 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.9438358849999999, 2024-09-14T22:37:01.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062362895000000002, 2024-09-14T22:37:01.437 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:01.437 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:01.437 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:01.437 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:01.437 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:01.437 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:01.437 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82590 2024-09-14T22:37:01.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.437 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:01.437 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:01.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:37:01.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:01.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:37:01.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:01.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419928+0000", 2024-09-14T22:37:01.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4470000000000001e-06 2024-09-14T22:37:01.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:01.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419942+0000", 2024-09-14T22:37:01.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4229e-05 2024-09-14T22:37:01.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:01.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419972+0000", 2024-09-14T22:37:01.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.065e-05 2024-09-14T22:37:01.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:01.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419985+0000", 2024-09-14T22:37:01.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2903e-05 2024-09-14T22:37:01.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:01.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481960+0000", 2024-09-14T22:37:01.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061974435000000001 2024-09-14T22:37:01.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:01.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482106+0000", 2024-09-14T22:37:01.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014643800000000001 2024-09-14T22:37:01.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:01.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482289+0000", 2024-09-14T22:37:01.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000182793 2024-09-14T22:37:01.441 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.441 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:01.441 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.441 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82592 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:01.441 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:37:01.441 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.943759663, 2024-09-14T22:37:01.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062631507000000003, 2024-09-14T22:37:01.441 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:01.441 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:01.441 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:01.441 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:01.441 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:01.442 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:01.442 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82592 2024-09-14T22:37:01.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.442 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:01.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:01.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:37:01.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:01.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:37:01.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.443 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:01.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420017+0000", 2024-09-14T22:37:01.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4744e-05 2024-09-14T22:37:01.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.443 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:01.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420036+0000", 2024-09-14T22:37:01.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8680000000000001e-05 2024-09-14T22:37:01.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.443 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:01.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420054+0000", 2024-09-14T22:37:01.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7799999999999999e-05 2024-09-14T22:37:01.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.444 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:01.444 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420067+0000", 2024-09-14T22:37:01.444 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3013000000000001e-05 2024-09-14T22:37:01.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.444 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:01.444 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482358+0000", 2024-09-14T22:37:01.444 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062291445000000001 2024-09-14T22:37:01.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.444 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:01.444 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482469+0000", 2024-09-14T22:37:01.444 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110608 2024-09-14T22:37:01.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:01.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482634+0000", 2024-09-14T22:37:01.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000165217 2024-09-14T22:37:01.445 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.445 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:01.445 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.445 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82594 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1654784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:01.445 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:37:01.445 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.9436616870000001, 2024-09-14T22:37:01.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062893849000000002, 2024-09-14T22:37:01.446 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:01.446 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:01.446 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:01.446 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:01.446 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:01.446 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:01.446 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82594 2024-09-14T22:37:01.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.446 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:01.446 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.446 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:01.446 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:37:01.446 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.446 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.447 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:01.447 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:37:01.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.447 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:01.447 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420102+0000", 2024-09-14T22:37:01.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5e-06 2024-09-14T22:37:01.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.447 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:01.447 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420115+0000", 2024-09-14T22:37:01.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.324e-05 2024-09-14T22:37:01.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.448 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:01.448 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420133+0000", 2024-09-14T22:37:01.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8213e-05 2024-09-14T22:37:01.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.448 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:01.448 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420147+0000", 2024-09-14T22:37:01.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3284e-05 2024-09-14T22:37:01.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.448 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:01.448 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482688+0000", 2024-09-14T22:37:01.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062541425999999997 2024-09-14T22:37:01.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.449 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:01.449 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482794+0000", 2024-09-14T22:37:01.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010575500000000001 2024-09-14T22:37:01.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.449 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:01.449 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482994+0000", 2024-09-14T22:37:01.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020043099999999999 2024-09-14T22:37:01.449 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.449 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:01.449 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.450 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82596 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:01.450 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:37:01.450 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.943598387, 2024-09-14T22:37:01.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063187238000000007, 2024-09-14T22:37:01.450 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:01.450 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:01.450 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:01.450 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:01.450 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:01.450 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:01.450 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82596 2024-09-14T22:37:01.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.450 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:01.450 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:01.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:37:01.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:01.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:37:01.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:01.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420165+0000", 2024-09-14T22:37:01.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4100000000000001e-06 2024-09-14T22:37:01.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:01.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420173+0000", 2024-09-14T22:37:01.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8129999999999996e-06 2024-09-14T22:37:01.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:01.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420190+0000", 2024-09-14T22:37:01.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7076999999999999e-05 2024-09-14T22:37:01.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:01.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420203+0000", 2024-09-14T22:37:01.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2683e-05 2024-09-14T22:37:01.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:01.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483060+0000", 2024-09-14T22:37:01.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062857114000000006 2024-09-14T22:37:01.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:01.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483167+0000", 2024-09-14T22:37:01.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010713999999999999 2024-09-14T22:37:01.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:01.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483351+0000", 2024-09-14T22:37:01.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018400099999999999 2024-09-14T22:37:01.453 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.454 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:01.454 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.454 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.454 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.454 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82597 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1949696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:01.454 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:37:01.454 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.943541561, 2024-09-14T22:37:01.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063466513000000002, 2024-09-14T22:37:01.454 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:01.454 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:01.454 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:01.454 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:01.454 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:01.454 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:01.454 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82597 2024-09-14T22:37:01.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.455 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:01.455 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.455 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:01.455 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:37:01.455 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.455 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.455 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:01.455 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:37:01.455 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:01.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.455 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.455 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:01.455 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420221+0000", 2024-09-14T22:37:01.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.043e-06 2024-09-14T22:37:01.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:01.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420229+0000", 2024-09-14T22:37:01.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0469999999999994e-06 2024-09-14T22:37:01.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:01.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420246+0000", 2024-09-14T22:37:01.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6339999999999999e-05 2024-09-14T22:37:01.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:01.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420258+0000", 2024-09-14T22:37:01.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2296000000000001e-05 2024-09-14T22:37:01.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:01.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483402+0000", 2024-09-14T22:37:01.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063144318000000005 2024-09-14T22:37:01.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:01.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483530+0000", 2024-09-14T22:37:01.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012713199999999999 2024-09-14T22:37:01.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:01.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:01.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:01.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483687+0000", 2024-09-14T22:37:01.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015733699999999999 2024-09-14T22:37:01.458 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.458 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:01.458 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.458 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:01.458 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:01.458 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:01.458 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:37:01.460 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:37:01.461 INFO:teuthology.orchestra.run.smithi001.stdout:2298 read 0x4bb6d68 thru 0x4bc48fc (0xdb95 bytes) 2024-09-14T22:37:01.462 INFO:teuthology.orchestra.run.smithi001.stdout:2299 punch from 0xb55563a to 0xb558dba, (0x3780 bytes) 2024-09-14T22:37:01.462 INFO:teuthology.orchestra.run.smithi001.stdout:2300 write 0xb587445 thru 0xb5896b2 (0x226e bytes) 2024-09-14T22:37:01.462 INFO:teuthology.orchestra.run.smithi001.stdout:2301 write 0x3fd796c thru 0x3fe3719 (0xbdae bytes) 2024-09-14T22:37:01.462 INFO:teuthology.orchestra.run.smithi001.stdout:2302 write 0xdedd6c8 thru 0xdeea788 (0xd0c1 bytes) 2024-09-14T22:37:01.462 INFO:teuthology.orchestra.run.smithi001.stdout:2303 read 0x377d101 thru 0x3787d8e (0xac8e bytes) 2024-09-14T22:37:01.462 INFO:teuthology.orchestra.run.smithi001.stdout:2304 write 0x97614c1 thru 0x9763bc2 (0x2702 bytes) 2024-09-14T22:37:01.462 INFO:teuthology.orchestra.run.smithi001.stdout:2307 read 0xaf9efbb thru 0xafa3d8d (0x4dd3 bytes) 2024-09-14T22:37:01.462 INFO:teuthology.orchestra.run.smithi001.stdout:2308 write 0x8b6d0a1 thru 0x8b70faf (0x3f0f bytes) 2024-09-14T22:37:01.462 INFO:teuthology.orchestra.run.smithi001.stdout:2309 write 0x60d9133 thru 0x60e5d5e (0xcc2c bytes) 2024-09-14T22:37:01.462 INFO:teuthology.orchestra.run.smithi001.stdout:2310 punch from 0x4bcc31d to 0x4bda090, (0xdd73 bytes) 2024-09-14T22:37:01.462 INFO:teuthology.orchestra.run.smithi001.stdout:2311 read 0x5840feb thru 0x58425ec (0x1602 bytes) 2024-09-14T22:37:01.462 INFO:teuthology.orchestra.run.smithi001.stdout:2312 read 0x8366d47 thru 0x836bdba (0x5074 bytes) 2024-09-14T22:37:01.462 INFO:teuthology.orchestra.run.smithi001.stdout:2314 write 0xea98af7 thru 0xeaa7ab2 (0xefbc bytes) 2024-09-14T22:37:01.463 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:01.455+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:01.463 INFO:teuthology.orchestra.run.smithi001.stdout:2316 read 0xc6097b9 thru 0xc60f59b (0x5de3 bytes) 2024-09-14T22:37:01.490 INFO:teuthology.orchestra.run.smithi001.stdout:2317 trunc from 0xece5473 to 0x3c165da 2024-09-14T22:37:01.520 INFO:teuthology.orchestra.run.smithi001.stdout:2319 writesame 0x2567bdc thru 0x2574519 data_size 0x2(0xc93e bytes) 2024-09-14T22:37:01.551 INFO:teuthology.orchestra.run.smithi001.stdout:2321 trunc from 0x3c165da to 0x8f4425e 2024-09-14T22:37:01.561 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:01.555+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:01.563 INFO:teuthology.orchestra.run.smithi001.stdout:2322 write 0x5a1bd79 thru 0x5a23f99 (0x8221 bytes) 2024-09-14T22:37:01.571 INFO:teuthology.orchestra.run.smithi001.stdout:2324 punch from 0x3fea9f1 to 0x3ff8906, (0xdf15 bytes) 2024-09-14T22:37:01.571 INFO:teuthology.orchestra.run.smithi001.stdout:2325 punch from 0x5eedbeb to 0x5efa330, (0xc745 bytes) 2024-09-14T22:37:01.584 INFO:teuthology.orchestra.run.smithi001.stdout:2326 writesame 0x9141c1e thru 0x914983c data_size 0x1f(0x7c1f bytes) 2024-09-14T22:37:01.584 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:01.600 INFO:teuthology.orchestra.run.smithi001.stdout:2330 write 0xb0e21a6 thru 0xb0e2baa (0xa05 bytes) 2024-09-14T22:37:01.622 INFO:teuthology.orchestra.run.smithi001.stdout:2332 write 0xe90f695 thru 0xe9119a7 (0x2313 bytes) 2024-09-14T22:37:01.626 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:37:01.626 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:37:01.627 INFO:teuthology.orchestra.run.smithi001.stdout:2334 clone 17 order 24 su 65536 sc 4 2024-09-14T22:37:01.644 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:37:01.644 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:01.644 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:01.644 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:37:01.644 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:01.645 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:37:01.645 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-14T22:37:01.657 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:37:01.661 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:37:01.661 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:37:01.661 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:01.659+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:01.678 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:37:01.679 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:37:01.696 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:37:01.696 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:37:01.714 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:37:01.715 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:37:01.732 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:37:01.733 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:37:01.736 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:37:01.736 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:37:01.762 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:01.755+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:01.779 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:01.779 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:01.793 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:37:01.879 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:01.875+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:01.959 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:01.959 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:01.964 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:01.959+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:01.974 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:37:02.065 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:02.063+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:02.076 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T22:37:02.115 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:02.115 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:02.132 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:37:02.166 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:02.159+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:02.267 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:02.263+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:02.300 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:02.300 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:02.316 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:37:02.333 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-14T22:37:02.368 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:02.363+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:02.464 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:02.464 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:02.469 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:02.463+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:02.481 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:37:02.570 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:02.567+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:02.603 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:02.603 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:02.619 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:37:02.695 INFO:teuthology.orchestra.run.smithi001.stdout:2335 read 0x705fffe thru 0x70646a3 (0x46a6 bytes) 2024-09-14T22:37:02.696 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:02.691+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:02.697 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:02.697 INFO:teuthology.orchestra.run.smithi001.stdout:2337 read 0xcefd883 thru 0xcf0af6f (0xd6ed bytes) 2024-09-14T22:37:02.698 INFO:teuthology.orchestra.run.smithi001.stdout:2338 write 0xdde0a59 thru 0xdde7888 (0x6e30 bytes) 2024-09-14T22:37:02.716 INFO:teuthology.orchestra.run.smithi001.stdout:2339 read 0x7fd0984 thru 0x7fd7bfd (0x727a bytes) 2024-09-14T22:37:02.716 INFO:teuthology.orchestra.run.smithi001.stdout:2341 punch from 0x76a5c41 to 0x76b497b, (0xed3a bytes) 2024-09-14T22:37:02.716 INFO:teuthology.orchestra.run.smithi001.stdout:2342 read 0x61c0dc4 thru 0x61c2c6b (0x1ea8 bytes) 2024-09-14T22:37:02.720 INFO:teuthology.orchestra.run.smithi001.stdout:2343 trunc from 0xe9119a8 to 0xb80ee9c 2024-09-14T22:37:02.766 INFO:teuthology.orchestra.run.smithi001.stdout:2344 trunc from 0xb80ee9c to 0xa36a28d 2024-09-14T22:37:02.771 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:02.767+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:02.776 INFO:teuthology.orchestra.run.smithi001.stdout:2347 write 0x1c5fb8f thru 0x1c663aa (0x681c bytes) 2024-09-14T22:37:02.787 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:02.787 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:02.804 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:37:02.818 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:02.818 INFO:teuthology.orchestra.run.smithi001.stdout:2351 punch from 0x6d4ac6f to 0x6d5a14d, (0xf4de bytes) 2024-09-14T22:37:02.818 INFO:teuthology.orchestra.run.smithi001.stdout:2353 read 0x5525a01 thru 0x552f0fc (0x96fc bytes) 2024-09-14T22:37:02.819 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:02.819 INFO:teuthology.orchestra.run.smithi001.stdout:2355 read 0x643953 thru 0x64ae10 (0x74be bytes) 2024-09-14T22:37:02.821 INFO:teuthology.orchestra.run.smithi001.stdout:2356 trunc from 0xa36a28d to 0xac3e33a 2024-09-14T22:37:02.829 INFO:teuthology.orchestra.run.smithi001.stdout:2357 read 0x105f8ad thru 0x1062b89 (0x32dd bytes) 2024-09-14T22:37:02.841 INFO:teuthology.orchestra.run.smithi001.stdout:2358 write 0xc269ac0 thru 0xc276be2 (0xd123 bytes) 2024-09-14T22:37:02.850 INFO:teuthology.orchestra.run.smithi001.stdout:2360 read 0x97c0f5f thru 0x97c2bad (0x1c4f bytes) 2024-09-14T22:37:02.851 INFO:teuthology.orchestra.run.smithi001.stdout:2361 write 0x46486ec thru 0x4655c9b (0xd5b0 bytes) 2024-09-14T22:37:02.868 INFO:teuthology.orchestra.run.smithi001.stdout:2362 read 0xacbaac7 thru 0xacbb262 (0x79c bytes) 2024-09-14T22:37:02.869 INFO:teuthology.orchestra.run.smithi001.stdout:2364 read 0xa05e746 thru 0xa06e0a7 (0xf962 bytes) 2024-09-14T22:37:02.870 INFO:teuthology.orchestra.run.smithi001.stdout:2365 write 0x41ada42 thru 0x41afb96 (0x2155 bytes) 2024-09-14T22:37:02.872 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:02.867+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:02.883 INFO:teuthology.orchestra.run.smithi001.stdout:2366 write 0x388205b thru 0x38861a2 (0x4148 bytes) 2024-09-14T22:37:02.917 INFO:teuthology.orchestra.run.smithi001.stdout:2368 write 0x311a656 thru 0x311f357 (0x4d02 bytes) 2024-09-14T22:37:02.921 INFO:teuthology.orchestra.run.smithi001.stdout:2369 punch from 0x8565c97 to 0x85688c6, (0x2c2f bytes) 2024-09-14T22:37:02.921 INFO:teuthology.orchestra.run.smithi001.stdout:2370 write 0x7ce3c0d thru 0x7ce573b (0x1b2f bytes) 2024-09-14T22:37:02.924 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:02.924 INFO:teuthology.orchestra.run.smithi001.stdout:2372 punch from 0xb4b532a to 0xb4bccc6, (0x799c bytes) 2024-09-14T22:37:02.925 INFO:teuthology.orchestra.run.smithi001.stdout:2373 trunc from 0xc276be3 to 0xccdf6ba 2024-09-14T22:37:02.935 INFO:teuthology.orchestra.run.smithi001.stdout:2374 write 0x96b0f81 thru 0x96bf349 (0xe3c9 bytes) 2024-09-14T22:37:02.938 INFO:teuthology.orchestra.run.smithi001.stdout:2377 write 0xbf1aa67 thru 0xbf1b665 (0xbff bytes) 2024-09-14T22:37:02.939 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:02.939 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:02.941 INFO:teuthology.orchestra.run.smithi001.stdout:2378 read 0x92a769e thru 0x92b5ad1 (0xe434 bytes) 2024-09-14T22:37:02.942 INFO:teuthology.orchestra.run.smithi001.stdout:2379 read 0x235a375 thru 0x23626cc (0x8358 bytes) 2024-09-14T22:37:02.943 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:02.943 INFO:teuthology.orchestra.run.smithi001.stdout:2383 read 0xa8f5823 thru 0xa902821 (0xcfff bytes) 2024-09-14T22:37:02.944 INFO:teuthology.orchestra.run.smithi001.stdout:2386 write 0x180a9f6 thru 0x1817ea1 (0xd4ac bytes) 2024-09-14T22:37:02.956 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:37:02.958 INFO:teuthology.orchestra.run.smithi001.stdout:2387 trunc from 0xccdf6ba to 0x60cec48 2024-09-14T22:37:02.991 INFO:teuthology.orchestra.run.smithi001.stdout:2388 read 0x3432b7 thru 0x347bec (0x4936 bytes) 2024-09-14T22:37:02.991 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:02.987+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:02.992 INFO:teuthology.orchestra.run.smithi001.stdout:2390 read 0x3c15ca7 thru 0x3c17fc7 (0x2321 bytes) 2024-09-14T22:37:02.993 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:02.993 INFO:teuthology.orchestra.run.smithi001.stdout:2392 write 0x4680a73 thru 0x468c52b (0xbab9 bytes) 2024-09-14T22:37:03.004 INFO:teuthology.orchestra.run.smithi001.stdout:2394 trunc from 0x60cec48 to 0xcbc73fc 2024-09-14T22:37:03.014 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:03.014 INFO:teuthology.orchestra.run.smithi001.stdout:2396 read 0xc76cd0f thru 0xc774f27 (0x8219 bytes) 2024-09-14T22:37:03.025 INFO:teuthology.orchestra.run.smithi001.stdout:2397 writesame 0xe1c28b3 thru 0xe1c8ed2 data_size 0x4(0x6620 bytes) 2024-09-14T22:37:03.025 INFO:teuthology.orchestra.run.smithi001.stdout:2398 writesame 0xd261ba2 thru 0xd26254a data_size 0x1(0x9a9 bytes) 2024-09-14T22:37:03.038 INFO:teuthology.orchestra.run.smithi001.stdout:2399 trunc from 0xe1c8ed3 to 0xa05d7b2 2024-09-14T22:37:03.067 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2024-09-14T22:37:03.074 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:03.071+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:03.076 INFO:teuthology.orchestra.run.smithi001.stdout:2400 write 0xccc23f4 thru 0xccc58a1 (0x34ae bytes) 2024-09-14T22:37:03.083 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:03.083 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:03.087 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:03.097 INFO:teuthology.orchestra.run.smithi001.stdout:2402 write 0xe010927 thru 0xe01475f (0x3e39 bytes) 2024-09-14T22:37:03.099 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:37:03.105 INFO:teuthology.orchestra.run.smithi001.stdout:2403 punch from 0x8106bab to 0x810f09e, (0x84f3 bytes) 2024-09-14T22:37:03.105 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:03.121 INFO:teuthology.orchestra.run.smithi001.stdout:2405 trunc from 0xe014760 to 0x6dba5c 2024-09-14T22:37:03.175 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:03.171+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:03.196 INFO:teuthology.orchestra.run.smithi001.stdout:2406 writesame 0xc855294 thru 0xc85c060 data_size 0x1(0x6dcd bytes) 2024-09-14T22:37:03.196 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:03.196 INFO:teuthology.orchestra.run.smithi001.stdout:2408 writesame 0x65cb565 thru 0x65d074b data_size 0x1d(0x51e7 bytes) 2024-09-14T22:37:03.205 INFO:teuthology.orchestra.run.smithi001.stdout:2411 punch from 0xb86a77a to 0xb86c70d, (0x1f93 bytes) 2024-09-14T22:37:03.205 INFO:teuthology.orchestra.run.smithi001.stdout:2413 trunc from 0xc85c061 to 0xc9599da 2024-09-14T22:37:03.214 INFO:teuthology.orchestra.run.smithi001.stdout:2415 write 0x2cd340f thru 0x2cd38cc (0x4be bytes) 2024-09-14T22:37:03.217 INFO:teuthology.orchestra.run.smithi001.stdout:2416 trunc from 0xc9599da to 0xc15cf54 2024-09-14T22:37:03.226 INFO:teuthology.orchestra.run.smithi001.stdout:2417 write 0x4b4d58c thru 0x4b5a716 (0xd18b bytes) 2024-09-14T22:37:03.229 INFO:teuthology.orchestra.run.smithi001.stdout:2418 punch from 0x87a20ac to 0x87a71f8, (0x514c bytes) 2024-09-14T22:37:03.229 INFO:teuthology.orchestra.run.smithi001.stdout:2419 write 0xa9772ec thru 0xa9780f6 (0xe0b bytes) 2024-09-14T22:37:03.235 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:03.235 INFO:teuthology.orchestra.run.smithi001.stdout:2424 write 0x878d58c thru 0x878efc8 (0x1a3d bytes) 2024-09-14T22:37:03.242 INFO:teuthology.orchestra.run.smithi001.stdout:2426 read 0x708e60a thru 0x709c468 (0xde5f bytes) 2024-09-14T22:37:03.243 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:03.243 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:03.248 INFO:teuthology.orchestra.run.smithi001.stdout:2427 trunc from 0xc15cf54 to 0x90bda24 2024-09-14T22:37:03.261 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:37:03.261 INFO:teuthology.orchestra.run.smithi001.stdout:2428 read 0x74b67e9 thru 0x74c2621 (0xbe39 bytes) 2024-09-14T22:37:03.262 INFO:teuthology.orchestra.run.smithi001.stdout:2430 write 0x53069a thru 0x53ee3f (0xe7a6 bytes) 2024-09-14T22:37:03.276 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:03.271+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:03.278 INFO:teuthology.orchestra.run.smithi001.stdout:2431 trunc from 0x90bda24 to 0x165bcf0 2024-09-14T22:37:03.301 INFO:teuthology.orchestra.run.smithi001.stdout:2433 read 0x15a1a1c thru 0x15aaa90 (0x9075 bytes) 2024-09-14T22:37:03.327 INFO:teuthology.orchestra.run.smithi001.stdout:2434 trunc from 0x165bcf0 to 0xe4ca62c 2024-09-14T22:37:03.338 INFO:teuthology.orchestra.run.smithi001.stdout:2435 write 0x2f16b42 thru 0x2f17feb (0x14aa bytes) 2024-09-14T22:37:03.344 INFO:teuthology.orchestra.run.smithi001.stdout:2436 writesame 0x96572d0 thru 0x96602dc data_size 0x1(0x900d bytes) 2024-09-14T22:37:03.377 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:03.375+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:03.382 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:03.383 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:03.392 INFO:teuthology.orchestra.run.smithi001.stdout:2437 read 0xb8aa252 thru 0xb8b10b8 (0x6e67 bytes) 2024-09-14T22:37:03.394 INFO:teuthology.orchestra.run.smithi001.stdout:2440 trunc from 0xe4ca62c to 0xd9d2fb7 2024-09-14T22:37:03.396 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:37:03.404 INFO:teuthology.orchestra.run.smithi001.stdout:2441 write 0xb184280 thru 0xb18f49b (0xb21c bytes) 2024-09-14T22:37:03.413 INFO:teuthology.orchestra.run.smithi001.stdout:2442 writesame 0x92bf9e6 thru 0x92c4124 data_size 0x1(0x473f bytes) 2024-09-14T22:37:03.441 INFO:teuthology.orchestra.run.smithi001.stdout:2443 trunc from 0xd9d2fb7 to 0x3617bb0 2024-09-14T22:37:03.462 INFO:teuthology.orchestra.run.smithi001.stdout:2445 punch from 0x2450bfd to 0x245a19a, (0x959d bytes) 2024-09-14T22:37:03.478 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:03.471+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:03.496 INFO:teuthology.orchestra.run.smithi001.stdout:2446 write 0xe233254 thru 0xe241ac2 (0xe86f bytes) 2024-09-14T22:37:03.506 INFO:teuthology.orchestra.run.smithi001.stdout:2449 writesame 0x9f841a1 thru 0x9f88f40 data_size 0x20(0x4da0 bytes) 2024-09-14T22:37:03.512 INFO:teuthology.orchestra.run.smithi001.stdout:2450 read 0x909d0b0 thru 0x90a73dd (0xa32e bytes) 2024-09-14T22:37:03.512 INFO:teuthology.orchestra.run.smithi001.stdout:2452 write 0x18cd884 thru 0x18ce4e0 (0xc5d bytes) 2024-09-14T22:37:03.530 INFO:teuthology.orchestra.run.smithi001.stdout:2453 trunc from 0xe241ac3 to 0x79cf98b 2024-09-14T22:37:03.539 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:03.540 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:03.553 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:37:03.576 INFO:teuthology.orchestra.run.smithi001.stdout:2455 writesame 0xc2c3b12 thru 0xc2cb2b1 data_size 0x10(0x77a0 bytes) 2024-09-14T22:37:03.576 INFO:teuthology.orchestra.run.smithi001.stdout:2456 punch from 0x81c920f to 0x81d7f0d, (0xecfe bytes) 2024-09-14T22:37:03.595 INFO:teuthology.orchestra.run.smithi001.stdout:2457 writesame 0xe3ddde1 thru 0xe3dfe5f data_size 0x3(0x207f bytes) 2024-09-14T22:37:03.595 INFO:teuthology.orchestra.run.smithi001.stdout:2458 write 0x850c858 thru 0x8511dee (0x5597 bytes) 2024-09-14T22:37:03.595 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:03.591+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:03.602 INFO:teuthology.orchestra.run.smithi001.stdout:2459 read 0x806962a thru 0x8077b17 (0xe4ee bytes) 2024-09-14T22:37:03.602 INFO:teuthology.orchestra.run.smithi001.stdout:2460 read 0x7a3c047 thru 0x7a3e1d1 (0x218b bytes) 2024-09-14T22:37:03.603 INFO:teuthology.orchestra.run.smithi001.stdout:2461 punch from 0x46ca2de to 0x46d8fba, (0xecdc bytes) 2024-09-14T22:37:03.603 INFO:teuthology.orchestra.run.smithi001.stdout:2462 read 0x50e2738 thru 0x50e4972 (0x223b bytes) 2024-09-14T22:37:03.619 INFO:teuthology.orchestra.run.smithi001.stdout:2463 trunc from 0xe3dfe60 to 0x8394e0d 2024-09-14T22:37:03.634 INFO:teuthology.orchestra.run.smithi001.stdout:2464 writesame 0x5ed9b1c thru 0x5ee08db data_size 0x20(0x6dc0 bytes) 2024-09-14T22:37:03.647 INFO:teuthology.orchestra.run.smithi001.stdout:2465 trunc from 0x8394e0d to 0xa7f5a53 2024-09-14T22:37:03.660 INFO:teuthology.orchestra.run.smithi001.stdout:2467 read 0x831c376 thru 0x832747c (0xb107 bytes) 2024-09-14T22:37:03.660 INFO:teuthology.orchestra.run.smithi001.stdout:2468 read 0x79fbff thru 0x7a9cad (0xa0af bytes) 2024-09-14T22:37:03.662 INFO:teuthology.orchestra.run.smithi001.stdout:2471 read 0x6ba65a5 thru 0x6bb15b5 (0xb011 bytes) 2024-09-14T22:37:03.679 INFO:teuthology.orchestra.run.smithi001.stdout:2472 trunc from 0xa7f5a53 to 0x44e50d7 2024-09-14T22:37:03.681 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:03.675+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:03.727 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:03.727 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:03.737 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:37:03.738 INFO:teuthology.orchestra.run.smithi001.stdout:2473 write 0xa808b19 thru 0xa81612b (0xd613 bytes) 2024-09-14T22:37:03.744 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:37:03.747 INFO:teuthology.orchestra.run.smithi001.stdout:2474 read 0x913c542 thru 0x9143431 (0x6ef0 bytes) 2024-09-14T22:37:03.766 INFO:teuthology.orchestra.run.smithi001.stdout:2475 write 0xd71e8e2 thru 0xd728488 (0x9ba7 bytes) 2024-09-14T22:37:03.773 INFO:teuthology.orchestra.run.smithi001.stdout:2477 punch from 0xb79a09 to 0xb876e6, (0xdcdd bytes) 2024-09-14T22:37:03.788 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:03.783+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:03.789 INFO:teuthology.orchestra.run.smithi001.stdout:2478 trunc from 0xd728489 to 0x5de81d9 2024-09-14T22:37:03.808 INFO:teuthology.orchestra.run.smithi001.stdout:2479 write 0x355371b thru 0x35625f8 (0xeede bytes) 2024-09-14T22:37:03.812 INFO:teuthology.orchestra.run.smithi001.stdout:2481 trunc from 0x5de81d9 to 0x52dfe40 2024-09-14T22:37:03.822 INFO:teuthology.orchestra.run.smithi001.stdout:2482 writesame 0x70178 thru 0x7883a data_size 0x1(0x86c3 bytes) 2024-09-14T22:37:03.876 INFO:teuthology.orchestra.run.smithi001.stdout:2483 write 0xe80e926 thru 0xe8134e2 (0x4bbd bytes) 2024-09-14T22:37:03.883 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:03.875+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:03.883 INFO:teuthology.orchestra.run.smithi001.stdout:2485 read 0x76f2911 thru 0x76fd4d1 (0xabc1 bytes) 2024-09-14T22:37:03.883 INFO:teuthology.orchestra.run.smithi001.stdout:2486 punch from 0x6d90dff to 0x6da0278, (0xf479 bytes) 2024-09-14T22:37:03.894 INFO:teuthology.orchestra.run.smithi001.stdout:2487 trunc from 0xe8134e3 to 0x7166fd8 2024-09-14T22:37:03.902 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:37:03.902 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:37:03.917 INFO:teuthology.orchestra.run.smithi001.stdout:2489 read 0x4bb7da0 thru 0x4bbdf3b (0x619c bytes) 2024-09-14T22:37:03.917 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:03.920 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:37:03.921 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:37:03.921 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-14T22:37:03.924 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:03.924 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:03.927 INFO:teuthology.orchestra.run.smithi001.stdout:2491 writesame 0x85cf0f8 thru 0x85dca83 data_size 0x54(0xd98c bytes) 2024-09-14T22:37:03.927 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:03.927 INFO:teuthology.orchestra.run.smithi001.stdout:2495 read 0x6bca5e6 thru 0x6bcee56 (0x4871 bytes) 2024-09-14T22:37:03.928 INFO:teuthology.orchestra.run.smithi001.stdout:2496 read 0x252a56e thru 0x252b358 (0xdeb bytes) 2024-09-14T22:37:03.928 INFO:teuthology.orchestra.run.smithi001.stdout:2499 read 0x2c8356a thru 0x2c85f0b (0x29a2 bytes) 2024-09-14T22:37:03.928 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:03.928 INFO:teuthology.orchestra.run.smithi001.stdout:2502 punch from 0x275ebd5 to 0x2766985, (0x7db0 bytes) 2024-09-14T22:37:03.938 INFO:teuthology.orchestra.run.smithi001.stdout:2503 write 0x980006a thru 0x98026ef (0x2686 bytes) 2024-09-14T22:37:03.939 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:37:03.940 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:37:03.940 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:37:03.942 INFO:teuthology.orchestra.run.smithi001.stdout:2504 read 0x398c0d5 thru 0x3995386 (0x92b2 bytes) 2024-09-14T22:37:03.942 INFO:teuthology.orchestra.run.smithi001.stdout:2506 write 0x7ac3670 thru 0x7acbfe7 (0x8978 bytes) 2024-09-14T22:37:03.948 INFO:teuthology.orchestra.run.smithi001.stdout:2509 read 0x1b13bb4 thru 0x1b2050a (0xc957 bytes) 2024-09-14T22:37:03.948 INFO:teuthology.orchestra.run.smithi001.stdout:2510 punch from 0x1d09397 to 0x1d09c5d, (0x8c6 bytes) 2024-09-14T22:37:03.957 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:37:03.958 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:37:03.958 INFO:teuthology.orchestra.run.smithi001.stdout:2511 write 0xd32d334 thru 0xd33abc8 (0xd895 bytes) 2024-09-14T22:37:03.966 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:03.967 INFO:teuthology.orchestra.run.smithi001.stdout:2513 trunc from 0xd33abc9 to 0xcd57632 2024-09-14T22:37:03.975 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:03.975 INFO:teuthology.orchestra.run.smithi001.stdout:2515 punch from 0x6020388 to 0x602867c, (0x82f4 bytes) 2024-09-14T22:37:03.975 INFO:teuthology.orchestra.run.smithi001.stdout:2516 punch from 0xc770eed to 0xc77f720, (0xe833 bytes) 2024-09-14T22:37:03.975 INFO:teuthology.orchestra.run.smithi001.stdout:2518 punch from 0xae5da35 to 0xae6689a, (0x8e65 bytes) 2024-09-14T22:37:03.975 INFO:teuthology.orchestra.run.smithi001.stdout:2520 writesame 0x339a2b0 thru 0x339bbd9 data_size 0x1(0x192a bytes) 2024-09-14T22:37:03.977 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:37:03.977 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:37:03.978 INFO:teuthology.orchestra.run.smithi001.stdout:2521 punch from 0x3d40598 to 0x3d434dc, (0x2f44 bytes) 2024-09-14T22:37:03.978 INFO:teuthology.orchestra.run.smithi001.stdout:2522 writesame 0x3586a5a thru 0x3593443 data_size 0xf(0xc9ea bytes) 2024-09-14T22:37:03.982 INFO:teuthology.orchestra.run.smithi001.stdout:2523 read 0x842b21d thru 0x842e2c2 (0x30a6 bytes) 2024-09-14T22:37:03.982 INFO:teuthology.orchestra.run.smithi001.stdout:2524 read 0xa62e91d thru 0xa631012 (0x26f6 bytes) 2024-09-14T22:37:03.982 INFO:teuthology.orchestra.run.smithi001.stdout:2525 punch from 0x52d09c7 to 0x52d90a7, (0x86e0 bytes) 2024-09-14T22:37:03.982 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:03.982 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:03.979+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:03.996 INFO:teuthology.orchestra.run.smithi001.stdout:2528 trunc from 0xcd57632 to 0x5250874 2024-09-14T22:37:03.998 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:37:03.998 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:37:04.018 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:37:04.018 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:37:04.021 INFO:teuthology.orchestra.run.smithi001.stdout:2529 trunc from 0x5250874 to 0x23ed7de 2024-09-14T22:37:04.022 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:37:04.022 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:37:04.037 INFO:teuthology.orchestra.run.smithi001.stdout:2530 read 0x6fc5c7 thru 0x7028c2 (0x62fc bytes) 2024-09-14T22:37:04.038 INFO:teuthology.orchestra.run.smithi001.stdout:2532 write 0x4c60e2 thru 0x4c7878 (0x1797 bytes) 2024-09-14T22:37:04.078 INFO:teuthology.orchestra.run.smithi001.stdout:2533 writesame 0xe55af61 thru 0xe5674d0 data_size 0x30(0xc570 bytes) 2024-09-14T22:37:04.078 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:04.078 INFO:teuthology.orchestra.run.smithi001.stdout:2535 write 0x2cf3875 thru 0x2cffa31 (0xc1bd bytes) 2024-09-14T22:37:04.082 INFO:teuthology.orchestra.run.smithi001.stdout:2536 read 0xbabf6a7 thru 0xbac5518 (0x5e72 bytes) 2024-09-14T22:37:04.082 INFO:teuthology.orchestra.run.smithi001.stdout:2537 read 0x78715cf thru 0x787c226 (0xac58 bytes) 2024-09-14T22:37:04.082 INFO:teuthology.orchestra.run.smithi001.stdout:2539 read 0x8023ef thru 0x803a28 (0x163a bytes) 2024-09-14T22:37:04.082 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:04.075+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:04.083 INFO:teuthology.orchestra.run.smithi001.stdout:2540 punch from 0x52971fb to 0x529e4de, (0x72e3 bytes) 2024-09-14T22:37:04.116 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:04.117 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:04.117 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:37:04.117 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:04.117 INFO:teuthology.orchestra.run.smithi001.stdout:2541 trunc from 0xe5674d1 to 0xed0e6 2024-09-14T22:37:04.132 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:37:04.167 INFO:teuthology.orchestra.run.smithi001.stdout:2542 write 0x89dadec thru 0x89e3b5c (0x8d71 bytes) 2024-09-14T22:37:04.177 INFO:teuthology.orchestra.run.smithi001.stdout:2543 write 0x2656eb8 thru 0x2665ff2 (0xf13b bytes) 2024-09-14T22:37:04.181 INFO:teuthology.orchestra.run.smithi001.stdout:2544 punch from 0x411f3d0 to 0x4122fd3, (0x3c03 bytes) 2024-09-14T22:37:04.181 INFO:teuthology.orchestra.run.smithi001.stdout:2545 write 0x14899ed thru 0x148f5c0 (0x5bd4 bytes) 2024-09-14T22:37:04.197 INFO:teuthology.orchestra.run.smithi001.stdout:2546 trunc from 0x89e3b5d to 0xd68da67 2024-09-14T22:37:04.209 INFO:teuthology.orchestra.run.smithi001.stdout:2547 read 0x5b289ef thru 0x5b34720 (0xbd32 bytes) 2024-09-14T22:37:04.209 INFO:teuthology.orchestra.run.smithi001.stdout:2549 punch from 0xa809add to 0xa810664, (0x6b87 bytes) 2024-09-14T22:37:04.210 INFO:teuthology.orchestra.run.smithi001.stdout:2552 write 0x973344 thru 0x975ae8 (0x27a5 bytes) 2024-09-14T22:37:04.213 INFO:teuthology.orchestra.run.smithi001.stdout:2553 read 0xc201b2e thru 0xc20457e (0x2a51 bytes) 2024-09-14T22:37:04.213 INFO:teuthology.orchestra.run.smithi001.stdout:2554 punch from 0x38e252d to 0x38e8c11, (0x66e4 bytes) 2024-09-14T22:37:04.213 INFO:teuthology.orchestra.run.smithi001.stdout:2555 write 0x1f92a20 thru 0x1f9d15b (0xa73c bytes) 2024-09-14T22:37:04.217 INFO:teuthology.orchestra.run.smithi001.stdout:2557 writesame 0x2d5c4d6 thru 0x2d65b9c data_size 0x1d(0x96c7 bytes) 2024-09-14T22:37:04.221 INFO:teuthology.orchestra.run.smithi001.stdout:2558 read 0x60330b9 thru 0x6034744 (0x168c bytes) 2024-09-14T22:37:04.221 INFO:teuthology.orchestra.run.smithi001.stdout:2559 read 0x2e50e29 thru 0x2e554da (0x46b2 bytes) 2024-09-14T22:37:04.222 INFO:teuthology.orchestra.run.smithi001.stdout:2560 read 0xab3f8b8 thru 0xab4c57b (0xccc4 bytes) 2024-09-14T22:37:04.222 INFO:teuthology.orchestra.run.smithi001.stdout:2561 read 0x173e0dc thru 0x1745661 (0x7586 bytes) 2024-09-14T22:37:04.223 INFO:teuthology.orchestra.run.smithi001.stdout:2563 write 0x2225bab thru 0x2235210 (0xf666 bytes) 2024-09-14T22:37:04.224 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:04.219+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:04.226 INFO:teuthology.orchestra.run.smithi001.stdout:2564 read 0x5bdfb76 thru 0x5be982f (0x9cba bytes) 2024-09-14T22:37:04.240 INFO:teuthology.orchestra.run.smithi001.stdout:2565 trunc from 0xd68da67 to 0x812ca4d 2024-09-14T22:37:04.255 INFO:teuthology.orchestra.run.smithi001.stdout:2566 write 0x44941e6 thru 0x4495988 (0x17a3 bytes) 2024-09-14T22:37:04.266 INFO:teuthology.orchestra.run.smithi001.stdout:2567 trunc from 0x812ca4d to 0x97e7145 2024-09-14T22:37:04.279 INFO:teuthology.orchestra.run.smithi001.stdout:2568 read 0x1a1d2b3 thru 0x1a25d59 (0x8aa7 bytes) 2024-09-14T22:37:04.280 INFO:teuthology.orchestra.run.smithi001.stdout:2570 write 0x2ac9235 thru 0x2ad797c (0xe748 bytes) 2024-09-14T22:37:04.284 INFO:teuthology.orchestra.run.smithi001.stdout:2571 read 0x27455af thru 0x27500e6 (0xab38 bytes) 2024-09-14T22:37:04.284 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:04.279+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:04.297 INFO:teuthology.orchestra.run.smithi001.stdout:2573 trunc from 0x97e7145 to 0xdb7a0e2 2024-09-14T22:37:04.305 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:04.305 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:04.305 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:37:04.305 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:37:04.306 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:04.320 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:37:04.324 INFO:teuthology.orchestra.run.smithi001.stdout:2576 write 0xea25473 thru 0xea30351 (0xaedf bytes) 2024-09-14T22:37:04.363 INFO:teuthology.orchestra.run.smithi001.stdout:2578 trunc from 0xea30352 to 0x27c66bc 2024-09-14T22:37:04.385 INFO:teuthology.orchestra.run.smithi001.stdout:2579 read 0x944691 thru 0x94a0d0 (0x5a40 bytes) 2024-09-14T22:37:04.386 INFO:teuthology.orchestra.run.smithi001.stdout:2581 clone 18 order 25 su 1048576 sc 14 2024-09-14T22:37:04.411 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:04.407+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:04.486 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:04.483+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:04.487 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:04.487 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:04.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:04.487 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:04.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.487 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20869 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7168000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:37:04.487 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483633+0000", 2024-09-14T22:37:04.487 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 31.998021908999998, 2024-09-14T22:37:04.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028181490999999999, 2024-09-14T22:37:04.487 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:04.487 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:04.488 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:04.488 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:04.488 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:04.488 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:04.488 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20869 2024-09-14T22:37:04.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.488 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:04.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.488 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:04.488 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483633+0000", 2024-09-14T22:37:04.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:04.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483633+0000", 2024-09-14T22:37:04.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:04.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483634+0000", 2024-09-14T22:37:04.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2599999999999998e-07 2024-09-14T22:37:04.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:04.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483636+0000", 2024-09-14T22:37:04.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1679999999999998e-06 2024-09-14T22:37:04.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:04.490 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483642+0000", 2024-09-14T22:37:04.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2149999999999998e-06 2024-09-14T22:37:04.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:04.490 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483647+0000", 2024-09-14T22:37:04.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7679999999999999e-06 2024-09-14T22:37:04.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:04.490 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.510694+0000", 2024-09-14T22:37:04.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027047073000000001 2024-09-14T22:37:04.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:04.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.510771+0000", 2024-09-14T22:37:04.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6814e-05 2024-09-14T22:37:04.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:04.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511815+0000", 2024-09-14T22:37:04.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0010440270000000001 2024-09-14T22:37:04.491 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.491 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:04.491 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.492 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20870 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7196672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:37:04.492 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483652+0000", 2024-09-14T22:37:04.492 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 31.998002907, 2024-09-14T22:37:04.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028301248000000001, 2024-09-14T22:37:04.492 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:04.492 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:04.492 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:04.492 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:04.492 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:04.492 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:04.492 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20870 2024-09-14T22:37:04.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.493 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:04.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.493 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:04.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483652+0000", 2024-09-14T22:37:04.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.493 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:04.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483652+0000", 2024-09-14T22:37:04.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.494 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:04.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483653+0000", 2024-09-14T22:37:04.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3099999999999997e-07 2024-09-14T22:37:04.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.494 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:04.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483655+0000", 2024-09-14T22:37:04.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.877e-06 2024-09-14T22:37:04.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.494 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:04.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483661+0000", 2024-09-14T22:37:04.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6049999999999997e-06 2024-09-14T22:37:04.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.495 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:04.495 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483666+0000", 2024-09-14T22:37:04.495 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3410000000000002e-06 2024-09-14T22:37:04.495 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.495 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:04.495 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511854+0000", 2024-09-14T22:37:04.495 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028187773999999999 2024-09-14T22:37:04.495 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.495 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:04.496 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511906+0000", 2024-09-14T22:37:04.496 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2012000000000003e-05 2024-09-14T22:37:04.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.496 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:04.496 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511954+0000", 2024-09-14T22:37:04.496 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8007999999999998e-05 2024-09-14T22:37:04.496 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.496 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:04.496 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.497 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20871 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7225344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:37:04.497 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483671+0000", 2024-09-14T22:37:04.497 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 31.997983984000001, 2024-09-14T22:37:04.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028405392000000002, 2024-09-14T22:37:04.497 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:04.497 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:04.497 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:04.497 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:04.497 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:04.497 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:04.497 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20871 2024-09-14T22:37:04.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.497 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:04.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:04.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483671+0000", 2024-09-14T22:37:04.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:04.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483671+0000", 2024-09-14T22:37:04.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:04.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483672+0000", 2024-09-14T22:37:04.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:37:04.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:04.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483674+0000", 2024-09-14T22:37:04.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3190000000000002e-06 2024-09-14T22:37:04.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:04.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483680+0000", 2024-09-14T22:37:04.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6860000000000003e-06 2024-09-14T22:37:04.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.500 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:04.500 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483684+0000", 2024-09-14T22:37:04.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3769999999999998e-06 2024-09-14T22:37:04.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.500 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:04.500 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511982+0000", 2024-09-14T22:37:04.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028297478000000001 2024-09-14T22:37:04.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.500 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:04.500 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512027+0000", 2024-09-14T22:37:04.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5290000000000002e-05 2024-09-14T22:37:04.501 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.501 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:04.501 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512077+0000", 2024-09-14T22:37:04.501 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9876000000000002e-05 2024-09-14T22:37:04.501 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.501 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:04.501 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.501 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.501 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20872 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7254016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:37:04.501 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483689+0000", 2024-09-14T22:37:04.501 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 31.997965914000002, 2024-09-14T22:37:04.502 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028494049, 2024-09-14T22:37:04.502 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:04.502 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:04.502 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:04.502 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:04.502 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:04.502 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:04.502 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20872 2024-09-14T22:37:04.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.502 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:04.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.502 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:04.502 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483689+0000", 2024-09-14T22:37:04.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.503 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:04.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483689+0000", 2024-09-14T22:37:04.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.503 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:04.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483690+0000", 2024-09-14T22:37:04.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:37:04.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.503 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:04.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483692+0000", 2024-09-14T22:37:04.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.778e-06 2024-09-14T22:37:04.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:04.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483697+0000", 2024-09-14T22:37:04.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7069999999999997e-06 2024-09-14T22:37:04.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:04.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483702+0000", 2024-09-14T22:37:04.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4689999999999999e-06 2024-09-14T22:37:04.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:04.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512104+0000", 2024-09-14T22:37:04.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028401948999999999 2024-09-14T22:37:04.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:04.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512142+0000", 2024-09-14T22:37:04.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8114999999999997e-05 2024-09-14T22:37:04.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:04.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512183+0000", 2024-09-14T22:37:04.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1496999999999999e-05 2024-09-14T22:37:04.505 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.506 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:04.506 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.506 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20873 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7282688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:37:04.506 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483707+0000", 2024-09-14T22:37:04.506 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 31.9979482, 2024-09-14T22:37:04.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028581662000000001, 2024-09-14T22:37:04.506 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:04.506 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:04.506 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:04.506 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:04.506 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:04.506 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:04.507 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20873 2024-09-14T22:37:04.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.507 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:04.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.507 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:04.507 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483707+0000", 2024-09-14T22:37:04.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.507 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:04.507 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483707+0000", 2024-09-14T22:37:04.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.507 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:04.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483707+0000", 2024-09-14T22:37:04.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2500000000000001e-07 2024-09-14T22:37:04.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:04.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483709+0000", 2024-09-14T22:37:04.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.891e-06 2024-09-14T22:37:04.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:04.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483715+0000", 2024-09-14T22:37:04.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7019999999999998e-06 2024-09-14T22:37:04.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:04.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483720+0000", 2024-09-14T22:37:04.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4900000000000002e-06 2024-09-14T22:37:04.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:04.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512213+0000", 2024-09-14T22:37:04.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028493622 2024-09-14T22:37:04.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:04.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512248+0000", 2024-09-14T22:37:04.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5228000000000002e-05 2024-09-14T22:37:04.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:04.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512289+0000", 2024-09-14T22:37:04.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0404000000000001e-05 2024-09-14T22:37:04.510 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.510 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:04.510 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.510 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20874 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7311360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:37:04.510 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483725+0000", 2024-09-14T22:37:04.510 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 31.997930561, 2024-09-14T22:37:04.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028777193999999999, 2024-09-14T22:37:04.510 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:04.510 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:04.510 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:04.511 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:04.511 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:04.511 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:04.511 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20874 2024-09-14T22:37:04.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.511 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:04.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.511 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:04.511 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483725+0000", 2024-09-14T22:37:04.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.511 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:04.511 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483725+0000", 2024-09-14T22:37:04.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:04.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483725+0000", 2024-09-14T22:37:04.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8000000000000001e-07 2024-09-14T22:37:04.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:04.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483727+0000", 2024-09-14T22:37:04.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8920000000000001e-06 2024-09-14T22:37:04.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:04.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483733+0000", 2024-09-14T22:37:04.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6840000000000001e-06 2024-09-14T22:37:04.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:04.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483737+0000", 2024-09-14T22:37:04.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4440000000000001e-06 2024-09-14T22:37:04.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:04.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512326+0000", 2024-09-14T22:37:04.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028588612999999999 2024-09-14T22:37:04.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:04.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512399+0000", 2024-09-14T22:37:04.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2782999999999998e-05 2024-09-14T22:37:04.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:04.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512502+0000", 2024-09-14T22:37:04.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000103398 2024-09-14T22:37:04.514 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.514 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:04.514 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.515 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20912 3.1c 3:3b9f022e:::rbd_data.113c4558bb1.000000000000009a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 497664~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:37:04.515 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.484800+0000", 2024-09-14T22:37:04.515 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 31.996855032999999, 2024-09-14T22:37:04.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045787688999999999, 2024-09-14T22:37:04.515 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:04.515 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:04.515 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:04.515 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:04.515 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:04.515 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:04.515 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20912 2024-09-14T22:37:04.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.516 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:04.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.516 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:04.516 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484800+0000", 2024-09-14T22:37:04.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.516 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:04.516 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484800+0000", 2024-09-14T22:37:04.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.516 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:04.517 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484801+0000", 2024-09-14T22:37:04.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1939999999999999e-06 2024-09-14T22:37:04.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.517 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:04.517 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484809+0000", 2024-09-14T22:37:04.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2989999999999999e-06 2024-09-14T22:37:04.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.517 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:04.517 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484823+0000", 2024-09-14T22:37:04.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3986e-05 2024-09-14T22:37:04.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:04.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484831+0000", 2024-09-14T22:37:04.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4009999999999997e-06 2024-09-14T22:37:04.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:04.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512318+0000", 2024-09-14T22:37:04.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027487208999999999 2024-09-14T22:37:04.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:04.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512450+0000", 2024-09-14T22:37:04.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000131681 2024-09-14T22:37:04.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:37:04.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512666+0000", 2024-09-14T22:37:04.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021547 2024-09-14T22:37:04.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:04.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.513788+0000", 2024-09-14T22:37:04.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001122975 2024-09-14T22:37:04.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:04.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.530473+0000", 2024-09-14T22:37:04.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.016684984999999999 2024-09-14T22:37:04.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:04.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.530533+0000", 2024-09-14T22:37:04.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9296999999999998e-05 2024-09-14T22:37:04.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:04.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.530588+0000", 2024-09-14T22:37:04.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5192000000000003e-05 2024-09-14T22:37:04.520 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.520 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:04.520 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.521 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:30512 3.1d 3:b9e08d52:::rbd_data.113cda0183a0.0000000000000008:6 [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:04.521 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.331867+0000", 2024-09-14T22:37:04.521 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 31.149788079, 2024-09-14T22:37:04.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027901088000000001, 2024-09-14T22:37:04.521 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:04.521 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:04.521 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:04.521 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:04.521 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:04.521 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:04.521 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 30512 2024-09-14T22:37:04.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.521 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:04.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:04.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331867+0000", 2024-09-14T22:37:04.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:04.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331867+0000", 2024-09-14T22:37:04.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:04.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331868+0000", 2024-09-14T22:37:04.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8999999999999996e-07 2024-09-14T22:37:04.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:04.523 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331871+0000", 2024-09-14T22:37:04.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.163e-06 2024-09-14T22:37:04.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:04.523 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331880+0000", 2024-09-14T22:37:04.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3449999999999995e-06 2024-09-14T22:37:04.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:04.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331888+0000", 2024-09-14T22:37:04.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5320000000000001e-06 2024-09-14T22:37:04.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:04.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.358443+0000", 2024-09-14T22:37:04.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026554845000000001 2024-09-14T22:37:04.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:04.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.359649+0000", 2024-09-14T22:37:04.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001205991 2024-09-14T22:37:04.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:04.525 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.359768+0000", 2024-09-14T22:37:04.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119522 2024-09-14T22:37:04.525 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.525 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:04.525 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.525 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:30513 3.1d 3:b9e08d52:::rbd_data.113cda0183a0.0000000000000008:6 [read 456704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:04.525 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.331897+0000", 2024-09-14T22:37:04.525 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 31.149757853000001, 2024-09-14T22:37:04.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028098122999999999, 2024-09-14T22:37:04.525 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:04.525 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:04.526 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:04.526 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:04.526 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:04.526 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:04.526 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 30513 2024-09-14T22:37:04.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.526 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:04.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.526 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:04.526 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331897+0000", 2024-09-14T22:37:04.526 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.526 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:04.526 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331897+0000", 2024-09-14T22:37:04.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:04.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331898+0000", 2024-09-14T22:37:04.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:37:04.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:04.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331901+0000", 2024-09-14T22:37:04.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7889999999999999e-06 2024-09-14T22:37:04.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:04.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331910+0000", 2024-09-14T22:37:04.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.346e-06 2024-09-14T22:37:04.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.528 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:04.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331917+0000", 2024-09-14T22:37:04.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2239999999999998e-06 2024-09-14T22:37:04.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.528 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:04.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.359807+0000", 2024-09-14T22:37:04.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027889225 2024-09-14T22:37:04.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:04.529 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.359895+0000", 2024-09-14T22:37:04.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8721000000000007e-05 2024-09-14T22:37:04.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:04.529 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.359996+0000", 2024-09-14T22:37:04.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010022300000000001 2024-09-14T22:37:04.529 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.529 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:04.529 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.529 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:30514 3.1d 3:b9e08d52:::rbd_data.113cda0183a0.0000000000000008:6 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:04.530 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.331927+0000", 2024-09-14T22:37:04.530 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 31.149728039999999, 2024-09-14T22:37:04.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028476330000000001, 2024-09-14T22:37:04.530 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:04.530 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:04.530 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:04.530 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:04.530 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:04.530 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:04.530 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 30514 2024-09-14T22:37:04.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.530 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:04.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:04.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331927+0000", 2024-09-14T22:37:04.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:04.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331927+0000", 2024-09-14T22:37:04.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:04.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331928+0000", 2024-09-14T22:37:04.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5700000000000002e-07 2024-09-14T22:37:04.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.532 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:04.532 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331931+0000", 2024-09-14T22:37:04.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9909999999999998e-06 2024-09-14T22:37:04.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.532 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:04.532 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331940+0000", 2024-09-14T22:37:04.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.312e-06 2024-09-14T22:37:04.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.532 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:04.532 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331948+0000", 2024-09-14T22:37:04.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5630000000000003e-06 2024-09-14T22:37:04.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:04.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360032+0000", 2024-09-14T22:37:04.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028083877 2024-09-14T22:37:04.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:04.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360282+0000", 2024-09-14T22:37:04.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00025062100000000003 2024-09-14T22:37:04.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:04.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360404+0000", 2024-09-14T22:37:04.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000121409 2024-09-14T22:37:04.534 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.534 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:04.534 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.534 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:30515 3.1d 3:b9e08d52:::rbd_data.113cda0183a0.0000000000000008:6 [read 514048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:04.534 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.331957+0000", 2024-09-14T22:37:04.534 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 31.149698215000001, 2024-09-14T22:37:04.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028684729999999999, 2024-09-14T22:37:04.534 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:04.534 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:04.534 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:04.534 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:04.535 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:04.535 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:04.535 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 30515 2024-09-14T22:37:04.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.535 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:04.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:04.535 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331957+0000", 2024-09-14T22:37:04.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:04.535 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331957+0000", 2024-09-14T22:37:04.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:04.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331958+0000", 2024-09-14T22:37:04.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1200000000000003e-07 2024-09-14T22:37:04.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:04.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331960+0000", 2024-09-14T22:37:04.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6929999999999999e-06 2024-09-14T22:37:04.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:04.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331970+0000", 2024-09-14T22:37:04.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4020000000000008e-06 2024-09-14T22:37:04.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:04.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331977+0000", 2024-09-14T22:37:04.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7660000000000001e-06 2024-09-14T22:37:04.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:04.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360444+0000", 2024-09-14T22:37:04.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028467399000000001 2024-09-14T22:37:04.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:04.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360536+0000", 2024-09-14T22:37:04.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1618000000000003e-05 2024-09-14T22:37:04.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.538 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:04.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360642+0000", 2024-09-14T22:37:04.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010624 2024-09-14T22:37:04.538 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.538 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:04.538 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.538 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:46576 3.1d 3:b9b1f07f:::rbd_data.113c8642facc.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T22:37:04.538 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:37.379223+0000", 2024-09-14T22:37:04.538 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.102432690000001, 2024-09-14T22:37:04.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072706044999999997, 2024-09-14T22:37:04.538 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:04.538 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:04.539 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:04.539 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:04.539 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:04.539 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:04.539 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 46576 2024-09-14T22:37:04.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.539 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:04.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:04.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379223+0000", 2024-09-14T22:37:04.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:04.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379223+0000", 2024-09-14T22:37:04.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:04.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379226+0000", 2024-09-14T22:37:04.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0929999999999999e-06 2024-09-14T22:37:04.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:04.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379240+0000", 2024-09-14T22:37:04.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4606999999999999e-05 2024-09-14T22:37:04.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:04.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379272+0000", 2024-09-14T22:37:04.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2073000000000003e-05 2024-09-14T22:37:04.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:04.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379288+0000", 2024-09-14T22:37:04.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6036999999999999e-05 2024-09-14T22:37:04.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:04.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379451+0000", 2024-09-14T22:37:04.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000162795 2024-09-14T22:37:04.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:04.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379702+0000", 2024-09-14T22:37:04.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00025106500000000002 2024-09-14T22:37:04.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:37:04.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.394067+0000", 2024-09-14T22:37:04.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014364564999999999 2024-09-14T22:37:04.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:04.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.451737+0000", 2024-09-14T22:37:04.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057670185999999998 2024-09-14T22:37:04.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:04.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.451823+0000", 2024-09-14T22:37:04.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6075000000000005e-05 2024-09-14T22:37:04.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:04.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.451881+0000", 2024-09-14T22:37:04.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7454999999999998e-05 2024-09-14T22:37:04.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:04.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.451929+0000", 2024-09-14T22:37:04.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8093999999999997e-05 2024-09-14T22:37:04.543 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.543 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:04.544 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.544 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:74169 3.16 3:69acee67:::rbd_data.113cce28790.0000000000000067:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T22:37:04.544 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:55.614872+0000", 2024-09-14T22:37:04.544 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.8667835040000007, 2024-09-14T22:37:04.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036045858, 2024-09-14T22:37:04.544 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:04.544 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:04.544 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:04.544 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:04.544 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:04.544 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:04.545 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74169 2024-09-14T22:37:04.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.545 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:04.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:04.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.614872+0000", 2024-09-14T22:37:04.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:04.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.614872+0000", 2024-09-14T22:37:04.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:04.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.614875+0000", 2024-09-14T22:37:04.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2799999999999999e-06 2024-09-14T22:37:04.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:04.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.614891+0000", 2024-09-14T22:37:04.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5613000000000001e-05 2024-09-14T22:37:04.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:04.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.614931+0000", 2024-09-14T22:37:04.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0540000000000001e-05 2024-09-14T22:37:04.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.547 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:04.547 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.614956+0000", 2024-09-14T22:37:04.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4939e-05 2024-09-14T22:37:04.547 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.547 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:04.547 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.615096+0000", 2024-09-14T22:37:04.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013952600000000001 2024-09-14T22:37:04.547 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.547 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:04.547 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.615345+0000", 2024-09-14T22:37:04.548 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024922099999999999 2024-09-14T22:37:04.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.548 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.548 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T22:37:04.548 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.621370+0000", 2024-09-14T22:37:04.548 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0060251239999999998 2024-09-14T22:37:04.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.548 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.548 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:04.548 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.650734+0000", 2024-09-14T22:37:04.548 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029364155999999999 2024-09-14T22:37:04.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.549 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:04.549 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.650760+0000", 2024-09-14T22:37:04.549 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5446999999999999e-05 2024-09-14T22:37:04.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.549 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:04.549 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.650842+0000", 2024-09-14T22:37:04.549 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2268999999999994e-05 2024-09-14T22:37:04.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.549 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:04.549 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.650918+0000", 2024-09-14T22:37:04.549 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5742999999999996e-05 2024-09-14T22:37:04.550 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.550 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:04.550 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.550 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.550 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:85215 3.16 3:688cf0a4:::rbd_data.113cff0d0fb7.000000000000000a:12 [read 344064~12288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:04.550 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:00.251379+0000", 2024-09-14T22:37:04.550 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.2302761679999996, 2024-09-14T22:37:04.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027899154999999998, 2024-09-14T22:37:04.550 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:04.550 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:04.550 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:04.550 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:04.551 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:04.551 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:04.551 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 85215 2024-09-14T22:37:04.551 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.551 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:04.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.551 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:04.551 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.251379+0000", 2024-09-14T22:37:04.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.551 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.551 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:04.551 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.251379+0000", 2024-09-14T22:37:04.552 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.552 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:04.552 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.251380+0000", 2024-09-14T22:37:04.552 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.77e-07 2024-09-14T22:37:04.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.552 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:04.552 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.251382+0000", 2024-09-14T22:37:04.552 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1610000000000001e-06 2024-09-14T22:37:04.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.552 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:04.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.251387+0000", 2024-09-14T22:37:04.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7180000000000001e-06 2024-09-14T22:37:04.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.553 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.553 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:04.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.251392+0000", 2024-09-14T22:37:04.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.78e-06 2024-09-14T22:37:04.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.553 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.553 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:04.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.279091+0000", 2024-09-14T22:37:04.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027699242999999998 2024-09-14T22:37:04.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.553 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.553 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:04.554 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.279182+0000", 2024-09-14T22:37:04.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0383999999999994e-05 2024-09-14T22:37:04.554 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.554 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:04.554 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.279278+0000", 2024-09-14T22:37:04.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6491999999999997e-05 2024-09-14T22:37:04.554 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.554 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:04.554 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.554 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.554 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:85216 3.16 3:688cf0a4:::rbd_data.113cff0d0fb7.000000000000000a:12 [read 434176~12288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:04.554 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:00.251398+0000", 2024-09-14T22:37:04.554 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.2302572319999996, 2024-09-14T22:37:04.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028095181, 2024-09-14T22:37:04.555 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:04.555 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:04.555 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:04.555 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:04.555 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:04.555 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:04.555 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 85216 2024-09-14T22:37:04.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.555 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:04.555 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.555 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:04.555 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.251398+0000", 2024-09-14T22:37:04.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.556 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.556 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:04.556 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.251398+0000", 2024-09-14T22:37:04.556 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.556 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.556 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:04.556 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.251398+0000", 2024-09-14T22:37:04.556 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9200000000000002e-07 2024-09-14T22:37:04.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.556 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.556 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:04.556 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.251400+0000", 2024-09-14T22:37:04.556 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9599999999999999e-06 2024-09-14T22:37:04.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.557 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:04.557 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.251406+0000", 2024-09-14T22:37:04.557 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.643e-06 2024-09-14T22:37:04.557 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.557 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:04.557 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.251410+0000", 2024-09-14T22:37:04.557 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2459999999999997e-06 2024-09-14T22:37:04.557 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.557 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:04.557 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.279312+0000", 2024-09-14T22:37:04.557 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027901433 2024-09-14T22:37:04.557 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.558 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:04.558 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.279393+0000", 2024-09-14T22:37:04.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1384999999999994e-05 2024-09-14T22:37:04.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.558 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:04.558 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.279493+0000", 2024-09-14T22:37:04.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000100122 2024-09-14T22:37:04.558 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.558 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:04.558 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.558 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:85373 3.16 3:688cf0a4:::rbd_data.113cff0d0fb7.000000000000000a:12 [read 172032~12288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:04.558 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:00.254699+0000", 2024-09-14T22:37:04.559 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.2269561419999997, 2024-09-14T22:37:04.559 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028028539000000002, 2024-09-14T22:37:04.559 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:04.559 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:04.559 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:04.559 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:04.559 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:04.559 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:04.559 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 85373 2024-09-14T22:37:04.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.559 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:04.559 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.559 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:04.559 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254699+0000", 2024-09-14T22:37:04.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.560 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.560 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.560 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:04.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254699+0000", 2024-09-14T22:37:04.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.560 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.560 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.560 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:04.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254700+0000", 2024-09-14T22:37:04.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.46e-07 2024-09-14T22:37:04.560 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.560 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.560 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:04.561 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254702+0000", 2024-09-14T22:37:04.561 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1739999999999999e-06 2024-09-14T22:37:04.561 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.561 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.561 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:04.561 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254708+0000", 2024-09-14T22:37:04.561 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.173e-06 2024-09-14T22:37:04.561 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.561 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.561 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:04.561 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254713+0000", 2024-09-14T22:37:04.561 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6040000000000003e-06 2024-09-14T22:37:04.562 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.562 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.562 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:04.562 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.282544+0000", 2024-09-14T22:37:04.562 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027831655 2024-09-14T22:37:04.562 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.562 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.562 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:04.562 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.282627+0000", 2024-09-14T22:37:04.562 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2891000000000003e-05 2024-09-14T22:37:04.562 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.562 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.562 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:04.562 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.282728+0000", 2024-09-14T22:37:04.562 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000100596 2024-09-14T22:37:04.563 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.563 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:04.563 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.563 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.563 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:85374 3.16 3:688cf0a4:::rbd_data.113cff0d0fb7.000000000000000a:12 [read 262144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:04.563 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:00.254720+0000", 2024-09-14T22:37:04.563 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.2269357110000003, 2024-09-14T22:37:04.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028235064000000001, 2024-09-14T22:37:04.563 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:04.563 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:04.563 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:04.563 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:04.564 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:04.564 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:04.564 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 85374 2024-09-14T22:37:04.564 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.564 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:04.564 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.564 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:04.564 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254720+0000", 2024-09-14T22:37:04.564 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.564 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.564 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.564 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:04.564 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254720+0000", 2024-09-14T22:37:04.564 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.564 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.565 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.565 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:04.565 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254720+0000", 2024-09-14T22:37:04.565 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4499999999999997e-07 2024-09-14T22:37:04.565 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.565 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.565 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:04.565 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254723+0000", 2024-09-14T22:37:04.565 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5919999999999999e-06 2024-09-14T22:37:04.565 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.565 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.565 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:04.565 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254729+0000", 2024-09-14T22:37:04.565 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5640000000000002e-06 2024-09-14T22:37:04.566 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.566 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.566 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:04.566 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254734+0000", 2024-09-14T22:37:04.566 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8049999999999997e-06 2024-09-14T22:37:04.566 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.566 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.566 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:04.566 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.282761+0000", 2024-09-14T22:37:04.566 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028026669000000001 2024-09-14T22:37:04.566 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.566 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.566 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:04.566 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.282840+0000", 2024-09-14T22:37:04.566 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9621000000000002e-05 2024-09-14T22:37:04.567 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.567 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.567 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:04.567 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.282955+0000", 2024-09-14T22:37:04.567 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000114368 2024-09-14T22:37:04.567 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.567 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:04.567 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.567 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.567 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.567 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:85375 3.16 3:688cf0a4:::rbd_data.113cff0d0fb7.000000000000000a:12 [read 356352~12288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:04.567 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:00.254740+0000", 2024-09-14T22:37:04.567 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.2269151750000002, 2024-09-14T22:37:04.567 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028433526000000001, 2024-09-14T22:37:04.567 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:04.568 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:04.568 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:04.568 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:04.568 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:04.568 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:04.568 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 85375 2024-09-14T22:37:04.568 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.568 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:04.568 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.568 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:04.568 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254740+0000", 2024-09-14T22:37:04.568 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.568 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.568 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.568 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:04.568 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254740+0000", 2024-09-14T22:37:04.569 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.569 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.569 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.569 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:04.569 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254741+0000", 2024-09-14T22:37:04.569 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8299999999999998e-07 2024-09-14T22:37:04.569 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.569 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.569 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:04.569 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254743+0000", 2024-09-14T22:37:04.569 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.154e-06 2024-09-14T22:37:04.569 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.569 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.569 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:04.569 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254748+0000", 2024-09-14T22:37:04.570 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7459999999999997e-06 2024-09-14T22:37:04.570 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.570 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.570 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:04.570 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254753+0000", 2024-09-14T22:37:04.570 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6600000000000003e-06 2024-09-14T22:37:04.570 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.570 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.570 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:04.570 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.282989+0000", 2024-09-14T22:37:04.570 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028235752999999999 2024-09-14T22:37:04.570 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.570 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.570 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:04.571 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.283067+0000", 2024-09-14T22:37:04.571 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8559000000000005e-05 2024-09-14T22:37:04.571 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.571 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.571 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:04.571 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.283174+0000", 2024-09-14T22:37:04.571 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106271 2024-09-14T22:37:04.571 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.571 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:04.571 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.571 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.571 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.571 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:85376 3.16 3:688cf0a4:::rbd_data.113cff0d0fb7.000000000000000a:12 [read 446464~12288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:04.571 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:00.254759+0000", 2024-09-14T22:37:04.571 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.2268960870000001, 2024-09-14T22:37:04.572 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028655568999999999, 2024-09-14T22:37:04.572 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:04.572 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:04.572 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:04.572 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:04.572 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:04.572 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:04.572 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 85376 2024-09-14T22:37:04.572 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.572 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:04.572 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.572 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:04.572 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254759+0000", 2024-09-14T22:37:04.573 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.573 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.573 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.573 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:04.573 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254759+0000", 2024-09-14T22:37:04.573 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.573 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.573 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.573 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:04.573 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254760+0000", 2024-09-14T22:37:04.573 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5600000000000001e-07 2024-09-14T22:37:04.573 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.573 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.573 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:04.574 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254762+0000", 2024-09-14T22:37:04.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.142e-06 2024-09-14T22:37:04.574 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.574 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.574 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:04.574 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254768+0000", 2024-09-14T22:37:04.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9819999999999996e-06 2024-09-14T22:37:04.574 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.574 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.574 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:04.574 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254773+0000", 2024-09-14T22:37:04.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0690000000000001e-06 2024-09-14T22:37:04.574 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.574 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.574 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:04.575 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.283207+0000", 2024-09-14T22:37:04.575 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028434207999999999 2024-09-14T22:37:04.575 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.575 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.575 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:04.575 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.283296+0000", 2024-09-14T22:37:04.575 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8636999999999999e-05 2024-09-14T22:37:04.575 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.575 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.575 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:04.575 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.283415+0000", 2024-09-14T22:37:04.575 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119175 2024-09-14T22:37:04.575 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.575 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:04.576 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.576 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.576 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.576 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:85461 3.16 3:68c13f99:::rbd_data.113cff0d0fb7.000000000000001b:12 [read 380928~12288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:04.576 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:00.261582+0000", 2024-09-14T22:37:04.576 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.2200732949999997, 2024-09-14T22:37:04.576 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029353918999999999, 2024-09-14T22:37:04.576 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:04.576 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:04.576 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:04.576 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:04.576 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:04.576 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:04.576 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 85461 2024-09-14T22:37:04.577 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.577 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:04.577 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.577 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:04.577 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.261582+0000", 2024-09-14T22:37:04.577 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.577 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.577 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.577 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:04.577 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.261582+0000", 2024-09-14T22:37:04.577 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:04.577 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.577 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.577 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:04.578 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.261582+0000", 2024-09-14T22:37:04.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:37:04.578 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.578 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:04.578 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.261585+0000", 2024-09-14T22:37:04.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.492e-06 2024-09-14T22:37:04.578 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.578 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:04.578 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.261591+0000", 2024-09-14T22:37:04.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5760000000000002e-06 2024-09-14T22:37:04.578 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.579 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:04.579 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.261598+0000", 2024-09-14T22:37:04.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4370000000000004e-06 2024-09-14T22:37:04.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.579 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:04.579 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.284316+0000", 2024-09-14T22:37:04.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022718341 2024-09-14T22:37:04.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.579 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:04.579 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.284384+0000", 2024-09-14T22:37:04.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7342999999999995e-05 2024-09-14T22:37:04.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:04.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:04.579 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:04.580 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.290936+0000", 2024-09-14T22:37:04.580 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0065523430000000004 2024-09-14T22:37:04.580 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.580 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:04.580 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.580 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:04.580 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:04.580 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:04.581 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:37:04.623 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:04.619+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:04.687 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:04.683+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:04.750 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:04.750 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:04.750 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:37:04.750 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:04.763 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:37:04.807 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:04.803+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:04.885 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:04.886 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:04.886 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:37:04.886 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:37:04.886 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:04.889 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:04.887+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:04.899 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:37:04.991 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:04.987+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:05.072 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:05.072 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:05.073 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:05.073 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:05.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.073 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:72252 3.2 3:442c6d72:::rbd_data.113cb4c83968.0000000000000015:head [zero 4456448~393216] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-14T22:37:05.073 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:37:05.073 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.367592381, 2024-09-14T22:37:05.073 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0176591500000001, 2024-09-14T22:37:05.073 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.073 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.073 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T22:37:05.073 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.073 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.074 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.074 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 72252 2024-09-14T22:37:05.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.074 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.074 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.074 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:37:05.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.074 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.074 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:37:05.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.075 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.075 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700101+0000", 2024-09-14T22:37:05.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4570000000000001e-06 2024-09-14T22:37:05.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.075 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.075 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700114+0000", 2024-09-14T22:37:05.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2799999999999999e-05 2024-09-14T22:37:05.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.075 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.076 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.076 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700146+0000", 2024-09-14T22:37:05.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2116000000000002e-05 2024-09-14T22:37:05.076 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.076 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.076 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.076 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700170+0000", 2024-09-14T22:37:05.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3770000000000001e-05 2024-09-14T22:37:05.076 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.076 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.076 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.076 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700285+0000", 2024-09-14T22:37:05.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000115228 2024-09-14T22:37:05.077 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.077 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op must wait for map", 2024-09-14T22:37:05.077 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700297+0000", 2024-09-14T22:37:05.077 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1681999999999999e-05 2024-09-14T22:37:05.077 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.077 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.077 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.703312+0000", 2024-09-14T22:37:05.077 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003015046 2024-09-14T22:37:05.077 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.077 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:37:05.077 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.703327+0000", 2024-09-14T22:37:05.078 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5481999999999998e-05 2024-09-14T22:37:05.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.078 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.078 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.078 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:51.717566+0000", 2024-09-14T22:37:05.078 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0142383509999999 2024-09-14T22:37:05.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.078 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.078 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.078 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:51.717757+0000", 2024-09-14T22:37:05.078 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000191218 2024-09-14T22:37:05.078 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.078 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.078 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.079 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.079 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.079 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:74156 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T22:37:05.079 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:37:05.079 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.5531236709999998, 2024-09-14T22:37:05.079 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062073521, 2024-09-14T22:37:05.079 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.079 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.079 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:05.079 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.079 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.079 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.079 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74156 2024-09-14T22:37:05.079 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.080 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.080 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.080 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.080 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:37:05.080 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.080 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.080 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.080 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.080 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:37:05.080 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.080 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.080 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.080 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.080 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514569+0000", 2024-09-14T22:37:05.081 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0229999999999999e-06 2024-09-14T22:37:05.081 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.081 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.081 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.081 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514596+0000", 2024-09-14T22:37:05.081 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6903e-05 2024-09-14T22:37:05.081 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.081 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.081 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.081 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514623+0000", 2024-09-14T22:37:05.081 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7206e-05 2024-09-14T22:37:05.081 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.082 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.082 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.082 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514637+0000", 2024-09-14T22:37:05.082 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:37:05.082 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.082 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.082 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.082 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514781+0000", 2024-09-14T22:37:05.082 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014394899999999999 2024-09-14T22:37:05.082 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.082 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.082 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.082 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.515005+0000", 2024-09-14T22:37:05.082 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022458499999999999 2024-09-14T22:37:05.083 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.083 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.083 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:37:05.083 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.527785+0000", 2024-09-14T22:37:05.083 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012780112 2024-09-14T22:37:05.083 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.083 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.083 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:05.083 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576209+0000", 2024-09-14T22:37:05.083 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.048423992999999999 2024-09-14T22:37:05.083 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.083 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.083 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:05.083 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576346+0000", 2024-09-14T22:37:05.084 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000136909 2024-09-14T22:37:05.084 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.084 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.084 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:05.084 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576517+0000", 2024-09-14T22:37:05.084 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000170848 2024-09-14T22:37:05.084 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.084 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.084 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.084 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576640+0000", 2024-09-14T22:37:05.084 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000122953 2024-09-14T22:37:05.084 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.084 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.085 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.085 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.085 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.085 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82513 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 557056~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:05.085 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:37:05.085 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6484918520000003, 2024-09-14T22:37:05.085 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059345486000000003, 2024-09-14T22:37:05.085 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.085 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.085 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.085 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.085 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.085 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.085 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82513 2024-09-14T22:37:05.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.086 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.086 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:37:05.086 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.086 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:37:05.086 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.086 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419200+0000", 2024-09-14T22:37:05.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.184e-06 2024-09-14T22:37:05.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419207+0000", 2024-09-14T22:37:05.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2690000000000002e-06 2024-09-14T22:37:05.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419224+0000", 2024-09-14T22:37:05.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6623e-05 2024-09-14T22:37:05.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.088 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.088 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419236+0000", 2024-09-14T22:37:05.088 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2367e-05 2024-09-14T22:37:05.088 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.088 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.088 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.088 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478294+0000", 2024-09-14T22:37:05.088 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059057927000000003 2024-09-14T22:37:05.088 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.088 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.088 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.088 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478402+0000", 2024-09-14T22:37:05.088 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107921 2024-09-14T22:37:05.088 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.089 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.089 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.089 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478544+0000", 2024-09-14T22:37:05.089 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014219499999999999 2024-09-14T22:37:05.089 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.089 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.089 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.089 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.089 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.089 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82514 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 704512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:05.089 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:37:05.089 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6484327490000004, 2024-09-14T22:37:05.089 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059580793, 2024-09-14T22:37:05.089 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.090 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.090 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.090 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.090 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.090 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.090 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82514 2024-09-14T22:37:05.090 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.090 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.090 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.090 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.090 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:37:05.090 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.090 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.090 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.091 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:37:05.091 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.091 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.091 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419259+0000", 2024-09-14T22:37:05.091 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.353e-06 2024-09-14T22:37:05.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.091 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.091 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419276+0000", 2024-09-14T22:37:05.091 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7567e-05 2024-09-14T22:37:05.092 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.092 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.092 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419294+0000", 2024-09-14T22:37:05.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.713e-05 2024-09-14T22:37:05.092 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.092 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.092 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419306+0000", 2024-09-14T22:37:05.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.279e-05 2024-09-14T22:37:05.092 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.092 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.092 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478589+0000", 2024-09-14T22:37:05.093 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059282978 2024-09-14T22:37:05.093 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.093 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.093 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.093 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478694+0000", 2024-09-14T22:37:05.093 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000104561 2024-09-14T22:37:05.093 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.093 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.093 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.093 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478838+0000", 2024-09-14T22:37:05.093 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014441399999999999 2024-09-14T22:37:05.093 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.093 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.094 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.094 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.094 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.094 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82515 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:05.094 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:37:05.094 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6483675990000002, 2024-09-14T22:37:05.094 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059815123999999997, 2024-09-14T22:37:05.094 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.094 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.094 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.094 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.094 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.094 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.094 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82515 2024-09-14T22:37:05.095 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.095 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.095 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.095 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.095 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:37:05.095 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.095 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.095 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.095 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.095 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:37:05.095 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.095 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.095 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.095 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.095 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419324+0000", 2024-09-14T22:37:05.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2130000000000001e-06 2024-09-14T22:37:05.096 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.096 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.096 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419331+0000", 2024-09-14T22:37:05.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4730000000000003e-06 2024-09-14T22:37:05.096 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.096 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.096 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419348+0000", 2024-09-14T22:37:05.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6269999999999998e-05 2024-09-14T22:37:05.096 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.096 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419360+0000", 2024-09-14T22:37:05.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2243e-05 2024-09-14T22:37:05.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.097 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478884+0000", 2024-09-14T22:37:05.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059524369000000001 2024-09-14T22:37:05.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.097 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478990+0000", 2024-09-14T22:37:05.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010569 2024-09-14T22:37:05.098 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.098 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.098 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.098 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479138+0000", 2024-09-14T22:37:05.098 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000147866 2024-09-14T22:37:05.098 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.098 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.098 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.098 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.098 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.098 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82516 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 999424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:05.098 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:37:05.099 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.648315094, 2024-09-14T22:37:05.099 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061979595999999998, 2024-09-14T22:37:05.099 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.099 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.099 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.099 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.099 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.099 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.099 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82516 2024-09-14T22:37:05.099 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.099 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.099 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.099 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.099 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:37:05.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.100 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.100 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.100 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:37:05.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.100 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.100 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.100 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419376+0000", 2024-09-14T22:37:05.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.237e-06 2024-09-14T22:37:05.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.100 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.101 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419383+0000", 2024-09-14T22:37:05.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8170000000000001e-06 2024-09-14T22:37:05.101 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.101 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419400+0000", 2024-09-14T22:37:05.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6636000000000001e-05 2024-09-14T22:37:05.101 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.101 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419412+0000", 2024-09-14T22:37:05.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2357e-05 2024-09-14T22:37:05.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481080+0000", 2024-09-14T22:37:05.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061668032999999997 2024-09-14T22:37:05.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481193+0000", 2024-09-14T22:37:05.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112335 2024-09-14T22:37:05.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481355+0000", 2024-09-14T22:37:05.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016218100000000001 2024-09-14T22:37:05.103 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.103 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.103 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.103 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.103 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.103 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82538 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1097728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:05.103 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:37:05.103 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6482630240000002, 2024-09-14T22:37:05.103 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059971705, 2024-09-14T22:37:05.103 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.103 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.103 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.103 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.103 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.104 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.104 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82538 2024-09-14T22:37:05.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.104 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.104 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.104 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.104 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:37:05.104 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.104 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.104 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.104 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:37:05.104 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.104 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419429+0000", 2024-09-14T22:37:05.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.497e-06 2024-09-14T22:37:05.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419436+0000", 2024-09-14T22:37:05.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2099999999999996e-06 2024-09-14T22:37:05.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419453+0000", 2024-09-14T22:37:05.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6840000000000001e-05 2024-09-14T22:37:05.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419465+0000", 2024-09-14T22:37:05.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2283000000000001e-05 2024-09-14T22:37:05.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479182+0000", 2024-09-14T22:37:05.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059717339000000001 2024-09-14T22:37:05.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479297+0000", 2024-09-14T22:37:05.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000114259 2024-09-14T22:37:05.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.107 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479399+0000", 2024-09-14T22:37:05.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102277 2024-09-14T22:37:05.107 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.107 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.107 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.107 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82539 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:05.107 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:37:05.107 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6482101739999999, 2024-09-14T22:37:05.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060166935999999997, 2024-09-14T22:37:05.108 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.108 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.108 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.108 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.108 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.108 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.108 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82539 2024-09-14T22:37:05.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.108 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.108 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:37:05.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.109 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:37:05.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.109 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419482+0000", 2024-09-14T22:37:05.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4130000000000001e-06 2024-09-14T22:37:05.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.109 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419488+0000", 2024-09-14T22:37:05.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8460000000000002e-06 2024-09-14T22:37:05.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.110 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419505+0000", 2024-09-14T22:37:05.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6477000000000002e-05 2024-09-14T22:37:05.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.110 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419518+0000", 2024-09-14T22:37:05.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2797e-05 2024-09-14T22:37:05.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479438+0000", 2024-09-14T22:37:05.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059919897 2024-09-14T22:37:05.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479547+0000", 2024-09-14T22:37:05.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109807 2024-09-14T22:37:05.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479647+0000", 2024-09-14T22:37:05.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9698999999999995e-05 2024-09-14T22:37:05.111 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.112 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.112 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.112 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82542 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:05.112 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:37:05.112 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6481577309999995, 2024-09-14T22:37:05.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060367430999999999, 2024-09-14T22:37:05.112 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.112 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.112 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.112 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.112 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.113 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.113 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82542 2024-09-14T22:37:05.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.113 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:37:05.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:37:05.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419534+0000", 2024-09-14T22:37:05.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T22:37:05.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419542+0000", 2024-09-14T22:37:05.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8199999999999997e-06 2024-09-14T22:37:05.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419559+0000", 2024-09-14T22:37:05.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6852999999999999e-05 2024-09-14T22:37:05.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419571+0000", 2024-09-14T22:37:05.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2364000000000001e-05 2024-09-14T22:37:05.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479686+0000", 2024-09-14T22:37:05.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060115522999999997 2024-09-14T22:37:05.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479795+0000", 2024-09-14T22:37:05.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010831399999999999 2024-09-14T22:37:05.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479900+0000", 2024-09-14T22:37:05.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105284 2024-09-14T22:37:05.116 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.116 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.116 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.117 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82544 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:05.117 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:37:05.117 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6481039949999996, 2024-09-14T22:37:05.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060590515999999997, 2024-09-14T22:37:05.117 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.117 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.117 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.117 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.117 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.117 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.117 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82544 2024-09-14T22:37:05.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.118 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:37:05.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:37:05.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419588+0000", 2024-09-14T22:37:05.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.35e-06 2024-09-14T22:37:05.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419595+0000", 2024-09-14T22:37:05.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4030000000000001e-06 2024-09-14T22:37:05.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419612+0000", 2024-09-14T22:37:05.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6670000000000001e-05 2024-09-14T22:37:05.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419624+0000", 2024-09-14T22:37:05.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-05 2024-09-14T22:37:05.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479957+0000", 2024-09-14T22:37:05.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060333287999999999 2024-09-14T22:37:05.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480074+0000", 2024-09-14T22:37:05.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116443 2024-09-14T22:37:05.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480177+0000", 2024-09-14T22:37:05.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000103102 2024-09-14T22:37:05.121 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.121 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.121 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.121 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82546 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:05.121 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:37:05.121 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6480510759999998, 2024-09-14T22:37:05.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060795428999999998, 2024-09-14T22:37:05.121 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.121 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.121 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.121 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.121 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.122 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.122 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82546 2024-09-14T22:37:05.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.122 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:37:05.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:37:05.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419640+0000", 2024-09-14T22:37:05.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.254e-06 2024-09-14T22:37:05.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419647+0000", 2024-09-14T22:37:05.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9e-06 2024-09-14T22:37:05.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419664+0000", 2024-09-14T22:37:05.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6345999999999999e-05 2024-09-14T22:37:05.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419676+0000", 2024-09-14T22:37:05.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.224e-05 2024-09-14T22:37:05.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480218+0000", 2024-09-14T22:37:05.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060542133999999997 2024-09-14T22:37:05.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480336+0000", 2024-09-14T22:37:05.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117472 2024-09-14T22:37:05.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480435+0000", 2024-09-14T22:37:05.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9083e-05 2024-09-14T22:37:05.125 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.125 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.125 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.125 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82548 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:05.126 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:37:05.126 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6479990659999997, 2024-09-14T22:37:05.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061011636000000001, 2024-09-14T22:37:05.126 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.126 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.126 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.126 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.126 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.126 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.126 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82548 2024-09-14T22:37:05.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.126 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.126 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.126 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:37:05.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:37:05.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419693+0000", 2024-09-14T22:37:05.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.387e-06 2024-09-14T22:37:05.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419699+0000", 2024-09-14T22:37:05.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8730000000000001e-06 2024-09-14T22:37:05.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419716+0000", 2024-09-14T22:37:05.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6416e-05 2024-09-14T22:37:05.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419728+0000", 2024-09-14T22:37:05.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2597e-05 2024-09-14T22:37:05.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480476+0000", 2024-09-14T22:37:05.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060747989000000002 2024-09-14T22:37:05.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480583+0000", 2024-09-14T22:37:05.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010638500000000001 2024-09-14T22:37:05.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480703+0000", 2024-09-14T22:37:05.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011998900000000001 2024-09-14T22:37:05.129 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.129 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.129 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.130 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82550 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1982464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:05.130 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:37:05.130 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6479463130000003, 2024-09-14T22:37:05.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061295483999999997, 2024-09-14T22:37:05.130 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.130 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.130 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.130 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.130 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.130 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.130 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82550 2024-09-14T22:37:05.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.130 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:37:05.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:37:05.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419745+0000", 2024-09-14T22:37:05.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.406e-06 2024-09-14T22:37:05.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419753+0000", 2024-09-14T22:37:05.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8140000000000001e-06 2024-09-14T22:37:05.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419770+0000", 2024-09-14T22:37:05.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6719999999999999e-05 2024-09-14T22:37:05.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419782+0000", 2024-09-14T22:37:05.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2252999999999999e-05 2024-09-14T22:37:05.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480745+0000", 2024-09-14T22:37:05.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060962385000000001 2024-09-14T22:37:05.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480900+0000", 2024-09-14T22:37:05.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015513899999999999 2024-09-14T22:37:05.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481039+0000", 2024-09-14T22:37:05.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013976700000000001 2024-09-14T22:37:05.134 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.134 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.134 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.134 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82588 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1064960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:05.134 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:37:05.134 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6478924230000001, 2024-09-14T22:37:05.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061863732999999997, 2024-09-14T22:37:05.134 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.134 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.135 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.135 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.135 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.135 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.135 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82588 2024-09-14T22:37:05.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.135 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.135 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.135 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.135 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:37:05.135 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.135 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.135 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:37:05.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419799+0000", 2024-09-14T22:37:05.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4160000000000001e-06 2024-09-14T22:37:05.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419807+0000", 2024-09-14T22:37:05.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2169999999999994e-06 2024-09-14T22:37:05.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419824+0000", 2024-09-14T22:37:05.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6776e-05 2024-09-14T22:37:05.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419837+0000", 2024-09-14T22:37:05.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2680000000000001e-05 2024-09-14T22:37:05.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481395+0000", 2024-09-14T22:37:05.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061558416999999997 2024-09-14T22:37:05.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481532+0000", 2024-09-14T22:37:05.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000136861 2024-09-14T22:37:05.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481662+0000", 2024-09-14T22:37:05.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012936600000000001 2024-09-14T22:37:05.138 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.138 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.138 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.139 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82589 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:05.139 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:37:05.139 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6478378280000001, 2024-09-14T22:37:05.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062064592000000002, 2024-09-14T22:37:05.139 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.139 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.139 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.139 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.139 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.139 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.139 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82589 2024-09-14T22:37:05.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.139 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.139 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.140 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:37:05.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.140 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.140 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:37:05.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.140 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.140 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419854+0000", 2024-09-14T22:37:05.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.367e-06 2024-09-14T22:37:05.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419862+0000", 2024-09-14T22:37:05.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8930000000000005e-06 2024-09-14T22:37:05.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419896+0000", 2024-09-14T22:37:05.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4134000000000002e-05 2024-09-14T22:37:05.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419909+0000", 2024-09-14T22:37:05.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3332999999999999e-05 2024-09-14T22:37:05.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481702+0000", 2024-09-14T22:37:05.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061792947000000001 2024-09-14T22:37:05.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481808+0000", 2024-09-14T22:37:05.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105655 2024-09-14T22:37:05.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481917+0000", 2024-09-14T22:37:05.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109263 2024-09-14T22:37:05.143 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.143 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.143 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.143 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82590 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1359872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:05.143 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:37:05.143 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6477642210000001, 2024-09-14T22:37:05.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062362895000000002, 2024-09-14T22:37:05.143 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.143 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.143 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.143 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.144 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.144 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.144 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82590 2024-09-14T22:37:05.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.144 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.144 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:37:05.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.144 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:37:05.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419928+0000", 2024-09-14T22:37:05.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4470000000000001e-06 2024-09-14T22:37:05.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419942+0000", 2024-09-14T22:37:05.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4229e-05 2024-09-14T22:37:05.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419972+0000", 2024-09-14T22:37:05.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.065e-05 2024-09-14T22:37:05.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419985+0000", 2024-09-14T22:37:05.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2903e-05 2024-09-14T22:37:05.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481960+0000", 2024-09-14T22:37:05.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061974435000000001 2024-09-14T22:37:05.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482106+0000", 2024-09-14T22:37:05.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014643800000000001 2024-09-14T22:37:05.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482289+0000", 2024-09-14T22:37:05.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000182793 2024-09-14T22:37:05.147 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.147 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.147 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.148 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82592 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:05.148 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:37:05.148 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6476879990000004, 2024-09-14T22:37:05.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062631507000000003, 2024-09-14T22:37:05.148 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.148 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.148 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.148 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.148 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.148 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.148 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82592 2024-09-14T22:37:05.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.148 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.149 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:37:05.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.149 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:37:05.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.149 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420017+0000", 2024-09-14T22:37:05.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4744e-05 2024-09-14T22:37:05.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420036+0000", 2024-09-14T22:37:05.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8680000000000001e-05 2024-09-14T22:37:05.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420054+0000", 2024-09-14T22:37:05.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7799999999999999e-05 2024-09-14T22:37:05.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420067+0000", 2024-09-14T22:37:05.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3013000000000001e-05 2024-09-14T22:37:05.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482358+0000", 2024-09-14T22:37:05.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062291445000000001 2024-09-14T22:37:05.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482469+0000", 2024-09-14T22:37:05.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110608 2024-09-14T22:37:05.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.152 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482634+0000", 2024-09-14T22:37:05.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000165217 2024-09-14T22:37:05.152 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.152 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.152 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.152 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82594 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1654784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:05.152 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:37:05.152 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6475900230000002, 2024-09-14T22:37:05.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062893849000000002, 2024-09-14T22:37:05.153 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.153 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.153 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.153 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.153 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.153 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.153 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82594 2024-09-14T22:37:05.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.153 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:37:05.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:37:05.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420102+0000", 2024-09-14T22:37:05.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5e-06 2024-09-14T22:37:05.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420115+0000", 2024-09-14T22:37:05.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.324e-05 2024-09-14T22:37:05.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420133+0000", 2024-09-14T22:37:05.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8213e-05 2024-09-14T22:37:05.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420147+0000", 2024-09-14T22:37:05.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3284e-05 2024-09-14T22:37:05.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482688+0000", 2024-09-14T22:37:05.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062541425999999997 2024-09-14T22:37:05.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482794+0000", 2024-09-14T22:37:05.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010575500000000001 2024-09-14T22:37:05.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482994+0000", 2024-09-14T22:37:05.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020043099999999999 2024-09-14T22:37:05.156 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.156 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.156 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.157 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82596 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:05.157 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:37:05.157 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6475267230000004, 2024-09-14T22:37:05.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063187238000000007, 2024-09-14T22:37:05.157 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.157 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.157 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.157 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.157 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.157 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.157 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82596 2024-09-14T22:37:05.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.157 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:37:05.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:37:05.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420165+0000", 2024-09-14T22:37:05.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4100000000000001e-06 2024-09-14T22:37:05.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420173+0000", 2024-09-14T22:37:05.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8129999999999996e-06 2024-09-14T22:37:05.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420190+0000", 2024-09-14T22:37:05.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7076999999999999e-05 2024-09-14T22:37:05.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420203+0000", 2024-09-14T22:37:05.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2683e-05 2024-09-14T22:37:05.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483060+0000", 2024-09-14T22:37:05.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062857114000000006 2024-09-14T22:37:05.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483167+0000", 2024-09-14T22:37:05.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010713999999999999 2024-09-14T22:37:05.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483351+0000", 2024-09-14T22:37:05.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018400099999999999 2024-09-14T22:37:05.161 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.161 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.161 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.161 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82597 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1949696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:05.161 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:37:05.161 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6474698969999997, 2024-09-14T22:37:05.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063466513000000002, 2024-09-14T22:37:05.161 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.161 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.162 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.162 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.162 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.162 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.162 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82597 2024-09-14T22:37:05.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.162 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:37:05.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:37:05.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420221+0000", 2024-09-14T22:37:05.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.043e-06 2024-09-14T22:37:05.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420229+0000", 2024-09-14T22:37:05.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0469999999999994e-06 2024-09-14T22:37:05.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420246+0000", 2024-09-14T22:37:05.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6339999999999999e-05 2024-09-14T22:37:05.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420258+0000", 2024-09-14T22:37:05.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2296000000000001e-05 2024-09-14T22:37:05.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483402+0000", 2024-09-14T22:37:05.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063144318000000005 2024-09-14T22:37:05.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483530+0000", 2024-09-14T22:37:05.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012713199999999999 2024-09-14T22:37:05.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483687+0000", 2024-09-14T22:37:05.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015733699999999999 2024-09-14T22:37:05.165 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.165 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.165 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.165 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.166 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.166 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:05.166 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T22:37:05.167 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:05.163+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:05.168 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:37:05.219 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone17 from 0x27c66bc (overlap 0xed0e6) to 0x76873 2024-09-14T22:37:05.244 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [2] live_osds: [0, 1, 3, 4, 5, 6, 7] 2024-09-14T22:37:05.244 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:37:05.244 INFO:tasks.thrashosds.thrasher:Reviving osd 2 2024-09-14T22:37:05.244 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-14T22:37:05.244 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-14T22:37:05.246 INFO:tasks.ceph.osd.2:Started 2024-09-14T22:37:05.247 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:37:05.267 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:05.263+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:05.306 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-14T22:37:05.343 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:05.343 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:05.343 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:37:05.343 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:05.357 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:37:05.370 INFO:teuthology.orchestra.run.smithi001.stderr:no valid command found; 10 closest matches: 2024-09-14T22:37:05.370 INFO:teuthology.orchestra.run.smithi001.stderr:0 2024-09-14T22:37:05.371 INFO:teuthology.orchestra.run.smithi001.stderr:1 2024-09-14T22:37:05.371 INFO:teuthology.orchestra.run.smithi001.stderr:2 2024-09-14T22:37:05.371 INFO:teuthology.orchestra.run.smithi001.stderr:abort 2024-09-14T22:37:05.371 INFO:teuthology.orchestra.run.smithi001.stderr:assert 2024-09-14T22:37:05.371 INFO:teuthology.orchestra.run.smithi001.stderr:config diff 2024-09-14T22:37:05.371 INFO:teuthology.orchestra.run.smithi001.stderr:config diff get 2024-09-14T22:37:05.371 INFO:teuthology.orchestra.run.smithi001.stderr:config get 2024-09-14T22:37:05.371 INFO:teuthology.orchestra.run.smithi001.stderr:config help [] 2024-09-14T22:37:05.371 INFO:teuthology.orchestra.run.smithi001.stderr:config set ... 2024-09-14T22:37:05.371 INFO:teuthology.orchestra.run.smithi001.stderr:admin_socket: invalid command 2024-09-14T22:37:05.373 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T22:37:05.373 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-14T22:37:05.399 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:05.395+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:05.468 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:05.463+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:05.517 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:05.517 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:05.518 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:37:05.518 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:37:05.518 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:05.531 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:37:05.569 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:05.567+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:05.670 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:05.667+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:05.700 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:05.700 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:05.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:05.700 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:05.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.700 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:37:05.701 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:37:05.701 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 65.689564524000005, 2024-09-14T22:37:05.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.398029152, 2024-09-14T22:37:05.701 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.701 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.701 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:05.701 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.701 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:37:05.701 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:37:05.701 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 1 2024-09-14T22:37:05.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.701 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:37:05.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:37:05.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006183+0000", 2024-09-14T22:37:05.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9059999999999998e-06 2024-09-14T22:37:05.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006198+0000", 2024-09-14T22:37:05.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.523e-05 2024-09-14T22:37:05.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006219+0000", 2024-09-14T22:37:05.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1342999999999999e-05 2024-09-14T22:37:05.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006240+0000", 2024-09-14T22:37:05.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1186999999999999e-05 2024-09-14T22:37:05.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006363+0000", 2024-09-14T22:37:05.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012258200000000001 2024-09-14T22:37:05.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:37:05.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006392+0000", 2024-09-14T22:37:05.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9607e-05 2024-09-14T22:37:05.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401084+0000", 2024-09-14T22:37:05.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.39469142699999998 2024-09-14T22:37:05.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401394+0000", 2024-09-14T22:37:05.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030987799999999998 2024-09-14T22:37:05.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:37:05.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401860+0000", 2024-09-14T22:37:05.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00046654299999999998 2024-09-14T22:37:05.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:05.706 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.403386+0000", 2024-09-14T22:37:05.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001525322 2024-09-14T22:37:05.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:05.706 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404042+0000", 2024-09-14T22:37:05.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065676099999999996 2024-09-14T22:37:05.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:05.706 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404136+0000", 2024-09-14T22:37:05.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4032999999999999e-05 2024-09-14T22:37:05.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404209+0000", 2024-09-14T22:37:05.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2333e-05 2024-09-14T22:37:05.707 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.707 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.707 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.707 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2079 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11401216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:05.707 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450429+0000", 2024-09-14T22:37:05.707 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.245314735999997, 2024-09-14T22:37:05.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055279107000000001, 2024-09-14T22:37:05.708 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.708 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.708 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.708 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.708 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.708 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.708 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2079 2024-09-14T22:37:05.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.708 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450429+0000", 2024-09-14T22:37:05.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450429+0000", 2024-09-14T22:37:05.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450431+0000", 2024-09-14T22:37:05.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1039999999999999e-06 2024-09-14T22:37:05.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450435+0000", 2024-09-14T22:37:05.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9300000000000002e-06 2024-09-14T22:37:05.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450455+0000", 2024-09-14T22:37:05.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9242999999999999e-05 2024-09-14T22:37:05.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450470+0000", 2024-09-14T22:37:05.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5523e-05 2024-09-14T22:37:05.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.505413+0000", 2024-09-14T22:37:05.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054943193000000001 2024-09-14T22:37:05.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.505538+0000", 2024-09-14T22:37:05.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012467199999999999 2024-09-14T22:37:05.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.505709+0000", 2024-09-14T22:37:05.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017044200000000001 2024-09-14T22:37:05.712 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.712 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.712 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.712 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2080 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11409408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:05.712 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450489+0000", 2024-09-14T22:37:05.712 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.245254989000003, 2024-09-14T22:37:05.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055572178, 2024-09-14T22:37:05.712 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.712 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.712 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.712 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.712 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.713 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.713 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2080 2024-09-14T22:37:05.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.713 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450489+0000", 2024-09-14T22:37:05.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450489+0000", 2024-09-14T22:37:05.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450490+0000", 2024-09-14T22:37:05.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.023e-06 2024-09-14T22:37:05.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450495+0000", 2024-09-14T22:37:05.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1399999999999999e-06 2024-09-14T22:37:05.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450512+0000", 2024-09-14T22:37:05.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6892999999999999e-05 2024-09-14T22:37:05.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450525+0000", 2024-09-14T22:37:05.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2877000000000001e-05 2024-09-14T22:37:05.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.505787+0000", 2024-09-14T22:37:05.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055261432999999999 2024-09-14T22:37:05.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.716 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.505899+0000", 2024-09-14T22:37:05.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112456 2024-09-14T22:37:05.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.716 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.716 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506061+0000", 2024-09-14T22:37:05.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000162356 2024-09-14T22:37:05.716 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.716 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.716 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.716 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2081 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11417600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:05.717 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450544+0000", 2024-09-14T22:37:05.717 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.245199669999998, 2024-09-14T22:37:05.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.05598185, 2024-09-14T22:37:05.717 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.717 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.717 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.717 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.717 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.717 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.717 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2081 2024-09-14T22:37:05.717 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.717 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.717 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.717 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450544+0000", 2024-09-14T22:37:05.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450544+0000", 2024-09-14T22:37:05.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450546+0000", 2024-09-14T22:37:05.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5370000000000001e-06 2024-09-14T22:37:05.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450553+0000", 2024-09-14T22:37:05.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8630000000000002e-06 2024-09-14T22:37:05.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450571+0000", 2024-09-14T22:37:05.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7657000000000001e-05 2024-09-14T22:37:05.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450585+0000", 2024-09-14T22:37:05.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4056000000000001e-05 2024-09-14T22:37:05.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506185+0000", 2024-09-14T22:37:05.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055599920999999997 2024-09-14T22:37:05.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506334+0000", 2024-09-14T22:37:05.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014919900000000001 2024-09-14T22:37:05.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.721 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.721 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506526+0000", 2024-09-14T22:37:05.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019261699999999999 2024-09-14T22:37:05.721 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.721 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.721 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.721 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2082 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11425792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:05.721 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450602+0000", 2024-09-14T22:37:05.721 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.245141943, 2024-09-14T22:37:05.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056238654999999999, 2024-09-14T22:37:05.721 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.721 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.722 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.722 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.722 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.722 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.722 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2082 2024-09-14T22:37:05.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.722 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.722 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.722 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450602+0000", 2024-09-14T22:37:05.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450602+0000", 2024-09-14T22:37:05.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450603+0000", 2024-09-14T22:37:05.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1060000000000001e-06 2024-09-14T22:37:05.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450608+0000", 2024-09-14T22:37:05.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1499999999999998e-06 2024-09-14T22:37:05.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450626+0000", 2024-09-14T22:37:05.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7484000000000001e-05 2024-09-14T22:37:05.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450639+0000", 2024-09-14T22:37:05.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3485999999999999e-05 2024-09-14T22:37:05.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506599+0000", 2024-09-14T22:37:05.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055959395000000002 2024-09-14T22:37:05.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506706+0000", 2024-09-14T22:37:05.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010703499999999999 2024-09-14T22:37:05.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506841+0000", 2024-09-14T22:37:05.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013499900000000001 2024-09-14T22:37:05.725 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.725 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.725 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.726 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2083 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11433984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:05.726 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450658+0000", 2024-09-14T22:37:05.726 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.245086516999997, 2024-09-14T22:37:05.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056459267, 2024-09-14T22:37:05.726 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.726 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.726 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.726 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.726 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.726 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.726 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2083 2024-09-14T22:37:05.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.727 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450658+0000", 2024-09-14T22:37:05.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450658+0000", 2024-09-14T22:37:05.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450659+0000", 2024-09-14T22:37:05.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T22:37:05.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.728 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450664+0000", 2024-09-14T22:37:05.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4700000000000001e-06 2024-09-14T22:37:05.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.728 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450684+0000", 2024-09-14T22:37:05.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.916e-05 2024-09-14T22:37:05.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450698+0000", 2024-09-14T22:37:05.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4100000000000001e-05 2024-09-14T22:37:05.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506881+0000", 2024-09-14T22:37:05.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056183250999999997 2024-09-14T22:37:05.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506982+0000", 2024-09-14T22:37:05.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010090900000000001 2024-09-14T22:37:05.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507117+0000", 2024-09-14T22:37:05.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013510400000000001 2024-09-14T22:37:05.730 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.730 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.730 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.730 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2084 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11442176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:05.730 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450716+0000", 2024-09-14T22:37:05.730 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.245028658000003, 2024-09-14T22:37:05.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056802328999999999, 2024-09-14T22:37:05.730 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.731 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.731 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.731 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.731 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.731 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.731 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2084 2024-09-14T22:37:05.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.731 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450716+0000", 2024-09-14T22:37:05.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450716+0000", 2024-09-14T22:37:05.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450717+0000", 2024-09-14T22:37:05.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.08e-06 2024-09-14T22:37:05.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450722+0000", 2024-09-14T22:37:05.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5099999999999998e-06 2024-09-14T22:37:05.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450740+0000", 2024-09-14T22:37:05.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7836999999999999e-05 2024-09-14T22:37:05.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450757+0000", 2024-09-14T22:37:05.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7513e-05 2024-09-14T22:37:05.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507224+0000", 2024-09-14T22:37:05.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056466954 2024-09-14T22:37:05.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507355+0000", 2024-09-14T22:37:05.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000130207 2024-09-14T22:37:05.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507518+0000", 2024-09-14T22:37:05.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016322799999999999 2024-09-14T22:37:05.734 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.734 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.734 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.735 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2085 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11450368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:05.735 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450774+0000", 2024-09-14T22:37:05.735 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.244969855000001, 2024-09-14T22:37:05.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057099952000000002, 2024-09-14T22:37:05.735 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.735 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.735 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.735 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.735 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.735 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.735 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2085 2024-09-14T22:37:05.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.735 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450774+0000", 2024-09-14T22:37:05.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450774+0000", 2024-09-14T22:37:05.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450775+0000", 2024-09-14T22:37:05.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1030000000000001e-06 2024-09-14T22:37:05.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450781+0000", 2024-09-14T22:37:05.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.22e-06 2024-09-14T22:37:05.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450798+0000", 2024-09-14T22:37:05.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7350000000000002e-05 2024-09-14T22:37:05.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450812+0000", 2024-09-14T22:37:05.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.358e-05 2024-09-14T22:37:05.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507576+0000", 2024-09-14T22:37:05.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056764468999999998 2024-09-14T22:37:05.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507714+0000", 2024-09-14T22:37:05.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013826800000000001 2024-09-14T22:37:05.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507874+0000", 2024-09-14T22:37:05.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015996200000000001 2024-09-14T22:37:05.739 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.739 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.739 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.739 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2086 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11458560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:05.739 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450830+0000", 2024-09-14T22:37:05.739 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.244914565000002, 2024-09-14T22:37:05.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057480495999999999, 2024-09-14T22:37:05.740 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.740 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.740 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.740 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.740 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.740 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.740 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2086 2024-09-14T22:37:05.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.740 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450830+0000", 2024-09-14T22:37:05.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450830+0000", 2024-09-14T22:37:05.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450831+0000", 2024-09-14T22:37:05.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0860000000000001e-06 2024-09-14T22:37:05.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450836+0000", 2024-09-14T22:37:05.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4999999999999999e-06 2024-09-14T22:37:05.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450853+0000", 2024-09-14T22:37:05.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7184e-05 2024-09-14T22:37:05.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450869+0000", 2024-09-14T22:37:05.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5965999999999999e-05 2024-09-14T22:37:05.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507972+0000", 2024-09-14T22:37:05.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057103095999999999 2024-09-14T22:37:05.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.743 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508119+0000", 2024-09-14T22:37:05.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014651200000000001 2024-09-14T22:37:05.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.743 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508310+0000", 2024-09-14T22:37:05.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019115199999999999 2024-09-14T22:37:05.743 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.743 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.743 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.744 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2087 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11466752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:05.744 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450888+0000", 2024-09-14T22:37:05.744 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.244856519000002, 2024-09-14T22:37:05.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057792806000000002, 2024-09-14T22:37:05.744 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.744 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.744 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.744 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.744 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.744 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.744 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2087 2024-09-14T22:37:05.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.745 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450888+0000", 2024-09-14T22:37:05.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450888+0000", 2024-09-14T22:37:05.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450889+0000", 2024-09-14T22:37:05.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4270000000000001e-06 2024-09-14T22:37:05.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450895+0000", 2024-09-14T22:37:05.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4700000000000001e-06 2024-09-14T22:37:05.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450913+0000", 2024-09-14T22:37:05.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7958999999999999e-05 2024-09-14T22:37:05.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450926+0000", 2024-09-14T22:37:05.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3842999999999999e-05 2024-09-14T22:37:05.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508394+0000", 2024-09-14T22:37:05.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057467795000000002 2024-09-14T22:37:05.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508516+0000", 2024-09-14T22:37:05.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000121822 2024-09-14T22:37:05.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508680+0000", 2024-09-14T22:37:05.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016448999999999999 2024-09-14T22:37:05.748 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.748 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.748 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.748 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2088 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11474944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:05.748 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450943+0000", 2024-09-14T22:37:05.748 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.244800906000002, 2024-09-14T22:37:05.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058110368000000003, 2024-09-14T22:37:05.749 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.749 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.749 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.749 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.749 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.749 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.749 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2088 2024-09-14T22:37:05.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.749 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450943+0000", 2024-09-14T22:37:05.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450943+0000", 2024-09-14T22:37:05.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450944+0000", 2024-09-14T22:37:05.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2100000000000001e-06 2024-09-14T22:37:05.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450950+0000", 2024-09-14T22:37:05.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3329999999999996e-06 2024-09-14T22:37:05.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450968+0000", 2024-09-14T22:37:05.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7802999999999999e-05 2024-09-14T22:37:05.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450981+0000", 2024-09-14T22:37:05.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.349e-05 2024-09-14T22:37:05.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508752+0000", 2024-09-14T22:37:05.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057770957999999997 2024-09-14T22:37:05.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508884+0000", 2024-09-14T22:37:05.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013186900000000001 2024-09-14T22:37:05.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509054+0000", 2024-09-14T22:37:05.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000169705 2024-09-14T22:37:05.752 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.752 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.752 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.753 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2089 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11483136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:05.753 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450998+0000", 2024-09-14T22:37:05.753 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.244746012999997, 2024-09-14T22:37:05.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058422566000000002, 2024-09-14T22:37:05.753 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.753 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.753 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.753 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.753 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.753 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.753 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2089 2024-09-14T22:37:05.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.754 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450998+0000", 2024-09-14T22:37:05.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450998+0000", 2024-09-14T22:37:05.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450999+0000", 2024-09-14T22:37:05.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0100000000000001e-06 2024-09-14T22:37:05.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451014+0000", 2024-09-14T22:37:05.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4486e-05 2024-09-14T22:37:05.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451032+0000", 2024-09-14T22:37:05.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7893e-05 2024-09-14T22:37:05.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451046+0000", 2024-09-14T22:37:05.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4783999999999999e-05 2024-09-14T22:37:05.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509131+0000", 2024-09-14T22:37:05.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058085070000000003 2024-09-14T22:37:05.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509253+0000", 2024-09-14T22:37:05.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012173299999999999 2024-09-14T22:37:05.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509421+0000", 2024-09-14T22:37:05.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016759000000000001 2024-09-14T22:37:05.757 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.757 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.757 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.757 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2090 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11491328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:05.757 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.451064+0000", 2024-09-14T22:37:05.757 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.244680236999997, 2024-09-14T22:37:05.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058723076999999999, 2024-09-14T22:37:05.758 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.758 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.758 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.758 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.758 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.758 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.758 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2090 2024-09-14T22:37:05.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.758 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.758 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451064+0000", 2024-09-14T22:37:05.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451064+0000", 2024-09-14T22:37:05.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451065+0000", 2024-09-14T22:37:05.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2160000000000001e-06 2024-09-14T22:37:05.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451070+0000", 2024-09-14T22:37:05.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1340000000000003e-06 2024-09-14T22:37:05.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451088+0000", 2024-09-14T22:37:05.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7476e-05 2024-09-14T22:37:05.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451101+0000", 2024-09-14T22:37:05.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3723999999999999e-05 2024-09-14T22:37:05.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509492+0000", 2024-09-14T22:37:05.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058390899000000003 2024-09-14T22:37:05.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509618+0000", 2024-09-14T22:37:05.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012593899999999999 2024-09-14T22:37:05.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509787+0000", 2024-09-14T22:37:05.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000168689 2024-09-14T22:37:05.761 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.761 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.762 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.762 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2091 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11499520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:05.762 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.451119+0000", 2024-09-14T22:37:05.762 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.244625184, 2024-09-14T22:37:05.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059045274000000002, 2024-09-14T22:37:05.762 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.762 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.762 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.762 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.762 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.762 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.762 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2091 2024-09-14T22:37:05.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.763 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451119+0000", 2024-09-14T22:37:05.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451119+0000", 2024-09-14T22:37:05.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.764 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451120+0000", 2024-09-14T22:37:05.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0669999999999999e-06 2024-09-14T22:37:05.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.764 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.764 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451125+0000", 2024-09-14T22:37:05.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3329999999999996e-06 2024-09-14T22:37:05.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.764 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.764 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451143+0000", 2024-09-14T22:37:05.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7309999999999999e-05 2024-09-14T22:37:05.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451157+0000", 2024-09-14T22:37:05.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3835999999999999e-05 2024-09-14T22:37:05.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509863+0000", 2024-09-14T22:37:05.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058706345 2024-09-14T22:37:05.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509985+0000", 2024-09-14T22:37:05.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000121736 2024-09-14T22:37:05.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510164+0000", 2024-09-14T22:37:05.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017964700000000001 2024-09-14T22:37:05.766 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.766 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.766 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.766 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2092 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11507712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:05.766 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.451202+0000", 2024-09-14T22:37:05.766 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.244542512000002, 2024-09-14T22:37:05.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059345031999999999, 2024-09-14T22:37:05.767 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.767 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.767 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.767 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.767 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.767 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.767 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2092 2024-09-14T22:37:05.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.767 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451202+0000", 2024-09-14T22:37:05.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451202+0000", 2024-09-14T22:37:05.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451203+0000", 2024-09-14T22:37:05.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6199999999999999e-06 2024-09-14T22:37:05.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451209+0000", 2024-09-14T22:37:05.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1999999999999999e-06 2024-09-14T22:37:05.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451230+0000", 2024-09-14T22:37:05.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0247000000000001e-05 2024-09-14T22:37:05.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451245+0000", 2024-09-14T22:37:05.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4993e-05 2024-09-14T22:37:05.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510239+0000", 2024-09-14T22:37:05.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058994321000000002 2024-09-14T22:37:05.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510371+0000", 2024-09-14T22:37:05.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013195899999999999 2024-09-14T22:37:05.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510547+0000", 2024-09-14T22:37:05.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000175692 2024-09-14T22:37:05.770 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.770 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.771 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.771 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2093 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11515904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:05.771 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.451264+0000", 2024-09-14T22:37:05.771 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.244480475000003, 2024-09-14T22:37:05.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059636293999999999, 2024-09-14T22:37:05.771 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.771 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.771 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.771 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.771 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.772 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.772 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2093 2024-09-14T22:37:05.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.772 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451264+0000", 2024-09-14T22:37:05.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451264+0000", 2024-09-14T22:37:05.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451265+0000", 2024-09-14T22:37:05.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3400000000000001e-06 2024-09-14T22:37:05.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451270+0000", 2024-09-14T22:37:05.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4360000000000001e-06 2024-09-14T22:37:05.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451289+0000", 2024-09-14T22:37:05.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8383e-05 2024-09-14T22:37:05.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451303+0000", 2024-09-14T22:37:05.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.455e-05 2024-09-14T22:37:05.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510627+0000", 2024-09-14T22:37:05.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059323280999999999 2024-09-14T22:37:05.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510745+0000", 2024-09-14T22:37:05.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011867900000000001 2024-09-14T22:37:05.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510900+0000", 2024-09-14T22:37:05.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000154625 2024-09-14T22:37:05.775 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.775 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.775 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.776 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2094 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11524096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:05.776 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.451321+0000", 2024-09-14T22:37:05.776 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.244423488999999, 2024-09-14T22:37:05.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059938827, 2024-09-14T22:37:05.776 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.776 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.776 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.776 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.776 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.776 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.776 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2094 2024-09-14T22:37:05.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.777 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451321+0000", 2024-09-14T22:37:05.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451321+0000", 2024-09-14T22:37:05.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451322+0000", 2024-09-14T22:37:05.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0529999999999999e-06 2024-09-14T22:37:05.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451328+0000", 2024-09-14T22:37:05.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8300000000000001e-06 2024-09-14T22:37:05.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451344+0000", 2024-09-14T22:37:05.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6750000000000001e-05 2024-09-14T22:37:05.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451358+0000", 2024-09-14T22:37:05.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3200000000000001e-05 2024-09-14T22:37:05.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510968+0000", 2024-09-14T22:37:05.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059610567000000003 2024-09-14T22:37:05.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.511094+0000", 2024-09-14T22:37:05.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012629199999999999 2024-09-14T22:37:05.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.511260+0000", 2024-09-14T22:37:05.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016513500000000001 2024-09-14T22:37:05.780 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.780 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.780 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.780 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2095 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11532288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:05.780 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.451375+0000", 2024-09-14T22:37:05.780 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.244369315999997, 2024-09-14T22:37:05.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060228067000000003, 2024-09-14T22:37:05.781 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.781 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.781 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:05.781 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.781 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.781 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.781 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2095 2024-09-14T22:37:05.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.781 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451375+0000", 2024-09-14T22:37:05.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451375+0000", 2024-09-14T22:37:05.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451376+0000", 2024-09-14T22:37:05.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0470000000000001e-06 2024-09-14T22:37:05.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451381+0000", 2024-09-14T22:37:05.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.536e-06 2024-09-14T22:37:05.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451399+0000", 2024-09-14T22:37:05.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7200000000000001e-05 2024-09-14T22:37:05.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451412+0000", 2024-09-14T22:37:05.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3726999999999999e-05 2024-09-14T22:37:05.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.511308+0000", 2024-09-14T22:37:05.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059895176000000001 2024-09-14T22:37:05.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.511444+0000", 2024-09-14T22:37:05.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013621600000000001 2024-09-14T22:37:05.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.511603+0000", 2024-09-14T22:37:05.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015916500000000001 2024-09-14T22:37:05.784 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.784 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.784 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.785 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:45459 3.11 3:8ce265cb:::rbd_data.113cc4e35b3.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:37:05.785 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:36.190430+0000", 2024-09-14T22:37:05.785 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.505314572, 2024-09-14T22:37:05.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.06125162, 2024-09-14T22:37:05.785 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.785 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.785 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:05.785 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.785 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.785 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.785 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 45459 2024-09-14T22:37:05.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.785 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190430+0000", 2024-09-14T22:37:05.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190430+0000", 2024-09-14T22:37:05.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190432+0000", 2024-09-14T22:37:05.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3039999999999999e-06 2024-09-14T22:37:05.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190443+0000", 2024-09-14T22:37:05.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1178999999999999e-05 2024-09-14T22:37:05.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190474+0000", 2024-09-14T22:37:05.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0803000000000003e-05 2024-09-14T22:37:05.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190489+0000", 2024-09-14T22:37:05.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.517e-05 2024-09-14T22:37:05.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190623+0000", 2024-09-14T22:37:05.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013405599999999999 2024-09-14T22:37:05.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190752+0000", 2024-09-14T22:37:05.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012870899999999999 2024-09-14T22:37:05.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:37:05.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.203210+0000", 2024-09-14T22:37:05.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012458484000000001 2024-09-14T22:37:05.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:05.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.251517+0000", 2024-09-14T22:37:05.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.048306453999999999 2024-09-14T22:37:05.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:05.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.251571+0000", 2024-09-14T22:37:05.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4228999999999999e-05 2024-09-14T22:37:05.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:05.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.251628+0000", 2024-09-14T22:37:05.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6922000000000002e-05 2024-09-14T22:37:05.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.251681+0000", 2024-09-14T22:37:05.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3310000000000003e-05 2024-09-14T22:37:05.790 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.790 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.790 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.791 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:45462 3.11 3:8ce265cb:::rbd_data.113cc4e35b3.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:37:05.791 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:36.258125+0000", 2024-09-14T22:37:05.791 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.437618994000001, 2024-09-14T22:37:05.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059222599000000001, 2024-09-14T22:37:05.791 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:05.791 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:05.791 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:05.791 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:05.791 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:05.791 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:05.791 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 45462 2024-09-14T22:37:05.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.791 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:05.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:05.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258125+0000", 2024-09-14T22:37:05.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:05.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258125+0000", 2024-09-14T22:37:05.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:05.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:05.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258127+0000", 2024-09-14T22:37:05.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3159999999999999e-06 2024-09-14T22:37:05.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:05.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258139+0000", 2024-09-14T22:37:05.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1667e-05 2024-09-14T22:37:05.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:05.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258170+0000", 2024-09-14T22:37:05.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0929999999999997e-05 2024-09-14T22:37:05.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:05.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258185+0000", 2024-09-14T22:37:05.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4423000000000001e-05 2024-09-14T22:37:05.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:05.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258291+0000", 2024-09-14T22:37:05.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106285 2024-09-14T22:37:05.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:05.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258446+0000", 2024-09-14T22:37:05.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015486499999999999 2024-09-14T22:37:05.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:37:05.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.269615+0000", 2024-09-14T22:37:05.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011169095 2024-09-14T22:37:05.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:05.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.317113+0000", 2024-09-14T22:37:05.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.047498382999999998 2024-09-14T22:37:05.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:05.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.317144+0000", 2024-09-14T22:37:05.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0970000000000003e-05 2024-09-14T22:37:05.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:05.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.317233+0000", 2024-09-14T22:37:05.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9205999999999999e-05 2024-09-14T22:37:05.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:05.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:05.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:05.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.317348+0000", 2024-09-14T22:37:05.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000114459 2024-09-14T22:37:05.796 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.796 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.797 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.797 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:05.797 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:05.797 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:05.799 INFO:teuthology.orchestra.run.smithi001.stdout:2583 writesame 0x502c03 thru 0x511ea7 data_size 0x1(0xf2a5 bytes) 2024-09-14T22:37:05.799 INFO:teuthology.orchestra.run.smithi001.stdout:2586 read 0x783efc thru 0x78fd80 (0xbe85 bytes) 2024-09-14T22:37:05.799 INFO:teuthology.orchestra.run.smithi001.stdout:2587 read 0x56db76 thru 0x56f92f (0x1dba bytes) 2024-09-14T22:37:05.799 INFO:teuthology.orchestra.run.smithi001.stdout:2588 read 0xd1969 thru 0xda537 (0x8bcf bytes) 2024-09-14T22:37:05.799 INFO:teuthology.orchestra.run.smithi001.stdout:2589 read 0x278ef6b thru 0x2793521 (0x45b7 bytes) 2024-09-14T22:37:05.799 INFO:teuthology.orchestra.run.smithi001.stdout:2590 writesame 0x281773a thru 0x281b6ec data_size 0x1(0x3fb3 bytes) 2024-09-14T22:37:05.799 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:05.795+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:05.800 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:37:05.800 INFO:teuthology.orchestra.run.smithi001.stdout:2591 punch from 0x238107a to 0x238d142, (0xc0c8 bytes) 2024-09-14T22:37:05.800 INFO:teuthology.orchestra.run.smithi001.stdout:2592 punch from 0x461df9 to 0x46205a, (0x261 bytes) 2024-09-14T22:37:05.821 INFO:teuthology.orchestra.run.smithi001.stdout:2593 write 0x7b7a66b thru 0x7b85c6e (0xb604 bytes) 2024-09-14T22:37:05.831 INFO:teuthology.orchestra.run.smithi001.stdout:2595 read 0x4053e35 thru 0x4054f6a (0x1136 bytes) 2024-09-14T22:37:05.831 INFO:teuthology.orchestra.run.smithi001.stdout:2596 writesame 0x14c8c8a thru 0x14cf933 data_size 0x7(0x6caa bytes) 2024-09-14T22:37:05.844 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:05.844 INFO:teuthology.orchestra.run.smithi001.stdout:2598 read 0x2a6ec0f thru 0x2a78c23 (0xa015 bytes) 2024-09-14T22:37:05.844 INFO:teuthology.orchestra.run.smithi001.stdout:2599 punch from 0x889ffb to 0x893fe0, (0x9fe5 bytes) 2024-09-14T22:37:05.844 INFO:teuthology.orchestra.run.smithi001.stdout:2600 read 0x181b64e thru 0x1820887 (0x523a bytes) 2024-09-14T22:37:05.845 INFO:teuthology.orchestra.run.smithi001.stdout:2601 read 0x4c49cb8 thru 0x4c504a8 (0x67f1 bytes) 2024-09-14T22:37:05.846 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:05.846 INFO:teuthology.orchestra.run.smithi001.stdout:2603 write 0x292b10e thru 0x292f9fd (0x48f0 bytes) 2024-09-14T22:37:05.853 INFO:teuthology.orchestra.run.smithi001.stdout:2604 read 0x539850a thru 0x53a14c8 (0x8fbf bytes) 2024-09-14T22:37:05.854 INFO:teuthology.orchestra.run.smithi001.stdout:2606 punch from 0x6d7e1ca to 0x6d86465, (0x829b bytes) 2024-09-14T22:37:05.854 INFO:teuthology.orchestra.run.smithi001.stdout:2607 read 0x10e5a80 thru 0x10e8e9d (0x341e bytes) 2024-09-14T22:37:05.855 INFO:teuthology.orchestra.run.smithi001.stdout:2608 write 0x2b6b613 thru 0x2b7b192 (0xfb80 bytes) 2024-09-14T22:37:05.865 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:05.865 INFO:teuthology.orchestra.run.smithi001.stdout:2611 write 0x7315e thru 0x7a986 (0x7829 bytes) 2024-09-14T22:37:05.880 INFO:teuthology.orchestra.run.smithi001.stdout:2613 punch from 0x10353a6 to 0x1038230, (0x2e8a bytes) 2024-09-14T22:37:05.899 INFO:teuthology.orchestra.run.smithi001.stdout:2614 write 0xaa0b926 thru 0xaa12b27 (0x7202 bytes) 2024-09-14T22:37:05.899 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:05.895+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:05.909 INFO:teuthology.orchestra.run.smithi001.stdout:2615 clone 19 order 18 su 1024 sc 9 2024-09-14T22:37:05.938 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:05.938 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:37:05.938 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:37:05.938 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:05.952 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:37:06.023 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:37:06.041 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:06.039+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:06.075 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2024-09-14T22:37:06.101 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:06.099+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:06.133 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:06.133 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:37:06.133 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:37:06.133 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:37:06.133 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:06.151 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:37:06.195 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:37:06.196 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:37:06.202 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:06.195+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:06.212 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:37:06.212 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:37:06.212 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-14T22:37:06.213 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone18 from 0xaa12b28 (overlap 0x27c66bc) to 0x1fd1efc 2024-09-14T22:37:06.232 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:37:06.232 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:37:06.257 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:37:06.257 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:37:06.278 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:37:06.278 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:37:06.298 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:37:06.299 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:37:06.302 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:06.299+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:06.319 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:06.319 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:06.320 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:37:06.321 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:37:06.325 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:37:06.325 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:37:06.335 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:37:06.388 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-14T22:37:06.395 INFO:teuthology.orchestra.run.smithi001.stdout:2616 trunc from 0xaa12b28 to 0xbb5f090 2024-09-14T22:37:06.403 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:06.399+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:06.409 INFO:teuthology.orchestra.run.smithi001.stdout:2617 write 0x7d27950 thru 0x7d2ad8f (0x3440 bytes) 2024-09-14T22:37:06.505 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:06.503+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:06.531 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:06.531 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:06.549 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:37:06.605 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:06.599+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:06.674 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:06.671+0000 7f73aa34c7c0 -1 Falling back to public interface 2024-09-14T22:37:06.706 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:06.703+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:06.714 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:06.714 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:06.728 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:37:06.800 INFO:teuthology.orchestra.run.smithi001.stdout:2618 trunc from 0xbb5f090 to 0x46a726 2024-09-14T22:37:06.806 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:06.803+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:06.908 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:06.903+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:06.936 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:06.936 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:06.953 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:37:07.039 INFO:teuthology.orchestra.run.smithi001.stdout:2621 punch from 0x3d09cc to 0x3da79f, (0x9dd3 bytes) 2024-09-14T22:37:07.040 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:07.035+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:07.057 INFO:teuthology.orchestra.run.smithi001.stdout:2623 write 0x4080340 thru 0x4087060 (0x6d21 bytes) 2024-09-14T22:37:07.095 INFO:teuthology.orchestra.run.smithi001.stdout:2624 trunc from 0x4087061 to 0xe59a648 2024-09-14T22:37:07.105 INFO:teuthology.orchestra.run.smithi001.stdout:2625 writesame 0x54a359c thru 0x54a8bc6 data_size 0x39(0x562b bytes) 2024-09-14T22:37:07.110 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:07.103+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:07.117 INFO:teuthology.orchestra.run.smithi001.stdout:2626 read 0x8a2d81b thru 0x8a365cd (0x8db3 bytes) 2024-09-14T22:37:07.118 INFO:teuthology.orchestra.run.smithi001.stdout:2627 punch from 0xd8720c0 to 0xd881251, (0xf191 bytes) 2024-09-14T22:37:07.118 INFO:teuthology.orchestra.run.smithi001.stdout:2628 write 0x22f4445 thru 0x22fce57 (0x8a13 bytes) 2024-09-14T22:37:07.129 INFO:teuthology.orchestra.run.smithi001.stdout:2629 read 0x6fc7e06 thru 0x6fd319d (0xb398 bytes) 2024-09-14T22:37:07.129 INFO:teuthology.orchestra.run.smithi001.stdout:2630 punch from 0x34a1e50 to 0x34b0f3b, (0xf0eb bytes) 2024-09-14T22:37:07.132 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:07.132 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:07.139 INFO:teuthology.orchestra.run.smithi001.stdout:2631 trunc from 0xe59a648 to 0x82f30c1 2024-09-14T22:37:07.149 INFO:teuthology.orchestra.run.smithi001.stdout:2632 read 0x1d82481 thru 0x1d84a93 (0x2613 bytes) 2024-09-14T22:37:07.149 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:07.152 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:37:07.160 INFO:teuthology.orchestra.run.smithi001.stdout:2634 write 0x98ce5cd thru 0x98d04ea (0x1f1e bytes) 2024-09-14T22:37:07.174 INFO:teuthology.orchestra.run.smithi001.stdout:2635 read 0x93e975c thru 0x93f41fe (0xaaa3 bytes) 2024-09-14T22:37:07.189 INFO:teuthology.orchestra.run.smithi001.stdout:2636 trunc from 0x98d04eb to 0x58b904 2024-09-14T22:37:07.211 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:07.203+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:07.244 INFO:teuthology.orchestra.run.smithi001.stdout:2637 write 0xc40383e thru 0xc410da5 (0xd568 bytes) 2024-09-14T22:37:07.258 INFO:teuthology.orchestra.run.smithi001.stdout:2639 write 0x167de1b thru 0x1686c4f (0x8e35 bytes) 2024-09-14T22:37:07.283 INFO:teuthology.orchestra.run.smithi001.stdout:2640 read 0x3ab54dc thru 0x3ac069c (0xb1c1 bytes) 2024-09-14T22:37:07.283 INFO:teuthology.orchestra.run.smithi001.stdout:2641 punch from 0x9d9ae33 to 0x9da07cb, (0x5998 bytes) 2024-09-14T22:37:07.283 INFO:teuthology.orchestra.run.smithi001.stdout:2642 read 0x5004dfd thru 0x500e47f (0x9683 bytes) 2024-09-14T22:37:07.284 INFO:teuthology.orchestra.run.smithi001.stdout:2643 writesame 0xacca73d thru 0xacd86bf data_size 0x1(0xdf83 bytes) 2024-09-14T22:37:07.297 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:07.298 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:07.311 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:07.307+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:07.312 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:37:07.320 INFO:teuthology.orchestra.run.smithi001.stdout:2644 read 0x1a15e1b thru 0x1a1e43d (0x8623 bytes) 2024-09-14T22:37:07.320 INFO:teuthology.orchestra.run.smithi001.stdout:2645 punch from 0xab69226 to 0xab6c0ce, (0x2ea8 bytes) 2024-09-14T22:37:07.320 INFO:teuthology.orchestra.run.smithi001.stdout:2646 write 0x2a8fe89 thru 0x2a9992d (0x9aa5 bytes) 2024-09-14T22:37:07.333 INFO:teuthology.orchestra.run.smithi001.stdout:2647 writesame 0x737eaa0 thru 0x7388bbd data_size 0x29(0xa11e bytes) 2024-09-14T22:37:07.347 INFO:teuthology.orchestra.run.smithi001.stdout:2648 write 0x759885e thru 0x759a78c (0x1f2f bytes) 2024-09-14T22:37:07.361 INFO:teuthology.orchestra.run.smithi001.stdout:2649 punch from 0xa01b598 to 0xa0299e8, (0xe450 bytes) 2024-09-14T22:37:07.361 INFO:teuthology.orchestra.run.smithi001.stdout:2650 read 0xb2945ce thru 0xb2a3a1a (0xf44d bytes) 2024-09-14T22:37:07.361 INFO:teuthology.orchestra.run.smithi001.stdout:2653 write 0x5f0560b thru 0x5f0ae67 (0x585d bytes) 2024-09-14T22:37:07.374 INFO:teuthology.orchestra.run.smithi001.stdout:2655 read 0x2abca10 thru 0x2ac9716 (0xcd07 bytes) 2024-09-14T22:37:07.375 INFO:teuthology.orchestra.run.smithi001.stdout:2656 read 0xbdaac75 thru 0xbdb708e (0xc41a bytes) 2024-09-14T22:37:07.377 INFO:teuthology.orchestra.run.smithi001.stdout:2657 trunc from 0xc410da6 to 0xd23c733 2024-09-14T22:37:07.387 INFO:teuthology.orchestra.run.smithi001.stdout:2658 writesame 0x90842d1 thru 0x908c206 data_size 0x13(0x7f36 bytes) 2024-09-14T22:37:07.399 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:07.399 INFO:teuthology.orchestra.run.smithi001.stdout:2660 write 0x6120024 thru 0x61226f9 (0x26d6 bytes) 2024-09-14T22:37:07.410 INFO:teuthology.orchestra.run.smithi001.stdout:2661 punch from 0x256cae8 to 0x25738e6, (0x6dfe bytes) 2024-09-14T22:37:07.412 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:07.407+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:07.422 INFO:teuthology.orchestra.run.smithi001.stdout:2662 write 0xec9e695 thru 0xec9fed4 (0x1840 bytes) 2024-09-14T22:37:07.435 INFO:teuthology.orchestra.run.smithi001.stdout:2663 read 0xbef368b thru 0xbef7d35 (0x46ab bytes) 2024-09-14T22:37:07.435 INFO:teuthology.orchestra.run.smithi001.stdout:2664 write 0x22153c8 thru 0x221dc55 (0x888e bytes) 2024-09-14T22:37:07.457 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:07.457 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:07.470 INFO:teuthology.orchestra.run.smithi001.stdout:2665 trunc from 0xec9fed5 to 0xe32bba 2024-09-14T22:37:07.479 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:37:07.521 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:07.519+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:07.526 INFO:teuthology.orchestra.run.smithi001.stdout:2668 trunc from 0xe32bba to 0x1746aa5 2024-09-14T22:37:07.534 INFO:teuthology.orchestra.run.smithi001.stdout:2670 read 0xb8e4a3 thru 0xb97dd6 (0x9934 bytes) 2024-09-14T22:37:07.551 INFO:teuthology.orchestra.run.smithi001.stdout:2671 write 0x81ecd1b thru 0x81f7587 (0xa86d bytes) 2024-09-14T22:37:07.564 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:07.564 INFO:teuthology.orchestra.run.smithi001.stdout:2674 read 0x2d2a0c7 thru 0x2d2e173 (0x40ad bytes) 2024-09-14T22:37:07.564 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:07.564 INFO:teuthology.orchestra.run.smithi001.stdout:2677 writesame 0x3b9b399 thru 0x3ba2bc6 data_size 0x1(0x782e bytes) 2024-09-14T22:37:07.581 INFO:teuthology.orchestra.run.smithi001.stdout:2679 write 0x314523b thru 0x314b6b1 (0x6477 bytes) 2024-09-14T22:37:07.594 INFO:teuthology.orchestra.run.smithi001.stdout:2680 read 0x7cc7b30 thru 0x7cd32aa (0xb77b bytes) 2024-09-14T22:37:07.594 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:07.595 INFO:teuthology.orchestra.run.smithi001.stdout:2682 writesame 0x147907c thru 0x1487f37 data_size 0x42(0xeebc bytes) 2024-09-14T22:37:07.608 INFO:teuthology.orchestra.run.smithi001.stdout:2683 punch from 0x99627a to 0x9a2452, (0xc1d8 bytes) 2024-09-14T22:37:07.608 INFO:teuthology.orchestra.run.smithi001.stdout:2684 write 0xee46d5 thru 0xeee1b8 (0x9ae4 bytes) 2024-09-14T22:37:07.614 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:07.607+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:07.622 INFO:teuthology.orchestra.run.smithi001.stdout:2685 read 0x476ead0 thru 0x4779e09 (0xb33a bytes) 2024-09-14T22:37:07.624 INFO:teuthology.orchestra.run.smithi001.stdout:2686 trunc from 0x81f7588 to 0x6bc5a91 2024-09-14T22:37:07.632 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:07.632 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:07.643 INFO:teuthology.orchestra.run.smithi001.stdout:2687 writesame 0x623ac95 thru 0x6245407 data_size 0x21(0xa773 bytes) 2024-09-14T22:37:07.647 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:37:07.656 INFO:teuthology.orchestra.run.smithi001.stdout:2688 punch from 0x17594f1 to 0x175fbb5, (0x66c4 bytes) 2024-09-14T22:37:07.656 INFO:teuthology.orchestra.run.smithi001.stdout:2689 write 0x560b359 thru 0x5610a1a (0x56c2 bytes) 2024-09-14T22:37:07.672 INFO:teuthology.orchestra.run.smithi001.stdout:2690 writesame 0x227c3a9 thru 0x2288043 data_size 0x1(0xbc9b bytes) 2024-09-14T22:37:07.709 INFO:teuthology.orchestra.run.smithi001.stdout:2691 punch from 0x254f880 to 0x255b809, (0xbf89 bytes) 2024-09-14T22:37:07.717 INFO:teuthology.orchestra.run.smithi001.stdout:2692 trunc from 0x6bc5a91 to 0x2a31d9d 2024-09-14T22:37:07.735 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:07.731+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:07.750 INFO:teuthology.orchestra.run.smithi001.stdout:2694 write 0x355a278 thru 0x3560b70 (0x68f9 bytes) 2024-09-14T22:37:07.786 INFO:teuthology.orchestra.run.smithi001.stdout:2695 trunc from 0x3560b71 to 0xd6773bb 2024-09-14T22:37:07.799 INFO:teuthology.orchestra.run.smithi001.stdout:2696 write 0x42a995b thru 0x42b610c (0xc7b2 bytes) 2024-09-14T22:37:07.816 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:07.811+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:07.819 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:07.819 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:07.824 INFO:teuthology.orchestra.run.smithi001.stdout:2697 read 0xd2b3bda thru 0xd2be265 (0xa68c bytes) 2024-09-14T22:37:07.824 INFO:teuthology.orchestra.run.smithi001.stdout:2698 write 0x189b2f5 thru 0x18a00c7 (0x4dd3 bytes) 2024-09-14T22:37:07.835 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:37:07.841 INFO:teuthology.orchestra.run.smithi001.stdout:2699 punch from 0x560d9a6 to 0x56156a1, (0x7cfb bytes) 2024-09-14T22:37:07.841 INFO:teuthology.orchestra.run.smithi001.stdout:2700 read 0xc1dc1e8 thru 0xc1e18e1 (0x56fa bytes) 2024-09-14T22:37:07.851 INFO:teuthology.orchestra.run.smithi001.stdout:2701 trunc from 0xd6773bb to 0x912fb0c 2024-09-14T22:37:07.863 INFO:teuthology.orchestra.run.smithi001.stdout:2703 write 0x78f3392 thru 0x7902a6d (0xf6dc bytes) 2024-09-14T22:37:07.900 INFO:teuthology.orchestra.run.smithi001.stdout:2704 trunc from 0x912fb0c to 0x38917ce 2024-09-14T22:37:07.921 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:07.921 INFO:teuthology.orchestra.run.smithi001.stdout:2707 write 0x2c4294c thru 0x2c491fc (0x68b1 bytes) 2024-09-14T22:37:07.923 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:07.919+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:07.940 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:07.954 INFO:teuthology.orchestra.run.smithi001.stdout:2710 write 0x459daa2 thru 0x45a35f1 (0x5b50 bytes) 2024-09-14T22:37:07.975 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:07.975 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:07.989 INFO:teuthology.orchestra.run.smithi001.stdout:2715 write 0x4bcf16b thru 0x4bd436e (0x5204 bytes) 2024-09-14T22:37:08.013 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:08.013 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:08.018 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:08.015+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:08.028 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:37:08.039 INFO:teuthology.orchestra.run.smithi001.stdout:2718 writesame 0x9422200 thru 0x942a21c data_size 0x1(0x801d bytes) 2024-09-14T22:37:08.054 INFO:teuthology.orchestra.run.smithi001.stdout:2719 writesame 0xb50b8fe thru 0xb511b12 data_size 0x11(0x6215 bytes) 2024-09-14T22:37:08.054 INFO:teuthology.orchestra.run.smithi001.stdout:2720 read 0x26b0574 thru 0x26b93e5 (0x8e72 bytes) 2024-09-14T22:37:08.055 INFO:teuthology.orchestra.run.smithi001.stdout:2723 write 0x30ffa3c thru 0x3105361 (0x5926 bytes) 2024-09-14T22:37:08.078 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:08.078 INFO:teuthology.orchestra.run.smithi001.stdout:2728 read 0x7b0b867 thru 0x7b142e0 (0x8a7a bytes) 2024-09-14T22:37:08.078 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:08.078 INFO:teuthology.orchestra.run.smithi001.stdout:2730 punch from 0x8f3fef6 to 0x8f4c270, (0xc37a bytes) 2024-09-14T22:37:08.078 INFO:teuthology.orchestra.run.smithi001.stdout:2733 read 0xa44e84e thru 0xa450120 (0x18d3 bytes) 2024-09-14T22:37:08.078 INFO:teuthology.orchestra.run.smithi001.stdout:2734 punch from 0x7c78153 to 0x7c7be0a, (0x3cb7 bytes) 2024-09-14T22:37:08.078 INFO:teuthology.orchestra.run.smithi001.stdout:2735 punch from 0x6d006cf to 0x6d05469, (0x4d9a bytes) 2024-09-14T22:37:08.078 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:08.079 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:08.085 INFO:teuthology.orchestra.run.smithi001.stdout:2739 trunc from 0xb511b13 to 0xe17f730 2024-09-14T22:37:08.096 INFO:teuthology.orchestra.run.smithi001.stdout:2740 read 0x5702f35 thru 0x5709aeb (0x6bb7 bytes) 2024-09-14T22:37:08.096 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:08.096 INFO:teuthology.orchestra.run.smithi001.stdout:2743 read 0x41fd1ff thru 0x42044b5 (0x72b7 bytes) 2024-09-14T22:37:08.096 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:08.097 INFO:teuthology.orchestra.run.smithi001.stdout:2745 writesame 0x4b6065a thru 0x4b6fc95 data_size 0x67(0xf63c bytes) 2024-09-14T22:37:08.102 INFO:teuthology.orchestra.run.smithi001.stdout:2746 read 0xc0181ca thru 0xc01e7fa (0x6631 bytes) 2024-09-14T22:37:08.102 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:08.102 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:08.106 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T22:37:08.119 INFO:teuthology.orchestra.run.smithi001.stdout:2749 trunc from 0xe17f730 to 0x7eb4a9f 2024-09-14T22:37:08.119 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:08.115+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:08.127 INFO:teuthology.orchestra.run.smithi001.stdout:2750 write 0x1863b93 thru 0x186983e (0x5cac bytes) 2024-09-14T22:37:08.135 INFO:teuthology.orchestra.run.smithi001.stdout:2751 trunc from 0x7eb4a9f to 0x5adee29 2024-09-14T22:37:08.144 INFO:teuthology.orchestra.run.smithi001.stdout:2752 read 0x130e929 thru 0x13152d6 (0x69ae bytes) 2024-09-14T22:37:08.147 INFO:teuthology.orchestra.run.smithi001.stdout:2755 trunc from 0x5adee29 to 0x72e07b0 2024-09-14T22:37:08.164 INFO:teuthology.orchestra.run.smithi001.stdout:2756 trunc from 0x72e07b0 to 0x2b2083b 2024-09-14T22:37:08.197 INFO:teuthology.orchestra.run.smithi001.stdout:2757 punch from 0x27e038a to 0x27e6a5d, (0x66d3 bytes) 2024-09-14T22:37:08.208 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:08.208 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:08.209 INFO:teuthology.orchestra.run.smithi001.stdout:2758 write 0x42931be thru 0x42999f7 (0x683a bytes) 2024-09-14T22:37:08.220 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:08.215+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:08.224 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:37:08.227 INFO:teuthology.orchestra.run.smithi001.stdout:2759 write 0x459f9ef thru 0x45a52d9 (0x58eb bytes) 2024-09-14T22:37:08.240 INFO:teuthology.orchestra.run.smithi001.stdout:2760 punch from 0x291f38a to 0x2928be3, (0x9859 bytes) 2024-09-14T22:37:08.254 INFO:teuthology.orchestra.run.smithi001.stdout:2761 write 0x85d774b thru 0x85e4ec9 (0xd77f bytes) 2024-09-14T22:37:08.266 INFO:teuthology.orchestra.run.smithi001.stdout:2762 writesame 0x8533f53 thru 0x85428a4 data_size 0x6e(0xe952 bytes) 2024-09-14T22:37:08.288 INFO:teuthology.orchestra.run.smithi001.stdout:2764 writesame 0xdbfe850 thru 0xdc0abf9 data_size 0x5(0xc3aa bytes) 2024-09-14T22:37:08.288 INFO:teuthology.orchestra.run.smithi001.stdout:2765 write 0x1e45aa5 thru 0x1e4ce76 (0x73d2 bytes) 2024-09-14T22:37:08.300 INFO:teuthology.orchestra.run.smithi001.stdout:2767 write 0x85433e4 thru 0x854bd79 (0x8996 bytes) 2024-09-14T22:37:08.304 INFO:teuthology.orchestra.run.smithi001.stdout:2768 read 0x4434012 thru 0x4438a66 (0x4a55 bytes) 2024-09-14T22:37:08.305 INFO:teuthology.orchestra.run.smithi001.stdout:2769 write 0x8f80f5d thru 0x8f8ad80 (0x9e24 bytes) 2024-09-14T22:37:08.315 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:08.321 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:08.315+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:08.326 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:37:08.327 INFO:teuthology.orchestra.run.smithi001.stdout:2771 trunc from 0xdc0abfa to 0x5e6d5ee 2024-09-14T22:37:08.342 INFO:teuthology.orchestra.run.smithi001.stdout:2772 punch from 0x38d1d97 to 0x38d60c9, (0x4332 bytes) 2024-09-14T22:37:08.342 INFO:teuthology.orchestra.run.smithi001.stdout:2773 write 0x279be9c thru 0x27a2e07 (0x6f6c bytes) 2024-09-14T22:37:08.358 INFO:teuthology.orchestra.run.smithi001.stdout:2776 trunc from 0x5e6d5ee to 0x72c0cff 2024-09-14T22:37:08.377 INFO:teuthology.orchestra.run.smithi001.stdout:2777 writesame 0x7ed7719 thru 0x7ee5585 data_size 0x1(0xde6d bytes) 2024-09-14T22:37:08.386 INFO:teuthology.orchestra.run.smithi001.stdout:2779 trunc from 0x7ee5586 to 0xe0270ae 2024-09-14T22:37:08.395 INFO:teuthology.orchestra.run.smithi001.stdout:2780 read 0x3fb088e thru 0x3fbbe13 (0xb586 bytes) 2024-09-14T22:37:08.407 INFO:teuthology.orchestra.run.smithi001.stdout:2781 write 0xea9bf7f thru 0xeaa51f5 (0x9277 bytes) 2024-09-14T22:37:08.407 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:08.407 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:08.423 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:37:08.424 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:08.419+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:08.425 INFO:teuthology.orchestra.run.smithi001.stdout:2782 read 0x204d929 thru 0x20558e3 (0x7fbb bytes) 2024-09-14T22:37:08.425 INFO:teuthology.orchestra.run.smithi001.stdout:2783 writesame 0xa83ce5 thru 0xa9194c data_size 0xc(0xdc68 bytes) 2024-09-14T22:37:08.442 INFO:teuthology.orchestra.run.smithi001.stdout:2785 writesame 0xb5a5f40 thru 0xb5acc0d data_size 0x2(0x6cce bytes) 2024-09-14T22:37:08.460 INFO:teuthology.orchestra.run.smithi001.stdout:2786 write 0xa4895a thru 0xa56717 (0xddbe bytes) 2024-09-14T22:37:08.465 INFO:teuthology.orchestra.run.smithi001.stdout:2787 write 0xdfc88fa thru 0xdfc97bf (0xec6 bytes) 2024-09-14T22:37:08.473 INFO:teuthology.orchestra.run.smithi001.stdout:2788 read 0x5abfb4c thru 0x5ac54d6 (0x598b bytes) 2024-09-14T22:37:08.473 INFO:teuthology.orchestra.run.smithi001.stdout:2789 read 0xe729e0e thru 0xe735b27 (0xbd1a bytes) 2024-09-14T22:37:08.473 INFO:teuthology.orchestra.run.smithi001.stdout:2790 punch from 0x4af1424 to 0x4af4983, (0x355f bytes) 2024-09-14T22:37:08.473 INFO:teuthology.orchestra.run.smithi001.stdout:2792 punch from 0xc74ec4b to 0xc74ff7a, (0x132f bytes) 2024-09-14T22:37:08.473 INFO:teuthology.orchestra.run.smithi001.stdout:2793 write 0x7e4d8ce thru 0x7e52e90 (0x55c3 bytes) 2024-09-14T22:37:08.495 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:37:08.495 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:37:08.502 INFO:teuthology.orchestra.run.smithi001.stdout:2794 trunc from 0xeaa51f6 to 0x2663ed5 2024-09-14T22:37:08.520 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:37:08.520 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:37:08.521 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-14T22:37:08.523 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:08.519+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:08.529 INFO:teuthology.orchestra.run.smithi001.stdout:2795 punch from 0xc532de to 0xc58ca0, (0x59c2 bytes) 2024-09-14T22:37:08.529 INFO:teuthology.orchestra.run.smithi001.stdout:2796 read 0x70c46a thru 0x71bb83 (0xf71a bytes) 2024-09-14T22:37:08.529 INFO:teuthology.orchestra.run.smithi001.stdout:2797 punch from 0x18cc890 to 0x18d3068, (0x67d8 bytes) 2024-09-14T22:37:08.546 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2024-09-14T22:37:08.549 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:37:08.550 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:37:08.559 INFO:teuthology.orchestra.run.smithi001.stdout:2798 write 0xc172ee6 thru 0xc177588 (0x46a3 bytes) 2024-09-14T22:37:08.573 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:08.573 INFO:teuthology.orchestra.run.smithi001.stdout:2801 read 0xb2e70fd thru 0xb2eaa01 (0x3905 bytes) 2024-09-14T22:37:08.573 INFO:teuthology.orchestra.run.smithi001.stdout:2803 read 0xc144218 thru 0xc149a92 (0x587b bytes) 2024-09-14T22:37:08.574 INFO:teuthology.orchestra.run.smithi001.stdout:2804 write 0x2daa2ea thru 0x2db9ad4 (0xf7eb bytes) 2024-09-14T22:37:08.583 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:37:08.583 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:37:08.584 INFO:teuthology.orchestra.run.smithi001.stdout:2805 read 0x103e92f thru 0x1040009 (0x16db bytes) 2024-09-14T22:37:08.584 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:08.584 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:08.584 INFO:teuthology.orchestra.run.smithi001.stdout:2808 write 0x557d507 thru 0x5588f80 (0xba7a bytes) 2024-09-14T22:37:08.606 INFO:teuthology.orchestra.run.smithi001.stdout:2810 write 0xc124f65 thru 0xc133cb8 (0xed54 bytes) 2024-09-14T22:37:08.608 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:37:08.608 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:37:08.609 INFO:teuthology.orchestra.run.smithi001.stdout:2811 write 0xaad6f23 thru 0xaad9300 (0x23de bytes) 2024-09-14T22:37:08.611 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:08.611 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:08.611 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:08.611 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:08.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.611 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20870 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7196672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:37:08.611 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483652+0000", 2024-09-14T22:37:08.612 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.123607012000001, 2024-09-14T22:37:08.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028301248000000001, 2024-09-14T22:37:08.612 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:08.612 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:08.612 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:08.612 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:08.612 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:08.612 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:08.612 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20870 2024-09-14T22:37:08.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.612 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:08.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:08.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483652+0000", 2024-09-14T22:37:08.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.613 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:08.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483652+0000", 2024-09-14T22:37:08.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.613 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:08.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483653+0000", 2024-09-14T22:37:08.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3099999999999997e-07 2024-09-14T22:37:08.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.613 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:08.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483655+0000", 2024-09-14T22:37:08.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.877e-06 2024-09-14T22:37:08.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.614 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:08.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483661+0000", 2024-09-14T22:37:08.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6049999999999997e-06 2024-09-14T22:37:08.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.614 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:08.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483666+0000", 2024-09-14T22:37:08.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3410000000000002e-06 2024-09-14T22:37:08.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.614 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:08.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511854+0000", 2024-09-14T22:37:08.615 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028187773999999999 2024-09-14T22:37:08.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.615 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.615 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:08.615 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511906+0000", 2024-09-14T22:37:08.615 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2012000000000003e-05 2024-09-14T22:37:08.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.615 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.615 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:08.615 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511954+0000", 2024-09-14T22:37:08.615 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8007999999999998e-05 2024-09-14T22:37:08.615 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.615 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:08.615 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.616 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.616 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20871 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7225344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:37:08.616 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483671+0000", 2024-09-14T22:37:08.616 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.123588089000002, 2024-09-14T22:37:08.616 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028405392000000002, 2024-09-14T22:37:08.616 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:08.616 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:08.616 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:08.616 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:08.616 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:08.616 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:08.616 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20871 2024-09-14T22:37:08.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.617 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:08.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.617 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:08.617 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483671+0000", 2024-09-14T22:37:08.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.617 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:08.617 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483671+0000", 2024-09-14T22:37:08.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.617 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:08.618 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483672+0000", 2024-09-14T22:37:08.618 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:37:08.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.618 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:08.618 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483674+0000", 2024-09-14T22:37:08.618 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3190000000000002e-06 2024-09-14T22:37:08.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.618 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:08.618 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483680+0000", 2024-09-14T22:37:08.618 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6860000000000003e-06 2024-09-14T22:37:08.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:08.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483684+0000", 2024-09-14T22:37:08.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3769999999999998e-06 2024-09-14T22:37:08.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:08.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.511982+0000", 2024-09-14T22:37:08.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028297478000000001 2024-09-14T22:37:08.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:08.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512027+0000", 2024-09-14T22:37:08.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5290000000000002e-05 2024-09-14T22:37:08.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:08.620 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512077+0000", 2024-09-14T22:37:08.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9876000000000002e-05 2024-09-14T22:37:08.620 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.620 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:08.620 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.620 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20872 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7254016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:37:08.620 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483689+0000", 2024-09-14T22:37:08.620 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.123570018999999, 2024-09-14T22:37:08.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028494049, 2024-09-14T22:37:08.621 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:08.621 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:08.621 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:08.621 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:08.621 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:08.621 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:08.621 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20872 2024-09-14T22:37:08.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.621 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:08.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.621 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:08.621 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483689+0000", 2024-09-14T22:37:08.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:08.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483689+0000", 2024-09-14T22:37:08.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:08.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483690+0000", 2024-09-14T22:37:08.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:37:08.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:08.623 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483692+0000", 2024-09-14T22:37:08.623 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.778e-06 2024-09-14T22:37:08.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.623 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:08.623 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483697+0000", 2024-09-14T22:37:08.623 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7069999999999997e-06 2024-09-14T22:37:08.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.623 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:08.623 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483702+0000", 2024-09-14T22:37:08.623 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4689999999999999e-06 2024-09-14T22:37:08.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.624 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.624 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:08.624 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512104+0000", 2024-09-14T22:37:08.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028401948999999999 2024-09-14T22:37:08.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.624 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.624 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:08.624 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512142+0000", 2024-09-14T22:37:08.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8114999999999997e-05 2024-09-14T22:37:08.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.624 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.624 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:08.624 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512183+0000", 2024-09-14T22:37:08.625 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1496999999999999e-05 2024-09-14T22:37:08.625 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.625 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:08.625 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.625 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.625 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20873 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7282688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:37:08.625 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483707+0000", 2024-09-14T22:37:08.625 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.123552304999997, 2024-09-14T22:37:08.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028581662000000001, 2024-09-14T22:37:08.626 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:08.626 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:08.626 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:08.626 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:08.626 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:08.626 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:08.626 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20873 2024-09-14T22:37:08.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.626 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:08.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:08.627 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483707+0000", 2024-09-14T22:37:08.627 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.627 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:08.627 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483707+0000", 2024-09-14T22:37:08.627 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.627 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:08.627 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483707+0000", 2024-09-14T22:37:08.627 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2500000000000001e-07 2024-09-14T22:37:08.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:08.628 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483709+0000", 2024-09-14T22:37:08.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.891e-06 2024-09-14T22:37:08.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:08.628 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483715+0000", 2024-09-14T22:37:08.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7019999999999998e-06 2024-09-14T22:37:08.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:08.628 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483720+0000", 2024-09-14T22:37:08.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4900000000000002e-06 2024-09-14T22:37:08.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.629 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.629 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:08.629 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512213+0000", 2024-09-14T22:37:08.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028493622 2024-09-14T22:37:08.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.629 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.629 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:08.629 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512248+0000", 2024-09-14T22:37:08.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5228000000000002e-05 2024-09-14T22:37:08.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.629 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.629 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:08.629 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512289+0000", 2024-09-14T22:37:08.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0404000000000001e-05 2024-09-14T22:37:08.629 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.630 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:08.630 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.630 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.630 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20874 3.c 3:3557682d:::rbd_data.113cda0183a0.000000000000000a:6 [read 7311360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:37:08.630 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.483725+0000", 2024-09-14T22:37:08.630 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.123534665999998, 2024-09-14T22:37:08.630 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028777193999999999, 2024-09-14T22:37:08.630 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:08.630 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:08.630 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:08.630 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:08.630 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:08.631 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:08.631 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20874 2024-09-14T22:37:08.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.631 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:08.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:08.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483725+0000", 2024-09-14T22:37:08.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:08.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483725+0000", 2024-09-14T22:37:08.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.632 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:08.632 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483725+0000", 2024-09-14T22:37:08.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8000000000000001e-07 2024-09-14T22:37:08.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.632 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:08.632 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483727+0000", 2024-09-14T22:37:08.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8920000000000001e-06 2024-09-14T22:37:08.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.632 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:08.632 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483733+0000", 2024-09-14T22:37:08.633 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6840000000000001e-06 2024-09-14T22:37:08.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.633 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:08.633 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.483737+0000", 2024-09-14T22:37:08.633 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4440000000000001e-06 2024-09-14T22:37:08.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.633 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:08.633 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512326+0000", 2024-09-14T22:37:08.633 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028588612999999999 2024-09-14T22:37:08.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.633 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:08.633 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512399+0000", 2024-09-14T22:37:08.634 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2782999999999998e-05 2024-09-14T22:37:08.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.634 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.634 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:08.634 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512502+0000", 2024-09-14T22:37:08.634 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000103398 2024-09-14T22:37:08.634 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.634 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:08.634 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.634 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.634 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:20912 3.1c 3:3b9f022e:::rbd_data.113c4558bb1.000000000000009a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 497664~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-14T22:37:08.634 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:32.484800+0000", 2024-09-14T22:37:08.634 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.122459138000004, 2024-09-14T22:37:08.634 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045787688999999999, 2024-09-14T22:37:08.635 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:08.635 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:08.635 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:08.635 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:08.635 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:08.635 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:08.635 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20912 2024-09-14T22:37:08.635 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.635 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:08.635 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.635 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:08.635 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484800+0000", 2024-09-14T22:37:08.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.635 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.636 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.636 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:08.636 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484800+0000", 2024-09-14T22:37:08.636 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.636 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.636 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:08.636 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484801+0000", 2024-09-14T22:37:08.636 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1939999999999999e-06 2024-09-14T22:37:08.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.636 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.636 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:08.636 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484809+0000", 2024-09-14T22:37:08.637 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2989999999999999e-06 2024-09-14T22:37:08.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.637 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.637 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:08.637 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484823+0000", 2024-09-14T22:37:08.637 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3986e-05 2024-09-14T22:37:08.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.637 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.637 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:08.637 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.484831+0000", 2024-09-14T22:37:08.637 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4009999999999997e-06 2024-09-14T22:37:08.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.637 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.637 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:08.637 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512318+0000", 2024-09-14T22:37:08.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027487208999999999 2024-09-14T22:37:08.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:08.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512450+0000", 2024-09-14T22:37:08.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000131681 2024-09-14T22:37:08.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:37:08.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.512666+0000", 2024-09-14T22:37:08.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021547 2024-09-14T22:37:08.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:08.639 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.513788+0000", 2024-09-14T22:37:08.639 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001122975 2024-09-14T22:37:08.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.639 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.639 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:08.639 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.530473+0000", 2024-09-14T22:37:08.639 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.016684984999999999 2024-09-14T22:37:08.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.639 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.639 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:08.639 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.530533+0000", 2024-09-14T22:37:08.640 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9296999999999998e-05 2024-09-14T22:37:08.640 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.640 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.640 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:08.640 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:32.530588+0000", 2024-09-14T22:37:08.640 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5192000000000003e-05 2024-09-14T22:37:08.640 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.640 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:08.640 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.640 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.640 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.640 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:30514 3.1d 3:b9e08d52:::rbd_data.113cda0183a0.0000000000000008:6 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:08.641 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.331927+0000", 2024-09-14T22:37:08.641 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 35.275332145, 2024-09-14T22:37:08.641 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028476330000000001, 2024-09-14T22:37:08.641 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:08.641 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:08.641 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:08.641 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:08.641 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:08.641 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:08.641 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 30514 2024-09-14T22:37:08.641 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.641 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:08.642 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.642 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:08.642 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331927+0000", 2024-09-14T22:37:08.642 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.642 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.642 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:08.642 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331927+0000", 2024-09-14T22:37:08.642 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.642 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.642 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:08.642 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331928+0000", 2024-09-14T22:37:08.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5700000000000002e-07 2024-09-14T22:37:08.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.643 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:08.643 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331931+0000", 2024-09-14T22:37:08.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9909999999999998e-06 2024-09-14T22:37:08.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.643 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:08.643 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331940+0000", 2024-09-14T22:37:08.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.312e-06 2024-09-14T22:37:08.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.644 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:08.644 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331948+0000", 2024-09-14T22:37:08.644 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5630000000000003e-06 2024-09-14T22:37:08.644 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.644 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.644 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:08.644 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360032+0000", 2024-09-14T22:37:08.644 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028083877 2024-09-14T22:37:08.644 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.644 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.644 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:08.644 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360282+0000", 2024-09-14T22:37:08.644 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00025062100000000003 2024-09-14T22:37:08.644 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.644 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.645 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:08.645 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360404+0000", 2024-09-14T22:37:08.645 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000121409 2024-09-14T22:37:08.645 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.645 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:08.645 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.645 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.645 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.645 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:30515 3.1d 3:b9e08d52:::rbd_data.113cda0183a0.0000000000000008:6 [read 514048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:08.645 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:33.331957+0000", 2024-09-14T22:37:08.645 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 35.275302320000002, 2024-09-14T22:37:08.645 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028684729999999999, 2024-09-14T22:37:08.645 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:08.646 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:08.646 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:08.646 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:08.646 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:08.646 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:08.646 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 30515 2024-09-14T22:37:08.646 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.646 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:08.646 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.646 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:08.646 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331957+0000", 2024-09-14T22:37:08.646 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.646 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.646 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.647 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:08.647 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331957+0000", 2024-09-14T22:37:08.647 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.647 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:08.647 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331958+0000", 2024-09-14T22:37:08.647 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1200000000000003e-07 2024-09-14T22:37:08.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.647 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:08.647 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331960+0000", 2024-09-14T22:37:08.647 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6929999999999999e-06 2024-09-14T22:37:08.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.648 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:08.648 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331970+0000", 2024-09-14T22:37:08.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4020000000000008e-06 2024-09-14T22:37:08.648 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.648 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.648 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:08.648 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.331977+0000", 2024-09-14T22:37:08.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7660000000000001e-06 2024-09-14T22:37:08.648 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.648 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.648 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:08.648 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360444+0000", 2024-09-14T22:37:08.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028467399000000001 2024-09-14T22:37:08.648 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.649 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.649 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:08.649 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360536+0000", 2024-09-14T22:37:08.649 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1618000000000003e-05 2024-09-14T22:37:08.649 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.649 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.649 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:08.649 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:33.360642+0000", 2024-09-14T22:37:08.649 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010624 2024-09-14T22:37:08.649 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.649 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:08.649 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.649 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.649 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.649 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:46576 3.1d 3:b9b1f07f:::rbd_data.113c8642facc.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T22:37:08.650 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:37.379223+0000", 2024-09-14T22:37:08.650 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 31.228036795000001, 2024-09-14T22:37:08.650 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072706044999999997, 2024-09-14T22:37:08.650 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:08.650 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:08.650 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:08.650 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:08.650 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:08.650 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:08.650 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 46576 2024-09-14T22:37:08.650 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.650 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:08.650 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.650 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:08.650 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379223+0000", 2024-09-14T22:37:08.650 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.651 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.651 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.651 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:08.651 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379223+0000", 2024-09-14T22:37:08.651 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.651 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.651 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.651 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:08.651 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379226+0000", 2024-09-14T22:37:08.651 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0929999999999999e-06 2024-09-14T22:37:08.651 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.651 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.651 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:08.651 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379240+0000", 2024-09-14T22:37:08.651 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4606999999999999e-05 2024-09-14T22:37:08.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.652 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:08.652 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379272+0000", 2024-09-14T22:37:08.652 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2073000000000003e-05 2024-09-14T22:37:08.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.652 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:08.652 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379288+0000", 2024-09-14T22:37:08.652 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6036999999999999e-05 2024-09-14T22:37:08.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.652 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:08.652 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379451+0000", 2024-09-14T22:37:08.652 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000162795 2024-09-14T22:37:08.653 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.653 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.653 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:08.653 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.379702+0000", 2024-09-14T22:37:08.653 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00025106500000000002 2024-09-14T22:37:08.653 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.653 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.653 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:37:08.653 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.394067+0000", 2024-09-14T22:37:08.653 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014364564999999999 2024-09-14T22:37:08.653 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.653 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.653 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:08.653 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.451737+0000", 2024-09-14T22:37:08.653 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057670185999999998 2024-09-14T22:37:08.654 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.654 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.654 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:08.654 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.451823+0000", 2024-09-14T22:37:08.654 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6075000000000005e-05 2024-09-14T22:37:08.654 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.654 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.654 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:08.654 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.451881+0000", 2024-09-14T22:37:08.654 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7454999999999998e-05 2024-09-14T22:37:08.654 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.654 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.654 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:08.654 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:37.451929+0000", 2024-09-14T22:37:08.654 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8093999999999997e-05 2024-09-14T22:37:08.655 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.655 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:08.655 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.655 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.655 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.655 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:74169 3.16 3:69acee67:::rbd_data.113cce28790.0000000000000067:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T22:37:08.655 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:55.614872+0000", 2024-09-14T22:37:08.655 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.992387609, 2024-09-14T22:37:08.655 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036045858, 2024-09-14T22:37:08.655 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:08.655 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:08.655 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:08.655 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:08.655 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:08.655 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:08.655 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74169 2024-09-14T22:37:08.656 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.656 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:08.656 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.656 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:08.656 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.614872+0000", 2024-09-14T22:37:08.656 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.656 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.656 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.656 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:08.656 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.614872+0000", 2024-09-14T22:37:08.656 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.656 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.656 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.657 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:08.657 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.614875+0000", 2024-09-14T22:37:08.657 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2799999999999999e-06 2024-09-14T22:37:08.657 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.657 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.657 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:08.657 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.614891+0000", 2024-09-14T22:37:08.657 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5613000000000001e-05 2024-09-14T22:37:08.657 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.657 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.657 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:08.657 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.614931+0000", 2024-09-14T22:37:08.657 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0540000000000001e-05 2024-09-14T22:37:08.658 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.658 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.658 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:08.658 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.614956+0000", 2024-09-14T22:37:08.658 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4939e-05 2024-09-14T22:37:08.658 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.658 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.658 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:08.658 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.615096+0000", 2024-09-14T22:37:08.658 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013952600000000001 2024-09-14T22:37:08.658 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.658 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.658 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:08.658 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.615345+0000", 2024-09-14T22:37:08.658 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024922099999999999 2024-09-14T22:37:08.659 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.659 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.659 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T22:37:08.659 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.621370+0000", 2024-09-14T22:37:08.659 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0060251239999999998 2024-09-14T22:37:08.659 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.659 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.659 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:08.659 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.650734+0000", 2024-09-14T22:37:08.659 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029364155999999999 2024-09-14T22:37:08.659 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.659 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.659 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:08.660 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.650760+0000", 2024-09-14T22:37:08.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5446999999999999e-05 2024-09-14T22:37:08.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.660 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:08.660 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.650842+0000", 2024-09-14T22:37:08.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2268999999999994e-05 2024-09-14T22:37:08.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.660 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:08.660 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.650918+0000", 2024-09-14T22:37:08.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5742999999999996e-05 2024-09-14T22:37:08.660 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.660 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:08.661 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.661 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:85374 3.16 3:688cf0a4:::rbd_data.113cff0d0fb7.000000000000000a:12 [read 262144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:08.661 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:00.254720+0000", 2024-09-14T22:37:08.661 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.3525398160000002, 2024-09-14T22:37:08.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028235064000000001, 2024-09-14T22:37:08.661 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:08.661 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:08.661 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:08.661 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:08.661 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:08.661 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:08.661 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 85374 2024-09-14T22:37:08.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.662 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:08.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:08.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254720+0000", 2024-09-14T22:37:08.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:08.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254720+0000", 2024-09-14T22:37:08.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:08.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254720+0000", 2024-09-14T22:37:08.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4499999999999997e-07 2024-09-14T22:37:08.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.663 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:08.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254723+0000", 2024-09-14T22:37:08.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5919999999999999e-06 2024-09-14T22:37:08.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.663 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:08.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254729+0000", 2024-09-14T22:37:08.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5640000000000002e-06 2024-09-14T22:37:08.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.663 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:08.664 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254734+0000", 2024-09-14T22:37:08.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8049999999999997e-06 2024-09-14T22:37:08.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.664 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:08.664 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.282761+0000", 2024-09-14T22:37:08.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028026669000000001 2024-09-14T22:37:08.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.664 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:08.664 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.282840+0000", 2024-09-14T22:37:08.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9621000000000002e-05 2024-09-14T22:37:08.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:08.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.282955+0000", 2024-09-14T22:37:08.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000114368 2024-09-14T22:37:08.665 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.665 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:08.665 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.665 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:85375 3.16 3:688cf0a4:::rbd_data.113cff0d0fb7.000000000000000a:12 [read 356352~12288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:08.665 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:00.254740+0000", 2024-09-14T22:37:08.665 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.3525192799999992, 2024-09-14T22:37:08.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028433526000000001, 2024-09-14T22:37:08.665 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:08.666 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:08.666 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:08.666 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:08.666 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:08.666 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:08.666 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 85375 2024-09-14T22:37:08.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.666 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:08.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:08.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254740+0000", 2024-09-14T22:37:08.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:08.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254740+0000", 2024-09-14T22:37:08.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:08.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254741+0000", 2024-09-14T22:37:08.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8299999999999998e-07 2024-09-14T22:37:08.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:08.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254743+0000", 2024-09-14T22:37:08.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.154e-06 2024-09-14T22:37:08.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:08.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254748+0000", 2024-09-14T22:37:08.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7459999999999997e-06 2024-09-14T22:37:08.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:08.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254753+0000", 2024-09-14T22:37:08.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6600000000000003e-06 2024-09-14T22:37:08.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:08.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.282989+0000", 2024-09-14T22:37:08.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028235752999999999 2024-09-14T22:37:08.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:08.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.283067+0000", 2024-09-14T22:37:08.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8559000000000005e-05 2024-09-14T22:37:08.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:08.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.283174+0000", 2024-09-14T22:37:08.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106271 2024-09-14T22:37:08.669 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.669 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:08.669 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.669 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:85376 3.16 3:688cf0a4:::rbd_data.113cff0d0fb7.000000000000000a:12 [read 446464~12288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:08.670 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:00.254759+0000", 2024-09-14T22:37:08.670 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.3525001920000008, 2024-09-14T22:37:08.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028655568999999999, 2024-09-14T22:37:08.670 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:08.670 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:08.670 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:08.670 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:08.670 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:08.670 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:08.670 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 85376 2024-09-14T22:37:08.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.670 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:08.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:08.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254759+0000", 2024-09-14T22:37:08.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:08.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254759+0000", 2024-09-14T22:37:08.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:08.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254760+0000", 2024-09-14T22:37:08.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5600000000000001e-07 2024-09-14T22:37:08.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:08.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254762+0000", 2024-09-14T22:37:08.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.142e-06 2024-09-14T22:37:08.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:08.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254768+0000", 2024-09-14T22:37:08.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9819999999999996e-06 2024-09-14T22:37:08.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:08.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.254773+0000", 2024-09-14T22:37:08.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0690000000000001e-06 2024-09-14T22:37:08.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:08.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.283207+0000", 2024-09-14T22:37:08.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028434207999999999 2024-09-14T22:37:08.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.673 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:08.673 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.283296+0000", 2024-09-14T22:37:08.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8636999999999999e-05 2024-09-14T22:37:08.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.673 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:08.673 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.283415+0000", 2024-09-14T22:37:08.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119175 2024-09-14T22:37:08.673 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.673 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:08.673 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.673 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:85461 3.16 3:68c13f99:::rbd_data.113cff0d0fb7.000000000000001b:12 [read 380928~12288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:08.674 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:00.261582+0000", 2024-09-14T22:37:08.674 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.3456773999999996, 2024-09-14T22:37:08.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029353918999999999, 2024-09-14T22:37:08.674 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:08.674 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:08.674 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:08.674 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:08.674 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:08.674 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:08.674 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 85461 2024-09-14T22:37:08.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.674 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:08.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:08.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.261582+0000", 2024-09-14T22:37:08.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:08.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.261582+0000", 2024-09-14T22:37:08.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:08.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.261582+0000", 2024-09-14T22:37:08.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:37:08.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:08.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.261585+0000", 2024-09-14T22:37:08.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.492e-06 2024-09-14T22:37:08.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:08.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.261591+0000", 2024-09-14T22:37:08.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5760000000000002e-06 2024-09-14T22:37:08.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:08.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.261598+0000", 2024-09-14T22:37:08.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4370000000000004e-06 2024-09-14T22:37:08.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:08.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.284316+0000", 2024-09-14T22:37:08.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022718341 2024-09-14T22:37:08.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:08.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.284384+0000", 2024-09-14T22:37:08.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7342999999999995e-05 2024-09-14T22:37:08.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:08.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:00.290936+0000", 2024-09-14T22:37:08.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0065523430000000004 2024-09-14T22:37:08.677 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.677 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:08.677 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.678 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:88836 3.16 3:6b6ec4f7:::rbd_object_map.113c5a3f285d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-14T22:37:08.678 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:06.215341+0000", 2024-09-14T22:37:08.678 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 2.3919181360000001, 2024-09-14T22:37:08.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029987278999999999, 2024-09-14T22:37:08.678 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:08.678 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:08.678 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:08.678 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:08.678 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:08.678 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:08.678 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 88836 2024-09-14T22:37:08.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.678 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:08.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:08.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.215341+0000", 2024-09-14T22:37:08.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:08.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.215341+0000", 2024-09-14T22:37:08.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:08.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.215342+0000", 2024-09-14T22:37:08.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1230000000000001e-06 2024-09-14T22:37:08.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:08.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.215349+0000", 2024-09-14T22:37:08.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5769999999999999e-06 2024-09-14T22:37:08.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:08.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.215367+0000", 2024-09-14T22:37:08.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8136000000000001e-05 2024-09-14T22:37:08.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:08.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.215382+0000", 2024-09-14T22:37:08.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4433000000000001e-05 2024-09-14T22:37:08.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:08.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.232470+0000", 2024-09-14T22:37:08.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017088394999999999 2024-09-14T22:37:08.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:08.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.232747+0000", 2024-09-14T22:37:08.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027697799999999999 2024-09-14T22:37:08.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T22:37:08.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.233591+0000", 2024-09-14T22:37:08.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00084451899999999996 2024-09-14T22:37:08.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:08.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.237719+0000", 2024-09-14T22:37:08.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004127436 2024-09-14T22:37:08.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:08.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.245185+0000", 2024-09-14T22:37:08.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0074657389999999999 2024-09-14T22:37:08.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:08.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.245266+0000", 2024-09-14T22:37:08.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1145999999999999e-05 2024-09-14T22:37:08.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:08.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.245329+0000", 2024-09-14T22:37:08.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2797000000000002e-05 2024-09-14T22:37:08.683 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.683 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:08.683 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.683 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:88837 3.16 3:6b6ec4f7:::rbd_object_map.113c5a3f285d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-14T22:37:08.683 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:06.215399+0000", 2024-09-14T22:37:08.683 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 2.3918607330000001, 2024-09-14T22:37:08.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030449014, 2024-09-14T22:37:08.683 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:08.683 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:08.683 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:08.683 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:08.684 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:08.684 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:08.684 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 88837 2024-09-14T22:37:08.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.684 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:08.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:08.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.215399+0000", 2024-09-14T22:37:08.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:08.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.215399+0000", 2024-09-14T22:37:08.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:08.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.215400+0000", 2024-09-14T22:37:08.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1999999999999999e-06 2024-09-14T22:37:08.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:08.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.215435+0000", 2024-09-14T22:37:08.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5102999999999999e-05 2024-09-14T22:37:08.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:08.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.215476+0000", 2024-09-14T22:37:08.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0578999999999999e-05 2024-09-14T22:37:08.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:08.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.215492+0000", 2024-09-14T22:37:08.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6297e-05 2024-09-14T22:37:08.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:08.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.234601+0000", 2024-09-14T22:37:08.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.019108664000000001 2024-09-14T22:37:08.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:08.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.234778+0000", 2024-09-14T22:37:08.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000177515 2024-09-14T22:37:08.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T22:37:08.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.235940+0000", 2024-09-14T22:37:08.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001161528 2024-09-14T22:37:08.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:08.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.245356+0000", 2024-09-14T22:37:08.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0094160489999999993 2024-09-14T22:37:08.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:08.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.245596+0000", 2024-09-14T22:37:08.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024065100000000001 2024-09-14T22:37:08.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:08.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.245747+0000", 2024-09-14T22:37:08.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000150832 2024-09-14T22:37:08.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:08.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.245848+0000", 2024-09-14T22:37:08.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000100596 2024-09-14T22:37:08.688 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.689 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:08.689 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.689 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:88839 3.16 3:6b6ec4f7:::rbd_object_map.113c5a3f285d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-14T22:37:08.689 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:06.216706+0000", 2024-09-14T22:37:08.689 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 2.3905532570000001, 2024-09-14T22:37:08.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029656426999999999, 2024-09-14T22:37:08.689 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:08.689 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:08.689 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:08.689 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:08.689 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:08.690 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:08.690 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 88839 2024-09-14T22:37:08.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.690 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:08.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:08.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.216706+0000", 2024-09-14T22:37:08.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:08.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.216706+0000", 2024-09-14T22:37:08.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:08.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.216709+0000", 2024-09-14T22:37:08.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7259999999999998e-06 2024-09-14T22:37:08.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:08.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.216723+0000", 2024-09-14T22:37:08.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4430000000000001e-05 2024-09-14T22:37:08.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:08.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.216757+0000", 2024-09-14T22:37:08.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3917000000000001e-05 2024-09-14T22:37:08.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.692 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:08.692 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.216779+0000", 2024-09-14T22:37:08.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1339999999999999e-05 2024-09-14T22:37:08.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.692 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:08.692 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.238293+0000", 2024-09-14T22:37:08.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021514234 2024-09-14T22:37:08.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.692 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:08.693 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.238564+0000", 2024-09-14T22:37:08.693 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027089099999999999 2024-09-14T22:37:08.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.693 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T22:37:08.693 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.239510+0000", 2024-09-14T22:37:08.693 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00094636899999999996 2024-09-14T22:37:08.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.693 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:08.693 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.246071+0000", 2024-09-14T22:37:08.693 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0065612759999999996 2024-09-14T22:37:08.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.694 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:08.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.246124+0000", 2024-09-14T22:37:08.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2636000000000002e-05 2024-09-14T22:37:08.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.694 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:08.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.246264+0000", 2024-09-14T22:37:08.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013956600000000001 2024-09-14T22:37:08.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.694 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:08.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.246363+0000", 2024-09-14T22:37:08.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9042000000000005e-05 2024-09-14T22:37:08.694 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.695 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:08.695 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.695 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.695 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:88841 3.16 3:6b6ec4f7:::rbd_object_map.113c5a3f285d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-14T22:37:08.695 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:06.222309+0000", 2024-09-14T22:37:08.695 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 2.3849500030000002, 2024-09-14T22:37:08.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028695439, 2024-09-14T22:37:08.695 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:08.695 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:08.695 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:08.695 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:08.695 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:08.695 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:08.695 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 88841 2024-09-14T22:37:08.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.696 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:08.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.696 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:08.696 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.222309+0000", 2024-09-14T22:37:08.696 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.696 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.696 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:08.696 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.222309+0000", 2024-09-14T22:37:08.696 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.696 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.696 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:08.696 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.222311+0000", 2024-09-14T22:37:08.696 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4899999999999999e-06 2024-09-14T22:37:08.696 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:08.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.222319+0000", 2024-09-14T22:37:08.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4239999999999993e-06 2024-09-14T22:37:08.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:08.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.222340+0000", 2024-09-14T22:37:08.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1152999999999999e-05 2024-09-14T22:37:08.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:08.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.222356+0000", 2024-09-14T22:37:08.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5537e-05 2024-09-14T22:37:08.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:08.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.243140+0000", 2024-09-14T22:37:08.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020783553 2024-09-14T22:37:08.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:08.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.243397+0000", 2024-09-14T22:37:08.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000257877 2024-09-14T22:37:08.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T22:37:08.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.244290+0000", 2024-09-14T22:37:08.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00089247999999999997 2024-09-14T22:37:08.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:08.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.250600+0000", 2024-09-14T22:37:08.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0063104149999999998 2024-09-14T22:37:08.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:08.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.250707+0000", 2024-09-14T22:37:08.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106189 2024-09-14T22:37:08.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:08.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.250859+0000", 2024-09-14T22:37:08.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015257500000000001 2024-09-14T22:37:08.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.700 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:08.700 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.251005+0000", 2024-09-14T22:37:08.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014574600000000001 2024-09-14T22:37:08.700 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.700 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:08.700 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.700 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:88842 3.16 3:6b6ec4f7:::rbd_object_map.113c5a3f285d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-14T22:37:08.700 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:06.224476+0000", 2024-09-14T22:37:08.700 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 2.3827830570000001, 2024-09-14T22:37:08.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028662149000000001, 2024-09-14T22:37:08.701 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:08.701 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:08.701 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:08.701 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:08.701 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:08.701 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:08.701 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 88842 2024-09-14T22:37:08.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.701 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:08.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:08.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.224476+0000", 2024-09-14T22:37:08.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:08.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.224476+0000", 2024-09-14T22:37:08.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:08.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.224479+0000", 2024-09-14T22:37:08.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7669999999999999e-06 2024-09-14T22:37:08.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:08.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.224493+0000", 2024-09-14T22:37:08.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.418e-05 2024-09-14T22:37:08.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:08.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.224531+0000", 2024-09-14T22:37:08.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8102999999999998e-05 2024-09-14T22:37:08.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:08.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.224552+0000", 2024-09-14T22:37:08.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0619999999999999e-05 2024-09-14T22:37:08.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:08.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.246680+0000", 2024-09-14T22:37:08.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022127838 2024-09-14T22:37:08.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:08.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.246842+0000", 2024-09-14T22:37:08.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016226899999999999 2024-09-14T22:37:08.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T22:37:08.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.247657+0000", 2024-09-14T22:37:08.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00081517299999999998 2024-09-14T22:37:08.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:08.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.251050+0000", 2024-09-14T22:37:08.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0033925230000000002 2024-09-14T22:37:08.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:08.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.252911+0000", 2024-09-14T22:37:08.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0018616380000000001 2024-09-14T22:37:08.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:08.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.253059+0000", 2024-09-14T22:37:08.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014763600000000001 2024-09-14T22:37:08.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:08.706 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.253139+0000", 2024-09-14T22:37:08.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9401999999999996e-05 2024-09-14T22:37:08.706 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.706 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:08.706 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.706 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:88844 3.16 3:6b6ec4f7:::rbd_object_map.113c5a3f285d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-14T22:37:08.706 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:06.224901+0000", 2024-09-14T22:37:08.706 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 2.3823582669999999, 2024-09-14T22:37:08.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029418771999999999, 2024-09-14T22:37:08.707 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:08.707 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:08.707 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:08.707 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:08.707 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:08.707 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:08.707 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 88844 2024-09-14T22:37:08.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.707 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:08.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:08.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.224901+0000", 2024-09-14T22:37:08.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:08.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.224901+0000", 2024-09-14T22:37:08.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:08.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:08.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.224904+0000", 2024-09-14T22:37:08.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5569999999999998e-06 2024-09-14T22:37:08.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:08.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.224916+0000", 2024-09-14T22:37:08.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2089999999999999e-05 2024-09-14T22:37:08.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:08.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.224949+0000", 2024-09-14T22:37:08.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2786000000000001e-05 2024-09-14T22:37:08.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:08.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.224970+0000", 2024-09-14T22:37:08.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1027e-05 2024-09-14T22:37:08.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:08.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.248806+0000", 2024-09-14T22:37:08.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023836600999999999 2024-09-14T22:37:08.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:08.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.249021+0000", 2024-09-14T22:37:08.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021489499999999999 2024-09-14T22:37:08.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T22:37:08.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.249687+0000", 2024-09-14T22:37:08.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00066561799999999998 2024-09-14T22:37:08.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:08.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.253713+0000", 2024-09-14T22:37:08.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0040259969999999999 2024-09-14T22:37:08.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:08.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.254059+0000", 2024-09-14T22:37:08.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000346206 2024-09-14T22:37:08.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:08.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.254206+0000", 2024-09-14T22:37:08.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014756700000000001 2024-09-14T22:37:08.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:08.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:08.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:08.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:06.254320+0000", 2024-09-14T22:37:08.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000113428 2024-09-14T22:37:08.712 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.712 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:08.712 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.712 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:08.712 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:08.712 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:08.713 INFO:teuthology.orchestra.run.smithi001.stdout:2812 read 0xa6033d8 thru 0xa6102b9 (0xcee2 bytes) 2024-09-14T22:37:08.713 INFO:teuthology.orchestra.run.smithi001.stdout:2813 punch from 0x6f2e861 to 0x6f337f4, (0x4f93 bytes) 2024-09-14T22:37:08.713 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:08.714 INFO:teuthology.orchestra.run.smithi001.stdout:2815 trunc from 0xc177589 to 0xbecd514 2024-09-14T22:37:08.714 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:37:08.714 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:37:08.715 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:37:08.715 INFO:teuthology.orchestra.run.smithi001.stdout:2816 read 0x4c56911 thru 0x4c616e4 (0xadd4 bytes) 2024-09-14T22:37:08.715 INFO:teuthology.orchestra.run.smithi001.stdout:2817 writesame 0xeb4f409 thru 0xeb5ba54 data_size 0x94(0xc64c bytes) 2024-09-14T22:37:08.715 INFO:teuthology.orchestra.run.smithi001.stdout:2818 writesame 0xcbb62bf thru 0xcbc5fbe data_size 0x20(0xfd00 bytes) 2024-09-14T22:37:08.716 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:37:08.716 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:37:08.716 INFO:teuthology.orchestra.run.smithi001.stdout:2821 punch from 0x271cccf to 0x272a6fd, (0xda2e bytes) 2024-09-14T22:37:08.716 INFO:teuthology.orchestra.run.smithi001.stdout:2822 read 0x3079b29 thru 0x3080c78 (0x7150 bytes) 2024-09-14T22:37:08.716 INFO:teuthology.orchestra.run.smithi001.stdout:2823 punch from 0xbef9525 to 0xbf06ac4, (0xd59f bytes) 2024-09-14T22:37:08.716 INFO:teuthology.orchestra.run.smithi001.stdout:2824 punch from 0x29afe1b to 0x29b616a, (0x634f bytes) 2024-09-14T22:37:08.717 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:37:08.717 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:37:08.718 INFO:teuthology.orchestra.run.smithi001.stdout:2827 writesame 0xeca942d thru 0xecaa894 data_size 0x4(0x1468 bytes) 2024-09-14T22:37:08.718 INFO:teuthology.orchestra.run.smithi001.stdout:2828 punch from 0xaf54692 to 0xaf5724f, (0x2bbd bytes) 2024-09-14T22:37:08.718 INFO:teuthology.orchestra.run.smithi001.stdout:2829 read 0xe5c8d07 thru 0xe5d52bc (0xc5b6 bytes) 2024-09-14T22:37:08.718 INFO:teuthology.orchestra.run.smithi001.stdout:2830 punch from 0x6e9d92c to 0x6ea6eea, (0x95be bytes) 2024-09-14T22:37:08.718 INFO:teuthology.orchestra.run.smithi001.stdout:2831 trunc from 0xecaa895 to 0x4ee0e64 2024-09-14T22:37:08.718 INFO:teuthology.orchestra.run.smithi001.stdout:2832 write 0x638553f thru 0x638a103 (0x4bc5 bytes) 2024-09-14T22:37:08.718 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:08.711+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:08.726 INFO:teuthology.orchestra.run.smithi001.stdout:2834 trunc from 0x638a104 to 0x3c985f4 2024-09-14T22:37:08.739 INFO:teuthology.orchestra.run.smithi001.stdout:2835 read 0x782049 thru 0x78cb06 (0xaabe bytes) 2024-09-14T22:37:08.739 INFO:teuthology.orchestra.run.smithi001.stdout:2836 write 0x1845758 thru 0x184b6c7 (0x5f70 bytes) 2024-09-14T22:37:08.745 INFO:teuthology.orchestra.run.smithi001.stdout:2839 punch from 0x26a9a99 to 0x26acd17, (0x327e bytes) 2024-09-14T22:37:08.759 INFO:teuthology.orchestra.run.smithi001.stdout:2841 writesame 0x6166850 thru 0x616aafc data_size 0xd(0x42ad bytes) 2024-09-14T22:37:08.759 INFO:teuthology.orchestra.run.smithi001.stdout:2842 read 0x35f108c thru 0x35f501f (0x3f94 bytes) 2024-09-14T22:37:08.778 INFO:teuthology.orchestra.run.smithi001.stdout:2843 writesame 0xb6cd221 thru 0xb6db1c9 data_size 0x1(0xdfa9 bytes) 2024-09-14T22:37:08.778 INFO:teuthology.orchestra.run.smithi001.stdout:2844 write 0x757c3f2 thru 0x757ee19 (0x2a28 bytes) 2024-09-14T22:37:08.793 INFO:teuthology.orchestra.run.smithi001.stdout:2845 write 0x3d0fd2d thru 0x3d13f2c (0x4200 bytes) 2024-09-14T22:37:08.810 INFO:teuthology.orchestra.run.smithi001.stdout:2846 write 0xad57938 thru 0xad6483a (0xcf03 bytes) 2024-09-14T22:37:08.814 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:08.807+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:08.839 INFO:teuthology.orchestra.run.smithi001.stdout:2847 write 0xd33dc9e thru 0xd34c520 (0xe883 bytes) 2024-09-14T22:37:08.857 INFO:teuthology.orchestra.run.smithi001.stdout:2849 trunc from 0xd34c521 to 0xbc6090e 2024-09-14T22:37:08.876 INFO:teuthology.orchestra.run.smithi001.stdout:2850 read 0xa27698a thru 0xa27c3c7 (0x5a3e bytes) 2024-09-14T22:37:08.876 INFO:teuthology.orchestra.run.smithi001.stdout:2851 read 0x895d5ee thru 0x895d64f (0x62 bytes) 2024-09-14T22:37:08.877 INFO:teuthology.orchestra.run.smithi001.stdout:2852 write 0x12de410 thru 0x12ec510 (0xe101 bytes) 2024-09-14T22:37:08.892 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:08.892 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:08.892 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:37:08.892 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:08.893 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:08.893 INFO:teuthology.orchestra.run.smithi001.stdout:2854 read 0x35d826c thru 0x35dbc62 (0x39f7 bytes) 2024-09-14T22:37:08.893 INFO:teuthology.orchestra.run.smithi001.stdout:2857 read 0x70efc3f thru 0x70f33ec (0x37ae bytes) 2024-09-14T22:37:08.893 INFO:teuthology.orchestra.run.smithi001.stdout:2858 punch from 0x2645458 to 0x2653b03, (0xe6ab bytes) 2024-09-14T22:37:08.893 INFO:teuthology.orchestra.run.smithi001.stdout:2860 read 0x9940ed8 thru 0x9949499 (0x85c2 bytes) 2024-09-14T22:37:08.893 INFO:teuthology.orchestra.run.smithi001.stdout:2862 read 0xa091e97 thru 0xa092a6f (0xbd9 bytes) 2024-09-14T22:37:08.906 INFO:teuthology.orchestra.run.smithi001.stdout:2863 write 0xc9f64a4 thru 0xc9fc547 (0x60a4 bytes) 2024-09-14T22:37:08.907 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:37:08.919 INFO:teuthology.orchestra.run.smithi001.stdout:2867 writesame 0xc5ebb68 thru 0xc5f915c data_size 0x1(0xd5f5 bytes) 2024-09-14T22:37:08.955 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:08.951+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:08.958 INFO:teuthology.orchestra.run.smithi001.stdout:2868 trunc from 0xc9fc548 to 0x64bfc79 2024-09-14T22:37:08.999 INFO:teuthology.orchestra.run.smithi001.stdout:2869 write 0x9f1b66d thru 0x9f20380 (0x4d14 bytes) 2024-09-14T22:37:09.016 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:09.011+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:09.019 INFO:teuthology.orchestra.run.smithi001.stdout:2870 write 0x68be004 thru 0x68c4144 (0x6141 bytes) 2024-09-14T22:37:09.037 INFO:teuthology.orchestra.run.smithi001.stdout:2872 write 0x88ce66d thru 0x88d6999 (0x832d bytes) 2024-09-14T22:37:09.052 INFO:teuthology.orchestra.run.smithi001.stdout:2873 punch from 0x2a0f846 to 0x2a15b32, (0x62ec bytes) 2024-09-14T22:37:09.052 INFO:teuthology.orchestra.run.smithi001.stdout:2874 read 0x694a557 thru 0x6954ef0 (0xa99a bytes) 2024-09-14T22:37:09.053 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:09.054 INFO:teuthology.orchestra.run.smithi001.stdout:2876 writesame 0x8cc12b4 thru 0x8cc96e3 data_size 0x48(0x8430 bytes) 2024-09-14T22:37:09.072 INFO:teuthology.orchestra.run.smithi001.stdout:2879 punch from 0x7eea130 to 0x7eef326, (0x51f6 bytes) 2024-09-14T22:37:09.072 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:09.072 INFO:teuthology.orchestra.run.smithi001.stdout:2882 write 0x126a82d thru 0x126cb60 (0x2334 bytes) 2024-09-14T22:37:09.077 INFO:teuthology.orchestra.run.smithi001.stdout:2883 punch from 0x9d24c2a to 0x9d3168e, (0xca64 bytes) 2024-09-14T22:37:09.077 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:09.077 INFO:teuthology.orchestra.run.smithi001.stdout:2885 punch from 0x3df8a01 to 0x3e05e4b, (0xd44a bytes) 2024-09-14T22:37:09.084 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:09.084 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:09.085 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:37:09.085 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:37:09.085 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:09.099 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:37:09.100 INFO:teuthology.orchestra.run.smithi001.stdout:2886 writesame 0xe89c20b thru 0xe8a7a5b data_size 0x1(0xb851 bytes) 2024-09-14T22:37:09.109 INFO:teuthology.orchestra.run.smithi001.stdout:2887 trunc from 0xe8a7a5c to 0x9cef91b 2024-09-14T22:37:09.128 INFO:teuthology.orchestra.run.smithi001.stdout:2888 read 0x495c300 thru 0x49622f7 (0x5ff8 bytes) 2024-09-14T22:37:09.128 INFO:teuthology.orchestra.run.smithi001.stdout:2889 writesame 0x3395bd thru 0x33bb96 data_size 0x13(0x25da bytes) 2024-09-14T22:37:09.133 INFO:teuthology.orchestra.run.smithi001.stdout:2890 punch from 0x4362cf8 to 0x43679bf, (0x4cc7 bytes) 2024-09-14T22:37:09.143 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:09.139+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:09.143 INFO:teuthology.orchestra.run.smithi001.stdout:2892 write 0xb246a5c thru 0xb24b6e6 (0x4c8b bytes) 2024-09-14T22:37:09.163 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:09.163 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:09.164 INFO:teuthology.orchestra.run.smithi001.stdout:2896 punch from 0x98a6211 to 0x98aa65a, (0x4449 bytes) 2024-09-14T22:37:09.164 INFO:teuthology.orchestra.run.smithi001.stdout:2898 read 0x32e4a56 thru 0x32e8b5c (0x4107 bytes) 2024-09-14T22:37:09.164 INFO:teuthology.orchestra.run.smithi001.stdout:2900 read 0x7bb014 thru 0x7c7dfe (0xcdeb bytes) 2024-09-14T22:37:09.164 INFO:teuthology.orchestra.run.smithi001.stdout:2901 write 0x43fc470 thru 0x4401797 (0x5328 bytes) 2024-09-14T22:37:09.187 INFO:teuthology.orchestra.run.smithi001.stdout:2903 punch from 0x81649a0 to 0x816723d, (0x289d bytes) 2024-09-14T22:37:09.187 INFO:teuthology.orchestra.run.smithi001.stdout:2904 punch from 0x9ad882 to 0x9b7506, (0x9c84 bytes) 2024-09-14T22:37:09.187 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:09.187 INFO:teuthology.orchestra.run.smithi001.stdout:2906 read 0x34eb40e thru 0x34f3091 (0x7c84 bytes) 2024-09-14T22:37:09.201 INFO:teuthology.orchestra.run.smithi001.stdout:2910 writesame 0xcc08b50 thru 0xcc10734 data_size 0x17(0x7be5 bytes) 2024-09-14T22:37:09.217 INFO:teuthology.orchestra.run.smithi001.stdout:2911 writesame 0xeb67057 thru 0xeb74040 data_size 0x9(0xcfea bytes) 2024-09-14T22:37:09.217 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:09.211+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:09.220 INFO:teuthology.orchestra.run.smithi001.stdout:2914 trunc from 0xeb74041 to 0xdac4aaa 2024-09-14T22:37:09.232 INFO:teuthology.orchestra.run.smithi001.stdout:2915 write 0x4eb2d6b thru 0x4ebb4ec (0x8782 bytes) 2024-09-14T22:37:09.233 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:09.233 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:09.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:09.233 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:09.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.233 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:72252 3.2 3:442c6d72:::rbd_data.113cb4c83968.0000000000000015:head [zero 4456448~393216] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-14T22:37:09.233 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:37:09.233 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.527561921, 2024-09-14T22:37:09.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0176591500000001, 2024-09-14T22:37:09.234 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.234 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.234 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T22:37:09.234 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.234 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.234 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.234 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 72252 2024-09-14T22:37:09.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.234 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:37:09.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:37:09.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700101+0000", 2024-09-14T22:37:09.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4570000000000001e-06 2024-09-14T22:37:09.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700114+0000", 2024-09-14T22:37:09.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2799999999999999e-05 2024-09-14T22:37:09.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700146+0000", 2024-09-14T22:37:09.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2116000000000002e-05 2024-09-14T22:37:09.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700170+0000", 2024-09-14T22:37:09.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3770000000000001e-05 2024-09-14T22:37:09.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700285+0000", 2024-09-14T22:37:09.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000115228 2024-09-14T22:37:09.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op must wait for map", 2024-09-14T22:37:09.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700297+0000", 2024-09-14T22:37:09.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1681999999999999e-05 2024-09-14T22:37:09.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.703312+0000", 2024-09-14T22:37:09.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003015046 2024-09-14T22:37:09.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:37:09.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.703327+0000", 2024-09-14T22:37:09.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5481999999999998e-05 2024-09-14T22:37:09.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:51.717566+0000", 2024-09-14T22:37:09.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0142383509999999 2024-09-14T22:37:09.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:51.717757+0000", 2024-09-14T22:37:09.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000191218 2024-09-14T22:37:09.239 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.239 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.239 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.239 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:74156 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T22:37:09.239 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:37:09.239 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 13.713093211, 2024-09-14T22:37:09.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062073521, 2024-09-14T22:37:09.239 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.239 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.240 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:09.240 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.240 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.240 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.240 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74156 2024-09-14T22:37:09.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.240 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:37:09.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:37:09.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514569+0000", 2024-09-14T22:37:09.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0229999999999999e-06 2024-09-14T22:37:09.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514596+0000", 2024-09-14T22:37:09.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6903e-05 2024-09-14T22:37:09.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514623+0000", 2024-09-14T22:37:09.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7206e-05 2024-09-14T22:37:09.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514637+0000", 2024-09-14T22:37:09.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:37:09.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514781+0000", 2024-09-14T22:37:09.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014394899999999999 2024-09-14T22:37:09.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.515005+0000", 2024-09-14T22:37:09.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022458499999999999 2024-09-14T22:37:09.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:37:09.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.527785+0000", 2024-09-14T22:37:09.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012780112 2024-09-14T22:37:09.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:09.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576209+0000", 2024-09-14T22:37:09.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.048423992999999999 2024-09-14T22:37:09.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:09.244 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576346+0000", 2024-09-14T22:37:09.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000136909 2024-09-14T22:37:09.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:09.244 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576517+0000", 2024-09-14T22:37:09.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000170848 2024-09-14T22:37:09.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.244 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576640+0000", 2024-09-14T22:37:09.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000122953 2024-09-14T22:37:09.245 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.245 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.245 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.245 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82513 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 557056~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:09.245 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:37:09.245 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.8084613919999999, 2024-09-14T22:37:09.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059345486000000003, 2024-09-14T22:37:09.245 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.245 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.245 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.245 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.246 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.246 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.246 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82513 2024-09-14T22:37:09.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.246 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:37:09.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:37:09.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419200+0000", 2024-09-14T22:37:09.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.184e-06 2024-09-14T22:37:09.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419207+0000", 2024-09-14T22:37:09.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2690000000000002e-06 2024-09-14T22:37:09.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419224+0000", 2024-09-14T22:37:09.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6623e-05 2024-09-14T22:37:09.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.248 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.248 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419236+0000", 2024-09-14T22:37:09.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2367e-05 2024-09-14T22:37:09.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.248 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.248 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478294+0000", 2024-09-14T22:37:09.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059057927000000003 2024-09-14T22:37:09.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.248 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.248 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478402+0000", 2024-09-14T22:37:09.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107921 2024-09-14T22:37:09.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.249 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.249 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478544+0000", 2024-09-14T22:37:09.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014219499999999999 2024-09-14T22:37:09.249 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.249 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.249 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.249 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82514 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 704512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:09.249 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:37:09.249 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.808402289, 2024-09-14T22:37:09.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059580793, 2024-09-14T22:37:09.250 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.250 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.250 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.250 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.250 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.250 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.250 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82514 2024-09-14T22:37:09.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.250 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:37:09.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:37:09.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419259+0000", 2024-09-14T22:37:09.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.353e-06 2024-09-14T22:37:09.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419276+0000", 2024-09-14T22:37:09.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7567e-05 2024-09-14T22:37:09.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419294+0000", 2024-09-14T22:37:09.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.713e-05 2024-09-14T22:37:09.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419306+0000", 2024-09-14T22:37:09.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.279e-05 2024-09-14T22:37:09.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478589+0000", 2024-09-14T22:37:09.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059282978 2024-09-14T22:37:09.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478694+0000", 2024-09-14T22:37:09.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000104561 2024-09-14T22:37:09.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478838+0000", 2024-09-14T22:37:09.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014441399999999999 2024-09-14T22:37:09.253 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.253 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.253 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.254 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82515 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:09.254 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:37:09.254 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.8083371390000007, 2024-09-14T22:37:09.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059815123999999997, 2024-09-14T22:37:09.254 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.254 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.254 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.254 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.254 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.254 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.254 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82515 2024-09-14T22:37:09.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.255 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:37:09.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:37:09.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419324+0000", 2024-09-14T22:37:09.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2130000000000001e-06 2024-09-14T22:37:09.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419331+0000", 2024-09-14T22:37:09.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4730000000000003e-06 2024-09-14T22:37:09.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419348+0000", 2024-09-14T22:37:09.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6269999999999998e-05 2024-09-14T22:37:09.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419360+0000", 2024-09-14T22:37:09.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2243e-05 2024-09-14T22:37:09.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478884+0000", 2024-09-14T22:37:09.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059524369000000001 2024-09-14T22:37:09.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478990+0000", 2024-09-14T22:37:09.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010569 2024-09-14T22:37:09.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479138+0000", 2024-09-14T22:37:09.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000147866 2024-09-14T22:37:09.258 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.258 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.258 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.258 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82516 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 999424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:09.258 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:37:09.258 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.8082846339999996, 2024-09-14T22:37:09.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061979595999999998, 2024-09-14T22:37:09.258 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.258 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.259 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.259 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.259 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.259 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.259 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82516 2024-09-14T22:37:09.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.259 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:37:09.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:37:09.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419376+0000", 2024-09-14T22:37:09.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.237e-06 2024-09-14T22:37:09.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419383+0000", 2024-09-14T22:37:09.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8170000000000001e-06 2024-09-14T22:37:09.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419400+0000", 2024-09-14T22:37:09.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6636000000000001e-05 2024-09-14T22:37:09.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.261 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419412+0000", 2024-09-14T22:37:09.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2357e-05 2024-09-14T22:37:09.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.261 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481080+0000", 2024-09-14T22:37:09.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061668032999999997 2024-09-14T22:37:09.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481193+0000", 2024-09-14T22:37:09.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112335 2024-09-14T22:37:09.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481355+0000", 2024-09-14T22:37:09.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016218100000000001 2024-09-14T22:37:09.262 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.262 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.262 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.262 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82538 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1097728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:09.263 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:37:09.263 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.8082325640000008, 2024-09-14T22:37:09.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059971705, 2024-09-14T22:37:09.263 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.263 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.263 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.263 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.263 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.263 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.263 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82538 2024-09-14T22:37:09.263 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.263 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.263 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:37:09.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:37:09.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419429+0000", 2024-09-14T22:37:09.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.497e-06 2024-09-14T22:37:09.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419436+0000", 2024-09-14T22:37:09.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2099999999999996e-06 2024-09-14T22:37:09.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419453+0000", 2024-09-14T22:37:09.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6840000000000001e-05 2024-09-14T22:37:09.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419465+0000", 2024-09-14T22:37:09.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2283000000000001e-05 2024-09-14T22:37:09.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479182+0000", 2024-09-14T22:37:09.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059717339000000001 2024-09-14T22:37:09.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479297+0000", 2024-09-14T22:37:09.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000114259 2024-09-14T22:37:09.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479399+0000", 2024-09-14T22:37:09.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102277 2024-09-14T22:37:09.267 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.267 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.267 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.267 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82539 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:09.267 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:37:09.267 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.8081797139999995, 2024-09-14T22:37:09.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060166935999999997, 2024-09-14T22:37:09.268 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.268 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.268 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.268 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.268 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.268 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.268 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82539 2024-09-14T22:37:09.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.268 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.268 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.268 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:37:09.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:37:09.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419482+0000", 2024-09-14T22:37:09.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4130000000000001e-06 2024-09-14T22:37:09.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419488+0000", 2024-09-14T22:37:09.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8460000000000002e-06 2024-09-14T22:37:09.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419505+0000", 2024-09-14T22:37:09.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6477000000000002e-05 2024-09-14T22:37:09.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419518+0000", 2024-09-14T22:37:09.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2797e-05 2024-09-14T22:37:09.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479438+0000", 2024-09-14T22:37:09.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059919897 2024-09-14T22:37:09.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479547+0000", 2024-09-14T22:37:09.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109807 2024-09-14T22:37:09.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.272 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479647+0000", 2024-09-14T22:37:09.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9698999999999995e-05 2024-09-14T22:37:09.272 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.272 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.272 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.272 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.272 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82542 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:09.272 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:37:09.272 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.808127271, 2024-09-14T22:37:09.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060367430999999999, 2024-09-14T22:37:09.272 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.272 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.272 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.273 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.273 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.273 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.273 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82542 2024-09-14T22:37:09.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.273 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:37:09.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:37:09.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.274 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419534+0000", 2024-09-14T22:37:09.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T22:37:09.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.274 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419542+0000", 2024-09-14T22:37:09.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8199999999999997e-06 2024-09-14T22:37:09.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419559+0000", 2024-09-14T22:37:09.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6852999999999999e-05 2024-09-14T22:37:09.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419571+0000", 2024-09-14T22:37:09.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2364000000000001e-05 2024-09-14T22:37:09.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479686+0000", 2024-09-14T22:37:09.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060115522999999997 2024-09-14T22:37:09.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479795+0000", 2024-09-14T22:37:09.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010831399999999999 2024-09-14T22:37:09.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479900+0000", 2024-09-14T22:37:09.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105284 2024-09-14T22:37:09.276 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.276 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.276 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.276 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82544 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:09.276 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:37:09.277 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.8080735350000001, 2024-09-14T22:37:09.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060590515999999997, 2024-09-14T22:37:09.277 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.277 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.277 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.277 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.277 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.277 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.277 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82544 2024-09-14T22:37:09.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.277 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:37:09.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:37:09.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.280 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419588+0000", 2024-09-14T22:37:09.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.35e-06 2024-09-14T22:37:09.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.280 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.280 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419595+0000", 2024-09-14T22:37:09.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4030000000000001e-06 2024-09-14T22:37:09.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.280 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.280 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419612+0000", 2024-09-14T22:37:09.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6670000000000001e-05 2024-09-14T22:37:09.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419624+0000", 2024-09-14T22:37:09.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-05 2024-09-14T22:37:09.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479957+0000", 2024-09-14T22:37:09.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060333287999999999 2024-09-14T22:37:09.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480074+0000", 2024-09-14T22:37:09.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116443 2024-09-14T22:37:09.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480177+0000", 2024-09-14T22:37:09.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000103102 2024-09-14T22:37:09.282 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.282 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.282 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.282 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82546 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:09.282 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:37:09.282 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.8080206160000003, 2024-09-14T22:37:09.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060795428999999998, 2024-09-14T22:37:09.283 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.283 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.283 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.283 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.283 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.283 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.283 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82546 2024-09-14T22:37:09.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.283 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:37:09.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:37:09.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419640+0000", 2024-09-14T22:37:09.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.254e-06 2024-09-14T22:37:09.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419647+0000", 2024-09-14T22:37:09.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9e-06 2024-09-14T22:37:09.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.285 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.285 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419664+0000", 2024-09-14T22:37:09.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6345999999999999e-05 2024-09-14T22:37:09.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.285 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.285 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419676+0000", 2024-09-14T22:37:09.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.224e-05 2024-09-14T22:37:09.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480218+0000", 2024-09-14T22:37:09.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060542133999999997 2024-09-14T22:37:09.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480336+0000", 2024-09-14T22:37:09.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117472 2024-09-14T22:37:09.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480435+0000", 2024-09-14T22:37:09.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9083e-05 2024-09-14T22:37:09.286 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.287 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.287 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.287 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82548 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:09.287 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:37:09.287 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.8079686059999993, 2024-09-14T22:37:09.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061011636000000001, 2024-09-14T22:37:09.287 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.287 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.287 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.287 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.287 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.287 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.288 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82548 2024-09-14T22:37:09.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.288 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:37:09.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:37:09.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419693+0000", 2024-09-14T22:37:09.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.387e-06 2024-09-14T22:37:09.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419699+0000", 2024-09-14T22:37:09.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8730000000000001e-06 2024-09-14T22:37:09.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419716+0000", 2024-09-14T22:37:09.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6416e-05 2024-09-14T22:37:09.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419728+0000", 2024-09-14T22:37:09.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2597e-05 2024-09-14T22:37:09.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480476+0000", 2024-09-14T22:37:09.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060747989000000002 2024-09-14T22:37:09.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480583+0000", 2024-09-14T22:37:09.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010638500000000001 2024-09-14T22:37:09.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480703+0000", 2024-09-14T22:37:09.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011998900000000001 2024-09-14T22:37:09.291 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.291 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.291 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.291 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82550 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1982464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:09.291 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:37:09.291 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.8079158530000008, 2024-09-14T22:37:09.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061295483999999997, 2024-09-14T22:37:09.291 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.291 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.291 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.292 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.292 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.292 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.292 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82550 2024-09-14T22:37:09.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.292 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:37:09.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:37:09.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419745+0000", 2024-09-14T22:37:09.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.406e-06 2024-09-14T22:37:09.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419753+0000", 2024-09-14T22:37:09.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8140000000000001e-06 2024-09-14T22:37:09.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419770+0000", 2024-09-14T22:37:09.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6719999999999999e-05 2024-09-14T22:37:09.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419782+0000", 2024-09-14T22:37:09.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2252999999999999e-05 2024-09-14T22:37:09.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480745+0000", 2024-09-14T22:37:09.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060962385000000001 2024-09-14T22:37:09.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480900+0000", 2024-09-14T22:37:09.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015513899999999999 2024-09-14T22:37:09.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481039+0000", 2024-09-14T22:37:09.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013976700000000001 2024-09-14T22:37:09.295 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.295 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.295 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.296 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82588 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1064960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:09.296 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:37:09.296 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.8078619630000006, 2024-09-14T22:37:09.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061863732999999997, 2024-09-14T22:37:09.296 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.296 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.296 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.296 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.296 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.296 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.296 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82588 2024-09-14T22:37:09.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.296 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:37:09.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:37:09.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419799+0000", 2024-09-14T22:37:09.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4160000000000001e-06 2024-09-14T22:37:09.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419807+0000", 2024-09-14T22:37:09.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2169999999999994e-06 2024-09-14T22:37:09.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419824+0000", 2024-09-14T22:37:09.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6776e-05 2024-09-14T22:37:09.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419837+0000", 2024-09-14T22:37:09.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2680000000000001e-05 2024-09-14T22:37:09.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.299 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481395+0000", 2024-09-14T22:37:09.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061558416999999997 2024-09-14T22:37:09.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.299 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481532+0000", 2024-09-14T22:37:09.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000136861 2024-09-14T22:37:09.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.299 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481662+0000", 2024-09-14T22:37:09.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012936600000000001 2024-09-14T22:37:09.300 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.300 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.300 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.300 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82589 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:09.300 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:37:09.300 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.8078073680000006, 2024-09-14T22:37:09.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062064592000000002, 2024-09-14T22:37:09.300 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.300 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.300 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.300 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.301 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.301 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.301 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82589 2024-09-14T22:37:09.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.301 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:37:09.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:37:09.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419854+0000", 2024-09-14T22:37:09.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.367e-06 2024-09-14T22:37:09.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419862+0000", 2024-09-14T22:37:09.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8930000000000005e-06 2024-09-14T22:37:09.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419896+0000", 2024-09-14T22:37:09.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4134000000000002e-05 2024-09-14T22:37:09.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.303 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.303 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419909+0000", 2024-09-14T22:37:09.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3332999999999999e-05 2024-09-14T22:37:09.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.303 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.303 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481702+0000", 2024-09-14T22:37:09.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061792947000000001 2024-09-14T22:37:09.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.303 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.303 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481808+0000", 2024-09-14T22:37:09.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105655 2024-09-14T22:37:09.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.304 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481917+0000", 2024-09-14T22:37:09.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109263 2024-09-14T22:37:09.304 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.304 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.304 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.304 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82590 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1359872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:09.304 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:37:09.304 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.8077337609999997, 2024-09-14T22:37:09.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062362895000000002, 2024-09-14T22:37:09.305 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.305 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.305 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.305 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.305 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.305 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.305 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82590 2024-09-14T22:37:09.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.305 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:37:09.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:37:09.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419928+0000", 2024-09-14T22:37:09.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4470000000000001e-06 2024-09-14T22:37:09.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419942+0000", 2024-09-14T22:37:09.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4229e-05 2024-09-14T22:37:09.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419972+0000", 2024-09-14T22:37:09.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.065e-05 2024-09-14T22:37:09.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419985+0000", 2024-09-14T22:37:09.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2903e-05 2024-09-14T22:37:09.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481960+0000", 2024-09-14T22:37:09.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061974435000000001 2024-09-14T22:37:09.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.308 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482106+0000", 2024-09-14T22:37:09.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014643800000000001 2024-09-14T22:37:09.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.308 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482289+0000", 2024-09-14T22:37:09.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000182793 2024-09-14T22:37:09.308 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.309 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.309 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.309 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82592 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:09.309 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:37:09.309 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.8076575390000009, 2024-09-14T22:37:09.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062631507000000003, 2024-09-14T22:37:09.309 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.309 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.309 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.309 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.310 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.310 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.310 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82592 2024-09-14T22:37:09.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.310 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:37:09.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:37:09.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420017+0000", 2024-09-14T22:37:09.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4744e-05 2024-09-14T22:37:09.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420036+0000", 2024-09-14T22:37:09.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8680000000000001e-05 2024-09-14T22:37:09.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.312 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420054+0000", 2024-09-14T22:37:09.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7799999999999999e-05 2024-09-14T22:37:09.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.312 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420067+0000", 2024-09-14T22:37:09.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3013000000000001e-05 2024-09-14T22:37:09.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.312 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482358+0000", 2024-09-14T22:37:09.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062291445000000001 2024-09-14T22:37:09.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.313 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482469+0000", 2024-09-14T22:37:09.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110608 2024-09-14T22:37:09.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.313 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482634+0000", 2024-09-14T22:37:09.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000165217 2024-09-14T22:37:09.313 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.314 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.314 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.314 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82594 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1654784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:09.314 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:37:09.314 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.8075595629999999, 2024-09-14T22:37:09.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062893849000000002, 2024-09-14T22:37:09.314 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.314 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.314 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.314 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.314 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.314 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.314 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82594 2024-09-14T22:37:09.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.315 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:37:09.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:37:09.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420102+0000", 2024-09-14T22:37:09.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5e-06 2024-09-14T22:37:09.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420115+0000", 2024-09-14T22:37:09.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.324e-05 2024-09-14T22:37:09.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420133+0000", 2024-09-14T22:37:09.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8213e-05 2024-09-14T22:37:09.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420147+0000", 2024-09-14T22:37:09.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3284e-05 2024-09-14T22:37:09.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482688+0000", 2024-09-14T22:37:09.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062541425999999997 2024-09-14T22:37:09.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482794+0000", 2024-09-14T22:37:09.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010575500000000001 2024-09-14T22:37:09.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482994+0000", 2024-09-14T22:37:09.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020043099999999999 2024-09-14T22:37:09.318 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.318 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.318 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.318 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82596 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:09.318 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:37:09.318 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.8074962630000009, 2024-09-14T22:37:09.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063187238000000007, 2024-09-14T22:37:09.318 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.318 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.318 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.319 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.319 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.319 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.319 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82596 2024-09-14T22:37:09.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.319 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:37:09.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:37:09.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420165+0000", 2024-09-14T22:37:09.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4100000000000001e-06 2024-09-14T22:37:09.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420173+0000", 2024-09-14T22:37:09.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8129999999999996e-06 2024-09-14T22:37:09.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420190+0000", 2024-09-14T22:37:09.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7076999999999999e-05 2024-09-14T22:37:09.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420203+0000", 2024-09-14T22:37:09.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2683e-05 2024-09-14T22:37:09.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483060+0000", 2024-09-14T22:37:09.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062857114000000006 2024-09-14T22:37:09.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483167+0000", 2024-09-14T22:37:09.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010713999999999999 2024-09-14T22:37:09.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483351+0000", 2024-09-14T22:37:09.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018400099999999999 2024-09-14T22:37:09.322 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.323 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.323 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.323 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82597 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1949696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:09.323 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:37:09.323 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.8074394369999993, 2024-09-14T22:37:09.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063466513000000002, 2024-09-14T22:37:09.323 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.323 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.323 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.323 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.324 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.324 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.324 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82597 2024-09-14T22:37:09.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.324 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:37:09.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:37:09.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420221+0000", 2024-09-14T22:37:09.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.043e-06 2024-09-14T22:37:09.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420229+0000", 2024-09-14T22:37:09.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0469999999999994e-06 2024-09-14T22:37:09.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420246+0000", 2024-09-14T22:37:09.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6339999999999999e-05 2024-09-14T22:37:09.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420258+0000", 2024-09-14T22:37:09.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2296000000000001e-05 2024-09-14T22:37:09.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483402+0000", 2024-09-14T22:37:09.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063144318000000005 2024-09-14T22:37:09.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483530+0000", 2024-09-14T22:37:09.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012713199999999999 2024-09-14T22:37:09.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483687+0000", 2024-09-14T22:37:09.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015733699999999999 2024-09-14T22:37:09.327 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.327 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.328 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.328 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.328 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.328 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:09.329 INFO:teuthology.orchestra.run.smithi001.stdout:2916 trunc from 0xdac4aaa to 0x577db3 2024-09-14T22:37:09.329 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:37:09.330 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:09.323+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:09.377 INFO:teuthology.orchestra.run.smithi001.stdout:2917 read 0x54e9b0 thru 0x5572f6 (0x8947 bytes) 2024-09-14T22:37:09.377 INFO:teuthology.orchestra.run.smithi001.stdout:2919 read 0x466644 thru 0x46b867 (0x5224 bytes) 2024-09-14T22:37:09.378 INFO:teuthology.orchestra.run.smithi001.stdout:2920 punch from 0x459de0 to 0x461134, (0x7354 bytes) 2024-09-14T22:37:09.408 INFO:teuthology.orchestra.run.smithi001.stdout:2921 trunc from 0x577db3 to 0xe5584fe 2024-09-14T22:37:09.419 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:09.419 INFO:teuthology.orchestra.run.smithi001.stdout:2923 read 0x445f4fb thru 0x44642e9 (0x4def bytes) 2024-09-14T22:37:09.420 INFO:teuthology.orchestra.run.smithi001.stdout:2924 write 0x368ada1 thru 0x36987c1 (0xda21 bytes) 2024-09-14T22:37:09.430 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:09.423+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:09.447 INFO:teuthology.orchestra.run.smithi001.stdout:2925 trunc from 0xe5584fe to 0xa725c93 2024-09-14T22:37:09.458 INFO:teuthology.orchestra.run.smithi001.stdout:2926 read 0x2027762 thru 0x2029b07 (0x23a6 bytes) 2024-09-14T22:37:09.473 INFO:teuthology.orchestra.run.smithi001.stdout:2927 write 0xc85e35f thru 0xc86a9ce (0xc670 bytes) 2024-09-14T22:37:09.495 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:09.495 INFO:teuthology.orchestra.run.smithi001.stdout:2929 read 0x237e145 thru 0x23850ef (0x6fab bytes) 2024-09-14T22:37:09.495 INFO:teuthology.orchestra.run.smithi001.stdout:2930 write 0x657a35b thru 0x6586743 (0xc3e9 bytes) 2024-09-14T22:37:09.503 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:09.503 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:09.503 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:37:09.503 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:09.519 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:37:09.520 INFO:teuthology.orchestra.run.smithi001.stdout:2931 writesame 0x7bf6b8c thru 0x7bfa59b data_size 0x8(0x3a10 bytes) 2024-09-14T22:37:09.531 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:09.527+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:09.546 INFO:teuthology.orchestra.run.smithi001.stdout:2932 write 0xdd0312 thru 0xddd26c (0xcf5b bytes) 2024-09-14T22:37:09.570 INFO:teuthology.orchestra.run.smithi001.stdout:2933 write 0xa1595a1 thru 0xa1598f8 (0x358 bytes) 2024-09-14T22:37:09.577 INFO:teuthology.orchestra.run.smithi001.stdout:2934 write 0x8edff92 thru 0x8ee1a32 (0x1aa1 bytes) 2024-09-14T22:37:09.600 INFO:teuthology.orchestra.run.smithi001.stdout:2935 write 0x4aa0f8b thru 0x4aae652 (0xd6c8 bytes) 2024-09-14T22:37:09.620 INFO:teuthology.orchestra.run.smithi001.stdout:2936 read 0xc956ff thru 0xca36f4 (0xdff6 bytes) 2024-09-14T22:37:09.620 INFO:teuthology.orchestra.run.smithi001.stdout:2937 read 0x46e42c3 thru 0x46eb614 (0x7352 bytes) 2024-09-14T22:37:09.621 INFO:teuthology.orchestra.run.smithi001.stdout:2939 read 0x687258d thru 0x687a1ca (0x7c3e bytes) 2024-09-14T22:37:09.621 INFO:teuthology.orchestra.run.smithi001.stdout:2941 write 0xaaa8e5d thru 0xaab500e (0xc1b2 bytes) 2024-09-14T22:37:09.632 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:09.627+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:09.642 INFO:teuthology.orchestra.run.smithi001.stdout:2942 read 0xbef8265 thru 0xbefeb53 (0x68ef bytes) 2024-09-14T22:37:09.642 INFO:teuthology.orchestra.run.smithi001.stdout:2944 read 0xc746db thru 0xc7a2d3 (0x5bf9 bytes) 2024-09-14T22:37:09.642 INFO:teuthology.orchestra.run.smithi001.stdout:2948 punch from 0x40ae505 to 0x40b2f4a, (0x4a45 bytes) 2024-09-14T22:37:09.642 INFO:teuthology.orchestra.run.smithi001.stdout:2949 write 0xc6f23f6 thru 0xc6f753a (0x5145 bytes) 2024-09-14T22:37:09.665 INFO:teuthology.orchestra.run.smithi001.stdout:2951 read 0x566584f thru 0x566b720 (0x5ed2 bytes) 2024-09-14T22:37:09.665 INFO:teuthology.orchestra.run.smithi001.stdout:2952 write 0x7cdbba6 thru 0x7ce2764 (0x6bbf bytes) 2024-09-14T22:37:09.669 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:09.670 INFO:teuthology.orchestra.run.smithi001.stdout:2954 writesame 0x7eccf48 thru 0x7ecfc9a data_size 0x1(0x2d53 bytes) 2024-09-14T22:37:09.697 INFO:teuthology.orchestra.run.smithi001.stdout:2955 writesame 0xd38e2eb thru 0xd39abb0 data_size 0x1f(0xc8c6 bytes) 2024-09-14T22:37:09.699 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:09.699 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:09.699 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:37:09.699 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:37:09.699 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:09.710 INFO:teuthology.orchestra.run.smithi001.stdout:2957 trunc from 0xd39abb1 to 0x6d942d8 2024-09-14T22:37:09.713 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:37:09.733 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:09.731+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:09.754 INFO:teuthology.orchestra.run.smithi001.stdout:2958 trunc from 0x6d942d8 to 0xd5ddd35 2024-09-14T22:37:09.784 INFO:teuthology.orchestra.run.smithi001.stdout:2959 trunc from 0xd5ddd35 to 0x4e3307e 2024-09-14T22:37:09.803 INFO:teuthology.orchestra.run.smithi001.stdout:2960 punch from 0x4481488 to 0x4489dc2, (0x893a bytes) 2024-09-14T22:37:09.817 INFO:teuthology.orchestra.run.smithi001.stdout:2961 writesame 0x626815f thru 0x626f001 data_size 0x9(0x6ea3 bytes) 2024-09-14T22:37:09.834 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:09.831+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:09.846 INFO:teuthology.orchestra.run.smithi001.stdout:2962 write 0xcb4ddfe thru 0xcb4f284 (0x1487 bytes) 2024-09-14T22:37:09.867 INFO:teuthology.orchestra.run.smithi001.stdout:2963 trunc from 0xcb4f285 to 0xdef7896 2024-09-14T22:37:09.888 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:09.888 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:09.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:09.888 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:09.888 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.888 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:37:09.888 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:37:09.888 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 69.876312439000003, 2024-09-14T22:37:09.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.398029152, 2024-09-14T22:37:09.889 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.889 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.889 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:09.889 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.889 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:37:09.889 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:37:09.889 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 1 2024-09-14T22:37:09.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.889 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:37:09.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:37:09.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006183+0000", 2024-09-14T22:37:09.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9059999999999998e-06 2024-09-14T22:37:09.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006198+0000", 2024-09-14T22:37:09.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.523e-05 2024-09-14T22:37:09.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006219+0000", 2024-09-14T22:37:09.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1342999999999999e-05 2024-09-14T22:37:09.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.892 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006240+0000", 2024-09-14T22:37:09.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1186999999999999e-05 2024-09-14T22:37:09.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.892 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.892 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006363+0000", 2024-09-14T22:37:09.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012258200000000001 2024-09-14T22:37:09.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.892 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:37:09.892 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006392+0000", 2024-09-14T22:37:09.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9607e-05 2024-09-14T22:37:09.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401084+0000", 2024-09-14T22:37:09.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.39469142699999998 2024-09-14T22:37:09.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401394+0000", 2024-09-14T22:37:09.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030987799999999998 2024-09-14T22:37:09.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:37:09.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401860+0000", 2024-09-14T22:37:09.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00046654299999999998 2024-09-14T22:37:09.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.894 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:09.894 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.403386+0000", 2024-09-14T22:37:09.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001525322 2024-09-14T22:37:09.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.894 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:09.894 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404042+0000", 2024-09-14T22:37:09.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065676099999999996 2024-09-14T22:37:09.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.894 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:09.894 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404136+0000", 2024-09-14T22:37:09.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4032999999999999e-05 2024-09-14T22:37:09.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.895 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404209+0000", 2024-09-14T22:37:09.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2333e-05 2024-09-14T22:37:09.895 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.895 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.895 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.895 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2079 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11401216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:09.895 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450429+0000", 2024-09-14T22:37:09.895 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.432062651000003, 2024-09-14T22:37:09.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055279107000000001, 2024-09-14T22:37:09.896 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.896 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.896 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.896 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.896 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.896 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.896 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2079 2024-09-14T22:37:09.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.896 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.896 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.896 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450429+0000", 2024-09-14T22:37:09.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450429+0000", 2024-09-14T22:37:09.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450431+0000", 2024-09-14T22:37:09.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1039999999999999e-06 2024-09-14T22:37:09.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450435+0000", 2024-09-14T22:37:09.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9300000000000002e-06 2024-09-14T22:37:09.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450455+0000", 2024-09-14T22:37:09.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9242999999999999e-05 2024-09-14T22:37:09.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450470+0000", 2024-09-14T22:37:09.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5523e-05 2024-09-14T22:37:09.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.505413+0000", 2024-09-14T22:37:09.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054943193000000001 2024-09-14T22:37:09.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.899 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.899 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.505538+0000", 2024-09-14T22:37:09.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012467199999999999 2024-09-14T22:37:09.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.899 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.899 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.505709+0000", 2024-09-14T22:37:09.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017044200000000001 2024-09-14T22:37:09.899 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.899 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.899 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.899 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2080 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11409408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:09.899 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450489+0000", 2024-09-14T22:37:09.900 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.432002904000001, 2024-09-14T22:37:09.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055572178, 2024-09-14T22:37:09.900 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.900 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.900 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.900 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.900 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.900 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.900 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2080 2024-09-14T22:37:09.900 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.900 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.900 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.900 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450489+0000", 2024-09-14T22:37:09.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.901 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.901 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.901 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.901 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450489+0000", 2024-09-14T22:37:09.901 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.901 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.901 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.901 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.901 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450490+0000", 2024-09-14T22:37:09.901 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.023e-06 2024-09-14T22:37:09.901 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.901 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.901 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.901 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450495+0000", 2024-09-14T22:37:09.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1399999999999999e-06 2024-09-14T22:37:09.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450512+0000", 2024-09-14T22:37:09.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6892999999999999e-05 2024-09-14T22:37:09.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450525+0000", 2024-09-14T22:37:09.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2877000000000001e-05 2024-09-14T22:37:09.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.505787+0000", 2024-09-14T22:37:09.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055261432999999999 2024-09-14T22:37:09.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.505899+0000", 2024-09-14T22:37:09.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112456 2024-09-14T22:37:09.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506061+0000", 2024-09-14T22:37:09.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000162356 2024-09-14T22:37:09.903 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.903 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.904 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.904 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2081 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11417600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:09.904 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450544+0000", 2024-09-14T22:37:09.904 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.431947585000003, 2024-09-14T22:37:09.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.05598185, 2024-09-14T22:37:09.904 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.904 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.904 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.904 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.904 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.904 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.904 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2081 2024-09-14T22:37:09.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.905 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.905 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.905 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450544+0000", 2024-09-14T22:37:09.905 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.905 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.905 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450544+0000", 2024-09-14T22:37:09.905 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.906 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.906 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450546+0000", 2024-09-14T22:37:09.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5370000000000001e-06 2024-09-14T22:37:09.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.906 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.906 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450553+0000", 2024-09-14T22:37:09.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8630000000000002e-06 2024-09-14T22:37:09.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.906 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.906 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450571+0000", 2024-09-14T22:37:09.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7657000000000001e-05 2024-09-14T22:37:09.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450585+0000", 2024-09-14T22:37:09.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4056000000000001e-05 2024-09-14T22:37:09.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506185+0000", 2024-09-14T22:37:09.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055599920999999997 2024-09-14T22:37:09.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506334+0000", 2024-09-14T22:37:09.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014919900000000001 2024-09-14T22:37:09.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506526+0000", 2024-09-14T22:37:09.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019261699999999999 2024-09-14T22:37:09.908 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.908 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.908 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.909 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2082 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11425792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:09.909 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450602+0000", 2024-09-14T22:37:09.909 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.431889857999998, 2024-09-14T22:37:09.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056238654999999999, 2024-09-14T22:37:09.909 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.909 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.909 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.909 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.909 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.909 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.910 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2082 2024-09-14T22:37:09.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.910 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.910 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450602+0000", 2024-09-14T22:37:09.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.910 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450602+0000", 2024-09-14T22:37:09.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450603+0000", 2024-09-14T22:37:09.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1060000000000001e-06 2024-09-14T22:37:09.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450608+0000", 2024-09-14T22:37:09.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1499999999999998e-06 2024-09-14T22:37:09.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450626+0000", 2024-09-14T22:37:09.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7484000000000001e-05 2024-09-14T22:37:09.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450639+0000", 2024-09-14T22:37:09.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3485999999999999e-05 2024-09-14T22:37:09.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506599+0000", 2024-09-14T22:37:09.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055959395000000002 2024-09-14T22:37:09.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506706+0000", 2024-09-14T22:37:09.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010703499999999999 2024-09-14T22:37:09.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506841+0000", 2024-09-14T22:37:09.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013499900000000001 2024-09-14T22:37:09.913 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.914 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.914 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.914 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2083 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11433984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:09.914 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450658+0000", 2024-09-14T22:37:09.914 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.431834432000002, 2024-09-14T22:37:09.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056459267, 2024-09-14T22:37:09.914 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.914 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.914 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.914 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.915 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.915 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.915 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2083 2024-09-14T22:37:09.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.915 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450658+0000", 2024-09-14T22:37:09.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450658+0000", 2024-09-14T22:37:09.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450659+0000", 2024-09-14T22:37:09.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T22:37:09.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450664+0000", 2024-09-14T22:37:09.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4700000000000001e-06 2024-09-14T22:37:09.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450684+0000", 2024-09-14T22:37:09.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.916e-05 2024-09-14T22:37:09.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450698+0000", 2024-09-14T22:37:09.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4100000000000001e-05 2024-09-14T22:37:09.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506881+0000", 2024-09-14T22:37:09.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056183250999999997 2024-09-14T22:37:09.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.506982+0000", 2024-09-14T22:37:09.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010090900000000001 2024-09-14T22:37:09.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507117+0000", 2024-09-14T22:37:09.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013510400000000001 2024-09-14T22:37:09.918 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.918 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.918 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.919 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2084 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11442176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:09.919 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450716+0000", 2024-09-14T22:37:09.919 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.431776573, 2024-09-14T22:37:09.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056802328999999999, 2024-09-14T22:37:09.919 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.919 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.919 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.919 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.919 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.919 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.919 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2084 2024-09-14T22:37:09.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.920 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450716+0000", 2024-09-14T22:37:09.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450716+0000", 2024-09-14T22:37:09.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450717+0000", 2024-09-14T22:37:09.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.08e-06 2024-09-14T22:37:09.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450722+0000", 2024-09-14T22:37:09.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5099999999999998e-06 2024-09-14T22:37:09.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450740+0000", 2024-09-14T22:37:09.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7836999999999999e-05 2024-09-14T22:37:09.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450757+0000", 2024-09-14T22:37:09.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7513e-05 2024-09-14T22:37:09.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507224+0000", 2024-09-14T22:37:09.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056466954 2024-09-14T22:37:09.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507355+0000", 2024-09-14T22:37:09.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000130207 2024-09-14T22:37:09.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.923 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.923 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507518+0000", 2024-09-14T22:37:09.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016322799999999999 2024-09-14T22:37:09.923 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.923 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.923 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.923 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2085 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11450368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:09.923 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450774+0000", 2024-09-14T22:37:09.923 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.431717769999999, 2024-09-14T22:37:09.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057099952000000002, 2024-09-14T22:37:09.923 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.923 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.923 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.924 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.924 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.924 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.924 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2085 2024-09-14T22:37:09.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.924 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450774+0000", 2024-09-14T22:37:09.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450774+0000", 2024-09-14T22:37:09.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450775+0000", 2024-09-14T22:37:09.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1030000000000001e-06 2024-09-14T22:37:09.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450781+0000", 2024-09-14T22:37:09.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.22e-06 2024-09-14T22:37:09.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450798+0000", 2024-09-14T22:37:09.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7350000000000002e-05 2024-09-14T22:37:09.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450812+0000", 2024-09-14T22:37:09.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.358e-05 2024-09-14T22:37:09.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507576+0000", 2024-09-14T22:37:09.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056764468999999998 2024-09-14T22:37:09.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507714+0000", 2024-09-14T22:37:09.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013826800000000001 2024-09-14T22:37:09.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.927 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507874+0000", 2024-09-14T22:37:09.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015996200000000001 2024-09-14T22:37:09.927 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.927 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.927 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.927 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2086 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11458560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:09.927 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450830+0000", 2024-09-14T22:37:09.927 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.43166248, 2024-09-14T22:37:09.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057480495999999999, 2024-09-14T22:37:09.928 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.928 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.928 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.928 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.928 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.928 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.928 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2086 2024-09-14T22:37:09.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.928 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450830+0000", 2024-09-14T22:37:09.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450830+0000", 2024-09-14T22:37:09.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450831+0000", 2024-09-14T22:37:09.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0860000000000001e-06 2024-09-14T22:37:09.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450836+0000", 2024-09-14T22:37:09.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4999999999999999e-06 2024-09-14T22:37:09.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450853+0000", 2024-09-14T22:37:09.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7184e-05 2024-09-14T22:37:09.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450869+0000", 2024-09-14T22:37:09.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5965999999999999e-05 2024-09-14T22:37:09.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.507972+0000", 2024-09-14T22:37:09.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057103095999999999 2024-09-14T22:37:09.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508119+0000", 2024-09-14T22:37:09.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014651200000000001 2024-09-14T22:37:09.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508310+0000", 2024-09-14T22:37:09.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019115199999999999 2024-09-14T22:37:09.931 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.931 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.931 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.932 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2087 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11466752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:09.932 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450888+0000", 2024-09-14T22:37:09.932 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.431604434, 2024-09-14T22:37:09.932 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057792806000000002, 2024-09-14T22:37:09.932 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.932 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.932 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.932 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.932 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.932 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.932 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2087 2024-09-14T22:37:09.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.932 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.932 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450888+0000", 2024-09-14T22:37:09.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450888+0000", 2024-09-14T22:37:09.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450889+0000", 2024-09-14T22:37:09.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4270000000000001e-06 2024-09-14T22:37:09.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450895+0000", 2024-09-14T22:37:09.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4700000000000001e-06 2024-09-14T22:37:09.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450913+0000", 2024-09-14T22:37:09.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7958999999999999e-05 2024-09-14T22:37:09.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450926+0000", 2024-09-14T22:37:09.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3842999999999999e-05 2024-09-14T22:37:09.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508394+0000", 2024-09-14T22:37:09.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057467795000000002 2024-09-14T22:37:09.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508516+0000", 2024-09-14T22:37:09.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000121822 2024-09-14T22:37:09.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.936 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508680+0000", 2024-09-14T22:37:09.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016448999999999999 2024-09-14T22:37:09.936 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.936 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.936 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.936 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2088 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11474944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:09.936 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450943+0000", 2024-09-14T22:37:09.936 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.431548821, 2024-09-14T22:37:09.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058110368000000003, 2024-09-14T22:37:09.937 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.937 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.937 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.937 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.937 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.937 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.937 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2088 2024-09-14T22:37:09.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.937 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.937 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.937 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.937 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450943+0000", 2024-09-14T22:37:09.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.938 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.938 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450943+0000", 2024-09-14T22:37:09.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.938 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.938 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450944+0000", 2024-09-14T22:37:09.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2100000000000001e-06 2024-09-14T22:37:09.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450950+0000", 2024-09-14T22:37:09.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3329999999999996e-06 2024-09-14T22:37:09.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450968+0000", 2024-09-14T22:37:09.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7802999999999999e-05 2024-09-14T22:37:09.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450981+0000", 2024-09-14T22:37:09.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.349e-05 2024-09-14T22:37:09.940 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.940 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.940 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508752+0000", 2024-09-14T22:37:09.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057770957999999997 2024-09-14T22:37:09.940 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.940 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.940 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.508884+0000", 2024-09-14T22:37:09.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013186900000000001 2024-09-14T22:37:09.940 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.940 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.941 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509054+0000", 2024-09-14T22:37:09.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000169705 2024-09-14T22:37:09.941 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.941 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.941 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.941 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2089 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11483136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:09.941 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.450998+0000", 2024-09-14T22:37:09.941 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.431493928000002, 2024-09-14T22:37:09.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058422566000000002, 2024-09-14T22:37:09.941 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.941 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.942 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.942 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.942 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.942 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.942 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2089 2024-09-14T22:37:09.942 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.942 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.942 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.942 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.942 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450998+0000", 2024-09-14T22:37:09.942 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.942 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.942 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.942 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450998+0000", 2024-09-14T22:37:09.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.450999+0000", 2024-09-14T22:37:09.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0100000000000001e-06 2024-09-14T22:37:09.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451014+0000", 2024-09-14T22:37:09.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4486e-05 2024-09-14T22:37:09.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451032+0000", 2024-09-14T22:37:09.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7893e-05 2024-09-14T22:37:09.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451046+0000", 2024-09-14T22:37:09.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4783999999999999e-05 2024-09-14T22:37:09.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509131+0000", 2024-09-14T22:37:09.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058085070000000003 2024-09-14T22:37:09.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509253+0000", 2024-09-14T22:37:09.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012173299999999999 2024-09-14T22:37:09.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509421+0000", 2024-09-14T22:37:09.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016759000000000001 2024-09-14T22:37:09.945 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.945 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.945 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.946 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2090 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11491328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:09.946 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.451064+0000", 2024-09-14T22:37:09.946 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.431428152000002, 2024-09-14T22:37:09.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058723076999999999, 2024-09-14T22:37:09.946 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.946 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.946 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.946 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.946 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.946 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.946 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2090 2024-09-14T22:37:09.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.946 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451064+0000", 2024-09-14T22:37:09.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451064+0000", 2024-09-14T22:37:09.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451065+0000", 2024-09-14T22:37:09.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2160000000000001e-06 2024-09-14T22:37:09.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451070+0000", 2024-09-14T22:37:09.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1340000000000003e-06 2024-09-14T22:37:09.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451088+0000", 2024-09-14T22:37:09.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7476e-05 2024-09-14T22:37:09.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451101+0000", 2024-09-14T22:37:09.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3723999999999999e-05 2024-09-14T22:37:09.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509492+0000", 2024-09-14T22:37:09.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058390899000000003 2024-09-14T22:37:09.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509618+0000", 2024-09-14T22:37:09.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012593899999999999 2024-09-14T22:37:09.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509787+0000", 2024-09-14T22:37:09.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000168689 2024-09-14T22:37:09.950 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.950 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.950 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.950 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2091 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11499520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:09.950 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.451119+0000", 2024-09-14T22:37:09.950 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.431373098999998, 2024-09-14T22:37:09.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059045274000000002, 2024-09-14T22:37:09.950 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.950 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.950 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.950 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.951 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.951 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.951 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2091 2024-09-14T22:37:09.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.951 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451119+0000", 2024-09-14T22:37:09.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451119+0000", 2024-09-14T22:37:09.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451120+0000", 2024-09-14T22:37:09.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0669999999999999e-06 2024-09-14T22:37:09.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451125+0000", 2024-09-14T22:37:09.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3329999999999996e-06 2024-09-14T22:37:09.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.953 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451143+0000", 2024-09-14T22:37:09.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7309999999999999e-05 2024-09-14T22:37:09.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.953 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451157+0000", 2024-09-14T22:37:09.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3835999999999999e-05 2024-09-14T22:37:09.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.953 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509863+0000", 2024-09-14T22:37:09.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058706345 2024-09-14T22:37:09.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.509985+0000", 2024-09-14T22:37:09.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000121736 2024-09-14T22:37:09.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510164+0000", 2024-09-14T22:37:09.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017964700000000001 2024-09-14T22:37:09.954 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.954 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.954 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.954 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2092 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11507712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:09.954 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.451202+0000", 2024-09-14T22:37:09.955 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.431290427, 2024-09-14T22:37:09.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059345031999999999, 2024-09-14T22:37:09.955 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.955 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.955 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.955 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.955 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.955 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.955 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2092 2024-09-14T22:37:09.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.955 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451202+0000", 2024-09-14T22:37:09.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451202+0000", 2024-09-14T22:37:09.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451203+0000", 2024-09-14T22:37:09.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6199999999999999e-06 2024-09-14T22:37:09.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451209+0000", 2024-09-14T22:37:09.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1999999999999999e-06 2024-09-14T22:37:09.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.957 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.957 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451230+0000", 2024-09-14T22:37:09.957 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0247000000000001e-05 2024-09-14T22:37:09.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.957 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.957 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451245+0000", 2024-09-14T22:37:09.957 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4993e-05 2024-09-14T22:37:09.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.957 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.957 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510239+0000", 2024-09-14T22:37:09.958 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058994321000000002 2024-09-14T22:37:09.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.958 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.958 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.958 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510371+0000", 2024-09-14T22:37:09.958 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013195899999999999 2024-09-14T22:37:09.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.958 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.958 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.958 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510547+0000", 2024-09-14T22:37:09.958 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000175692 2024-09-14T22:37:09.958 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.958 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.958 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.958 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.959 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2093 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11515904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:09.959 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.451264+0000", 2024-09-14T22:37:09.959 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.431228390000001, 2024-09-14T22:37:09.959 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059636293999999999, 2024-09-14T22:37:09.959 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.959 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.959 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.959 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.959 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.959 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.959 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2093 2024-09-14T22:37:09.959 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.959 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.959 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.959 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451264+0000", 2024-09-14T22:37:09.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451264+0000", 2024-09-14T22:37:09.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451265+0000", 2024-09-14T22:37:09.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3400000000000001e-06 2024-09-14T22:37:09.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.961 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451270+0000", 2024-09-14T22:37:09.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4360000000000001e-06 2024-09-14T22:37:09.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.961 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451289+0000", 2024-09-14T22:37:09.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8383e-05 2024-09-14T22:37:09.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.961 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451303+0000", 2024-09-14T22:37:09.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.455e-05 2024-09-14T22:37:09.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510627+0000", 2024-09-14T22:37:09.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059323280999999999 2024-09-14T22:37:09.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510745+0000", 2024-09-14T22:37:09.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011867900000000001 2024-09-14T22:37:09.963 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.963 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.963 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.963 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510900+0000", 2024-09-14T22:37:09.963 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000154625 2024-09-14T22:37:09.963 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.963 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.963 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.963 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.963 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.963 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2094 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11524096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:09.963 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.451321+0000", 2024-09-14T22:37:09.963 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.431171403999997, 2024-09-14T22:37:09.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059938827, 2024-09-14T22:37:09.964 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.964 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.964 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.964 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.964 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.964 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.964 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2094 2024-09-14T22:37:09.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.964 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451321+0000", 2024-09-14T22:37:09.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451321+0000", 2024-09-14T22:37:09.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451322+0000", 2024-09-14T22:37:09.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0529999999999999e-06 2024-09-14T22:37:09.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451328+0000", 2024-09-14T22:37:09.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8300000000000001e-06 2024-09-14T22:37:09.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451344+0000", 2024-09-14T22:37:09.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6750000000000001e-05 2024-09-14T22:37:09.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451358+0000", 2024-09-14T22:37:09.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3200000000000001e-05 2024-09-14T22:37:09.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.967 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.967 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.510968+0000", 2024-09-14T22:37:09.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059610567000000003 2024-09-14T22:37:09.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.967 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.967 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.511094+0000", 2024-09-14T22:37:09.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012629199999999999 2024-09-14T22:37:09.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.967 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.968 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.511260+0000", 2024-09-14T22:37:09.968 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016513500000000001 2024-09-14T22:37:09.968 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.968 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.968 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.968 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.968 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.968 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:2095 3.7 3:e5c186b1:::rbd_data.113c3b74a67f.000000000000000a:4 [read 11532288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:09.968 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.451375+0000", 2024-09-14T22:37:09.968 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.431117231000002, 2024-09-14T22:37:09.968 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060228067000000003, 2024-09-14T22:37:09.968 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.969 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.969 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:09.969 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.969 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.969 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.969 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 2095 2024-09-14T22:37:09.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.969 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451375+0000", 2024-09-14T22:37:09.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451375+0000", 2024-09-14T22:37:09.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451376+0000", 2024-09-14T22:37:09.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0470000000000001e-06 2024-09-14T22:37:09.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451381+0000", 2024-09-14T22:37:09.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.536e-06 2024-09-14T22:37:09.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451399+0000", 2024-09-14T22:37:09.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7200000000000001e-05 2024-09-14T22:37:09.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.451412+0000", 2024-09-14T22:37:09.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3726999999999999e-05 2024-09-14T22:37:09.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.511308+0000", 2024-09-14T22:37:09.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059895176000000001 2024-09-14T22:37:09.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.511444+0000", 2024-09-14T22:37:09.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013621600000000001 2024-09-14T22:37:09.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.511603+0000", 2024-09-14T22:37:09.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015916500000000001 2024-09-14T22:37:09.973 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.973 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.973 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.973 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:45459 3.11 3:8ce265cb:::rbd_data.113cc4e35b3.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:37:09.973 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:36.190430+0000", 2024-09-14T22:37:09.973 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.692062487000001, 2024-09-14T22:37:09.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.06125162, 2024-09-14T22:37:09.974 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.974 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.974 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:09.974 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.974 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.974 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.974 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 45459 2024-09-14T22:37:09.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.974 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.974 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190430+0000", 2024-09-14T22:37:09.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190430+0000", 2024-09-14T22:37:09.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190432+0000", 2024-09-14T22:37:09.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3039999999999999e-06 2024-09-14T22:37:09.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190443+0000", 2024-09-14T22:37:09.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1178999999999999e-05 2024-09-14T22:37:09.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.976 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.976 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190474+0000", 2024-09-14T22:37:09.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0803000000000003e-05 2024-09-14T22:37:09.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.976 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.976 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190489+0000", 2024-09-14T22:37:09.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.517e-05 2024-09-14T22:37:09.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.976 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190623+0000", 2024-09-14T22:37:09.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013405599999999999 2024-09-14T22:37:09.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.190752+0000", 2024-09-14T22:37:09.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012870899999999999 2024-09-14T22:37:09.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:37:09.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.203210+0000", 2024-09-14T22:37:09.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012458484000000001 2024-09-14T22:37:09.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:09.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.251517+0000", 2024-09-14T22:37:09.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.048306453999999999 2024-09-14T22:37:09.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:09.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.251571+0000", 2024-09-14T22:37:09.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4228999999999999e-05 2024-09-14T22:37:09.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:09.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.251628+0000", 2024-09-14T22:37:09.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6922000000000002e-05 2024-09-14T22:37:09.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.979 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.979 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.251681+0000", 2024-09-14T22:37:09.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3310000000000003e-05 2024-09-14T22:37:09.979 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.979 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.979 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.979 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:45462 3.11 3:8ce265cb:::rbd_data.113cc4e35b3.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:37:09.979 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:36.258125+0000", 2024-09-14T22:37:09.979 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.624366909000003, 2024-09-14T22:37:09.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059222599000000001, 2024-09-14T22:37:09.979 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:09.979 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:09.980 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:09.980 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:09.980 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:09.980 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:09.980 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 45462 2024-09-14T22:37:09.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.980 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:09.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.980 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:09.980 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258125+0000", 2024-09-14T22:37:09.980 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.980 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:09.980 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258125+0000", 2024-09-14T22:37:09.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:09.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.981 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:09.981 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258127+0000", 2024-09-14T22:37:09.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3159999999999999e-06 2024-09-14T22:37:09.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.981 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:09.981 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258139+0000", 2024-09-14T22:37:09.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1667e-05 2024-09-14T22:37:09.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.981 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:09.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258170+0000", 2024-09-14T22:37:09.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0929999999999997e-05 2024-09-14T22:37:09.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:09.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258185+0000", 2024-09-14T22:37:09.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4423000000000001e-05 2024-09-14T22:37:09.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:09.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258291+0000", 2024-09-14T22:37:09.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106285 2024-09-14T22:37:09.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:09.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.258446+0000", 2024-09-14T22:37:09.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015486499999999999 2024-09-14T22:37:09.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:37:09.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.269615+0000", 2024-09-14T22:37:09.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011169095 2024-09-14T22:37:09.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:09.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.317113+0000", 2024-09-14T22:37:09.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.047498382999999998 2024-09-14T22:37:09.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.984 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:09.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.317144+0000", 2024-09-14T22:37:09.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0970000000000003e-05 2024-09-14T22:37:09.984 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.984 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:09.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.317233+0000", 2024-09-14T22:37:09.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9205999999999999e-05 2024-09-14T22:37:09.984 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:09.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:09.984 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:09.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:36.317348+0000", 2024-09-14T22:37:09.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000114459 2024-09-14T22:37:09.984 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.984 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.985 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.985 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:09.985 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:09.985 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:09.986 INFO:teuthology.orchestra.run.smithi001.stdout:2964 write 0xe69cbe9 thru 0xe6a249b (0x58b3 bytes) 2024-09-14T22:37:09.986 INFO:teuthology.orchestra.run.smithi001.stdout:2966 punch from 0x65e55a1 to 0x65ee5c8, (0x9027 bytes) 2024-09-14T22:37:09.986 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:09.986 INFO:teuthology.orchestra.run.smithi001.stdout:2969 trunc from 0xe6a249c to 0x8291091 2024-09-14T22:37:09.987 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:09.979+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:09.987 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:37:09.990 INFO:teuthology.orchestra.run.smithi001.stdout:2971 writesame 0xd3e38f7 thru 0xd3e97d9 data_size 0x3(0x5ee3 bytes) 2024-09-14T22:37:09.990 INFO:teuthology.orchestra.run.smithi001.stdout:2974 writesame 0x85124a2 thru 0x85159ee data_size 0x1(0x354d bytes) 2024-09-14T22:37:10.012 INFO:teuthology.orchestra.run.smithi001.stdout:2975 trunc from 0xd3e97da to 0xe832122 2024-09-14T22:37:10.024 INFO:teuthology.orchestra.run.smithi001.stdout:2976 writesame 0x1d790c thru 0x1e3d8e data_size 0x29(0xc483 bytes) 2024-09-14T22:37:10.087 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:10.083+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:10.171 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:10.171 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:37:10.172 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:37:10.172 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:10.188 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:10.183+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:10.189 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:37:10.289 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:10.283+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:10.357 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:10.357 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:37:10.357 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:37:10.357 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:37:10.357 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:10.374 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:37:10.374 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:37:10.421 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:10.419+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:10.490 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:10.487+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:10.499 INFO:teuthology.orchestra.run.smithi001.stderr:no valid command found; 10 closest matches: 2024-09-14T22:37:10.499 INFO:teuthology.orchestra.run.smithi001.stderr:0 2024-09-14T22:37:10.499 INFO:teuthology.orchestra.run.smithi001.stderr:1 2024-09-14T22:37:10.499 INFO:teuthology.orchestra.run.smithi001.stderr:2 2024-09-14T22:37:10.499 INFO:teuthology.orchestra.run.smithi001.stderr:abort 2024-09-14T22:37:10.499 INFO:teuthology.orchestra.run.smithi001.stderr:assert 2024-09-14T22:37:10.499 INFO:teuthology.orchestra.run.smithi001.stderr:bluestore allocator dump block 2024-09-14T22:37:10.499 INFO:teuthology.orchestra.run.smithi001.stderr:bluestore allocator fragmentation block 2024-09-14T22:37:10.499 INFO:teuthology.orchestra.run.smithi001.stderr:bluestore allocator fragmentation histogram block [] [] 2024-09-14T22:37:10.500 INFO:teuthology.orchestra.run.smithi001.stderr:bluestore allocator score block 2024-09-14T22:37:10.500 INFO:teuthology.orchestra.run.smithi001.stderr:config diff 2024-09-14T22:37:10.500 INFO:teuthology.orchestra.run.smithi001.stderr:admin_socket: invalid command 2024-09-14T22:37:10.501 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T22:37:10.502 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-14T22:37:10.506 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:10.507 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:37:10.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:37:10.507 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:37:10.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.507 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3022 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14301184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:10.507 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559594+0000", 2024-09-14T22:37:10.507 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.944609620000001, 2024-09-14T22:37:10.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028916481000000001, 2024-09-14T22:37:10.508 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:10.508 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:10.508 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:10.508 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:10.508 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:10.508 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:10.508 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3022 2024-09-14T22:37:10.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.508 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:10.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:10.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559594+0000", 2024-09-14T22:37:10.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:10.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559594+0000", 2024-09-14T22:37:10.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:10.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559594+0000", 2024-09-14T22:37:10.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7000000000000004e-07 2024-09-14T22:37:10.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:10.510 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559604+0000", 2024-09-14T22:37:10.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6020000000000006e-06 2024-09-14T22:37:10.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:10.510 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559615+0000", 2024-09-14T22:37:10.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1032999999999999e-05 2024-09-14T22:37:10.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:10.510 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559638+0000", 2024-09-14T22:37:10.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3255999999999999e-05 2024-09-14T22:37:10.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:10.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588323+0000", 2024-09-14T22:37:10.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.02868447 2024-09-14T22:37:10.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:10.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588443+0000", 2024-09-14T22:37:10.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120056 2024-09-14T22:37:10.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:10.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588510+0000", 2024-09-14T22:37:10.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7393999999999999e-05 2024-09-14T22:37:10.512 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.512 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:10.512 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.512 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3023 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14309376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:10.512 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559651+0000", 2024-09-14T22:37:10.512 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.944552133999998, 2024-09-14T22:37:10.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029146231000000002, 2024-09-14T22:37:10.513 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:10.513 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:10.513 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:10.513 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:10.513 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:10.513 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:10.513 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3023 2024-09-14T22:37:10.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.513 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:10.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.514 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:10.514 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559651+0000", 2024-09-14T22:37:10.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.514 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:10.514 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559651+0000", 2024-09-14T22:37:10.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.514 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:10.514 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559652+0000", 2024-09-14T22:37:10.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2699999999999998e-07 2024-09-14T22:37:10.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:10.515 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559656+0000", 2024-09-14T22:37:10.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7019999999999999e-06 2024-09-14T22:37:10.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:10.515 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559666+0000", 2024-09-14T22:37:10.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0400999999999999e-05 2024-09-14T22:37:10.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:10.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559674+0000", 2024-09-14T22:37:10.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8890000000000002e-06 2024-09-14T22:37:10.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:10.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588565+0000", 2024-09-14T22:37:10.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028891217 2024-09-14T22:37:10.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:10.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588679+0000", 2024-09-14T22:37:10.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114058 2024-09-14T22:37:10.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:10.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588797+0000", 2024-09-14T22:37:10.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118137 2024-09-14T22:37:10.517 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.517 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:10.517 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.517 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3024 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14317568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:10.517 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559685+0000", 2024-09-14T22:37:10.517 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.944518023000001, 2024-09-14T22:37:10.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029326498999999999, 2024-09-14T22:37:10.518 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:10.518 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:10.518 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:10.518 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:10.518 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:10.518 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:10.518 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3024 2024-09-14T22:37:10.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.518 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:10.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:10.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559685+0000", 2024-09-14T22:37:10.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:10.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559685+0000", 2024-09-14T22:37:10.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:10.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559686+0000", 2024-09-14T22:37:10.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0999999999999998e-07 2024-09-14T22:37:10.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:10.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559697+0000", 2024-09-14T22:37:10.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.147e-05 2024-09-14T22:37:10.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:10.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559708+0000", 2024-09-14T22:37:10.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0546e-05 2024-09-14T22:37:10.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:10.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559716+0000", 2024-09-14T22:37:10.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.704e-06 2024-09-14T22:37:10.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:10.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588833+0000", 2024-09-14T22:37:10.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029116442999999999 2024-09-14T22:37:10.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:10.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588920+0000", 2024-09-14T22:37:10.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6916999999999995e-05 2024-09-14T22:37:10.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:10.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589012+0000", 2024-09-14T22:37:10.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1809000000000001e-05 2024-09-14T22:37:10.521 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.522 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:10.522 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.522 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3025 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14325760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:10.522 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559729+0000", 2024-09-14T22:37:10.522 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.944474124000003, 2024-09-14T22:37:10.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029422501, 2024-09-14T22:37:10.522 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:10.522 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:10.522 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:10.522 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:10.522 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:10.522 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:10.523 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3025 2024-09-14T22:37:10.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.523 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:10.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.523 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:10.523 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559729+0000", 2024-09-14T22:37:10.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.523 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:10.523 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559729+0000", 2024-09-14T22:37:10.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:10.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559730+0000", 2024-09-14T22:37:10.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8100000000000002e-07 2024-09-14T22:37:10.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:10.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559733+0000", 2024-09-14T22:37:10.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4740000000000001e-06 2024-09-14T22:37:10.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:10.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559744+0000", 2024-09-14T22:37:10.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0489000000000001e-05 2024-09-14T22:37:10.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:10.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559760+0000", 2024-09-14T22:37:10.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5817000000000001e-05 2024-09-14T22:37:10.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:10.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589050+0000", 2024-09-14T22:37:10.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.02929056 2024-09-14T22:37:10.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:10.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589101+0000", 2024-09-14T22:37:10.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0992000000000002e-05 2024-09-14T22:37:10.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:10.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589152+0000", 2024-09-14T22:37:10.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0387999999999999e-05 2024-09-14T22:37:10.526 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.526 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:10.526 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.526 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3026 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14333952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:10.526 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559795+0000", 2024-09-14T22:37:10.527 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.944408299999999, 2024-09-14T22:37:10.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029483812000000002, 2024-09-14T22:37:10.527 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:10.527 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:10.527 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:10.527 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:10.527 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:10.527 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:10.527 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3026 2024-09-14T22:37:10.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.527 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:10.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.528 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:10.528 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559795+0000", 2024-09-14T22:37:10.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.528 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:10.528 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559795+0000", 2024-09-14T22:37:10.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.528 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:10.528 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559796+0000", 2024-09-14T22:37:10.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3200000000000003e-07 2024-09-14T22:37:10.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:10.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559800+0000", 2024-09-14T22:37:10.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2930000000000002e-06 2024-09-14T22:37:10.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:10.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559812+0000", 2024-09-14T22:37:10.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2053e-05 2024-09-14T22:37:10.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:10.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559821+0000", 2024-09-14T22:37:10.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8869999999999998e-06 2024-09-14T22:37:10.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:10.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589177+0000", 2024-09-14T22:37:10.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029356308000000001 2024-09-14T22:37:10.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:10.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589229+0000", 2024-09-14T22:37:10.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1326000000000003e-05 2024-09-14T22:37:10.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:10.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589279+0000", 2024-09-14T22:37:10.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0012999999999997e-05 2024-09-14T22:37:10.531 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.531 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:10.531 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.532 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3027 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14342144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:10.532 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559833+0000", 2024-09-14T22:37:10.532 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.944370571999997, 2024-09-14T22:37:10.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029619187000000002, 2024-09-14T22:37:10.532 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:10.532 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:10.532 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:10.532 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:10.532 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:10.532 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:10.532 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3027 2024-09-14T22:37:10.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.533 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:10.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.533 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:10.533 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559833+0000", 2024-09-14T22:37:10.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.533 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:10.533 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559833+0000", 2024-09-14T22:37:10.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:10.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559833+0000", 2024-09-14T22:37:10.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9400000000000005e-07 2024-09-14T22:37:10.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:10.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559845+0000", 2024-09-14T22:37:10.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1623e-05 2024-09-14T22:37:10.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:10.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559856+0000", 2024-09-14T22:37:10.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0832999999999999e-05 2024-09-14T22:37:10.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:10.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559864+0000", 2024-09-14T22:37:10.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3119999999999994e-06 2024-09-14T22:37:10.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:10.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589299+0000", 2024-09-14T22:37:10.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029434699000000002 2024-09-14T22:37:10.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:10.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589354+0000", 2024-09-14T22:37:10.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5467999999999998e-05 2024-09-14T22:37:10.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:10.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589452+0000", 2024-09-14T22:37:10.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7558000000000003e-05 2024-09-14T22:37:10.536 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.536 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:10.536 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.536 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3028 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14350336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:10.537 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559875+0000", 2024-09-14T22:37:10.537 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.944327684000001, 2024-09-14T22:37:10.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029775586999999999, 2024-09-14T22:37:10.537 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:10.537 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:10.537 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:10.537 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:10.537 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:10.537 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:10.537 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3028 2024-09-14T22:37:10.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.537 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:10.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.537 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:10.537 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559875+0000", 2024-09-14T22:37:10.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:10.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559875+0000", 2024-09-14T22:37:10.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:10.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559876+0000", 2024-09-14T22:37:10.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5199999999999996e-07 2024-09-14T22:37:10.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:10.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559879+0000", 2024-09-14T22:37:10.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.095e-06 2024-09-14T22:37:10.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:10.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559889+0000", 2024-09-14T22:37:10.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0019e-05 2024-09-14T22:37:10.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:10.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559897+0000", 2024-09-14T22:37:10.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9189999999999999e-06 2024-09-14T22:37:10.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:10.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589486+0000", 2024-09-14T22:37:10.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029588482999999999 2024-09-14T22:37:10.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:10.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589578+0000", 2024-09-14T22:37:10.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2329000000000003e-05 2024-09-14T22:37:10.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:10.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589651+0000", 2024-09-14T22:37:10.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2990000000000004e-05 2024-09-14T22:37:10.540 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.540 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:10.540 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.541 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4068 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13252608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:10.541 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680102+0000", 2024-09-14T22:37:10.541 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.824100864999998, 2024-09-14T22:37:10.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028907656, 2024-09-14T22:37:10.541 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:10.541 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:10.541 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:10.541 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:10.541 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:10.541 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:10.541 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4068 2024-09-14T22:37:10.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.542 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:10.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.542 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:10.542 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680102+0000", 2024-09-14T22:37:10.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.542 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:10.542 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680102+0000", 2024-09-14T22:37:10.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.542 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:10.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680103+0000", 2024-09-14T22:37:10.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6600000000000002e-07 2024-09-14T22:37:10.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:10.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680105+0000", 2024-09-14T22:37:10.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.616e-06 2024-09-14T22:37:10.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:10.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680114+0000", 2024-09-14T22:37:10.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4120000000000001e-06 2024-09-14T22:37:10.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:10.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680119+0000", 2024-09-14T22:37:10.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6210000000000001e-06 2024-09-14T22:37:10.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:10.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.708813+0000", 2024-09-14T22:37:10.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028693330999999999 2024-09-14T22:37:10.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:10.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.708903+0000", 2024-09-14T22:37:10.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0297000000000006e-05 2024-09-14T22:37:10.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.545 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:10.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709010+0000", 2024-09-14T22:37:10.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106913 2024-09-14T22:37:10.545 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.545 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:10.545 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.545 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4069 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13260800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:10.546 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680128+0000", 2024-09-14T22:37:10.546 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.82407559, 2024-09-14T22:37:10.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029145476, 2024-09-14T22:37:10.546 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:10.546 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:10.546 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:10.546 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:10.546 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:10.546 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:10.546 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4069 2024-09-14T22:37:10.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.546 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:10.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:10.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680128+0000", 2024-09-14T22:37:10.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:10.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680128+0000", 2024-09-14T22:37:10.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:10.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680128+0000", 2024-09-14T22:37:10.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7599999999999998e-07 2024-09-14T22:37:10.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:10.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680131+0000", 2024-09-14T22:37:10.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9050000000000001e-06 2024-09-14T22:37:10.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:10.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680138+0000", 2024-09-14T22:37:10.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2559999999999996e-06 2024-09-14T22:37:10.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:10.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680144+0000", 2024-09-14T22:37:10.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4779999999999999e-06 2024-09-14T22:37:10.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:10.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709054+0000", 2024-09-14T22:37:10.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028910207 2024-09-14T22:37:10.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:10.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709180+0000", 2024-09-14T22:37:10.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012608699999999999 2024-09-14T22:37:10.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.550 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:10.550 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709273+0000", 2024-09-14T22:37:10.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3166999999999998e-05 2024-09-14T22:37:10.550 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.550 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:10.550 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.550 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4070 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13268992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:10.550 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680151+0000", 2024-09-14T22:37:10.550 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.824052307999999, 2024-09-14T22:37:10.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029327986, 2024-09-14T22:37:10.550 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:10.550 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:10.551 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:10.551 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:10.551 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:10.551 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:10.551 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4070 2024-09-14T22:37:10.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.551 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:10.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.551 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:10.551 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680151+0000", 2024-09-14T22:37:10.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.551 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:10.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680151+0000", 2024-09-14T22:37:10.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:10.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680151+0000", 2024-09-14T22:37:10.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:37:10.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:10.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680154+0000", 2024-09-14T22:37:10.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3350000000000001e-06 2024-09-14T22:37:10.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:10.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680162+0000", 2024-09-14T22:37:10.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.473e-06 2024-09-14T22:37:10.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:10.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680177+0000", 2024-09-14T22:37:10.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.465e-05 2024-09-14T22:37:10.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:10.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709356+0000", 2024-09-14T22:37:10.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029178744 2024-09-14T22:37:10.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:10.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709428+0000", 2024-09-14T22:37:10.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2428000000000005e-05 2024-09-14T22:37:10.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:10.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709479+0000", 2024-09-14T22:37:10.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0915999999999998e-05 2024-09-14T22:37:10.554 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.554 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:10.555 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.555 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4071 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13277184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:10.555 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680185+0000", 2024-09-14T22:37:10.555 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.824017808000001, 2024-09-14T22:37:10.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029452392000000001, 2024-09-14T22:37:10.555 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:10.555 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:10.555 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:10.555 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:10.555 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:10.556 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:10.556 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4071 2024-09-14T22:37:10.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.556 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:10.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:10.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680185+0000", 2024-09-14T22:37:10.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:10.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680185+0000", 2024-09-14T22:37:10.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:10.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680186+0000", 2024-09-14T22:37:10.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6100000000000001e-07 2024-09-14T22:37:10.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:10.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680188+0000", 2024-09-14T22:37:10.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.509e-06 2024-09-14T22:37:10.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:10.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680197+0000", 2024-09-14T22:37:10.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8550000000000008e-06 2024-09-14T22:37:10.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:10.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680203+0000", 2024-09-14T22:37:10.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5849999999999999e-06 2024-09-14T22:37:10.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:10.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709514+0000", 2024-09-14T22:37:10.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029311054 2024-09-14T22:37:10.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:10.559 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709585+0000", 2024-09-14T22:37:10.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0875000000000002e-05 2024-09-14T22:37:10.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:10.559 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709638+0000", 2024-09-14T22:37:10.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3053000000000002e-05 2024-09-14T22:37:10.559 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.559 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:10.559 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.560 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4072 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13285376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:10.560 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680211+0000", 2024-09-14T22:37:10.560 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.823992527000001, 2024-09-14T22:37:10.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029591177999999999, 2024-09-14T22:37:10.560 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:10.560 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:10.560 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:10.560 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:10.560 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:10.560 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:10.560 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4072 2024-09-14T22:37:10.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.561 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:10.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:10.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680211+0000", 2024-09-14T22:37:10.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:10.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680211+0000", 2024-09-14T22:37:10.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:10.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680211+0000", 2024-09-14T22:37:10.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6699999999999999e-07 2024-09-14T22:37:10.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:10.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680214+0000", 2024-09-14T22:37:10.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0149999999999999e-06 2024-09-14T22:37:10.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:10.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680222+0000", 2024-09-14T22:37:10.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.371e-06 2024-09-14T22:37:10.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:10.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680228+0000", 2024-09-14T22:37:10.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6230000000000002e-06 2024-09-14T22:37:10.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:10.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709670+0000", 2024-09-14T22:37:10.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029441694000000001 2024-09-14T22:37:10.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:10.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709741+0000", 2024-09-14T22:37:10.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1668999999999994e-05 2024-09-14T22:37:10.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.564 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:10.564 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709802+0000", 2024-09-14T22:37:10.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0338999999999999e-05 2024-09-14T22:37:10.564 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.564 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:10.564 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.564 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4073 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13293568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:10.564 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680236+0000", 2024-09-14T22:37:10.565 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.823967168000003, 2024-09-14T22:37:10.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029755238999999999, 2024-09-14T22:37:10.565 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:10.565 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:10.565 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:10.565 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:10.565 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:10.565 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:10.565 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4073 2024-09-14T22:37:10.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.565 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:10.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:10.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680236+0000", 2024-09-14T22:37:10.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:10.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680236+0000", 2024-09-14T22:37:10.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:10.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680236+0000", 2024-09-14T22:37:10.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5499999999999998e-07 2024-09-14T22:37:10.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:10.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680239+0000", 2024-09-14T22:37:10.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4380000000000002e-06 2024-09-14T22:37:10.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:10.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680248+0000", 2024-09-14T22:37:10.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6319999999999997e-06 2024-09-14T22:37:10.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:10.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680261+0000", 2024-09-14T22:37:10.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3193000000000001e-05 2024-09-14T22:37:10.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:10.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709831+0000", 2024-09-14T22:37:10.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029570486 2024-09-14T22:37:10.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:10.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709883+0000", 2024-09-14T22:37:10.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2290999999999998e-05 2024-09-14T22:37:10.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:10.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709991+0000", 2024-09-14T22:37:10.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107744 2024-09-14T22:37:10.569 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.569 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:10.569 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.569 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4074 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13301760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:10.569 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680269+0000", 2024-09-14T22:37:10.569 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.823934270000002, 2024-09-14T22:37:10.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030007875999999999, 2024-09-14T22:37:10.569 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:10.570 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:10.570 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:10.570 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:10.570 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:10.570 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:10.570 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4074 2024-09-14T22:37:10.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.570 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:10.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:10.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680269+0000", 2024-09-14T22:37:10.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:10.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680269+0000", 2024-09-14T22:37:10.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:10.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680269+0000", 2024-09-14T22:37:10.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7300000000000001e-07 2024-09-14T22:37:10.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:10.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680277+0000", 2024-09-14T22:37:10.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8480000000000001e-06 2024-09-14T22:37:10.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:10.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680286+0000", 2024-09-14T22:37:10.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0529999999999996e-06 2024-09-14T22:37:10.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:10.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680292+0000", 2024-09-14T22:37:10.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.728e-06 2024-09-14T22:37:10.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:10.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.710024+0000", 2024-09-14T22:37:10.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029732176999999999 2024-09-14T22:37:10.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:10.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.710109+0000", 2024-09-14T22:37:10.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4715000000000005e-05 2024-09-14T22:37:10.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:10.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.710277+0000", 2024-09-14T22:37:10.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000167882 2024-09-14T22:37:10.573 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.573 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:10.574 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.574 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:12664 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T22:37:10.574 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:28.325596+0000", 2024-09-14T22:37:10.574 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.178606756000001, 2024-09-14T22:37:10.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028964581, 2024-09-14T22:37:10.574 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:10.574 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:10.574 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:10.574 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:10.574 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:10.574 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:10.574 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12664 2024-09-14T22:37:10.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.575 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:10.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:10.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325596+0000", 2024-09-14T22:37:10.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:10.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325596+0000", 2024-09-14T22:37:10.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:10.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325597+0000", 2024-09-14T22:37:10.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4300000000000002e-07 2024-09-14T22:37:10.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:10.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325600+0000", 2024-09-14T22:37:10.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2219999999999998e-06 2024-09-14T22:37:10.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:10.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325610+0000", 2024-09-14T22:37:10.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2650000000000002e-06 2024-09-14T22:37:10.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:10.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325617+0000", 2024-09-14T22:37:10.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1199999999999996e-06 2024-09-14T22:37:10.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:10.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354438+0000", 2024-09-14T22:37:10.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028821205999999999 2024-09-14T22:37:10.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:10.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354495+0000", 2024-09-14T22:37:10.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6752999999999997e-05 2024-09-14T22:37:10.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:10.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354561+0000", 2024-09-14T22:37:10.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6272000000000005e-05 2024-09-14T22:37:10.578 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.578 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:10.578 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.578 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:12665 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 391168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T22:37:10.578 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:28.325626+0000", 2024-09-14T22:37:10.579 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.178577433999997, 2024-09-14T22:37:10.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029080366, 2024-09-14T22:37:10.579 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:10.579 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:10.579 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:10.579 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:10.579 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:10.579 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:10.579 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12665 2024-09-14T22:37:10.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.579 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:10.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:10.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325626+0000", 2024-09-14T22:37:10.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:10.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325626+0000", 2024-09-14T22:37:10.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:10.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325626+0000", 2024-09-14T22:37:10.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.06e-07 2024-09-14T22:37:10.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:10.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325629+0000", 2024-09-14T22:37:10.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7149999999999998e-06 2024-09-14T22:37:10.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:10.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325637+0000", 2024-09-14T22:37:10.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7360000000000001e-06 2024-09-14T22:37:10.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:10.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325643+0000", 2024-09-14T22:37:10.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2480000000000001e-06 2024-09-14T22:37:10.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:10.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354589+0000", 2024-09-14T22:37:10.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028945538 2024-09-14T22:37:10.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:10.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354635+0000", 2024-09-14T22:37:10.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6374e-05 2024-09-14T22:37:10.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:10.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354706+0000", 2024-09-14T22:37:10.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1049000000000004e-05 2024-09-14T22:37:10.583 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.583 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:10.583 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.583 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:13927 3.1b 3:ddfd6241:::rbd_data.113cda0183a0.0000000000000009:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-14T22:37:10.583 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:31.537348+0000", 2024-09-14T22:37:10.583 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 38.966854834000003, 2024-09-14T22:37:10.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032156573000000001, 2024-09-14T22:37:10.583 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:10.583 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:10.583 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:10.583 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:10.583 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:10.584 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:10.584 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 13927 2024-09-14T22:37:10.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.584 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:10.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:10.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537348+0000", 2024-09-14T22:37:10.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:10.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537348+0000", 2024-09-14T22:37:10.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:10.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537349+0000", 2024-09-14T22:37:10.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.068e-06 2024-09-14T22:37:10.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:10.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537358+0000", 2024-09-14T22:37:10.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.653e-06 2024-09-14T22:37:10.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:10.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537374+0000", 2024-09-14T22:37:10.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5594e-05 2024-09-14T22:37:10.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:10.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537383+0000", 2024-09-14T22:37:10.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0929999999999992e-06 2024-09-14T22:37:10.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:10.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537429+0000", 2024-09-14T22:37:10.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5948000000000001e-05 2024-09-14T22:37:10.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:10.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537490+0000", 2024-09-14T22:37:10.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1002000000000003e-05 2024-09-14T22:37:10.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 4", 2024-09-14T22:37:10.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.542781+0000", 2024-09-14T22:37:10.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.005290861 2024-09-14T22:37:10.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:10.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.569350+0000", 2024-09-14T22:37:10.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026569299000000001 2024-09-14T22:37:10.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:37:10.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.569375+0000", 2024-09-14T22:37:10.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5448000000000001e-05 2024-09-14T22:37:10.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:37:10.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.569444+0000", 2024-09-14T22:37:10.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9184999999999996e-05 2024-09-14T22:37:10.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:10.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.569505+0000", 2024-09-14T22:37:10.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0421999999999998e-05 2024-09-14T22:37:10.588 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.588 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:10.588 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.589 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:85875 3.12 3:498b1eba:::rbd_data.113cd62ea8a5.000000000000006a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:10.589 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:00.460932+0000", 2024-09-14T22:37:10.589 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.043270933000001, 2024-09-14T22:37:10.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044152693999999999, 2024-09-14T22:37:10.589 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:10.589 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:10.589 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:10.589 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:10.589 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:10.589 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:10.589 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 85875 2024-09-14T22:37:10.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.590 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:10.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:10.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.460932+0000", 2024-09-14T22:37:10.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:10.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.460932+0000", 2024-09-14T22:37:10.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:10.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.460935+0000", 2024-09-14T22:37:10.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6280000000000001e-06 2024-09-14T22:37:10.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:10.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.460947+0000", 2024-09-14T22:37:10.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2269000000000001e-05 2024-09-14T22:37:10.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:10.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.460973+0000", 2024-09-14T22:37:10.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5939999999999999e-05 2024-09-14T22:37:10.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:10.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.460985+0000", 2024-09-14T22:37:10.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.181e-05 2024-09-14T22:37:10.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:10.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.461102+0000", 2024-09-14T22:37:10.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000117413 2024-09-14T22:37:10.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:10.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.461349+0000", 2024-09-14T22:37:10.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00024703300000000002 2024-09-14T22:37:10.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 7", 2024-09-14T22:37:10.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.468711+0000", 2024-09-14T22:37:10.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0073619180000000003 2024-09-14T22:37:10.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:37:10.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.504629+0000", 2024-09-14T22:37:10.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035917273 2024-09-14T22:37:10.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:10.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.504776+0000", 2024-09-14T22:37:10.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014738300000000001 2024-09-14T22:37:10.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:37:10.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.504951+0000", 2024-09-14T22:37:10.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017548100000000001 2024-09-14T22:37:10.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:10.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.505085+0000", 2024-09-14T22:37:10.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013354600000000001 2024-09-14T22:37:10.595 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.595 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:10.595 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.595 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:85883 3.12 3:4cf65cce:::rbd_data.113cd62ea8a5.0000000000000015:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:10.595 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:00.524670+0000", 2024-09-14T22:37:10.595 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9795329089999996, 2024-09-14T22:37:10.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050663316, 2024-09-14T22:37:10.595 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:10.595 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:10.595 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:10.595 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:10.595 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:10.596 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:10.596 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 85883 2024-09-14T22:37:10.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.596 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:10.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:10.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.524670+0000", 2024-09-14T22:37:10.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:10.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.524670+0000", 2024-09-14T22:37:10.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:10.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.524672+0000", 2024-09-14T22:37:10.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1629999999999999e-06 2024-09-14T22:37:10.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:10.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.524688+0000", 2024-09-14T22:37:10.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5787e-05 2024-09-14T22:37:10.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:10.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.524720+0000", 2024-09-14T22:37:10.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1294000000000002e-05 2024-09-14T22:37:10.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:10.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.524736+0000", 2024-09-14T22:37:10.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.607e-05 2024-09-14T22:37:10.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:10.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.524863+0000", 2024-09-14T22:37:10.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012693399999999999 2024-09-14T22:37:10.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:10.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.525060+0000", 2024-09-14T22:37:10.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000197732 2024-09-14T22:37:10.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 7", 2024-09-14T22:37:10.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.532377+0000", 2024-09-14T22:37:10.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0073167680000000004 2024-09-14T22:37:10.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:10.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.575115+0000", 2024-09-14T22:37:10.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042737626000000001 2024-09-14T22:37:10.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:37:10.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.575151+0000", 2024-09-14T22:37:10.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6211000000000003e-05 2024-09-14T22:37:10.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:37:10.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.575245+0000", 2024-09-14T22:37:10.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4648999999999995e-05 2024-09-14T22:37:10.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:10.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.575334+0000", 2024-09-14T22:37:10.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8082000000000001e-05 2024-09-14T22:37:10.601 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.601 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:10.601 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.601 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:86178 3.1 3:83964f10:::rbd_data.113cd62ea8a5.0000000000000027:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:10.601 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:01.316148+0000", 2024-09-14T22:37:10.601 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.1880556060000007, 2024-09-14T22:37:10.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.036603839999999999, 2024-09-14T22:37:10.601 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:10.602 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:10.602 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:10.602 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:10.602 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:10.602 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:10.602 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 86178 2024-09-14T22:37:10.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.602 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:10.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:10.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.316148+0000", 2024-09-14T22:37:10.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:10.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.316148+0000", 2024-09-14T22:37:10.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:10.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:10.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.316150+0000", 2024-09-14T22:37:10.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.43e-06 2024-09-14T22:37:10.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:10.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.316163+0000", 2024-09-14T22:37:10.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2581e-05 2024-09-14T22:37:10.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:10.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.316194+0000", 2024-09-14T22:37:10.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1794000000000001e-05 2024-09-14T22:37:10.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:10.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.316210+0000", 2024-09-14T22:37:10.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5947000000000002e-05 2024-09-14T22:37:10.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:10.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.316343+0000", 2024-09-14T22:37:10.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000132843 2024-09-14T22:37:10.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:10.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.316486+0000", 2024-09-14T22:37:10.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014334300000000001 2024-09-14T22:37:10.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 4", 2024-09-14T22:37:10.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.324078+0000", 2024-09-14T22:37:10.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0075916990000000004 2024-09-14T22:37:10.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:10.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.352551+0000", 2024-09-14T22:37:10.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028472515 2024-09-14T22:37:10.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:37:10.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.352587+0000", 2024-09-14T22:37:10.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6196999999999999e-05 2024-09-14T22:37:10.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:37:10.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.352676+0000", 2024-09-14T22:37:10.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8862000000000004e-05 2024-09-14T22:37:10.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:10.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:10.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:10.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.352751+0000", 2024-09-14T22:37:10.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5629000000000004e-05 2024-09-14T22:37:10.607 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.607 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:10.607 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.607 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:10.607 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:10.607 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:10.607 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T22:37:10.609 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:37:10.609 INFO:teuthology.orchestra.run.smithi001.stdout:2979 trunc from 0xe832122 to 0x59ffab4 2024-09-14T22:37:10.609 INFO:teuthology.orchestra.run.smithi001.stdout:2980 clone 20 order 19 su 65536 sc 5 2024-09-14T22:37:10.610 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:10.603+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:10.709 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:10.707+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:10.718 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:37:10.778 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone19 from 0x59ffab4 (overlap 0x46a726) to 0x0 2024-09-14T22:37:10.803 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:10.804 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:37:10.804 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:37:10.804 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:10.810 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:10.807+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:10.827 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:37:10.877 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:37:10.877 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:37:10.913 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:37:10.913 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:37:10.913 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-14T22:37:10.913 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:10.911+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:10.937 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:37:10.937 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:37:10.948 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-14T22:37:10.973 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:37:10.974 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:37:11.003 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:11.004 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:11.008 INFO:teuthology.orchestra.run.smithi001.stdout:2981 punch from 0x3c52abe to 0x3c5328b, (0x7cd bytes) 2024-09-14T22:37:11.010 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:37:11.010 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:37:11.013 INFO:teuthology.orchestra.run.smithi001.stdout:2983 read 0x14dc6be thru 0x14e2052 (0x5995 bytes) 2024-09-14T22:37:11.014 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:11.007+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:11.014 INFO:teuthology.orchestra.run.smithi001.stdout:2985 trunc from 0x59ffab4 to 0x5371f49 2024-09-14T22:37:11.023 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:37:11.047 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:37:11.047 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:37:11.052 INFO:teuthology.orchestra.run.smithi001.stdout:2986 writesame 0xe0b5add thru 0xe0b9ffe data_size 0x1(0x4522 bytes) 2024-09-14T22:37:11.052 INFO:teuthology.orchestra.run.smithi001.stdout:2987 read 0xd1809d5 thru 0xd18c3d3 (0xb9ff bytes) 2024-09-14T22:37:11.052 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:11.053 INFO:teuthology.orchestra.run.smithi001.stdout:2990 write 0x5deba63 thru 0x5df81cc (0xc76a bytes) 2024-09-14T22:37:11.058 INFO:teuthology.orchestra.run.smithi001.stdout:2991 write 0x56b7266 thru 0x56bca09 (0x57a4 bytes) 2024-09-14T22:37:11.063 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:11.064 INFO:teuthology.orchestra.run.smithi001.stdout:2993 writesame 0x8d9c89 thru 0x8dd2de data_size 0x1a(0x3656 bytes) 2024-09-14T22:37:11.070 INFO:teuthology.orchestra.run.smithi001.stdout:2995 read 0x1385873 thru 0x139395e (0xe0ec bytes) 2024-09-14T22:37:11.083 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:37:11.084 INFO:teuthology.orchestra.run.smithi001.stdout:2996 trunc from 0xe0b9fff to 0x476634a 2024-09-14T22:37:11.084 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:37:11.087 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:37:11.088 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:37:11.108 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:11.183 INFO:teuthology.orchestra.run.smithi001.stdout:2998 read 0x2a20cf2 thru 0x2a290a2 (0x83b1 bytes) 2024-09-14T22:37:11.185 INFO:teuthology.orchestra.run.smithi001.stdout:2999 writesame 0x6c58783 thru 0x6c612f2 data_size 0x61(0x8b70 bytes) 2024-09-14T22:37:11.185 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:11.185 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:11.185 INFO:teuthology.orchestra.run.smithi001.stdout:3003 trunc from 0x6c612f3 to 0xae251ae 2024-09-14T22:37:11.185 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:11.185 INFO:teuthology.orchestra.run.smithi001.stdout:3005 punch from 0x204450e to 0x20489a9, (0x449b bytes) 2024-09-14T22:37:11.185 INFO:teuthology.orchestra.run.smithi001.stdout:3006 writesame 0x48979ba thru 0x4899951 data_size 0xc(0x1f98 bytes) 2024-09-14T22:37:11.186 INFO:teuthology.orchestra.run.smithi001.stdout:3007 write 0xb9fa9d0 thru 0xb9fde31 (0x3462 bytes) 2024-09-14T22:37:11.186 INFO:teuthology.orchestra.run.smithi001.stdout:3008 read 0x27dc36a thru 0x27e027a (0x3f11 bytes) 2024-09-14T22:37:11.186 INFO:teuthology.orchestra.run.smithi001.stdout:3009 trunc from 0xb9fde32 to 0x37c7c3f 2024-09-14T22:37:11.186 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:11.186 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:11.186 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:11.179+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:11.201 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:37:11.212 INFO:teuthology.orchestra.run.smithi001.stdout:3010 writesame 0xb99f7b2 thru 0xb9af740 data_size 0x1(0xff8f bytes) 2024-09-14T22:37:11.212 INFO:teuthology.orchestra.run.smithi001.stdout:3011 read 0xaad0a3b thru 0xaadee02 (0xe3c8 bytes) 2024-09-14T22:37:11.213 INFO:teuthology.orchestra.run.smithi001.stdout:3012 read 0x282c471 thru 0x283ad10 (0xe8a0 bytes) 2024-09-14T22:37:11.213 INFO:teuthology.orchestra.run.smithi001.stdout:3013 write 0x32f154b thru 0x3300342 (0xedf8 bytes) 2024-09-14T22:37:11.219 INFO:teuthology.orchestra.run.smithi001.stdout:3016 write 0xd07f8b thru 0xd0d6e3 (0x5759 bytes) 2024-09-14T22:37:11.229 INFO:teuthology.orchestra.run.smithi001.stdout:3017 write 0x7e294fb thru 0x7e3911c (0xfc22 bytes) 2024-09-14T22:37:11.237 INFO:teuthology.orchestra.run.smithi001.stdout:3018 write 0xa37bfb1 thru 0xa385873 (0x98c3 bytes) 2024-09-14T22:37:11.244 INFO:teuthology.orchestra.run.smithi001.stdout:3019 write 0x166da8a thru 0x1673db6 (0x632d bytes) 2024-09-14T22:37:11.252 INFO:teuthology.orchestra.run.smithi001.stdout:3020 read 0x5a4704a thru 0x5a54768 (0xd71f bytes) 2024-09-14T22:37:11.252 INFO:teuthology.orchestra.run.smithi001.stdout:3021 read 0x5d7f74 thru 0x5dbb71 (0x3bfe bytes) 2024-09-14T22:37:11.254 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:11.251+0000 7f73aa34c7c0 -1 osd.2 30 log_to_monitors true 2024-09-14T22:37:11.261 INFO:teuthology.orchestra.run.smithi001.stdout:3022 writesame 0xdc8300a thru 0xdc8aa68 data_size 0x1(0x7a5f bytes) 2024-09-14T22:37:11.261 INFO:teuthology.orchestra.run.smithi001.stdout:3023 write 0x8d3a828 thru 0x8d3b7ae (0xf87 bytes) 2024-09-14T22:37:11.265 INFO:teuthology.orchestra.run.smithi001.stdout:3025 punch from 0x984ce62 to 0x984f7f5, (0x2993 bytes) 2024-09-14T22:37:11.265 INFO:teuthology.orchestra.run.smithi001.stdout:3026 write 0xc07e205 thru 0xc083f90 (0x5d8c bytes) 2024-09-14T22:37:11.271 INFO:teuthology.orchestra.run.smithi001.stdout:3027 writesame 0xb94534b thru 0xb95282d data_size 0x1(0xd4e3 bytes) 2024-09-14T22:37:11.286 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:11.279+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:11.299 INFO:teuthology.orchestra.run.smithi001.stdout:3028 read 0x83e485a thru 0x83f1861 (0xd008 bytes) 2024-09-14T22:37:11.299 INFO:teuthology.orchestra.run.smithi001.stdout:3029 read 0xcefb2be thru 0xcf09b90 (0xe8d3 bytes) 2024-09-14T22:37:11.299 INFO:teuthology.orchestra.run.smithi001.stdout:3030 write 0x5dc23cf thru 0x5dc27f2 (0x424 bytes) 2024-09-14T22:37:11.304 INFO:teuthology.orchestra.run.smithi001.stdout:3031 read 0xd54b3 thru 0xe24b6 (0xd004 bytes) 2024-09-14T22:37:11.307 INFO:teuthology.orchestra.run.smithi001.stdout:3032 punch from 0x19028c4 to 0x1903602, (0xd3e bytes) 2024-09-14T22:37:11.307 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:11.307 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:11.308 INFO:teuthology.orchestra.run.smithi001.stdout:3037 read 0xa6da962 thru 0xa6e5ea2 (0xb541 bytes) 2024-09-14T22:37:11.308 INFO:teuthology.orchestra.run.smithi001.stdout:3038 punch from 0xb8be2f8 to 0xb8c8ced, (0xa9f5 bytes) 2024-09-14T22:37:11.308 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:11.308 INFO:teuthology.orchestra.run.smithi001.stdout:3040 write 0x5ea11d7 thru 0x5ea6ca1 (0x5acb bytes) 2024-09-14T22:37:11.312 INFO:teuthology.orchestra.run.smithi001.stdout:3041 write 0x9b165ab thru 0x9b1d49e (0x6ef4 bytes) 2024-09-14T22:37:11.316 INFO:teuthology.orchestra.run.smithi001.stdout:3043 punch from 0x73f44ce to 0x73f6a4c, (0x257e bytes) 2024-09-14T22:37:11.316 INFO:teuthology.orchestra.run.smithi001.stdout:3044 writesame 0x81f1374 thru 0x81ffa79 data_size 0x3(0xe706 bytes) 2024-09-14T22:37:11.342 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:11.342 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:11.344 INFO:teuthology.orchestra.run.smithi001.stdout:3045 read 0xbded1d9 thru 0xbdf86be (0xb4e6 bytes) 2024-09-14T22:37:11.344 INFO:teuthology.orchestra.run.smithi001.stdout:3046 writesame 0xc12a22 thru 0xc1dac7 data_size 0x3(0xb0a6 bytes) 2024-09-14T22:37:11.353 INFO:teuthology.orchestra.run.smithi001.stdout:3048 write 0x683c3e6 thru 0x6846fa9 (0xabc4 bytes) 2024-09-14T22:37:11.357 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:37:11.364 INFO:teuthology.orchestra.run.smithi001.stdout:3049 trunc from 0xdc8aa69 to 0xa8b63d3 2024-09-14T22:37:11.378 INFO:teuthology.orchestra.run.smithi001.stdout:3051 read 0xc8d91d thru 0xc9c683 (0xed67 bytes) 2024-09-14T22:37:11.378 INFO:teuthology.orchestra.run.smithi001.stdout:3056 read 0x89d183b thru 0x89d5fd2 (0x4798 bytes) 2024-09-14T22:37:11.378 INFO:teuthology.orchestra.run.smithi001.stdout:3057 read 0x9b5a144 thru 0x9b62be7 (0x8aa4 bytes) 2024-09-14T22:37:11.378 INFO:teuthology.orchestra.run.smithi001.stdout:3058 read 0x3b976d5 thru 0x3ba523c (0xdb68 bytes) 2024-09-14T22:37:11.386 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:11.383+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:11.390 INFO:teuthology.orchestra.run.smithi001.stdout:3062 write 0xe7fe0f4 thru 0xe80ac5e (0xcb6b bytes) 2024-09-14T22:37:11.394 INFO:teuthology.orchestra.run.smithi001.stdout:3063 writesame 0x33d4e3e thru 0x33d6a96 data_size 0x29(0x1c59 bytes) 2024-09-14T22:37:11.400 INFO:teuthology.orchestra.run.smithi001.stdout:3065 read 0xa34cfd5 thru 0xa35410d (0x7139 bytes) 2024-09-14T22:37:11.400 INFO:teuthology.orchestra.run.smithi001.stdout:3066 write 0x578472b thru 0x578fb45 (0xb41b bytes) 2024-09-14T22:37:11.414 INFO:teuthology.orchestra.run.smithi001.stdout:3067 trunc from 0xe80ac5f to 0x6fb173c 2024-09-14T22:37:11.428 INFO:teuthology.orchestra.run.smithi001.stdout:3068 read 0x1cd26b3 thru 0x1cd9501 (0x6e4f bytes) 2024-09-14T22:37:11.428 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:11.428 INFO:teuthology.orchestra.run.smithi001.stdout:3070 read 0x3d5acb thru 0x3db286 (0x57bc bytes) 2024-09-14T22:37:11.431 INFO:teuthology.orchestra.run.smithi001.stdout:3071 trunc from 0x6fb173c to 0x8235563 2024-09-14T22:37:11.454 INFO:teuthology.orchestra.run.smithi001.stdout:3073 write 0xeadc438 thru 0xeaea4ab (0xe074 bytes) 2024-09-14T22:37:11.461 INFO:teuthology.orchestra.run.smithi001.stdout:3075 writesame 0xc6efb1c thru 0xc6f1be6 data_size 0x5(0x20cb bytes) 2024-09-14T22:37:11.469 INFO:teuthology.orchestra.run.smithi001.stdout:3076 punch from 0xd138247 to 0xd142f85, (0xad3e bytes) 2024-09-14T22:37:11.469 INFO:teuthology.orchestra.run.smithi001.stdout:3077 read 0x98fde18 thru 0x99060b2 (0x829b bytes) 2024-09-14T22:37:11.469 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:11.469 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:11.469 INFO:teuthology.orchestra.run.smithi001.stdout:3083 write 0x69b6412 thru 0x69b89fb (0x25ea bytes) 2024-09-14T22:37:11.474 INFO:teuthology.orchestra.run.smithi001.stdout:3086 read 0x98692d9 thru 0x9875fc5 (0xcced bytes) 2024-09-14T22:37:11.474 INFO:teuthology.orchestra.run.smithi001.stdout:3087 punch from 0x62e6f7a to 0x62f4e8b, (0xdf11 bytes) 2024-09-14T22:37:11.474 INFO:teuthology.orchestra.run.smithi001.stdout:3088 write 0x7a2bf3 thru 0x7b0780 (0xdb8e bytes) 2024-09-14T22:37:11.482 INFO:teuthology.orchestra.run.smithi001.stdout:3089 writesame 0x8e794ce thru 0x8e7ebfd data_size 0x12(0x5730 bytes) 2024-09-14T22:37:11.487 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:11.483+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:11.488 INFO:teuthology.orchestra.run.smithi001.stdout:3090 write 0x1be46cc thru 0x1beffa2 (0xb8d7 bytes) 2024-09-14T22:37:11.494 INFO:teuthology.orchestra.run.smithi001.stdout:3091 write 0x68a6854 thru 0x68b02a2 (0x9a4f bytes) 2024-09-14T22:37:11.499 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:11.499 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:11.501 INFO:teuthology.orchestra.run.smithi001.stdout:3093 read 0xda46d6d thru 0xda4bf1d (0x51b1 bytes) 2024-09-14T22:37:11.501 INFO:teuthology.orchestra.run.smithi001.stdout:3094 write 0x1ed771e thru 0x1edc5b5 (0x4e98 bytes) 2024-09-14T22:37:11.508 INFO:teuthology.orchestra.run.smithi001.stdout:3096 punch from 0xdd7fddb to 0xdd8a952, (0xab77 bytes) 2024-09-14T22:37:11.515 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:37:11.524 INFO:teuthology.orchestra.run.smithi001.stdout:3097 trunc from 0xeaea4ac to 0x242ec26 2024-09-14T22:37:11.559 INFO:teuthology.orchestra.run.smithi001.stdout:3098 trunc from 0x242ec26 to 0x573518d 2024-09-14T22:37:11.565 INFO:teuthology.orchestra.run.smithi001.stdout:3099 write 0xd59516 thru 0xd5e40a (0x4ef5 bytes) 2024-09-14T22:37:11.569 INFO:teuthology.orchestra.run.smithi001.stdout:3100 punch from 0x5018f7c to 0x501f364, (0x63e8 bytes) 2024-09-14T22:37:11.569 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:11.569 INFO:teuthology.orchestra.run.smithi001.stdout:3104 read 0x8cffb thru 0x8d053 (0x59 bytes) 2024-09-14T22:37:11.570 INFO:teuthology.orchestra.run.smithi001.stdout:3105 read 0x2e77e thru 0x3a3c9 (0xbc4c bytes) 2024-09-14T22:37:11.574 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:11.574 INFO:teuthology.orchestra.run.smithi001.stdout:3108 punch from 0x4cf1687 to 0x4cf58b3, (0x422c bytes) 2024-09-14T22:37:11.574 INFO:teuthology.orchestra.run.smithi001.stdout:3109 read 0x1ee3465 thru 0x1eece0c (0x99a8 bytes) 2024-09-14T22:37:11.585 INFO:teuthology.orchestra.run.smithi001.stdout:3111 writesame 0x7d22638 thru 0x7d2e37c data_size 0x1f(0xbd45 bytes) 2024-09-14T22:37:11.585 INFO:teuthology.orchestra.run.smithi001.stdout:3112 punch from 0x37233f3 to 0x3725af2, (0x26ff bytes) 2024-09-14T22:37:11.585 INFO:teuthology.orchestra.run.smithi001.stdout:3113 punch from 0x531fd1e to 0x532ab5f, (0xae41 bytes) 2024-09-14T22:37:11.585 INFO:teuthology.orchestra.run.smithi001.stdout:3114 read 0x4ddc8f3 thru 0x4de1b1b (0x5229 bytes) 2024-09-14T22:37:11.585 INFO:teuthology.orchestra.run.smithi001.stdout:3115 write 0x63e8828 thru 0x63e8b95 (0x36e bytes) 2024-09-14T22:37:11.587 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:11.583+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:11.590 INFO:teuthology.orchestra.run.smithi001.stdout:3117 punch from 0x3a9dc89 to 0x3aab27e, (0xd5f5 bytes) 2024-09-14T22:37:11.593 INFO:teuthology.orchestra.run.smithi001.stdout:3119 trunc from 0x7d2e37d to 0x9f99d6a 2024-09-14T22:37:11.605 INFO:teuthology.orchestra.run.smithi001.stdout:3120 trunc from 0x9f99d6a to 0xdb0f024 2024-09-14T22:37:11.613 INFO:teuthology.orchestra.run.smithi001.stdout:3122 read 0xb0ee2c1 thru 0xb0f4126 (0x5e66 bytes) 2024-09-14T22:37:11.613 INFO:teuthology.orchestra.run.smithi001.stdout:3124 read 0x227b54c thru 0x228471c (0x91d1 bytes) 2024-09-14T22:37:11.613 INFO:teuthology.orchestra.run.smithi001.stdout:3125 read 0x509bd9 thru 0x516815 (0xcc3d bytes) 2024-09-14T22:37:11.613 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:11.613 INFO:teuthology.orchestra.run.smithi001.stdout:3127 punch from 0x645a8e to 0x651c91, (0xc203 bytes) 2024-09-14T22:37:11.613 INFO:teuthology.orchestra.run.smithi001.stdout:3128 writesame 0x74d6284 thru 0x74e4607 data_size 0x2(0xe384 bytes) 2024-09-14T22:37:11.615 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2024-09-14T22:37:11.654 INFO:teuthology.orchestra.run.smithi001.stdout:3129 trunc from 0xdb0f024 to 0xd0270f2 2024-09-14T22:37:11.657 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:11.655+0000 7f739e12a640 -1 osd.2 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T22:37:11.660 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:11.660 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:11.665 INFO:teuthology.orchestra.run.smithi001.stdout:3130 punch from 0x7b017de to 0x7b0e9ed, (0xd20f bytes) 2024-09-14T22:37:11.674 INFO:teuthology.orchestra.run.smithi001.stdout:3131 write 0xeb2f09e thru 0xeb37bf9 (0x8b5c bytes) 2024-09-14T22:37:11.676 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:37:11.690 INFO:teuthology.orchestra.run.smithi001.stdout:3132 trunc from 0xeb37bfa to 0x7394af6 2024-09-14T22:37:11.704 INFO:teuthology.orchestra.run.smithi001.stdout:3133 read 0x6f67a6b thru 0x6f70e3f (0x93d5 bytes) 2024-09-14T22:37:11.715 INFO:teuthology.orchestra.run.smithi001.stdout:3134 write 0xaf3bab1 thru 0xaf3ea5b (0x2fab bytes) 2024-09-14T22:37:11.719 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:11.719 INFO:teuthology.orchestra.run.smithi001.stdout:3138 punch from 0x411a4a to 0x41dbd6, (0xc18c bytes) 2024-09-14T22:37:11.723 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:11.719+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:11.728 INFO:teuthology.orchestra.run.smithi001.stdout:3140 writesame 0xcbab855 thru 0xcbade7e data_size 0x5(0x262a bytes) 2024-09-14T22:37:11.728 INFO:teuthology.orchestra.run.smithi001.stdout:3142 write 0x404bbf4 thru 0x4055e72 (0xa27f bytes) 2024-09-14T22:37:11.733 INFO:teuthology.orchestra.run.smithi001.stdout:3144 write 0xce6059 thru 0xcf180a (0xb7b2 bytes) 2024-09-14T22:37:11.748 INFO:teuthology.orchestra.run.smithi001.stdout:3145 write 0x3226e9a thru 0x322b2dc (0x4443 bytes) 2024-09-14T22:37:11.752 INFO:teuthology.orchestra.run.smithi001.stdout:3146 writesame 0xbdc6ac7 thru 0xbdcab3e data_size 0x4(0x4078 bytes) 2024-09-14T22:37:11.758 INFO:teuthology.orchestra.run.smithi001.stdout:3147 writesame 0x3e9c2a thru 0x3f1809 data_size 0x10(0x7be0 bytes) 2024-09-14T22:37:11.771 INFO:teuthology.orchestra.run.smithi001.stdout:3149 punch from 0x31f528a to 0x31f6964, (0x16da bytes) 2024-09-14T22:37:11.771 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:11.771 INFO:teuthology.orchestra.run.smithi001.stdout:3151 read 0x72ddca7 thru 0x72dde7d (0x1d7 bytes) 2024-09-14T22:37:11.772 INFO:teuthology.orchestra.run.smithi001.stdout:3152 read 0xa8dad23 thru 0xa8dee13 (0x40f1 bytes) 2024-09-14T22:37:11.772 INFO:teuthology.orchestra.run.smithi001.stdout:3158 write 0x66a20dd thru 0x66ad1ef (0xb113 bytes) 2024-09-14T22:37:11.776 INFO:teuthology.orchestra.run.smithi001.stdout:3159 write 0x9a3a2de thru 0x9a40b4a (0x686d bytes) 2024-09-14T22:37:11.782 INFO:teuthology.orchestra.run.smithi001.stdout:3161 punch from 0x888f467 to 0x8893566, (0x40ff bytes) 2024-09-14T22:37:11.782 INFO:teuthology.orchestra.run.smithi001.stdout:3162 read 0xb2bcb07 thru 0xb2c39ef (0x6ee9 bytes) 2024-09-14T22:37:11.782 INFO:teuthology.orchestra.run.smithi001.stdout:3163 punch from 0x8bb2c69 to 0x8bb523e, (0x25d5 bytes) 2024-09-14T22:37:11.782 INFO:teuthology.orchestra.run.smithi001.stdout:3164 read 0xb774ae7 thru 0xb77dbdd (0x90f7 bytes) 2024-09-14T22:37:11.782 INFO:teuthology.orchestra.run.smithi001.stdout:3165 read 0x6b1dc5d thru 0x6b29502 (0xb8a6 bytes) 2024-09-14T22:37:11.789 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:11.783+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:11.790 INFO:teuthology.orchestra.run.smithi001.stdout:3166 writesame 0xdee682b thru 0xdef2c6f data_size 0xd(0xc445 bytes) 2024-09-14T22:37:11.797 INFO:teuthology.orchestra.run.smithi001.stdout:3167 trunc from 0xdef2c70 to 0x9729386 2024-09-14T22:37:11.811 INFO:teuthology.orchestra.run.smithi001.stdout:3169 read 0x36f0eed thru 0x36f232a (0x143e bytes) 2024-09-14T22:37:11.811 INFO:teuthology.orchestra.run.smithi001.stdout:3170 read 0x6ceae5f thru 0x6cf0ae7 (0x5c89 bytes) 2024-09-14T22:37:11.819 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:11.819 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:11.831 INFO:teuthology.orchestra.run.smithi001.stdout:3171 write 0xdbfa321 thru 0xdbfc2b6 (0x1f96 bytes) 2024-09-14T22:37:11.833 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:37:11.835 INFO:teuthology.orchestra.run.smithi001.stdout:3173 write 0x89b8ea8 thru 0x89c876c (0xf8c5 bytes) 2024-09-14T22:37:11.841 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:11.841 INFO:teuthology.orchestra.run.smithi001.stdout:3176 punch from 0x8ae1706 to 0x8ae6bf9, (0x54f3 bytes) 2024-09-14T22:37:11.842 INFO:teuthology.orchestra.run.smithi001.stdout:3178 writesame 0x6b7c741 thru 0x6b7de28 data_size 0x4(0x16e8 bytes) 2024-09-14T22:37:11.846 INFO:teuthology.orchestra.run.smithi001.stdout:3179 write 0x582bd5a thru 0x582bf26 (0x1cd bytes) 2024-09-14T22:37:11.850 INFO:teuthology.orchestra.run.smithi001.stdout:3180 writesame 0xb208480 thru 0xb213996 data_size 0x65(0xb517 bytes) 2024-09-14T22:37:11.856 INFO:teuthology.orchestra.run.smithi001.stdout:3181 read 0x642d44c thru 0x6433481 (0x6036 bytes) 2024-09-14T22:37:11.856 INFO:teuthology.orchestra.run.smithi001.stdout:3183 punch from 0x1e18bf8 to 0x1e22019, (0x9421 bytes) 2024-09-14T22:37:11.856 INFO:teuthology.orchestra.run.smithi001.stdout:3185 writesame 0xa4a86e8 thru 0xa4b37fc data_size 0x1b(0xb115 bytes) 2024-09-14T22:37:11.862 INFO:teuthology.orchestra.run.smithi001.stdout:3186 read 0x2ef2c1a thru 0x2ef9381 (0x6768 bytes) 2024-09-14T22:37:11.862 INFO:teuthology.orchestra.run.smithi001.stdout:3187 punch from 0x30c6ec8 to 0x30cb6fb, (0x4833 bytes) 2024-09-14T22:37:11.862 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:11.862 INFO:teuthology.orchestra.run.smithi001.stdout:3190 write 0xd5848a thru 0xd62231 (0x9da8 bytes) 2024-09-14T22:37:11.890 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:11.887+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:11.894 INFO:teuthology.orchestra.run.smithi001.stdout:3191 trunc from 0xdbfc2b7 to 0x4b0e6a0 2024-09-14T22:37:11.915 INFO:teuthology.orchestra.run.smithi001.stdout:3192 read 0x3e048f8 thru 0x3e06ea0 (0x25a9 bytes) 2024-09-14T22:37:11.915 INFO:teuthology.orchestra.run.smithi001.stdout:3194 read 0xa09f34 thru 0xa19b55 (0xfc22 bytes) 2024-09-14T22:37:11.935 INFO:teuthology.orchestra.run.smithi001.stdout:3195 trunc from 0x4b0e6a0 to 0xd26d184 2024-09-14T22:37:11.945 INFO:teuthology.orchestra.run.smithi001.stdout:3196 flatten 2024-09-14T22:37:11.991 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:11.987+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:12.013 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:12.013 INFO:teuthology.orchestra.run.smithi001.stdout:3198 read 0x6547ecf thru 0x654ef0a (0x703c bytes) 2024-09-14T22:37:12.013 INFO:teuthology.orchestra.run.smithi001.stdout:3199 write 0x8a3e7a2 thru 0x8a46837 (0x8096 bytes) 2024-09-14T22:37:12.018 INFO:teuthology.orchestra.run.smithi001.stdout:3200 writesame 0x8e53010 thru 0x8e6091b data_size 0x3a(0xd90c bytes) 2024-09-14T22:37:12.020 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:12.020 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:12.024 INFO:teuthology.orchestra.run.smithi001.stdout:3201 read 0x66b809d thru 0x66c7447 (0xf3ab bytes) 2024-09-14T22:37:12.024 INFO:teuthology.orchestra.run.smithi001.stdout:3204 write 0xbca1773 thru 0xbcae0ff (0xc98d bytes) 2024-09-14T22:37:12.028 INFO:teuthology.orchestra.run.smithi001.stdout:3205 punch from 0x27278ca to 0x2732c4e, (0xb384 bytes) 2024-09-14T22:37:12.028 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:12.028 INFO:teuthology.orchestra.run.smithi001.stdout:3207 write 0xb366f27 thru 0xb37422e (0xd308 bytes) 2024-09-14T22:37:12.035 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:37:12.036 INFO:teuthology.orchestra.run.smithi001.stdout:3208 read 0xaf74383 thru 0xaf7dae7 (0x9765 bytes) 2024-09-14T22:37:12.036 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:12.036 INFO:teuthology.orchestra.run.smithi001.stdout:3210 read 0x2aef5c thru 0x2b409b (0x5140 bytes) 2024-09-14T22:37:12.041 INFO:teuthology.orchestra.run.smithi001.stdout:3211 trunc from 0xd26d184 to 0xa9f19a7 2024-09-14T22:37:12.068 INFO:teuthology.orchestra.run.smithi001.stdout:3212 write 0xabc2595 thru 0xabd2088 (0xfaf4 bytes) 2024-09-14T22:37:12.075 INFO:teuthology.orchestra.run.smithi001.stdout:3213 read 0x703e362 thru 0x7041870 (0x350f bytes) 2024-09-14T22:37:12.076 INFO:teuthology.orchestra.run.smithi001.stdout:3214 read 0x83fa636 thru 0x840380b (0x91d6 bytes) 2024-09-14T22:37:12.076 INFO:teuthology.orchestra.run.smithi001.stdout:3215 write 0x2f9ae12 thru 0x2f9dcef (0x2ede bytes) 2024-09-14T22:37:12.080 INFO:teuthology.orchestra.run.smithi001.stdout:3217 punch from 0x9de8110 to 0x9deb28d, (0x317d bytes) 2024-09-14T22:37:12.091 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:12.087+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:12.093 INFO:teuthology.orchestra.run.smithi001.stdout:3218 writesame 0xbf4d4a3 thru 0xbf4f993 data_size 0x7(0x24f1 bytes) 2024-09-14T22:37:12.093 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:12.093 INFO:teuthology.orchestra.run.smithi001.stdout:3220 punch from 0x6f97feb to 0x6fa6ea8, (0xeebd bytes) 2024-09-14T22:37:12.093 INFO:teuthology.orchestra.run.smithi001.stdout:3221 read 0x1de9c58 thru 0x1df19cc (0x7d75 bytes) 2024-09-14T22:37:12.093 INFO:teuthology.orchestra.run.smithi001.stdout:3222 write 0xaf12109 thru 0xaf130e5 (0xfdd bytes) 2024-09-14T22:37:12.098 INFO:teuthology.orchestra.run.smithi001.stdout:3223 read 0x7da0ddf thru 0x7daed4b (0xdf6d bytes) 2024-09-14T22:37:12.098 INFO:teuthology.orchestra.run.smithi001.stdout:3224 write 0x7772f02 thru 0x777edc5 (0xbec4 bytes) 2024-09-14T22:37:12.104 INFO:teuthology.orchestra.run.smithi001.stdout:3225 write 0x25131fd thru 0x2513bba (0x9be bytes) 2024-09-14T22:37:12.106 INFO:teuthology.orchestra.run.smithi001.stdout:3227 punch from 0x254b3dd to 0x25588bb, (0xd4de bytes) 2024-09-14T22:37:12.112 INFO:teuthology.orchestra.run.smithi001.stdout:3228 trunc from 0xbf4f994 to 0x99e6491 2024-09-14T22:37:12.132 INFO:teuthology.orchestra.run.smithi001.stdout:3229 punch from 0x2752c7 to 0x282f8b, (0xdcc4 bytes) 2024-09-14T22:37:12.133 INFO:teuthology.orchestra.run.smithi001.stdout:3230 read 0x7b85ca2 thru 0x7b8d4e4 (0x7843 bytes) 2024-09-14T22:37:12.133 INFO:teuthology.orchestra.run.smithi001.stdout:3231 punch from 0x100ecd1 to 0x101610a, (0x7439 bytes) 2024-09-14T22:37:12.135 INFO:teuthology.orchestra.run.smithi001.stdout:3233 trunc from 0x99e6491 to 0xa7fbda6 2024-09-14T22:37:12.149 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:12.149 INFO:teuthology.orchestra.run.smithi001.stdout:3236 write 0x9c1e872 thru 0x9c26d7a (0x8509 bytes) 2024-09-14T22:37:12.158 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:12.158 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:12.159 INFO:teuthology.orchestra.run.smithi001.stdout:3237 writesame 0x2d4cff6 thru 0x2d50206 data_size 0x1(0x3211 bytes) 2024-09-14T22:37:12.174 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:37:12.177 INFO:teuthology.orchestra.run.smithi001.stdout:3238 clone 21 order 24 su 65536 sc 8 2024-09-14T22:37:12.215 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:12.211+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:12.293 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:12.291+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:12.337 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:12.337 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:12.351 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:37:12.393 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:12.391+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:12.470 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:12.470 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:12.484 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:37:12.494 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:12.491+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:12.595 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:12.591+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:12.602 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:12.602 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:12.616 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:37:12.696 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:12.691+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:12.782 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:12.782 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:12.796 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:37:12.797 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:12.791+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:12.899 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:12.895+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:12.914 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:12.914 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:12.929 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:37:13.000 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:12.995+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:13.051 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:13.052 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:13.066 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:37:13.088 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:37:13.133 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:13.131+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:13.202 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:13.195+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:13.230 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:13.230 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:13.244 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:37:13.331 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:13.327+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:13.404 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:13.399+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:13.411 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:13.411 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:13.425 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:37:13.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-14T22:37:13.423+0000 7ff72affd640 0 --2- 172.21.15.1:0/3814488637 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff72c13c4b0 0x7ff72c145140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-14T22:37:13.451 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:37:13.451 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:37:13.477 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:37:13.477 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:37:13.501 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:37:13.502 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-14T22:37:13.505 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:13.503+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:13.530 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:37:13.530 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:37:13.555 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:37:13.555 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:37:13.583 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:37:13.583 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:37:13.593 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:13.593 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:13.606 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:13.599+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:13.608 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:37:13.611 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:37:13.612 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:37:13.637 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T22:37:13.641 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:37:13.641 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:37:13.707 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:13.703+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:13.744 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:13.744 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:37:13.744 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:37:13.745 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:13.759 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:37:13.849 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:13.847+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:13.909 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:13.907+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:13.919 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:13.919 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:37:13.919 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:37:13.919 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:37:13.919 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:13.934 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:37:14.010 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:14.003+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:14.105 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:14.105 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:37:14.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:37:14.106 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:37:14.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.106 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3022 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14301184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:14.106 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559594+0000", 2024-09-14T22:37:14.106 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.543392963999999, 2024-09-14T22:37:14.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028916481000000001, 2024-09-14T22:37:14.106 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.106 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.106 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.106 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.106 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.106 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.107 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3022 2024-09-14T22:37:14.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.107 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559594+0000", 2024-09-14T22:37:14.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559594+0000", 2024-09-14T22:37:14.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559594+0000", 2024-09-14T22:37:14.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7000000000000004e-07 2024-09-14T22:37:14.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559604+0000", 2024-09-14T22:37:14.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6020000000000006e-06 2024-09-14T22:37:14.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559615+0000", 2024-09-14T22:37:14.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1032999999999999e-05 2024-09-14T22:37:14.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559638+0000", 2024-09-14T22:37:14.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3255999999999999e-05 2024-09-14T22:37:14.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588323+0000", 2024-09-14T22:37:14.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.02868447 2024-09-14T22:37:14.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588443+0000", 2024-09-14T22:37:14.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120056 2024-09-14T22:37:14.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588510+0000", 2024-09-14T22:37:14.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7393999999999999e-05 2024-09-14T22:37:14.110 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.110 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.110 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.111 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3023 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14309376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:14.111 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559651+0000", 2024-09-14T22:37:14.111 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.543335478000003, 2024-09-14T22:37:14.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029146231000000002, 2024-09-14T22:37:14.111 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.111 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.111 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.111 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.111 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.111 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.111 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3023 2024-09-14T22:37:14.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.111 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559651+0000", 2024-09-14T22:37:14.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559651+0000", 2024-09-14T22:37:14.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559652+0000", 2024-09-14T22:37:14.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2699999999999998e-07 2024-09-14T22:37:14.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559656+0000", 2024-09-14T22:37:14.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7019999999999999e-06 2024-09-14T22:37:14.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559666+0000", 2024-09-14T22:37:14.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0400999999999999e-05 2024-09-14T22:37:14.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559674+0000", 2024-09-14T22:37:14.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8890000000000002e-06 2024-09-14T22:37:14.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588565+0000", 2024-09-14T22:37:14.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028891217 2024-09-14T22:37:14.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588679+0000", 2024-09-14T22:37:14.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114058 2024-09-14T22:37:14.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588797+0000", 2024-09-14T22:37:14.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118137 2024-09-14T22:37:14.115 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.115 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.115 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.115 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3024 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14317568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:14.116 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559685+0000", 2024-09-14T22:37:14.116 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.543301366999998, 2024-09-14T22:37:14.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029326498999999999, 2024-09-14T22:37:14.116 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.116 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.116 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.116 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.116 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.116 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.116 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3024 2024-09-14T22:37:14.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.116 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559685+0000", 2024-09-14T22:37:14.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559685+0000", 2024-09-14T22:37:14.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559686+0000", 2024-09-14T22:37:14.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0999999999999998e-07 2024-09-14T22:37:14.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559697+0000", 2024-09-14T22:37:14.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.147e-05 2024-09-14T22:37:14.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559708+0000", 2024-09-14T22:37:14.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0546e-05 2024-09-14T22:37:14.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559716+0000", 2024-09-14T22:37:14.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.704e-06 2024-09-14T22:37:14.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588833+0000", 2024-09-14T22:37:14.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029116442999999999 2024-09-14T22:37:14.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588920+0000", 2024-09-14T22:37:14.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6916999999999995e-05 2024-09-14T22:37:14.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589012+0000", 2024-09-14T22:37:14.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1809000000000001e-05 2024-09-14T22:37:14.120 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.120 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.120 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.120 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3025 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14325760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:14.120 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559729+0000", 2024-09-14T22:37:14.120 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.543257468, 2024-09-14T22:37:14.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029422501, 2024-09-14T22:37:14.120 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.120 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.120 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.121 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.121 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.121 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.121 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3025 2024-09-14T22:37:14.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.121 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559729+0000", 2024-09-14T22:37:14.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559729+0000", 2024-09-14T22:37:14.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.122 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559730+0000", 2024-09-14T22:37:14.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8100000000000002e-07 2024-09-14T22:37:14.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.122 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559733+0000", 2024-09-14T22:37:14.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4740000000000001e-06 2024-09-14T22:37:14.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559744+0000", 2024-09-14T22:37:14.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0489000000000001e-05 2024-09-14T22:37:14.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559760+0000", 2024-09-14T22:37:14.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5817000000000001e-05 2024-09-14T22:37:14.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589050+0000", 2024-09-14T22:37:14.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.02929056 2024-09-14T22:37:14.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589101+0000", 2024-09-14T22:37:14.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0992000000000002e-05 2024-09-14T22:37:14.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589152+0000", 2024-09-14T22:37:14.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0387999999999999e-05 2024-09-14T22:37:14.124 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.124 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.124 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.124 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3026 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14333952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:14.125 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559795+0000", 2024-09-14T22:37:14.125 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.543191643999997, 2024-09-14T22:37:14.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029483812000000002, 2024-09-14T22:37:14.125 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.125 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.125 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.125 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.125 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.125 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.125 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3026 2024-09-14T22:37:14.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.125 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559795+0000", 2024-09-14T22:37:14.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559795+0000", 2024-09-14T22:37:14.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559796+0000", 2024-09-14T22:37:14.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3200000000000003e-07 2024-09-14T22:37:14.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559800+0000", 2024-09-14T22:37:14.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2930000000000002e-06 2024-09-14T22:37:14.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559812+0000", 2024-09-14T22:37:14.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2053e-05 2024-09-14T22:37:14.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559821+0000", 2024-09-14T22:37:14.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8869999999999998e-06 2024-09-14T22:37:14.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589177+0000", 2024-09-14T22:37:14.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029356308000000001 2024-09-14T22:37:14.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589229+0000", 2024-09-14T22:37:14.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1326000000000003e-05 2024-09-14T22:37:14.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589279+0000", 2024-09-14T22:37:14.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0012999999999997e-05 2024-09-14T22:37:14.129 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.129 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.129 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.129 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3027 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14342144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:14.129 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559833+0000", 2024-09-14T22:37:14.129 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.543153916000001, 2024-09-14T22:37:14.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029619187000000002, 2024-09-14T22:37:14.129 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.130 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.130 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.130 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.130 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.130 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.130 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3027 2024-09-14T22:37:14.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.130 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559833+0000", 2024-09-14T22:37:14.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559833+0000", 2024-09-14T22:37:14.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559833+0000", 2024-09-14T22:37:14.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9400000000000005e-07 2024-09-14T22:37:14.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559845+0000", 2024-09-14T22:37:14.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1623e-05 2024-09-14T22:37:14.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559856+0000", 2024-09-14T22:37:14.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0832999999999999e-05 2024-09-14T22:37:14.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559864+0000", 2024-09-14T22:37:14.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3119999999999994e-06 2024-09-14T22:37:14.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589299+0000", 2024-09-14T22:37:14.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029434699000000002 2024-09-14T22:37:14.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.133 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.133 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589354+0000", 2024-09-14T22:37:14.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5467999999999998e-05 2024-09-14T22:37:14.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.133 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.133 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589452+0000", 2024-09-14T22:37:14.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7558000000000003e-05 2024-09-14T22:37:14.133 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.133 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.134 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.134 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3028 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14350336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:14.134 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559875+0000", 2024-09-14T22:37:14.134 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.543111027999998, 2024-09-14T22:37:14.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029775586999999999, 2024-09-14T22:37:14.134 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.134 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.134 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.134 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.134 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.134 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.134 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3028 2024-09-14T22:37:14.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.135 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559875+0000", 2024-09-14T22:37:14.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559875+0000", 2024-09-14T22:37:14.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559876+0000", 2024-09-14T22:37:14.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5199999999999996e-07 2024-09-14T22:37:14.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559879+0000", 2024-09-14T22:37:14.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.095e-06 2024-09-14T22:37:14.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559889+0000", 2024-09-14T22:37:14.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0019e-05 2024-09-14T22:37:14.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559897+0000", 2024-09-14T22:37:14.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9189999999999999e-06 2024-09-14T22:37:14.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589486+0000", 2024-09-14T22:37:14.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029588482999999999 2024-09-14T22:37:14.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.138 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.138 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589578+0000", 2024-09-14T22:37:14.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2329000000000003e-05 2024-09-14T22:37:14.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.138 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.138 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589651+0000", 2024-09-14T22:37:14.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2990000000000004e-05 2024-09-14T22:37:14.138 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.138 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.138 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.139 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4068 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13252608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:14.139 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680102+0000", 2024-09-14T22:37:14.139 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.422884209000003, 2024-09-14T22:37:14.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028907656, 2024-09-14T22:37:14.139 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.139 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.139 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.139 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.139 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.139 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.139 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4068 2024-09-14T22:37:14.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.140 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680102+0000", 2024-09-14T22:37:14.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680102+0000", 2024-09-14T22:37:14.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680103+0000", 2024-09-14T22:37:14.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6600000000000002e-07 2024-09-14T22:37:14.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680105+0000", 2024-09-14T22:37:14.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.616e-06 2024-09-14T22:37:14.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680114+0000", 2024-09-14T22:37:14.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4120000000000001e-06 2024-09-14T22:37:14.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680119+0000", 2024-09-14T22:37:14.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6210000000000001e-06 2024-09-14T22:37:14.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.708813+0000", 2024-09-14T22:37:14.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028693330999999999 2024-09-14T22:37:14.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.708903+0000", 2024-09-14T22:37:14.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0297000000000006e-05 2024-09-14T22:37:14.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709010+0000", 2024-09-14T22:37:14.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106913 2024-09-14T22:37:14.143 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.143 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.143 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.143 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4069 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13260800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:14.143 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680128+0000", 2024-09-14T22:37:14.143 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.422858933999997, 2024-09-14T22:37:14.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029145476, 2024-09-14T22:37:14.143 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.144 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.144 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.144 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.144 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.144 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.144 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4069 2024-09-14T22:37:14.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.144 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680128+0000", 2024-09-14T22:37:14.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680128+0000", 2024-09-14T22:37:14.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680128+0000", 2024-09-14T22:37:14.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7599999999999998e-07 2024-09-14T22:37:14.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680131+0000", 2024-09-14T22:37:14.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9050000000000001e-06 2024-09-14T22:37:14.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680138+0000", 2024-09-14T22:37:14.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2559999999999996e-06 2024-09-14T22:37:14.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680144+0000", 2024-09-14T22:37:14.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4779999999999999e-06 2024-09-14T22:37:14.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709054+0000", 2024-09-14T22:37:14.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028910207 2024-09-14T22:37:14.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709180+0000", 2024-09-14T22:37:14.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012608699999999999 2024-09-14T22:37:14.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709273+0000", 2024-09-14T22:37:14.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3166999999999998e-05 2024-09-14T22:37:14.147 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.147 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.148 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.148 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4070 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13268992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:14.148 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680151+0000", 2024-09-14T22:37:14.148 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.422835652000003, 2024-09-14T22:37:14.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029327986, 2024-09-14T22:37:14.148 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.148 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.148 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.148 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.148 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.148 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.148 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4070 2024-09-14T22:37:14.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.149 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680151+0000", 2024-09-14T22:37:14.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680151+0000", 2024-09-14T22:37:14.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680151+0000", 2024-09-14T22:37:14.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:37:14.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680154+0000", 2024-09-14T22:37:14.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3350000000000001e-06 2024-09-14T22:37:14.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680162+0000", 2024-09-14T22:37:14.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.473e-06 2024-09-14T22:37:14.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680177+0000", 2024-09-14T22:37:14.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.465e-05 2024-09-14T22:37:14.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709356+0000", 2024-09-14T22:37:14.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029178744 2024-09-14T22:37:14.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709428+0000", 2024-09-14T22:37:14.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2428000000000005e-05 2024-09-14T22:37:14.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.152 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709479+0000", 2024-09-14T22:37:14.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0915999999999998e-05 2024-09-14T22:37:14.152 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.152 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.152 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.152 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4071 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13277184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:14.153 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680185+0000", 2024-09-14T22:37:14.153 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.422801151999998, 2024-09-14T22:37:14.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029452392000000001, 2024-09-14T22:37:14.153 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.153 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.153 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.153 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.153 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.153 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.153 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4071 2024-09-14T22:37:14.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.154 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680185+0000", 2024-09-14T22:37:14.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680185+0000", 2024-09-14T22:37:14.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680186+0000", 2024-09-14T22:37:14.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6100000000000001e-07 2024-09-14T22:37:14.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680188+0000", 2024-09-14T22:37:14.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.509e-06 2024-09-14T22:37:14.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680197+0000", 2024-09-14T22:37:14.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8550000000000008e-06 2024-09-14T22:37:14.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680203+0000", 2024-09-14T22:37:14.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5849999999999999e-06 2024-09-14T22:37:14.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709514+0000", 2024-09-14T22:37:14.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029311054 2024-09-14T22:37:14.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.157 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.157 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709585+0000", 2024-09-14T22:37:14.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0875000000000002e-05 2024-09-14T22:37:14.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.157 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.157 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709638+0000", 2024-09-14T22:37:14.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3053000000000002e-05 2024-09-14T22:37:14.157 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.157 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.157 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.158 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4072 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13285376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:14.158 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680211+0000", 2024-09-14T22:37:14.158 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.422775870999999, 2024-09-14T22:37:14.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029591177999999999, 2024-09-14T22:37:14.158 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.158 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.158 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.158 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.158 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.158 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.158 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4072 2024-09-14T22:37:14.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.158 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680211+0000", 2024-09-14T22:37:14.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680211+0000", 2024-09-14T22:37:14.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680211+0000", 2024-09-14T22:37:14.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6699999999999999e-07 2024-09-14T22:37:14.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680214+0000", 2024-09-14T22:37:14.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0149999999999999e-06 2024-09-14T22:37:14.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680222+0000", 2024-09-14T22:37:14.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.371e-06 2024-09-14T22:37:14.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680228+0000", 2024-09-14T22:37:14.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6230000000000002e-06 2024-09-14T22:37:14.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.161 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.161 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709670+0000", 2024-09-14T22:37:14.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029441694000000001 2024-09-14T22:37:14.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.161 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.161 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709741+0000", 2024-09-14T22:37:14.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1668999999999994e-05 2024-09-14T22:37:14.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709802+0000", 2024-09-14T22:37:14.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0338999999999999e-05 2024-09-14T22:37:14.162 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.162 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.162 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.162 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4073 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13293568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:14.162 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680236+0000", 2024-09-14T22:37:14.162 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.422750512, 2024-09-14T22:37:14.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029755238999999999, 2024-09-14T22:37:14.162 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.162 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.163 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.163 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.163 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.163 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.163 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4073 2024-09-14T22:37:14.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.163 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680236+0000", 2024-09-14T22:37:14.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680236+0000", 2024-09-14T22:37:14.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680236+0000", 2024-09-14T22:37:14.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5499999999999998e-07 2024-09-14T22:37:14.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680239+0000", 2024-09-14T22:37:14.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4380000000000002e-06 2024-09-14T22:37:14.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.165 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680248+0000", 2024-09-14T22:37:14.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6319999999999997e-06 2024-09-14T22:37:14.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.165 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.165 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680261+0000", 2024-09-14T22:37:14.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3193000000000001e-05 2024-09-14T22:37:14.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.165 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.165 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709831+0000", 2024-09-14T22:37:14.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029570486 2024-09-14T22:37:14.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.166 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.166 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709883+0000", 2024-09-14T22:37:14.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2290999999999998e-05 2024-09-14T22:37:14.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.166 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.166 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709991+0000", 2024-09-14T22:37:14.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107744 2024-09-14T22:37:14.166 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.166 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.166 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.166 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4074 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13301760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:14.166 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680269+0000", 2024-09-14T22:37:14.167 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.422717614, 2024-09-14T22:37:14.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030007875999999999, 2024-09-14T22:37:14.167 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.167 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.167 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.167 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.167 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.167 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.167 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4074 2024-09-14T22:37:14.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.167 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680269+0000", 2024-09-14T22:37:14.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680269+0000", 2024-09-14T22:37:14.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680269+0000", 2024-09-14T22:37:14.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7300000000000001e-07 2024-09-14T22:37:14.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680277+0000", 2024-09-14T22:37:14.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8480000000000001e-06 2024-09-14T22:37:14.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680286+0000", 2024-09-14T22:37:14.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0529999999999996e-06 2024-09-14T22:37:14.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680292+0000", 2024-09-14T22:37:14.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.728e-06 2024-09-14T22:37:14.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.710024+0000", 2024-09-14T22:37:14.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029732176999999999 2024-09-14T22:37:14.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.710109+0000", 2024-09-14T22:37:14.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4715000000000005e-05 2024-09-14T22:37:14.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.171 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.171 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.710277+0000", 2024-09-14T22:37:14.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000167882 2024-09-14T22:37:14.171 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.171 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.171 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.171 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:12664 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T22:37:14.171 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:28.325596+0000", 2024-09-14T22:37:14.171 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 45.777390099999998, 2024-09-14T22:37:14.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028964581, 2024-09-14T22:37:14.172 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.172 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.172 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.172 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.172 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.172 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.172 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12664 2024-09-14T22:37:14.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.172 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325596+0000", 2024-09-14T22:37:14.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325596+0000", 2024-09-14T22:37:14.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325597+0000", 2024-09-14T22:37:14.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4300000000000002e-07 2024-09-14T22:37:14.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325600+0000", 2024-09-14T22:37:14.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2219999999999998e-06 2024-09-14T22:37:14.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.174 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.174 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325610+0000", 2024-09-14T22:37:14.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2650000000000002e-06 2024-09-14T22:37:14.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.174 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.174 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325617+0000", 2024-09-14T22:37:14.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1199999999999996e-06 2024-09-14T22:37:14.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.175 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.175 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354438+0000", 2024-09-14T22:37:14.175 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028821205999999999 2024-09-14T22:37:14.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.175 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.175 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354495+0000", 2024-09-14T22:37:14.175 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6752999999999997e-05 2024-09-14T22:37:14.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.175 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.175 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354561+0000", 2024-09-14T22:37:14.175 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6272000000000005e-05 2024-09-14T22:37:14.175 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.176 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.176 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.176 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:12665 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 391168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T22:37:14.176 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:28.325626+0000", 2024-09-14T22:37:14.176 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 45.777360778000002, 2024-09-14T22:37:14.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029080366, 2024-09-14T22:37:14.176 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.176 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.176 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.176 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.176 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.176 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.177 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12665 2024-09-14T22:37:14.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.177 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325626+0000", 2024-09-14T22:37:14.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325626+0000", 2024-09-14T22:37:14.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325626+0000", 2024-09-14T22:37:14.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.06e-07 2024-09-14T22:37:14.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325629+0000", 2024-09-14T22:37:14.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7149999999999998e-06 2024-09-14T22:37:14.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325637+0000", 2024-09-14T22:37:14.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7360000000000001e-06 2024-09-14T22:37:14.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325643+0000", 2024-09-14T22:37:14.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2480000000000001e-06 2024-09-14T22:37:14.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354589+0000", 2024-09-14T22:37:14.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028945538 2024-09-14T22:37:14.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354635+0000", 2024-09-14T22:37:14.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6374e-05 2024-09-14T22:37:14.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354706+0000", 2024-09-14T22:37:14.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1049000000000004e-05 2024-09-14T22:37:14.180 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.180 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.180 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.180 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:13927 3.1b 3:ddfd6241:::rbd_data.113cda0183a0.0000000000000009:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-14T22:37:14.180 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:31.537348+0000", 2024-09-14T22:37:14.180 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.565638178, 2024-09-14T22:37:14.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032156573000000001, 2024-09-14T22:37:14.181 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.181 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.181 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:14.181 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.181 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.181 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.181 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 13927 2024-09-14T22:37:14.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.181 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537348+0000", 2024-09-14T22:37:14.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537348+0000", 2024-09-14T22:37:14.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537349+0000", 2024-09-14T22:37:14.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.068e-06 2024-09-14T22:37:14.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537358+0000", 2024-09-14T22:37:14.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.653e-06 2024-09-14T22:37:14.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537374+0000", 2024-09-14T22:37:14.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5594e-05 2024-09-14T22:37:14.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537383+0000", 2024-09-14T22:37:14.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0929999999999992e-06 2024-09-14T22:37:14.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.184 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.184 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537429+0000", 2024-09-14T22:37:14.184 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5948000000000001e-05 2024-09-14T22:37:14.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.184 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.184 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537490+0000", 2024-09-14T22:37:14.184 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1002000000000003e-05 2024-09-14T22:37:14.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.184 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 4", 2024-09-14T22:37:14.185 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.542781+0000", 2024-09-14T22:37:14.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.005290861 2024-09-14T22:37:14.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.185 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:14.185 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.569350+0000", 2024-09-14T22:37:14.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026569299000000001 2024-09-14T22:37:14.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.185 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:37:14.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.569375+0000", 2024-09-14T22:37:14.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5448000000000001e-05 2024-09-14T22:37:14.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.186 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:37:14.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.569444+0000", 2024-09-14T22:37:14.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9184999999999996e-05 2024-09-14T22:37:14.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.186 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.569505+0000", 2024-09-14T22:37:14.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0421999999999998e-05 2024-09-14T22:37:14.186 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.186 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.187 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.187 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:85875 3.12 3:498b1eba:::rbd_data.113cd62ea8a5.000000000000006a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:14.187 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:00.460932+0000", 2024-09-14T22:37:14.187 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.642054277, 2024-09-14T22:37:14.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044152693999999999, 2024-09-14T22:37:14.187 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.187 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.187 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:14.187 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.187 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.187 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.187 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 85875 2024-09-14T22:37:14.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.188 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.460932+0000", 2024-09-14T22:37:14.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.460932+0000", 2024-09-14T22:37:14.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.460935+0000", 2024-09-14T22:37:14.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6280000000000001e-06 2024-09-14T22:37:14.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.460947+0000", 2024-09-14T22:37:14.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2269000000000001e-05 2024-09-14T22:37:14.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.460973+0000", 2024-09-14T22:37:14.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5939999999999999e-05 2024-09-14T22:37:14.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.460985+0000", 2024-09-14T22:37:14.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.181e-05 2024-09-14T22:37:14.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.461102+0000", 2024-09-14T22:37:14.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000117413 2024-09-14T22:37:14.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.461349+0000", 2024-09-14T22:37:14.191 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00024703300000000002 2024-09-14T22:37:14.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.191 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 7", 2024-09-14T22:37:14.191 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.468711+0000", 2024-09-14T22:37:14.191 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0073619180000000003 2024-09-14T22:37:14.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.191 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:37:14.191 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.504629+0000", 2024-09-14T22:37:14.191 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035917273 2024-09-14T22:37:14.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:14.192 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.504776+0000", 2024-09-14T22:37:14.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014738300000000001 2024-09-14T22:37:14.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:37:14.192 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.504951+0000", 2024-09-14T22:37:14.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017548100000000001 2024-09-14T22:37:14.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.192 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.505085+0000", 2024-09-14T22:37:14.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013354600000000001 2024-09-14T22:37:14.192 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.193 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.193 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.193 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:85883 3.12 3:4cf65cce:::rbd_data.113cd62ea8a5.0000000000000015:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:14.193 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:00.524670+0000", 2024-09-14T22:37:14.193 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.578316253000001, 2024-09-14T22:37:14.193 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050663316, 2024-09-14T22:37:14.193 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.193 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.193 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:14.193 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.193 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.194 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.194 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 85883 2024-09-14T22:37:14.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.194 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.524670+0000", 2024-09-14T22:37:14.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.524670+0000", 2024-09-14T22:37:14.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.524672+0000", 2024-09-14T22:37:14.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1629999999999999e-06 2024-09-14T22:37:14.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.524688+0000", 2024-09-14T22:37:14.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5787e-05 2024-09-14T22:37:14.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.524720+0000", 2024-09-14T22:37:14.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1294000000000002e-05 2024-09-14T22:37:14.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.524736+0000", 2024-09-14T22:37:14.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.607e-05 2024-09-14T22:37:14.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.524863+0000", 2024-09-14T22:37:14.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012693399999999999 2024-09-14T22:37:14.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.197 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.197 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.525060+0000", 2024-09-14T22:37:14.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000197732 2024-09-14T22:37:14.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.197 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 7", 2024-09-14T22:37:14.197 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.532377+0000", 2024-09-14T22:37:14.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0073167680000000004 2024-09-14T22:37:14.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.197 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:14.197 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.575115+0000", 2024-09-14T22:37:14.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042737626000000001 2024-09-14T22:37:14.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:37:14.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.575151+0000", 2024-09-14T22:37:14.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6211000000000003e-05 2024-09-14T22:37:14.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:37:14.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.575245+0000", 2024-09-14T22:37:14.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4648999999999995e-05 2024-09-14T22:37:14.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.575334+0000", 2024-09-14T22:37:14.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8082000000000001e-05 2024-09-14T22:37:14.199 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.199 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.199 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.199 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:86178 3.1 3:83964f10:::rbd_data.113cd62ea8a5.0000000000000027:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:14.199 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:01.316148+0000", 2024-09-14T22:37:14.199 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 12.78683895, 2024-09-14T22:37:14.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.036603839999999999, 2024-09-14T22:37:14.199 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.199 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.199 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:14.200 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.200 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.200 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.200 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 86178 2024-09-14T22:37:14.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.200 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.316148+0000", 2024-09-14T22:37:14.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.316148+0000", 2024-09-14T22:37:14.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.316150+0000", 2024-09-14T22:37:14.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.43e-06 2024-09-14T22:37:14.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.316163+0000", 2024-09-14T22:37:14.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2581e-05 2024-09-14T22:37:14.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.316194+0000", 2024-09-14T22:37:14.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1794000000000001e-05 2024-09-14T22:37:14.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.316210+0000", 2024-09-14T22:37:14.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5947000000000002e-05 2024-09-14T22:37:14.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.316343+0000", 2024-09-14T22:37:14.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000132843 2024-09-14T22:37:14.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.316486+0000", 2024-09-14T22:37:14.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014334300000000001 2024-09-14T22:37:14.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 4", 2024-09-14T22:37:14.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.324078+0000", 2024-09-14T22:37:14.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0075916990000000004 2024-09-14T22:37:14.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:14.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.352551+0000", 2024-09-14T22:37:14.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028472515 2024-09-14T22:37:14.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:37:14.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.352587+0000", 2024-09-14T22:37:14.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6196999999999999e-05 2024-09-14T22:37:14.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:37:14.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.352676+0000", 2024-09-14T22:37:14.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8862000000000004e-05 2024-09-14T22:37:14.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.352751+0000", 2024-09-14T22:37:14.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5629000000000004e-05 2024-09-14T22:37:14.205 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.205 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.205 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.205 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.205 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.205 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:14.207 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:37:14.207 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:14.203+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:14.308 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:14.303+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:14.377 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:14.378 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:37:14.378 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:37:14.378 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:14.392 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:37:14.433 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:14.431+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:14.509 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:14.507+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:14.557 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:14.557 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:37:14.558 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:37:14.558 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:37:14.558 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:14.572 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:37:14.613 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:14.611+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:14.622 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2024-09-14T22:37:14.711 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:14.707+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:14.739 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:14.739 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:37:14.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:37:14.739 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:37:14.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.740 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4149 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13129728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:14.740 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703281+0000", 2024-09-14T22:37:14.740 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.033959408999998, 2024-09-14T22:37:14.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045957114, 2024-09-14T22:37:14.740 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.740 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.740 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.740 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.740 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.740 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.740 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4149 2024-09-14T22:37:14.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.740 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703281+0000", 2024-09-14T22:37:14.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703281+0000", 2024-09-14T22:37:14.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703282+0000", 2024-09-14T22:37:14.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4199999999999996e-07 2024-09-14T22:37:14.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703289+0000", 2024-09-14T22:37:14.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0770000000000002e-06 2024-09-14T22:37:14.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703296+0000", 2024-09-14T22:37:14.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8129999999999998e-06 2024-09-14T22:37:14.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703307+0000", 2024-09-14T22:37:14.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1663e-05 2024-09-14T22:37:14.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749073+0000", 2024-09-14T22:37:14.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.04576537 2024-09-14T22:37:14.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749156+0000", 2024-09-14T22:37:14.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3399999999999994e-05 2024-09-14T22:37:14.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749239+0000", 2024-09-14T22:37:14.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2249000000000005e-05 2024-09-14T22:37:14.744 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.744 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.744 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.744 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4150 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13137920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:14.744 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703318+0000", 2024-09-14T22:37:14.744 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.033923215000002, 2024-09-14T22:37:14.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046170585, 2024-09-14T22:37:14.744 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.744 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.745 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.745 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.745 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.745 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.745 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4150 2024-09-14T22:37:14.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.745 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703318+0000", 2024-09-14T22:37:14.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703318+0000", 2024-09-14T22:37:14.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703318+0000", 2024-09-14T22:37:14.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8800000000000003e-07 2024-09-14T22:37:14.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703321+0000", 2024-09-14T22:37:14.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7860000000000001e-06 2024-09-14T22:37:14.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703328+0000", 2024-09-14T22:37:14.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2810000000000003e-06 2024-09-14T22:37:14.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703340+0000", 2024-09-14T22:37:14.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1745e-05 2024-09-14T22:37:14.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749286+0000", 2024-09-14T22:37:14.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045946222000000002 2024-09-14T22:37:14.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749390+0000", 2024-09-14T22:37:14.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000103614 2024-09-14T22:37:14.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749488+0000", 2024-09-14T22:37:14.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8449000000000006e-05 2024-09-14T22:37:14.748 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.748 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.748 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.748 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4151 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13146112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:14.748 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703348+0000", 2024-09-14T22:37:14.749 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.033893052000003, 2024-09-14T22:37:14.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046349509999999997, 2024-09-14T22:37:14.749 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.749 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.749 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.749 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.749 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.749 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.749 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4151 2024-09-14T22:37:14.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.749 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703348+0000", 2024-09-14T22:37:14.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703348+0000", 2024-09-14T22:37:14.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703348+0000", 2024-09-14T22:37:14.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6899999999999998e-07 2024-09-14T22:37:14.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703351+0000", 2024-09-14T22:37:14.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5950000000000001e-06 2024-09-14T22:37:14.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703357+0000", 2024-09-14T22:37:14.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4150000000000004e-06 2024-09-14T22:37:14.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703364+0000", 2024-09-14T22:37:14.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4069999999999998e-06 2024-09-14T22:37:14.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749530+0000", 2024-09-14T22:37:14.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046166281000000003 2024-09-14T22:37:14.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749615+0000", 2024-09-14T22:37:14.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5409999999999997e-05 2024-09-14T22:37:14.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749697+0000", 2024-09-14T22:37:14.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1933000000000003e-05 2024-09-14T22:37:14.753 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.753 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.753 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.753 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4152 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13154304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:14.753 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703371+0000", 2024-09-14T22:37:14.753 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.03386965, 2024-09-14T22:37:14.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046552233999999998, 2024-09-14T22:37:14.753 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.753 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.753 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.753 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.754 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.754 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.754 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4152 2024-09-14T22:37:14.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.754 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703371+0000", 2024-09-14T22:37:14.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703371+0000", 2024-09-14T22:37:14.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703371+0000", 2024-09-14T22:37:14.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.34e-07 2024-09-14T22:37:14.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703374+0000", 2024-09-14T22:37:14.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.227e-06 2024-09-14T22:37:14.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703380+0000", 2024-09-14T22:37:14.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4699999999999999e-06 2024-09-14T22:37:14.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703385+0000", 2024-09-14T22:37:14.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.259e-06 2024-09-14T22:37:14.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749742+0000", 2024-09-14T22:37:14.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046357031 2024-09-14T22:37:14.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749825+0000", 2024-09-14T22:37:14.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2674000000000002e-05 2024-09-14T22:37:14.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749923+0000", 2024-09-14T22:37:14.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8239000000000006e-05 2024-09-14T22:37:14.757 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.757 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.757 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.758 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4153 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13162496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:14.758 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703392+0000", 2024-09-14T22:37:14.758 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.033848354, 2024-09-14T22:37:14.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046675756999999998, 2024-09-14T22:37:14.758 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.758 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.758 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.758 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.758 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.758 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.758 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4153 2024-09-14T22:37:14.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.759 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.759 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.759 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703392+0000", 2024-09-14T22:37:14.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.759 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.759 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703392+0000", 2024-09-14T22:37:14.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.759 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.759 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.705934+0000", 2024-09-14T22:37:14.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025418540000000001 2024-09-14T22:37:14.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.705940+0000", 2024-09-14T22:37:14.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0950000000000001e-06 2024-09-14T22:37:14.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.705952+0000", 2024-09-14T22:37:14.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1973e-05 2024-09-14T22:37:14.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.705958+0000", 2024-09-14T22:37:14.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.541e-06 2024-09-14T22:37:14.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749969+0000", 2024-09-14T22:37:14.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044011300000000003 2024-09-14T22:37:14.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.750020+0000", 2024-09-14T22:37:14.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1054000000000003e-05 2024-09-14T22:37:14.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.750068+0000", 2024-09-14T22:37:14.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7939999999999998e-05 2024-09-14T22:37:14.762 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.762 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.762 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.762 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4165 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13260800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:14.762 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706178+0000", 2024-09-14T22:37:14.762 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.031062442, 2024-09-14T22:37:14.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045915594999999997, 2024-09-14T22:37:14.762 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.763 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.763 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.763 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.763 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.763 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.763 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4165 2024-09-14T22:37:14.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.763 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706178+0000", 2024-09-14T22:37:14.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706178+0000", 2024-09-14T22:37:14.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706179+0000", 2024-09-14T22:37:14.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4999999999999998e-07 2024-09-14T22:37:14.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706181+0000", 2024-09-14T22:37:14.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4679999999999999e-06 2024-09-14T22:37:14.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706189+0000", 2024-09-14T22:37:14.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.994e-06 2024-09-14T22:37:14.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706195+0000", 2024-09-14T22:37:14.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6119999999999998e-06 2024-09-14T22:37:14.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752003+0000", 2024-09-14T22:37:14.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045807618000000001 2024-09-14T22:37:14.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752050+0000", 2024-09-14T22:37:14.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7301e-05 2024-09-14T22:37:14.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752094+0000", 2024-09-14T22:37:14.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4152e-05 2024-09-14T22:37:14.766 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.766 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.766 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.766 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4166 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13268992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:14.767 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706205+0000", 2024-09-14T22:37:14.767 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.031036253000003, 2024-09-14T22:37:14.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046092665999999997, 2024-09-14T22:37:14.767 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.767 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.767 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.767 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.767 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.767 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.767 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4166 2024-09-14T22:37:14.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.767 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706205+0000", 2024-09-14T22:37:14.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.768 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706205+0000", 2024-09-14T22:37:14.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.768 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706205+0000", 2024-09-14T22:37:14.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:37:14.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706208+0000", 2024-09-14T22:37:14.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8049999999999998e-06 2024-09-14T22:37:14.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706215+0000", 2024-09-14T22:37:14.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2010000000000002e-06 2024-09-14T22:37:14.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706223+0000", 2024-09-14T22:37:14.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7260000000000002e-06 2024-09-14T22:37:14.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752129+0000", 2024-09-14T22:37:14.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045906532 2024-09-14T22:37:14.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752213+0000", 2024-09-14T22:37:14.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4060000000000005e-05 2024-09-14T22:37:14.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752297+0000", 2024-09-14T22:37:14.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3807999999999994e-05 2024-09-14T22:37:14.771 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.771 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.771 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.771 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4167 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13277184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:14.771 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706233+0000", 2024-09-14T22:37:14.771 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.031007641999999, 2024-09-14T22:37:14.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046190500000000002, 2024-09-14T22:37:14.771 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.771 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.771 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.771 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.772 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.772 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.772 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4167 2024-09-14T22:37:14.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.772 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706233+0000", 2024-09-14T22:37:14.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706233+0000", 2024-09-14T22:37:14.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706234+0000", 2024-09-14T22:37:14.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.75e-07 2024-09-14T22:37:14.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706237+0000", 2024-09-14T22:37:14.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9890000000000001e-06 2024-09-14T22:37:14.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706244+0000", 2024-09-14T22:37:14.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6240000000000002e-06 2024-09-14T22:37:14.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706255+0000", 2024-09-14T22:37:14.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0328e-05 2024-09-14T22:37:14.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752338+0000", 2024-09-14T22:37:14.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046083172999999998 2024-09-14T22:37:14.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752381+0000", 2024-09-14T22:37:14.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2973999999999999e-05 2024-09-14T22:37:14.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752424+0000", 2024-09-14T22:37:14.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2936999999999999e-05 2024-09-14T22:37:14.775 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.775 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.775 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.775 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4168 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13285376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:14.775 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706261+0000", 2024-09-14T22:37:14.775 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.030980298000003, 2024-09-14T22:37:14.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046378635000000001, 2024-09-14T22:37:14.776 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.776 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.776 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.776 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.776 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.776 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.776 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4168 2024-09-14T22:37:14.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.776 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706261+0000", 2024-09-14T22:37:14.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706261+0000", 2024-09-14T22:37:14.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706261+0000", 2024-09-14T22:37:14.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2899999999999999e-07 2024-09-14T22:37:14.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706264+0000", 2024-09-14T22:37:14.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.774e-06 2024-09-14T22:37:14.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706267+0000", 2024-09-14T22:37:14.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7179999999999998e-06 2024-09-14T22:37:14.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706274+0000", 2024-09-14T22:37:14.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7599999999999997e-06 2024-09-14T22:37:14.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752458+0000", 2024-09-14T22:37:14.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046183825999999997 2024-09-14T22:37:14.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752545+0000", 2024-09-14T22:37:14.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7272000000000001e-05 2024-09-14T22:37:14.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752639+0000", 2024-09-14T22:37:14.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3956000000000006e-05 2024-09-14T22:37:14.779 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.779 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.779 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.780 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4169 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13293568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:14.780 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706278+0000", 2024-09-14T22:37:14.780 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.030962571000003, 2024-09-14T22:37:14.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046489181999999997, 2024-09-14T22:37:14.780 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.780 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.780 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.780 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.780 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.780 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.780 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4169 2024-09-14T22:37:14.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.780 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706278+0000", 2024-09-14T22:37:14.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706278+0000", 2024-09-14T22:37:14.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706279+0000", 2024-09-14T22:37:14.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6100000000000002e-07 2024-09-14T22:37:14.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706280+0000", 2024-09-14T22:37:14.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2589999999999999e-06 2024-09-14T22:37:14.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706283+0000", 2024-09-14T22:37:14.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.501e-06 2024-09-14T22:37:14.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706286+0000", 2024-09-14T22:37:14.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5409999999999999e-06 2024-09-14T22:37:14.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752680+0000", 2024-09-14T22:37:14.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046393886000000002 2024-09-14T22:37:14.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752723+0000", 2024-09-14T22:37:14.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2870000000000001e-05 2024-09-14T22:37:14.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752767+0000", 2024-09-14T22:37:14.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4864000000000003e-05 2024-09-14T22:37:14.784 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.784 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.784 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.784 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4170 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13301760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:14.784 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706289+0000", 2024-09-14T22:37:14.784 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.030951672999997, 2024-09-14T22:37:14.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046691115999999998, 2024-09-14T22:37:14.784 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.784 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.784 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.784 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.785 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.785 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.785 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4170 2024-09-14T22:37:14.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.785 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706289+0000", 2024-09-14T22:37:14.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706289+0000", 2024-09-14T22:37:14.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.786 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706289+0000", 2024-09-14T22:37:14.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3200000000000001e-07 2024-09-14T22:37:14.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.786 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706296+0000", 2024-09-14T22:37:14.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.922e-06 2024-09-14T22:37:14.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.786 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706303+0000", 2024-09-14T22:37:14.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3980000000000004e-06 2024-09-14T22:37:14.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706307+0000", 2024-09-14T22:37:14.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6750000000000001e-06 2024-09-14T22:37:14.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752799+0000", 2024-09-14T22:37:14.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046491609000000003 2024-09-14T22:37:14.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752884+0000", 2024-09-14T22:37:14.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4876999999999994e-05 2024-09-14T22:37:14.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752980+0000", 2024-09-14T22:37:14.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6403000000000005e-05 2024-09-14T22:37:14.788 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.788 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.788 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.788 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6292 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8329216~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:14.788 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966066+0000", 2024-09-14T22:37:14.788 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.771174735000002, 2024-09-14T22:37:14.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045967367000000002, 2024-09-14T22:37:14.789 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.789 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.789 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.789 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.789 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.789 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.789 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6292 2024-09-14T22:37:14.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.789 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.789 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966066+0000", 2024-09-14T22:37:14.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966066+0000", 2024-09-14T22:37:14.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966067+0000", 2024-09-14T22:37:14.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6499999999999999e-07 2024-09-14T22:37:14.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966069+0000", 2024-09-14T22:37:14.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4059999999999999e-06 2024-09-14T22:37:14.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966077+0000", 2024-09-14T22:37:14.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5120000000000002e-06 2024-09-14T22:37:14.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966083+0000", 2024-09-14T22:37:14.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5590000000000002e-06 2024-09-14T22:37:14.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.011820+0000", 2024-09-14T22:37:14.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045737281999999997 2024-09-14T22:37:14.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.011911+0000", 2024-09-14T22:37:14.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0153000000000002e-05 2024-09-14T22:37:14.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012033+0000", 2024-09-14T22:37:14.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012289000000000001 2024-09-14T22:37:14.792 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.792 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.792 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.793 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6293 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8337408~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:14.793 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966091+0000", 2024-09-14T22:37:14.793 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.771149641000001, 2024-09-14T22:37:14.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046181672, 2024-09-14T22:37:14.793 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.793 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.793 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.793 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.793 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.793 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.793 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6293 2024-09-14T22:37:14.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.794 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966091+0000", 2024-09-14T22:37:14.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966091+0000", 2024-09-14T22:37:14.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966092+0000", 2024-09-14T22:37:14.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:37:14.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.795 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966094+0000", 2024-09-14T22:37:14.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5830000000000001e-06 2024-09-14T22:37:14.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.795 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966102+0000", 2024-09-14T22:37:14.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7470000000000005e-06 2024-09-14T22:37:14.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.795 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966107+0000", 2024-09-14T22:37:14.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1209999999999998e-06 2024-09-14T22:37:14.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012079+0000", 2024-09-14T22:37:14.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045971953000000003 2024-09-14T22:37:14.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012168+0000", 2024-09-14T22:37:14.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9398e-05 2024-09-14T22:37:14.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012273+0000", 2024-09-14T22:37:14.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000104483 2024-09-14T22:37:14.797 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.797 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.797 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.797 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6294 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8345600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:14.797 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966115+0000", 2024-09-14T22:37:14.797 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.771126277999997, 2024-09-14T22:37:14.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046297036999999999, 2024-09-14T22:37:14.797 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.797 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.797 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.798 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.798 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.798 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.798 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6294 2024-09-14T22:37:14.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.798 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966115+0000", 2024-09-14T22:37:14.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966115+0000", 2024-09-14T22:37:14.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966115+0000", 2024-09-14T22:37:14.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7e-07 2024-09-14T22:37:14.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966118+0000", 2024-09-14T22:37:14.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.571e-06 2024-09-14T22:37:14.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966125+0000", 2024-09-14T22:37:14.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1690000000000003e-06 2024-09-14T22:37:14.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966130+0000", 2024-09-14T22:37:14.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3290000000000001e-06 2024-09-14T22:37:14.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012298+0000", 2024-09-14T22:37:14.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046168168000000002 2024-09-14T22:37:14.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012348+0000", 2024-09-14T22:37:14.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9663999999999997e-05 2024-09-14T22:37:14.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012412+0000", 2024-09-14T22:37:14.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3665999999999997e-05 2024-09-14T22:37:14.801 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.801 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.801 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.801 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6295 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8353792~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:14.801 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966138+0000", 2024-09-14T22:37:14.802 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.7711033, 2024-09-14T22:37:14.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046505112000000001, 2024-09-14T22:37:14.802 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.802 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.802 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.802 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.802 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.802 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.802 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6295 2024-09-14T22:37:14.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.802 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966138+0000", 2024-09-14T22:37:14.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966138+0000", 2024-09-14T22:37:14.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966138+0000", 2024-09-14T22:37:14.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5900000000000002e-07 2024-09-14T22:37:14.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966140+0000", 2024-09-14T22:37:14.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4710000000000001e-06 2024-09-14T22:37:14.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966147+0000", 2024-09-14T22:37:14.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6760000000000001e-06 2024-09-14T22:37:14.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966153+0000", 2024-09-14T22:37:14.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4779999999999999e-06 2024-09-14T22:37:14.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012452+0000", 2024-09-14T22:37:14.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046299298000000003 2024-09-14T22:37:14.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012537+0000", 2024-09-14T22:37:14.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5439999999999995e-05 2024-09-14T22:37:14.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012643+0000", 2024-09-14T22:37:14.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010529 2024-09-14T22:37:14.805 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.805 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.805 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.806 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6296 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8361984~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:14.806 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966160+0000", 2024-09-14T22:37:14.806 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.771081246000001, 2024-09-14T22:37:14.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046601423000000003, 2024-09-14T22:37:14.806 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.806 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.806 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.806 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.806 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.806 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.806 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6296 2024-09-14T22:37:14.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.806 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966160+0000", 2024-09-14T22:37:14.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966160+0000", 2024-09-14T22:37:14.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966160+0000", 2024-09-14T22:37:14.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7999999999999995e-07 2024-09-14T22:37:14.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966163+0000", 2024-09-14T22:37:14.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.514e-06 2024-09-14T22:37:14.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966169+0000", 2024-09-14T22:37:14.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8880000000000002e-06 2024-09-14T22:37:14.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966173+0000", 2024-09-14T22:37:14.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9189999999999998e-06 2024-09-14T22:37:14.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012660+0000", 2024-09-14T22:37:14.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046486522000000002 2024-09-14T22:37:14.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.809 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012703+0000", 2024-09-14T22:37:14.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2715e-05 2024-09-14T22:37:14.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.809 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012761+0000", 2024-09-14T22:37:14.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8285000000000002e-05 2024-09-14T22:37:14.809 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.809 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.809 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.810 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6297 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8370176~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:14.810 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966180+0000", 2024-09-14T22:37:14.810 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.771060442, 2024-09-14T22:37:14.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046808482999999998, 2024-09-14T22:37:14.810 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.810 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.810 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.810 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.810 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.810 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.810 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6297 2024-09-14T22:37:14.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.810 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966180+0000", 2024-09-14T22:37:14.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966180+0000", 2024-09-14T22:37:14.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966181+0000", 2024-09-14T22:37:14.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0600000000000001e-07 2024-09-14T22:37:14.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966183+0000", 2024-09-14T22:37:14.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3539999999999998e-06 2024-09-14T22:37:14.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966190+0000", 2024-09-14T22:37:14.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.489e-06 2024-09-14T22:37:14.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966195+0000", 2024-09-14T22:37:14.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3890000000000004e-06 2024-09-14T22:37:14.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012799+0000", 2024-09-14T22:37:14.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046604200999999998 2024-09-14T22:37:14.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012883+0000", 2024-09-14T22:37:14.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3714000000000005e-05 2024-09-14T22:37:14.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012989+0000", 2024-09-14T22:37:14.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010593 2024-09-14T22:37:14.813 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.814 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.814 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.814 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6298 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8378368~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:14.814 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966203+0000", 2024-09-14T22:37:14.814 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.771037380999999, 2024-09-14T22:37:14.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046904649999999999, 2024-09-14T22:37:14.814 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.814 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.814 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.814 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.814 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.814 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.814 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6298 2024-09-14T22:37:14.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.815 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966203+0000", 2024-09-14T22:37:14.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966203+0000", 2024-09-14T22:37:14.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966205+0000", 2024-09-14T22:37:14.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5009999999999999e-06 2024-09-14T22:37:14.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966207+0000", 2024-09-14T22:37:14.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2019999999999998e-06 2024-09-14T22:37:14.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966214+0000", 2024-09-14T22:37:14.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.815e-06 2024-09-14T22:37:14.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966219+0000", 2024-09-14T22:37:14.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0590000000000002e-06 2024-09-14T22:37:14.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013005+0000", 2024-09-14T22:37:14.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046785852000000003 2024-09-14T22:37:14.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013050+0000", 2024-09-14T22:37:14.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.511e-05 2024-09-14T22:37:14.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013108+0000", 2024-09-14T22:37:14.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8111e-05 2024-09-14T22:37:14.818 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.818 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.818 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.818 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6299 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8386560~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:14.818 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966238+0000", 2024-09-14T22:37:14.818 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.771003059999998, 2024-09-14T22:37:14.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047091099999999997, 2024-09-14T22:37:14.818 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.818 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.818 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.818 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.818 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.818 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.818 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6299 2024-09-14T22:37:14.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.819 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966238+0000", 2024-09-14T22:37:14.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966238+0000", 2024-09-14T22:37:14.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966238+0000", 2024-09-14T22:37:14.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1500000000000005e-07 2024-09-14T22:37:14.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966241+0000", 2024-09-14T22:37:14.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8270000000000002e-06 2024-09-14T22:37:14.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966249+0000", 2024-09-14T22:37:14.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0029999999999994e-06 2024-09-14T22:37:14.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966255+0000", 2024-09-14T22:37:14.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7760000000000003e-06 2024-09-14T22:37:14.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013136+0000", 2024-09-14T22:37:14.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046881379000000001 2024-09-14T22:37:14.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013223+0000", 2024-09-14T22:37:14.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6681000000000006e-05 2024-09-14T22:37:14.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013329+0000", 2024-09-14T22:37:14.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105919 2024-09-14T22:37:14.822 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.822 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.822 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.822 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:31370 3.0 3:02f48f58:::rbd_data.113cda0183a0.0000000000000000:6 [read 813056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:14.822 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:37:14.822 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 41.374842966999999, 2024-09-14T22:37:14.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047519796000000003, 2024-09-14T22:37:14.822 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:14.822 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:14.822 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:14.823 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:14.823 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:14.823 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:14.823 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 31370 2024-09-14T22:37:14.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.823 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:14.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:14.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:37:14.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:14.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:37:14.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:14.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:14.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:37:14.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0999999999999999e-07 2024-09-14T22:37:14.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:14.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362401+0000", 2024-09-14T22:37:14.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2900000000000001e-06 2024-09-14T22:37:14.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:14.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362406+0000", 2024-09-14T22:37:14.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6960000000000002e-06 2024-09-14T22:37:14.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:14.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362412+0000", 2024-09-14T22:37:14.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.823e-06 2024-09-14T22:37:14.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:14.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.400270+0000", 2024-09-14T22:37:14.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037857737000000002 2024-09-14T22:37:14.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:14.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.400342+0000", 2024-09-14T22:37:14.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1797000000000004e-05 2024-09-14T22:37:14.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:14.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:14.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:14.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.409918+0000", 2024-09-14T22:37:14.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0095760429999999994 2024-09-14T22:37:14.826 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.826 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.826 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.826 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:14.826 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:14.826 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:14.827 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:37:14.828 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:14.823+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:14.929 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:14.923+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:14.999 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:14.999 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:37:14.999 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:37:14.999 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:15.013 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:37:15.029 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:15.023+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:15.131 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:15.123+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:15.134 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:15.134 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:37:15.134 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:37:15.134 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:37:15.134 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:15.148 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:37:15.231 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:15.227+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:15.277 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:15.278 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:37:15.278 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:37:15.278 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:37:15.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.278 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35306 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1425408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:15.278 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507069+0000", 2024-09-14T22:37:15.278 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 41.766853613999999, 2024-09-14T22:37:15.278 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045725259999999997, 2024-09-14T22:37:15.278 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:15.278 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:15.279 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:15.279 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:15.279 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:15.279 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.279 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35306 2024-09-14T22:37:15.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.279 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:15.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.279 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:15.279 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507069+0000", 2024-09-14T22:37:15.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:15.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507069+0000", 2024-09-14T22:37:15.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:15.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507070+0000", 2024-09-14T22:37:15.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0400000000000002e-07 2024-09-14T22:37:15.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:15.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507072+0000", 2024-09-14T22:37:15.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.891e-06 2024-09-14T22:37:15.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:15.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507078+0000", 2024-09-14T22:37:15.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0279999999999997e-06 2024-09-14T22:37:15.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507082+0000", 2024-09-14T22:37:15.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4680000000000003e-06 2024-09-14T22:37:15.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:15.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552632+0000", 2024-09-14T22:37:15.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045550144000000001 2024-09-14T22:37:15.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:15.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552707+0000", 2024-09-14T22:37:15.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4148999999999998e-05 2024-09-14T22:37:15.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:15.283 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552795+0000", 2024-09-14T22:37:15.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8176000000000004e-05 2024-09-14T22:37:15.283 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.283 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:15.283 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.283 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35307 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1454080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:15.283 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:37:15.283 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 41.766835063000002, 2024-09-14T22:37:15.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045891358, 2024-09-14T22:37:15.283 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:15.283 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:15.283 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:15.284 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:15.284 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:15.284 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.284 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35307 2024-09-14T22:37:15.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.284 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:15.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:15.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:37:15.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:15.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:37:15.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:15.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:37:15.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4700000000000002e-07 2024-09-14T22:37:15.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:15.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507093+0000", 2024-09-14T22:37:15.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9520000000000002e-06 2024-09-14T22:37:15.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:15.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507099+0000", 2024-09-14T22:37:15.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0240000000000003e-06 2024-09-14T22:37:15.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507105+0000", 2024-09-14T22:37:15.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5520000000000004e-06 2024-09-14T22:37:15.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:15.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552826+0000", 2024-09-14T22:37:15.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045721494000000001 2024-09-14T22:37:15.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.287 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:15.287 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552896+0000", 2024-09-14T22:37:15.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9519999999999998e-05 2024-09-14T22:37:15.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.287 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:15.287 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552979+0000", 2024-09-14T22:37:15.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3468999999999997e-05 2024-09-14T22:37:15.287 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.287 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:15.287 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.288 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35308 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1482752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:15.288 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:37:15.288 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 41.766812067000004, 2024-09-14T22:37:15.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046076533000000003, 2024-09-14T22:37:15.288 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:15.288 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:15.288 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:15.288 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:15.288 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:15.288 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.288 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35308 2024-09-14T22:37:15.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.288 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:15.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:15.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:37:15.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:15.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:37:15.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:15.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:37:15.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:37:15.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:15.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507113+0000", 2024-09-14T22:37:15.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9099999999999999e-06 2024-09-14T22:37:15.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:15.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507119+0000", 2024-09-14T22:37:15.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7440000000000004e-06 2024-09-14T22:37:15.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507124+0000", 2024-09-14T22:37:15.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2129999999999999e-06 2024-09-14T22:37:15.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:15.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553007+0000", 2024-09-14T22:37:15.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045882488999999999 2024-09-14T22:37:15.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:15.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553076+0000", 2024-09-14T22:37:15.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9233000000000006e-05 2024-09-14T22:37:15.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:15.292 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553188+0000", 2024-09-14T22:37:15.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111557 2024-09-14T22:37:15.292 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.292 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:15.292 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.292 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35309 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1511424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:15.292 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:37:15.292 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 41.766793161999999, 2024-09-14T22:37:15.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046231869000000002, 2024-09-14T22:37:15.292 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:15.292 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:15.292 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:15.293 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:15.293 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:15.293 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.293 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35309 2024-09-14T22:37:15.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.293 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:15.293 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.293 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:15.293 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:37:15.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.293 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.293 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:15.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:37:15.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:15.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:37:15.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9099999999999999e-07 2024-09-14T22:37:15.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:15.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507132+0000", 2024-09-14T22:37:15.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8190000000000001e-06 2024-09-14T22:37:15.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:15.295 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507139+0000", 2024-09-14T22:37:15.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9419999999999998e-06 2024-09-14T22:37:15.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.295 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507145+0000", 2024-09-14T22:37:15.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2600000000000002e-06 2024-09-14T22:37:15.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:15.295 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553225+0000", 2024-09-14T22:37:15.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046079991000000001 2024-09-14T22:37:15.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.296 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:15.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553284+0000", 2024-09-14T22:37:15.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8638999999999999e-05 2024-09-14T22:37:15.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.296 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:15.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553362+0000", 2024-09-14T22:37:15.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7826999999999999e-05 2024-09-14T22:37:15.296 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.296 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:15.296 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.297 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35310 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1540096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:15.297 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507151+0000", 2024-09-14T22:37:15.297 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 41.766771706, 2024-09-14T22:37:15.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046444541999999998, 2024-09-14T22:37:15.297 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:15.297 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:15.297 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:15.297 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:15.297 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:15.297 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.297 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35310 2024-09-14T22:37:15.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.297 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:15.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:15.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507151+0000", 2024-09-14T22:37:15.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:15.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507151+0000", 2024-09-14T22:37:15.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:15.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507152+0000", 2024-09-14T22:37:15.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1699999999999999e-07 2024-09-14T22:37:15.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:15.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507154+0000", 2024-09-14T22:37:15.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.902e-06 2024-09-14T22:37:15.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:15.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507159+0000", 2024-09-14T22:37:15.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.823e-06 2024-09-14T22:37:15.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507165+0000", 2024-09-14T22:37:15.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.587e-06 2024-09-14T22:37:15.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:15.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553395+0000", 2024-09-14T22:37:15.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046230318999999999 2024-09-14T22:37:15.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:15.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553434+0000", 2024-09-14T22:37:15.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8680999999999998e-05 2024-09-14T22:37:15.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:15.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553596+0000", 2024-09-14T22:37:15.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016191299999999999 2024-09-14T22:37:15.301 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.301 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:15.301 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.301 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35311 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1568768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:15.302 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:37:15.302 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 41.766752453000002, 2024-09-14T22:37:15.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046623592999999998, 2024-09-14T22:37:15.302 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:15.302 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:15.302 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:15.302 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:15.302 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:15.302 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.302 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35311 2024-09-14T22:37:15.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.302 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:15.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:15.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:37:15.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:15.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:37:15.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:15.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:37:15.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:37:15.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:15.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507176+0000", 2024-09-14T22:37:15.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3349999999999997e-06 2024-09-14T22:37:15.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:15.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507200+0000", 2024-09-14T22:37:15.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4003e-05 2024-09-14T22:37:15.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507206+0000", 2024-09-14T22:37:15.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.976e-06 2024-09-14T22:37:15.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:15.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553627+0000", 2024-09-14T22:37:15.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046420284999999999 2024-09-14T22:37:15.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:15.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553696+0000", 2024-09-14T22:37:15.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9567999999999995e-05 2024-09-14T22:37:15.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.306 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:15.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553794+0000", 2024-09-14T22:37:15.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8060000000000006e-05 2024-09-14T22:37:15.306 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.306 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:15.306 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.306 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35566 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1484800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:15.306 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514668+0000", 2024-09-14T22:37:15.306 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 41.759254675999998, 2024-09-14T22:37:15.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045644974999999997, 2024-09-14T22:37:15.307 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:15.307 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:15.307 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:15.307 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:15.307 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:15.307 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.307 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35566 2024-09-14T22:37:15.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.307 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:15.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:15.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514668+0000", 2024-09-14T22:37:15.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:15.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514668+0000", 2024-09-14T22:37:15.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:15.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514669+0000", 2024-09-14T22:37:15.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9099999999999999e-07 2024-09-14T22:37:15.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:15.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514671+0000", 2024-09-14T22:37:15.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3800000000000001e-06 2024-09-14T22:37:15.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:15.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514677+0000", 2024-09-14T22:37:15.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9660000000000001e-06 2024-09-14T22:37:15.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514682+0000", 2024-09-14T22:37:15.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0939999999999998e-06 2024-09-14T22:37:15.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:15.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560133+0000", 2024-09-14T22:37:15.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045450398000000003 2024-09-14T22:37:15.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:15.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560207+0000", 2024-09-14T22:37:15.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4196999999999995e-05 2024-09-14T22:37:15.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:15.311 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560313+0000", 2024-09-14T22:37:15.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106549 2024-09-14T22:37:15.311 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.311 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:15.311 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.311 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35567 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1513472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:15.311 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:37:15.311 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 41.759234137, 2024-09-14T22:37:15.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045793215999999998, 2024-09-14T22:37:15.311 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:15.311 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:15.312 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:15.312 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:15.312 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:15.312 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.312 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35567 2024-09-14T22:37:15.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.312 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:15.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.312 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:15.312 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:37:15.312 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.312 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:15.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:37:15.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:15.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:37:15.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1899999999999998e-07 2024-09-14T22:37:15.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:15.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514692+0000", 2024-09-14T22:37:15.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2699999999999999e-06 2024-09-14T22:37:15.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:15.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514698+0000", 2024-09-14T22:37:15.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0939999999999996e-06 2024-09-14T22:37:15.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514703+0000", 2024-09-14T22:37:15.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3870000000000003e-06 2024-09-14T22:37:15.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:15.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560353+0000", 2024-09-14T22:37:15.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045649914 2024-09-14T22:37:15.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.315 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:15.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560416+0000", 2024-09-14T22:37:15.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3096999999999996e-05 2024-09-14T22:37:15.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.315 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:15.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560482+0000", 2024-09-14T22:37:15.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6035e-05 2024-09-14T22:37:15.315 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.315 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:15.315 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.316 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35568 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1542144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:15.316 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:37:15.316 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 41.759209339000002, 2024-09-14T22:37:15.316 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045995921000000002, 2024-09-14T22:37:15.316 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:15.316 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:15.316 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:15.316 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:15.316 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:15.316 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.316 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35568 2024-09-14T22:37:15.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.316 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:15.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:15.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:37:15.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:15.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:37:15.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:15.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:37:15.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4099999999999999e-07 2024-09-14T22:37:15.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:15.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514720+0000", 2024-09-14T22:37:15.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6869999999999999e-06 2024-09-14T22:37:15.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:15.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514727+0000", 2024-09-14T22:37:15.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3540000000000002e-06 2024-09-14T22:37:15.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514732+0000", 2024-09-14T22:37:15.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7690000000000004e-06 2024-09-14T22:37:15.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:15.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560523+0000", 2024-09-14T22:37:15.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045791373000000003 2024-09-14T22:37:15.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:15.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560606+0000", 2024-09-14T22:37:15.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2690999999999998e-05 2024-09-14T22:37:15.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.320 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:15.320 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560710+0000", 2024-09-14T22:37:15.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000103606 2024-09-14T22:37:15.320 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.320 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:15.320 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.320 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35569 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1570816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:15.320 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:37:15.320 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 41.759183530000001, 2024-09-14T22:37:15.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046110630999999999, 2024-09-14T22:37:15.321 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:15.321 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:15.321 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:15.321 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:15.321 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:15.321 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.321 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35569 2024-09-14T22:37:15.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.321 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:15.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.321 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:15.321 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:37:15.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:15.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:37:15.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:15.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:37:15.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.82e-07 2024-09-14T22:37:15.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:15.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514743+0000", 2024-09-14T22:37:15.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.269e-06 2024-09-14T22:37:15.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:15.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514750+0000", 2024-09-14T22:37:15.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7340000000000002e-06 2024-09-14T22:37:15.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514755+0000", 2024-09-14T22:37:15.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7260000000000002e-06 2024-09-14T22:37:15.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:15.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560731+0000", 2024-09-14T22:37:15.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045976377999999998 2024-09-14T22:37:15.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:15.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560777+0000", 2024-09-14T22:37:15.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5868999999999997e-05 2024-09-14T22:37:15.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:15.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560850+0000", 2024-09-14T22:37:15.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3172999999999999e-05 2024-09-14T22:37:15.325 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.325 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:15.325 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.325 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:86180 3.18 3:1d0c52d8:::rbd_data.113cd62ea8a5.000000000000004b:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:15.325 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:01.356417+0000", 2024-09-14T22:37:15.325 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.917506435, 2024-09-14T22:37:15.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049143245000000002, 2024-09-14T22:37:15.325 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:15.325 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:15.325 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:15.326 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:15.326 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:15.326 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.326 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 86180 2024-09-14T22:37:15.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.326 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:15.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:15.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356417+0000", 2024-09-14T22:37:15.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:15.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356417+0000", 2024-09-14T22:37:15.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:15.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356420+0000", 2024-09-14T22:37:15.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2030000000000001e-06 2024-09-14T22:37:15.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:15.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356435+0000", 2024-09-14T22:37:15.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5287000000000001e-05 2024-09-14T22:37:15.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:15.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356472+0000", 2024-09-14T22:37:15.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7311000000000002e-05 2024-09-14T22:37:15.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356492+0000", 2024-09-14T22:37:15.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9519999999999999e-05 2024-09-14T22:37:15.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:15.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356670+0000", 2024-09-14T22:37:15.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000177567 2024-09-14T22:37:15.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:15.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356924+0000", 2024-09-14T22:37:15.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025399299999999999 2024-09-14T22:37:15.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 0", 2024-09-14T22:37:15.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.365052+0000", 2024-09-14T22:37:15.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0081289380000000005 2024-09-14T22:37:15.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:15.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.405396+0000", 2024-09-14T22:37:15.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040343228000000002 2024-09-14T22:37:15.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:37:15.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.405421+0000", 2024-09-14T22:37:15.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5763e-05 2024-09-14T22:37:15.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:37:15.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.405491+0000", 2024-09-14T22:37:15.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9721999999999994e-05 2024-09-14T22:37:15.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:15.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.405560+0000", 2024-09-14T22:37:15.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8713000000000004e-05 2024-09-14T22:37:15.331 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.331 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:15.331 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.331 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98378 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 919552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:15.331 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148183+0000", 2024-09-14T22:37:15.331 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.1257401570000001, 2024-09-14T22:37:15.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045614549999999997, 2024-09-14T22:37:15.332 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:15.332 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:15.332 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:15.332 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:15.332 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:15.332 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.332 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98378 2024-09-14T22:37:15.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.332 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:15.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:15.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148183+0000", 2024-09-14T22:37:15.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:15.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148183+0000", 2024-09-14T22:37:15.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:15.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148183+0000", 2024-09-14T22:37:15.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0800000000000001e-07 2024-09-14T22:37:15.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:15.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148184+0000", 2024-09-14T22:37:15.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2780000000000001e-06 2024-09-14T22:37:15.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:15.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148189+0000", 2024-09-14T22:37:15.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2699999999999998e-06 2024-09-14T22:37:15.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148191+0000", 2024-09-14T22:37:15.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7659999999999999e-06 2024-09-14T22:37:15.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:15.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193708+0000", 2024-09-14T22:37:15.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045516562000000003 2024-09-14T22:37:15.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:15.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193751+0000", 2024-09-14T22:37:15.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2811000000000001e-05 2024-09-14T22:37:15.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:15.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193797+0000", 2024-09-14T22:37:15.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6655e-05 2024-09-14T22:37:15.336 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.336 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:15.336 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.336 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98379 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 928768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:15.336 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148195+0000", 2024-09-14T22:37:15.336 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.1257283280000001, 2024-09-14T22:37:15.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045817405999999998, 2024-09-14T22:37:15.336 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:15.336 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:15.336 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:15.336 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:15.337 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:15.337 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.337 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98379 2024-09-14T22:37:15.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.337 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:15.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:15.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148195+0000", 2024-09-14T22:37:15.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:15.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148195+0000", 2024-09-14T22:37:15.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:15.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148195+0000", 2024-09-14T22:37:15.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3999999999999998e-07 2024-09-14T22:37:15.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:15.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148196+0000", 2024-09-14T22:37:15.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.341e-06 2024-09-14T22:37:15.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:15.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148200+0000", 2024-09-14T22:37:15.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9040000000000002e-06 2024-09-14T22:37:15.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148203+0000", 2024-09-14T22:37:15.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.959e-06 2024-09-14T22:37:15.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:15.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193841+0000", 2024-09-14T22:37:15.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045637886000000003 2024-09-14T22:37:15.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:15.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193933+0000", 2024-09-14T22:37:15.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1823999999999994e-05 2024-09-14T22:37:15.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:15.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194012+0000", 2024-09-14T22:37:15.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9252000000000006e-05 2024-09-14T22:37:15.340 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.340 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:15.340 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.341 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98380 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 937984~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:15.341 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148206+0000", 2024-09-14T22:37:15.341 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.12571669, 2024-09-14T22:37:15.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045937354, 2024-09-14T22:37:15.341 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:15.341 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:15.341 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:15.341 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:15.341 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:15.341 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.341 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98380 2024-09-14T22:37:15.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.341 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:15.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:15.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148206+0000", 2024-09-14T22:37:15.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:15.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148206+0000", 2024-09-14T22:37:15.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:15.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148207+0000", 2024-09-14T22:37:15.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1e-07 2024-09-14T22:37:15.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.343 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:15.343 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148221+0000", 2024-09-14T22:37:15.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4679e-05 2024-09-14T22:37:15.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.343 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:15.343 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148228+0000", 2024-09-14T22:37:15.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3820000000000001e-06 2024-09-14T22:37:15.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148231+0000", 2024-09-14T22:37:15.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2849999999999999e-06 2024-09-14T22:37:15.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:15.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194054+0000", 2024-09-14T22:37:15.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045822596 2024-09-14T22:37:15.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:15.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194098+0000", 2024-09-14T22:37:15.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4604000000000002e-05 2024-09-14T22:37:15.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:15.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194144+0000", 2024-09-14T22:37:15.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5597999999999999e-05 2024-09-14T22:37:15.345 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.345 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:15.345 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.345 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98381 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 947200~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:15.345 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148236+0000", 2024-09-14T22:37:15.345 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.1256871970000004, 2024-09-14T22:37:15.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046111129000000001, 2024-09-14T22:37:15.345 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:15.346 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:15.346 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:15.346 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:15.346 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:15.346 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.346 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98381 2024-09-14T22:37:15.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.346 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:15.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:15.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148236+0000", 2024-09-14T22:37:15.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:15.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148236+0000", 2024-09-14T22:37:15.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:15.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148236+0000", 2024-09-14T22:37:15.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8799999999999998e-07 2024-09-14T22:37:15.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:15.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148238+0000", 2024-09-14T22:37:15.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6559999999999999e-06 2024-09-14T22:37:15.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:15.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148246+0000", 2024-09-14T22:37:15.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3440000000000001e-06 2024-09-14T22:37:15.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148251+0000", 2024-09-14T22:37:15.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5480000000000003e-06 2024-09-14T22:37:15.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:15.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194187+0000", 2024-09-14T22:37:15.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045935847000000002 2024-09-14T22:37:15.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:15.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194267+0000", 2024-09-14T22:37:15.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0267999999999998e-05 2024-09-14T22:37:15.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:15.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194347+0000", 2024-09-14T22:37:15.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0178000000000004e-05 2024-09-14T22:37:15.349 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.349 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:15.349 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.350 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98382 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 956416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:15.350 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148254+0000", 2024-09-14T22:37:15.350 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.125668825, 2024-09-14T22:37:15.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046224227, 2024-09-14T22:37:15.350 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:15.350 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:15.350 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:15.350 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:15.350 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:15.350 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.350 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98382 2024-09-14T22:37:15.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.350 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:15.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:15.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148254+0000", 2024-09-14T22:37:15.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:15.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148254+0000", 2024-09-14T22:37:15.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:15.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148254+0000", 2024-09-14T22:37:15.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3099999999999999e-07 2024-09-14T22:37:15.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:15.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148256+0000", 2024-09-14T22:37:15.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5540000000000001e-06 2024-09-14T22:37:15.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:15.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148259+0000", 2024-09-14T22:37:15.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2499999999999998e-06 2024-09-14T22:37:15.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148262+0000", 2024-09-14T22:37:15.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8310000000000001e-06 2024-09-14T22:37:15.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:15.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194387+0000", 2024-09-14T22:37:15.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046125082999999997 2024-09-14T22:37:15.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:15.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194430+0000", 2024-09-14T22:37:15.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3074999999999997e-05 2024-09-14T22:37:15.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:15.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194478+0000", 2024-09-14T22:37:15.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8202999999999999e-05 2024-09-14T22:37:15.354 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.354 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:15.354 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.354 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98383 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 965632~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:15.354 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148265+0000", 2024-09-14T22:37:15.354 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.1256577109999997, 2024-09-14T22:37:15.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046409141000000001, 2024-09-14T22:37:15.355 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:15.355 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:15.355 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:15.355 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:15.355 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:15.355 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.355 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98383 2024-09-14T22:37:15.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.355 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:15.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:15.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148265+0000", 2024-09-14T22:37:15.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:15.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148265+0000", 2024-09-14T22:37:15.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:15.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148266+0000", 2024-09-14T22:37:15.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0800000000000001e-07 2024-09-14T22:37:15.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:15.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148267+0000", 2024-09-14T22:37:15.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1659999999999999e-06 2024-09-14T22:37:15.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:15.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148271+0000", 2024-09-14T22:37:15.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1289999999999998e-06 2024-09-14T22:37:15.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148274+0000", 2024-09-14T22:37:15.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5939999999999998e-06 2024-09-14T22:37:15.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:15.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194523+0000", 2024-09-14T22:37:15.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046248162000000002 2024-09-14T22:37:15.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:15.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194603+0000", 2024-09-14T22:37:15.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0161999999999996e-05 2024-09-14T22:37:15.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:15.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194675+0000", 2024-09-14T22:37:15.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1719999999999998e-05 2024-09-14T22:37:15.358 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.359 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:15.359 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.359 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98384 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 974848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:15.359 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148278+0000", 2024-09-14T22:37:15.359 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.1256453830000002, 2024-09-14T22:37:15.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046527369999999998, 2024-09-14T22:37:15.359 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:15.359 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:15.359 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:15.359 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:15.359 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:15.360 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.360 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98384 2024-09-14T22:37:15.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.360 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:15.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:15.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148278+0000", 2024-09-14T22:37:15.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:15.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148278+0000", 2024-09-14T22:37:15.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:15.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.149987+0000", 2024-09-14T22:37:15.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017093430000000001 2024-09-14T22:37:15.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:15.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.149993+0000", 2024-09-14T22:37:15.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6219999999999997e-06 2024-09-14T22:37:15.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:15.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150006+0000", 2024-09-14T22:37:15.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3095000000000001e-05 2024-09-14T22:37:15.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150014+0000", 2024-09-14T22:37:15.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6340000000000007e-06 2024-09-14T22:37:15.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:15.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194715+0000", 2024-09-14T22:37:15.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044700349 2024-09-14T22:37:15.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:15.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194759+0000", 2024-09-14T22:37:15.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3807000000000003e-05 2024-09-14T22:37:15.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:15.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194805+0000", 2024-09-14T22:37:15.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6520000000000002e-05 2024-09-14T22:37:15.363 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.363 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:15.363 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.363 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98390 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 1030144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:15.363 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.150152+0000", 2024-09-14T22:37:15.363 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.1237706960000002, 2024-09-14T22:37:15.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045642708999999997, 2024-09-14T22:37:15.364 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:15.364 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:15.364 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:15.364 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:15.364 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:15.364 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.364 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98390 2024-09-14T22:37:15.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.364 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:15.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:15.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150152+0000", 2024-09-14T22:37:15.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:15.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150152+0000", 2024-09-14T22:37:15.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:15.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150153+0000", 2024-09-14T22:37:15.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.08e-07 2024-09-14T22:37:15.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:15.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150155+0000", 2024-09-14T22:37:15.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2910000000000002e-06 2024-09-14T22:37:15.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:15.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150162+0000", 2024-09-14T22:37:15.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4099999999999996e-06 2024-09-14T22:37:15.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150167+0000", 2024-09-14T22:37:15.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3199999999999999e-06 2024-09-14T22:37:15.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:15.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195704+0000", 2024-09-14T22:37:15.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045537243999999998 2024-09-14T22:37:15.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:15.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195749+0000", 2024-09-14T22:37:15.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4866e-05 2024-09-14T22:37:15.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:15.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195795+0000", 2024-09-14T22:37:15.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.617e-05 2024-09-14T22:37:15.367 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.367 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:15.368 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.368 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98391 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 1039360~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:15.368 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.150175+0000", 2024-09-14T22:37:15.368 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.1237478599999999, 2024-09-14T22:37:15.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045825232, 2024-09-14T22:37:15.368 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:15.368 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:15.368 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:15.368 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:15.368 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:15.368 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.368 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98391 2024-09-14T22:37:15.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.369 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:15.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:15.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150175+0000", 2024-09-14T22:37:15.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:15.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150175+0000", 2024-09-14T22:37:15.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:15.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150176+0000", 2024-09-14T22:37:15.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9700000000000002e-07 2024-09-14T22:37:15.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:15.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150178+0000", 2024-09-14T22:37:15.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.407e-06 2024-09-14T22:37:15.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:15.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150185+0000", 2024-09-14T22:37:15.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5189999999999998e-06 2024-09-14T22:37:15.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.371 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150190+0000", 2024-09-14T22:37:15.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2399999999999998e-06 2024-09-14T22:37:15.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.371 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:15.371 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195838+0000", 2024-09-14T22:37:15.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045648265 2024-09-14T22:37:15.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.371 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:15.371 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195918+0000", 2024-09-14T22:37:15.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0094999999999998e-05 2024-09-14T22:37:15.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:15.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.196000+0000", 2024-09-14T22:37:15.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2309000000000001e-05 2024-09-14T22:37:15.372 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.372 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:15.372 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.372 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.372 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:15.372 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:15.373 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:37:15.374 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:15.367+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:15.475 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:15.471+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:15.502 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:37:15.512 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:15.512 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:15.513 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:37:15.513 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:15.521 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-14T22:37:15.523 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:15.523 INFO:teuthology.orchestra.run.smithi001.stdout:3240 read 0x9a119b8 thru 0x9a14018 (0x2661 bytes) 2024-09-14T22:37:15.527 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:37:15.541 INFO:teuthology.orchestra.run.smithi001.stdout:3241 write 0xd1f5ce5 thru 0xd204a50 (0xed6c bytes) 2024-09-14T22:37:15.547 INFO:teuthology.orchestra.run.smithi001.stdout:3242 writesame 0x3b167b8 thru 0x3b20b0a data_size 0xb(0xa353 bytes) 2024-09-14T22:37:15.565 INFO:teuthology.orchestra.run.smithi001.stdout:3243 trunc from 0xd204a51 to 0xabebfab 2024-09-14T22:37:15.576 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:15.571+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:15.585 INFO:teuthology.orchestra.run.smithi001.stdout:3245 writesame 0x5fd4fbb thru 0x5fdf5e0 data_size 0x8b(0xa626 bytes) 2024-09-14T22:37:15.596 INFO:teuthology.orchestra.run.smithi001.stdout:3246 write 0x60a0908 thru 0x60ab74d (0xae46 bytes) 2024-09-14T22:37:15.599 INFO:teuthology.orchestra.run.smithi001.stdout:3248 read 0x9d53bd0 thru 0x9d550eb (0x151c bytes) 2024-09-14T22:37:15.599 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:15.600 INFO:teuthology.orchestra.run.smithi001.stdout:3250 read 0xa83e197 thru 0xa84742b (0x9295 bytes) 2024-09-14T22:37:15.600 INFO:teuthology.orchestra.run.smithi001.stdout:3251 read 0x441d107 thru 0x4423b80 (0x6a7a bytes) 2024-09-14T22:37:15.601 INFO:teuthology.orchestra.run.smithi001.stdout:3252 read 0x6234d46 thru 0x6242e53 (0xe10e bytes) 2024-09-14T22:37:15.601 INFO:teuthology.orchestra.run.smithi001.stdout:3254 writesame 0x95566e1 thru 0x9556a4a data_size 0x2(0x36a bytes) 2024-09-14T22:37:15.602 INFO:teuthology.orchestra.run.smithi001.stdout:3256 writesame 0xa50e3e0 thru 0xa510ed8 data_size 0x13(0x2af9 bytes) 2024-09-14T22:37:15.605 INFO:teuthology.orchestra.run.smithi001.stdout:3257 read 0x33627bb thru 0x3371f34 (0xf77a bytes) 2024-09-14T22:37:15.605 INFO:teuthology.orchestra.run.smithi001.stdout:3259 punch from 0xa9e13a to 0xaaac9e, (0xcb64 bytes) 2024-09-14T22:37:15.605 INFO:teuthology.orchestra.run.smithi001.stdout:3260 clone 22 order 25 su 131072 sc 8 2024-09-14T22:37:15.661 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:15.661 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:37:15.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.661 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDPGPush(3.4 69/67 [PushOp(3:269c8772:::rbd_data.113cff0d0fb7.0000000000000004:head, version: 56'138, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:269c8772:::rbd_data.113cff0d0fb7.0000000000000004:head@56'138, size: 524288, copy_subset: [(0, 524288)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 524288, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T22:37:15.661 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:15.660725+0000", 2024-09-14T22:37:15.661 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.00096856700000000004, 2024-09-14T22:37:15.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010015270000000001, 2024-09-14T22:37:15.662 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:15.662 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:15.662 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:15.662 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:15.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:15.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:15.660725+0000", 2024-09-14T22:37:15.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:15.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:15.660725+0000", 2024-09-14T22:37:15.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:15.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:15.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:15.660726+0000", 2024-09-14T22:37:15.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0800000000000003e-07 2024-09-14T22:37:15.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:15.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:15.661041+0000", 2024-09-14T22:37:15.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031528200000000001 2024-09-14T22:37:15.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:15.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:15.661061+0000", 2024-09-14T22:37:15.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9556e-05 2024-09-14T22:37:15.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:15.661073+0000", 2024-09-14T22:37:15.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2182e-05 2024-09-14T22:37:15.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:15.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:15.661138+0000", 2024-09-14T22:37:15.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5011999999999994e-05 2024-09-14T22:37:15.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:15.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:15.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:15.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:15.661145+0000", 2024-09-14T22:37:15.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.379e-06 2024-09-14T22:37:15.664 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.665 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:15.665 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.665 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:15.665 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:37:15.665 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 1 2024-09-14T22:37:15.665 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:15.665 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:37:15.677 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:15.671+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:15.699 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:15.700 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:15.700 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:37:15.700 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:37:15.700 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:15.716 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:37:15.746 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone21 from 0xabebfab (overlap 0xa7fbda6) to 0x43318a8 2024-09-14T22:37:15.778 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:15.775+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:15.817 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:37:15.818 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:37:15.834 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:37:15.834 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:37:15.853 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:37:15.853 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:37:15.870 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:37:15.870 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:37:15.879 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:15.875+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:15.882 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:15.882 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:15.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:15.882 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:15.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.883 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6376 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8230912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:15.883 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932578+0000", 2024-09-14T22:37:15.883 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.945244158000001, 2024-09-14T22:37:15.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051558771000000003, 2024-09-14T22:37:15.883 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:15.883 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:15.883 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:15.883 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:15.883 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:15.883 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.883 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6376 2024-09-14T22:37:15.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.883 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:15.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.884 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:15.884 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932578+0000", 2024-09-14T22:37:15.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.884 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:15.884 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932578+0000", 2024-09-14T22:37:15.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.884 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:15.884 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932579+0000", 2024-09-14T22:37:15.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8599999999999996e-07 2024-09-14T22:37:15.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:15.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932597+0000", 2024-09-14T22:37:15.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7771e-05 2024-09-14T22:37:15.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:15.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932618+0000", 2024-09-14T22:37:15.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1026000000000002e-05 2024-09-14T22:37:15.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932635+0000", 2024-09-14T22:37:15.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.702e-05 2024-09-14T22:37:15.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:15.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.983951+0000", 2024-09-14T22:37:15.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051316639999999997 2024-09-14T22:37:15.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:15.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984031+0000", 2024-09-14T22:37:15.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0085000000000003e-05 2024-09-14T22:37:15.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:15.887 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984137+0000", 2024-09-14T22:37:15.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105243 2024-09-14T22:37:15.887 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.887 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:15.887 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.887 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6377 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8239104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:15.887 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932688+0000", 2024-09-14T22:37:15.887 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.945134127000003, 2024-09-14T22:37:15.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051650696000000003, 2024-09-14T22:37:15.887 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:15.887 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:15.888 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:15.888 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:15.888 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:15.888 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.888 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6377 2024-09-14T22:37:15.888 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.888 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:15.888 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.888 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:15.888 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932688+0000", 2024-09-14T22:37:15.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.888 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.888 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:15.889 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932688+0000", 2024-09-14T22:37:15.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:15.889 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932689+0000", 2024-09-14T22:37:15.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4309999999999999e-06 2024-09-14T22:37:15.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:15.889 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932695+0000", 2024-09-14T22:37:15.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8889999999999999e-06 2024-09-14T22:37:15.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:15.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932713+0000", 2024-09-14T22:37:15.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7416999999999999e-05 2024-09-14T22:37:15.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932726+0000", 2024-09-14T22:37:15.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3145e-05 2024-09-14T22:37:15.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:15.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984181+0000", 2024-09-14T22:37:15.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051455630000000002 2024-09-14T22:37:15.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:15.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984247+0000", 2024-09-14T22:37:15.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5350000000000003e-05 2024-09-14T22:37:15.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:15.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984339+0000", 2024-09-14T22:37:15.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1834000000000002e-05 2024-09-14T22:37:15.891 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.891 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:15.891 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.892 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6378 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8247296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:15.892 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932743+0000", 2024-09-14T22:37:15.892 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.945078893000002, 2024-09-14T22:37:15.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051819373000000002, 2024-09-14T22:37:15.892 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:15.892 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:15.892 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:15.892 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:15.892 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:15.892 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.892 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6378 2024-09-14T22:37:15.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.893 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:15.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:15.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932743+0000", 2024-09-14T22:37:15.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:15.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932743+0000", 2024-09-14T22:37:15.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:15.894 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932745+0000", 2024-09-14T22:37:15.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.637e-06 2024-09-14T22:37:15.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.894 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:15.894 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932750+0000", 2024-09-14T22:37:15.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7859999999999996e-06 2024-09-14T22:37:15.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.894 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:15.894 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932767+0000", 2024-09-14T22:37:15.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7621999999999998e-05 2024-09-14T22:37:15.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.895 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932781+0000", 2024-09-14T22:37:15.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3403e-05 2024-09-14T22:37:15.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.895 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:15.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984383+0000", 2024-09-14T22:37:15.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051602249000000003 2024-09-14T22:37:15.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.895 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:15.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984461+0000", 2024-09-14T22:37:15.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8467000000000006e-05 2024-09-14T22:37:15.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.896 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:15.896 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984562+0000", 2024-09-14T22:37:15.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101209 2024-09-14T22:37:15.896 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.896 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:15.896 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.896 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6379 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8255488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:15.896 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932796+0000", 2024-09-14T22:37:15.896 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.945025502, 2024-09-14T22:37:15.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051961245000000003, 2024-09-14T22:37:15.897 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:15.897 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:15.897 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:15.897 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:15.897 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:15.897 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.897 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6379 2024-09-14T22:37:15.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.897 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:15.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:15.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932796+0000", 2024-09-14T22:37:15.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:15.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932796+0000", 2024-09-14T22:37:15.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:15.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932797+0000", 2024-09-14T22:37:15.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9700000000000005e-07 2024-09-14T22:37:15.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:15.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932823+0000", 2024-09-14T22:37:15.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5273999999999999e-05 2024-09-14T22:37:15.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.899 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:15.899 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932840+0000", 2024-09-14T22:37:15.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7774e-05 2024-09-14T22:37:15.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.899 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.899 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932862+0000", 2024-09-14T22:37:15.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1957e-05 2024-09-14T22:37:15.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.899 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:15.899 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984603+0000", 2024-09-14T22:37:15.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051740543 2024-09-14T22:37:15.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.900 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:15.900 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984669+0000", 2024-09-14T22:37:15.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.567e-05 2024-09-14T22:37:15.900 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.900 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:15.900 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984758+0000", 2024-09-14T22:37:15.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9129999999999995e-05 2024-09-14T22:37:15.900 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.900 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:15.900 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.900 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.900 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6380 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8263680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:15.901 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932879+0000", 2024-09-14T22:37:15.901 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.944943422999998, 2024-09-14T22:37:15.901 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052102860000000001, 2024-09-14T22:37:15.901 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:15.901 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:15.901 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:15.901 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:15.901 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:15.901 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.901 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6380 2024-09-14T22:37:15.901 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.901 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:15.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:15.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932879+0000", 2024-09-14T22:37:15.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:15.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932879+0000", 2024-09-14T22:37:15.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:15.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932880+0000", 2024-09-14T22:37:15.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.054e-06 2024-09-14T22:37:15.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:15.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932885+0000", 2024-09-14T22:37:15.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1290000000000004e-06 2024-09-14T22:37:15.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:15.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932901+0000", 2024-09-14T22:37:15.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5908000000000001e-05 2024-09-14T22:37:15.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.904 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.904 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932913+0000", 2024-09-14T22:37:15.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2408999999999999e-05 2024-09-14T22:37:15.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.904 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:15.904 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984800+0000", 2024-09-14T22:37:15.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051886671000000002 2024-09-14T22:37:15.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.904 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:15.905 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984879+0000", 2024-09-14T22:37:15.905 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9200000000000001e-05 2024-09-14T22:37:15.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.905 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:15.905 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984981+0000", 2024-09-14T22:37:15.905 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102489 2024-09-14T22:37:15.905 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.905 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:15.905 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.906 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6381 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8271872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:15.906 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932929+0000", 2024-09-14T22:37:15.906 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.944893092000001, 2024-09-14T22:37:15.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052251484000000001, 2024-09-14T22:37:15.906 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:15.906 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:15.906 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:15.906 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:15.906 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:15.906 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.906 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6381 2024-09-14T22:37:15.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.907 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:15.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:15.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932929+0000", 2024-09-14T22:37:15.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:15.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932929+0000", 2024-09-14T22:37:15.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:15.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932930+0000", 2024-09-14T22:37:15.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3399999999999998e-07 2024-09-14T22:37:15.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:15.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932934+0000", 2024-09-14T22:37:15.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5719999999999996e-06 2024-09-14T22:37:15.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:15.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932961+0000", 2024-09-14T22:37:15.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6437000000000001e-05 2024-09-14T22:37:15.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932973+0000", 2024-09-14T22:37:15.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2097e-05 2024-09-14T22:37:15.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.909 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:15.909 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985021+0000", 2024-09-14T22:37:15.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052047955999999999 2024-09-14T22:37:15.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.909 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:15.909 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985089+0000", 2024-09-14T22:37:15.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7725999999999995e-05 2024-09-14T22:37:15.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.909 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:15.909 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985180+0000", 2024-09-14T22:37:15.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1861999999999996e-05 2024-09-14T22:37:15.910 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.910 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:15.910 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.910 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6382 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8280064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:15.910 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933013+0000", 2024-09-14T22:37:15.910 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.944808930000001, 2024-09-14T22:37:15.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052391118, 2024-09-14T22:37:15.910 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:15.910 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:15.910 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:15.910 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:15.911 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:15.911 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.911 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6382 2024-09-14T22:37:15.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.911 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:15.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:15.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933013+0000", 2024-09-14T22:37:15.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:15.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933013+0000", 2024-09-14T22:37:15.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:15.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933014+0000", 2024-09-14T22:37:15.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.375e-06 2024-09-14T22:37:15.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:15.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933020+0000", 2024-09-14T22:37:15.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0220000000000001e-06 2024-09-14T22:37:15.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:15.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933038+0000", 2024-09-14T22:37:15.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7694999999999999e-05 2024-09-14T22:37:15.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933051+0000", 2024-09-14T22:37:15.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2721999999999999e-05 2024-09-14T22:37:15.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:15.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985224+0000", 2024-09-14T22:37:15.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052172757 2024-09-14T22:37:15.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:15.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985303+0000", 2024-09-14T22:37:15.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9426999999999997e-05 2024-09-14T22:37:15.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.914 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:15.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985404+0000", 2024-09-14T22:37:15.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010111999999999999 2024-09-14T22:37:15.914 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.914 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:15.914 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.914 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6383 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8288256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:15.914 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933082+0000", 2024-09-14T22:37:15.915 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.944739949999999, 2024-09-14T22:37:15.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052525761999999997, 2024-09-14T22:37:15.915 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:15.915 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:15.915 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:15.915 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:15.915 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:15.915 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.915 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6383 2024-09-14T22:37:15.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.915 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:15.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:15.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933082+0000", 2024-09-14T22:37:15.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:15.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933082+0000", 2024-09-14T22:37:15.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:15.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933084+0000", 2024-09-14T22:37:15.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5659999999999999e-06 2024-09-14T22:37:15.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:15.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933089+0000", 2024-09-14T22:37:15.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.271e-06 2024-09-14T22:37:15.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:15.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933105+0000", 2024-09-14T22:37:15.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5685999999999999e-05 2024-09-14T22:37:15.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933118+0000", 2024-09-14T22:37:15.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3188e-05 2024-09-14T22:37:15.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:15.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985448+0000", 2024-09-14T22:37:15.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052330176999999999 2024-09-14T22:37:15.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:15.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985514+0000", 2024-09-14T22:37:15.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6029000000000001e-05 2024-09-14T22:37:15.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:15.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985608+0000", 2024-09-14T22:37:15.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3844999999999993e-05 2024-09-14T22:37:15.919 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.919 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:15.919 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.919 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6384 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8296448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:15.919 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933134+0000", 2024-09-14T22:37:15.919 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.944687590999997, 2024-09-14T22:37:15.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052688835000000003, 2024-09-14T22:37:15.919 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:15.919 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:15.919 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:15.919 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:15.919 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:15.920 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.920 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6384 2024-09-14T22:37:15.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.920 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:15.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:15.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933134+0000", 2024-09-14T22:37:15.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:15.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933134+0000", 2024-09-14T22:37:15.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:15.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933135+0000", 2024-09-14T22:37:15.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.78e-07 2024-09-14T22:37:15.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:15.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933141+0000", 2024-09-14T22:37:15.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1540000000000001e-06 2024-09-14T22:37:15.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:15.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933156+0000", 2024-09-14T22:37:15.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5488000000000001e-05 2024-09-14T22:37:15.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933169+0000", 2024-09-14T22:37:15.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2649e-05 2024-09-14T22:37:15.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:15.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985650+0000", 2024-09-14T22:37:15.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052481112000000003 2024-09-14T22:37:15.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.923 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:15.923 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985724+0000", 2024-09-14T22:37:15.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3953000000000002e-05 2024-09-14T22:37:15.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.923 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:15.923 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985823+0000", 2024-09-14T22:37:15.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9500999999999995e-05 2024-09-14T22:37:15.923 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.923 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:15.923 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.924 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6385 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8304640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:15.924 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933185+0000", 2024-09-14T22:37:15.924 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.944636887999998, 2024-09-14T22:37:15.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052892844000000001, 2024-09-14T22:37:15.924 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:15.924 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:15.924 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:15.924 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:15.924 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:15.924 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.924 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6385 2024-09-14T22:37:15.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.924 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:15.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:15.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933185+0000", 2024-09-14T22:37:15.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:15.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933185+0000", 2024-09-14T22:37:15.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:15.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933186+0000", 2024-09-14T22:37:15.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0139999999999999e-06 2024-09-14T22:37:15.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:15.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933191+0000", 2024-09-14T22:37:15.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7230000000000004e-06 2024-09-14T22:37:15.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:15.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933207+0000", 2024-09-14T22:37:15.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.658e-05 2024-09-14T22:37:15.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.927 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933220+0000", 2024-09-14T22:37:15.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2583e-05 2024-09-14T22:37:15.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:15.927 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985880+0000", 2024-09-14T22:37:15.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052660140000000001 2024-09-14T22:37:15.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:15.927 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985965+0000", 2024-09-14T22:37:15.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4472e-05 2024-09-14T22:37:15.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:15.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986078+0000", 2024-09-14T22:37:15.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011333200000000001 2024-09-14T22:37:15.928 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.928 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:15.928 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.928 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6386 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8312832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:15.928 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933236+0000", 2024-09-14T22:37:15.928 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.944585994000001, 2024-09-14T22:37:15.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053075342999999997, 2024-09-14T22:37:15.929 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:15.929 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:15.929 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:15.929 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:15.929 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:15.929 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.929 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6386 2024-09-14T22:37:15.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.929 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:15.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:15.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933236+0000", 2024-09-14T22:37:15.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:15.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933236+0000", 2024-09-14T22:37:15.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:15.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933237+0000", 2024-09-14T22:37:15.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6500000000000008e-07 2024-09-14T22:37:15.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:15.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933242+0000", 2024-09-14T22:37:15.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.938e-06 2024-09-14T22:37:15.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:15.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933258+0000", 2024-09-14T22:37:15.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6334e-05 2024-09-14T22:37:15.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933270+0000", 2024-09-14T22:37:15.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1973999999999999e-05 2024-09-14T22:37:15.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.932 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.932 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:15.932 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986123+0000", 2024-09-14T22:37:15.932 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052852769000000001 2024-09-14T22:37:15.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:15.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986204+0000", 2024-09-14T22:37:15.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1458999999999994e-05 2024-09-14T22:37:15.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:15.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986311+0000", 2024-09-14T22:37:15.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010690400000000001 2024-09-14T22:37:15.933 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.933 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:15.934 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.934 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6387 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8321024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:15.934 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933335+0000", 2024-09-14T22:37:15.934 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.944486724000001, 2024-09-14T22:37:15.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053182363000000003, 2024-09-14T22:37:15.934 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:15.934 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:15.934 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:15.934 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:15.934 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:15.934 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.935 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6387 2024-09-14T22:37:15.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.935 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:15.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:15.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933335+0000", 2024-09-14T22:37:15.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:15.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933335+0000", 2024-09-14T22:37:15.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:15.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933337+0000", 2024-09-14T22:37:15.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3829999999999999e-06 2024-09-14T22:37:15.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.936 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:15.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933343+0000", 2024-09-14T22:37:15.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1489999999999999e-06 2024-09-14T22:37:15.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.936 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:15.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933361+0000", 2024-09-14T22:37:15.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8105e-05 2024-09-14T22:37:15.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.937 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.937 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933374+0000", 2024-09-14T22:37:15.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.33e-05 2024-09-14T22:37:15.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.937 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.937 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:15.937 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986355+0000", 2024-09-14T22:37:15.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052981053 2024-09-14T22:37:15.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.937 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.937 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:15.937 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986424+0000", 2024-09-14T22:37:15.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8864999999999999e-05 2024-09-14T22:37:15.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.938 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:15.938 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986518+0000", 2024-09-14T22:37:15.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3508e-05 2024-09-14T22:37:15.938 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.938 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:15.938 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.938 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6388 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8329216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:15.938 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933390+0000", 2024-09-14T22:37:15.938 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.944432138000003, 2024-09-14T22:37:15.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053371605000000003, 2024-09-14T22:37:15.938 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:15.938 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:15.939 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:15.939 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:15.939 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:15.939 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.939 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6388 2024-09-14T22:37:15.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.939 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:15.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:15.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933390+0000", 2024-09-14T22:37:15.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.940 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:15.940 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933390+0000", 2024-09-14T22:37:15.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.940 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.940 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:15.940 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933391+0000", 2024-09-14T22:37:15.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.443e-06 2024-09-14T22:37:15.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.941 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:15.941 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933396+0000", 2024-09-14T22:37:15.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1680000000000003e-06 2024-09-14T22:37:15.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.941 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:15.941 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933412+0000", 2024-09-14T22:37:15.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5302999999999999e-05 2024-09-14T22:37:15.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.942 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.942 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933439+0000", 2024-09-14T22:37:15.942 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6747999999999999e-05 2024-09-14T22:37:15.942 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.942 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.942 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:15.942 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986562+0000", 2024-09-14T22:37:15.942 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053123161000000002 2024-09-14T22:37:15.942 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.942 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.942 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:15.942 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986641+0000", 2024-09-14T22:37:15.942 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9086999999999997e-05 2024-09-14T22:37:15.942 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.942 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:15.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986761+0000", 2024-09-14T22:37:15.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120695 2024-09-14T22:37:15.943 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.943 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:15.943 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.943 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6389 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8337408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:15.943 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933456+0000", 2024-09-14T22:37:15.943 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.944366287000001, 2024-09-14T22:37:15.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053514182, 2024-09-14T22:37:15.943 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:15.944 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:15.944 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:15.944 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:15.944 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:15.944 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.944 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6389 2024-09-14T22:37:15.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.944 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:15.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:15.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933456+0000", 2024-09-14T22:37:15.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:15.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933456+0000", 2024-09-14T22:37:15.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:15.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933457+0000", 2024-09-14T22:37:15.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0020000000000001e-06 2024-09-14T22:37:15.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:15.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933462+0000", 2024-09-14T22:37:15.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8400000000000002e-06 2024-09-14T22:37:15.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.946 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:15.946 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933477+0000", 2024-09-14T22:37:15.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.592e-05 2024-09-14T22:37:15.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.946 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.946 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933489+0000", 2024-09-14T22:37:15.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1955e-05 2024-09-14T22:37:15.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:15.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986798+0000", 2024-09-14T22:37:15.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053308263000000002 2024-09-14T22:37:15.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:15.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986871+0000", 2024-09-14T22:37:15.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3518999999999999e-05 2024-09-14T22:37:15.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:15.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986970+0000", 2024-09-14T22:37:15.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8683000000000004e-05 2024-09-14T22:37:15.948 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.948 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:15.948 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.948 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6390 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8345600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:15.948 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933504+0000", 2024-09-14T22:37:15.948 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.944317501999997, 2024-09-14T22:37:15.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053719579000000003, 2024-09-14T22:37:15.948 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:15.949 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:15.949 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:15.949 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:15.949 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:15.949 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.949 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6390 2024-09-14T22:37:15.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.949 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:15.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:15.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933504+0000", 2024-09-14T22:37:15.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.950 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:15.950 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933504+0000", 2024-09-14T22:37:15.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.950 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:15.950 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933505+0000", 2024-09-14T22:37:15.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8299999999999995e-07 2024-09-14T22:37:15.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.950 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:15.950 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933520+0000", 2024-09-14T22:37:15.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4948e-05 2024-09-14T22:37:15.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:15.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933538+0000", 2024-09-14T22:37:15.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7286e-05 2024-09-14T22:37:15.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933550+0000", 2024-09-14T22:37:15.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2223e-05 2024-09-14T22:37:15.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:15.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987028+0000", 2024-09-14T22:37:15.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053478500999999998 2024-09-14T22:37:15.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:15.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987111+0000", 2024-09-14T22:37:15.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3158999999999994e-05 2024-09-14T22:37:15.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:15.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987224+0000", 2024-09-14T22:37:15.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011257899999999999 2024-09-14T22:37:15.952 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.952 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:15.953 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.953 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6391 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8353792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:15.953 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933576+0000", 2024-09-14T22:37:15.953 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.944246057000001, 2024-09-14T22:37:15.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053858196999999997, 2024-09-14T22:37:15.953 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:15.953 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:15.953 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:15.953 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:15.953 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:15.953 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.953 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6391 2024-09-14T22:37:15.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.954 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:15.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:15.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933576+0000", 2024-09-14T22:37:15.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:15.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933576+0000", 2024-09-14T22:37:15.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:15.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933578+0000", 2024-09-14T22:37:15.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9539999999999998e-06 2024-09-14T22:37:15.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:15.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933584+0000", 2024-09-14T22:37:15.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9030000000000001e-06 2024-09-14T22:37:15.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:15.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933601+0000", 2024-09-14T22:37:15.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7703e-05 2024-09-14T22:37:15.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933614+0000", 2024-09-14T22:37:15.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2330999999999999e-05 2024-09-14T22:37:15.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:15.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987275+0000", 2024-09-14T22:37:15.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053661470000000003 2024-09-14T22:37:15.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:15.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987342+0000", 2024-09-14T22:37:15.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6266000000000006e-05 2024-09-14T22:37:15.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.957 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:15.957 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987434+0000", 2024-09-14T22:37:15.957 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2570000000000003e-05 2024-09-14T22:37:15.957 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.957 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:15.957 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.957 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6392 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8361984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:15.957 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933648+0000", 2024-09-14T22:37:15.957 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.944173669000001, 2024-09-14T22:37:15.957 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054032430999999999, 2024-09-14T22:37:15.958 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:15.958 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:15.958 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:15.958 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:15.958 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:15.958 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.958 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6392 2024-09-14T22:37:15.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.958 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:15.958 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.958 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:15.958 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933648+0000", 2024-09-14T22:37:15.958 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.958 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.959 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:15.959 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933648+0000", 2024-09-14T22:37:15.959 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.959 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.959 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.959 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:15.959 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933650+0000", 2024-09-14T22:37:15.959 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3969999999999999e-06 2024-09-14T22:37:15.959 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.959 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.959 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:15.959 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933655+0000", 2024-09-14T22:37:15.959 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7690000000000002e-06 2024-09-14T22:37:15.959 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:15.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933672+0000", 2024-09-14T22:37:15.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6359999999999999e-05 2024-09-14T22:37:15.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933696+0000", 2024-09-14T22:37:15.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4630999999999998e-05 2024-09-14T22:37:15.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:15.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987490+0000", 2024-09-14T22:37:15.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053793256999999997 2024-09-14T22:37:15.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.961 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:15.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987572+0000", 2024-09-14T22:37:15.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1897999999999994e-05 2024-09-14T22:37:15.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.961 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:15.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987681+0000", 2024-09-14T22:37:15.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109119 2024-09-14T22:37:15.961 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.962 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:15.962 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.962 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6393 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8370176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:15.962 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933713+0000", 2024-09-14T22:37:15.962 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.944108714999999, 2024-09-14T22:37:15.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054194111000000003, 2024-09-14T22:37:15.962 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:15.962 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:15.963 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:15.963 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:15.963 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:15.963 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.963 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6393 2024-09-14T22:37:15.963 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.963 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:15.963 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.963 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:15.963 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933713+0000", 2024-09-14T22:37:15.963 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.963 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.963 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.963 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:15.964 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933713+0000", 2024-09-14T22:37:15.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:15.964 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933714+0000", 2024-09-14T22:37:15.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2109999999999999e-06 2024-09-14T22:37:15.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:15.964 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933721+0000", 2024-09-14T22:37:15.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2489999999999998e-06 2024-09-14T22:37:15.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:15.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933751+0000", 2024-09-14T22:37:15.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0239999999999998e-05 2024-09-14T22:37:15.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933764+0000", 2024-09-14T22:37:15.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.322e-05 2024-09-14T22:37:15.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:15.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987737+0000", 2024-09-14T22:37:15.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053972539 2024-09-14T22:37:15.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:15.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987813+0000", 2024-09-14T22:37:15.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6635000000000001e-05 2024-09-14T22:37:15.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:15.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987907+0000", 2024-09-14T22:37:15.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4017000000000005e-05 2024-09-14T22:37:15.967 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.967 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:15.967 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.967 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6394 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8378368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:15.967 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933781+0000", 2024-09-14T22:37:15.967 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.944040956000002, 2024-09-14T22:37:15.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054378276000000003, 2024-09-14T22:37:15.967 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:15.967 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:15.967 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:15.967 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:15.968 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:15.968 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.968 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6394 2024-09-14T22:37:15.968 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.968 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:15.968 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.968 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:15.968 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933781+0000", 2024-09-14T22:37:15.968 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.968 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.968 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.968 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:15.968 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933781+0000", 2024-09-14T22:37:15.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:15.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933782+0000", 2024-09-14T22:37:15.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9199999999999999e-07 2024-09-14T22:37:15.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:15.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933788+0000", 2024-09-14T22:37:15.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9340000000000003e-06 2024-09-14T22:37:15.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:15.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933803+0000", 2024-09-14T22:37:15.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5362999999999998e-05 2024-09-14T22:37:15.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933816+0000", 2024-09-14T22:37:15.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2456999999999999e-05 2024-09-14T22:37:15.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:15.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987961+0000", 2024-09-14T22:37:15.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054145581999999998 2024-09-14T22:37:15.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:15.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988042+0000", 2024-09-14T22:37:15.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1075000000000006e-05 2024-09-14T22:37:15.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:15.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988159+0000", 2024-09-14T22:37:15.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116873 2024-09-14T22:37:15.971 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.971 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:15.971 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.972 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6395 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8386560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:15.972 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:37:15.972 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.943990061999997, 2024-09-14T22:37:15.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054534379000000001, 2024-09-14T22:37:15.972 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:15.972 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:15.972 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:15.972 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:15.972 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:15.972 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:15.972 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6395 2024-09-14T22:37:15.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.973 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:15.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:15.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:37:15.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:15.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:37:15.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:15.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:15.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933833+0000", 2024-09-14T22:37:15.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3859999999999999e-06 2024-09-14T22:37:15.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:15.974 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933838+0000", 2024-09-14T22:37:15.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8709999999999996e-06 2024-09-14T22:37:15.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:15.974 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933854+0000", 2024-09-14T22:37:15.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6189e-05 2024-09-14T22:37:15.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:15.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933871+0000", 2024-09-14T22:37:15.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.626e-05 2024-09-14T22:37:15.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:15.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988198+0000", 2024-09-14T22:37:15.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054327581999999999 2024-09-14T22:37:15.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:15.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988274+0000", 2024-09-14T22:37:15.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5759000000000004e-05 2024-09-14T22:37:15.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:15.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:15.976 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:15.976 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988366+0000", 2024-09-14T22:37:15.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2331999999999996e-05 2024-09-14T22:37:15.976 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.976 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:15.976 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.976 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:15.976 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:15.976 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:15.977 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:37:15.977 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:37:15.978 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:37:15.978 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:37:15.979 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:37:15.979 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:37:15.979 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:37:15.980 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:37:15.980 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:37:16.025 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:16.023+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:16.082 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:16.075+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:16.145 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:16.145 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:16.159 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:37:16.203 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:16.199+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:16.284 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:16.279+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:16.321 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:16.321 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:16.333 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:37:16.419 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:16.415+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:16.485 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:16.483+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:16.499 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:16.499 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:16.513 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:37:16.586 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:16.583+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:16.649 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T22:37:16.688 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:16.688 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:16.689 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:16.683+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:16.706 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:37:16.795 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:16.791+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:16.864 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:16.864 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:16.878 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:37:16.889 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:16.887+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:16.990 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:16.983+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:17.042 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:17.042 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:17.057 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:37:17.099 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:17.095+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:17.191 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:17.187+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:17.221 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:17.221 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:17.234 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:37:17.293 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:17.291+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:17.357 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:17.357 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:17.370 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:37:17.393 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:17.391+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:17.487 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:17.488 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:17.494 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:17.491+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:17.501 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:37:17.595 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:17.591+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:17.638 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:17.638 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:17.652 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:37:17.663 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2024-09-14T22:37:17.700 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:17.695+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:17.786 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:17.786 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:17.797 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:17.795+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:17.801 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:37:17.854 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-14T22:37:17.898 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:17.895+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:17.982 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:37:17.986 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:17.986 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:17.998 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:17.995+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:18.005 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:37:18.099 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:18.095+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:18.138 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:37:18.138 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:37:18.166 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:37:18.167 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:37:18.173 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:18.174 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:37:18.174 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:37:18.174 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:18.188 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:37:18.198 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:37:18.198 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:37:18.200 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:18.195+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:18.213 INFO:teuthology.orchestra.run.smithi001.stdout:3261 read 0x3f5d348 thru 0x3f5fbfd (0x28b6 bytes) 2024-09-14T22:37:18.215 INFO:teuthology.orchestra.run.smithi001.stdout:3262 writesame 0x6d8d1a2 thru 0x6d9d0ba data_size 0x25(0xff19 bytes) 2024-09-14T22:37:18.222 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:37:18.222 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:37:18.253 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:37:18.253 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:37:18.277 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:37:18.277 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:37:18.294 INFO:teuthology.orchestra.run.smithi001.stdout:3263 trunc from 0xabebfab to 0xdf212b3 2024-09-14T22:37:18.300 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:18.295+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:18.301 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:37:18.301 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:37:18.306 INFO:teuthology.orchestra.run.smithi001.stdout:3264 read 0xb693efb thru 0xb696774 (0x287a bytes) 2024-09-14T22:37:18.307 INFO:teuthology.orchestra.run.smithi001.stdout:3265 write 0x3503bbe thru 0x35050ee (0x1531 bytes) 2024-09-14T22:37:18.333 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:37:18.333 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:37:18.337 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:18.337 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:37:18.337 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:37:18.337 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:37:18.337 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:18.354 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:37:18.382 INFO:teuthology.orchestra.run.smithi001.stdout:3266 writesame 0x682d93c thru 0x6832435 data_size 0x15(0x4afa bytes) 2024-09-14T22:37:18.401 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:18.395+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:18.453 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:18.453 INFO:teuthology.orchestra.run.smithi001.stdout:3268 punch from 0x78fdc1c to 0x7900aff, (0x2ee3 bytes) 2024-09-14T22:37:18.464 INFO:teuthology.orchestra.run.smithi001.stdout:3269 trunc from 0xdf212b3 to 0x5faf381 2024-09-14T22:37:18.502 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:18.499+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:18.524 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:18.524 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:37:18.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:37:18.525 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:37:18.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.525 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3022 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14301184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:18.525 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559594+0000", 2024-09-14T22:37:18.525 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 54.964305887000002, 2024-09-14T22:37:18.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028916481000000001, 2024-09-14T22:37:18.525 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:18.525 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:18.525 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:18.525 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:18.525 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:18.525 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:18.525 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3022 2024-09-14T22:37:18.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.526 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:18.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:18.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559594+0000", 2024-09-14T22:37:18.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:18.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559594+0000", 2024-09-14T22:37:18.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:18.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559594+0000", 2024-09-14T22:37:18.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7000000000000004e-07 2024-09-14T22:37:18.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.527 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:18.527 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559604+0000", 2024-09-14T22:37:18.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6020000000000006e-06 2024-09-14T22:37:18.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.527 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:18.527 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559615+0000", 2024-09-14T22:37:18.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1032999999999999e-05 2024-09-14T22:37:18.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.528 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:18.528 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559638+0000", 2024-09-14T22:37:18.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3255999999999999e-05 2024-09-14T22:37:18.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.528 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:18.528 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588323+0000", 2024-09-14T22:37:18.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.02868447 2024-09-14T22:37:18.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.528 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:18.528 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588443+0000", 2024-09-14T22:37:18.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120056 2024-09-14T22:37:18.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:18.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588510+0000", 2024-09-14T22:37:18.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7393999999999999e-05 2024-09-14T22:37:18.529 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.529 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:18.529 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.529 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3023 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14309376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:18.529 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559651+0000", 2024-09-14T22:37:18.529 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 54.964248400999999, 2024-09-14T22:37:18.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029146231000000002, 2024-09-14T22:37:18.529 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:18.530 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:18.530 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:18.530 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:18.530 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:18.530 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:18.530 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3023 2024-09-14T22:37:18.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.530 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:18.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:18.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559651+0000", 2024-09-14T22:37:18.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:18.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559651+0000", 2024-09-14T22:37:18.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:18.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559652+0000", 2024-09-14T22:37:18.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2699999999999998e-07 2024-09-14T22:37:18.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:18.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559656+0000", 2024-09-14T22:37:18.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7019999999999999e-06 2024-09-14T22:37:18.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:18.532 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559666+0000", 2024-09-14T22:37:18.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0400999999999999e-05 2024-09-14T22:37:18.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.532 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:18.532 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559674+0000", 2024-09-14T22:37:18.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8890000000000002e-06 2024-09-14T22:37:18.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.532 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:18.532 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588565+0000", 2024-09-14T22:37:18.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028891217 2024-09-14T22:37:18.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.533 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:18.533 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588679+0000", 2024-09-14T22:37:18.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114058 2024-09-14T22:37:18.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.533 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:18.533 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588797+0000", 2024-09-14T22:37:18.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118137 2024-09-14T22:37:18.533 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.533 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:18.533 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.534 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3024 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14317568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:18.534 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559685+0000", 2024-09-14T22:37:18.534 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 54.964214290000001, 2024-09-14T22:37:18.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029326498999999999, 2024-09-14T22:37:18.534 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:18.534 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:18.535 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:18.535 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:18.535 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:18.535 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:18.535 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3024 2024-09-14T22:37:18.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.535 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:18.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:18.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559685+0000", 2024-09-14T22:37:18.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:18.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559685+0000", 2024-09-14T22:37:18.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:18.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559686+0000", 2024-09-14T22:37:18.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0999999999999998e-07 2024-09-14T22:37:18.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:18.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559697+0000", 2024-09-14T22:37:18.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.147e-05 2024-09-14T22:37:18.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.537 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:18.537 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559708+0000", 2024-09-14T22:37:18.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0546e-05 2024-09-14T22:37:18.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.537 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:18.537 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559716+0000", 2024-09-14T22:37:18.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.704e-06 2024-09-14T22:37:18.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.537 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:18.537 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588833+0000", 2024-09-14T22:37:18.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029116442999999999 2024-09-14T22:37:18.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:18.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588920+0000", 2024-09-14T22:37:18.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6916999999999995e-05 2024-09-14T22:37:18.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:18.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589012+0000", 2024-09-14T22:37:18.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1809000000000001e-05 2024-09-14T22:37:18.538 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.538 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:18.538 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.538 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3025 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14325760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:18.539 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559729+0000", 2024-09-14T22:37:18.539 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 54.964170391000003, 2024-09-14T22:37:18.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029422501, 2024-09-14T22:37:18.539 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:18.539 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:18.539 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:18.539 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:18.539 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:18.539 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:18.539 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3025 2024-09-14T22:37:18.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.539 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:18.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:18.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559729+0000", 2024-09-14T22:37:18.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:18.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559729+0000", 2024-09-14T22:37:18.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:18.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559730+0000", 2024-09-14T22:37:18.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8100000000000002e-07 2024-09-14T22:37:18.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.541 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:18.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559733+0000", 2024-09-14T22:37:18.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4740000000000001e-06 2024-09-14T22:37:18.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.541 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:18.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559744+0000", 2024-09-14T22:37:18.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0489000000000001e-05 2024-09-14T22:37:18.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.541 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:18.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559760+0000", 2024-09-14T22:37:18.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5817000000000001e-05 2024-09-14T22:37:18.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.542 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:18.542 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589050+0000", 2024-09-14T22:37:18.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.02929056 2024-09-14T22:37:18.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.542 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:18.542 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589101+0000", 2024-09-14T22:37:18.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0992000000000002e-05 2024-09-14T22:37:18.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.542 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:18.542 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589152+0000", 2024-09-14T22:37:18.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0387999999999999e-05 2024-09-14T22:37:18.543 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.543 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:18.543 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.543 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3026 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14333952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:18.543 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559795+0000", 2024-09-14T22:37:18.543 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 54.964104567, 2024-09-14T22:37:18.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029483812000000002, 2024-09-14T22:37:18.543 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:18.544 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:18.544 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:18.544 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:18.544 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:18.544 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:18.544 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3026 2024-09-14T22:37:18.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.544 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:18.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:18.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559795+0000", 2024-09-14T22:37:18.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.545 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:18.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559795+0000", 2024-09-14T22:37:18.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.545 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:18.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559796+0000", 2024-09-14T22:37:18.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3200000000000003e-07 2024-09-14T22:37:18.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.545 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:18.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559800+0000", 2024-09-14T22:37:18.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2930000000000002e-06 2024-09-14T22:37:18.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.546 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:18.546 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559812+0000", 2024-09-14T22:37:18.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2053e-05 2024-09-14T22:37:18.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.546 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:18.546 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559821+0000", 2024-09-14T22:37:18.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8869999999999998e-06 2024-09-14T22:37:18.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:18.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589177+0000", 2024-09-14T22:37:18.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029356308000000001 2024-09-14T22:37:18.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:18.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589229+0000", 2024-09-14T22:37:18.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1326000000000003e-05 2024-09-14T22:37:18.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:18.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589279+0000", 2024-09-14T22:37:18.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0012999999999997e-05 2024-09-14T22:37:18.548 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.548 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:18.548 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.548 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3027 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14342144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:18.548 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559833+0000", 2024-09-14T22:37:18.548 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 54.964066838999997, 2024-09-14T22:37:18.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029619187000000002, 2024-09-14T22:37:18.548 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:18.548 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:18.549 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:18.549 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:18.549 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:18.549 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:18.549 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3027 2024-09-14T22:37:18.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.549 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:18.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:18.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559833+0000", 2024-09-14T22:37:18.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:18.550 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559833+0000", 2024-09-14T22:37:18.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.550 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:18.550 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559833+0000", 2024-09-14T22:37:18.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9400000000000005e-07 2024-09-14T22:37:18.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.550 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:18.550 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559845+0000", 2024-09-14T22:37:18.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1623e-05 2024-09-14T22:37:18.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.551 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:18.551 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559856+0000", 2024-09-14T22:37:18.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0832999999999999e-05 2024-09-14T22:37:18.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.551 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:18.551 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559864+0000", 2024-09-14T22:37:18.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3119999999999994e-06 2024-09-14T22:37:18.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.551 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:18.551 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589299+0000", 2024-09-14T22:37:18.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029434699000000002 2024-09-14T22:37:18.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:18.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589354+0000", 2024-09-14T22:37:18.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5467999999999998e-05 2024-09-14T22:37:18.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:18.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589452+0000", 2024-09-14T22:37:18.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7558000000000003e-05 2024-09-14T22:37:18.552 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.552 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:18.552 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.553 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3028 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14350336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:18.553 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559875+0000", 2024-09-14T22:37:18.553 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 54.964023951000001, 2024-09-14T22:37:18.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029775586999999999, 2024-09-14T22:37:18.553 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:18.553 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:18.553 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:18.553 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:18.553 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:18.553 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:18.553 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3028 2024-09-14T22:37:18.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.553 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:18.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:18.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559875+0000", 2024-09-14T22:37:18.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:18.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559875+0000", 2024-09-14T22:37:18.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:18.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559876+0000", 2024-09-14T22:37:18.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5199999999999996e-07 2024-09-14T22:37:18.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.555 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:18.555 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559879+0000", 2024-09-14T22:37:18.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.095e-06 2024-09-14T22:37:18.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.555 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:18.555 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559889+0000", 2024-09-14T22:37:18.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0019e-05 2024-09-14T22:37:18.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:18.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559897+0000", 2024-09-14T22:37:18.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9189999999999999e-06 2024-09-14T22:37:18.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:18.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589486+0000", 2024-09-14T22:37:18.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029588482999999999 2024-09-14T22:37:18.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:18.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589578+0000", 2024-09-14T22:37:18.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2329000000000003e-05 2024-09-14T22:37:18.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:18.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589651+0000", 2024-09-14T22:37:18.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2990000000000004e-05 2024-09-14T22:37:18.557 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.557 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:18.557 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.557 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4068 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13252608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:18.557 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680102+0000", 2024-09-14T22:37:18.557 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 54.843797131999999, 2024-09-14T22:37:18.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028907656, 2024-09-14T22:37:18.557 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:18.558 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:18.558 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:18.558 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:18.558 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:18.558 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:18.558 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4068 2024-09-14T22:37:18.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.558 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:18.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:18.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680102+0000", 2024-09-14T22:37:18.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:18.559 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680102+0000", 2024-09-14T22:37:18.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:18.559 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680103+0000", 2024-09-14T22:37:18.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6600000000000002e-07 2024-09-14T22:37:18.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:18.559 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680105+0000", 2024-09-14T22:37:18.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.616e-06 2024-09-14T22:37:18.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.560 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:18.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680114+0000", 2024-09-14T22:37:18.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4120000000000001e-06 2024-09-14T22:37:18.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.560 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:18.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680119+0000", 2024-09-14T22:37:18.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6210000000000001e-06 2024-09-14T22:37:18.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.560 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:18.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.708813+0000", 2024-09-14T22:37:18.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028693330999999999 2024-09-14T22:37:18.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:18.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.708903+0000", 2024-09-14T22:37:18.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0297000000000006e-05 2024-09-14T22:37:18.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:18.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709010+0000", 2024-09-14T22:37:18.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106913 2024-09-14T22:37:18.561 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.561 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:18.561 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.562 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4069 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13260800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:18.562 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680128+0000", 2024-09-14T22:37:18.562 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 54.843771857, 2024-09-14T22:37:18.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029145476, 2024-09-14T22:37:18.562 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:18.562 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:18.562 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:18.562 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:18.562 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:18.562 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:18.562 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4069 2024-09-14T22:37:18.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.563 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:18.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:18.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680128+0000", 2024-09-14T22:37:18.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:18.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680128+0000", 2024-09-14T22:37:18.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:18.564 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680128+0000", 2024-09-14T22:37:18.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7599999999999998e-07 2024-09-14T22:37:18.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.564 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:18.564 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680131+0000", 2024-09-14T22:37:18.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9050000000000001e-06 2024-09-14T22:37:18.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:18.565 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680138+0000", 2024-09-14T22:37:18.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2559999999999996e-06 2024-09-14T22:37:18.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:18.565 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680144+0000", 2024-09-14T22:37:18.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4779999999999999e-06 2024-09-14T22:37:18.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:18.565 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709054+0000", 2024-09-14T22:37:18.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028910207 2024-09-14T22:37:18.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:18.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709180+0000", 2024-09-14T22:37:18.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012608699999999999 2024-09-14T22:37:18.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:18.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709273+0000", 2024-09-14T22:37:18.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3166999999999998e-05 2024-09-14T22:37:18.566 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.566 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:18.566 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.567 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4070 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13268992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:18.567 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680151+0000", 2024-09-14T22:37:18.567 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 54.843748574999999, 2024-09-14T22:37:18.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029327986, 2024-09-14T22:37:18.567 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:18.567 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:18.567 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:18.567 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:18.567 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:18.567 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:18.567 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4070 2024-09-14T22:37:18.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.568 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:18.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:18.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680151+0000", 2024-09-14T22:37:18.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:18.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680151+0000", 2024-09-14T22:37:18.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:18.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680151+0000", 2024-09-14T22:37:18.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:37:18.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:18.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680154+0000", 2024-09-14T22:37:18.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3350000000000001e-06 2024-09-14T22:37:18.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:18.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680162+0000", 2024-09-14T22:37:18.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.473e-06 2024-09-14T22:37:18.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:18.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680177+0000", 2024-09-14T22:37:18.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.465e-05 2024-09-14T22:37:18.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:18.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709356+0000", 2024-09-14T22:37:18.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029178744 2024-09-14T22:37:18.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:18.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709428+0000", 2024-09-14T22:37:18.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2428000000000005e-05 2024-09-14T22:37:18.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:18.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709479+0000", 2024-09-14T22:37:18.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0915999999999998e-05 2024-09-14T22:37:18.571 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.571 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:18.571 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.571 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4071 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13277184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:18.572 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680185+0000", 2024-09-14T22:37:18.572 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 54.843714075000001, 2024-09-14T22:37:18.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029452392000000001, 2024-09-14T22:37:18.572 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:18.572 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:18.572 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:18.572 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:18.572 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:18.572 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:18.572 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4071 2024-09-14T22:37:18.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.572 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:18.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:18.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680185+0000", 2024-09-14T22:37:18.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:18.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680185+0000", 2024-09-14T22:37:18.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:18.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680186+0000", 2024-09-14T22:37:18.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6100000000000001e-07 2024-09-14T22:37:18.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:18.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680188+0000", 2024-09-14T22:37:18.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.509e-06 2024-09-14T22:37:18.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:18.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680197+0000", 2024-09-14T22:37:18.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8550000000000008e-06 2024-09-14T22:37:18.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:18.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680203+0000", 2024-09-14T22:37:18.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5849999999999999e-06 2024-09-14T22:37:18.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:18.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709514+0000", 2024-09-14T22:37:18.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029311054 2024-09-14T22:37:18.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:18.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709585+0000", 2024-09-14T22:37:18.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0875000000000002e-05 2024-09-14T22:37:18.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:18.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709638+0000", 2024-09-14T22:37:18.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3053000000000002e-05 2024-09-14T22:37:18.576 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.576 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:18.576 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.576 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4072 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13285376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:18.576 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680211+0000", 2024-09-14T22:37:18.576 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 54.843688794000002, 2024-09-14T22:37:18.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029591177999999999, 2024-09-14T22:37:18.577 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:18.577 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:18.577 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:18.577 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:18.577 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:18.577 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:18.577 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4072 2024-09-14T22:37:18.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.577 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:18.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:18.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680211+0000", 2024-09-14T22:37:18.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:18.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680211+0000", 2024-09-14T22:37:18.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:18.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680211+0000", 2024-09-14T22:37:18.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6699999999999999e-07 2024-09-14T22:37:18.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:18.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680214+0000", 2024-09-14T22:37:18.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0149999999999999e-06 2024-09-14T22:37:18.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:18.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680222+0000", 2024-09-14T22:37:18.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.371e-06 2024-09-14T22:37:18.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:18.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680228+0000", 2024-09-14T22:37:18.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6230000000000002e-06 2024-09-14T22:37:18.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:18.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709670+0000", 2024-09-14T22:37:18.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029441694000000001 2024-09-14T22:37:18.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:18.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709741+0000", 2024-09-14T22:37:18.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1668999999999994e-05 2024-09-14T22:37:18.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:18.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709802+0000", 2024-09-14T22:37:18.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0338999999999999e-05 2024-09-14T22:37:18.581 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.581 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:18.581 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.581 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4073 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13293568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:18.581 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680236+0000", 2024-09-14T22:37:18.581 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 54.843663435000003, 2024-09-14T22:37:18.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029755238999999999, 2024-09-14T22:37:18.581 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:18.581 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:18.581 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:18.581 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:18.582 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:18.582 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:18.582 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4073 2024-09-14T22:37:18.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.582 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:18.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:18.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680236+0000", 2024-09-14T22:37:18.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:18.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680236+0000", 2024-09-14T22:37:18.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:18.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680236+0000", 2024-09-14T22:37:18.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5499999999999998e-07 2024-09-14T22:37:18.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:18.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680239+0000", 2024-09-14T22:37:18.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4380000000000002e-06 2024-09-14T22:37:18.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:18.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680248+0000", 2024-09-14T22:37:18.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6319999999999997e-06 2024-09-14T22:37:18.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:18.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680261+0000", 2024-09-14T22:37:18.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3193000000000001e-05 2024-09-14T22:37:18.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:18.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709831+0000", 2024-09-14T22:37:18.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029570486 2024-09-14T22:37:18.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:18.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709883+0000", 2024-09-14T22:37:18.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2290999999999998e-05 2024-09-14T22:37:18.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:18.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709991+0000", 2024-09-14T22:37:18.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107744 2024-09-14T22:37:18.585 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.585 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:18.585 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.585 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4074 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13301760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:18.585 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680269+0000", 2024-09-14T22:37:18.585 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 54.843630537000003, 2024-09-14T22:37:18.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030007875999999999, 2024-09-14T22:37:18.585 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:18.585 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:18.585 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:18.586 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:18.586 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:18.586 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:18.586 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4074 2024-09-14T22:37:18.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.586 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:18.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:18.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680269+0000", 2024-09-14T22:37:18.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:18.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680269+0000", 2024-09-14T22:37:18.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:18.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680269+0000", 2024-09-14T22:37:18.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7300000000000001e-07 2024-09-14T22:37:18.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:18.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680277+0000", 2024-09-14T22:37:18.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8480000000000001e-06 2024-09-14T22:37:18.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:18.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680286+0000", 2024-09-14T22:37:18.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0529999999999996e-06 2024-09-14T22:37:18.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:18.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680292+0000", 2024-09-14T22:37:18.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.728e-06 2024-09-14T22:37:18.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:18.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.710024+0000", 2024-09-14T22:37:18.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029732176999999999 2024-09-14T22:37:18.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:18.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.710109+0000", 2024-09-14T22:37:18.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4715000000000005e-05 2024-09-14T22:37:18.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:18.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.710277+0000", 2024-09-14T22:37:18.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000167882 2024-09-14T22:37:18.589 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.589 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:18.589 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.589 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:12664 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T22:37:18.589 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:28.325596+0000", 2024-09-14T22:37:18.590 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.198303023000001, 2024-09-14T22:37:18.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028964581, 2024-09-14T22:37:18.590 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:18.590 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:18.590 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:18.590 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:18.590 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:18.590 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:18.590 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12664 2024-09-14T22:37:18.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.590 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:18.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:18.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325596+0000", 2024-09-14T22:37:18.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:18.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325596+0000", 2024-09-14T22:37:18.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:18.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325597+0000", 2024-09-14T22:37:18.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4300000000000002e-07 2024-09-14T22:37:18.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:18.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325600+0000", 2024-09-14T22:37:18.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2219999999999998e-06 2024-09-14T22:37:18.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:18.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325610+0000", 2024-09-14T22:37:18.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2650000000000002e-06 2024-09-14T22:37:18.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:18.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325617+0000", 2024-09-14T22:37:18.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1199999999999996e-06 2024-09-14T22:37:18.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:18.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354438+0000", 2024-09-14T22:37:18.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028821205999999999 2024-09-14T22:37:18.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:18.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354495+0000", 2024-09-14T22:37:18.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6752999999999997e-05 2024-09-14T22:37:18.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:18.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354561+0000", 2024-09-14T22:37:18.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6272000000000005e-05 2024-09-14T22:37:18.593 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.594 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:18.594 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.594 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:12665 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 391168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T22:37:18.594 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:28.325626+0000", 2024-09-14T22:37:18.594 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.198273700999998, 2024-09-14T22:37:18.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029080366, 2024-09-14T22:37:18.594 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:18.594 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:18.594 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:18.594 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:18.594 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:18.594 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:18.594 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12665 2024-09-14T22:37:18.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.595 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:18.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:18.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325626+0000", 2024-09-14T22:37:18.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:18.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325626+0000", 2024-09-14T22:37:18.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:18.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325626+0000", 2024-09-14T22:37:18.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.06e-07 2024-09-14T22:37:18.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:18.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325629+0000", 2024-09-14T22:37:18.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7149999999999998e-06 2024-09-14T22:37:18.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:18.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325637+0000", 2024-09-14T22:37:18.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7360000000000001e-06 2024-09-14T22:37:18.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:18.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325643+0000", 2024-09-14T22:37:18.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2480000000000001e-06 2024-09-14T22:37:18.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:18.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354589+0000", 2024-09-14T22:37:18.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028945538 2024-09-14T22:37:18.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:18.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354635+0000", 2024-09-14T22:37:18.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6374e-05 2024-09-14T22:37:18.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:18.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354706+0000", 2024-09-14T22:37:18.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1049000000000004e-05 2024-09-14T22:37:18.598 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.598 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:18.598 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.598 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:13927 3.1b 3:ddfd6241:::rbd_data.113cda0183a0.0000000000000009:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-14T22:37:18.598 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:31.537348+0000", 2024-09-14T22:37:18.599 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.986551101000003, 2024-09-14T22:37:18.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032156573000000001, 2024-09-14T22:37:18.599 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:18.599 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:18.599 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:18.599 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:18.599 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:18.599 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:18.599 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 13927 2024-09-14T22:37:18.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.599 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:18.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:18.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537348+0000", 2024-09-14T22:37:18.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:18.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537348+0000", 2024-09-14T22:37:18.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:18.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537349+0000", 2024-09-14T22:37:18.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.068e-06 2024-09-14T22:37:18.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:18.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537358+0000", 2024-09-14T22:37:18.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.653e-06 2024-09-14T22:37:18.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:18.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537374+0000", 2024-09-14T22:37:18.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5594e-05 2024-09-14T22:37:18.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:18.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537383+0000", 2024-09-14T22:37:18.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0929999999999992e-06 2024-09-14T22:37:18.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:18.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537429+0000", 2024-09-14T22:37:18.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5948000000000001e-05 2024-09-14T22:37:18.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:18.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537490+0000", 2024-09-14T22:37:18.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1002000000000003e-05 2024-09-14T22:37:18.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 4", 2024-09-14T22:37:18.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.542781+0000", 2024-09-14T22:37:18.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.005290861 2024-09-14T22:37:18.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:18.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.569350+0000", 2024-09-14T22:37:18.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026569299000000001 2024-09-14T22:37:18.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:37:18.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.569375+0000", 2024-09-14T22:37:18.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5448000000000001e-05 2024-09-14T22:37:18.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:37:18.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.569444+0000", 2024-09-14T22:37:18.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9184999999999996e-05 2024-09-14T22:37:18.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:18.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.569505+0000", 2024-09-14T22:37:18.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0421999999999998e-05 2024-09-14T22:37:18.604 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.604 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:18.604 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.604 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:85875 3.12 3:498b1eba:::rbd_data.113cd62ea8a5.000000000000006a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:18.604 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:00.460932+0000", 2024-09-14T22:37:18.604 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.062967199999999, 2024-09-14T22:37:18.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044152693999999999, 2024-09-14T22:37:18.604 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:18.605 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:18.605 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:18.605 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:18.605 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:18.605 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:18.605 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 85875 2024-09-14T22:37:18.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.605 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:18.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:18.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.460932+0000", 2024-09-14T22:37:18.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:18.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.460932+0000", 2024-09-14T22:37:18.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:18.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.460935+0000", 2024-09-14T22:37:18.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6280000000000001e-06 2024-09-14T22:37:18.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:18.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.460947+0000", 2024-09-14T22:37:18.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2269000000000001e-05 2024-09-14T22:37:18.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:18.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.460973+0000", 2024-09-14T22:37:18.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5939999999999999e-05 2024-09-14T22:37:18.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:18.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.460985+0000", 2024-09-14T22:37:18.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.181e-05 2024-09-14T22:37:18.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:18.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.461102+0000", 2024-09-14T22:37:18.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000117413 2024-09-14T22:37:18.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:18.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.461349+0000", 2024-09-14T22:37:18.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00024703300000000002 2024-09-14T22:37:18.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 7", 2024-09-14T22:37:18.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.468711+0000", 2024-09-14T22:37:18.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0073619180000000003 2024-09-14T22:37:18.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:37:18.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.504629+0000", 2024-09-14T22:37:18.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035917273 2024-09-14T22:37:18.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:18.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.504776+0000", 2024-09-14T22:37:18.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014738300000000001 2024-09-14T22:37:18.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:37:18.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.504951+0000", 2024-09-14T22:37:18.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017548100000000001 2024-09-14T22:37:18.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:18.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.505085+0000", 2024-09-14T22:37:18.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013354600000000001 2024-09-14T22:37:18.610 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.610 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:18.610 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.610 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:85883 3.12 3:4cf65cce:::rbd_data.113cd62ea8a5.0000000000000015:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:18.610 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:00.524670+0000", 2024-09-14T22:37:18.610 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 17.999229176, 2024-09-14T22:37:18.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050663316, 2024-09-14T22:37:18.610 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:18.611 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:18.611 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:18.611 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:18.611 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:18.611 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:18.611 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 85883 2024-09-14T22:37:18.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.611 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:18.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:18.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.524670+0000", 2024-09-14T22:37:18.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:18.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.524670+0000", 2024-09-14T22:37:18.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:18.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.524672+0000", 2024-09-14T22:37:18.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1629999999999999e-06 2024-09-14T22:37:18.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:18.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.524688+0000", 2024-09-14T22:37:18.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5787e-05 2024-09-14T22:37:18.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:18.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.524720+0000", 2024-09-14T22:37:18.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1294000000000002e-05 2024-09-14T22:37:18.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:18.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.524736+0000", 2024-09-14T22:37:18.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.607e-05 2024-09-14T22:37:18.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:18.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.524863+0000", 2024-09-14T22:37:18.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012693399999999999 2024-09-14T22:37:18.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:18.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.525060+0000", 2024-09-14T22:37:18.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000197732 2024-09-14T22:37:18.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 7", 2024-09-14T22:37:18.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.532377+0000", 2024-09-14T22:37:18.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0073167680000000004 2024-09-14T22:37:18.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:18.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.575115+0000", 2024-09-14T22:37:18.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042737626000000001 2024-09-14T22:37:18.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:37:18.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.575151+0000", 2024-09-14T22:37:18.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6211000000000003e-05 2024-09-14T22:37:18.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:37:18.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.575245+0000", 2024-09-14T22:37:18.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4648999999999995e-05 2024-09-14T22:37:18.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.616 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:18.616 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.575334+0000", 2024-09-14T22:37:18.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8082000000000001e-05 2024-09-14T22:37:18.616 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.616 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:18.616 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.616 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:86178 3.1 3:83964f10:::rbd_data.113cd62ea8a5.0000000000000027:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:18.616 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:01.316148+0000", 2024-09-14T22:37:18.616 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 17.207751872999999, 2024-09-14T22:37:18.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.036603839999999999, 2024-09-14T22:37:18.616 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:18.617 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:18.617 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:18.617 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:18.617 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:18.617 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:18.617 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 86178 2024-09-14T22:37:18.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.617 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:18.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:18.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.316148+0000", 2024-09-14T22:37:18.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:18.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.316148+0000", 2024-09-14T22:37:18.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:18.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:18.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.316150+0000", 2024-09-14T22:37:18.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.43e-06 2024-09-14T22:37:18.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:18.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.316163+0000", 2024-09-14T22:37:18.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2581e-05 2024-09-14T22:37:18.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:18.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.316194+0000", 2024-09-14T22:37:18.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1794000000000001e-05 2024-09-14T22:37:18.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:18.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.316210+0000", 2024-09-14T22:37:18.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5947000000000002e-05 2024-09-14T22:37:18.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:18.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.316343+0000", 2024-09-14T22:37:18.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000132843 2024-09-14T22:37:18.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:18.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.316486+0000", 2024-09-14T22:37:18.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014334300000000001 2024-09-14T22:37:18.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 4", 2024-09-14T22:37:18.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.324078+0000", 2024-09-14T22:37:18.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0075916990000000004 2024-09-14T22:37:18.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:18.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.352551+0000", 2024-09-14T22:37:18.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028472515 2024-09-14T22:37:18.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:37:18.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.352587+0000", 2024-09-14T22:37:18.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6196999999999999e-05 2024-09-14T22:37:18.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:37:18.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.352676+0000", 2024-09-14T22:37:18.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8862000000000004e-05 2024-09-14T22:37:18.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:18.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:18.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:18.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.352751+0000", 2024-09-14T22:37:18.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5629000000000004e-05 2024-09-14T22:37:18.621 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.621 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:18.622 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.622 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:18.622 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:18.622 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:18.623 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:37:18.624 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:18.619+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:18.678 INFO:teuthology.orchestra.run.smithi001.stdout:3272 write 0xd96a85d thru 0xd96ee55 (0x45f9 bytes) 2024-09-14T22:37:18.682 INFO:teuthology.orchestra.run.smithi001.stdout:3273 read 0x3c1708b thru 0x3c1cd29 (0x5c9f bytes) 2024-09-14T22:37:18.683 INFO:teuthology.orchestra.run.smithi001.stdout:3274 read 0x9d035b9 thru 0x9d0a55e (0x6fa6 bytes) 2024-09-14T22:37:18.684 INFO:teuthology.orchestra.run.smithi001.stdout:3276 writesame 0x589e44 thru 0x5957ff data_size 0x2(0xb9bc bytes) 2024-09-14T22:37:18.712 INFO:teuthology.orchestra.run.smithi001.stdout:3277 write 0xcdf94f3 thru 0xcdfc8cf (0x33dd bytes) 2024-09-14T22:37:18.724 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:18.719+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:18.729 INFO:teuthology.orchestra.run.smithi001.stdout:3280 writesame 0xec77660 thru 0xec7eaf4 data_size 0x2f(0x7495 bytes) 2024-09-14T22:37:18.731 INFO:teuthology.orchestra.run.smithi001.stdout:3281 read 0x1c014c thru 0x1ce854 (0xe709 bytes) 2024-09-14T22:37:18.733 INFO:teuthology.orchestra.run.smithi001.stdout:3282 punch from 0x57a8e76 to 0x57b4100, (0xb28a bytes) 2024-09-14T22:37:18.733 INFO:teuthology.orchestra.run.smithi001.stdout:3283 read 0x3f4544b thru 0x3f4ce71 (0x7a27 bytes) 2024-09-14T22:37:18.734 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:18.734 INFO:teuthology.orchestra.run.smithi001.stdout:3285 write 0x9bb2c11 thru 0x9bbf35b (0xc74b bytes) 2024-09-14T22:37:18.737 INFO:teuthology.orchestra.run.smithi001.stdout:3288 write 0x629ff36 thru 0x62a6dd1 (0x6e9c bytes) 2024-09-14T22:37:18.741 INFO:teuthology.orchestra.run.smithi001.stdout:3289 read 0x38d4d09 thru 0x38d9a46 (0x4d3e bytes) 2024-09-14T22:37:18.741 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:18.742 INFO:teuthology.orchestra.run.smithi001.stdout:3291 punch from 0xd472227 to 0xd4817fe, (0xf5d7 bytes) 2024-09-14T22:37:18.742 INFO:teuthology.orchestra.run.smithi001.stdout:3293 punch from 0x85f7986 to 0x860522d, (0xd8a7 bytes) 2024-09-14T22:37:18.742 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:18.769 INFO:teuthology.orchestra.run.smithi001.stdout:3295 trunc from 0xec7eaf5 to 0x2ad6aca 2024-09-14T22:37:18.787 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:18.787 INFO:teuthology.orchestra.run.smithi001.stdout:3298 read 0x9e83a thru 0x9f265 (0xa2c bytes) 2024-09-14T22:37:18.788 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:18.788 INFO:teuthology.orchestra.run.smithi001.stdout:3300 read 0x1445e46 thru 0x1450202 (0xa3bd bytes) 2024-09-14T22:37:18.789 INFO:teuthology.orchestra.run.smithi001.stdout:3301 punch from 0x297240c to 0x298149b, (0xf08f bytes) 2024-09-14T22:37:18.789 INFO:teuthology.orchestra.run.smithi001.stdout:3302 read 0x151ae12 thru 0x1523130 (0x831f bytes) 2024-09-14T22:37:18.804 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:18.804 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:37:18.804 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:37:18.804 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:18.817 INFO:teuthology.orchestra.run.smithi001.stdout:3303 writesame 0xacfd01b thru 0xad0a9b4 data_size 0x2e(0xd99a bytes) 2024-09-14T22:37:18.819 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:37:18.820 INFO:teuthology.orchestra.run.smithi001.stdout:3306 write 0x19946c1 thru 0x1994e9f (0x7df bytes) 2024-09-14T22:37:18.823 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:18.823 INFO:teuthology.orchestra.run.smithi001.stdout:3309 read 0x9c0a6c0 thru 0x9c0df13 (0x3854 bytes) 2024-09-14T22:37:18.824 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:18.824 INFO:teuthology.orchestra.run.smithi001.stdout:3311 write 0x5afca8b thru 0x5b0c4df (0xfa55 bytes) 2024-09-14T22:37:18.825 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:18.819+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:18.845 INFO:teuthology.orchestra.run.smithi001.stdout:3312 writesame 0xe147ac5 thru 0xe14ce42 data_size 0x1(0x537e bytes) 2024-09-14T22:37:18.847 INFO:teuthology.orchestra.run.smithi001.stdout:3313 write 0x76b9c9d thru 0x76c0cf3 (0x7057 bytes) 2024-09-14T22:37:18.863 INFO:teuthology.orchestra.run.smithi001.stdout:3314 writesame 0xeb77407 thru 0xeb85ba7 data_size 0x2b(0xe7a1 bytes) 2024-09-14T22:37:18.866 INFO:teuthology.orchestra.run.smithi001.stdout:3315 read 0xe30da62 thru 0xe312d50 (0x52ef bytes) 2024-09-14T22:37:18.866 INFO:teuthology.orchestra.run.smithi001.stdout:3316 read 0x7488c21 thru 0x748f0c3 (0x64a3 bytes) 2024-09-14T22:37:18.867 INFO:teuthology.orchestra.run.smithi001.stdout:3317 write 0x45e6aea thru 0x45efa40 (0x8f57 bytes) 2024-09-14T22:37:18.869 INFO:teuthology.orchestra.run.smithi001.stdout:3319 read 0xbbdb941 thru 0xbbe4a86 (0x9146 bytes) 2024-09-14T22:37:18.870 INFO:teuthology.orchestra.run.smithi001.stdout:3320 writesame 0xcb4d8d8 thru 0xcb594ac data_size 0x3b(0xbbd5 bytes) 2024-09-14T22:37:18.873 INFO:teuthology.orchestra.run.smithi001.stdout:3321 writesame 0xa00e1c2 thru 0xa017dc0 data_size 0x31(0x9bff bytes) 2024-09-14T22:37:18.876 INFO:teuthology.orchestra.run.smithi001.stdout:3322 read 0x27ca9d5 thru 0x27d01c1 (0x57ed bytes) 2024-09-14T22:37:18.877 INFO:teuthology.orchestra.run.smithi001.stdout:3323 read 0xbddbe9b thru 0xbdeb719 (0xf87f bytes) 2024-09-14T22:37:18.879 INFO:teuthology.orchestra.run.smithi001.stdout:3325 write 0x1af0496 thru 0x1afa71d (0xa288 bytes) 2024-09-14T22:37:18.881 INFO:teuthology.orchestra.run.smithi001.stdout:3326 write 0x5cb551c thru 0x5cc5426 (0xff0b bytes) 2024-09-14T22:37:18.885 INFO:teuthology.orchestra.run.smithi001.stdout:3328 write 0x43ad76d thru 0x43b2994 (0x5228 bytes) 2024-09-14T22:37:18.887 INFO:teuthology.orchestra.run.smithi001.stdout:3329 write 0x2dbb8ac thru 0x2dcb304 (0xfa59 bytes) 2024-09-14T22:37:18.891 INFO:teuthology.orchestra.run.smithi001.stdout:3331 write 0x414cf2 thru 0x415dc1 (0x10d0 bytes) 2024-09-14T22:37:18.893 INFO:teuthology.orchestra.run.smithi001.stdout:3332 write 0x9279b65 thru 0x92805bd (0x6a59 bytes) 2024-09-14T22:37:18.896 INFO:teuthology.orchestra.run.smithi001.stdout:3334 write 0x5aaf586 thru 0x5ab3bd1 (0x464c bytes) 2024-09-14T22:37:18.898 INFO:teuthology.orchestra.run.smithi001.stdout:3335 write 0xbbec8fe thru 0xbbef2e1 (0x29e4 bytes) 2024-09-14T22:37:18.925 INFO:teuthology.orchestra.run.smithi001.stdout:3336 trunc from 0xeb85ba8 to 0x2992b97 2024-09-14T22:37:18.926 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:18.919+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:18.945 INFO:teuthology.orchestra.run.smithi001.stdout:3337 read 0xb48566 thru 0xb54383 (0xbe1e bytes) 2024-09-14T22:37:18.946 INFO:teuthology.orchestra.run.smithi001.stdout:3338 read 0x81b65b thru 0x81bdb2 (0x758 bytes) 2024-09-14T22:37:18.947 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:18.947 INFO:teuthology.orchestra.run.smithi001.stdout:3343 punch from 0x148f376 to 0x149e7b9, (0xf443 bytes) 2024-09-14T22:37:18.947 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:18.947 INFO:teuthology.orchestra.run.smithi001.stdout:3346 punch from 0x28fb75d to 0x29050e6, (0x9989 bytes) 2024-09-14T22:37:18.956 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:18.956 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:37:18.956 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:37:18.956 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:37:18.956 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:18.957 INFO:teuthology.orchestra.run.smithi001.stdout:3347 trunc from 0x2992b97 to 0x6e46327 2024-09-14T22:37:18.972 INFO:teuthology.orchestra.run.smithi001.stdout:3348 read 0x6d80fae thru 0x6d8bae3 (0xab36 bytes) 2024-09-14T22:37:18.972 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:37:18.973 INFO:teuthology.orchestra.run.smithi001.stdout:3349 write 0x6e0249a thru 0x6e0d4d0 (0xb037 bytes) 2024-09-14T22:37:19.002 INFO:teuthology.orchestra.run.smithi001.stdout:3350 write 0xda2db28 thru 0xda33c62 (0x613b bytes) 2024-09-14T22:37:19.005 INFO:teuthology.orchestra.run.smithi001.stdout:3351 read 0x3e71cf2 thru 0x3e78ba1 (0x6eb0 bytes) 2024-09-14T22:37:19.006 INFO:teuthology.orchestra.run.smithi001.stdout:3352 read 0x4b70221 thru 0x4b7b631 (0xb411 bytes) 2024-09-14T22:37:19.007 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:19.007 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:19.007 INFO:teuthology.orchestra.run.smithi001.stdout:3356 write 0x3aa49d5 thru 0x3ab2792 (0xddbe bytes) 2024-09-14T22:37:19.012 INFO:teuthology.orchestra.run.smithi001.stdout:3358 trunc from 0xda33c63 to 0xeaed19b 2024-09-14T22:37:19.026 INFO:teuthology.orchestra.run.smithi001.stdout:3360 write 0x96ced33 thru 0x96d399a (0x4c68 bytes) 2024-09-14T22:37:19.027 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:19.019+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:19.030 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:19.030 INFO:teuthology.orchestra.run.smithi001.stdout:3362 read 0xc86e4d5 thru 0xc86ef80 (0xaac bytes) 2024-09-14T22:37:19.031 INFO:teuthology.orchestra.run.smithi001.stdout:3363 writesame 0xe5f0e67 thru 0xe5f62f6 data_size 0x10(0x5490 bytes) 2024-09-14T22:37:19.034 INFO:teuthology.orchestra.run.smithi001.stdout:3364 writesame 0xe61122b thru 0xe620d33 data_size 0x1(0xfb09 bytes) 2024-09-14T22:37:19.108 INFO:teuthology.orchestra.run.smithi001.stdout:3365 punch from 0x81481e2 to 0x815488c, (0xc6aa bytes) 2024-09-14T22:37:19.108 INFO:teuthology.orchestra.run.smithi001.stdout:3366 read 0x5c520e4 thru 0x5c527eb (0x708 bytes) 2024-09-14T22:37:19.108 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:19.108 INFO:teuthology.orchestra.run.smithi001.stdout:3369 write 0x72431b4 thru 0x724407f (0xecc bytes) 2024-09-14T22:37:19.112 INFO:teuthology.orchestra.run.smithi001.stdout:3370 write 0x5261c5e thru 0x52657ef (0x3b92 bytes) 2024-09-14T22:37:19.127 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:19.123+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:19.131 INFO:teuthology.orchestra.run.smithi001.stdout:3371 writesame 0xee15d15 thru 0xee17258 data_size 0x2(0x1544 bytes) 2024-09-14T22:37:19.133 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:19.134 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:37:19.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:37:19.134 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:37:19.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.134 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4149 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13129728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:19.134 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703281+0000", 2024-09-14T22:37:19.134 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 55.428676920000001, 2024-09-14T22:37:19.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045957114, 2024-09-14T22:37:19.134 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.134 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.134 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.134 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.135 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.135 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.135 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4149 2024-09-14T22:37:19.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.135 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703281+0000", 2024-09-14T22:37:19.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703281+0000", 2024-09-14T22:37:19.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703282+0000", 2024-09-14T22:37:19.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4199999999999996e-07 2024-09-14T22:37:19.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703289+0000", 2024-09-14T22:37:19.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0770000000000002e-06 2024-09-14T22:37:19.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703296+0000", 2024-09-14T22:37:19.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8129999999999998e-06 2024-09-14T22:37:19.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703307+0000", 2024-09-14T22:37:19.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1663e-05 2024-09-14T22:37:19.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749073+0000", 2024-09-14T22:37:19.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.04576537 2024-09-14T22:37:19.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.138 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.138 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749156+0000", 2024-09-14T22:37:19.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3399999999999994e-05 2024-09-14T22:37:19.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.138 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.138 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749239+0000", 2024-09-14T22:37:19.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2249000000000005e-05 2024-09-14T22:37:19.138 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.138 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.138 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.139 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4150 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13137920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:19.139 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703318+0000", 2024-09-14T22:37:19.139 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 55.428640725999998, 2024-09-14T22:37:19.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046170585, 2024-09-14T22:37:19.139 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.139 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.139 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.139 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.139 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.139 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.139 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4150 2024-09-14T22:37:19.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.139 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703318+0000", 2024-09-14T22:37:19.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703318+0000", 2024-09-14T22:37:19.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703318+0000", 2024-09-14T22:37:19.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8800000000000003e-07 2024-09-14T22:37:19.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703321+0000", 2024-09-14T22:37:19.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7860000000000001e-06 2024-09-14T22:37:19.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703328+0000", 2024-09-14T22:37:19.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2810000000000003e-06 2024-09-14T22:37:19.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703340+0000", 2024-09-14T22:37:19.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1745e-05 2024-09-14T22:37:19.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749286+0000", 2024-09-14T22:37:19.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045946222000000002 2024-09-14T22:37:19.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749390+0000", 2024-09-14T22:37:19.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000103614 2024-09-14T22:37:19.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749488+0000", 2024-09-14T22:37:19.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8449000000000006e-05 2024-09-14T22:37:19.143 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.143 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.143 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.143 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4151 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13146112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:19.143 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703348+0000", 2024-09-14T22:37:19.143 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 55.428610562999999, 2024-09-14T22:37:19.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046349509999999997, 2024-09-14T22:37:19.144 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.144 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.144 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.144 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.144 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.144 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.144 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4151 2024-09-14T22:37:19.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.144 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703348+0000", 2024-09-14T22:37:19.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703348+0000", 2024-09-14T22:37:19.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703348+0000", 2024-09-14T22:37:19.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6899999999999998e-07 2024-09-14T22:37:19.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703351+0000", 2024-09-14T22:37:19.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5950000000000001e-06 2024-09-14T22:37:19.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703357+0000", 2024-09-14T22:37:19.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4150000000000004e-06 2024-09-14T22:37:19.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703364+0000", 2024-09-14T22:37:19.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4069999999999998e-06 2024-09-14T22:37:19.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749530+0000", 2024-09-14T22:37:19.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046166281000000003 2024-09-14T22:37:19.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749615+0000", 2024-09-14T22:37:19.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5409999999999997e-05 2024-09-14T22:37:19.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749697+0000", 2024-09-14T22:37:19.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1933000000000003e-05 2024-09-14T22:37:19.147 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.148 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.148 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.148 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4152 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13154304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:19.148 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703371+0000", 2024-09-14T22:37:19.148 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 55.428587161000003, 2024-09-14T22:37:19.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046552233999999998, 2024-09-14T22:37:19.148 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.148 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.148 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.149 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.149 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.149 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.149 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4152 2024-09-14T22:37:19.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.149 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703371+0000", 2024-09-14T22:37:19.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703371+0000", 2024-09-14T22:37:19.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703371+0000", 2024-09-14T22:37:19.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.34e-07 2024-09-14T22:37:19.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703374+0000", 2024-09-14T22:37:19.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.227e-06 2024-09-14T22:37:19.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703380+0000", 2024-09-14T22:37:19.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4699999999999999e-06 2024-09-14T22:37:19.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703385+0000", 2024-09-14T22:37:19.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.259e-06 2024-09-14T22:37:19.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749742+0000", 2024-09-14T22:37:19.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046357031 2024-09-14T22:37:19.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.152 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749825+0000", 2024-09-14T22:37:19.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2674000000000002e-05 2024-09-14T22:37:19.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.152 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749923+0000", 2024-09-14T22:37:19.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8239000000000006e-05 2024-09-14T22:37:19.152 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.153 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.153 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.153 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4153 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13162496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:19.153 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703392+0000", 2024-09-14T22:37:19.153 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 55.428565865000003, 2024-09-14T22:37:19.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046675756999999998, 2024-09-14T22:37:19.153 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.153 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.153 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.153 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.153 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.154 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.154 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4153 2024-09-14T22:37:19.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.154 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703392+0000", 2024-09-14T22:37:19.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703392+0000", 2024-09-14T22:37:19.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.705934+0000", 2024-09-14T22:37:19.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025418540000000001 2024-09-14T22:37:19.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.705940+0000", 2024-09-14T22:37:19.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0950000000000001e-06 2024-09-14T22:37:19.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.705952+0000", 2024-09-14T22:37:19.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1973e-05 2024-09-14T22:37:19.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.705958+0000", 2024-09-14T22:37:19.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.541e-06 2024-09-14T22:37:19.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749969+0000", 2024-09-14T22:37:19.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044011300000000003 2024-09-14T22:37:19.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.750020+0000", 2024-09-14T22:37:19.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1054000000000003e-05 2024-09-14T22:37:19.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.157 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.157 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.750068+0000", 2024-09-14T22:37:19.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7939999999999998e-05 2024-09-14T22:37:19.157 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.157 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.157 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.157 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4165 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13260800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:19.157 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706178+0000", 2024-09-14T22:37:19.157 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 55.425779953000003, 2024-09-14T22:37:19.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045915594999999997, 2024-09-14T22:37:19.157 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.157 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.158 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.158 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.158 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.158 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.158 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4165 2024-09-14T22:37:19.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.158 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706178+0000", 2024-09-14T22:37:19.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706178+0000", 2024-09-14T22:37:19.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706179+0000", 2024-09-14T22:37:19.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4999999999999998e-07 2024-09-14T22:37:19.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706181+0000", 2024-09-14T22:37:19.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4679999999999999e-06 2024-09-14T22:37:19.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706189+0000", 2024-09-14T22:37:19.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.994e-06 2024-09-14T22:37:19.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706195+0000", 2024-09-14T22:37:19.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6119999999999998e-06 2024-09-14T22:37:19.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752003+0000", 2024-09-14T22:37:19.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045807618000000001 2024-09-14T22:37:19.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.161 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752050+0000", 2024-09-14T22:37:19.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7301e-05 2024-09-14T22:37:19.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.161 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.161 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752094+0000", 2024-09-14T22:37:19.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4152e-05 2024-09-14T22:37:19.161 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.161 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.161 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.161 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4166 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13268992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:19.161 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706205+0000", 2024-09-14T22:37:19.162 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 55.425753764, 2024-09-14T22:37:19.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046092665999999997, 2024-09-14T22:37:19.162 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.162 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.162 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.162 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.162 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.162 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.162 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4166 2024-09-14T22:37:19.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.162 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706205+0000", 2024-09-14T22:37:19.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706205+0000", 2024-09-14T22:37:19.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706205+0000", 2024-09-14T22:37:19.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:37:19.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706208+0000", 2024-09-14T22:37:19.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8049999999999998e-06 2024-09-14T22:37:19.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706215+0000", 2024-09-14T22:37:19.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2010000000000002e-06 2024-09-14T22:37:19.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706223+0000", 2024-09-14T22:37:19.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7260000000000002e-06 2024-09-14T22:37:19.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752129+0000", 2024-09-14T22:37:19.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045906532 2024-09-14T22:37:19.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.165 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.165 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752213+0000", 2024-09-14T22:37:19.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4060000000000005e-05 2024-09-14T22:37:19.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.165 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.165 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752297+0000", 2024-09-14T22:37:19.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3807999999999994e-05 2024-09-14T22:37:19.165 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.165 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.165 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.165 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4167 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13277184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:19.166 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706233+0000", 2024-09-14T22:37:19.166 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 55.425725153000002, 2024-09-14T22:37:19.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046190500000000002, 2024-09-14T22:37:19.166 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.166 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.166 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.166 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.166 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.166 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.166 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4167 2024-09-14T22:37:19.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.166 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.166 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.166 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706233+0000", 2024-09-14T22:37:19.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706233+0000", 2024-09-14T22:37:19.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706234+0000", 2024-09-14T22:37:19.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.75e-07 2024-09-14T22:37:19.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706237+0000", 2024-09-14T22:37:19.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9890000000000001e-06 2024-09-14T22:37:19.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706244+0000", 2024-09-14T22:37:19.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6240000000000002e-06 2024-09-14T22:37:19.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706255+0000", 2024-09-14T22:37:19.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0328e-05 2024-09-14T22:37:19.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752338+0000", 2024-09-14T22:37:19.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046083172999999998 2024-09-14T22:37:19.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752381+0000", 2024-09-14T22:37:19.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2973999999999999e-05 2024-09-14T22:37:19.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752424+0000", 2024-09-14T22:37:19.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2936999999999999e-05 2024-09-14T22:37:19.169 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.169 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.169 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.170 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4168 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13285376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:19.170 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706261+0000", 2024-09-14T22:37:19.170 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 55.425697808999999, 2024-09-14T22:37:19.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046378635000000001, 2024-09-14T22:37:19.170 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.170 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.170 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.170 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.170 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.170 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.170 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4168 2024-09-14T22:37:19.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.170 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706261+0000", 2024-09-14T22:37:19.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.171 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.171 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706261+0000", 2024-09-14T22:37:19.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.171 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.171 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706261+0000", 2024-09-14T22:37:19.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2899999999999999e-07 2024-09-14T22:37:19.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.171 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.171 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706264+0000", 2024-09-14T22:37:19.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.774e-06 2024-09-14T22:37:19.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706267+0000", 2024-09-14T22:37:19.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7179999999999998e-06 2024-09-14T22:37:19.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706274+0000", 2024-09-14T22:37:19.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7599999999999997e-06 2024-09-14T22:37:19.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752458+0000", 2024-09-14T22:37:19.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046183825999999997 2024-09-14T22:37:19.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752545+0000", 2024-09-14T22:37:19.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7272000000000001e-05 2024-09-14T22:37:19.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752639+0000", 2024-09-14T22:37:19.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3956000000000006e-05 2024-09-14T22:37:19.173 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.173 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.173 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.174 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4169 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13293568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:19.174 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706278+0000", 2024-09-14T22:37:19.174 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 55.425680082, 2024-09-14T22:37:19.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046489181999999997, 2024-09-14T22:37:19.174 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.174 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.174 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.174 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.174 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.174 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.174 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4169 2024-09-14T22:37:19.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.174 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.175 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.175 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706278+0000", 2024-09-14T22:37:19.175 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.175 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.175 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706278+0000", 2024-09-14T22:37:19.175 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.175 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.175 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706279+0000", 2024-09-14T22:37:19.175 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6100000000000002e-07 2024-09-14T22:37:19.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706280+0000", 2024-09-14T22:37:19.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2589999999999999e-06 2024-09-14T22:37:19.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706283+0000", 2024-09-14T22:37:19.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.501e-06 2024-09-14T22:37:19.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706286+0000", 2024-09-14T22:37:19.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5409999999999999e-06 2024-09-14T22:37:19.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752680+0000", 2024-09-14T22:37:19.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046393886000000002 2024-09-14T22:37:19.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752723+0000", 2024-09-14T22:37:19.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2870000000000001e-05 2024-09-14T22:37:19.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752767+0000", 2024-09-14T22:37:19.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4864000000000003e-05 2024-09-14T22:37:19.178 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.178 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.178 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.178 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4170 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13301760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:19.178 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706289+0000", 2024-09-14T22:37:19.178 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 55.425669184, 2024-09-14T22:37:19.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046691115999999998, 2024-09-14T22:37:19.178 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.178 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.178 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.178 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.178 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.178 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.178 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4170 2024-09-14T22:37:19.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.179 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706289+0000", 2024-09-14T22:37:19.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706289+0000", 2024-09-14T22:37:19.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706289+0000", 2024-09-14T22:37:19.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3200000000000001e-07 2024-09-14T22:37:19.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706296+0000", 2024-09-14T22:37:19.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.922e-06 2024-09-14T22:37:19.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706303+0000", 2024-09-14T22:37:19.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3980000000000004e-06 2024-09-14T22:37:19.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706307+0000", 2024-09-14T22:37:19.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6750000000000001e-06 2024-09-14T22:37:19.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752799+0000", 2024-09-14T22:37:19.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046491609000000003 2024-09-14T22:37:19.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752884+0000", 2024-09-14T22:37:19.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4876999999999994e-05 2024-09-14T22:37:19.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752980+0000", 2024-09-14T22:37:19.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6403000000000005e-05 2024-09-14T22:37:19.182 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.182 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.182 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.182 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6292 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8329216~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:19.182 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966066+0000", 2024-09-14T22:37:19.182 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 55.165892245999999, 2024-09-14T22:37:19.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045967367000000002, 2024-09-14T22:37:19.182 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.182 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.182 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.183 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.183 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.183 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.183 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6292 2024-09-14T22:37:19.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.183 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966066+0000", 2024-09-14T22:37:19.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966066+0000", 2024-09-14T22:37:19.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.184 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.184 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966067+0000", 2024-09-14T22:37:19.184 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6499999999999999e-07 2024-09-14T22:37:19.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.184 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.184 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966069+0000", 2024-09-14T22:37:19.184 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4059999999999999e-06 2024-09-14T22:37:19.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.184 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.184 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966077+0000", 2024-09-14T22:37:19.184 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5120000000000002e-06 2024-09-14T22:37:19.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.185 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.185 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966083+0000", 2024-09-14T22:37:19.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5590000000000002e-06 2024-09-14T22:37:19.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.185 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.185 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.011820+0000", 2024-09-14T22:37:19.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045737281999999997 2024-09-14T22:37:19.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.185 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.185 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.011911+0000", 2024-09-14T22:37:19.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0153000000000002e-05 2024-09-14T22:37:19.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.186 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012033+0000", 2024-09-14T22:37:19.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012289000000000001 2024-09-14T22:37:19.186 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.186 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.186 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.186 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6293 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8337408~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:19.186 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966091+0000", 2024-09-14T22:37:19.186 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 55.165867151999997, 2024-09-14T22:37:19.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046181672, 2024-09-14T22:37:19.186 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.187 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.187 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.187 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.187 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.187 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.187 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6293 2024-09-14T22:37:19.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.187 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966091+0000", 2024-09-14T22:37:19.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966091+0000", 2024-09-14T22:37:19.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966092+0000", 2024-09-14T22:37:19.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:37:19.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966094+0000", 2024-09-14T22:37:19.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5830000000000001e-06 2024-09-14T22:37:19.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966102+0000", 2024-09-14T22:37:19.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7470000000000005e-06 2024-09-14T22:37:19.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966107+0000", 2024-09-14T22:37:19.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1209999999999998e-06 2024-09-14T22:37:19.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012079+0000", 2024-09-14T22:37:19.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045971953000000003 2024-09-14T22:37:19.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012168+0000", 2024-09-14T22:37:19.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9398e-05 2024-09-14T22:37:19.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012273+0000", 2024-09-14T22:37:19.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000104483 2024-09-14T22:37:19.190 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.190 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.190 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.190 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6294 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8345600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:19.190 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966115+0000", 2024-09-14T22:37:19.190 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 55.165843789, 2024-09-14T22:37:19.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046297036999999999, 2024-09-14T22:37:19.191 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.191 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.191 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.191 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.191 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.191 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.191 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6294 2024-09-14T22:37:19.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.191 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.191 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.191 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966115+0000", 2024-09-14T22:37:19.191 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.192 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966115+0000", 2024-09-14T22:37:19.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.192 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966115+0000", 2024-09-14T22:37:19.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7e-07 2024-09-14T22:37:19.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.192 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966118+0000", 2024-09-14T22:37:19.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.571e-06 2024-09-14T22:37:19.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.193 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.193 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966125+0000", 2024-09-14T22:37:19.193 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1690000000000003e-06 2024-09-14T22:37:19.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.193 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.193 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966130+0000", 2024-09-14T22:37:19.193 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3290000000000001e-06 2024-09-14T22:37:19.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.193 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.193 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012298+0000", 2024-09-14T22:37:19.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046168168000000002 2024-09-14T22:37:19.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012348+0000", 2024-09-14T22:37:19.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9663999999999997e-05 2024-09-14T22:37:19.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012412+0000", 2024-09-14T22:37:19.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3665999999999997e-05 2024-09-14T22:37:19.194 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.194 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.194 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.195 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6295 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8353792~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:19.195 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966138+0000", 2024-09-14T22:37:19.195 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 55.165820811000003, 2024-09-14T22:37:19.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046505112000000001, 2024-09-14T22:37:19.195 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.195 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.195 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.195 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.195 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.195 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.195 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6295 2024-09-14T22:37:19.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.195 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966138+0000", 2024-09-14T22:37:19.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966138+0000", 2024-09-14T22:37:19.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966138+0000", 2024-09-14T22:37:19.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5900000000000002e-07 2024-09-14T22:37:19.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.197 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966140+0000", 2024-09-14T22:37:19.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4710000000000001e-06 2024-09-14T22:37:19.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.197 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.197 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966147+0000", 2024-09-14T22:37:19.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6760000000000001e-06 2024-09-14T22:37:19.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.197 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.197 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966153+0000", 2024-09-14T22:37:19.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4779999999999999e-06 2024-09-14T22:37:19.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012452+0000", 2024-09-14T22:37:19.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046299298000000003 2024-09-14T22:37:19.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012537+0000", 2024-09-14T22:37:19.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5439999999999995e-05 2024-09-14T22:37:19.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012643+0000", 2024-09-14T22:37:19.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010529 2024-09-14T22:37:19.198 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.198 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.199 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.199 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6296 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8361984~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:19.199 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966160+0000", 2024-09-14T22:37:19.199 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 55.165798756999997, 2024-09-14T22:37:19.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046601423000000003, 2024-09-14T22:37:19.199 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.199 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.199 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.199 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.199 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.199 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.200 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6296 2024-09-14T22:37:19.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.200 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966160+0000", 2024-09-14T22:37:19.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966160+0000", 2024-09-14T22:37:19.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966160+0000", 2024-09-14T22:37:19.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7999999999999995e-07 2024-09-14T22:37:19.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966163+0000", 2024-09-14T22:37:19.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.514e-06 2024-09-14T22:37:19.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966169+0000", 2024-09-14T22:37:19.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8880000000000002e-06 2024-09-14T22:37:19.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966173+0000", 2024-09-14T22:37:19.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9189999999999998e-06 2024-09-14T22:37:19.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012660+0000", 2024-09-14T22:37:19.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046486522000000002 2024-09-14T22:37:19.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012703+0000", 2024-09-14T22:37:19.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2715e-05 2024-09-14T22:37:19.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012761+0000", 2024-09-14T22:37:19.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8285000000000002e-05 2024-09-14T22:37:19.203 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.203 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.203 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.203 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6297 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8370176~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:19.203 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966180+0000", 2024-09-14T22:37:19.203 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 55.165777953000003, 2024-09-14T22:37:19.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046808482999999998, 2024-09-14T22:37:19.203 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.203 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.203 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.204 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.204 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.204 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.204 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6297 2024-09-14T22:37:19.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.204 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966180+0000", 2024-09-14T22:37:19.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966180+0000", 2024-09-14T22:37:19.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966181+0000", 2024-09-14T22:37:19.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0600000000000001e-07 2024-09-14T22:37:19.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966183+0000", 2024-09-14T22:37:19.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3539999999999998e-06 2024-09-14T22:37:19.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966190+0000", 2024-09-14T22:37:19.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.489e-06 2024-09-14T22:37:19.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.206 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.206 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966195+0000", 2024-09-14T22:37:19.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3890000000000004e-06 2024-09-14T22:37:19.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.206 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.206 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012799+0000", 2024-09-14T22:37:19.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046604200999999998 2024-09-14T22:37:19.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.206 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.206 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012883+0000", 2024-09-14T22:37:19.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3714000000000005e-05 2024-09-14T22:37:19.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.207 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012989+0000", 2024-09-14T22:37:19.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010593 2024-09-14T22:37:19.207 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.207 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.207 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.207 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6298 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8378368~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:19.207 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966203+0000", 2024-09-14T22:37:19.207 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 55.165754892000002, 2024-09-14T22:37:19.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046904649999999999, 2024-09-14T22:37:19.207 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.207 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.207 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.207 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.208 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.208 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.208 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6298 2024-09-14T22:37:19.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.208 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966203+0000", 2024-09-14T22:37:19.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966203+0000", 2024-09-14T22:37:19.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966205+0000", 2024-09-14T22:37:19.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5009999999999999e-06 2024-09-14T22:37:19.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966207+0000", 2024-09-14T22:37:19.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2019999999999998e-06 2024-09-14T22:37:19.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966214+0000", 2024-09-14T22:37:19.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.815e-06 2024-09-14T22:37:19.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966219+0000", 2024-09-14T22:37:19.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0590000000000002e-06 2024-09-14T22:37:19.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013005+0000", 2024-09-14T22:37:19.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046785852000000003 2024-09-14T22:37:19.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013050+0000", 2024-09-14T22:37:19.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.511e-05 2024-09-14T22:37:19.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.211 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.211 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.211 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013108+0000", 2024-09-14T22:37:19.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8111e-05 2024-09-14T22:37:19.211 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.211 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.211 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.211 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.211 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6299 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8386560~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:19.211 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966238+0000", 2024-09-14T22:37:19.211 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 55.165720571000001, 2024-09-14T22:37:19.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047091099999999997, 2024-09-14T22:37:19.211 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.212 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.212 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.212 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.212 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.212 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.212 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6299 2024-09-14T22:37:19.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.212 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.212 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.212 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966238+0000", 2024-09-14T22:37:19.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966238+0000", 2024-09-14T22:37:19.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966238+0000", 2024-09-14T22:37:19.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1500000000000005e-07 2024-09-14T22:37:19.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966241+0000", 2024-09-14T22:37:19.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8270000000000002e-06 2024-09-14T22:37:19.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966249+0000", 2024-09-14T22:37:19.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0029999999999994e-06 2024-09-14T22:37:19.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966255+0000", 2024-09-14T22:37:19.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7760000000000003e-06 2024-09-14T22:37:19.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013136+0000", 2024-09-14T22:37:19.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046881379000000001 2024-09-14T22:37:19.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013223+0000", 2024-09-14T22:37:19.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6681000000000006e-05 2024-09-14T22:37:19.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013329+0000", 2024-09-14T22:37:19.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105919 2024-09-14T22:37:19.216 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.216 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.216 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.216 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:31370 3.0 3:02f48f58:::rbd_data.113cda0183a0.0000000000000000:6 [read 813056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:19.216 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:37:19.216 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 45.769560478000002, 2024-09-14T22:37:19.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047519796000000003, 2024-09-14T22:37:19.216 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.216 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.216 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.216 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.216 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.217 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.217 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 31370 2024-09-14T22:37:19.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.217 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.217 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.217 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:37:19.217 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.217 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.217 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:37:19.217 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:37:19.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0999999999999999e-07 2024-09-14T22:37:19.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362401+0000", 2024-09-14T22:37:19.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2900000000000001e-06 2024-09-14T22:37:19.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362406+0000", 2024-09-14T22:37:19.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6960000000000002e-06 2024-09-14T22:37:19.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362412+0000", 2024-09-14T22:37:19.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.823e-06 2024-09-14T22:37:19.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.400270+0000", 2024-09-14T22:37:19.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037857737000000002 2024-09-14T22:37:19.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.400342+0000", 2024-09-14T22:37:19.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1797000000000004e-05 2024-09-14T22:37:19.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.409918+0000", 2024-09-14T22:37:19.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0095760429999999994 2024-09-14T22:37:19.220 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.220 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.220 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.220 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.220 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.220 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:19.221 INFO:teuthology.orchestra.run.smithi001.stdout:3372 trunc from 0xee17259 to 0xbbc9883 2024-09-14T22:37:19.221 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:37:19.222 INFO:teuthology.orchestra.run.smithi001.stdout:3375 write 0x26c0fbc thru 0x26c350d (0x2552 bytes) 2024-09-14T22:37:19.223 INFO:teuthology.orchestra.run.smithi001.stdout:3376 write 0x2845b45 thru 0x284b160 (0x561c bytes) 2024-09-14T22:37:19.223 INFO:teuthology.orchestra.run.smithi001.stdout:3377 writesame 0xd2376ed thru 0xd2414c4 data_size 0x4(0x9dd8 bytes) 2024-09-14T22:37:19.223 INFO:teuthology.orchestra.run.smithi001.stdout:3379 writesame 0x9761853 thru 0x976962c data_size 0x59(0x7dda bytes) 2024-09-14T22:37:19.223 INFO:teuthology.orchestra.run.smithi001.stdout:3380 write 0x28f50e5 thru 0x2903fea (0xef06 bytes) 2024-09-14T22:37:19.223 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:19.223 INFO:teuthology.orchestra.run.smithi001.stdout:3382 write 0x215b95d thru 0x215fb5d (0x4201 bytes) 2024-09-14T22:37:19.223 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:19.223 INFO:teuthology.orchestra.run.smithi001.stdout:3386 read 0xa906ec4 thru 0xa91374c (0xc889 bytes) 2024-09-14T22:37:19.223 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:19.223 INFO:teuthology.orchestra.run.smithi001.stdout:3389 trunc from 0xd2414c5 to 0x4f28877 2024-09-14T22:37:19.258 INFO:teuthology.orchestra.run.smithi001.stdout:3390 write 0x704972b thru 0x7053c52 (0xa528 bytes) 2024-09-14T22:37:19.259 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:19.255+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:19.294 INFO:teuthology.orchestra.run.smithi001.stdout:3392 write 0xcce9ccc thru 0xccea75d (0xa92 bytes) 2024-09-14T22:37:19.296 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:19.296 INFO:teuthology.orchestra.run.smithi001.stdout:3397 write 0x36ba8be thru 0x36bfb28 (0x526b bytes) 2024-09-14T22:37:19.299 INFO:teuthology.orchestra.run.smithi001.stdout:3398 punch from 0xd8ed48 to 0xd9d298, (0xe550 bytes) 2024-09-14T22:37:19.299 INFO:teuthology.orchestra.run.smithi001.stdout:3399 read 0xba71a07 thru 0xba736ae (0x1ca8 bytes) 2024-09-14T22:37:19.299 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:19.299 INFO:teuthology.orchestra.run.smithi001.stdout:3402 writesame 0x25e5fd thru 0x261baa data_size 0x1(0x35ae bytes) 2024-09-14T22:37:19.310 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:19.310 INFO:teuthology.orchestra.run.smithi001.stdout:3405 write 0x68bb2c7 thru 0x68bdbb9 (0x28f3 bytes) 2024-09-14T22:37:19.313 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:19.313 INFO:teuthology.orchestra.run.smithi001.stdout:3407 write 0x2b3cd86 thru 0x2b4626b (0x94e6 bytes) 2024-09-14T22:37:19.317 INFO:teuthology.orchestra.run.smithi001.stdout:3409 read 0xa977e9f thru 0xa97f92d (0x7a8f bytes) 2024-09-14T22:37:19.317 INFO:teuthology.orchestra.run.smithi001.stdout:3410 write 0x59cc7e0 thru 0x59d74df (0xad00 bytes) 2024-09-14T22:37:19.322 INFO:teuthology.orchestra.run.smithi001.stdout:3412 read 0x8dbe84d thru 0x8dc2044 (0x37f8 bytes) 2024-09-14T22:37:19.322 INFO:teuthology.orchestra.run.smithi001.stdout:3413 read 0x88a46a4 thru 0x88a8719 (0x4076 bytes) 2024-09-14T22:37:19.323 INFO:teuthology.orchestra.run.smithi001.stdout:3415 write 0x7b8a791 thru 0x7b9579b (0xb00b bytes) 2024-09-14T22:37:19.327 INFO:teuthology.orchestra.run.smithi001.stdout:3416 writesame 0x9502e5f thru 0x950a7a9 data_size 0x1(0x794b bytes) 2024-09-14T22:37:19.330 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:19.327+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:19.366 INFO:teuthology.orchestra.run.smithi001.stdout:3417 writesame 0x6c60a0c thru 0x6c6b700 data_size 0x1(0xacf5 bytes) 2024-09-14T22:37:19.391 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:19.391 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:37:19.391 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:37:19.391 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:19.407 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:37:19.412 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:19.412 INFO:teuthology.orchestra.run.smithi001.stdout:3419 writesame 0xb9c9c2c thru 0xb9d1d40 data_size 0x5(0x8115 bytes) 2024-09-14T22:37:19.422 INFO:teuthology.orchestra.run.smithi001.stdout:3420 read 0x5d67907 thru 0x5d73691 (0xbd8b bytes) 2024-09-14T22:37:19.424 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:19.424 INFO:teuthology.orchestra.run.smithi001.stdout:3422 read 0x3c2066e thru 0x3c23ce6 (0x3679 bytes) 2024-09-14T22:37:19.424 INFO:teuthology.orchestra.run.smithi001.stdout:3423 read 0x6960297 thru 0x6961035 (0xd9f bytes) 2024-09-14T22:37:19.424 INFO:teuthology.orchestra.run.smithi001.stdout:3424 read 0x101efa4 thru 0x1025667 (0x66c4 bytes) 2024-09-14T22:37:19.425 INFO:teuthology.orchestra.run.smithi001.stdout:3425 punch from 0x57a7628 to 0x57b3328, (0xbd00 bytes) 2024-09-14T22:37:19.425 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:19.426 INFO:teuthology.orchestra.run.smithi001.stdout:3427 write 0x44d8b29 thru 0x44e139d (0x8875 bytes) 2024-09-14T22:37:19.429 INFO:teuthology.orchestra.run.smithi001.stdout:3428 writesame 0x1a9e67b thru 0x1aa523a data_size 0x20(0x6bc0 bytes) 2024-09-14T22:37:19.430 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:19.423+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:19.433 INFO:teuthology.orchestra.run.smithi001.stdout:3429 read 0x9f76b87 thru 0x9f8504b (0xe4c5 bytes) 2024-09-14T22:37:19.434 INFO:teuthology.orchestra.run.smithi001.stdout:3430 write 0x47b1eec thru 0x47b2020 (0x135 bytes) 2024-09-14T22:37:19.436 INFO:teuthology.orchestra.run.smithi001.stdout:3431 punch from 0x7006651 to 0x7013958, (0xd307 bytes) 2024-09-14T22:37:19.436 INFO:teuthology.orchestra.run.smithi001.stdout:3432 writesame 0x9e91074 thru 0x9e985c1 data_size 0x4e(0x754e bytes) 2024-09-14T22:37:19.440 INFO:teuthology.orchestra.run.smithi001.stdout:3433 read 0x78cda4e thru 0x78ce6df (0xc92 bytes) 2024-09-14T22:37:19.441 INFO:teuthology.orchestra.run.smithi001.stdout:3436 read 0xb1f4ee2 thru 0xb1fb7a6 (0x68c5 bytes) 2024-09-14T22:37:19.447 INFO:teuthology.orchestra.run.smithi001.stdout:3437 trunc from 0xccea75e to 0x978e802 2024-09-14T22:37:19.463 INFO:teuthology.orchestra.run.smithi001.stdout:3438 read 0x64e77f4 thru 0x64f7365 (0xfb72 bytes) 2024-09-14T22:37:19.476 INFO:teuthology.orchestra.run.smithi001.stdout:3439 trunc from 0x978e802 to 0x25d74e9 2024-09-14T22:37:19.516 INFO:teuthology.orchestra.run.smithi001.stdout:3440 write 0x938d57b thru 0x938fc56 (0x26dc bytes) 2024-09-14T22:37:19.520 INFO:teuthology.orchestra.run.smithi001.stdout:3443 read 0x8adba69 thru 0x8ae0859 (0x4df1 bytes) 2024-09-14T22:37:19.521 INFO:teuthology.orchestra.run.smithi001.stdout:3446 writesame 0x805ab62 thru 0x8069b11 data_size 0xd0(0xefb0 bytes) 2024-09-14T22:37:19.524 INFO:teuthology.orchestra.run.smithi001.stdout:3447 read 0x40a2fec thru 0x40a3de2 (0xdf7 bytes) 2024-09-14T22:37:19.530 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:19.527+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:19.541 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:19.541 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:37:19.541 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:37:19.541 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:37:19.541 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:19.542 INFO:teuthology.orchestra.run.smithi001.stdout:3449 write 0xb4a820b thru 0xb4ace02 (0x4bf8 bytes) 2024-09-14T22:37:19.560 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:37:19.563 INFO:teuthology.orchestra.run.smithi001.stdout:3450 trunc from 0xb4ace03 to 0x3953b54 2024-09-14T22:37:19.583 INFO:teuthology.orchestra.run.smithi001.stdout:3452 trunc from 0x3953b54 to 0x3d4c07d 2024-09-14T22:37:19.597 INFO:teuthology.orchestra.run.smithi001.stdout:3453 read 0x27a92f1 thru 0x27aa369 (0x1079 bytes) 2024-09-14T22:37:19.619 INFO:teuthology.orchestra.run.smithi001.stdout:3454 writesame 0x70eff04 thru 0x70f3d19 data_size 0x9(0x3e16 bytes) 2024-09-14T22:37:19.621 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:19.621 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:19.621 INFO:teuthology.orchestra.run.smithi001.stdout:3457 write 0x42e8d3c thru 0x42f8508 (0xf7cd bytes) 2024-09-14T22:37:19.631 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:19.627+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:19.633 INFO:teuthology.orchestra.run.smithi001.stdout:3458 trunc from 0x70f3d1a to 0xb7304d7 2024-09-14T22:37:19.647 INFO:teuthology.orchestra.run.smithi001.stdout:3459 read 0xb424b90 thru 0xb430c18 (0xc089 bytes) 2024-09-14T22:37:19.648 INFO:teuthology.orchestra.run.smithi001.stdout:3460 read 0xd66021 thru 0xd6c860 (0x6840 bytes) 2024-09-14T22:37:19.648 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:19.648 INFO:teuthology.orchestra.run.smithi001.stdout:3463 read 0xab179d4 thru 0xab1af90 (0x35bd bytes) 2024-09-14T22:37:19.649 INFO:teuthology.orchestra.run.smithi001.stdout:3464 read 0x5d240a thru 0x5d305f (0xc56 bytes) 2024-09-14T22:37:19.650 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:19.668 INFO:teuthology.orchestra.run.smithi001.stdout:3466 write 0xd42d8dd thru 0xd43b0fd (0xd821 bytes) 2024-09-14T22:37:19.694 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T22:37:19.695 INFO:teuthology.orchestra.run.smithi001.stdout:3467 trunc from 0xd43b0fe to 0x311020e 2024-09-14T22:37:19.705 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:19.705 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:37:19.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:37:19.705 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:37:19.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.705 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35306 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1425408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:19.705 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507069+0000", 2024-09-14T22:37:19.705 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.196794675, 2024-09-14T22:37:19.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045725259999999997, 2024-09-14T22:37:19.705 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.705 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.705 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.705 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.706 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.706 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.706 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35306 2024-09-14T22:37:19.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.706 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507069+0000", 2024-09-14T22:37:19.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507069+0000", 2024-09-14T22:37:19.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507070+0000", 2024-09-14T22:37:19.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0400000000000002e-07 2024-09-14T22:37:19.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507072+0000", 2024-09-14T22:37:19.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.891e-06 2024-09-14T22:37:19.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507078+0000", 2024-09-14T22:37:19.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0279999999999997e-06 2024-09-14T22:37:19.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507082+0000", 2024-09-14T22:37:19.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4680000000000003e-06 2024-09-14T22:37:19.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552632+0000", 2024-09-14T22:37:19.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045550144000000001 2024-09-14T22:37:19.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552707+0000", 2024-09-14T22:37:19.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4148999999999998e-05 2024-09-14T22:37:19.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552795+0000", 2024-09-14T22:37:19.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8176000000000004e-05 2024-09-14T22:37:19.709 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.709 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.709 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.709 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35307 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1454080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:19.710 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:37:19.710 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.196776124000003, 2024-09-14T22:37:19.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045891358, 2024-09-14T22:37:19.710 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.710 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.710 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.710 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.710 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.710 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.710 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35307 2024-09-14T22:37:19.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.710 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:37:19.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:37:19.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:37:19.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4700000000000002e-07 2024-09-14T22:37:19.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507093+0000", 2024-09-14T22:37:19.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9520000000000002e-06 2024-09-14T22:37:19.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507099+0000", 2024-09-14T22:37:19.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0240000000000003e-06 2024-09-14T22:37:19.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507105+0000", 2024-09-14T22:37:19.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5520000000000004e-06 2024-09-14T22:37:19.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552826+0000", 2024-09-14T22:37:19.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045721494000000001 2024-09-14T22:37:19.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552896+0000", 2024-09-14T22:37:19.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9519999999999998e-05 2024-09-14T22:37:19.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552979+0000", 2024-09-14T22:37:19.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3468999999999997e-05 2024-09-14T22:37:19.714 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.714 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.714 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.714 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35308 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1482752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:19.714 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:37:19.714 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.196753127999997, 2024-09-14T22:37:19.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046076533000000003, 2024-09-14T22:37:19.714 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.714 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.714 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.714 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.715 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.715 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.715 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35308 2024-09-14T22:37:19.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.715 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:37:19.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:37:19.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:37:19.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:37:19.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507113+0000", 2024-09-14T22:37:19.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9099999999999999e-06 2024-09-14T22:37:19.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.717 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507119+0000", 2024-09-14T22:37:19.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7440000000000004e-06 2024-09-14T22:37:19.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.717 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507124+0000", 2024-09-14T22:37:19.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2129999999999999e-06 2024-09-14T22:37:19.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553007+0000", 2024-09-14T22:37:19.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045882488999999999 2024-09-14T22:37:19.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553076+0000", 2024-09-14T22:37:19.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9233000000000006e-05 2024-09-14T22:37:19.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553188+0000", 2024-09-14T22:37:19.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111557 2024-09-14T22:37:19.719 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.719 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.719 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.719 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35309 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1511424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:19.719 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:37:19.719 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.196734223, 2024-09-14T22:37:19.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046231869000000002, 2024-09-14T22:37:19.719 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.719 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.719 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.719 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.719 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.720 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.720 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35309 2024-09-14T22:37:19.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.720 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:37:19.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:37:19.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:37:19.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9099999999999999e-07 2024-09-14T22:37:19.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507132+0000", 2024-09-14T22:37:19.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8190000000000001e-06 2024-09-14T22:37:19.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507139+0000", 2024-09-14T22:37:19.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9419999999999998e-06 2024-09-14T22:37:19.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.722 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507145+0000", 2024-09-14T22:37:19.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2600000000000002e-06 2024-09-14T22:37:19.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.722 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553225+0000", 2024-09-14T22:37:19.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046079991000000001 2024-09-14T22:37:19.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.722 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553284+0000", 2024-09-14T22:37:19.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8638999999999999e-05 2024-09-14T22:37:19.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553362+0000", 2024-09-14T22:37:19.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7826999999999999e-05 2024-09-14T22:37:19.723 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.723 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.723 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.723 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35310 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1540096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:19.723 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507151+0000", 2024-09-14T22:37:19.723 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.196712767000001, 2024-09-14T22:37:19.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046444541999999998, 2024-09-14T22:37:19.723 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.724 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.724 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.724 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.724 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.724 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.724 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35310 2024-09-14T22:37:19.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.724 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507151+0000", 2024-09-14T22:37:19.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507151+0000", 2024-09-14T22:37:19.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507152+0000", 2024-09-14T22:37:19.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1699999999999999e-07 2024-09-14T22:37:19.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507154+0000", 2024-09-14T22:37:19.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.902e-06 2024-09-14T22:37:19.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507159+0000", 2024-09-14T22:37:19.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.823e-06 2024-09-14T22:37:19.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507165+0000", 2024-09-14T22:37:19.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.587e-06 2024-09-14T22:37:19.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553395+0000", 2024-09-14T22:37:19.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046230318999999999 2024-09-14T22:37:19.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553434+0000", 2024-09-14T22:37:19.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8680999999999998e-05 2024-09-14T22:37:19.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553596+0000", 2024-09-14T22:37:19.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016191299999999999 2024-09-14T22:37:19.727 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.727 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.727 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.728 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35311 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1568768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:19.728 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:37:19.728 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.196693514000003, 2024-09-14T22:37:19.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046623592999999998, 2024-09-14T22:37:19.728 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.728 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.728 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.728 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.728 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.728 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.728 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35311 2024-09-14T22:37:19.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.729 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:37:19.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:37:19.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:37:19.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:37:19.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507176+0000", 2024-09-14T22:37:19.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3349999999999997e-06 2024-09-14T22:37:19.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507200+0000", 2024-09-14T22:37:19.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4003e-05 2024-09-14T22:37:19.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507206+0000", 2024-09-14T22:37:19.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.976e-06 2024-09-14T22:37:19.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.731 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.731 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553627+0000", 2024-09-14T22:37:19.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046420284999999999 2024-09-14T22:37:19.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.731 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.731 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553696+0000", 2024-09-14T22:37:19.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9567999999999995e-05 2024-09-14T22:37:19.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.731 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.731 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553794+0000", 2024-09-14T22:37:19.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8060000000000006e-05 2024-09-14T22:37:19.732 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.732 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.732 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.732 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35566 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1484800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:19.732 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514668+0000", 2024-09-14T22:37:19.732 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.189195736999999, 2024-09-14T22:37:19.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045644974999999997, 2024-09-14T22:37:19.732 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.732 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.732 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.732 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.733 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.733 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.733 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35566 2024-09-14T22:37:19.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.733 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514668+0000", 2024-09-14T22:37:19.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514668+0000", 2024-09-14T22:37:19.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514669+0000", 2024-09-14T22:37:19.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9099999999999999e-07 2024-09-14T22:37:19.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514671+0000", 2024-09-14T22:37:19.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3800000000000001e-06 2024-09-14T22:37:19.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514677+0000", 2024-09-14T22:37:19.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9660000000000001e-06 2024-09-14T22:37:19.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514682+0000", 2024-09-14T22:37:19.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0939999999999998e-06 2024-09-14T22:37:19.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560133+0000", 2024-09-14T22:37:19.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045450398000000003 2024-09-14T22:37:19.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560207+0000", 2024-09-14T22:37:19.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4196999999999995e-05 2024-09-14T22:37:19.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560313+0000", 2024-09-14T22:37:19.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106549 2024-09-14T22:37:19.736 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.736 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.736 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.736 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35567 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1513472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:19.736 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:37:19.736 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.189175198000001, 2024-09-14T22:37:19.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045793215999999998, 2024-09-14T22:37:19.736 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.736 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.736 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.736 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.736 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.737 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.737 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35567 2024-09-14T22:37:19.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.737 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:37:19.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:37:19.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:37:19.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1899999999999998e-07 2024-09-14T22:37:19.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514692+0000", 2024-09-14T22:37:19.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2699999999999999e-06 2024-09-14T22:37:19.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514698+0000", 2024-09-14T22:37:19.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0939999999999996e-06 2024-09-14T22:37:19.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514703+0000", 2024-09-14T22:37:19.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3870000000000003e-06 2024-09-14T22:37:19.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560353+0000", 2024-09-14T22:37:19.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045649914 2024-09-14T22:37:19.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560416+0000", 2024-09-14T22:37:19.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3096999999999996e-05 2024-09-14T22:37:19.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560482+0000", 2024-09-14T22:37:19.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6035e-05 2024-09-14T22:37:19.740 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.740 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.740 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.740 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35568 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1542144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:19.740 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:37:19.740 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.189150400000003, 2024-09-14T22:37:19.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045995921000000002, 2024-09-14T22:37:19.740 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.741 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.741 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.741 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.741 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.741 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.741 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35568 2024-09-14T22:37:19.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.741 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:37:19.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:37:19.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:37:19.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4099999999999999e-07 2024-09-14T22:37:19.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514720+0000", 2024-09-14T22:37:19.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6869999999999999e-06 2024-09-14T22:37:19.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514727+0000", 2024-09-14T22:37:19.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3540000000000002e-06 2024-09-14T22:37:19.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514732+0000", 2024-09-14T22:37:19.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7690000000000004e-06 2024-09-14T22:37:19.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560523+0000", 2024-09-14T22:37:19.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045791373000000003 2024-09-14T22:37:19.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560606+0000", 2024-09-14T22:37:19.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2690999999999998e-05 2024-09-14T22:37:19.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560710+0000", 2024-09-14T22:37:19.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000103606 2024-09-14T22:37:19.744 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.744 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.744 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.744 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35569 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1570816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:19.744 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:37:19.744 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.189124591000002, 2024-09-14T22:37:19.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046110630999999999, 2024-09-14T22:37:19.745 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.745 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.745 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.745 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.745 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.745 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.745 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35569 2024-09-14T22:37:19.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.745 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:37:19.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:37:19.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:37:19.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.82e-07 2024-09-14T22:37:19.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514743+0000", 2024-09-14T22:37:19.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.269e-06 2024-09-14T22:37:19.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514750+0000", 2024-09-14T22:37:19.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7340000000000002e-06 2024-09-14T22:37:19.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514755+0000", 2024-09-14T22:37:19.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7260000000000002e-06 2024-09-14T22:37:19.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560731+0000", 2024-09-14T22:37:19.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045976377999999998 2024-09-14T22:37:19.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560777+0000", 2024-09-14T22:37:19.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5868999999999997e-05 2024-09-14T22:37:19.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560850+0000", 2024-09-14T22:37:19.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3172999999999999e-05 2024-09-14T22:37:19.748 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.748 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.749 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.749 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:86180 3.18 3:1d0c52d8:::rbd_data.113cd62ea8a5.000000000000004b:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:19.749 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:01.356417+0000", 2024-09-14T22:37:19.749 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.347447496000001, 2024-09-14T22:37:19.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049143245000000002, 2024-09-14T22:37:19.749 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.749 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.749 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:19.749 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.749 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.749 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.749 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 86180 2024-09-14T22:37:19.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.750 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356417+0000", 2024-09-14T22:37:19.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356417+0000", 2024-09-14T22:37:19.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356420+0000", 2024-09-14T22:37:19.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2030000000000001e-06 2024-09-14T22:37:19.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356435+0000", 2024-09-14T22:37:19.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5287000000000001e-05 2024-09-14T22:37:19.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356472+0000", 2024-09-14T22:37:19.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7311000000000002e-05 2024-09-14T22:37:19.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356492+0000", 2024-09-14T22:37:19.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9519999999999999e-05 2024-09-14T22:37:19.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356670+0000", 2024-09-14T22:37:19.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000177567 2024-09-14T22:37:19.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356924+0000", 2024-09-14T22:37:19.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025399299999999999 2024-09-14T22:37:19.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 0", 2024-09-14T22:37:19.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.365052+0000", 2024-09-14T22:37:19.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0081289380000000005 2024-09-14T22:37:19.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:19.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.405396+0000", 2024-09-14T22:37:19.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040343228000000002 2024-09-14T22:37:19.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:37:19.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.405421+0000", 2024-09-14T22:37:19.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5763e-05 2024-09-14T22:37:19.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:37:19.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.405491+0000", 2024-09-14T22:37:19.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9721999999999994e-05 2024-09-14T22:37:19.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.405560+0000", 2024-09-14T22:37:19.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8713000000000004e-05 2024-09-14T22:37:19.754 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.754 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.755 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.755 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98378 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 919552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:19.755 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148183+0000", 2024-09-14T22:37:19.755 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.5556812180000001, 2024-09-14T22:37:19.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045614549999999997, 2024-09-14T22:37:19.755 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.755 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.755 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.755 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.755 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.755 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.755 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98378 2024-09-14T22:37:19.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.756 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148183+0000", 2024-09-14T22:37:19.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148183+0000", 2024-09-14T22:37:19.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148183+0000", 2024-09-14T22:37:19.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0800000000000001e-07 2024-09-14T22:37:19.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148184+0000", 2024-09-14T22:37:19.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2780000000000001e-06 2024-09-14T22:37:19.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148189+0000", 2024-09-14T22:37:19.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2699999999999998e-06 2024-09-14T22:37:19.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148191+0000", 2024-09-14T22:37:19.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7659999999999999e-06 2024-09-14T22:37:19.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193708+0000", 2024-09-14T22:37:19.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045516562000000003 2024-09-14T22:37:19.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193751+0000", 2024-09-14T22:37:19.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2811000000000001e-05 2024-09-14T22:37:19.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193797+0000", 2024-09-14T22:37:19.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6655e-05 2024-09-14T22:37:19.758 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.758 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.759 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.759 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98379 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 928768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:19.759 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148195+0000", 2024-09-14T22:37:19.759 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.5556693890000002, 2024-09-14T22:37:19.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045817405999999998, 2024-09-14T22:37:19.759 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.759 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.759 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.759 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.759 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.760 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.760 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98379 2024-09-14T22:37:19.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.760 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148195+0000", 2024-09-14T22:37:19.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148195+0000", 2024-09-14T22:37:19.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148195+0000", 2024-09-14T22:37:19.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3999999999999998e-07 2024-09-14T22:37:19.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148196+0000", 2024-09-14T22:37:19.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.341e-06 2024-09-14T22:37:19.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148200+0000", 2024-09-14T22:37:19.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9040000000000002e-06 2024-09-14T22:37:19.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148203+0000", 2024-09-14T22:37:19.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.959e-06 2024-09-14T22:37:19.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193841+0000", 2024-09-14T22:37:19.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045637886000000003 2024-09-14T22:37:19.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193933+0000", 2024-09-14T22:37:19.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1823999999999994e-05 2024-09-14T22:37:19.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194012+0000", 2024-09-14T22:37:19.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9252000000000006e-05 2024-09-14T22:37:19.763 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.763 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.763 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.763 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98380 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 937984~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:19.763 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148206+0000", 2024-09-14T22:37:19.763 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.555657751, 2024-09-14T22:37:19.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045937354, 2024-09-14T22:37:19.763 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.763 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.763 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.764 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.764 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.764 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.764 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98380 2024-09-14T22:37:19.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.764 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148206+0000", 2024-09-14T22:37:19.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148206+0000", 2024-09-14T22:37:19.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148207+0000", 2024-09-14T22:37:19.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1e-07 2024-09-14T22:37:19.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148221+0000", 2024-09-14T22:37:19.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4679e-05 2024-09-14T22:37:19.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148228+0000", 2024-09-14T22:37:19.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3820000000000001e-06 2024-09-14T22:37:19.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148231+0000", 2024-09-14T22:37:19.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2849999999999999e-06 2024-09-14T22:37:19.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194054+0000", 2024-09-14T22:37:19.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045822596 2024-09-14T22:37:19.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194098+0000", 2024-09-14T22:37:19.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4604000000000002e-05 2024-09-14T22:37:19.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194144+0000", 2024-09-14T22:37:19.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5597999999999999e-05 2024-09-14T22:37:19.767 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.767 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.767 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.767 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98381 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 947200~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:19.767 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148236+0000", 2024-09-14T22:37:19.767 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.5556282580000005, 2024-09-14T22:37:19.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046111129000000001, 2024-09-14T22:37:19.768 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.768 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.768 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.768 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.768 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.768 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.768 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98381 2024-09-14T22:37:19.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.768 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.768 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148236+0000", 2024-09-14T22:37:19.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148236+0000", 2024-09-14T22:37:19.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148236+0000", 2024-09-14T22:37:19.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8799999999999998e-07 2024-09-14T22:37:19.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148238+0000", 2024-09-14T22:37:19.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6559999999999999e-06 2024-09-14T22:37:19.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148246+0000", 2024-09-14T22:37:19.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3440000000000001e-06 2024-09-14T22:37:19.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148251+0000", 2024-09-14T22:37:19.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5480000000000003e-06 2024-09-14T22:37:19.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194187+0000", 2024-09-14T22:37:19.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045935847000000002 2024-09-14T22:37:19.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194267+0000", 2024-09-14T22:37:19.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0267999999999998e-05 2024-09-14T22:37:19.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194347+0000", 2024-09-14T22:37:19.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0178000000000004e-05 2024-09-14T22:37:19.771 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.771 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.771 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.771 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98382 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 956416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:19.771 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148254+0000", 2024-09-14T22:37:19.771 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.5556098859999992, 2024-09-14T22:37:19.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046224227, 2024-09-14T22:37:19.772 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.772 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.772 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.772 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.772 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.772 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.772 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98382 2024-09-14T22:37:19.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.772 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148254+0000", 2024-09-14T22:37:19.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148254+0000", 2024-09-14T22:37:19.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148254+0000", 2024-09-14T22:37:19.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3099999999999999e-07 2024-09-14T22:37:19.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148256+0000", 2024-09-14T22:37:19.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5540000000000001e-06 2024-09-14T22:37:19.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148259+0000", 2024-09-14T22:37:19.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2499999999999998e-06 2024-09-14T22:37:19.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148262+0000", 2024-09-14T22:37:19.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8310000000000001e-06 2024-09-14T22:37:19.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194387+0000", 2024-09-14T22:37:19.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046125082999999997 2024-09-14T22:37:19.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194430+0000", 2024-09-14T22:37:19.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3074999999999997e-05 2024-09-14T22:37:19.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194478+0000", 2024-09-14T22:37:19.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8202999999999999e-05 2024-09-14T22:37:19.775 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.775 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.775 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.776 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98383 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 965632~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:19.776 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148265+0000", 2024-09-14T22:37:19.776 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.5555987719999997, 2024-09-14T22:37:19.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046409141000000001, 2024-09-14T22:37:19.776 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.776 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.776 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.776 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.776 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.776 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.776 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98383 2024-09-14T22:37:19.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.776 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148265+0000", 2024-09-14T22:37:19.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148265+0000", 2024-09-14T22:37:19.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148266+0000", 2024-09-14T22:37:19.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0800000000000001e-07 2024-09-14T22:37:19.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148267+0000", 2024-09-14T22:37:19.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1659999999999999e-06 2024-09-14T22:37:19.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148271+0000", 2024-09-14T22:37:19.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1289999999999998e-06 2024-09-14T22:37:19.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148274+0000", 2024-09-14T22:37:19.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5939999999999998e-06 2024-09-14T22:37:19.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194523+0000", 2024-09-14T22:37:19.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046248162000000002 2024-09-14T22:37:19.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194603+0000", 2024-09-14T22:37:19.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0161999999999996e-05 2024-09-14T22:37:19.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194675+0000", 2024-09-14T22:37:19.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1719999999999998e-05 2024-09-14T22:37:19.779 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.779 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.780 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.780 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98384 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 974848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:19.780 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148278+0000", 2024-09-14T22:37:19.780 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.5555864439999993, 2024-09-14T22:37:19.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046527369999999998, 2024-09-14T22:37:19.780 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.780 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.780 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.780 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.780 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.780 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.780 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98384 2024-09-14T22:37:19.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.781 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148278+0000", 2024-09-14T22:37:19.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148278+0000", 2024-09-14T22:37:19.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.149987+0000", 2024-09-14T22:37:19.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017093430000000001 2024-09-14T22:37:19.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.149993+0000", 2024-09-14T22:37:19.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6219999999999997e-06 2024-09-14T22:37:19.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150006+0000", 2024-09-14T22:37:19.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3095000000000001e-05 2024-09-14T22:37:19.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150014+0000", 2024-09-14T22:37:19.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6340000000000007e-06 2024-09-14T22:37:19.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194715+0000", 2024-09-14T22:37:19.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044700349 2024-09-14T22:37:19.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194759+0000", 2024-09-14T22:37:19.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3807000000000003e-05 2024-09-14T22:37:19.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194805+0000", 2024-09-14T22:37:19.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6520000000000002e-05 2024-09-14T22:37:19.783 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.783 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.784 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.784 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98390 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 1030144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:19.784 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.150152+0000", 2024-09-14T22:37:19.784 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.5537117570000003, 2024-09-14T22:37:19.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045642708999999997, 2024-09-14T22:37:19.784 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.784 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.784 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.784 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.784 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.784 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.784 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98390 2024-09-14T22:37:19.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.785 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150152+0000", 2024-09-14T22:37:19.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150152+0000", 2024-09-14T22:37:19.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.786 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150153+0000", 2024-09-14T22:37:19.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.08e-07 2024-09-14T22:37:19.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.786 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150155+0000", 2024-09-14T22:37:19.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2910000000000002e-06 2024-09-14T22:37:19.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.786 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150162+0000", 2024-09-14T22:37:19.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4099999999999996e-06 2024-09-14T22:37:19.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150167+0000", 2024-09-14T22:37:19.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3199999999999999e-06 2024-09-14T22:37:19.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195704+0000", 2024-09-14T22:37:19.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045537243999999998 2024-09-14T22:37:19.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195749+0000", 2024-09-14T22:37:19.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4866e-05 2024-09-14T22:37:19.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195795+0000", 2024-09-14T22:37:19.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.617e-05 2024-09-14T22:37:19.788 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.788 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.788 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.788 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98391 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 1039360~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:19.788 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.150175+0000", 2024-09-14T22:37:19.788 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.5536889209999991, 2024-09-14T22:37:19.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045825232, 2024-09-14T22:37:19.788 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:19.788 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:19.788 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:19.788 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:19.788 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:19.789 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:19.789 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98391 2024-09-14T22:37:19.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.789 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:19.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.789 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:19.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150175+0000", 2024-09-14T22:37:19.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.789 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:19.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150175+0000", 2024-09-14T22:37:19.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:19.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.789 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:19.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150176+0000", 2024-09-14T22:37:19.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9700000000000002e-07 2024-09-14T22:37:19.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:19.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150178+0000", 2024-09-14T22:37:19.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.407e-06 2024-09-14T22:37:19.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:19.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150185+0000", 2024-09-14T22:37:19.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5189999999999998e-06 2024-09-14T22:37:19.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:19.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150190+0000", 2024-09-14T22:37:19.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2399999999999998e-06 2024-09-14T22:37:19.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:19.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195838+0000", 2024-09-14T22:37:19.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045648265 2024-09-14T22:37:19.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:19.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195918+0000", 2024-09-14T22:37:19.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0094999999999998e-05 2024-09-14T22:37:19.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:19.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:19.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:19.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.196000+0000", 2024-09-14T22:37:19.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2309000000000001e-05 2024-09-14T22:37:19.791 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.792 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.792 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.792 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:19.792 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:19.792 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:19.793 INFO:teuthology.orchestra.run.smithi001.stdout:3468 punch from 0x189666e to 0x18a2373, (0xbd05 bytes) 2024-09-14T22:37:19.793 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:19.794 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:37:19.795 INFO:teuthology.orchestra.run.smithi001.stdout:3471 write 0xce80090 thru 0xce83701 (0x3672 bytes) 2024-09-14T22:37:19.795 INFO:teuthology.orchestra.run.smithi001.stdout:3472 write 0x25a07bb thru 0x25a8b0a (0x8350 bytes) 2024-09-14T22:37:19.795 INFO:teuthology.orchestra.run.smithi001.stdout:3473 read 0x10b3373 thru 0x10b3ba2 (0x830 bytes) 2024-09-14T22:37:19.795 INFO:teuthology.orchestra.run.smithi001.stdout:3474 punch from 0xcdcabb9 to 0xcdcbd0d, (0x1154 bytes) 2024-09-14T22:37:19.795 INFO:teuthology.orchestra.run.smithi001.stdout:3475 punch from 0x78a1d83 to 0x78a93af, (0x762c bytes) 2024-09-14T22:37:19.795 INFO:teuthology.orchestra.run.smithi001.stdout:3476 writesame 0x6bf3d1e thru 0x6bf5011 data_size 0x2(0x12f4 bytes) 2024-09-14T22:37:19.795 INFO:teuthology.orchestra.run.smithi001.stdout:3478 punch from 0x645e9a4 to 0x6466dc2, (0x841e bytes) 2024-09-14T22:37:19.795 INFO:teuthology.orchestra.run.smithi001.stdout:3479 trunc from 0xce83702 to 0x7127692 2024-09-14T22:37:19.795 INFO:teuthology.orchestra.run.smithi001.stdout:3484 punch from 0x3c35b23 to 0x3c396bc, (0x3b99 bytes) 2024-09-14T22:37:19.795 INFO:teuthology.orchestra.run.smithi001.stdout:3485 writesame 0x3cfa98c thru 0x3cff780 data_size 0x1(0x4df5 bytes) 2024-09-14T22:37:19.796 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:19.787+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:19.811 INFO:teuthology.orchestra.run.smithi001.stdout:3486 writesame 0xebbfc7d thru 0xebc9c7e data_size 0x3(0xa002 bytes) 2024-09-14T22:37:19.814 INFO:teuthology.orchestra.run.smithi001.stdout:3488 read 0x92bf2b7 thru 0x92c78da (0x8624 bytes) 2024-09-14T22:37:19.815 INFO:teuthology.orchestra.run.smithi001.stdout:3490 write 0x5e7f1de thru 0x5e89f09 (0xad2c bytes) 2024-09-14T22:37:19.818 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:19.818 INFO:teuthology.orchestra.run.smithi001.stdout:3493 write 0x86a4786 thru 0x86b2d9a (0xe615 bytes) 2024-09-14T22:37:19.822 INFO:teuthology.orchestra.run.smithi001.stdout:3494 punch from 0x15109b3 to 0x15157dc, (0x4e29 bytes) 2024-09-14T22:37:19.822 INFO:teuthology.orchestra.run.smithi001.stdout:3495 writesame 0xcb33714 thru 0xcb37282 data_size 0x11(0x3b6f bytes) 2024-09-14T22:37:19.825 INFO:teuthology.orchestra.run.smithi001.stdout:3496 writesame 0x641b74a thru 0x6422391 data_size 0x38(0x6c48 bytes) 2024-09-14T22:37:19.827 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:19.848 INFO:teuthology.orchestra.run.smithi001.stdout:3499 trunc from 0xebc9c7f to 0x2049e08 2024-09-14T22:37:19.862 INFO:teuthology.orchestra.run.smithi001.stdout:3500 write 0x16954fe thru 0x169f7cb (0xa2ce bytes) 2024-09-14T22:37:19.873 INFO:teuthology.orchestra.run.smithi001.stdout:3503 write 0x2868032 thru 0x28688c9 (0x898 bytes) 2024-09-14T22:37:19.875 INFO:teuthology.orchestra.run.smithi001.stdout:3504 read 0x279cf6d thru 0x27a2e15 (0x5ea9 bytes) 2024-09-14T22:37:19.876 INFO:teuthology.orchestra.run.smithi001.stdout:3505 read 0x11b89df thru 0x11bf04c (0x666e bytes) 2024-09-14T22:37:19.893 INFO:teuthology.orchestra.run.smithi001.stdout:3506 trunc from 0x28688ca to 0xc1a87d5 2024-09-14T22:37:19.894 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:19.891+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:19.903 INFO:teuthology.orchestra.run.smithi001.stdout:3507 write 0x4ce3ead thru 0x4ce51c7 (0x131b bytes) 2024-09-14T22:37:19.919 INFO:teuthology.orchestra.run.smithi001.stdout:3508 writesame 0xdee3abd thru 0xdee4a45 data_size 0x1(0xf89 bytes) 2024-09-14T22:37:19.921 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:19.921 INFO:teuthology.orchestra.run.smithi001.stdout:3510 punch from 0xcb943ca to 0xcb969c2, (0x25f8 bytes) 2024-09-14T22:37:19.933 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:19.933 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:19.933 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:37:19.933 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:19.937 INFO:teuthology.orchestra.run.smithi001.stdout:3511 trunc from 0xdee4a46 to 0x2f05c99 2024-09-14T22:37:19.949 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:37:19.951 INFO:teuthology.orchestra.run.smithi001.stdout:3512 punch from 0x188655d to 0x1889def, (0x3892 bytes) 2024-09-14T22:37:19.978 INFO:teuthology.orchestra.run.smithi001.stdout:3513 writesame 0xe0572d5 thru 0xe060cfc data_size 0x4(0x9a28 bytes) 2024-09-14T22:37:19.981 INFO:teuthology.orchestra.run.smithi001.stdout:3514 read 0x1508cb3 thru 0x15136ab (0xa9f9 bytes) 2024-09-14T22:37:19.982 INFO:teuthology.orchestra.run.smithi001.stdout:3515 read 0xda3ddd4 thru 0xda4a7f1 (0xca1e bytes) 2024-09-14T22:37:19.983 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:19.983 INFO:teuthology.orchestra.run.smithi001.stdout:3517 write 0x51c2385 thru 0x51ca5b7 (0x8233 bytes) 2024-09-14T22:37:19.985 INFO:teuthology.orchestra.run.smithi001.stdout:3518 write 0x98d4846 thru 0x98d5c29 (0x13e4 bytes) 2024-09-14T22:37:19.986 INFO:teuthology.orchestra.run.smithi001.stdout:3519 read 0xbd82318 thru 0xbd8f774 (0xd45d bytes) 2024-09-14T22:37:19.987 INFO:teuthology.orchestra.run.smithi001.stdout:3520 punch from 0xa26ae70 to 0xa2769c8, (0xbb58 bytes) 2024-09-14T22:37:19.995 INFO:teuthology.orchestra.run.smithi001.stdout:3523 trunc from 0xe060cfd to 0x950eabe 2024-09-14T22:37:19.999 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:19.995+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:20.005 INFO:teuthology.orchestra.run.smithi001.stdout:3525 read 0x422d5bc thru 0x423c638 (0xf07d bytes) 2024-09-14T22:37:20.006 INFO:teuthology.orchestra.run.smithi001.stdout:3526 write 0x30f631f thru 0x30f8558 (0x223a bytes) 2024-09-14T22:37:20.008 INFO:teuthology.orchestra.run.smithi001.stdout:3527 write 0x290fbd thru 0x295065 (0x40a9 bytes) 2024-09-14T22:37:20.010 INFO:teuthology.orchestra.run.smithi001.stdout:3529 write 0x369e442 thru 0x36a1566 (0x3125 bytes) 2024-09-14T22:37:20.013 INFO:teuthology.orchestra.run.smithi001.stdout:3530 write 0x652f10c thru 0x6533b16 (0x4a0b bytes) 2024-09-14T22:37:20.015 INFO:teuthology.orchestra.run.smithi001.stdout:3531 write 0x761228c thru 0x7620ff7 (0xed6c bytes) 2024-09-14T22:37:20.017 INFO:teuthology.orchestra.run.smithi001.stdout:truncating to largest ever: 0xee65704 2024-09-14T22:37:20.025 INFO:teuthology.orchestra.run.smithi001.stdout:3532 trunc from 0x950eabe to 0xee65704 2024-09-14T22:37:20.033 INFO:teuthology.orchestra.run.smithi001.stdout:3533 write 0x81b57e4 thru 0x81b5bbe (0x3db bytes) 2024-09-14T22:37:20.036 INFO:teuthology.orchestra.run.smithi001.stdout:3535 read 0xa715f2f thru 0xa718735 (0x2807 bytes) 2024-09-14T22:37:20.036 INFO:teuthology.orchestra.run.smithi001.stdout:3536 clone 23 order 24 su 2097152 sc 5 2024-09-14T22:37:20.077 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:20.077 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:20.077 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:37:20.077 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:37:20.077 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:20.091 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:37:20.135 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:20.131+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:20.197 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:20.195+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:20.258 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:20.259 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:20.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:20.259 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:20.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.259 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6377 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8239104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:20.259 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932688+0000", 2024-09-14T22:37:20.259 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.321363859000002, 2024-09-14T22:37:20.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051650696000000003, 2024-09-14T22:37:20.259 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:20.259 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:20.259 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:20.259 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:20.259 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:20.259 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:20.259 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6377 2024-09-14T22:37:20.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.260 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:20.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:20.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932688+0000", 2024-09-14T22:37:20.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:20.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932688+0000", 2024-09-14T22:37:20.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:20.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932689+0000", 2024-09-14T22:37:20.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4309999999999999e-06 2024-09-14T22:37:20.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:20.261 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932695+0000", 2024-09-14T22:37:20.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8889999999999999e-06 2024-09-14T22:37:20.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:20.261 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932713+0000", 2024-09-14T22:37:20.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7416999999999999e-05 2024-09-14T22:37:20.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:20.261 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932726+0000", 2024-09-14T22:37:20.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3145e-05 2024-09-14T22:37:20.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:20.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984181+0000", 2024-09-14T22:37:20.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051455630000000002 2024-09-14T22:37:20.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:20.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984247+0000", 2024-09-14T22:37:20.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5350000000000003e-05 2024-09-14T22:37:20.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:20.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984339+0000", 2024-09-14T22:37:20.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1834000000000002e-05 2024-09-14T22:37:20.263 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.263 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:20.263 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.263 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.263 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.263 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6378 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8247296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:20.263 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932743+0000", 2024-09-14T22:37:20.263 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.321308625, 2024-09-14T22:37:20.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051819373000000002, 2024-09-14T22:37:20.263 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:20.263 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:20.263 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:20.263 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:20.263 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:20.264 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:20.264 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6378 2024-09-14T22:37:20.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.264 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:20.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:20.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932743+0000", 2024-09-14T22:37:20.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:20.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932743+0000", 2024-09-14T22:37:20.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:20.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932745+0000", 2024-09-14T22:37:20.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.637e-06 2024-09-14T22:37:20.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:20.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932750+0000", 2024-09-14T22:37:20.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7859999999999996e-06 2024-09-14T22:37:20.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:20.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932767+0000", 2024-09-14T22:37:20.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7621999999999998e-05 2024-09-14T22:37:20.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:20.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932781+0000", 2024-09-14T22:37:20.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3403e-05 2024-09-14T22:37:20.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:20.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984383+0000", 2024-09-14T22:37:20.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051602249000000003 2024-09-14T22:37:20.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:20.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984461+0000", 2024-09-14T22:37:20.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8467000000000006e-05 2024-09-14T22:37:20.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:20.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984562+0000", 2024-09-14T22:37:20.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101209 2024-09-14T22:37:20.267 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.267 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:20.267 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.267 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6379 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8255488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:20.267 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932796+0000", 2024-09-14T22:37:20.267 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.321255233999999, 2024-09-14T22:37:20.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051961245000000003, 2024-09-14T22:37:20.268 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:20.268 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:20.268 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:20.268 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:20.268 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:20.268 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:20.268 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6379 2024-09-14T22:37:20.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.268 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:20.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.268 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:20.268 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932796+0000", 2024-09-14T22:37:20.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:20.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932796+0000", 2024-09-14T22:37:20.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:20.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932797+0000", 2024-09-14T22:37:20.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9700000000000005e-07 2024-09-14T22:37:20.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:20.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932823+0000", 2024-09-14T22:37:20.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5273999999999999e-05 2024-09-14T22:37:20.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:20.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932840+0000", 2024-09-14T22:37:20.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7774e-05 2024-09-14T22:37:20.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:20.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932862+0000", 2024-09-14T22:37:20.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1957e-05 2024-09-14T22:37:20.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:20.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984603+0000", 2024-09-14T22:37:20.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051740543 2024-09-14T22:37:20.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:20.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984669+0000", 2024-09-14T22:37:20.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.567e-05 2024-09-14T22:37:20.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:20.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984758+0000", 2024-09-14T22:37:20.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9129999999999995e-05 2024-09-14T22:37:20.271 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.271 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:20.271 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.272 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.272 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6380 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8263680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:20.272 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932879+0000", 2024-09-14T22:37:20.272 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.321173154999997, 2024-09-14T22:37:20.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052102860000000001, 2024-09-14T22:37:20.272 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:20.272 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:20.272 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:20.272 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:20.272 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:20.272 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:20.272 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6380 2024-09-14T22:37:20.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.273 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:20.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:20.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932879+0000", 2024-09-14T22:37:20.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:20.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932879+0000", 2024-09-14T22:37:20.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:20.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932880+0000", 2024-09-14T22:37:20.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.054e-06 2024-09-14T22:37:20.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:20.274 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932885+0000", 2024-09-14T22:37:20.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1290000000000004e-06 2024-09-14T22:37:20.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:20.274 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932901+0000", 2024-09-14T22:37:20.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5908000000000001e-05 2024-09-14T22:37:20.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:20.274 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932913+0000", 2024-09-14T22:37:20.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2408999999999999e-05 2024-09-14T22:37:20.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:20.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984800+0000", 2024-09-14T22:37:20.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051886671000000002 2024-09-14T22:37:20.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:20.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984879+0000", 2024-09-14T22:37:20.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9200000000000001e-05 2024-09-14T22:37:20.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:20.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984981+0000", 2024-09-14T22:37:20.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102489 2024-09-14T22:37:20.276 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.276 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:20.276 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.276 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6381 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8271872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:20.276 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932929+0000", 2024-09-14T22:37:20.276 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.321122824, 2024-09-14T22:37:20.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052251484000000001, 2024-09-14T22:37:20.276 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:20.276 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:20.276 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:20.276 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:20.277 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:20.277 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:20.277 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6381 2024-09-14T22:37:20.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.277 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:20.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:20.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932929+0000", 2024-09-14T22:37:20.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:20.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932929+0000", 2024-09-14T22:37:20.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:20.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932930+0000", 2024-09-14T22:37:20.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3399999999999998e-07 2024-09-14T22:37:20.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:20.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932934+0000", 2024-09-14T22:37:20.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5719999999999996e-06 2024-09-14T22:37:20.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:20.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932961+0000", 2024-09-14T22:37:20.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6437000000000001e-05 2024-09-14T22:37:20.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:20.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932973+0000", 2024-09-14T22:37:20.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2097e-05 2024-09-14T22:37:20.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:20.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985021+0000", 2024-09-14T22:37:20.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052047955999999999 2024-09-14T22:37:20.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:20.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985089+0000", 2024-09-14T22:37:20.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7725999999999995e-05 2024-09-14T22:37:20.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.280 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:20.280 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985180+0000", 2024-09-14T22:37:20.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1861999999999996e-05 2024-09-14T22:37:20.280 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.280 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:20.280 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.280 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6382 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8280064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:20.280 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933013+0000", 2024-09-14T22:37:20.280 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.321038661999999, 2024-09-14T22:37:20.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052391118, 2024-09-14T22:37:20.281 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:20.281 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:20.281 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:20.281 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:20.281 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:20.281 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:20.281 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6382 2024-09-14T22:37:20.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.281 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:20.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:20.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933013+0000", 2024-09-14T22:37:20.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:20.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933013+0000", 2024-09-14T22:37:20.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:20.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933014+0000", 2024-09-14T22:37:20.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.375e-06 2024-09-14T22:37:20.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:20.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933020+0000", 2024-09-14T22:37:20.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0220000000000001e-06 2024-09-14T22:37:20.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:20.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933038+0000", 2024-09-14T22:37:20.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7694999999999999e-05 2024-09-14T22:37:20.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:20.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933051+0000", 2024-09-14T22:37:20.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2721999999999999e-05 2024-09-14T22:37:20.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:20.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985224+0000", 2024-09-14T22:37:20.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052172757 2024-09-14T22:37:20.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:20.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985303+0000", 2024-09-14T22:37:20.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9426999999999997e-05 2024-09-14T22:37:20.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:20.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985404+0000", 2024-09-14T22:37:20.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010111999999999999 2024-09-14T22:37:20.284 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.285 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:20.285 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.285 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6383 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8288256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:20.285 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933082+0000", 2024-09-14T22:37:20.285 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.320969681999998, 2024-09-14T22:37:20.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052525761999999997, 2024-09-14T22:37:20.285 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:20.285 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:20.285 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:20.285 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:20.285 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:20.285 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:20.286 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6383 2024-09-14T22:37:20.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.286 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:20.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:20.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933082+0000", 2024-09-14T22:37:20.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:20.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933082+0000", 2024-09-14T22:37:20.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.287 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:20.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933084+0000", 2024-09-14T22:37:20.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5659999999999999e-06 2024-09-14T22:37:20.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.287 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:20.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933089+0000", 2024-09-14T22:37:20.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.271e-06 2024-09-14T22:37:20.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.287 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:20.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933105+0000", 2024-09-14T22:37:20.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5685999999999999e-05 2024-09-14T22:37:20.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.287 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:20.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933118+0000", 2024-09-14T22:37:20.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3188e-05 2024-09-14T22:37:20.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:20.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985448+0000", 2024-09-14T22:37:20.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052330176999999999 2024-09-14T22:37:20.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:20.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985514+0000", 2024-09-14T22:37:20.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6029000000000001e-05 2024-09-14T22:37:20.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:20.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985608+0000", 2024-09-14T22:37:20.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3844999999999993e-05 2024-09-14T22:37:20.289 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.289 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:20.289 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.289 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6384 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8296448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:20.289 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933134+0000", 2024-09-14T22:37:20.289 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.320917323000003, 2024-09-14T22:37:20.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052688835000000003, 2024-09-14T22:37:20.289 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:20.289 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:20.289 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:20.290 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:20.290 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:20.290 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:20.290 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6384 2024-09-14T22:37:20.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.290 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:20.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:20.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933134+0000", 2024-09-14T22:37:20.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:20.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933134+0000", 2024-09-14T22:37:20.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:20.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933135+0000", 2024-09-14T22:37:20.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.78e-07 2024-09-14T22:37:20.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:20.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933141+0000", 2024-09-14T22:37:20.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1540000000000001e-06 2024-09-14T22:37:20.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:20.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933156+0000", 2024-09-14T22:37:20.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5488000000000001e-05 2024-09-14T22:37:20.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:20.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933169+0000", 2024-09-14T22:37:20.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2649e-05 2024-09-14T22:37:20.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:20.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985650+0000", 2024-09-14T22:37:20.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052481112000000003 2024-09-14T22:37:20.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:20.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985724+0000", 2024-09-14T22:37:20.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3953000000000002e-05 2024-09-14T22:37:20.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:20.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985823+0000", 2024-09-14T22:37:20.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9500999999999995e-05 2024-09-14T22:37:20.294 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.294 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:20.294 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.294 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6385 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8304640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:20.294 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933185+0000", 2024-09-14T22:37:20.294 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.320866619999997, 2024-09-14T22:37:20.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052892844000000001, 2024-09-14T22:37:20.294 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:20.294 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:20.294 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:20.294 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:20.295 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:20.295 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:20.295 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6385 2024-09-14T22:37:20.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.295 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:20.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:20.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933185+0000", 2024-09-14T22:37:20.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:20.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933185+0000", 2024-09-14T22:37:20.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:20.296 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933186+0000", 2024-09-14T22:37:20.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0139999999999999e-06 2024-09-14T22:37:20.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:20.296 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933191+0000", 2024-09-14T22:37:20.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7230000000000004e-06 2024-09-14T22:37:20.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:20.296 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933207+0000", 2024-09-14T22:37:20.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.658e-05 2024-09-14T22:37:20.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:20.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933220+0000", 2024-09-14T22:37:20.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2583e-05 2024-09-14T22:37:20.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:20.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985880+0000", 2024-09-14T22:37:20.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052660140000000001 2024-09-14T22:37:20.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:20.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985965+0000", 2024-09-14T22:37:20.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4472e-05 2024-09-14T22:37:20.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:20.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986078+0000", 2024-09-14T22:37:20.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011333200000000001 2024-09-14T22:37:20.298 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.298 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:20.298 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.299 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6386 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8312832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:20.299 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933236+0000", 2024-09-14T22:37:20.299 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.320815725999999, 2024-09-14T22:37:20.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053075342999999997, 2024-09-14T22:37:20.299 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:20.299 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:20.299 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:20.299 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:20.299 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:20.299 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:20.299 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6386 2024-09-14T22:37:20.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.300 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:20.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.300 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:20.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933236+0000", 2024-09-14T22:37:20.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.300 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:20.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933236+0000", 2024-09-14T22:37:20.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.300 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:20.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933237+0000", 2024-09-14T22:37:20.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6500000000000008e-07 2024-09-14T22:37:20.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:20.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933242+0000", 2024-09-14T22:37:20.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.938e-06 2024-09-14T22:37:20.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:20.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933258+0000", 2024-09-14T22:37:20.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6334e-05 2024-09-14T22:37:20.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:20.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933270+0000", 2024-09-14T22:37:20.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1973999999999999e-05 2024-09-14T22:37:20.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:20.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986123+0000", 2024-09-14T22:37:20.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052852769000000001 2024-09-14T22:37:20.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:20.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986204+0000", 2024-09-14T22:37:20.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1458999999999994e-05 2024-09-14T22:37:20.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.303 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:20.303 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986311+0000", 2024-09-14T22:37:20.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010690400000000001 2024-09-14T22:37:20.303 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.303 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:20.303 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.303 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6387 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8321024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:20.303 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933335+0000", 2024-09-14T22:37:20.303 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.320716456, 2024-09-14T22:37:20.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053182363000000003, 2024-09-14T22:37:20.303 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:20.304 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:20.304 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:20.304 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:20.304 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:20.304 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:20.304 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6387 2024-09-14T22:37:20.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.304 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:20.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:20.304 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933335+0000", 2024-09-14T22:37:20.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:20.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933335+0000", 2024-09-14T22:37:20.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:20.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933337+0000", 2024-09-14T22:37:20.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3829999999999999e-06 2024-09-14T22:37:20.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:20.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933343+0000", 2024-09-14T22:37:20.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1489999999999999e-06 2024-09-14T22:37:20.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:20.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933361+0000", 2024-09-14T22:37:20.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8105e-05 2024-09-14T22:37:20.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:20.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933374+0000", 2024-09-14T22:37:20.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.33e-05 2024-09-14T22:37:20.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:20.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986355+0000", 2024-09-14T22:37:20.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052981053 2024-09-14T22:37:20.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:20.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986424+0000", 2024-09-14T22:37:20.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8864999999999999e-05 2024-09-14T22:37:20.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:20.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986518+0000", 2024-09-14T22:37:20.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3508e-05 2024-09-14T22:37:20.307 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.307 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:20.307 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.308 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6388 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8329216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:20.308 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933390+0000", 2024-09-14T22:37:20.308 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.320661870000002, 2024-09-14T22:37:20.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053371605000000003, 2024-09-14T22:37:20.308 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:20.308 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:20.308 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:20.308 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:20.308 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:20.308 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:20.308 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6388 2024-09-14T22:37:20.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.308 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:20.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:20.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933390+0000", 2024-09-14T22:37:20.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:20.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933390+0000", 2024-09-14T22:37:20.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:20.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933391+0000", 2024-09-14T22:37:20.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.443e-06 2024-09-14T22:37:20.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:20.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933396+0000", 2024-09-14T22:37:20.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1680000000000003e-06 2024-09-14T22:37:20.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:20.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933412+0000", 2024-09-14T22:37:20.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5302999999999999e-05 2024-09-14T22:37:20.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:20.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933439+0000", 2024-09-14T22:37:20.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6747999999999999e-05 2024-09-14T22:37:20.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:20.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986562+0000", 2024-09-14T22:37:20.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053123161000000002 2024-09-14T22:37:20.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:20.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986641+0000", 2024-09-14T22:37:20.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9086999999999997e-05 2024-09-14T22:37:20.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:20.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986761+0000", 2024-09-14T22:37:20.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120695 2024-09-14T22:37:20.312 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.312 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:20.312 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.312 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6389 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8337408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:20.312 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933456+0000", 2024-09-14T22:37:20.312 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.320596019, 2024-09-14T22:37:20.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053514182, 2024-09-14T22:37:20.312 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:20.312 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:20.312 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:20.312 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:20.312 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:20.312 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:20.313 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6389 2024-09-14T22:37:20.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.313 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:20.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:20.313 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933456+0000", 2024-09-14T22:37:20.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:20.313 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933456+0000", 2024-09-14T22:37:20.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:20.314 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933457+0000", 2024-09-14T22:37:20.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0020000000000001e-06 2024-09-14T22:37:20.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:20.314 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933462+0000", 2024-09-14T22:37:20.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8400000000000002e-06 2024-09-14T22:37:20.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:20.314 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933477+0000", 2024-09-14T22:37:20.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.592e-05 2024-09-14T22:37:20.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:20.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933489+0000", 2024-09-14T22:37:20.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1955e-05 2024-09-14T22:37:20.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:20.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986798+0000", 2024-09-14T22:37:20.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053308263000000002 2024-09-14T22:37:20.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:20.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986871+0000", 2024-09-14T22:37:20.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3518999999999999e-05 2024-09-14T22:37:20.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:20.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986970+0000", 2024-09-14T22:37:20.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8683000000000004e-05 2024-09-14T22:37:20.316 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.316 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:20.316 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.316 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6390 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8345600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:20.316 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933504+0000", 2024-09-14T22:37:20.316 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.320547234000003, 2024-09-14T22:37:20.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053719579000000003, 2024-09-14T22:37:20.316 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:20.316 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:20.316 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:20.317 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:20.317 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:20.317 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:20.317 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6390 2024-09-14T22:37:20.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.317 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:20.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:20.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933504+0000", 2024-09-14T22:37:20.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:20.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933504+0000", 2024-09-14T22:37:20.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:20.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933505+0000", 2024-09-14T22:37:20.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8299999999999995e-07 2024-09-14T22:37:20.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:20.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933520+0000", 2024-09-14T22:37:20.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4948e-05 2024-09-14T22:37:20.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:20.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933538+0000", 2024-09-14T22:37:20.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7286e-05 2024-09-14T22:37:20.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:20.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933550+0000", 2024-09-14T22:37:20.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2223e-05 2024-09-14T22:37:20.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:20.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987028+0000", 2024-09-14T22:37:20.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053478500999999998 2024-09-14T22:37:20.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:20.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987111+0000", 2024-09-14T22:37:20.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3158999999999994e-05 2024-09-14T22:37:20.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:20.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987224+0000", 2024-09-14T22:37:20.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011257899999999999 2024-09-14T22:37:20.320 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.320 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:20.320 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.320 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6391 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8353792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:20.321 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933576+0000", 2024-09-14T22:37:20.321 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.320475789, 2024-09-14T22:37:20.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053858196999999997, 2024-09-14T22:37:20.321 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:20.321 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:20.321 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:20.321 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:20.321 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:20.321 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:20.321 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6391 2024-09-14T22:37:20.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.321 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:20.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:20.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933576+0000", 2024-09-14T22:37:20.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:20.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933576+0000", 2024-09-14T22:37:20.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:20.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933578+0000", 2024-09-14T22:37:20.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9539999999999998e-06 2024-09-14T22:37:20.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:20.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933584+0000", 2024-09-14T22:37:20.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9030000000000001e-06 2024-09-14T22:37:20.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:20.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933601+0000", 2024-09-14T22:37:20.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7703e-05 2024-09-14T22:37:20.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:20.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933614+0000", 2024-09-14T22:37:20.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2330999999999999e-05 2024-09-14T22:37:20.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:20.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987275+0000", 2024-09-14T22:37:20.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053661470000000003 2024-09-14T22:37:20.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:20.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987342+0000", 2024-09-14T22:37:20.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6266000000000006e-05 2024-09-14T22:37:20.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:20.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987434+0000", 2024-09-14T22:37:20.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2570000000000003e-05 2024-09-14T22:37:20.325 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.325 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:20.325 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.325 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6392 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8361984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:20.325 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933648+0000", 2024-09-14T22:37:20.325 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.320403401, 2024-09-14T22:37:20.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054032430999999999, 2024-09-14T22:37:20.325 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:20.325 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:20.325 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:20.325 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:20.325 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:20.325 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:20.326 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6392 2024-09-14T22:37:20.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.326 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:20.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:20.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933648+0000", 2024-09-14T22:37:20.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:20.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933648+0000", 2024-09-14T22:37:20.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:20.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933650+0000", 2024-09-14T22:37:20.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3969999999999999e-06 2024-09-14T22:37:20.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:20.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933655+0000", 2024-09-14T22:37:20.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7690000000000002e-06 2024-09-14T22:37:20.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:20.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933672+0000", 2024-09-14T22:37:20.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6359999999999999e-05 2024-09-14T22:37:20.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:20.328 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933696+0000", 2024-09-14T22:37:20.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4630999999999998e-05 2024-09-14T22:37:20.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:20.328 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987490+0000", 2024-09-14T22:37:20.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053793256999999997 2024-09-14T22:37:20.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:20.328 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987572+0000", 2024-09-14T22:37:20.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1897999999999994e-05 2024-09-14T22:37:20.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:20.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987681+0000", 2024-09-14T22:37:20.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109119 2024-09-14T22:37:20.329 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.329 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:20.329 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.329 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6393 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8370176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:20.329 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933713+0000", 2024-09-14T22:37:20.329 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.320338446999997, 2024-09-14T22:37:20.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054194111000000003, 2024-09-14T22:37:20.329 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:20.330 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:20.330 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:20.330 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:20.330 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:20.330 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:20.330 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6393 2024-09-14T22:37:20.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.330 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:20.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:20.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933713+0000", 2024-09-14T22:37:20.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:20.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933713+0000", 2024-09-14T22:37:20.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:20.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933714+0000", 2024-09-14T22:37:20.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2109999999999999e-06 2024-09-14T22:37:20.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:20.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933721+0000", 2024-09-14T22:37:20.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2489999999999998e-06 2024-09-14T22:37:20.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:20.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933751+0000", 2024-09-14T22:37:20.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0239999999999998e-05 2024-09-14T22:37:20.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:20.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933764+0000", 2024-09-14T22:37:20.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.322e-05 2024-09-14T22:37:20.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:20.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987737+0000", 2024-09-14T22:37:20.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053972539 2024-09-14T22:37:20.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:20.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987813+0000", 2024-09-14T22:37:20.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6635000000000001e-05 2024-09-14T22:37:20.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:20.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987907+0000", 2024-09-14T22:37:20.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4017000000000005e-05 2024-09-14T22:37:20.333 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.333 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:20.333 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.334 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6394 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8378368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:20.334 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933781+0000", 2024-09-14T22:37:20.334 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.320270688000001, 2024-09-14T22:37:20.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054378276000000003, 2024-09-14T22:37:20.334 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:20.334 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:20.334 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:20.334 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:20.334 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:20.334 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:20.334 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6394 2024-09-14T22:37:20.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.334 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:20.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:20.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933781+0000", 2024-09-14T22:37:20.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:20.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933781+0000", 2024-09-14T22:37:20.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:20.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933782+0000", 2024-09-14T22:37:20.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9199999999999999e-07 2024-09-14T22:37:20.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:20.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933788+0000", 2024-09-14T22:37:20.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9340000000000003e-06 2024-09-14T22:37:20.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:20.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933803+0000", 2024-09-14T22:37:20.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5362999999999998e-05 2024-09-14T22:37:20.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:20.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933816+0000", 2024-09-14T22:37:20.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2456999999999999e-05 2024-09-14T22:37:20.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:20.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987961+0000", 2024-09-14T22:37:20.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054145581999999998 2024-09-14T22:37:20.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:20.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988042+0000", 2024-09-14T22:37:20.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1075000000000006e-05 2024-09-14T22:37:20.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:20.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988159+0000", 2024-09-14T22:37:20.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116873 2024-09-14T22:37:20.338 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.338 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:20.338 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.338 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6395 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8386560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:20.338 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:37:20.338 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.320219794000003, 2024-09-14T22:37:20.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054534379000000001, 2024-09-14T22:37:20.338 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:20.338 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:20.338 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:20.338 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:20.339 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:20.339 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:20.339 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6395 2024-09-14T22:37:20.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.339 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:20.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:20.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:37:20.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:20.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:37:20.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:20.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933833+0000", 2024-09-14T22:37:20.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3859999999999999e-06 2024-09-14T22:37:20.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:20.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933838+0000", 2024-09-14T22:37:20.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8709999999999996e-06 2024-09-14T22:37:20.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:20.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933854+0000", 2024-09-14T22:37:20.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6189e-05 2024-09-14T22:37:20.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:20.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933871+0000", 2024-09-14T22:37:20.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.626e-05 2024-09-14T22:37:20.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:20.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988198+0000", 2024-09-14T22:37:20.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054327581999999999 2024-09-14T22:37:20.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:20.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988274+0000", 2024-09-14T22:37:20.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5759000000000004e-05 2024-09-14T22:37:20.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:20.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988366+0000", 2024-09-14T22:37:20.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2331999999999996e-05 2024-09-14T22:37:20.342 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.342 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:20.342 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.342 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104944 3.3 3:c3780f3b:::rbd_data.113ca561f7a9.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:20.342 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:37:20.342 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.2206179210000001, 2024-09-14T22:37:20.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073041087000000005, 2024-09-14T22:37:20.343 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:20.343 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:20.343 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:20.343 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:20.343 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:20.343 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:20.343 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104944 2024-09-14T22:37:20.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.343 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:20.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:20.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:37:20.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:20.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:37:20.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:20.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:20.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033436+0000", 2024-09-14T22:37:20.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2060000000000001e-06 2024-09-14T22:37:20.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:20.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033448+0000", 2024-09-14T22:37:20.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.169e-05 2024-09-14T22:37:20.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:20.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033474+0000", 2024-09-14T22:37:20.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6400000000000001e-05 2024-09-14T22:37:20.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:20.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033489+0000", 2024-09-14T22:37:20.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5067e-05 2024-09-14T22:37:20.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:20.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033658+0000", 2024-09-14T22:37:20.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000168565 2024-09-14T22:37:20.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:20.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033793+0000", 2024-09-14T22:37:20.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013536900000000001 2024-09-14T22:37:20.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:37:20.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.047559+0000", 2024-09-14T22:37:20.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013766294 2024-09-14T22:37:20.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:20.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106157+0000", 2024-09-14T22:37:20.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058597110000000001 2024-09-14T22:37:20.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.347 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:20.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106290+0000", 2024-09-14T22:37:20.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000133244 2024-09-14T22:37:20.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.347 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:20.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106396+0000", 2024-09-14T22:37:20.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106246 2024-09-14T22:37:20.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:20.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:20.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:20.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106475+0000", 2024-09-14T22:37:20.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8895999999999998e-05 2024-09-14T22:37:20.348 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.348 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:20.348 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.348 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:20.348 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:20.348 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:20.350 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:37:20.350 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:20.343+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:20.355 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:37:20.450 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:20.443+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:20.516 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:37:20.517 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:37:20.519 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:20.519 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:20.519 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:37:20.519 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:20.533 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:37:20.533 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:37:20.534 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:37:20.549 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:37:20.549 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:37:20.550 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:20.547+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:20.566 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:37:20.566 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:37:20.586 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:37:20.587 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:37:20.606 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:37:20.607 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:37:20.629 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:37:20.629 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:37:20.651 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:37:20.651 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:37:20.651 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:20.647+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:20.680 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2024-09-14T22:37:20.682 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-14T22:37:20.682 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:37:20.683 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-14T22:37:20.683 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 1.0 2024-09-14T22:37:20.704 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:20.704 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:20.719 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:37:20.763 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:20.759+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:20.786 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone22 from 0xee65704 (overlap 0x2049e08) to 0x6752ce 2024-09-14T22:37:20.849 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:20.849 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:20.853 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:20.847+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:20.864 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:37:20.901 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-14T22:37:20.955 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:20.951+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:21.037 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:21.037 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:21.051 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:37:21.055 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:21.051+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:21.090 INFO:teuthology.orchestra.run.smithi001.stdout:3537 trunc from 0xee65704 to 0xe11803c 2024-09-14T22:37:21.149 INFO:teuthology.orchestra.run.smithi001.stdout:3541 read 0x2e1fd7d thru 0x2e255c3 (0x5847 bytes) 2024-09-14T22:37:21.150 INFO:teuthology.orchestra.run.smithi001.stdout:3544 write 0xdebc858 thru 0xdec3d16 (0x74bf bytes) 2024-09-14T22:37:21.153 INFO:teuthology.orchestra.run.smithi001.stdout:3545 read 0x7f932db thru 0x7f9eac9 (0xb7ef bytes) 2024-09-14T22:37:21.154 INFO:teuthology.orchestra.run.smithi001.stdout:3547 write 0x2bf888e thru 0x2bf9879 (0xfec bytes) 2024-09-14T22:37:21.155 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:21.147+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:21.180 INFO:teuthology.orchestra.run.smithi001.stdout:3548 read 0x4fb84d4 thru 0x4fbe089 (0x5bb6 bytes) 2024-09-14T22:37:21.196 INFO:teuthology.orchestra.run.smithi001.stdout:3549 trunc from 0xe11803c to 0x73ce1af 2024-09-14T22:37:21.243 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:21.243 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:21.252 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:21.252 INFO:teuthology.orchestra.run.smithi001.stdout:3551 write 0x2f19281 thru 0x2f1f814 (0x6594 bytes) 2024-09-14T22:37:21.256 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:21.251+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:21.258 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:37:21.267 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:21.267 INFO:teuthology.orchestra.run.smithi001.stdout:3554 write 0x268d825 thru 0x269d169 (0xf945 bytes) 2024-09-14T22:37:21.291 INFO:teuthology.orchestra.run.smithi001.stdout:3555 trunc from 0x73ce1af to 0xc59993a 2024-09-14T22:37:21.301 INFO:teuthology.orchestra.run.smithi001.stdout:3556 trunc from 0xc59993a to 0xc3dd7b9 2024-09-14T22:37:21.323 INFO:teuthology.orchestra.run.smithi001.stdout:3557 write 0xd0e87a9 thru 0xd0f1e0f (0x9667 bytes) 2024-09-14T22:37:21.330 INFO:teuthology.orchestra.run.smithi001.stdout:3560 read 0x4ab92ba thru 0x4ac1b94 (0x88db bytes) 2024-09-14T22:37:21.332 INFO:teuthology.orchestra.run.smithi001.stdout:3563 trunc from 0xd0f1e10 to 0xd9a4cbe 2024-09-14T22:37:21.343 INFO:teuthology.orchestra.run.smithi001.stdout:3564 write 0x8e67cb6 thru 0x8e697eb (0x1b36 bytes) 2024-09-14T22:37:21.346 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:21.346 INFO:teuthology.orchestra.run.smithi001.stdout:3567 read 0x7a56000 thru 0x7a57eb6 (0x1eb7 bytes) 2024-09-14T22:37:21.347 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:21.348 INFO:teuthology.orchestra.run.smithi001.stdout:3569 writesame 0x480e9da thru 0x481ca90 data_size 0x1(0xe0b7 bytes) 2024-09-14T22:37:21.356 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:21.351+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:21.420 INFO:teuthology.orchestra.run.smithi001.stdout:3570 trunc from 0xd9a4cbe to 0x955e78d 2024-09-14T22:37:21.426 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:21.427 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:21.442 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:37:21.446 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:21.454 INFO:teuthology.orchestra.run.smithi001.stdout:3572 trunc from 0x955e78d to 0xc1eed99 2024-09-14T22:37:21.467 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:21.482 INFO:teuthology.orchestra.run.smithi001.stdout:3574 write 0xcb5052f thru 0xcb59cf2 (0x97c4 bytes) 2024-09-14T22:37:21.487 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:21.483+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:21.490 INFO:teuthology.orchestra.run.smithi001.stdout:3577 write 0x27be9a1 thru 0x27cde43 (0xf4a3 bytes) 2024-09-14T22:37:21.494 INFO:teuthology.orchestra.run.smithi001.stdout:3578 punch from 0xa3b2322 to 0xa3c0645, (0xe323 bytes) 2024-09-14T22:37:21.494 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:21.494 INFO:teuthology.orchestra.run.smithi001.stdout:3581 writesame 0xac335d1 thru 0xac3d5e3 data_size 0x1(0xa013 bytes) 2024-09-14T22:37:21.545 INFO:teuthology.orchestra.run.smithi001.stdout:3584 punch from 0x84e642e to 0x84ec649, (0x621b bytes) 2024-09-14T22:37:21.545 INFO:teuthology.orchestra.run.smithi001.stdout:3585 write 0x3fd0778 thru 0x3fdcb69 (0xc3f2 bytes) 2024-09-14T22:37:21.549 INFO:teuthology.orchestra.run.smithi001.stdout:3586 write 0x40c7065 thru 0x40d6a7c (0xfa18 bytes) 2024-09-14T22:37:21.557 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:21.555+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:21.565 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:21.586 INFO:teuthology.orchestra.run.smithi001.stdout:3591 trunc from 0xcb59cf3 to 0x50e4a5c 2024-09-14T22:37:21.616 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:21.616 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:21.631 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:37:21.640 INFO:teuthology.orchestra.run.smithi001.stdout:3592 trunc from 0x50e4a5c to 0x4099eba 2024-09-14T22:37:21.675 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:21.671+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:21.695 INFO:teuthology.orchestra.run.smithi001.stdout:moved 4 / 82 (4.87805%) 2024-09-14T22:37:21.696 INFO:teuthology.orchestra.run.smithi001.stdout:avg 10.25 2024-09-14T22:37:21.696 INFO:teuthology.orchestra.run.smithi001.stdout:stddev 3.07205 -> 3.15238 (expected baseline 2.99479) 2024-09-14T22:37:21.696 INFO:teuthology.orchestra.run.smithi001.stdout:min osd.2 with 5 -> 6 pgs (0.487805 -> 0.585366 * mean) 2024-09-14T22:37:21.696 INFO:teuthology.orchestra.run.smithi001.stdout:max osd.7 with 14 -> 16 pgs (1.36585 -> 1.56098 * mean) 2024-09-14T22:37:21.696 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:37:21.696 INFO:teuthology.orchestra.run.smithi001.stdout:oload 110 2024-09-14T22:37:21.696 INFO:teuthology.orchestra.run.smithi001.stdout:max_change 1 2024-09-14T22:37:21.696 INFO:teuthology.orchestra.run.smithi001.stdout:max_change_osds 4 2024-09-14T22:37:21.696 INFO:teuthology.orchestra.run.smithi001.stdout:average_utilization 0.0004 2024-09-14T22:37:21.696 INFO:teuthology.orchestra.run.smithi001.stdout:overload_utilization 0.0005 2024-09-14T22:37:21.696 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3 weight 1.0000 -> 0.7978 2024-09-14T22:37:21.696 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0 weight 1.0000 -> 0.8740 2024-09-14T22:37:21.711 INFO:teuthology.orchestra.run.smithi001.stdout:3594 trunc from 0x4099eba to 0x7780c6c 2024-09-14T22:37:21.726 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-14T22:37:21.726 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 3.0 2024-09-14T22:37:21.727 INFO:teuthology.orchestra.run.smithi001.stdout:3596 trunc from 0x7780c6c to 0x30d3e83 2024-09-14T22:37:21.754 INFO:teuthology.orchestra.run.smithi001.stdout:3599 write 0x6f2c6d4 thru 0x6f3aea7 (0xe7d4 bytes) 2024-09-14T22:37:21.760 INFO:teuthology.orchestra.run.smithi001.stdout:3600 read 0x6b37287 thru 0x6b45251 (0xdfcb bytes) 2024-09-14T22:37:21.760 INFO:teuthology.orchestra.run.smithi001.stdout:3601 read 0x5bddf71 thru 0x5bdfd7e (0x1e0e bytes) 2024-09-14T22:37:21.760 INFO:teuthology.orchestra.run.smithi001.stdout:3602 punch from 0x8cdd4c to 0x8dd9cc, (0xfc80 bytes) 2024-09-14T22:37:21.760 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:21.761 INFO:teuthology.orchestra.run.smithi001.stdout:3604 read 0x5da7397 thru 0x5dad344 (0x5fae bytes) 2024-09-14T22:37:21.761 INFO:teuthology.orchestra.run.smithi001.stdout:3605 read 0x4e58d47 thru 0x4e633c5 (0xa67f bytes) 2024-09-14T22:37:21.761 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:21.761 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:21.762 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:21.759+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:21.769 INFO:teuthology.orchestra.run.smithi001.stdout:3606 writesame 0x7443c99 thru 0x744f993 data_size 0x1(0xbcfb bytes) 2024-09-14T22:37:21.769 INFO:teuthology.orchestra.run.smithi001.stdout:3609 write 0x3e56fdc thru 0x3e60ae8 (0x9b0d bytes) 2024-09-14T22:37:21.776 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:37:21.779 INFO:teuthology.orchestra.run.smithi001.stdout:3610 trunc from 0x744f994 to 0x2f10460 2024-09-14T22:37:21.794 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:21.794 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:21.813 INFO:teuthology.orchestra.run.smithi001.stdout:3614 trunc from 0x2f10460 to 0xe2376b7 2024-09-14T22:37:21.820 INFO:teuthology.orchestra.run.smithi001.stdout:3615 write 0x2638bee thru 0x263daec (0x4eff bytes) 2024-09-14T22:37:21.822 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:21.822 INFO:teuthology.orchestra.run.smithi001.stdout:3617 punch from 0x8350290 to 0x835dc99, (0xda09 bytes) 2024-09-14T22:37:21.822 INFO:teuthology.orchestra.run.smithi001.stdout:3618 write 0xb079e4f thru 0xb088b50 (0xed02 bytes) 2024-09-14T22:37:21.863 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:21.859+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:21.943 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:21.943 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:21.957 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:37:22.005 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:22.003+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:22.065 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:22.059+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:22.067 INFO:teuthology.orchestra.run.smithi001.stdout:moved 0 / 82 (0%) 2024-09-14T22:37:22.067 INFO:teuthology.orchestra.run.smithi001.stdout:avg 10.25 2024-09-14T22:37:22.067 INFO:teuthology.orchestra.run.smithi001.stdout:stddev 3.15238 -> 3.15238 (expected baseline 2.99479) 2024-09-14T22:37:22.067 INFO:teuthology.orchestra.run.smithi001.stdout:min osd.2 with 6 -> 6 pgs (0.585366 -> 0.585366 * mean) 2024-09-14T22:37:22.067 INFO:teuthology.orchestra.run.smithi001.stdout:max osd.7 with 16 -> 16 pgs (1.56098 -> 1.56098 * mean) 2024-09-14T22:37:22.067 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:37:22.067 INFO:teuthology.orchestra.run.smithi001.stdout:oload 1000 2024-09-14T22:37:22.067 INFO:teuthology.orchestra.run.smithi001.stdout:max_change 3 2024-09-14T22:37:22.067 INFO:teuthology.orchestra.run.smithi001.stdout:max_change_osds 4 2024-09-14T22:37:22.068 INFO:teuthology.orchestra.run.smithi001.stdout:average_utilization 0.0004 2024-09-14T22:37:22.068 INFO:teuthology.orchestra.run.smithi001.stdout:overload_utilization 0.0045 2024-09-14T22:37:22.068 INFO:teuthology.orchestra.run.smithi001.stderr:no change 2024-09-14T22:37:22.079 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:22.079 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:22.088 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-14T22:37:22.088 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 1.0 2024-09-14T22:37:22.096 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:37:22.181 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:22.179+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:22.254 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:22.254 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:22.267 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:22.259+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:22.269 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:37:22.368 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:22.363+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:22.433 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:22.434 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:22.440 INFO:tasks.ceph.mgr.x.smithi176.stderr:2024-09-14T22:37:22.435+0000 7f10a3ce0640 -1 mgr.server reply reply (2) No such file or directory no change 2024-09-14T22:37:22.441 INFO:teuthology.orchestra.run.smithi001.stdout:moved 3 / 82 (3.65854%) 2024-09-14T22:37:22.441 INFO:teuthology.orchestra.run.smithi001.stdout:avg 10.25 2024-09-14T22:37:22.441 INFO:teuthology.orchestra.run.smithi001.stdout:stddev 3.15238 -> 3.4187 (expected baseline 2.99479) 2024-09-14T22:37:22.441 INFO:teuthology.orchestra.run.smithi001.stdout:min osd.2 with 6 -> 6 pgs (0.585366 -> 0.585366 * mean) 2024-09-14T22:37:22.441 INFO:teuthology.orchestra.run.smithi001.stdout:max osd.7 with 16 -> 16 pgs (1.56098 -> 1.56098 * mean) 2024-09-14T22:37:22.441 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:37:22.441 INFO:teuthology.orchestra.run.smithi001.stdout:oload 110 2024-09-14T22:37:22.441 INFO:teuthology.orchestra.run.smithi001.stdout:max_change 1 2024-09-14T22:37:22.441 INFO:teuthology.orchestra.run.smithi001.stdout:max_change_osds 4 2024-09-14T22:37:22.442 INFO:teuthology.orchestra.run.smithi001.stdout:average_utilization 0.0004 2024-09-14T22:37:22.442 INFO:teuthology.orchestra.run.smithi001.stdout:overload_utilization 0.0005 2024-09-14T22:37:22.442 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3 weight 0.7978 -> 0.6369 2024-09-14T22:37:22.442 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0 weight 0.8740 -> 0.7643 2024-09-14T22:37:22.442 INFO:teuthology.orchestra.run.smithi001.stderr:no change 2024-09-14T22:37:22.449 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:37:22.461 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-14T22:37:22.461 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-09-14T22:37:22.469 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:22.467+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:22.566 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:22.566 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:22.570 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:22.563+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:22.582 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:37:22.671 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:37:22.671 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:22.667+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:22.704 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T22:37:22.713 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:22.713 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:22.731 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:37:22.773 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:22.771+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:22.818 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:37:22.818 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:37:22.835 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:37:22.835 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:37:22.853 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:37:22.853 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:37:22.869 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:37:22.869 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:37:22.873 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:22.871+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:22.890 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:37:22.890 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:37:22.896 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:22.896 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:37:22.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:37:22.896 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:37:22.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.896 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3022 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14301184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:22.896 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559594+0000", 2024-09-14T22:37:22.896 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.336065458, 2024-09-14T22:37:22.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028916481000000001, 2024-09-14T22:37:22.896 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:22.897 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:22.897 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:22.897 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:22.897 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:22.897 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:22.897 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3022 2024-09-14T22:37:22.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.897 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:22.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:22.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559594+0000", 2024-09-14T22:37:22.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:22.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559594+0000", 2024-09-14T22:37:22.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:22.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559594+0000", 2024-09-14T22:37:22.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7000000000000004e-07 2024-09-14T22:37:22.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:22.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559604+0000", 2024-09-14T22:37:22.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6020000000000006e-06 2024-09-14T22:37:22.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:22.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559615+0000", 2024-09-14T22:37:22.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1032999999999999e-05 2024-09-14T22:37:22.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:22.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559638+0000", 2024-09-14T22:37:22.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3255999999999999e-05 2024-09-14T22:37:22.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:22.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588323+0000", 2024-09-14T22:37:22.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.02868447 2024-09-14T22:37:22.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:22.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588443+0000", 2024-09-14T22:37:22.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120056 2024-09-14T22:37:22.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:22.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588510+0000", 2024-09-14T22:37:22.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7393999999999999e-05 2024-09-14T22:37:22.901 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.901 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:22.901 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.901 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3023 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14309376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:22.901 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559651+0000", 2024-09-14T22:37:22.901 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.336007971999997, 2024-09-14T22:37:22.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029146231000000002, 2024-09-14T22:37:22.901 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:22.901 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:22.901 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:22.901 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:22.901 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:22.902 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:22.902 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3023 2024-09-14T22:37:22.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.902 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:22.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:22.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559651+0000", 2024-09-14T22:37:22.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:22.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559651+0000", 2024-09-14T22:37:22.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:22.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559652+0000", 2024-09-14T22:37:22.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2699999999999998e-07 2024-09-14T22:37:22.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:22.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559656+0000", 2024-09-14T22:37:22.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7019999999999999e-06 2024-09-14T22:37:22.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:22.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559666+0000", 2024-09-14T22:37:22.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0400999999999999e-05 2024-09-14T22:37:22.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.904 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:22.904 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559674+0000", 2024-09-14T22:37:22.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8890000000000002e-06 2024-09-14T22:37:22.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.904 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:22.904 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588565+0000", 2024-09-14T22:37:22.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028891217 2024-09-14T22:37:22.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:22.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588679+0000", 2024-09-14T22:37:22.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114058 2024-09-14T22:37:22.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:22.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588797+0000", 2024-09-14T22:37:22.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118137 2024-09-14T22:37:22.905 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.905 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:22.905 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3024 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14317568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559685+0000", 2024-09-14T22:37:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.335973860999999, 2024-09-14T22:37:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029326498999999999, 2024-09-14T22:37:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3024 2024-09-14T22:37:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:22.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559685+0000", 2024-09-14T22:37:22.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:22.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559685+0000", 2024-09-14T22:37:22.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:22.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559686+0000", 2024-09-14T22:37:22.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0999999999999998e-07 2024-09-14T22:37:22.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559697+0000", 2024-09-14T22:37:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.147e-05 2024-09-14T22:37:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559708+0000", 2024-09-14T22:37:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0546e-05 2024-09-14T22:37:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559716+0000", 2024-09-14T22:37:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.704e-06 2024-09-14T22:37:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:22.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588833+0000", 2024-09-14T22:37:22.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029116442999999999 2024-09-14T22:37:22.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:22.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.588920+0000", 2024-09-14T22:37:22.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6916999999999995e-05 2024-09-14T22:37:22.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:22.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589012+0000", 2024-09-14T22:37:22.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1809000000000001e-05 2024-09-14T22:37:22.910 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.910 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:22.910 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.910 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3025 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14325760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:22.910 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559729+0000", 2024-09-14T22:37:22.910 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.335929962000002, 2024-09-14T22:37:22.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029422501, 2024-09-14T22:37:22.910 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:22.910 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:22.910 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:22.910 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:22.910 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:22.910 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:22.911 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3025 2024-09-14T22:37:22.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.911 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:22.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:22.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559729+0000", 2024-09-14T22:37:22.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:22.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559729+0000", 2024-09-14T22:37:22.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:22.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559730+0000", 2024-09-14T22:37:22.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8100000000000002e-07 2024-09-14T22:37:22.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:22.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559733+0000", 2024-09-14T22:37:22.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4740000000000001e-06 2024-09-14T22:37:22.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:22.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559744+0000", 2024-09-14T22:37:22.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0489000000000001e-05 2024-09-14T22:37:22.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559760+0000", 2024-09-14T22:37:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5817000000000001e-05 2024-09-14T22:37:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589050+0000", 2024-09-14T22:37:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.02929056 2024-09-14T22:37:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589101+0000", 2024-09-14T22:37:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0992000000000002e-05 2024-09-14T22:37:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589152+0000", 2024-09-14T22:37:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0387999999999999e-05 2024-09-14T22:37:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3026 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14333952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559795+0000", 2024-09-14T22:37:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.335864137999998, 2024-09-14T22:37:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029483812000000002, 2024-09-14T22:37:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:22.915 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:22.915 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:22.915 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3026 2024-09-14T22:37:22.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.915 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:22.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:22.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559795+0000", 2024-09-14T22:37:22.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:22.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559795+0000", 2024-09-14T22:37:22.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:22.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559796+0000", 2024-09-14T22:37:22.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3200000000000003e-07 2024-09-14T22:37:22.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:22.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559800+0000", 2024-09-14T22:37:22.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2930000000000002e-06 2024-09-14T22:37:22.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559812+0000", 2024-09-14T22:37:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2053e-05 2024-09-14T22:37:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559821+0000", 2024-09-14T22:37:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8869999999999998e-06 2024-09-14T22:37:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589177+0000", 2024-09-14T22:37:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029356308000000001 2024-09-14T22:37:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:22.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589229+0000", 2024-09-14T22:37:22.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1326000000000003e-05 2024-09-14T22:37:22.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:22.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589279+0000", 2024-09-14T22:37:22.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0012999999999997e-05 2024-09-14T22:37:22.918 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.918 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:22.918 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.918 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3027 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14342144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:22.918 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559833+0000", 2024-09-14T22:37:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.335826410000003, 2024-09-14T22:37:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029619187000000002, 2024-09-14T22:37:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3027 2024-09-14T22:37:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559833+0000", 2024-09-14T22:37:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559833+0000", 2024-09-14T22:37:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559833+0000", 2024-09-14T22:37:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9400000000000005e-07 2024-09-14T22:37:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559845+0000", 2024-09-14T22:37:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1623e-05 2024-09-14T22:37:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559856+0000", 2024-09-14T22:37:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0832999999999999e-05 2024-09-14T22:37:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559864+0000", 2024-09-14T22:37:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3119999999999994e-06 2024-09-14T22:37:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589299+0000", 2024-09-14T22:37:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029434699000000002 2024-09-14T22:37:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589354+0000", 2024-09-14T22:37:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5467999999999998e-05 2024-09-14T22:37:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589452+0000", 2024-09-14T22:37:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7558000000000003e-05 2024-09-14T22:37:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:3028 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 14350336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.559875+0000", 2024-09-14T22:37:22.923 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.335783522, 2024-09-14T22:37:22.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029775586999999999, 2024-09-14T22:37:22.923 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:22.923 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:22.923 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:22.923 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:22.923 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:22.923 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:22.923 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 3028 2024-09-14T22:37:22.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.923 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:22.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:22.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559875+0000", 2024-09-14T22:37:22.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:22.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559875+0000", 2024-09-14T22:37:22.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:22.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559876+0000", 2024-09-14T22:37:22.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5199999999999996e-07 2024-09-14T22:37:22.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:22.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559879+0000", 2024-09-14T22:37:22.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.095e-06 2024-09-14T22:37:22.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:22.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559889+0000", 2024-09-14T22:37:22.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0019e-05 2024-09-14T22:37:22.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:22.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.559897+0000", 2024-09-14T22:37:22.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9189999999999999e-06 2024-09-14T22:37:22.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:22.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589486+0000", 2024-09-14T22:37:22.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029588482999999999 2024-09-14T22:37:22.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:22.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589578+0000", 2024-09-14T22:37:22.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2329000000000003e-05 2024-09-14T22:37:22.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:22.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.589651+0000", 2024-09-14T22:37:22.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2990000000000004e-05 2024-09-14T22:37:22.927 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.927 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:22.927 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.927 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4068 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13252608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:22.927 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680102+0000", 2024-09-14T22:37:22.927 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.215556702999997, 2024-09-14T22:37:22.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028907656, 2024-09-14T22:37:22.927 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:22.927 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:22.927 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:22.928 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:22.928 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:22.928 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:22.928 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4068 2024-09-14T22:37:22.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.928 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:22.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:22.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680102+0000", 2024-09-14T22:37:22.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:22.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680102+0000", 2024-09-14T22:37:22.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680103+0000", 2024-09-14T22:37:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6600000000000002e-07 2024-09-14T22:37:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680105+0000", 2024-09-14T22:37:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.616e-06 2024-09-14T22:37:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680114+0000", 2024-09-14T22:37:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4120000000000001e-06 2024-09-14T22:37:22.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:22.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680119+0000", 2024-09-14T22:37:22.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6210000000000001e-06 2024-09-14T22:37:22.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:22.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.708813+0000", 2024-09-14T22:37:22.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028693330999999999 2024-09-14T22:37:22.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.708903+0000", 2024-09-14T22:37:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0297000000000006e-05 2024-09-14T22:37:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709010+0000", 2024-09-14T22:37:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106913 2024-09-14T22:37:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4069 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13260800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680128+0000", 2024-09-14T22:37:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.215531427999998, 2024-09-14T22:37:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029145476, 2024-09-14T22:37:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4069 2024-09-14T22:37:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680128+0000", 2024-09-14T22:37:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680128+0000", 2024-09-14T22:37:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680128+0000", 2024-09-14T22:37:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7599999999999998e-07 2024-09-14T22:37:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680131+0000", 2024-09-14T22:37:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9050000000000001e-06 2024-09-14T22:37:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680138+0000", 2024-09-14T22:37:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2559999999999996e-06 2024-09-14T22:37:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680144+0000", 2024-09-14T22:37:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4779999999999999e-06 2024-09-14T22:37:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709054+0000", 2024-09-14T22:37:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028910207 2024-09-14T22:37:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709180+0000", 2024-09-14T22:37:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012608699999999999 2024-09-14T22:37:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709273+0000", 2024-09-14T22:37:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3166999999999998e-05 2024-09-14T22:37:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4070 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13268992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680151+0000", 2024-09-14T22:37:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.215508145999998, 2024-09-14T22:37:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029327986, 2024-09-14T22:37:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4070 2024-09-14T22:37:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680151+0000", 2024-09-14T22:37:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680151+0000", 2024-09-14T22:37:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680151+0000", 2024-09-14T22:37:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:37:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680154+0000", 2024-09-14T22:37:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3350000000000001e-06 2024-09-14T22:37:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680162+0000", 2024-09-14T22:37:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.473e-06 2024-09-14T22:37:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680177+0000", 2024-09-14T22:37:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.465e-05 2024-09-14T22:37:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709356+0000", 2024-09-14T22:37:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029178744 2024-09-14T22:37:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709428+0000", 2024-09-14T22:37:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2428000000000005e-05 2024-09-14T22:37:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709479+0000", 2024-09-14T22:37:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0915999999999998e-05 2024-09-14T22:37:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4071 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13277184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680185+0000", 2024-09-14T22:37:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.215473646, 2024-09-14T22:37:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029452392000000001, 2024-09-14T22:37:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4071 2024-09-14T22:37:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680185+0000", 2024-09-14T22:37:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680185+0000", 2024-09-14T22:37:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680186+0000", 2024-09-14T22:37:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6100000000000001e-07 2024-09-14T22:37:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680188+0000", 2024-09-14T22:37:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.509e-06 2024-09-14T22:37:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680197+0000", 2024-09-14T22:37:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8550000000000008e-06 2024-09-14T22:37:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680203+0000", 2024-09-14T22:37:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5849999999999999e-06 2024-09-14T22:37:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709514+0000", 2024-09-14T22:37:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029311054 2024-09-14T22:37:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709585+0000", 2024-09-14T22:37:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0875000000000002e-05 2024-09-14T22:37:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709638+0000", 2024-09-14T22:37:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3053000000000002e-05 2024-09-14T22:37:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4072 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13285376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680211+0000", 2024-09-14T22:37:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.215448365, 2024-09-14T22:37:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029591177999999999, 2024-09-14T22:37:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4072 2024-09-14T22:37:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680211+0000", 2024-09-14T22:37:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680211+0000", 2024-09-14T22:37:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680211+0000", 2024-09-14T22:37:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6699999999999999e-07 2024-09-14T22:37:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680214+0000", 2024-09-14T22:37:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0149999999999999e-06 2024-09-14T22:37:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680222+0000", 2024-09-14T22:37:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.371e-06 2024-09-14T22:37:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680228+0000", 2024-09-14T22:37:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6230000000000002e-06 2024-09-14T22:37:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709670+0000", 2024-09-14T22:37:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029441694000000001 2024-09-14T22:37:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709741+0000", 2024-09-14T22:37:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1668999999999994e-05 2024-09-14T22:37:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709802+0000", 2024-09-14T22:37:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0338999999999999e-05 2024-09-14T22:37:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4073 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13293568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680236+0000", 2024-09-14T22:37:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.215423006000002, 2024-09-14T22:37:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029755238999999999, 2024-09-14T22:37:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4073 2024-09-14T22:37:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680236+0000", 2024-09-14T22:37:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680236+0000", 2024-09-14T22:37:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680236+0000", 2024-09-14T22:37:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5499999999999998e-07 2024-09-14T22:37:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680239+0000", 2024-09-14T22:37:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4380000000000002e-06 2024-09-14T22:37:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680248+0000", 2024-09-14T22:37:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6319999999999997e-06 2024-09-14T22:37:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680261+0000", 2024-09-14T22:37:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3193000000000001e-05 2024-09-14T22:37:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709831+0000", 2024-09-14T22:37:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029570486 2024-09-14T22:37:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709883+0000", 2024-09-14T22:37:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2290999999999998e-05 2024-09-14T22:37:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.709991+0000", 2024-09-14T22:37:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107744 2024-09-14T22:37:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4074 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 13301760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.680269+0000", 2024-09-14T22:37:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.215390108000001, 2024-09-14T22:37:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030007875999999999, 2024-09-14T22:37:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4074 2024-09-14T22:37:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680269+0000", 2024-09-14T22:37:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680269+0000", 2024-09-14T22:37:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680269+0000", 2024-09-14T22:37:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7300000000000001e-07 2024-09-14T22:37:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680277+0000", 2024-09-14T22:37:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8480000000000001e-06 2024-09-14T22:37:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680286+0000", 2024-09-14T22:37:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0529999999999996e-06 2024-09-14T22:37:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.680292+0000", 2024-09-14T22:37:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.728e-06 2024-09-14T22:37:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.710024+0000", 2024-09-14T22:37:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029732176999999999 2024-09-14T22:37:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.710109+0000", 2024-09-14T22:37:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4715000000000005e-05 2024-09-14T22:37:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.710277+0000", 2024-09-14T22:37:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000167882 2024-09-14T22:37:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:12664 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T22:37:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:28.325596+0000", 2024-09-14T22:37:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 54.570062593999999, 2024-09-14T22:37:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028964581, 2024-09-14T22:37:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12664 2024-09-14T22:37:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325596+0000", 2024-09-14T22:37:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325596+0000", 2024-09-14T22:37:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325597+0000", 2024-09-14T22:37:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4300000000000002e-07 2024-09-14T22:37:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325600+0000", 2024-09-14T22:37:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2219999999999998e-06 2024-09-14T22:37:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325610+0000", 2024-09-14T22:37:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2650000000000002e-06 2024-09-14T22:37:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325617+0000", 2024-09-14T22:37:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1199999999999996e-06 2024-09-14T22:37:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354438+0000", 2024-09-14T22:37:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028821205999999999 2024-09-14T22:37:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354495+0000", 2024-09-14T22:37:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6752999999999997e-05 2024-09-14T22:37:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354561+0000", 2024-09-14T22:37:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6272000000000005e-05 2024-09-14T22:37:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:12665 3.1f 3:f8e4fcd6:::rbd_data.113c3b74a67f.0000000000000001:4 [read 391168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T22:37:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:28.325626+0000", 2024-09-14T22:37:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 54.570033272000003, 2024-09-14T22:37:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029080366, 2024-09-14T22:37:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12665 2024-09-14T22:37:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325626+0000", 2024-09-14T22:37:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325626+0000", 2024-09-14T22:37:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325626+0000", 2024-09-14T22:37:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.06e-07 2024-09-14T22:37:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325629+0000", 2024-09-14T22:37:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7149999999999998e-06 2024-09-14T22:37:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325637+0000", 2024-09-14T22:37:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7360000000000001e-06 2024-09-14T22:37:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.325643+0000", 2024-09-14T22:37:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2480000000000001e-06 2024-09-14T22:37:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354589+0000", 2024-09-14T22:37:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028945538 2024-09-14T22:37:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354635+0000", 2024-09-14T22:37:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6374e-05 2024-09-14T22:37:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:28.354706+0000", 2024-09-14T22:37:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1049000000000004e-05 2024-09-14T22:37:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:13927 3.1b 3:ddfd6241:::rbd_data.113cda0183a0.0000000000000009:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-14T22:37:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:31.537348+0000", 2024-09-14T22:37:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.358310672000002, 2024-09-14T22:37:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032156573000000001, 2024-09-14T22:37:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 13927 2024-09-14T22:37:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537348+0000", 2024-09-14T22:37:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537348+0000", 2024-09-14T22:37:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537349+0000", 2024-09-14T22:37:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.068e-06 2024-09-14T22:37:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537358+0000", 2024-09-14T22:37:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.653e-06 2024-09-14T22:37:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537374+0000", 2024-09-14T22:37:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5594e-05 2024-09-14T22:37:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537383+0000", 2024-09-14T22:37:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0929999999999992e-06 2024-09-14T22:37:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537429+0000", 2024-09-14T22:37:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5948000000000001e-05 2024-09-14T22:37:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.537490+0000", 2024-09-14T22:37:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1002000000000003e-05 2024-09-14T22:37:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 4", 2024-09-14T22:37:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.542781+0000", 2024-09-14T22:37:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.005290861 2024-09-14T22:37:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.569350+0000", 2024-09-14T22:37:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026569299000000001 2024-09-14T22:37:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:37:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.569375+0000", 2024-09-14T22:37:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5448000000000001e-05 2024-09-14T22:37:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:37:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.569444+0000", 2024-09-14T22:37:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9184999999999996e-05 2024-09-14T22:37:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:31.569505+0000", 2024-09-14T22:37:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0421999999999998e-05 2024-09-14T22:37:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:85875 3.12 3:498b1eba:::rbd_data.113cd62ea8a5.000000000000006a:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:00.460932+0000", 2024-09-14T22:37:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.434726771000001, 2024-09-14T22:37:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044152693999999999, 2024-09-14T22:37:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 85875 2024-09-14T22:37:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.460932+0000", 2024-09-14T22:37:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.460932+0000", 2024-09-14T22:37:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.460935+0000", 2024-09-14T22:37:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6280000000000001e-06 2024-09-14T22:37:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.460947+0000", 2024-09-14T22:37:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2269000000000001e-05 2024-09-14T22:37:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.460973+0000", 2024-09-14T22:37:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5939999999999999e-05 2024-09-14T22:37:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.460985+0000", 2024-09-14T22:37:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.181e-05 2024-09-14T22:37:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.461102+0000", 2024-09-14T22:37:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000117413 2024-09-14T22:37:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.461349+0000", 2024-09-14T22:37:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00024703300000000002 2024-09-14T22:37:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 7", 2024-09-14T22:37:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.468711+0000", 2024-09-14T22:37:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0073619180000000003 2024-09-14T22:37:22.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:37:22.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.504629+0000", 2024-09-14T22:37:22.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035917273 2024-09-14T22:37:22.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:22.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.504776+0000", 2024-09-14T22:37:22.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014738300000000001 2024-09-14T22:37:22.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:37:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.504951+0000", 2024-09-14T22:37:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017548100000000001 2024-09-14T22:37:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.505085+0000", 2024-09-14T22:37:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013354600000000001 2024-09-14T22:37:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:85883 3.12 3:4cf65cce:::rbd_data.113cd62ea8a5.0000000000000015:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:00.524670+0000", 2024-09-14T22:37:22.979 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.370988746999998, 2024-09-14T22:37:22.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050663316, 2024-09-14T22:37:22.979 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:22.979 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:22.979 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:22.979 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:22.979 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:22.979 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:22.979 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 85883 2024-09-14T22:37:22.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.979 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:22.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:22.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.524670+0000", 2024-09-14T22:37:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.524670+0000", 2024-09-14T22:37:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.524672+0000", 2024-09-14T22:37:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1629999999999999e-06 2024-09-14T22:37:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.524688+0000", 2024-09-14T22:37:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5787e-05 2024-09-14T22:37:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.524720+0000", 2024-09-14T22:37:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1294000000000002e-05 2024-09-14T22:37:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.524736+0000", 2024-09-14T22:37:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.607e-05 2024-09-14T22:37:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.524863+0000", 2024-09-14T22:37:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012693399999999999 2024-09-14T22:37:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.525060+0000", 2024-09-14T22:37:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000197732 2024-09-14T22:37:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 7", 2024-09-14T22:37:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.532377+0000", 2024-09-14T22:37:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0073167680000000004 2024-09-14T22:37:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.575115+0000", 2024-09-14T22:37:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042737626000000001 2024-09-14T22:37:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:37:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.575151+0000", 2024-09-14T22:37:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6211000000000003e-05 2024-09-14T22:37:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:37:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.575245+0000", 2024-09-14T22:37:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4648999999999995e-05 2024-09-14T22:37:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:00.575334+0000", 2024-09-14T22:37:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8082000000000001e-05 2024-09-14T22:37:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:86178 3.1 3:83964f10:::rbd_data.113cd62ea8a5.0000000000000027:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:01.316148+0000", 2024-09-14T22:37:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 21.579511444000001, 2024-09-14T22:37:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.036603839999999999, 2024-09-14T22:37:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 86178 2024-09-14T22:37:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.316148+0000", 2024-09-14T22:37:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.316148+0000", 2024-09-14T22:37:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.316150+0000", 2024-09-14T22:37:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.43e-06 2024-09-14T22:37:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.316163+0000", 2024-09-14T22:37:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2581e-05 2024-09-14T22:37:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.316194+0000", 2024-09-14T22:37:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1794000000000001e-05 2024-09-14T22:37:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.316210+0000", 2024-09-14T22:37:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5947000000000002e-05 2024-09-14T22:37:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.316343+0000", 2024-09-14T22:37:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000132843 2024-09-14T22:37:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.316486+0000", 2024-09-14T22:37:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014334300000000001 2024-09-14T22:37:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 4", 2024-09-14T22:37:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.324078+0000", 2024-09-14T22:37:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0075916990000000004 2024-09-14T22:37:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.352551+0000", 2024-09-14T22:37:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028472515 2024-09-14T22:37:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:37:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.352587+0000", 2024-09-14T22:37:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6196999999999999e-05 2024-09-14T22:37:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:37:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.352676+0000", 2024-09-14T22:37:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8862000000000004e-05 2024-09-14T22:37:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.352751+0000", 2024-09-14T22:37:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5629000000000004e-05 2024-09-14T22:37:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:22.991 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:22.993 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:37:22.993 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:37:22.994 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:37:22.994 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:37:22.994 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:37:22.994 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:37:22.995 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:37:22.995 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:22.991+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:23.094 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:23.091+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:23.157 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:23.157 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:37:23.157 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:37:23.157 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:23.169 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:37:23.194 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:23.191+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:23.295 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:23.291+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:23.335 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:23.335 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:37:23.335 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:37:23.336 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:37:23.336 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:23.348 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:37:23.437 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:23.435+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:23.497 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:23.495+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:23.518 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:23.518 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:37:23.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:37:23.519 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:37:23.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.519 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4149 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13129728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:23.519 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703281+0000", 2024-09-14T22:37:23.519 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.813432456999998, 2024-09-14T22:37:23.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045957114, 2024-09-14T22:37:23.519 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:23.519 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:23.519 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:23.519 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:23.519 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:23.519 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:23.519 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4149 2024-09-14T22:37:23.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.520 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:23.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:23.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703281+0000", 2024-09-14T22:37:23.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:23.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703281+0000", 2024-09-14T22:37:23.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:23.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703282+0000", 2024-09-14T22:37:23.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4199999999999996e-07 2024-09-14T22:37:23.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:23.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703289+0000", 2024-09-14T22:37:23.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0770000000000002e-06 2024-09-14T22:37:23.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:23.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703296+0000", 2024-09-14T22:37:23.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8129999999999998e-06 2024-09-14T22:37:23.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.522 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:23.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703307+0000", 2024-09-14T22:37:23.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1663e-05 2024-09-14T22:37:23.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.522 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:23.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749073+0000", 2024-09-14T22:37:23.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.04576537 2024-09-14T22:37:23.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.522 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:23.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749156+0000", 2024-09-14T22:37:23.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3399999999999994e-05 2024-09-14T22:37:23.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.523 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:23.523 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749239+0000", 2024-09-14T22:37:23.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2249000000000005e-05 2024-09-14T22:37:23.523 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.523 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:23.523 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.523 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4150 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13137920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:23.523 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703318+0000", 2024-09-14T22:37:23.523 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.813396263000001, 2024-09-14T22:37:23.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046170585, 2024-09-14T22:37:23.524 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:23.524 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:23.524 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:23.524 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:23.524 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:23.524 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:23.524 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4150 2024-09-14T22:37:23.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.524 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:23.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:23.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703318+0000", 2024-09-14T22:37:23.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:23.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703318+0000", 2024-09-14T22:37:23.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:23.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703318+0000", 2024-09-14T22:37:23.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8800000000000003e-07 2024-09-14T22:37:23.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:23.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703321+0000", 2024-09-14T22:37:23.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7860000000000001e-06 2024-09-14T22:37:23.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:23.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703328+0000", 2024-09-14T22:37:23.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2810000000000003e-06 2024-09-14T22:37:23.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:23.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703340+0000", 2024-09-14T22:37:23.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1745e-05 2024-09-14T22:37:23.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:23.527 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749286+0000", 2024-09-14T22:37:23.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045946222000000002 2024-09-14T22:37:23.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.527 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:23.527 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749390+0000", 2024-09-14T22:37:23.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000103614 2024-09-14T22:37:23.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.527 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:23.527 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749488+0000", 2024-09-14T22:37:23.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8449000000000006e-05 2024-09-14T22:37:23.527 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.527 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:23.528 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.528 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4151 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13146112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:23.528 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703348+0000", 2024-09-14T22:37:23.528 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.813366100000003, 2024-09-14T22:37:23.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046349509999999997, 2024-09-14T22:37:23.528 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:23.528 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:23.528 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:23.528 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:23.528 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:23.528 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:23.528 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4151 2024-09-14T22:37:23.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.529 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:23.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:23.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703348+0000", 2024-09-14T22:37:23.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:23.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703348+0000", 2024-09-14T22:37:23.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:23.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703348+0000", 2024-09-14T22:37:23.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6899999999999998e-07 2024-09-14T22:37:23.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:23.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703351+0000", 2024-09-14T22:37:23.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5950000000000001e-06 2024-09-14T22:37:23.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:23.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703357+0000", 2024-09-14T22:37:23.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4150000000000004e-06 2024-09-14T22:37:23.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:23.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703364+0000", 2024-09-14T22:37:23.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4069999999999998e-06 2024-09-14T22:37:23.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:23.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749530+0000", 2024-09-14T22:37:23.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046166281000000003 2024-09-14T22:37:23.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:23.532 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749615+0000", 2024-09-14T22:37:23.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5409999999999997e-05 2024-09-14T22:37:23.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.532 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:23.532 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749697+0000", 2024-09-14T22:37:23.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1933000000000003e-05 2024-09-14T22:37:23.532 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.532 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:23.532 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.533 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4152 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13154304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:23.533 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703371+0000", 2024-09-14T22:37:23.533 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.813342698, 2024-09-14T22:37:23.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046552233999999998, 2024-09-14T22:37:23.533 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:23.533 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:23.533 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:23.533 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:23.533 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:23.533 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:23.533 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4152 2024-09-14T22:37:23.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.534 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:23.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:23.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703371+0000", 2024-09-14T22:37:23.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:23.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703371+0000", 2024-09-14T22:37:23.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:23.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703371+0000", 2024-09-14T22:37:23.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.34e-07 2024-09-14T22:37:23.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:23.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703374+0000", 2024-09-14T22:37:23.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.227e-06 2024-09-14T22:37:23.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:23.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703380+0000", 2024-09-14T22:37:23.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4699999999999999e-06 2024-09-14T22:37:23.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:23.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703385+0000", 2024-09-14T22:37:23.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.259e-06 2024-09-14T22:37:23.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:23.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749742+0000", 2024-09-14T22:37:23.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046357031 2024-09-14T22:37:23.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:23.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749825+0000", 2024-09-14T22:37:23.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2674000000000002e-05 2024-09-14T22:37:23.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:23.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749923+0000", 2024-09-14T22:37:23.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8239000000000006e-05 2024-09-14T22:37:23.537 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.537 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:23.537 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.537 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4153 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13162496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:23.537 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703392+0000", 2024-09-14T22:37:23.537 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.813321402, 2024-09-14T22:37:23.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046675756999999998, 2024-09-14T22:37:23.537 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:23.537 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:23.537 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:23.537 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:23.537 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:23.538 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:23.538 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4153 2024-09-14T22:37:23.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.538 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:23.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:23.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703392+0000", 2024-09-14T22:37:23.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:23.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703392+0000", 2024-09-14T22:37:23.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:23.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.705934+0000", 2024-09-14T22:37:23.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025418540000000001 2024-09-14T22:37:23.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:23.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.705940+0000", 2024-09-14T22:37:23.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0950000000000001e-06 2024-09-14T22:37:23.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:23.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.705952+0000", 2024-09-14T22:37:23.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1973e-05 2024-09-14T22:37:23.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:23.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.705958+0000", 2024-09-14T22:37:23.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.541e-06 2024-09-14T22:37:23.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:23.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749969+0000", 2024-09-14T22:37:23.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044011300000000003 2024-09-14T22:37:23.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.541 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:23.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.750020+0000", 2024-09-14T22:37:23.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1054000000000003e-05 2024-09-14T22:37:23.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.541 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:23.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.750068+0000", 2024-09-14T22:37:23.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7939999999999998e-05 2024-09-14T22:37:23.541 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.541 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:23.541 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.541 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4165 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13260800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:23.541 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706178+0000", 2024-09-14T22:37:23.542 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.810535489999999, 2024-09-14T22:37:23.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045915594999999997, 2024-09-14T22:37:23.542 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:23.542 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:23.542 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:23.542 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:23.542 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:23.542 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:23.542 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4165 2024-09-14T22:37:23.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.542 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:23.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.542 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:23.542 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706178+0000", 2024-09-14T22:37:23.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:23.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706178+0000", 2024-09-14T22:37:23.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:23.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706179+0000", 2024-09-14T22:37:23.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4999999999999998e-07 2024-09-14T22:37:23.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:23.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706181+0000", 2024-09-14T22:37:23.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4679999999999999e-06 2024-09-14T22:37:23.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:23.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706189+0000", 2024-09-14T22:37:23.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.994e-06 2024-09-14T22:37:23.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:23.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706195+0000", 2024-09-14T22:37:23.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6119999999999998e-06 2024-09-14T22:37:23.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:23.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752003+0000", 2024-09-14T22:37:23.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045807618000000001 2024-09-14T22:37:23.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.545 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:23.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752050+0000", 2024-09-14T22:37:23.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7301e-05 2024-09-14T22:37:23.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.545 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:23.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752094+0000", 2024-09-14T22:37:23.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4152e-05 2024-09-14T22:37:23.545 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.545 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:23.545 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.546 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4166 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13268992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:23.546 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706205+0000", 2024-09-14T22:37:23.546 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.810509301000003, 2024-09-14T22:37:23.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046092665999999997, 2024-09-14T22:37:23.546 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:23.546 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:23.546 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:23.546 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:23.546 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:23.546 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:23.546 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4166 2024-09-14T22:37:23.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.547 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:23.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:23.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706205+0000", 2024-09-14T22:37:23.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:23.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706205+0000", 2024-09-14T22:37:23.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:23.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706205+0000", 2024-09-14T22:37:23.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:37:23.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:23.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706208+0000", 2024-09-14T22:37:23.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8049999999999998e-06 2024-09-14T22:37:23.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:23.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706215+0000", 2024-09-14T22:37:23.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2010000000000002e-06 2024-09-14T22:37:23.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:23.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706223+0000", 2024-09-14T22:37:23.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7260000000000002e-06 2024-09-14T22:37:23.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:23.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752129+0000", 2024-09-14T22:37:23.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045906532 2024-09-14T22:37:23.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.550 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:23.550 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752213+0000", 2024-09-14T22:37:23.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4060000000000005e-05 2024-09-14T22:37:23.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.550 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:23.550 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752297+0000", 2024-09-14T22:37:23.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3807999999999994e-05 2024-09-14T22:37:23.550 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.550 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:23.551 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.551 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4167 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13277184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:23.551 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706233+0000", 2024-09-14T22:37:23.551 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.810480689999999, 2024-09-14T22:37:23.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046190500000000002, 2024-09-14T22:37:23.551 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:23.551 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:23.551 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:23.551 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:23.551 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:23.551 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:23.551 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4167 2024-09-14T22:37:23.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.552 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:23.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:23.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706233+0000", 2024-09-14T22:37:23.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:23.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706233+0000", 2024-09-14T22:37:23.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:23.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706234+0000", 2024-09-14T22:37:23.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.75e-07 2024-09-14T22:37:23.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:23.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706237+0000", 2024-09-14T22:37:23.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9890000000000001e-06 2024-09-14T22:37:23.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:23.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706244+0000", 2024-09-14T22:37:23.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6240000000000002e-06 2024-09-14T22:37:23.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:23.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706255+0000", 2024-09-14T22:37:23.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0328e-05 2024-09-14T22:37:23.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:23.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752338+0000", 2024-09-14T22:37:23.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046083172999999998 2024-09-14T22:37:23.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.555 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:23.555 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752381+0000", 2024-09-14T22:37:23.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2973999999999999e-05 2024-09-14T22:37:23.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.555 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:23.555 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752424+0000", 2024-09-14T22:37:23.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2936999999999999e-05 2024-09-14T22:37:23.555 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.555 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:23.555 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.556 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4168 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13285376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:23.556 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706261+0000", 2024-09-14T22:37:23.556 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.810453346000003, 2024-09-14T22:37:23.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046378635000000001, 2024-09-14T22:37:23.556 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:23.556 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:23.556 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:23.556 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:23.556 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:23.556 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:23.556 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4168 2024-09-14T22:37:23.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.557 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:23.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:23.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706261+0000", 2024-09-14T22:37:23.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:23.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706261+0000", 2024-09-14T22:37:23.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:23.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706261+0000", 2024-09-14T22:37:23.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2899999999999999e-07 2024-09-14T22:37:23.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:23.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706264+0000", 2024-09-14T22:37:23.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.774e-06 2024-09-14T22:37:23.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:23.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706267+0000", 2024-09-14T22:37:23.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7179999999999998e-06 2024-09-14T22:37:23.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:23.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706274+0000", 2024-09-14T22:37:23.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7599999999999997e-06 2024-09-14T22:37:23.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:23.559 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752458+0000", 2024-09-14T22:37:23.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046183825999999997 2024-09-14T22:37:23.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:23.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752545+0000", 2024-09-14T22:37:23.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7272000000000001e-05 2024-09-14T22:37:23.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.560 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:23.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752639+0000", 2024-09-14T22:37:23.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3956000000000006e-05 2024-09-14T22:37:23.560 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.560 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:23.560 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.560 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4169 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13293568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:23.561 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706278+0000", 2024-09-14T22:37:23.561 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.810435619000003, 2024-09-14T22:37:23.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046489181999999997, 2024-09-14T22:37:23.561 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:23.561 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:23.561 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:23.561 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:23.561 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:23.561 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:23.561 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4169 2024-09-14T22:37:23.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.562 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:23.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:23.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706278+0000", 2024-09-14T22:37:23.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:23.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706278+0000", 2024-09-14T22:37:23.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:23.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706279+0000", 2024-09-14T22:37:23.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6100000000000002e-07 2024-09-14T22:37:23.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:23.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706280+0000", 2024-09-14T22:37:23.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2589999999999999e-06 2024-09-14T22:37:23.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:23.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706283+0000", 2024-09-14T22:37:23.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.501e-06 2024-09-14T22:37:23.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.564 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:23.564 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706286+0000", 2024-09-14T22:37:23.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5409999999999999e-06 2024-09-14T22:37:23.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:23.565 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752680+0000", 2024-09-14T22:37:23.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046393886000000002 2024-09-14T22:37:23.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:23.565 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752723+0000", 2024-09-14T22:37:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2870000000000001e-05 2024-09-14T22:37:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752767+0000", 2024-09-14T22:37:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4864000000000003e-05 2024-09-14T22:37:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4170 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13301760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706289+0000", 2024-09-14T22:37:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.810424720999997, 2024-09-14T22:37:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046691115999999998, 2024-09-14T22:37:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4170 2024-09-14T22:37:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706289+0000", 2024-09-14T22:37:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:23.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706289+0000", 2024-09-14T22:37:23.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:23.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706289+0000", 2024-09-14T22:37:23.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3200000000000001e-07 2024-09-14T22:37:23.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:23.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706296+0000", 2024-09-14T22:37:23.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.922e-06 2024-09-14T22:37:23.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:23.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706303+0000", 2024-09-14T22:37:23.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3980000000000004e-06 2024-09-14T22:37:23.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:23.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706307+0000", 2024-09-14T22:37:23.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6750000000000001e-06 2024-09-14T22:37:23.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:23.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752799+0000", 2024-09-14T22:37:23.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046491609000000003 2024-09-14T22:37:23.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:23.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752884+0000", 2024-09-14T22:37:23.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4876999999999994e-05 2024-09-14T22:37:23.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:23.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752980+0000", 2024-09-14T22:37:23.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6403000000000005e-05 2024-09-14T22:37:23.571 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.571 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:23.571 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.571 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6292 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8329216~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:23.571 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966066+0000", 2024-09-14T22:37:23.571 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.550647783000002, 2024-09-14T22:37:23.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045967367000000002, 2024-09-14T22:37:23.571 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:23.571 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:23.572 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:23.572 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:23.572 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:23.572 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:23.572 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6292 2024-09-14T22:37:23.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.572 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:23.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:23.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966066+0000", 2024-09-14T22:37:23.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:23.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966066+0000", 2024-09-14T22:37:23.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:23.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966067+0000", 2024-09-14T22:37:23.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6499999999999999e-07 2024-09-14T22:37:23.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:23.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966069+0000", 2024-09-14T22:37:23.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4059999999999999e-06 2024-09-14T22:37:23.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:23.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966077+0000", 2024-09-14T22:37:23.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5120000000000002e-06 2024-09-14T22:37:23.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:23.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966083+0000", 2024-09-14T22:37:23.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5590000000000002e-06 2024-09-14T22:37:23.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:23.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.011820+0000", 2024-09-14T22:37:23.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045737281999999997 2024-09-14T22:37:23.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:23.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.011911+0000", 2024-09-14T22:37:23.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0153000000000002e-05 2024-09-14T22:37:23.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:23.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012033+0000", 2024-09-14T22:37:23.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012289000000000001 2024-09-14T22:37:23.576 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.576 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:23.576 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.576 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6293 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8337408~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:23.576 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966091+0000", 2024-09-14T22:37:23.576 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.550622689000001, 2024-09-14T22:37:23.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046181672, 2024-09-14T22:37:23.576 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:23.577 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:23.577 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:23.577 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:23.577 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:23.577 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:23.577 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6293 2024-09-14T22:37:23.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.577 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:23.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:23.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966091+0000", 2024-09-14T22:37:23.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:23.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966091+0000", 2024-09-14T22:37:23.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:23.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966092+0000", 2024-09-14T22:37:23.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:37:23.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:23.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966094+0000", 2024-09-14T22:37:23.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5830000000000001e-06 2024-09-14T22:37:23.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:23.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966102+0000", 2024-09-14T22:37:23.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7470000000000005e-06 2024-09-14T22:37:23.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:23.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966107+0000", 2024-09-14T22:37:23.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1209999999999998e-06 2024-09-14T22:37:23.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:23.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012079+0000", 2024-09-14T22:37:23.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045971953000000003 2024-09-14T22:37:23.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:23.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012168+0000", 2024-09-14T22:37:23.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9398e-05 2024-09-14T22:37:23.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:23.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012273+0000", 2024-09-14T22:37:23.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000104483 2024-09-14T22:37:23.580 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.580 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:23.580 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.581 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6294 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8345600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:23.581 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966115+0000", 2024-09-14T22:37:23.581 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.550599325999997, 2024-09-14T22:37:23.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046297036999999999, 2024-09-14T22:37:23.581 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:23.581 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:23.581 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:23.581 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:23.581 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:23.581 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:23.581 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6294 2024-09-14T22:37:23.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.582 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:23.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:23.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966115+0000", 2024-09-14T22:37:23.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:23.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966115+0000", 2024-09-14T22:37:23.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:23.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966115+0000", 2024-09-14T22:37:23.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7e-07 2024-09-14T22:37:23.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:23.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966118+0000", 2024-09-14T22:37:23.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.571e-06 2024-09-14T22:37:23.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:23.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966125+0000", 2024-09-14T22:37:23.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1690000000000003e-06 2024-09-14T22:37:23.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:23.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966130+0000", 2024-09-14T22:37:23.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3290000000000001e-06 2024-09-14T22:37:23.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:23.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012298+0000", 2024-09-14T22:37:23.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046168168000000002 2024-09-14T22:37:23.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:23.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012348+0000", 2024-09-14T22:37:23.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9663999999999997e-05 2024-09-14T22:37:23.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:23.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012412+0000", 2024-09-14T22:37:23.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3665999999999997e-05 2024-09-14T22:37:23.585 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.585 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:23.585 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.585 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6295 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8353792~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:23.585 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966138+0000", 2024-09-14T22:37:23.585 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.550576348, 2024-09-14T22:37:23.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046505112000000001, 2024-09-14T22:37:23.586 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:23.586 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:23.586 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:23.586 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:23.586 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:23.586 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:23.586 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6295 2024-09-14T22:37:23.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.586 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:23.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:23.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966138+0000", 2024-09-14T22:37:23.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:23.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966138+0000", 2024-09-14T22:37:23.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:23.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966138+0000", 2024-09-14T22:37:23.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5900000000000002e-07 2024-09-14T22:37:23.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:23.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966140+0000", 2024-09-14T22:37:23.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4710000000000001e-06 2024-09-14T22:37:23.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:23.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966147+0000", 2024-09-14T22:37:23.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6760000000000001e-06 2024-09-14T22:37:23.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:23.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966153+0000", 2024-09-14T22:37:23.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4779999999999999e-06 2024-09-14T22:37:23.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:23.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012452+0000", 2024-09-14T22:37:23.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046299298000000003 2024-09-14T22:37:23.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:23.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012537+0000", 2024-09-14T22:37:23.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5439999999999995e-05 2024-09-14T22:37:23.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:23.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012643+0000", 2024-09-14T22:37:23.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010529 2024-09-14T22:37:23.590 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.590 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:23.590 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.590 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6296 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8361984~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:23.590 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966160+0000", 2024-09-14T22:37:23.590 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.550554294000001, 2024-09-14T22:37:23.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046601423000000003, 2024-09-14T22:37:23.590 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:23.591 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:23.591 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:23.591 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:23.591 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:23.591 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:23.591 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6296 2024-09-14T22:37:23.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.591 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:23.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:23.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966160+0000", 2024-09-14T22:37:23.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:23.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966160+0000", 2024-09-14T22:37:23.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:23.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966160+0000", 2024-09-14T22:37:23.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7999999999999995e-07 2024-09-14T22:37:23.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:23.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966163+0000", 2024-09-14T22:37:23.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.514e-06 2024-09-14T22:37:23.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:23.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966169+0000", 2024-09-14T22:37:23.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8880000000000002e-06 2024-09-14T22:37:23.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:23.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966173+0000", 2024-09-14T22:37:23.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9189999999999998e-06 2024-09-14T22:37:23.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:23.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012660+0000", 2024-09-14T22:37:23.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046486522000000002 2024-09-14T22:37:23.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:23.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012703+0000", 2024-09-14T22:37:23.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2715e-05 2024-09-14T22:37:23.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:23.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012761+0000", 2024-09-14T22:37:23.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8285000000000002e-05 2024-09-14T22:37:23.594 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.594 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:23.594 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.595 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6297 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8370176~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:23.595 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966180+0000", 2024-09-14T22:37:23.595 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.550533489999999, 2024-09-14T22:37:23.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046808482999999998, 2024-09-14T22:37:23.595 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:23.595 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:23.595 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:23.595 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:23.595 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:23.595 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:23.595 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6297 2024-09-14T22:37:23.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.596 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:23.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:23.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966180+0000", 2024-09-14T22:37:23.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:23.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966180+0000", 2024-09-14T22:37:23.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:23.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966181+0000", 2024-09-14T22:37:23.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0600000000000001e-07 2024-09-14T22:37:23.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:23.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966183+0000", 2024-09-14T22:37:23.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3539999999999998e-06 2024-09-14T22:37:23.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:23.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966190+0000", 2024-09-14T22:37:23.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.489e-06 2024-09-14T22:37:23.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:23.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966195+0000", 2024-09-14T22:37:23.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3890000000000004e-06 2024-09-14T22:37:23.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:23.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012799+0000", 2024-09-14T22:37:23.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046604200999999998 2024-09-14T22:37:23.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:23.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012883+0000", 2024-09-14T22:37:23.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3714000000000005e-05 2024-09-14T22:37:23.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:23.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012989+0000", 2024-09-14T22:37:23.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010593 2024-09-14T22:37:23.599 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.599 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:23.599 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.599 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6298 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8378368~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:23.599 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966203+0000", 2024-09-14T22:37:23.599 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.550510428999999, 2024-09-14T22:37:23.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046904649999999999, 2024-09-14T22:37:23.599 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:23.599 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:23.599 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:23.600 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:23.600 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:23.600 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:23.600 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6298 2024-09-14T22:37:23.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.600 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:23.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:23.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966203+0000", 2024-09-14T22:37:23.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:23.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966203+0000", 2024-09-14T22:37:23.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:23.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966205+0000", 2024-09-14T22:37:23.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5009999999999999e-06 2024-09-14T22:37:23.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:23.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966207+0000", 2024-09-14T22:37:23.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2019999999999998e-06 2024-09-14T22:37:23.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:23.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966214+0000", 2024-09-14T22:37:23.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.815e-06 2024-09-14T22:37:23.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:23.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966219+0000", 2024-09-14T22:37:23.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0590000000000002e-06 2024-09-14T22:37:23.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:23.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013005+0000", 2024-09-14T22:37:23.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046785852000000003 2024-09-14T22:37:23.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:23.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013050+0000", 2024-09-14T22:37:23.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.511e-05 2024-09-14T22:37:23.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:23.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013108+0000", 2024-09-14T22:37:23.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8111e-05 2024-09-14T22:37:23.606 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.606 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:23.606 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.606 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6299 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8386560~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:23.606 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966238+0000", 2024-09-14T22:37:23.606 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.550476107999998, 2024-09-14T22:37:23.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047091099999999997, 2024-09-14T22:37:23.606 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:23.606 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:23.606 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:23.606 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:23.606 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:23.607 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:23.607 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6299 2024-09-14T22:37:23.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.607 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:23.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:23.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966238+0000", 2024-09-14T22:37:23.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:23.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966238+0000", 2024-09-14T22:37:23.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:23.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966238+0000", 2024-09-14T22:37:23.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1500000000000005e-07 2024-09-14T22:37:23.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:23.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966241+0000", 2024-09-14T22:37:23.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8270000000000002e-06 2024-09-14T22:37:23.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:23.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966249+0000", 2024-09-14T22:37:23.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0029999999999994e-06 2024-09-14T22:37:23.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:23.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966255+0000", 2024-09-14T22:37:23.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7760000000000003e-06 2024-09-14T22:37:23.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:23.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013136+0000", 2024-09-14T22:37:23.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046881379000000001 2024-09-14T22:37:23.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:23.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013223+0000", 2024-09-14T22:37:23.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6681000000000006e-05 2024-09-14T22:37:23.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:23.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013329+0000", 2024-09-14T22:37:23.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105919 2024-09-14T22:37:23.610 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.610 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:23.610 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.610 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:31370 3.0 3:02f48f58:::rbd_data.113cda0183a0.0000000000000000:6 [read 813056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:23.610 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:37:23.610 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.154316014999999, 2024-09-14T22:37:23.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047519796000000003, 2024-09-14T22:37:23.610 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:23.610 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:23.611 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:23.611 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:23.611 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:23.611 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:23.611 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 31370 2024-09-14T22:37:23.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.611 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:23.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:23.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:37:23.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:23.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:37:23.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:23.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:23.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:37:23.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0999999999999999e-07 2024-09-14T22:37:23.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:23.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362401+0000", 2024-09-14T22:37:23.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2900000000000001e-06 2024-09-14T22:37:23.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:23.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362406+0000", 2024-09-14T22:37:23.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6960000000000002e-06 2024-09-14T22:37:23.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:23.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362412+0000", 2024-09-14T22:37:23.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.823e-06 2024-09-14T22:37:23.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:23.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.400270+0000", 2024-09-14T22:37:23.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037857737000000002 2024-09-14T22:37:23.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:23.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.400342+0000", 2024-09-14T22:37:23.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1797000000000004e-05 2024-09-14T22:37:23.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:23.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:23.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:23.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.409918+0000", 2024-09-14T22:37:23.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0095760429999999994 2024-09-14T22:37:23.614 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.614 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:23.614 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.614 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:23.614 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:23.614 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:23.616 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:37:23.616 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:23.611+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:23.699 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2024-09-14T22:37:23.708 INFO:teuthology.orchestra.run.smithi001.stdout:moved 1 / 82 (1.21951%) 2024-09-14T22:37:23.708 INFO:teuthology.orchestra.run.smithi001.stdout:avg 10.25 2024-09-14T22:37:23.708 INFO:teuthology.orchestra.run.smithi001.stdout:stddev 3.15238 -> 3.23071 (expected baseline 2.99479) 2024-09-14T22:37:23.708 INFO:teuthology.orchestra.run.smithi001.stdout:min osd.2 with 6 -> 6 pgs (0.585366 -> 0.585366 * mean) 2024-09-14T22:37:23.708 INFO:teuthology.orchestra.run.smithi001.stdout:max osd.7 with 16 -> 16 pgs (1.56098 -> 1.56098 * mean) 2024-09-14T22:37:23.708 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:37:23.709 INFO:teuthology.orchestra.run.smithi001.stdout:oload 110 2024-09-14T22:37:23.709 INFO:teuthology.orchestra.run.smithi001.stdout:max_change 0.05 2024-09-14T22:37:23.709 INFO:teuthology.orchestra.run.smithi001.stdout:max_change_osds 4 2024-09-14T22:37:23.709 INFO:teuthology.orchestra.run.smithi001.stdout:average_utilization 0.0004 2024-09-14T22:37:23.709 INFO:teuthology.orchestra.run.smithi001.stdout:overload_utilization 0.0005 2024-09-14T22:37:23.709 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3 weight 0.7978 -> 0.7478 2024-09-14T22:37:23.709 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0 weight 0.8740 -> 0.8240 2024-09-14T22:37:23.716 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:23.715+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:23.743 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-14T22:37:23.743 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 3.0 2024-09-14T22:37:23.808 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:23.808 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:37:23.808 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:37:23.808 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:23.817 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:23.815+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:23.823 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:37:23.906 INFO:teuthology.orchestra.run.smithi001.stdout:3619 punch from 0xdc14a66 to 0xdc23c30, (0xf1ca bytes) 2024-09-14T22:37:23.906 INFO:teuthology.orchestra.run.smithi001.stdout:3621 writesame 0x1d877d5 thru 0x1d8f1ca data_size 0x43(0x79f6 bytes) 2024-09-14T22:37:23.910 INFO:teuthology.orchestra.run.smithi001.stdout:3622 write 0xa9b0cc5 thru 0xa9bb1df (0xa51b bytes) 2024-09-14T22:37:23.916 INFO:teuthology.orchestra.run.smithi001.stdout:3623 write 0x9d777f3 thru 0x9d85df0 (0xe5fe bytes) 2024-09-14T22:37:23.918 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:23.911+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:23.921 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:23.922 INFO:teuthology.orchestra.run.smithi001.stdout:3626 write 0x3b48429 thru 0x3b523b4 (0x9f8c bytes) 2024-09-14T22:37:23.931 INFO:teuthology.orchestra.run.smithi001.stdout:3628 trunc from 0xe2376b7 to 0x846ac08 2024-09-14T22:37:23.948 INFO:teuthology.orchestra.run.smithi001.stdout:3629 writesame 0x537685e thru 0x537cafe data_size 0x1(0x62a1 bytes) 2024-09-14T22:37:23.982 INFO:teuthology.orchestra.run.smithi001.stdout:3630 write 0x7cde820 thru 0x7ce57a6 (0x6f87 bytes) 2024-09-14T22:37:23.992 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:23.992 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:37:23.992 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:37:23.992 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:37:23.992 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:24.001 INFO:teuthology.orchestra.run.smithi001.stdout:3633 write 0xd6296c3 thru 0xd62e845 (0x5183 bytes) 2024-09-14T22:37:24.006 INFO:teuthology.orchestra.run.smithi001.stdout:3634 read 0x54bcde1 thru 0x54bf454 (0x2674 bytes) 2024-09-14T22:37:24.007 INFO:teuthology.orchestra.run.smithi001.stdout:3635 write 0xb3c2e83 thru 0xb3cad9c (0x7f1a bytes) 2024-09-14T22:37:24.009 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:37:24.013 INFO:teuthology.orchestra.run.smithi001.stdout:3636 read 0xcdca5f6 thru 0xcdd75f3 (0xcffe bytes) 2024-09-14T22:37:24.015 INFO:teuthology.orchestra.run.smithi001.stdout:3638 read 0x5d8a941 thru 0x5d8f599 (0x4c59 bytes) 2024-09-14T22:37:24.015 INFO:teuthology.orchestra.run.smithi001.stdout:3640 writesame 0x266c676 thru 0x267a3a7 data_size 0x17(0xdd32 bytes) 2024-09-14T22:37:24.020 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:24.015+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:24.020 INFO:teuthology.orchestra.run.smithi001.stdout:3641 write 0xb1c5625 thru 0xb1d2dc5 (0xd7a1 bytes) 2024-09-14T22:37:24.041 INFO:teuthology.orchestra.run.smithi001.stdout:3642 trunc from 0xd62e846 to 0x348231e 2024-09-14T22:37:24.065 INFO:teuthology.orchestra.run.smithi001.stdout:3644 write 0x2fcbf59 thru 0x2fd1f6a (0x6012 bytes) 2024-09-14T22:37:24.068 INFO:teuthology.orchestra.run.smithi001.stdout:3645 write 0x32d3c3c thru 0x32dbd5d (0x8122 bytes) 2024-09-14T22:37:24.086 INFO:teuthology.orchestra.run.smithi001.stdout:3647 writesame 0x90db89a thru 0x90ea77d data_size 0x2(0xeee4 bytes) 2024-09-14T22:37:24.086 INFO:teuthology.orchestra.run.smithi001.stdout:3650 trunc from 0x90ea77e to 0x984d3ff 2024-09-14T22:37:24.095 INFO:teuthology.orchestra.run.smithi001.stdout:3651 punch from 0x3fad1b8 to 0x3fafa87, (0x28cf bytes) 2024-09-14T22:37:24.096 INFO:teuthology.orchestra.run.smithi001.stdout:3652 read 0x28a353f thru 0x28a86b8 (0x517a bytes) 2024-09-14T22:37:24.096 INFO:teuthology.orchestra.run.smithi001.stdout:3653 read 0x68ed4e6 thru 0x68fb635 (0xe150 bytes) 2024-09-14T22:37:24.097 INFO:teuthology.orchestra.run.smithi001.stdout:3654 flatten 2024-09-14T22:37:24.106 INFO:teuthology.orchestra.run.smithi001.stdout:3655 write 0x5a2fb43 thru 0x5a3147f (0x193d bytes) 2024-09-14T22:37:24.110 INFO:teuthology.orchestra.run.smithi001.stdout:3657 write 0x5bb6700 thru 0x5bbd712 (0x7013 bytes) 2024-09-14T22:37:24.112 INFO:teuthology.orchestra.run.smithi001.stdout:3659 punch from 0x5d9e269 to 0x5dab230, (0xcfc7 bytes) 2024-09-14T22:37:24.112 INFO:teuthology.orchestra.run.smithi001.stdout:3660 writesame 0x4166c6e thru 0x416a2e8 data_size 0x1(0x367b bytes) 2024-09-14T22:37:24.120 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:24.115+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:24.139 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:24.139 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:37:24.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:37:24.139 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:37:24.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.140 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35306 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1425408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:24.140 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507069+0000", 2024-09-14T22:37:24.140 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.631237411999997, 2024-09-14T22:37:24.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045725259999999997, 2024-09-14T22:37:24.141 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:24.141 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:24.141 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:24.141 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:24.141 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:24.141 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.142 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35306 2024-09-14T22:37:24.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.142 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:24.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:24.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507069+0000", 2024-09-14T22:37:24.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:24.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507069+0000", 2024-09-14T22:37:24.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:24.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507070+0000", 2024-09-14T22:37:24.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0400000000000002e-07 2024-09-14T22:37:24.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:24.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507072+0000", 2024-09-14T22:37:24.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.891e-06 2024-09-14T22:37:24.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:24.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507078+0000", 2024-09-14T22:37:24.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0279999999999997e-06 2024-09-14T22:37:24.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507082+0000", 2024-09-14T22:37:24.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4680000000000003e-06 2024-09-14T22:37:24.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:24.148 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552632+0000", 2024-09-14T22:37:24.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045550144000000001 2024-09-14T22:37:24.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:24.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552707+0000", 2024-09-14T22:37:24.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4148999999999998e-05 2024-09-14T22:37:24.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:24.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552795+0000", 2024-09-14T22:37:24.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8176000000000004e-05 2024-09-14T22:37:24.150 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.150 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:24.150 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.151 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35307 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1454080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:24.151 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:37:24.151 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.631218861000001, 2024-09-14T22:37:24.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045891358, 2024-09-14T22:37:24.151 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:24.152 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:24.152 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:24.152 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:24.152 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:24.152 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.153 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35307 2024-09-14T22:37:24.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.153 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:24.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:24.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:37:24.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:24.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:37:24.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:24.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:37:24.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4700000000000002e-07 2024-09-14T22:37:24.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:24.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507093+0000", 2024-09-14T22:37:24.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9520000000000002e-06 2024-09-14T22:37:24.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.157 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:24.157 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507099+0000", 2024-09-14T22:37:24.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0240000000000003e-06 2024-09-14T22:37:24.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507105+0000", 2024-09-14T22:37:24.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5520000000000004e-06 2024-09-14T22:37:24.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:24.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552826+0000", 2024-09-14T22:37:24.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045721494000000001 2024-09-14T22:37:24.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:24.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552896+0000", 2024-09-14T22:37:24.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9519999999999998e-05 2024-09-14T22:37:24.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:24.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552979+0000", 2024-09-14T22:37:24.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3468999999999997e-05 2024-09-14T22:37:24.161 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.161 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:24.161 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.162 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35308 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1482752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:24.162 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:37:24.162 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.631195865000002, 2024-09-14T22:37:24.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046076533000000003, 2024-09-14T22:37:24.162 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:24.163 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:24.163 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:24.163 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:24.163 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:24.163 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.164 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35308 2024-09-14T22:37:24.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.164 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:24.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:24.165 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:37:24.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.165 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:24.166 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:37:24.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.166 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:24.166 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:37:24.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:37:24.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:24.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507113+0000", 2024-09-14T22:37:24.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9099999999999999e-06 2024-09-14T22:37:24.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:24.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507119+0000", 2024-09-14T22:37:24.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7440000000000004e-06 2024-09-14T22:37:24.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507124+0000", 2024-09-14T22:37:24.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2129999999999999e-06 2024-09-14T22:37:24.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:24.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553007+0000", 2024-09-14T22:37:24.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045882488999999999 2024-09-14T22:37:24.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.171 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:24.171 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553076+0000", 2024-09-14T22:37:24.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9233000000000006e-05 2024-09-14T22:37:24.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.171 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:24.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553188+0000", 2024-09-14T22:37:24.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111557 2024-09-14T22:37:24.172 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.172 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:24.172 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.173 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35309 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1511424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:24.173 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:37:24.173 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.631176959999998, 2024-09-14T22:37:24.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046231869000000002, 2024-09-14T22:37:24.174 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:24.174 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:24.174 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:24.174 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:24.174 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:24.175 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.175 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35309 2024-09-14T22:37:24.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.175 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:24.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:24.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:37:24.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:24.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:37:24.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:24.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:37:24.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9099999999999999e-07 2024-09-14T22:37:24.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:24.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507132+0000", 2024-09-14T22:37:24.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8190000000000001e-06 2024-09-14T22:37:24.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:24.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507139+0000", 2024-09-14T22:37:24.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9419999999999998e-06 2024-09-14T22:37:24.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507145+0000", 2024-09-14T22:37:24.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2600000000000002e-06 2024-09-14T22:37:24.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:24.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553225+0000", 2024-09-14T22:37:24.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046079991000000001 2024-09-14T22:37:24.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:24.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553284+0000", 2024-09-14T22:37:24.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8638999999999999e-05 2024-09-14T22:37:24.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:24.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553362+0000", 2024-09-14T22:37:24.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7826999999999999e-05 2024-09-14T22:37:24.183 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.183 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:24.184 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.184 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35310 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1540096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:24.184 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507151+0000", 2024-09-14T22:37:24.184 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.631155503999999, 2024-09-14T22:37:24.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046444541999999998, 2024-09-14T22:37:24.185 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:24.185 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:24.185 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:24.185 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:24.186 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:24.186 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.186 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35310 2024-09-14T22:37:24.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.186 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:24.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:24.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507151+0000", 2024-09-14T22:37:24.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:24.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507151+0000", 2024-09-14T22:37:24.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:24.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507152+0000", 2024-09-14T22:37:24.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1699999999999999e-07 2024-09-14T22:37:24.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:24.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507154+0000", 2024-09-14T22:37:24.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.902e-06 2024-09-14T22:37:24.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:24.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507159+0000", 2024-09-14T22:37:24.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.823e-06 2024-09-14T22:37:24.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.191 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.191 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507165+0000", 2024-09-14T22:37:24.191 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.587e-06 2024-09-14T22:37:24.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.191 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:24.191 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553395+0000", 2024-09-14T22:37:24.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046230318999999999 2024-09-14T22:37:24.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:24.192 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553434+0000", 2024-09-14T22:37:24.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8680999999999998e-05 2024-09-14T22:37:24.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:24.192 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553596+0000", 2024-09-14T22:37:24.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016191299999999999 2024-09-14T22:37:24.192 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.192 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:24.193 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.193 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35311 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1568768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:24.193 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:37:24.193 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.631136251000001, 2024-09-14T22:37:24.193 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046623592999999998, 2024-09-14T22:37:24.193 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:24.193 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:24.193 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:24.193 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:24.194 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:24.194 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.194 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35311 2024-09-14T22:37:24.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.194 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:24.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:24.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:37:24.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:24.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:37:24.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:24.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:37:24.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:37:24.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:24.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507176+0000", 2024-09-14T22:37:24.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3349999999999997e-06 2024-09-14T22:37:24.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:24.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507200+0000", 2024-09-14T22:37:24.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4003e-05 2024-09-14T22:37:24.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507206+0000", 2024-09-14T22:37:24.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.976e-06 2024-09-14T22:37:24.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:24.197 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553627+0000", 2024-09-14T22:37:24.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046420284999999999 2024-09-14T22:37:24.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.197 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:24.197 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553696+0000", 2024-09-14T22:37:24.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9567999999999995e-05 2024-09-14T22:37:24.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.197 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:24.197 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553794+0000", 2024-09-14T22:37:24.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8060000000000006e-05 2024-09-14T22:37:24.198 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.198 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:24.198 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.198 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35566 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1484800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:24.198 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514668+0000", 2024-09-14T22:37:24.198 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.623638474000003, 2024-09-14T22:37:24.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045644974999999997, 2024-09-14T22:37:24.198 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:24.198 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:24.198 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:24.198 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:24.198 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:24.199 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.199 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35566 2024-09-14T22:37:24.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.199 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:24.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:24.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514668+0000", 2024-09-14T22:37:24.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:24.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514668+0000", 2024-09-14T22:37:24.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:24.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514669+0000", 2024-09-14T22:37:24.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9099999999999999e-07 2024-09-14T22:37:24.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:24.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514671+0000", 2024-09-14T22:37:24.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3800000000000001e-06 2024-09-14T22:37:24.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:24.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514677+0000", 2024-09-14T22:37:24.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9660000000000001e-06 2024-09-14T22:37:24.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514682+0000", 2024-09-14T22:37:24.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0939999999999998e-06 2024-09-14T22:37:24.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:24.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560133+0000", 2024-09-14T22:37:24.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045450398000000003 2024-09-14T22:37:24.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:24.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560207+0000", 2024-09-14T22:37:24.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4196999999999995e-05 2024-09-14T22:37:24.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:24.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560313+0000", 2024-09-14T22:37:24.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106549 2024-09-14T22:37:24.202 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.202 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:24.202 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.202 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35567 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1513472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:24.202 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:37:24.203 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.623617934999999, 2024-09-14T22:37:24.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045793215999999998, 2024-09-14T22:37:24.203 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:24.203 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:24.203 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:24.203 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:24.203 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:24.203 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.203 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35567 2024-09-14T22:37:24.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.203 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:24.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:24.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:37:24.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:24.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:37:24.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:24.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:37:24.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1899999999999998e-07 2024-09-14T22:37:24.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:24.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514692+0000", 2024-09-14T22:37:24.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2699999999999999e-06 2024-09-14T22:37:24.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:24.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514698+0000", 2024-09-14T22:37:24.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0939999999999996e-06 2024-09-14T22:37:24.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.206 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.206 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514703+0000", 2024-09-14T22:37:24.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3870000000000003e-06 2024-09-14T22:37:24.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.206 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:24.206 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560353+0000", 2024-09-14T22:37:24.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045649914 2024-09-14T22:37:24.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.206 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:24.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560416+0000", 2024-09-14T22:37:24.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3096999999999996e-05 2024-09-14T22:37:24.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:24.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560482+0000", 2024-09-14T22:37:24.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6035e-05 2024-09-14T22:37:24.208 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.208 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:24.208 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.209 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35568 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1542144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:24.209 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:37:24.209 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.623593137, 2024-09-14T22:37:24.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045995921000000002, 2024-09-14T22:37:24.209 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:24.209 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:24.209 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:24.209 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:24.209 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:24.209 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.209 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35568 2024-09-14T22:37:24.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.210 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:24.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:24.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:37:24.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:24.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:37:24.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:24.211 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:37:24.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4099999999999999e-07 2024-09-14T22:37:24.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.211 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.211 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:24.211 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514720+0000", 2024-09-14T22:37:24.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6869999999999999e-06 2024-09-14T22:37:24.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.211 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.211 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:24.211 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514727+0000", 2024-09-14T22:37:24.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3540000000000002e-06 2024-09-14T22:37:24.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.212 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.212 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514732+0000", 2024-09-14T22:37:24.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7690000000000004e-06 2024-09-14T22:37:24.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.212 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:24.212 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560523+0000", 2024-09-14T22:37:24.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045791373000000003 2024-09-14T22:37:24.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:24.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560606+0000", 2024-09-14T22:37:24.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2690999999999998e-05 2024-09-14T22:37:24.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:24.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560710+0000", 2024-09-14T22:37:24.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000103606 2024-09-14T22:37:24.213 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.213 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:24.213 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.214 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35569 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1570816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:24.214 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:37:24.214 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.623567328, 2024-09-14T22:37:24.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046110630999999999, 2024-09-14T22:37:24.214 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:24.214 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:24.214 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:24.214 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:24.214 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:24.214 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.214 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35569 2024-09-14T22:37:24.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.214 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:24.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:24.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:37:24.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:24.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:37:24.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:24.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:37:24.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.82e-07 2024-09-14T22:37:24.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:24.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514743+0000", 2024-09-14T22:37:24.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.269e-06 2024-09-14T22:37:24.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.217 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:24.217 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514750+0000", 2024-09-14T22:37:24.217 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7340000000000002e-06 2024-09-14T22:37:24.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.217 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.217 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514755+0000", 2024-09-14T22:37:24.217 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7260000000000002e-06 2024-09-14T22:37:24.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:24.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560731+0000", 2024-09-14T22:37:24.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045976377999999998 2024-09-14T22:37:24.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:24.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560777+0000", 2024-09-14T22:37:24.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5868999999999997e-05 2024-09-14T22:37:24.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:24.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560850+0000", 2024-09-14T22:37:24.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3172999999999999e-05 2024-09-14T22:37:24.219 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.219 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:24.219 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.219 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:86180 3.18 3:1d0c52d8:::rbd_data.113cd62ea8a5.000000000000004b:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:24.219 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:01.356417+0000", 2024-09-14T22:37:24.219 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.781890232999999, 2024-09-14T22:37:24.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049143245000000002, 2024-09-14T22:37:24.219 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:24.220 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:24.220 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:24.220 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:24.220 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:24.220 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.220 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 86180 2024-09-14T22:37:24.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.220 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:24.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:24.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356417+0000", 2024-09-14T22:37:24.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.221 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:24.221 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356417+0000", 2024-09-14T22:37:24.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.221 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:24.221 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356420+0000", 2024-09-14T22:37:24.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2030000000000001e-06 2024-09-14T22:37:24.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.221 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:24.221 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356435+0000", 2024-09-14T22:37:24.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5287000000000001e-05 2024-09-14T22:37:24.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:24.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356472+0000", 2024-09-14T22:37:24.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7311000000000002e-05 2024-09-14T22:37:24.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356492+0000", 2024-09-14T22:37:24.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9519999999999999e-05 2024-09-14T22:37:24.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:24.223 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356670+0000", 2024-09-14T22:37:24.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000177567 2024-09-14T22:37:24.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:24.223 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356924+0000", 2024-09-14T22:37:24.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025399299999999999 2024-09-14T22:37:24.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 0", 2024-09-14T22:37:24.223 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.365052+0000", 2024-09-14T22:37:24.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0081289380000000005 2024-09-14T22:37:24.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.224 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:24.224 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.405396+0000", 2024-09-14T22:37:24.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040343228000000002 2024-09-14T22:37:24.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.224 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:37:24.224 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.405421+0000", 2024-09-14T22:37:24.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5763e-05 2024-09-14T22:37:24.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.224 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:37:24.224 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.405491+0000", 2024-09-14T22:37:24.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9721999999999994e-05 2024-09-14T22:37:24.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:24.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.405560+0000", 2024-09-14T22:37:24.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8713000000000004e-05 2024-09-14T22:37:24.225 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.225 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:24.225 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.225 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98378 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 919552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:24.225 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148183+0000", 2024-09-14T22:37:24.226 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.990123955, 2024-09-14T22:37:24.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045614549999999997, 2024-09-14T22:37:24.226 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:24.226 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:24.226 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:24.226 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:24.226 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:24.226 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.226 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98378 2024-09-14T22:37:24.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.227 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:24.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:24.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148183+0000", 2024-09-14T22:37:24.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:24.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148183+0000", 2024-09-14T22:37:24.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.228 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:24.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148183+0000", 2024-09-14T22:37:24.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0800000000000001e-07 2024-09-14T22:37:24.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.228 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:24.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148184+0000", 2024-09-14T22:37:24.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2780000000000001e-06 2024-09-14T22:37:24.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:24.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148189+0000", 2024-09-14T22:37:24.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2699999999999998e-06 2024-09-14T22:37:24.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148191+0000", 2024-09-14T22:37:24.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7659999999999999e-06 2024-09-14T22:37:24.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:24.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193708+0000", 2024-09-14T22:37:24.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045516562000000003 2024-09-14T22:37:24.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:24.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193751+0000", 2024-09-14T22:37:24.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2811000000000001e-05 2024-09-14T22:37:24.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:24.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193797+0000", 2024-09-14T22:37:24.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6655e-05 2024-09-14T22:37:24.230 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.230 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:24.230 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.230 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98379 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 928768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:24.231 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148195+0000", 2024-09-14T22:37:24.231 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.990112126, 2024-09-14T22:37:24.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045817405999999998, 2024-09-14T22:37:24.231 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:24.231 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:24.231 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:24.231 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:24.231 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:24.231 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.231 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98379 2024-09-14T22:37:24.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.232 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:24.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:24.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148195+0000", 2024-09-14T22:37:24.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:24.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148195+0000", 2024-09-14T22:37:24.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:24.233 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148195+0000", 2024-09-14T22:37:24.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3999999999999998e-07 2024-09-14T22:37:24.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:24.233 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148196+0000", 2024-09-14T22:37:24.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.341e-06 2024-09-14T22:37:24.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:24.233 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148200+0000", 2024-09-14T22:37:24.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9040000000000002e-06 2024-09-14T22:37:24.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148203+0000", 2024-09-14T22:37:24.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.959e-06 2024-09-14T22:37:24.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:24.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193841+0000", 2024-09-14T22:37:24.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045637886000000003 2024-09-14T22:37:24.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:24.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193933+0000", 2024-09-14T22:37:24.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1823999999999994e-05 2024-09-14T22:37:24.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:24.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194012+0000", 2024-09-14T22:37:24.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9252000000000006e-05 2024-09-14T22:37:24.235 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.235 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:24.235 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.235 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98380 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 937984~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:24.235 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148206+0000", 2024-09-14T22:37:24.235 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.990100488, 2024-09-14T22:37:24.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045937354, 2024-09-14T22:37:24.236 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:24.236 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:24.236 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:24.236 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:24.236 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:24.236 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.236 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98380 2024-09-14T22:37:24.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.236 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:24.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:24.236 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148206+0000", 2024-09-14T22:37:24.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:24.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148206+0000", 2024-09-14T22:37:24.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:24.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148207+0000", 2024-09-14T22:37:24.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1e-07 2024-09-14T22:37:24.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:24.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148221+0000", 2024-09-14T22:37:24.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4679e-05 2024-09-14T22:37:24.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:24.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148228+0000", 2024-09-14T22:37:24.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3820000000000001e-06 2024-09-14T22:37:24.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148231+0000", 2024-09-14T22:37:24.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2849999999999999e-06 2024-09-14T22:37:24.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:24.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194054+0000", 2024-09-14T22:37:24.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045822596 2024-09-14T22:37:24.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:24.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194098+0000", 2024-09-14T22:37:24.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4604000000000002e-05 2024-09-14T22:37:24.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:24.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194144+0000", 2024-09-14T22:37:24.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5597999999999999e-05 2024-09-14T22:37:24.240 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.240 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:24.240 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.240 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98381 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 947200~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:24.241 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148236+0000", 2024-09-14T22:37:24.241 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.990070995, 2024-09-14T22:37:24.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046111129000000001, 2024-09-14T22:37:24.241 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:24.241 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:24.241 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:24.241 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:24.241 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:24.242 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.242 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98381 2024-09-14T22:37:24.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.242 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:24.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:24.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148236+0000", 2024-09-14T22:37:24.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:24.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148236+0000", 2024-09-14T22:37:24.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.243 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:24.243 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148236+0000", 2024-09-14T22:37:24.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8799999999999998e-07 2024-09-14T22:37:24.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.243 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:24.243 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148238+0000", 2024-09-14T22:37:24.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6559999999999999e-06 2024-09-14T22:37:24.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.243 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:24.243 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148246+0000", 2024-09-14T22:37:24.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3440000000000001e-06 2024-09-14T22:37:24.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148251+0000", 2024-09-14T22:37:24.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5480000000000003e-06 2024-09-14T22:37:24.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:24.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194187+0000", 2024-09-14T22:37:24.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045935847000000002 2024-09-14T22:37:24.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:24.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194267+0000", 2024-09-14T22:37:24.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0267999999999998e-05 2024-09-14T22:37:24.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:24.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194347+0000", 2024-09-14T22:37:24.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0178000000000004e-05 2024-09-14T22:37:24.245 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.245 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:24.245 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.245 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98382 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 956416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:24.245 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148254+0000", 2024-09-14T22:37:24.246 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.990052623, 2024-09-14T22:37:24.246 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046224227, 2024-09-14T22:37:24.246 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:24.246 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:24.246 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:24.246 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:24.246 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:24.246 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.246 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98382 2024-09-14T22:37:24.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.246 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:24.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.246 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:24.246 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148254+0000", 2024-09-14T22:37:24.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:24.247 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148254+0000", 2024-09-14T22:37:24.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:24.247 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148254+0000", 2024-09-14T22:37:24.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3099999999999999e-07 2024-09-14T22:37:24.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:24.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148256+0000", 2024-09-14T22:37:24.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5540000000000001e-06 2024-09-14T22:37:24.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:24.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148259+0000", 2024-09-14T22:37:24.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2499999999999998e-06 2024-09-14T22:37:24.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148262+0000", 2024-09-14T22:37:24.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8310000000000001e-06 2024-09-14T22:37:24.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:24.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194387+0000", 2024-09-14T22:37:24.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046125082999999997 2024-09-14T22:37:24.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:24.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194430+0000", 2024-09-14T22:37:24.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3074999999999997e-05 2024-09-14T22:37:24.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.250 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:24.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194478+0000", 2024-09-14T22:37:24.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8202999999999999e-05 2024-09-14T22:37:24.250 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.250 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:24.250 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.250 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98383 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 965632~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:24.250 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148265+0000", 2024-09-14T22:37:24.250 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.990041508999999, 2024-09-14T22:37:24.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046409141000000001, 2024-09-14T22:37:24.251 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:24.251 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:24.251 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:24.251 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:24.251 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:24.251 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.251 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98383 2024-09-14T22:37:24.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.251 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:24.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.251 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:24.251 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148265+0000", 2024-09-14T22:37:24.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.252 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:24.252 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148265+0000", 2024-09-14T22:37:24.252 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.252 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:24.252 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148266+0000", 2024-09-14T22:37:24.252 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0800000000000001e-07 2024-09-14T22:37:24.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.252 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:24.252 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148267+0000", 2024-09-14T22:37:24.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1659999999999999e-06 2024-09-14T22:37:24.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:24.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148271+0000", 2024-09-14T22:37:24.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1289999999999998e-06 2024-09-14T22:37:24.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148274+0000", 2024-09-14T22:37:24.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5939999999999998e-06 2024-09-14T22:37:24.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:24.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194523+0000", 2024-09-14T22:37:24.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046248162000000002 2024-09-14T22:37:24.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:24.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194603+0000", 2024-09-14T22:37:24.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0161999999999996e-05 2024-09-14T22:37:24.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:24.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194675+0000", 2024-09-14T22:37:24.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1719999999999998e-05 2024-09-14T22:37:24.254 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.255 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:24.255 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.255 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98384 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 974848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:24.255 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148278+0000", 2024-09-14T22:37:24.255 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.990029181000001, 2024-09-14T22:37:24.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046527369999999998, 2024-09-14T22:37:24.255 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:24.255 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:24.255 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:24.255 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:24.255 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:24.255 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.255 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98384 2024-09-14T22:37:24.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.256 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:24.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:24.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148278+0000", 2024-09-14T22:37:24.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:24.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148278+0000", 2024-09-14T22:37:24.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:24.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.149987+0000", 2024-09-14T22:37:24.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017093430000000001 2024-09-14T22:37:24.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:24.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.149993+0000", 2024-09-14T22:37:24.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6219999999999997e-06 2024-09-14T22:37:24.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:24.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150006+0000", 2024-09-14T22:37:24.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3095000000000001e-05 2024-09-14T22:37:24.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150014+0000", 2024-09-14T22:37:24.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6340000000000007e-06 2024-09-14T22:37:24.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:24.259 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194715+0000", 2024-09-14T22:37:24.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044700349 2024-09-14T22:37:24.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.259 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:24.259 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194759+0000", 2024-09-14T22:37:24.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3807000000000003e-05 2024-09-14T22:37:24.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.259 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:24.260 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194805+0000", 2024-09-14T22:37:24.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6520000000000002e-05 2024-09-14T22:37:24.260 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.260 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:24.260 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.260 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98390 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 1030144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:24.260 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.150152+0000", 2024-09-14T22:37:24.260 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.988154494, 2024-09-14T22:37:24.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045642708999999997, 2024-09-14T22:37:24.261 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:24.261 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:24.261 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:24.261 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:24.261 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:24.261 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.261 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98390 2024-09-14T22:37:24.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.261 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:24.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.261 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:24.261 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150152+0000", 2024-09-14T22:37:24.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:24.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150152+0000", 2024-09-14T22:37:24.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:24.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150153+0000", 2024-09-14T22:37:24.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.08e-07 2024-09-14T22:37:24.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:24.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150155+0000", 2024-09-14T22:37:24.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2910000000000002e-06 2024-09-14T22:37:24.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:24.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150162+0000", 2024-09-14T22:37:24.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4099999999999996e-06 2024-09-14T22:37:24.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150167+0000", 2024-09-14T22:37:24.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3199999999999999e-06 2024-09-14T22:37:24.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:24.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195704+0000", 2024-09-14T22:37:24.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045537243999999998 2024-09-14T22:37:24.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:24.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195749+0000", 2024-09-14T22:37:24.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4866e-05 2024-09-14T22:37:24.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:24.265 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195795+0000", 2024-09-14T22:37:24.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.617e-05 2024-09-14T22:37:24.265 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.265 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:24.265 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.265 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98391 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 1039360~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:24.265 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.150175+0000", 2024-09-14T22:37:24.265 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.988131658, 2024-09-14T22:37:24.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045825232, 2024-09-14T22:37:24.266 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:24.266 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:24.266 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:24.266 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:24.266 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:24.266 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.266 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98391 2024-09-14T22:37:24.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.266 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:24.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.266 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:24.266 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150175+0000", 2024-09-14T22:37:24.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:24.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150175+0000", 2024-09-14T22:37:24.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:24.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:24.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150176+0000", 2024-09-14T22:37:24.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9700000000000002e-07 2024-09-14T22:37:24.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:24.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150178+0000", 2024-09-14T22:37:24.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.407e-06 2024-09-14T22:37:24.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:24.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150185+0000", 2024-09-14T22:37:24.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5189999999999998e-06 2024-09-14T22:37:24.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.269 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.269 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150190+0000", 2024-09-14T22:37:24.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2399999999999998e-06 2024-09-14T22:37:24.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.269 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:24.269 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195838+0000", 2024-09-14T22:37:24.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045648265 2024-09-14T22:37:24.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.269 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:24.269 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195918+0000", 2024-09-14T22:37:24.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0094999999999998e-05 2024-09-14T22:37:24.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:24.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:24.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:24.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.196000+0000", 2024-09-14T22:37:24.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2309000000000001e-05 2024-09-14T22:37:24.270 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.270 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:24.270 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.270 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:24.270 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:24.270 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:24.271 INFO:teuthology.orchestra.run.smithi001.stdout:3663 write 0xadc3aea thru 0xadcf2a7 (0xb7be bytes) 2024-09-14T22:37:24.272 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:37:24.274 INFO:teuthology.orchestra.run.smithi001.stdout:3665 trunc from 0xadcf2a8 to 0x781f561 2024-09-14T22:37:24.274 INFO:teuthology.orchestra.run.smithi001.stdout:3666 read 0x74db1d5 thru 0x74e9d28 (0xeb54 bytes) 2024-09-14T22:37:24.274 INFO:teuthology.orchestra.run.smithi001.stdout:3667 write 0x10bb810 thru 0x10c410a (0x88fb bytes) 2024-09-14T22:37:24.274 INFO:teuthology.orchestra.run.smithi001.stdout:3669 writesame 0x39a7464 thru 0x39a999d data_size 0x5(0x253a bytes) 2024-09-14T22:37:24.274 INFO:teuthology.orchestra.run.smithi001.stdout:3670 punch from 0x5d32b45 to 0x5d3550c, (0x29c7 bytes) 2024-09-14T22:37:24.274 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:24.274 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:24.274 INFO:teuthology.orchestra.run.smithi001.stdout:3673 writesame 0x7260ea8 thru 0x7267e6d data_size 0x39(0x6fc6 bytes) 2024-09-14T22:37:24.274 INFO:teuthology.orchestra.run.smithi001.stdout:3675 trunc from 0x781f561 to 0x98996e8 2024-09-14T22:37:24.275 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:24.275 INFO:teuthology.orchestra.run.smithi001.stdout:3677 trunc from 0x98996e8 to 0x6530534 2024-09-14T22:37:24.275 INFO:teuthology.orchestra.run.smithi001.stdout:3678 writesame 0xe6ce1d5 thru 0xe6ce268 data_size 0x4(0x94 bytes) 2024-09-14T22:37:24.275 INFO:teuthology.orchestra.run.smithi001.stdout:3679 read 0xb9107f7 thru 0xb9176ea (0x6ef4 bytes) 2024-09-14T22:37:24.275 INFO:teuthology.orchestra.run.smithi001.stdout:3680 write 0x5f1db41 thru 0x5f22a5f (0x4f1f bytes) 2024-09-14T22:37:24.275 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:24.275 INFO:teuthology.orchestra.run.smithi001.stdout:3682 write 0x83f65bc thru 0x840353e (0xcf83 bytes) 2024-09-14T22:37:24.275 INFO:teuthology.orchestra.run.smithi001.stdout:3690 write 0xc089ba9 thru 0xc08bcb8 (0x2110 bytes) 2024-09-14T22:37:24.275 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:24.267+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:24.276 INFO:teuthology.orchestra.run.smithi001.stdout:3691 trunc from 0xe6ce269 to 0x6debfe2 2024-09-14T22:37:24.296 INFO:teuthology.orchestra.run.smithi001.stdout:3692 read 0x3682fdf thru 0x368d675 (0xa697 bytes) 2024-09-14T22:37:24.309 INFO:teuthology.orchestra.run.smithi001.stdout:3694 write 0x82be4ce thru 0x82ca929 (0xc45c bytes) 2024-09-14T22:37:24.320 INFO:teuthology.orchestra.run.smithi001.stdout:3695 trunc from 0x82ca92a to 0xe496bad 2024-09-14T22:37:24.335 INFO:teuthology.orchestra.run.smithi001.stdout:3696 trunc from 0xe496bad to 0xadb511a 2024-09-14T22:37:24.360 INFO:teuthology.orchestra.run.smithi001.stdout:3698 trunc from 0xadb511a to 0x496239e 2024-09-14T22:37:24.372 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:24.367+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:24.385 INFO:teuthology.orchestra.run.smithi001.stdout:3699 read 0x279389d thru 0x2795056 (0x17ba bytes) 2024-09-14T22:37:24.394 INFO:teuthology.orchestra.run.smithi001.stdout:3700 trunc from 0x496239e to 0x10539f3 2024-09-14T22:37:24.412 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:24.412 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:24.412 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:37:24.412 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:24.425 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:37:24.437 INFO:teuthology.orchestra.run.smithi001.stdout:3701 writesame 0x6d4819a thru 0x6d4e573 data_size 0x1(0x63da bytes) 2024-09-14T22:37:24.456 INFO:teuthology.orchestra.run.smithi001.stdout:3704 writesame 0x9b32db5 thru 0x9b36184 data_size 0x2(0x33d0 bytes) 2024-09-14T22:37:24.456 INFO:teuthology.orchestra.run.smithi001.stdout:3705 punch from 0x6da2903 to 0x6dab337, (0x8a34 bytes) 2024-09-14T22:37:24.457 INFO:teuthology.orchestra.run.smithi001.stdout:3706 write 0x9a028fb thru 0x9a055cc (0x2cd2 bytes) 2024-09-14T22:37:24.463 INFO:teuthology.orchestra.run.smithi001.stdout:3707 punch from 0x9353fe6 to 0x935752a, (0x3544 bytes) 2024-09-14T22:37:24.463 INFO:teuthology.orchestra.run.smithi001.stdout:3708 write 0x5e92254 thru 0x5e95074 (0x2e21 bytes) 2024-09-14T22:37:24.465 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:24.465 INFO:teuthology.orchestra.run.smithi001.stdout:3710 write 0x9ad0231 thru 0x9ad741b (0x71eb bytes) 2024-09-14T22:37:24.488 INFO:teuthology.orchestra.run.smithi001.stdout:3711 writesame 0xbaeaf36 thru 0xbaf9fd2 data_size 0x1(0xf09d bytes) 2024-09-14T22:37:24.489 INFO:teuthology.orchestra.run.smithi001.stdout:3712 write 0x627727 thru 0x6337e4 (0xc0be bytes) 2024-09-14T22:37:24.492 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:24.493 INFO:teuthology.orchestra.run.smithi001.stdout:3715 write 0xaa89460 thru 0xaa9851a (0xf0bb bytes) 2024-09-14T22:37:24.500 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:24.500 INFO:teuthology.orchestra.run.smithi001.stdout:3717 write 0x8eed6f7 thru 0x8efc8f1 (0xf1fb bytes) 2024-09-14T22:37:24.507 INFO:teuthology.orchestra.run.smithi001.stdout:3718 read 0x25117d1 thru 0x25172c2 (0x5af2 bytes) 2024-09-14T22:37:24.508 INFO:teuthology.orchestra.run.smithi001.stdout:3719 punch from 0x9855aa to 0x987cbe, (0x2714 bytes) 2024-09-14T22:37:24.511 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:24.507+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:24.531 INFO:teuthology.orchestra.run.smithi001.stdout:3720 write 0xeb495b9 thru 0xeb4f2d6 (0x5d1e bytes) 2024-09-14T22:37:24.538 INFO:teuthology.orchestra.run.smithi001.stdout:3721 write 0xe2e6759 thru 0xe2eb6fd (0x4fa5 bytes) 2024-09-14T22:37:24.545 INFO:teuthology.orchestra.run.smithi001.stdout:3722 write 0x1d79597 thru 0x1d82be0 (0x964a bytes) 2024-09-14T22:37:24.547 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:24.547 INFO:teuthology.orchestra.run.smithi001.stdout:3725 read 0x64e04e3 thru 0x64e3df1 (0x390f bytes) 2024-09-14T22:37:24.547 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:24.548 INFO:teuthology.orchestra.run.smithi001.stdout:3727 writesame 0xae27bfd thru 0xae31785 data_size 0x1(0x9b89 bytes) 2024-09-14T22:37:24.574 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:24.571+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:24.589 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:24.589 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:24.589 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:37:24.589 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:37:24.589 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:24.590 INFO:teuthology.orchestra.run.smithi001.stdout:3728 read 0xc17ade6 thru 0xc17ce15 (0x2030 bytes) 2024-09-14T22:37:24.590 INFO:teuthology.orchestra.run.smithi001.stdout:3729 writesame 0x2b2bd91 thru 0x2b37254 data_size 0x2e(0xb4c4 bytes) 2024-09-14T22:37:24.594 INFO:teuthology.orchestra.run.smithi001.stdout:3730 read 0x91abf32 thru 0x91afb05 (0x3bd4 bytes) 2024-09-14T22:37:24.595 INFO:teuthology.orchestra.run.smithi001.stdout:3731 punch from 0x46e11e8 to 0x46efdbd, (0xebd5 bytes) 2024-09-14T22:37:24.595 INFO:teuthology.orchestra.run.smithi001.stdout:3732 writesame 0x4702003 thru 0x470d1e2 data_size 0x10(0xb1e0 bytes) 2024-09-14T22:37:24.599 INFO:teuthology.orchestra.run.smithi001.stdout:3734 read 0x7886428 thru 0x788ba72 (0x564b bytes) 2024-09-14T22:37:24.599 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:24.600 INFO:teuthology.orchestra.run.smithi001.stdout:3737 read 0x9dc8399 thru 0x9dcbebf (0x3b27 bytes) 2024-09-14T22:37:24.600 INFO:teuthology.orchestra.run.smithi001.stdout:3738 write 0x3c4935d thru 0x3c5742e (0xe0d2 bytes) 2024-09-14T22:37:24.604 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:37:24.605 INFO:teuthology.orchestra.run.smithi001.stdout:3739 write 0x8e50dec thru 0x8e55b36 (0x4d4b bytes) 2024-09-14T22:37:24.608 INFO:teuthology.orchestra.run.smithi001.stdout:3740 read 0x829f2b0 thru 0x82a305a (0x3dab bytes) 2024-09-14T22:37:24.608 INFO:teuthology.orchestra.run.smithi001.stdout:3741 writesame 0x8ed82db thru 0x8ee1b57 data_size 0x1(0x987d bytes) 2024-09-14T22:37:24.626 INFO:teuthology.orchestra.run.smithi001.stdout:3742 read 0xcb1180c thru 0xcb1b1b7 (0x99ac bytes) 2024-09-14T22:37:24.626 INFO:teuthology.orchestra.run.smithi001.stdout:3744 writesame 0x6bf7af thru 0x6cd8d3 data_size 0x1(0xe125 bytes) 2024-09-14T22:37:24.667 INFO:teuthology.orchestra.run.smithi001.stdout:3745 writesame 0x8e212c3 thru 0x8e299f1 data_size 0x1(0x872f bytes) 2024-09-14T22:37:24.683 INFO:teuthology.orchestra.run.smithi001.stdout:3746 punch from 0x7a8d3b to 0x7afb91, (0x6e56 bytes) 2024-09-14T22:37:24.683 INFO:teuthology.orchestra.run.smithi001.stdout:3749 write 0x28e4965 thru 0x28ef6a0 (0xad3c bytes) 2024-09-14T22:37:24.687 INFO:teuthology.orchestra.run.smithi001.stdout:3751 write 0x7b81c45 thru 0x7b8d246 (0xb602 bytes) 2024-09-14T22:37:24.691 INFO:teuthology.orchestra.run.smithi001.stdout:3753 writesame 0x85dcb47 thru 0x85e19c9 data_size 0x1(0x4e83 bytes) 2024-09-14T22:37:24.695 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:24.691+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:24.701 INFO:teuthology.orchestra.run.smithi001.stdout:3756 read 0x129bd7b thru 0x129d150 (0x13d6 bytes) 2024-09-14T22:37:24.702 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:24.702 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:24.702 INFO:teuthology.orchestra.run.smithi001.stdout:3760 write 0xbd1846a thru 0xbd24c60 (0xc7f7 bytes) 2024-09-14T22:37:24.710 INFO:teuthology.orchestra.run.smithi001.stdout:3761 punch from 0xaee61f3 to 0xaef4e6e, (0xec7b bytes) 2024-09-14T22:37:24.711 INFO:teuthology.orchestra.run.smithi001.stdout:3762 read 0x6988fc1 thru 0x698de80 (0x4ec0 bytes) 2024-09-14T22:37:24.712 INFO:teuthology.orchestra.run.smithi001.stdout:3763 punch from 0xc994128 to 0xc9a16c8, (0xd5a0 bytes) 2024-09-14T22:37:24.712 INFO:teuthology.orchestra.run.smithi001.stdout:3764 write 0xe093746 thru 0xe0a18b6 (0xe171 bytes) 2024-09-14T22:37:24.724 INFO:teuthology.orchestra.run.smithi001.stdout:moved 4 / 82 (4.87805%) 2024-09-14T22:37:24.724 INFO:teuthology.orchestra.run.smithi001.stdout:avg 10.25 2024-09-14T22:37:24.724 INFO:teuthology.orchestra.run.smithi001.stdout:stddev 3.23071 -> 3.26917 (expected baseline 2.99479) 2024-09-14T22:37:24.724 INFO:teuthology.orchestra.run.smithi001.stdout:min osd.2 with 6 -> 5 pgs (0.585366 -> 0.487805 * mean) 2024-09-14T22:37:24.724 INFO:teuthology.orchestra.run.smithi001.stdout:max osd.7 with 16 -> 15 pgs (1.56098 -> 1.46341 * mean) 2024-09-14T22:37:24.725 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:37:24.725 INFO:teuthology.orchestra.run.smithi001.stdout:oload 1000 2024-09-14T22:37:24.725 INFO:teuthology.orchestra.run.smithi001.stdout:max_change 3 2024-09-14T22:37:24.725 INFO:teuthology.orchestra.run.smithi001.stdout:max_change_osds 4 2024-09-14T22:37:24.725 INFO:teuthology.orchestra.run.smithi001.stdout:average_utilization 0.0004 2024-09-14T22:37:24.725 INFO:teuthology.orchestra.run.smithi001.stdout:overload_utilization 0.0043 2024-09-14T22:37:24.725 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0 weight 0.8240 -> 1.0000 2024-09-14T22:37:24.730 INFO:teuthology.orchestra.run.smithi001.stdout:3765 trunc from 0xeb4f2d7 to 0xafda988 2024-09-14T22:37:24.776 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:24.771+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:24.779 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:24.779 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:24.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:24.779 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:24.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.780 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6377 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8239104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:24.780 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932688+0000", 2024-09-14T22:37:24.780 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 60.842278047999997, 2024-09-14T22:37:24.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051650696000000003, 2024-09-14T22:37:24.780 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:24.780 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:24.780 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:24.780 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:24.780 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:24.780 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.781 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6377 2024-09-14T22:37:24.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.781 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:24.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:24.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932688+0000", 2024-09-14T22:37:24.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:24.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932688+0000", 2024-09-14T22:37:24.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:24.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932689+0000", 2024-09-14T22:37:24.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4309999999999999e-06 2024-09-14T22:37:24.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:24.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932695+0000", 2024-09-14T22:37:24.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8889999999999999e-06 2024-09-14T22:37:24.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:24.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932713+0000", 2024-09-14T22:37:24.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7416999999999999e-05 2024-09-14T22:37:24.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932726+0000", 2024-09-14T22:37:24.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3145e-05 2024-09-14T22:37:24.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:24.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984181+0000", 2024-09-14T22:37:24.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051455630000000002 2024-09-14T22:37:24.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:24.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984247+0000", 2024-09-14T22:37:24.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5350000000000003e-05 2024-09-14T22:37:24.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:24.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984339+0000", 2024-09-14T22:37:24.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1834000000000002e-05 2024-09-14T22:37:24.784 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.784 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:24.784 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.784 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6378 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8247296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:24.784 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932743+0000", 2024-09-14T22:37:24.784 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 60.842222814000003, 2024-09-14T22:37:24.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051819373000000002, 2024-09-14T22:37:24.785 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:24.785 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:24.785 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:24.785 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:24.785 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:24.785 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.785 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6378 2024-09-14T22:37:24.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.785 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:24.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:24.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932743+0000", 2024-09-14T22:37:24.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:24.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932743+0000", 2024-09-14T22:37:24.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:24.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932745+0000", 2024-09-14T22:37:24.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.637e-06 2024-09-14T22:37:24.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:24.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932750+0000", 2024-09-14T22:37:24.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7859999999999996e-06 2024-09-14T22:37:24.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:24.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932767+0000", 2024-09-14T22:37:24.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7621999999999998e-05 2024-09-14T22:37:24.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932781+0000", 2024-09-14T22:37:24.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3403e-05 2024-09-14T22:37:24.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:24.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984383+0000", 2024-09-14T22:37:24.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051602249000000003 2024-09-14T22:37:24.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:24.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984461+0000", 2024-09-14T22:37:24.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8467000000000006e-05 2024-09-14T22:37:24.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:24.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984562+0000", 2024-09-14T22:37:24.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101209 2024-09-14T22:37:24.789 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.789 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:24.789 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.789 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6379 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8255488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:24.789 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932796+0000", 2024-09-14T22:37:24.789 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 60.842169423000001, 2024-09-14T22:37:24.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051961245000000003, 2024-09-14T22:37:24.789 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:24.789 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:24.789 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:24.790 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:24.790 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:24.790 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.790 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6379 2024-09-14T22:37:24.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.790 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:24.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:24.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932796+0000", 2024-09-14T22:37:24.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:24.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932796+0000", 2024-09-14T22:37:24.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:24.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932797+0000", 2024-09-14T22:37:24.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9700000000000005e-07 2024-09-14T22:37:24.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:24.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932823+0000", 2024-09-14T22:37:24.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5273999999999999e-05 2024-09-14T22:37:24.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:24.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932840+0000", 2024-09-14T22:37:24.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7774e-05 2024-09-14T22:37:24.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932862+0000", 2024-09-14T22:37:24.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1957e-05 2024-09-14T22:37:24.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:24.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984603+0000", 2024-09-14T22:37:24.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051740543 2024-09-14T22:37:24.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:24.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984669+0000", 2024-09-14T22:37:24.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.567e-05 2024-09-14T22:37:24.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:24.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984758+0000", 2024-09-14T22:37:24.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9129999999999995e-05 2024-09-14T22:37:24.794 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.794 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:24.794 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.794 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6380 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8263680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:24.794 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932879+0000", 2024-09-14T22:37:24.794 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 60.842087343999999, 2024-09-14T22:37:24.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052102860000000001, 2024-09-14T22:37:24.794 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:24.795 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:24.795 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:24.795 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:24.795 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:24.795 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.795 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6380 2024-09-14T22:37:24.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.795 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:24.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:24.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932879+0000", 2024-09-14T22:37:24.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:24.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932879+0000", 2024-09-14T22:37:24.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:24.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932880+0000", 2024-09-14T22:37:24.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.054e-06 2024-09-14T22:37:24.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:24.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932885+0000", 2024-09-14T22:37:24.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1290000000000004e-06 2024-09-14T22:37:24.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:24.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932901+0000", 2024-09-14T22:37:24.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5908000000000001e-05 2024-09-14T22:37:24.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932913+0000", 2024-09-14T22:37:24.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2408999999999999e-05 2024-09-14T22:37:24.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:24.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984800+0000", 2024-09-14T22:37:24.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.051886671000000002 2024-09-14T22:37:24.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:24.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984879+0000", 2024-09-14T22:37:24.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9200000000000001e-05 2024-09-14T22:37:24.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:24.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.984981+0000", 2024-09-14T22:37:24.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102489 2024-09-14T22:37:24.798 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.799 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:24.799 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.799 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6381 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8271872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:24.799 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.932929+0000", 2024-09-14T22:37:24.799 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 60.842037013000002, 2024-09-14T22:37:24.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052251484000000001, 2024-09-14T22:37:24.799 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:24.799 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:24.799 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:24.799 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:24.799 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:24.799 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.799 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6381 2024-09-14T22:37:24.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.800 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:24.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:24.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932929+0000", 2024-09-14T22:37:24.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:24.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932929+0000", 2024-09-14T22:37:24.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:24.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932930+0000", 2024-09-14T22:37:24.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3399999999999998e-07 2024-09-14T22:37:24.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:24.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932934+0000", 2024-09-14T22:37:24.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5719999999999996e-06 2024-09-14T22:37:24.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:24.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932961+0000", 2024-09-14T22:37:24.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6437000000000001e-05 2024-09-14T22:37:24.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.932973+0000", 2024-09-14T22:37:24.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2097e-05 2024-09-14T22:37:24.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:24.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985021+0000", 2024-09-14T22:37:24.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052047955999999999 2024-09-14T22:37:24.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:24.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985089+0000", 2024-09-14T22:37:24.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7725999999999995e-05 2024-09-14T22:37:24.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:24.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985180+0000", 2024-09-14T22:37:24.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1861999999999996e-05 2024-09-14T22:37:24.803 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.803 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:24.803 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.803 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6382 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8280064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:24.804 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933013+0000", 2024-09-14T22:37:24.804 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 60.841952851000002, 2024-09-14T22:37:24.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052391118, 2024-09-14T22:37:24.804 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:24.804 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:24.804 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:24.804 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:24.804 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:24.804 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.804 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6382 2024-09-14T22:37:24.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.804 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:24.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:24.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933013+0000", 2024-09-14T22:37:24.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:24.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933013+0000", 2024-09-14T22:37:24.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:24.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933014+0000", 2024-09-14T22:37:24.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.375e-06 2024-09-14T22:37:24.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:24.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933020+0000", 2024-09-14T22:37:24.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0220000000000001e-06 2024-09-14T22:37:24.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:24.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933038+0000", 2024-09-14T22:37:24.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7694999999999999e-05 2024-09-14T22:37:24.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933051+0000", 2024-09-14T22:37:24.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2721999999999999e-05 2024-09-14T22:37:24.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:24.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985224+0000", 2024-09-14T22:37:24.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052172757 2024-09-14T22:37:24.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:24.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985303+0000", 2024-09-14T22:37:24.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9426999999999997e-05 2024-09-14T22:37:24.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:24.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985404+0000", 2024-09-14T22:37:24.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010111999999999999 2024-09-14T22:37:24.808 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.808 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:24.808 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.808 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6383 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8288256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:24.808 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933082+0000", 2024-09-14T22:37:24.808 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 60.841883871, 2024-09-14T22:37:24.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052525761999999997, 2024-09-14T22:37:24.808 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:24.808 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:24.809 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:24.809 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:24.809 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:24.809 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.809 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6383 2024-09-14T22:37:24.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.809 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:24.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:24.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933082+0000", 2024-09-14T22:37:24.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:24.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933082+0000", 2024-09-14T22:37:24.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:24.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933084+0000", 2024-09-14T22:37:24.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5659999999999999e-06 2024-09-14T22:37:24.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:24.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933089+0000", 2024-09-14T22:37:24.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.271e-06 2024-09-14T22:37:24.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:24.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933105+0000", 2024-09-14T22:37:24.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5685999999999999e-05 2024-09-14T22:37:24.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933118+0000", 2024-09-14T22:37:24.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3188e-05 2024-09-14T22:37:24.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:24.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985448+0000", 2024-09-14T22:37:24.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052330176999999999 2024-09-14T22:37:24.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:24.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985514+0000", 2024-09-14T22:37:24.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6029000000000001e-05 2024-09-14T22:37:24.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:24.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985608+0000", 2024-09-14T22:37:24.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3844999999999993e-05 2024-09-14T22:37:24.813 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.813 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:24.813 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.813 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6384 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8296448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:24.813 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933134+0000", 2024-09-14T22:37:24.813 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 60.841831511999999, 2024-09-14T22:37:24.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052688835000000003, 2024-09-14T22:37:24.813 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:24.814 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:24.814 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:24.814 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:24.814 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:24.814 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.814 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6384 2024-09-14T22:37:24.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.814 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:24.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:24.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933134+0000", 2024-09-14T22:37:24.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:24.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933134+0000", 2024-09-14T22:37:24.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:24.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933135+0000", 2024-09-14T22:37:24.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.78e-07 2024-09-14T22:37:24.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:24.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933141+0000", 2024-09-14T22:37:24.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1540000000000001e-06 2024-09-14T22:37:24.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:24.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933156+0000", 2024-09-14T22:37:24.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5488000000000001e-05 2024-09-14T22:37:24.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933169+0000", 2024-09-14T22:37:24.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2649e-05 2024-09-14T22:37:24.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:24.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985650+0000", 2024-09-14T22:37:24.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052481112000000003 2024-09-14T22:37:24.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:24.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985724+0000", 2024-09-14T22:37:24.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3953000000000002e-05 2024-09-14T22:37:24.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:24.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985823+0000", 2024-09-14T22:37:24.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9500999999999995e-05 2024-09-14T22:37:24.818 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.818 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:24.818 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.818 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6385 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8304640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:24.818 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933185+0000", 2024-09-14T22:37:24.818 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 60.841780808999999, 2024-09-14T22:37:24.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052892844000000001, 2024-09-14T22:37:24.818 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:24.818 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:24.819 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:24.819 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:24.819 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:24.819 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.819 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6385 2024-09-14T22:37:24.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.819 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:24.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.819 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:24.819 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933185+0000", 2024-09-14T22:37:24.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.819 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:24.819 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933185+0000", 2024-09-14T22:37:24.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:24.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933186+0000", 2024-09-14T22:37:24.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0139999999999999e-06 2024-09-14T22:37:24.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:24.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933191+0000", 2024-09-14T22:37:24.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7230000000000004e-06 2024-09-14T22:37:24.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:24.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933207+0000", 2024-09-14T22:37:24.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.658e-05 2024-09-14T22:37:24.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933220+0000", 2024-09-14T22:37:24.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2583e-05 2024-09-14T22:37:24.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:24.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985880+0000", 2024-09-14T22:37:24.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052660140000000001 2024-09-14T22:37:24.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:24.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.985965+0000", 2024-09-14T22:37:24.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4472e-05 2024-09-14T22:37:24.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:24.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986078+0000", 2024-09-14T22:37:24.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011333200000000001 2024-09-14T22:37:24.823 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.823 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:24.823 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.823 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6386 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8312832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:24.823 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933236+0000", 2024-09-14T22:37:24.823 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 60.841729915000002, 2024-09-14T22:37:24.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053075342999999997, 2024-09-14T22:37:24.823 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:24.823 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:24.823 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:24.824 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:24.824 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:24.824 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.824 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6386 2024-09-14T22:37:24.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.824 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:24.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.824 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:24.824 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933236+0000", 2024-09-14T22:37:24.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.824 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:24.825 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933236+0000", 2024-09-14T22:37:24.825 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.825 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:24.825 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933237+0000", 2024-09-14T22:37:24.825 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6500000000000008e-07 2024-09-14T22:37:24.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.825 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:24.825 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933242+0000", 2024-09-14T22:37:24.825 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.938e-06 2024-09-14T22:37:24.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.826 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:24.826 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933258+0000", 2024-09-14T22:37:24.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6334e-05 2024-09-14T22:37:24.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.826 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.826 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933270+0000", 2024-09-14T22:37:24.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1973999999999999e-05 2024-09-14T22:37:24.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.826 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:24.826 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986123+0000", 2024-09-14T22:37:24.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052852769000000001 2024-09-14T22:37:24.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:24.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986204+0000", 2024-09-14T22:37:24.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1458999999999994e-05 2024-09-14T22:37:24.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:24.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986311+0000", 2024-09-14T22:37:24.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010690400000000001 2024-09-14T22:37:24.827 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.827 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:24.828 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.828 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6387 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8321024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:24.828 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933335+0000", 2024-09-14T22:37:24.828 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 60.841630645000002, 2024-09-14T22:37:24.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053182363000000003, 2024-09-14T22:37:24.828 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:24.828 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:24.828 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:24.828 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:24.828 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:24.828 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.828 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6387 2024-09-14T22:37:24.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.829 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:24.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:24.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933335+0000", 2024-09-14T22:37:24.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:24.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933335+0000", 2024-09-14T22:37:24.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:24.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933337+0000", 2024-09-14T22:37:24.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3829999999999999e-06 2024-09-14T22:37:24.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:24.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933343+0000", 2024-09-14T22:37:24.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1489999999999999e-06 2024-09-14T22:37:24.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:24.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933361+0000", 2024-09-14T22:37:24.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8105e-05 2024-09-14T22:37:24.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933374+0000", 2024-09-14T22:37:24.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.33e-05 2024-09-14T22:37:24.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:24.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986355+0000", 2024-09-14T22:37:24.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.052981053 2024-09-14T22:37:24.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:24.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986424+0000", 2024-09-14T22:37:24.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8864999999999999e-05 2024-09-14T22:37:24.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.832 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:24.832 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986518+0000", 2024-09-14T22:37:24.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3508e-05 2024-09-14T22:37:24.832 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.832 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:24.832 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.833 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6388 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8329216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:24.833 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933390+0000", 2024-09-14T22:37:24.833 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 60.841576058999998, 2024-09-14T22:37:24.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053371605000000003, 2024-09-14T22:37:24.833 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:24.833 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:24.833 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:24.833 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:24.834 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:24.834 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.834 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6388 2024-09-14T22:37:24.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.834 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:24.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.834 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:24.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933390+0000", 2024-09-14T22:37:24.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.834 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:24.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933390+0000", 2024-09-14T22:37:24.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.835 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:24.835 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933391+0000", 2024-09-14T22:37:24.835 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.443e-06 2024-09-14T22:37:24.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.835 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:24.835 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933396+0000", 2024-09-14T22:37:24.835 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1680000000000003e-06 2024-09-14T22:37:24.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:24.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933412+0000", 2024-09-14T22:37:24.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5302999999999999e-05 2024-09-14T22:37:24.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933439+0000", 2024-09-14T22:37:24.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6747999999999999e-05 2024-09-14T22:37:24.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:24.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986562+0000", 2024-09-14T22:37:24.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053123161000000002 2024-09-14T22:37:24.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.837 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:24.837 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986641+0000", 2024-09-14T22:37:24.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9086999999999997e-05 2024-09-14T22:37:24.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.837 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:24.837 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986761+0000", 2024-09-14T22:37:24.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120695 2024-09-14T22:37:24.837 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.837 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:24.837 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.838 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6389 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8337408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:24.838 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933456+0000", 2024-09-14T22:37:24.838 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 60.841510208000003, 2024-09-14T22:37:24.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053514182, 2024-09-14T22:37:24.838 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:24.838 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:24.838 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:24.838 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:24.838 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:24.838 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.838 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6389 2024-09-14T22:37:24.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.838 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:24.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:24.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933456+0000", 2024-09-14T22:37:24.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:24.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933456+0000", 2024-09-14T22:37:24.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:24.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933457+0000", 2024-09-14T22:37:24.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0020000000000001e-06 2024-09-14T22:37:24.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:24.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933462+0000", 2024-09-14T22:37:24.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8400000000000002e-06 2024-09-14T22:37:24.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:24.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933477+0000", 2024-09-14T22:37:24.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.592e-05 2024-09-14T22:37:24.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933489+0000", 2024-09-14T22:37:24.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1955e-05 2024-09-14T22:37:24.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:24.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986798+0000", 2024-09-14T22:37:24.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053308263000000002 2024-09-14T22:37:24.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:24.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986871+0000", 2024-09-14T22:37:24.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3518999999999999e-05 2024-09-14T22:37:24.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:24.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.986970+0000", 2024-09-14T22:37:24.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8683000000000004e-05 2024-09-14T22:37:24.841 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.841 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:24.841 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.842 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6390 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8345600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:24.842 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933504+0000", 2024-09-14T22:37:24.842 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 60.841461422999998, 2024-09-14T22:37:24.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053719579000000003, 2024-09-14T22:37:24.842 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:24.842 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:24.842 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:24.842 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:24.842 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:24.842 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.842 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6390 2024-09-14T22:37:24.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.843 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:24.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:24.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933504+0000", 2024-09-14T22:37:24.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:24.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933504+0000", 2024-09-14T22:37:24.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.844 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:24.844 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933505+0000", 2024-09-14T22:37:24.844 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8299999999999995e-07 2024-09-14T22:37:24.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.844 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:24.844 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933520+0000", 2024-09-14T22:37:24.844 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4948e-05 2024-09-14T22:37:24.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.844 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:24.844 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933538+0000", 2024-09-14T22:37:24.844 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7286e-05 2024-09-14T22:37:24.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933550+0000", 2024-09-14T22:37:24.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2223e-05 2024-09-14T22:37:24.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:24.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987028+0000", 2024-09-14T22:37:24.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053478500999999998 2024-09-14T22:37:24.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:24.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987111+0000", 2024-09-14T22:37:24.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3158999999999994e-05 2024-09-14T22:37:24.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:24.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987224+0000", 2024-09-14T22:37:24.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011257899999999999 2024-09-14T22:37:24.846 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.846 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:24.846 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.846 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6391 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8353792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:24.846 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933576+0000", 2024-09-14T22:37:24.846 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 60.841389978000002, 2024-09-14T22:37:24.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053858196999999997, 2024-09-14T22:37:24.846 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:24.847 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:24.847 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:24.847 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:24.847 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:24.847 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.847 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6391 2024-09-14T22:37:24.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.847 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:24.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:24.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933576+0000", 2024-09-14T22:37:24.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:24.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933576+0000", 2024-09-14T22:37:24.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:24.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933578+0000", 2024-09-14T22:37:24.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9539999999999998e-06 2024-09-14T22:37:24.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:24.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933584+0000", 2024-09-14T22:37:24.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9030000000000001e-06 2024-09-14T22:37:24.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:24.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933601+0000", 2024-09-14T22:37:24.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7703e-05 2024-09-14T22:37:24.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933614+0000", 2024-09-14T22:37:24.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2330999999999999e-05 2024-09-14T22:37:24.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:24.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987275+0000", 2024-09-14T22:37:24.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053661470000000003 2024-09-14T22:37:24.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:24.850 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987342+0000", 2024-09-14T22:37:24.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6266000000000006e-05 2024-09-14T22:37:24.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:24.850 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987434+0000", 2024-09-14T22:37:24.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2570000000000003e-05 2024-09-14T22:37:24.850 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.850 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:24.850 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.850 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6392 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8361984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:24.850 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933648+0000", 2024-09-14T22:37:24.851 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 60.841317590000003, 2024-09-14T22:37:24.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054032430999999999, 2024-09-14T22:37:24.851 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:24.851 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:24.851 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:24.851 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:24.851 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:24.851 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.851 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6392 2024-09-14T22:37:24.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.851 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:24.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:24.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933648+0000", 2024-09-14T22:37:24.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.852 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:24.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933648+0000", 2024-09-14T22:37:24.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.852 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:24.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933650+0000", 2024-09-14T22:37:24.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3969999999999999e-06 2024-09-14T22:37:24.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.852 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:24.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933655+0000", 2024-09-14T22:37:24.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7690000000000002e-06 2024-09-14T22:37:24.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:24.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933672+0000", 2024-09-14T22:37:24.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6359999999999999e-05 2024-09-14T22:37:24.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933696+0000", 2024-09-14T22:37:24.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4630999999999998e-05 2024-09-14T22:37:24.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:24.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987490+0000", 2024-09-14T22:37:24.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053793256999999997 2024-09-14T22:37:24.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:24.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987572+0000", 2024-09-14T22:37:24.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1897999999999994e-05 2024-09-14T22:37:24.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:24.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987681+0000", 2024-09-14T22:37:24.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109119 2024-09-14T22:37:24.855 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.855 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:24.855 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.855 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6393 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8370176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:24.855 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933713+0000", 2024-09-14T22:37:24.855 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 60.841252636, 2024-09-14T22:37:24.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054194111000000003, 2024-09-14T22:37:24.855 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:24.855 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:24.855 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:24.855 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:24.856 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:24.856 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.856 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6393 2024-09-14T22:37:24.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.856 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:24.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.856 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:24.856 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933713+0000", 2024-09-14T22:37:24.856 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.856 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:24.856 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933713+0000", 2024-09-14T22:37:24.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:24.857 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933714+0000", 2024-09-14T22:37:24.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2109999999999999e-06 2024-09-14T22:37:24.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:24.857 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933721+0000", 2024-09-14T22:37:24.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2489999999999998e-06 2024-09-14T22:37:24.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:24.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933751+0000", 2024-09-14T22:37:24.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0239999999999998e-05 2024-09-14T22:37:24.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933764+0000", 2024-09-14T22:37:24.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.322e-05 2024-09-14T22:37:24.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:24.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987737+0000", 2024-09-14T22:37:24.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053972539 2024-09-14T22:37:24.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:24.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987813+0000", 2024-09-14T22:37:24.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6635000000000001e-05 2024-09-14T22:37:24.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.859 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:24.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987907+0000", 2024-09-14T22:37:24.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4017000000000005e-05 2024-09-14T22:37:24.859 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.859 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:24.859 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.859 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6394 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8378368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:24.859 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933781+0000", 2024-09-14T22:37:24.859 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 60.841184877000003, 2024-09-14T22:37:24.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054378276000000003, 2024-09-14T22:37:24.860 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:24.860 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:24.860 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:24.860 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:24.860 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:24.860 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.860 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6394 2024-09-14T22:37:24.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.860 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:24.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:24.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933781+0000", 2024-09-14T22:37:24.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:24.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933781+0000", 2024-09-14T22:37:24.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:24.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933782+0000", 2024-09-14T22:37:24.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9199999999999999e-07 2024-09-14T22:37:24.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:24.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933788+0000", 2024-09-14T22:37:24.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9340000000000003e-06 2024-09-14T22:37:24.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:24.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933803+0000", 2024-09-14T22:37:24.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5362999999999998e-05 2024-09-14T22:37:24.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933816+0000", 2024-09-14T22:37:24.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2456999999999999e-05 2024-09-14T22:37:24.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:24.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.987961+0000", 2024-09-14T22:37:24.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054145581999999998 2024-09-14T22:37:24.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:24.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988042+0000", 2024-09-14T22:37:24.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1075000000000006e-05 2024-09-14T22:37:24.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:24.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988159+0000", 2024-09-14T22:37:24.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116873 2024-09-14T22:37:24.864 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.864 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:24.864 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.864 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6395 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8386560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:24.864 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:37:24.864 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 60.841133982999999, 2024-09-14T22:37:24.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054534379000000001, 2024-09-14T22:37:24.864 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:24.864 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:24.865 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:24.865 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:24.865 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:24.865 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.865 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6395 2024-09-14T22:37:24.865 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.865 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:24.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.865 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:24.865 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:37:24.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.865 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.865 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:24.865 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:37:24.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:24.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933833+0000", 2024-09-14T22:37:24.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3859999999999999e-06 2024-09-14T22:37:24.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:24.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933838+0000", 2024-09-14T22:37:24.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8709999999999996e-06 2024-09-14T22:37:24.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:24.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933854+0000", 2024-09-14T22:37:24.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6189e-05 2024-09-14T22:37:24.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933871+0000", 2024-09-14T22:37:24.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.626e-05 2024-09-14T22:37:24.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:24.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988198+0000", 2024-09-14T22:37:24.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054327581999999999 2024-09-14T22:37:24.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:24.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988274+0000", 2024-09-14T22:37:24.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5759000000000004e-05 2024-09-14T22:37:24.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.869 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:24.869 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988366+0000", 2024-09-14T22:37:24.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2331999999999996e-05 2024-09-14T22:37:24.869 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.870 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:24.870 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.870 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.870 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104944 3.3 3:c3780f3b:::rbd_data.113ca561f7a9.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:24.870 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:37:24.870 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.7415321099999996, 2024-09-14T22:37:24.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073041087000000005, 2024-09-14T22:37:24.871 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:24.871 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:24.871 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:24.871 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:24.871 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:24.872 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:24.872 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104944 2024-09-14T22:37:24.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.872 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:24.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:24.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:37:24.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:24.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:37:24.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:24.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.874 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:24.874 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033436+0000", 2024-09-14T22:37:24.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2060000000000001e-06 2024-09-14T22:37:24.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.875 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:24.875 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033448+0000", 2024-09-14T22:37:24.875 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.169e-05 2024-09-14T22:37:24.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:24.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033474+0000", 2024-09-14T22:37:24.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6400000000000001e-05 2024-09-14T22:37:24.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:24.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033489+0000", 2024-09-14T22:37:24.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5067e-05 2024-09-14T22:37:24.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:24.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033658+0000", 2024-09-14T22:37:24.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000168565 2024-09-14T22:37:24.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:24.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033793+0000", 2024-09-14T22:37:24.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013536900000000001 2024-09-14T22:37:24.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.879 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:37:24.879 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.047559+0000", 2024-09-14T22:37:24.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013766294 2024-09-14T22:37:24.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:24.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106157+0000", 2024-09-14T22:37:24.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058597110000000001 2024-09-14T22:37:24.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:24.881 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106290+0000", 2024-09-14T22:37:24.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000133244 2024-09-14T22:37:24.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:24.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106396+0000", 2024-09-14T22:37:24.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106246 2024-09-14T22:37:24.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:24.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:24.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:24.883 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106475+0000", 2024-09-14T22:37:24.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8895999999999998e-05 2024-09-14T22:37:24.883 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.883 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:24.883 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.883 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:24.883 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:24.884 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:24.886 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:37:24.886 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:24.883+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:24.986 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:24.983+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:24.996 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:37:25.054 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:25.054 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:25.054 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:37:25.054 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:25.068 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:37:25.111 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:25.107+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:25.153 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:37:25.153 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:37:25.169 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:37:25.169 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:37:25.186 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:37:25.186 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:37:25.187 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:25.183+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:25.188 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:25.188 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:25.188 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:37:25.188 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:37:25.188 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:25.203 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:37:25.204 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:37:25.204 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:37:25.222 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:37:25.222 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:37:25.244 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:37:25.245 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:37:25.264 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:37:25.265 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:37:25.284 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:37:25.284 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:37:25.288 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:25.287+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:25.367 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:25.367 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:25.381 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:37:25.390 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:25.387+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:25.491 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:25.483+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:25.499 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:25.499 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:25.513 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:37:25.591 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:25.587+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:25.683 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:25.683 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:25.692 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:25.691+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:25.696 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:37:25.737 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T22:37:25.751 INFO:teuthology.orchestra.run.smithi001.stdout:3766 punch from 0x26fae68 to 0x2701b0f, (0x6ca7 bytes) 2024-09-14T22:37:25.751 INFO:teuthology.orchestra.run.smithi001.stdout:3767 punch from 0x5fa4069 to 0x5fb275d, (0xe6f4 bytes) 2024-09-14T22:37:25.751 INFO:teuthology.orchestra.run.smithi001.stdout:3768 read 0x80f7475 thru 0x80fd518 (0x60a4 bytes) 2024-09-14T22:37:25.751 INFO:teuthology.orchestra.run.smithi001.stdout:3769 write 0x8433652 thru 0x843a883 (0x7232 bytes) 2024-09-14T22:37:25.754 INFO:teuthology.orchestra.run.smithi001.stdout:3770 punch from 0x2d58f34 to 0x2d683bd, (0xf489 bytes) 2024-09-14T22:37:25.754 INFO:teuthology.orchestra.run.smithi001.stdout:3771 read 0x95c6209 thru 0x95c7cc8 (0x1ac0 bytes) 2024-09-14T22:37:25.754 INFO:teuthology.orchestra.run.smithi001.stdout:3773 read 0x2e7c0ac thru 0x2e7ff11 (0x3e66 bytes) 2024-09-14T22:37:25.754 INFO:teuthology.orchestra.run.smithi001.stdout:3775 writesame 0x30ca37f thru 0x30d32a2 data_size 0x2(0x8f24 bytes) 2024-09-14T22:37:25.764 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:25.764 INFO:teuthology.orchestra.run.smithi001.stdout:3778 write 0x82d4a44 thru 0x82df77f (0xad3c bytes) 2024-09-14T22:37:25.770 INFO:teuthology.orchestra.run.smithi001.stdout:3779 read 0x2d79546 thru 0x2d7d6b9 (0x4174 bytes) 2024-09-14T22:37:25.771 INFO:teuthology.orchestra.run.smithi001.stdout:3780 write 0x1ad230a thru 0x1adbc3d (0x9934 bytes) 2024-09-14T22:37:25.774 INFO:teuthology.orchestra.run.smithi001.stdout:3781 write 0x735b91d thru 0x736364b (0x7d2f bytes) 2024-09-14T22:37:25.777 INFO:teuthology.orchestra.run.smithi001.stdout:3783 read 0x165eb1e thru 0x166b918 (0xcdfb bytes) 2024-09-14T22:37:25.778 INFO:teuthology.orchestra.run.smithi001.stdout:3784 write 0x7140773 thru 0x7140d7c (0x60a bytes) 2024-09-14T22:37:25.781 INFO:teuthology.orchestra.run.smithi001.stdout:3785 punch from 0x5da46e0 to 0x5da63f1, (0x1d11 bytes) 2024-09-14T22:37:25.781 INFO:teuthology.orchestra.run.smithi001.stdout:3787 read 0x39ce2a6 thru 0x39d99d0 (0xb72b bytes) 2024-09-14T22:37:25.782 INFO:teuthology.orchestra.run.smithi001.stdout:3788 writesame 0xcce2d7 thru 0xcda966 data_size 0x48(0xc690 bytes) 2024-09-14T22:37:25.793 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:25.787+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:25.795 INFO:teuthology.orchestra.run.smithi001.stdout:3789 write 0xbafa02e thru 0xbb098c0 (0xf893 bytes) 2024-09-14T22:37:25.805 INFO:teuthology.orchestra.run.smithi001.stdout:3790 trunc from 0xbb098c1 to 0x872c9a2 2024-09-14T22:37:25.825 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:25.825 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:25.827 INFO:teuthology.orchestra.run.smithi001.stdout:3791 writesame 0x4c4432f thru 0x4c49c81 data_size 0x1(0x5953 bytes) 2024-09-14T22:37:25.840 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:37:25.845 INFO:teuthology.orchestra.run.smithi001.stdout:3793 read 0x7437038 thru 0x743a303 (0x32cc bytes) 2024-09-14T22:37:25.845 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:25.845 INFO:teuthology.orchestra.run.smithi001.stdout:3795 read 0x83bb54c thru 0x83cb302 (0xfdb7 bytes) 2024-09-14T22:37:25.868 INFO:teuthology.orchestra.run.smithi001.stdout:3796 write 0xe6e6e1e thru 0xe6e7f96 (0x1179 bytes) 2024-09-14T22:37:25.874 INFO:teuthology.orchestra.run.smithi001.stdout:3797 writesame 0xdef39a thru 0xdfd0fa data_size 0x9(0xdd61 bytes) 2024-09-14T22:37:25.879 INFO:teuthology.orchestra.run.smithi001.stdout:3798 write 0x6648324 thru 0x6652aaf (0xa78c bytes) 2024-09-14T22:37:25.882 INFO:teuthology.orchestra.run.smithi001.stdout:3799 punch from 0x77db20b to 0x77dbab5, (0x8aa bytes) 2024-09-14T22:37:25.882 INFO:teuthology.orchestra.run.smithi001.stdout:3800 punch from 0x6b1f0b4 to 0x6b26e1b, (0x7d67 bytes) 2024-09-14T22:37:25.882 INFO:teuthology.orchestra.run.smithi001.stdout:3802 punch from 0xd834e78 to 0xd835a3c, (0xbc4 bytes) 2024-09-14T22:37:25.882 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:25.882 INFO:teuthology.orchestra.run.smithi001.stdout:3804 write 0xc95f581 thru 0xc96807c (0x8afc bytes) 2024-09-14T22:37:25.904 INFO:teuthology.orchestra.run.smithi001.stdout:3805 trunc from 0xe6e7f97 to 0xbf75cb 2024-09-14T22:37:25.935 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:25.931+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:25.935 INFO:teuthology.orchestra.run.smithi001.stdout:3808 write 0x81f5680 thru 0x8200f05 (0xb886 bytes) 2024-09-14T22:37:25.940 INFO:teuthology.orchestra.run.smithi001.stdout:3809 write 0x783d879 thru 0x7848a05 (0xb18d bytes) 2024-09-14T22:37:25.942 INFO:teuthology.orchestra.run.smithi001.stdout:3810 read 0x761048f thru 0x7610967 (0x4d9 bytes) 2024-09-14T22:37:25.942 INFO:teuthology.orchestra.run.smithi001.stdout:3811 read 0x8a72d3 thru 0x8afdf7 (0x8b25 bytes) 2024-09-14T22:37:25.943 INFO:teuthology.orchestra.run.smithi001.stdout:3813 punch from 0x61c03be to 0x61c2d8f, (0x29d1 bytes) 2024-09-14T22:37:25.943 INFO:teuthology.orchestra.run.smithi001.stdout:3814 read 0x1f46eed thru 0x1f51d65 (0xae79 bytes) 2024-09-14T22:37:25.944 INFO:teuthology.orchestra.run.smithi001.stdout:3816 read 0x6068479 thru 0x606abcb (0x2753 bytes) 2024-09-14T22:37:25.944 INFO:teuthology.orchestra.run.smithi001.stdout:3818 read 0x168bc65 thru 0x168f9fa (0x3d96 bytes) 2024-09-14T22:37:25.944 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:25.944 INFO:teuthology.orchestra.run.smithi001.stdout:3820 read 0x736c400 thru 0x7372a40 (0x6641 bytes) 2024-09-14T22:37:25.945 INFO:teuthology.orchestra.run.smithi001.stdout:3822 write 0x2c9e27a thru 0x2caaa45 (0xc7cc bytes) 2024-09-14T22:37:25.946 INFO:teuthology.orchestra.run.smithi001.stdout:3823 write 0x1108314 thru 0x1113bd1 (0xb8be bytes) 2024-09-14T22:37:25.948 INFO:teuthology.orchestra.run.smithi001.stdout:3824 writesame 0x686cdb9 thru 0x6878635 data_size 0x27(0xb87d bytes) 2024-09-14T22:37:25.969 INFO:teuthology.orchestra.run.smithi001.stdout:3825 write 0xe5def31 thru 0xe5e49ac (0x5a7c bytes) 2024-09-14T22:37:25.974 INFO:teuthology.orchestra.run.smithi001.stdout:3826 write 0x6c73e36 thru 0x6c7a95d (0x6b28 bytes) 2024-09-14T22:37:25.976 INFO:teuthology.orchestra.run.smithi001.stdout:3827 write 0x7e18753 thru 0x7e1c5f4 (0x3ea2 bytes) 2024-09-14T22:37:25.980 INFO:teuthology.orchestra.run.smithi001.stdout:3828 write 0x363f470 thru 0x3649f77 (0xab08 bytes) 2024-09-14T22:37:25.983 INFO:teuthology.orchestra.run.smithi001.stdout:3829 write 0x92758c2 thru 0x927da4d (0x818c bytes) 2024-09-14T22:37:25.985 INFO:teuthology.orchestra.run.smithi001.stdout:3830 punch from 0xde9a538 to 0xdea12e2, (0x6daa bytes) 2024-09-14T22:37:25.985 INFO:teuthology.orchestra.run.smithi001.stdout:3831 read 0x13ac0 thru 0x1579e (0x1cdf bytes) 2024-09-14T22:37:25.994 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:25.987+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:25.999 INFO:teuthology.orchestra.run.smithi001.stdout:3832 trunc from 0xe5e49ad to 0x59010a3 2024-09-14T22:37:26.015 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:26.015 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:26.030 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:37:26.036 INFO:teuthology.orchestra.run.smithi001.stdout:3833 write 0x9e7b2fe thru 0x9e828cd (0x75d0 bytes) 2024-09-14T22:37:26.045 INFO:teuthology.orchestra.run.smithi001.stdout:3834 trunc from 0x9e828ce to 0xd91a37e 2024-09-14T22:37:26.054 INFO:teuthology.orchestra.run.smithi001.stdout:3838 punch from 0xb9c9742 to 0xb9cd0ec, (0x39aa bytes) 2024-09-14T22:37:26.057 INFO:teuthology.orchestra.run.smithi001.stdout:3839 trunc from 0xd91a37e to 0xbfe2253 2024-09-14T22:37:26.066 INFO:teuthology.orchestra.run.smithi001.stdout:3840 write 0x9f3ad7c thru 0x9f3c0b2 (0x1337 bytes) 2024-09-14T22:37:26.069 INFO:teuthology.orchestra.run.smithi001.stdout:3842 writesame 0xb482041 thru 0xb48eb62 data_size 0xa2(0xcb22 bytes) 2024-09-14T22:37:26.079 INFO:teuthology.orchestra.run.smithi001.stdout:3844 trunc from 0xbfe2253 to 0x9450f6b 2024-09-14T22:37:26.103 INFO:teuthology.orchestra.run.smithi001.stdout:3845 write 0xc61c6c thru 0xc6f7ba (0xdb4f bytes) 2024-09-14T22:37:26.106 INFO:teuthology.orchestra.run.smithi001.stdout:3846 write 0x41a97d1 thru 0x41a9953 (0x183 bytes) 2024-09-14T22:37:26.115 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:26.111+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:26.123 INFO:teuthology.orchestra.run.smithi001.stdout:3847 writesame 0xac1e457 thru 0xac2a70d data_size 0x1(0xc2b7 bytes) 2024-09-14T22:37:26.123 INFO:teuthology.orchestra.run.smithi001.stdout:3848 punch from 0xa523797 to 0xa5295eb, (0x5e54 bytes) 2024-09-14T22:37:26.141 INFO:teuthology.orchestra.run.smithi001.stdout:3850 trunc from 0xac2a70e to 0x1e3fbc2 2024-09-14T22:37:26.194 INFO:teuthology.orchestra.run.smithi001.stdout:3851 write 0x9d3cf97 thru 0x9d4c720 (0xf78a bytes) 2024-09-14T22:37:26.195 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:26.191+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:26.199 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:26.199 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:26.201 INFO:teuthology.orchestra.run.smithi001.stdout:3852 read 0x1308a9f thru 0x131548b (0xc9ed bytes) 2024-09-14T22:37:26.213 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:37:26.229 INFO:teuthology.orchestra.run.smithi001.stdout:3853 write 0xea04ec9 thru 0xea11ba5 (0xccdd bytes) 2024-09-14T22:37:26.235 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:26.236 INFO:teuthology.orchestra.run.smithi001.stdout:3856 clone 24 order 21 su 8192 sc 6 2024-09-14T22:37:26.296 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:26.291+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:26.377 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:26.377 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:26.391 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:37:26.397 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:26.391+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:26.498 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:26.495+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:26.527 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:26.527 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:26.542 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:37:26.629 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:26.627+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:26.699 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:26.695+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:26.704 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:26.705 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:26.711 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2024-09-14T22:37:26.724 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:37:26.801 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:26.795+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:26.902 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:26.895+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:26.909 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:26.909 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:26.926 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:37:27.003 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:26.995+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:27.021 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-14T22:37:27.041 INFO:teuthology.orchestra.run.smithi001.stdout:3857 writesame 0xaca8c43 thru 0xaca94dd data_size 0x1(0x89b bytes) 2024-09-14T22:37:27.055 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:27.055 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:27.062 INFO:teuthology.orchestra.run.smithi001.stdout:3858 write 0x37053a9 thru 0x3706c68 (0x18c0 bytes) 2024-09-14T22:37:27.075 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:37:27.103 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:27.099+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:27.183 INFO:teuthology.orchestra.run.smithi001.stdout:3859 trunc from 0xea11ba6 to 0x554ab5a 2024-09-14T22:37:27.205 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:27.203+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:27.215 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:27.216 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:27.233 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:37:27.301 INFO:teuthology.orchestra.run.smithi001.stdout:3860 read 0x2d86ad3 thru 0x2d8e6d1 (0x7bff bytes) 2024-09-14T22:37:27.304 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:37:27.305 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:27.303+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:27.324 INFO:teuthology.orchestra.run.smithi001.stdout:3861 writesame 0xbd2486c thru 0xbd31f76 data_size 0x1(0xd70b bytes) 2024-09-14T22:37:27.324 INFO:teuthology.orchestra.run.smithi001.stdout:3863 read 0xc7192a thru 0xc77adf (0x61b6 bytes) 2024-09-14T22:37:27.325 INFO:teuthology.orchestra.run.smithi001.stdout:3865 read 0xa884a8c thru 0xa88a1aa (0x571f bytes) 2024-09-14T22:37:27.338 INFO:teuthology.orchestra.run.smithi001.stdout:3868 trunc from 0xbd31f77 to 0x41eafa8 2024-09-14T22:37:27.397 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:27.397 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:27.406 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:27.403+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:27.414 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:37:27.470 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:37:27.470 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:37:27.490 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:37:27.490 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:37:27.506 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:27.499+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:27.508 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:37:27.509 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:37:27.527 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:37:27.527 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:37:27.530 INFO:teuthology.orchestra.run.smithi001.stdout:3871 write 0xaf79d51 thru 0xaf7bc3c (0x1eec bytes) 2024-09-14T22:37:27.535 INFO:teuthology.orchestra.run.smithi001.stdout:3873 read 0x5718d43 thru 0x571ce19 (0x40d7 bytes) 2024-09-14T22:37:27.537 INFO:teuthology.orchestra.run.smithi001.stdout:3875 trunc from 0xaf7bc3d to 0xc2ff36d 2024-09-14T22:37:27.543 INFO:teuthology.orchestra.run.smithi001.stdout:3876 write 0x20b7509 thru 0x20bb798 (0x4290 bytes) 2024-09-14T22:37:27.549 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:37:27.549 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:37:27.574 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:37:27.575 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:37:27.595 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:37:27.596 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:37:27.599 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:27.600 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:37:27.600 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:37:27.600 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:27.607 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:27.603+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:27.612 INFO:teuthology.orchestra.run.smithi001.stdout:3877 writesame 0xbb001c9 thru 0xbb0acdf data_size 0x1(0xab17 bytes) 2024-09-14T22:37:27.615 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:37:27.615 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:37:27.617 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:37:27.625 INFO:teuthology.orchestra.run.smithi001.stdout:3878 read 0x561e68 thru 0x5675c3 (0x575c bytes) 2024-09-14T22:37:27.627 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:27.631 INFO:teuthology.orchestra.run.smithi001.stdout:3880 trunc from 0xc2ff36d to 0x87e1b28 2024-09-14T22:37:27.653 INFO:teuthology.orchestra.run.smithi001.stdout:3881 trunc from 0x87e1b28 to 0xdc040c5 2024-09-14T22:37:27.670 INFO:teuthology.orchestra.run.smithi001.stdout:3884 trunc from 0xdc040c5 to 0x6462e05 2024-09-14T22:37:27.677 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:27.679 INFO:teuthology.orchestra.run.smithi001.stdout:3886 trunc from 0x6462e05 to 0x4ed6166 2024-09-14T22:37:27.686 INFO:teuthology.orchestra.run.smithi001.stdout:3887 write 0x1d19dec thru 0x1d1cf04 (0x3119 bytes) 2024-09-14T22:37:27.708 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:27.703+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:27.744 INFO:teuthology.orchestra.run.smithi001.stdout:3888 write 0xe9db0ad thru 0xe9e4097 (0x8feb bytes) 2024-09-14T22:37:27.754 INFO:teuthology.orchestra.run.smithi001.stdout:3889 punch from 0xe7ea962 to 0xe7f4769, (0x9e07 bytes) 2024-09-14T22:37:27.754 INFO:teuthology.orchestra.run.smithi001.stdout:3890 write 0xe1acb72 thru 0xe1bc1c8 (0xf657 bytes) 2024-09-14T22:37:27.765 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:27.765 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:37:27.765 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:37:27.765 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:37:27.765 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:27.770 INFO:teuthology.orchestra.run.smithi001.stdout:3891 trunc from 0xe9e4098 to 0xa381d3c 2024-09-14T22:37:27.786 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:37:27.787 INFO:teuthology.orchestra.run.smithi001.stdout:3892 read 0x97d9e20 thru 0x97dab27 (0xd08 bytes) 2024-09-14T22:37:27.799 INFO:teuthology.orchestra.run.smithi001.stdout:3893 trunc from 0xa381d3c to 0x25d4c63 2024-09-14T22:37:27.809 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:27.803+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:27.910 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:27.907+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:27.927 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:27.927 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:37:27.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:37:27.928 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:37:27.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.928 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4149 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13129728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:27.928 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703281+0000", 2024-09-14T22:37:27.928 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 64.223643683000006, 2024-09-14T22:37:27.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045957114, 2024-09-14T22:37:27.928 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:27.928 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:27.928 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:27.929 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:27.929 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:27.929 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:27.929 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4149 2024-09-14T22:37:27.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.929 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:27.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:27.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703281+0000", 2024-09-14T22:37:27.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:27.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:27.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703281+0000", 2024-09-14T22:37:27.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:27.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:27.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703282+0000", 2024-09-14T22:37:27.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4199999999999996e-07 2024-09-14T22:37:27.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:27.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703289+0000", 2024-09-14T22:37:27.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0770000000000002e-06 2024-09-14T22:37:27.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:27.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703296+0000", 2024-09-14T22:37:27.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8129999999999998e-06 2024-09-14T22:37:27.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:27.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703307+0000", 2024-09-14T22:37:27.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1663e-05 2024-09-14T22:37:27.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:27.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749073+0000", 2024-09-14T22:37:27.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.04576537 2024-09-14T22:37:27.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:27.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749156+0000", 2024-09-14T22:37:27.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3399999999999994e-05 2024-09-14T22:37:27.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:27.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749239+0000", 2024-09-14T22:37:27.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2249000000000005e-05 2024-09-14T22:37:27.933 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:27.933 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:27.933 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:27.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.933 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4150 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13137920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:27.933 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703318+0000", 2024-09-14T22:37:27.933 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 64.223607489000003, 2024-09-14T22:37:27.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046170585, 2024-09-14T22:37:27.933 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:27.934 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:27.934 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:27.934 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:27.934 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:27.934 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:27.934 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4150 2024-09-14T22:37:27.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.934 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:27.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:27.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703318+0000", 2024-09-14T22:37:27.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:27.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:27.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703318+0000", 2024-09-14T22:37:27.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:27.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:27.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703318+0000", 2024-09-14T22:37:27.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8800000000000003e-07 2024-09-14T22:37:27.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:27.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703321+0000", 2024-09-14T22:37:27.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7860000000000001e-06 2024-09-14T22:37:27.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:27.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703328+0000", 2024-09-14T22:37:27.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2810000000000003e-06 2024-09-14T22:37:27.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:27.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703340+0000", 2024-09-14T22:37:27.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1745e-05 2024-09-14T22:37:27.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:27.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749286+0000", 2024-09-14T22:37:27.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045946222000000002 2024-09-14T22:37:27.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:27.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749390+0000", 2024-09-14T22:37:27.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000103614 2024-09-14T22:37:27.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:27.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749488+0000", 2024-09-14T22:37:27.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8449000000000006e-05 2024-09-14T22:37:27.938 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:27.938 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:27.938 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:27.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.939 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4151 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13146112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:27.939 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703348+0000", 2024-09-14T22:37:27.939 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 64.223577325999997, 2024-09-14T22:37:27.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046349509999999997, 2024-09-14T22:37:27.939 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:27.939 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:27.939 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:27.939 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:27.939 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:27.939 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:27.939 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4151 2024-09-14T22:37:27.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.940 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:27.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:27.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703348+0000", 2024-09-14T22:37:27.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:27.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:27.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703348+0000", 2024-09-14T22:37:27.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:27.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:27.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703348+0000", 2024-09-14T22:37:27.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6899999999999998e-07 2024-09-14T22:37:27.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:27.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703351+0000", 2024-09-14T22:37:27.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5950000000000001e-06 2024-09-14T22:37:27.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:27.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703357+0000", 2024-09-14T22:37:27.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4150000000000004e-06 2024-09-14T22:37:27.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:27.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703364+0000", 2024-09-14T22:37:27.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4069999999999998e-06 2024-09-14T22:37:27.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:27.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749530+0000", 2024-09-14T22:37:27.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046166281000000003 2024-09-14T22:37:27.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:27.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749615+0000", 2024-09-14T22:37:27.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5409999999999997e-05 2024-09-14T22:37:27.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:27.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749697+0000", 2024-09-14T22:37:27.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1933000000000003e-05 2024-09-14T22:37:27.943 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:27.943 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:27.943 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:27.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.943 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4152 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13154304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:27.943 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703371+0000", 2024-09-14T22:37:27.943 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 64.223553924000001, 2024-09-14T22:37:27.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046552233999999998, 2024-09-14T22:37:27.944 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:27.944 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:27.944 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:27.944 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:27.944 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:27.944 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:27.944 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4152 2024-09-14T22:37:27.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.944 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:27.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:27.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703371+0000", 2024-09-14T22:37:27.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:27.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:27.945 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703371+0000", 2024-09-14T22:37:27.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:27.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:27.945 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703371+0000", 2024-09-14T22:37:27.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.34e-07 2024-09-14T22:37:27.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:27.945 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703374+0000", 2024-09-14T22:37:27.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.227e-06 2024-09-14T22:37:27.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:27.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703380+0000", 2024-09-14T22:37:27.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4699999999999999e-06 2024-09-14T22:37:27.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:27.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703385+0000", 2024-09-14T22:37:27.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.259e-06 2024-09-14T22:37:27.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:27.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749742+0000", 2024-09-14T22:37:27.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046357031 2024-09-14T22:37:27.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:27.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749825+0000", 2024-09-14T22:37:27.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2674000000000002e-05 2024-09-14T22:37:27.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:27.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749923+0000", 2024-09-14T22:37:27.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8239000000000006e-05 2024-09-14T22:37:27.948 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:27.948 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:27.948 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:27.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.948 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4153 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13162496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:27.948 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.703392+0000", 2024-09-14T22:37:27.948 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 64.223532628000001, 2024-09-14T22:37:27.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046675756999999998, 2024-09-14T22:37:27.949 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:27.949 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:27.949 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:27.949 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:27.949 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:27.949 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:27.949 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4153 2024-09-14T22:37:27.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.949 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:27.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:27.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703392+0000", 2024-09-14T22:37:27.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:27.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:27.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.703392+0000", 2024-09-14T22:37:27.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:27.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:27.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.705934+0000", 2024-09-14T22:37:27.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025418540000000001 2024-09-14T22:37:27.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:27.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.705940+0000", 2024-09-14T22:37:27.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0950000000000001e-06 2024-09-14T22:37:27.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.951 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:27.951 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.705952+0000", 2024-09-14T22:37:27.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1973e-05 2024-09-14T22:37:27.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.951 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:27.951 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.705958+0000", 2024-09-14T22:37:27.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.541e-06 2024-09-14T22:37:27.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.951 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:27.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.749969+0000", 2024-09-14T22:37:27.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044011300000000003 2024-09-14T22:37:27.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:27.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.750020+0000", 2024-09-14T22:37:27.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1054000000000003e-05 2024-09-14T22:37:27.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:27.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.750068+0000", 2024-09-14T22:37:27.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7939999999999998e-05 2024-09-14T22:37:27.953 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:27.953 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:27.953 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:27.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.953 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4165 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13260800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:27.953 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706178+0000", 2024-09-14T22:37:27.953 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 64.220746715999994, 2024-09-14T22:37:27.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045915594999999997, 2024-09-14T22:37:27.954 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:27.954 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:27.954 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:27.954 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:27.954 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:27.954 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:27.954 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4165 2024-09-14T22:37:27.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.955 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:27.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:27.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706178+0000", 2024-09-14T22:37:27.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:27.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:27.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706178+0000", 2024-09-14T22:37:27.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:27.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:27.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706179+0000", 2024-09-14T22:37:27.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4999999999999998e-07 2024-09-14T22:37:27.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:27.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706181+0000", 2024-09-14T22:37:27.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4679999999999999e-06 2024-09-14T22:37:27.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:27.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706189+0000", 2024-09-14T22:37:27.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.994e-06 2024-09-14T22:37:27.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:27.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706195+0000", 2024-09-14T22:37:27.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6119999999999998e-06 2024-09-14T22:37:27.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:27.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752003+0000", 2024-09-14T22:37:27.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045807618000000001 2024-09-14T22:37:27.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:27.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752050+0000", 2024-09-14T22:37:27.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7301e-05 2024-09-14T22:37:27.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:27.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752094+0000", 2024-09-14T22:37:27.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4152e-05 2024-09-14T22:37:27.958 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:27.958 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:27.958 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:27.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.959 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4166 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13268992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:27.959 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706205+0000", 2024-09-14T22:37:27.959 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 64.220720526999997, 2024-09-14T22:37:27.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046092665999999997, 2024-09-14T22:37:27.959 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:27.959 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:27.959 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:27.959 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:27.959 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:27.959 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:27.959 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4166 2024-09-14T22:37:27.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.959 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:27.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:27.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706205+0000", 2024-09-14T22:37:27.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:27.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:27.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706205+0000", 2024-09-14T22:37:27.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:27.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:27.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706205+0000", 2024-09-14T22:37:27.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:37:27.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:27.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706208+0000", 2024-09-14T22:37:27.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8049999999999998e-06 2024-09-14T22:37:27.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:27.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706215+0000", 2024-09-14T22:37:27.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2010000000000002e-06 2024-09-14T22:37:27.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:27.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706223+0000", 2024-09-14T22:37:27.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7260000000000002e-06 2024-09-14T22:37:27.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:27.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752129+0000", 2024-09-14T22:37:27.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045906532 2024-09-14T22:37:27.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:27.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752213+0000", 2024-09-14T22:37:27.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4060000000000005e-05 2024-09-14T22:37:27.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:27.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752297+0000", 2024-09-14T22:37:27.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3807999999999994e-05 2024-09-14T22:37:27.963 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:27.963 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:27.963 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:27.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.963 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4167 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13277184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:27.963 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706233+0000", 2024-09-14T22:37:27.963 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 64.220691916000007, 2024-09-14T22:37:27.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046190500000000002, 2024-09-14T22:37:27.964 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:27.964 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:27.964 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:27.964 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:27.964 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:27.964 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:27.964 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4167 2024-09-14T22:37:27.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.964 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:27.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:27.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706233+0000", 2024-09-14T22:37:27.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:27.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:27.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706233+0000", 2024-09-14T22:37:27.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:27.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:27.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706234+0000", 2024-09-14T22:37:27.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.75e-07 2024-09-14T22:37:27.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:27.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706237+0000", 2024-09-14T22:37:27.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9890000000000001e-06 2024-09-14T22:37:27.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:27.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706244+0000", 2024-09-14T22:37:27.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6240000000000002e-06 2024-09-14T22:37:27.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:27.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706255+0000", 2024-09-14T22:37:27.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0328e-05 2024-09-14T22:37:27.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:27.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752338+0000", 2024-09-14T22:37:27.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046083172999999998 2024-09-14T22:37:27.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:27.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752381+0000", 2024-09-14T22:37:27.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2973999999999999e-05 2024-09-14T22:37:27.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:27.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752424+0000", 2024-09-14T22:37:27.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2936999999999999e-05 2024-09-14T22:37:27.968 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:27.968 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:27.968 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:27.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.968 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4168 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13285376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:27.968 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706261+0000", 2024-09-14T22:37:27.968 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 64.220664572000004, 2024-09-14T22:37:27.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046378635000000001, 2024-09-14T22:37:27.968 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:27.968 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:27.969 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:27.969 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:27.969 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:27.969 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:27.969 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4168 2024-09-14T22:37:27.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.969 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:27.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:27.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706261+0000", 2024-09-14T22:37:27.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:27.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:27.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706261+0000", 2024-09-14T22:37:27.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:27.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:27.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706261+0000", 2024-09-14T22:37:27.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2899999999999999e-07 2024-09-14T22:37:27.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:27.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706264+0000", 2024-09-14T22:37:27.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.774e-06 2024-09-14T22:37:27.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:27.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706267+0000", 2024-09-14T22:37:27.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7179999999999998e-06 2024-09-14T22:37:27.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:27.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706274+0000", 2024-09-14T22:37:27.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7599999999999997e-06 2024-09-14T22:37:27.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:27.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752458+0000", 2024-09-14T22:37:27.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046183825999999997 2024-09-14T22:37:27.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.973 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:27.973 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752545+0000", 2024-09-14T22:37:27.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7272000000000001e-05 2024-09-14T22:37:27.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.973 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:27.973 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752639+0000", 2024-09-14T22:37:27.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3956000000000006e-05 2024-09-14T22:37:27.973 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:27.974 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:27.974 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:27.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.974 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4169 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13293568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:27.974 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706278+0000", 2024-09-14T22:37:27.974 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 64.220646845000005, 2024-09-14T22:37:27.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046489181999999997, 2024-09-14T22:37:27.974 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:27.974 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:27.974 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:27.974 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:27.974 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:27.975 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:27.975 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4169 2024-09-14T22:37:27.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.975 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:27.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:27.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706278+0000", 2024-09-14T22:37:27.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:27.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:27.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706278+0000", 2024-09-14T22:37:27.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:27.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:27.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706279+0000", 2024-09-14T22:37:27.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6100000000000002e-07 2024-09-14T22:37:27.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:27.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706280+0000", 2024-09-14T22:37:27.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2589999999999999e-06 2024-09-14T22:37:27.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:27.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706283+0000", 2024-09-14T22:37:27.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.501e-06 2024-09-14T22:37:27.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:27.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706286+0000", 2024-09-14T22:37:27.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5409999999999999e-06 2024-09-14T22:37:27.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:27.978 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752680+0000", 2024-09-14T22:37:27.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046393886000000002 2024-09-14T22:37:27.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:27.978 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752723+0000", 2024-09-14T22:37:27.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2870000000000001e-05 2024-09-14T22:37:27.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:27.978 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752767+0000", 2024-09-14T22:37:27.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4864000000000003e-05 2024-09-14T22:37:27.978 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:27.979 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:27.979 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:27.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.979 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:4170 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 13301760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:37:27.979 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.706289+0000", 2024-09-14T22:37:27.979 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 64.220635947000005, 2024-09-14T22:37:27.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046691115999999998, 2024-09-14T22:37:27.979 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:27.979 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:27.979 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:27.979 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:27.980 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:27.980 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:27.980 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4170 2024-09-14T22:37:27.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.980 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:27.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:27.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706289+0000", 2024-09-14T22:37:27.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:27.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:27.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706289+0000", 2024-09-14T22:37:27.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:27.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:27.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706289+0000", 2024-09-14T22:37:27.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3200000000000001e-07 2024-09-14T22:37:27.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:27.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706296+0000", 2024-09-14T22:37:27.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.922e-06 2024-09-14T22:37:27.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:27.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706303+0000", 2024-09-14T22:37:27.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3980000000000004e-06 2024-09-14T22:37:27.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:27.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.706307+0000", 2024-09-14T22:37:27.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6750000000000001e-06 2024-09-14T22:37:27.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:27.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752799+0000", 2024-09-14T22:37:27.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046491609000000003 2024-09-14T22:37:27.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:27.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752884+0000", 2024-09-14T22:37:27.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4876999999999994e-05 2024-09-14T22:37:27.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:27.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.752980+0000", 2024-09-14T22:37:27.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6403000000000005e-05 2024-09-14T22:37:27.983 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:27.984 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:27.984 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:27.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.984 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6292 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8329216~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:27.984 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966066+0000", 2024-09-14T22:37:27.984 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 63.960859009000004, 2024-09-14T22:37:27.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045967367000000002, 2024-09-14T22:37:27.984 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:27.984 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:27.984 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:27.984 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:27.985 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:27.985 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:27.985 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6292 2024-09-14T22:37:27.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.985 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:27.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:27.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966066+0000", 2024-09-14T22:37:27.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:27.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:27.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966066+0000", 2024-09-14T22:37:27.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:27.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:27.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966067+0000", 2024-09-14T22:37:27.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6499999999999999e-07 2024-09-14T22:37:27.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:27.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966069+0000", 2024-09-14T22:37:27.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4059999999999999e-06 2024-09-14T22:37:27.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:27.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966077+0000", 2024-09-14T22:37:27.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5120000000000002e-06 2024-09-14T22:37:27.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:27.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966083+0000", 2024-09-14T22:37:27.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5590000000000002e-06 2024-09-14T22:37:27.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:27.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.011820+0000", 2024-09-14T22:37:27.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045737281999999997 2024-09-14T22:37:27.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:27.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.011911+0000", 2024-09-14T22:37:27.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0153000000000002e-05 2024-09-14T22:37:27.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:27.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012033+0000", 2024-09-14T22:37:27.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012289000000000001 2024-09-14T22:37:27.989 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:27.989 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:27.989 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:27.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.989 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6293 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8337408~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:27.989 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966091+0000", 2024-09-14T22:37:27.989 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 63.960833915000002, 2024-09-14T22:37:27.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046181672, 2024-09-14T22:37:27.989 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:27.989 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:27.989 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:27.990 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:27.990 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:27.990 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:27.990 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6293 2024-09-14T22:37:27.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.990 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:27.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:27.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966091+0000", 2024-09-14T22:37:27.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:27.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:27.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966091+0000", 2024-09-14T22:37:27.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:27.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:27.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966092+0000", 2024-09-14T22:37:27.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:37:27.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:27.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966094+0000", 2024-09-14T22:37:27.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5830000000000001e-06 2024-09-14T22:37:27.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:27.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966102+0000", 2024-09-14T22:37:27.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7470000000000005e-06 2024-09-14T22:37:27.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:27.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966107+0000", 2024-09-14T22:37:27.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1209999999999998e-06 2024-09-14T22:37:27.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:27.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012079+0000", 2024-09-14T22:37:27.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045971953000000003 2024-09-14T22:37:27.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:27.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012168+0000", 2024-09-14T22:37:27.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9398e-05 2024-09-14T22:37:27.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:27.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012273+0000", 2024-09-14T22:37:27.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000104483 2024-09-14T22:37:27.994 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:27.994 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:27.994 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:27.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.994 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6294 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8345600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:27.994 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966115+0000", 2024-09-14T22:37:27.994 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 63.960810551999998, 2024-09-14T22:37:27.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046297036999999999, 2024-09-14T22:37:27.994 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:27.994 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:27.995 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:27.995 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:27.995 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:27.995 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:27.995 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6294 2024-09-14T22:37:27.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.995 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:27.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:27.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966115+0000", 2024-09-14T22:37:27.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:27.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:27.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966115+0000", 2024-09-14T22:37:27.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:27.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:27.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966115+0000", 2024-09-14T22:37:27.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7e-07 2024-09-14T22:37:27.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:27.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966118+0000", 2024-09-14T22:37:27.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.571e-06 2024-09-14T22:37:27.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:27.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966125+0000", 2024-09-14T22:37:27.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1690000000000003e-06 2024-09-14T22:37:27.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:27.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966130+0000", 2024-09-14T22:37:27.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3290000000000001e-06 2024-09-14T22:37:27.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:27.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012298+0000", 2024-09-14T22:37:27.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046168168000000002 2024-09-14T22:37:27.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:27.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012348+0000", 2024-09-14T22:37:27.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9663999999999997e-05 2024-09-14T22:37:27.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:27.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012412+0000", 2024-09-14T22:37:27.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3665999999999997e-05 2024-09-14T22:37:27.998 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:27.998 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:27.998 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:27.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:27.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:27.999 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6295 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8353792~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:27.999 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966138+0000", 2024-09-14T22:37:27.999 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 63.960787574000001, 2024-09-14T22:37:27.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046505112000000001, 2024-09-14T22:37:27.999 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:27.999 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:27.999 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:27.999 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:27.999 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:27.999 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:27.999 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6295 2024-09-14T22:37:27.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.000 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:28.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:28.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966138+0000", 2024-09-14T22:37:28.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:28.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966138+0000", 2024-09-14T22:37:28.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:28.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966138+0000", 2024-09-14T22:37:28.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5900000000000002e-07 2024-09-14T22:37:28.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:28.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966140+0000", 2024-09-14T22:37:28.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4710000000000001e-06 2024-09-14T22:37:28.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:28.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966147+0000", 2024-09-14T22:37:28.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6760000000000001e-06 2024-09-14T22:37:28.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.002 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:28.002 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966153+0000", 2024-09-14T22:37:28.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4779999999999999e-06 2024-09-14T22:37:28.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.002 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:28.002 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012452+0000", 2024-09-14T22:37:28.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046299298000000003 2024-09-14T22:37:28.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.002 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:28.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012537+0000", 2024-09-14T22:37:28.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5439999999999995e-05 2024-09-14T22:37:28.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:28.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012643+0000", 2024-09-14T22:37:28.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010529 2024-09-14T22:37:28.003 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.003 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:28.003 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.003 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6296 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8361984~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:28.004 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966160+0000", 2024-09-14T22:37:28.004 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 63.960765520000002, 2024-09-14T22:37:28.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046601423000000003, 2024-09-14T22:37:28.004 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:28.004 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:28.004 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:28.004 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:28.004 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:28.004 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:28.004 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6296 2024-09-14T22:37:28.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.005 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:28.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:28.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966160+0000", 2024-09-14T22:37:28.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:28.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966160+0000", 2024-09-14T22:37:28.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:28.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966160+0000", 2024-09-14T22:37:28.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7999999999999995e-07 2024-09-14T22:37:28.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:28.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966163+0000", 2024-09-14T22:37:28.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.514e-06 2024-09-14T22:37:28.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:28.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966169+0000", 2024-09-14T22:37:28.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8880000000000002e-06 2024-09-14T22:37:28.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:28.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966173+0000", 2024-09-14T22:37:28.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9189999999999998e-06 2024-09-14T22:37:28.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:28.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012660+0000", 2024-09-14T22:37:28.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046486522000000002 2024-09-14T22:37:28.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:28.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012703+0000", 2024-09-14T22:37:28.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2715e-05 2024-09-14T22:37:28.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:28.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012761+0000", 2024-09-14T22:37:28.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8285000000000002e-05 2024-09-14T22:37:28.009 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.009 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:28.009 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.009 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6297 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8370176~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:28.009 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966180+0000", 2024-09-14T22:37:28.009 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 63.960744716000001, 2024-09-14T22:37:28.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046808482999999998, 2024-09-14T22:37:28.009 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:28.009 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:28.009 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:28.010 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:28.010 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:28.010 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:28.010 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6297 2024-09-14T22:37:28.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.010 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:28.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:28.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966180+0000", 2024-09-14T22:37:28.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:28.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966180+0000", 2024-09-14T22:37:28.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:28.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966181+0000", 2024-09-14T22:37:28.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0600000000000001e-07 2024-09-14T22:37:28.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:28.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966183+0000", 2024-09-14T22:37:28.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3539999999999998e-06 2024-09-14T22:37:28.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:28.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966190+0000", 2024-09-14T22:37:28.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.489e-06 2024-09-14T22:37:28.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:28.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966195+0000", 2024-09-14T22:37:28.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3890000000000004e-06 2024-09-14T22:37:28.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:28.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012799+0000", 2024-09-14T22:37:28.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046604200999999998 2024-09-14T22:37:28.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:28.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012883+0000", 2024-09-14T22:37:28.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3714000000000005e-05 2024-09-14T22:37:28.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:28.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.012989+0000", 2024-09-14T22:37:28.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010593 2024-09-14T22:37:28.013 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.013 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:28.014 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.014 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6298 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8378368~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:28.014 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966203+0000", 2024-09-14T22:37:28.014 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 63.960721655, 2024-09-14T22:37:28.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046904649999999999, 2024-09-14T22:37:28.014 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:28.014 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:28.014 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:28.014 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:28.015 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:28.015 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:28.015 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6298 2024-09-14T22:37:28.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.015 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:28.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:28.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966203+0000", 2024-09-14T22:37:28.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:28.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966203+0000", 2024-09-14T22:37:28.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:28.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966205+0000", 2024-09-14T22:37:28.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5009999999999999e-06 2024-09-14T22:37:28.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:28.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966207+0000", 2024-09-14T22:37:28.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2019999999999998e-06 2024-09-14T22:37:28.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:28.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966214+0000", 2024-09-14T22:37:28.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.815e-06 2024-09-14T22:37:28.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:28.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966219+0000", 2024-09-14T22:37:28.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0590000000000002e-06 2024-09-14T22:37:28.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:28.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013005+0000", 2024-09-14T22:37:28.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046785852000000003 2024-09-14T22:37:28.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:28.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013050+0000", 2024-09-14T22:37:28.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.511e-05 2024-09-14T22:37:28.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:28.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013108+0000", 2024-09-14T22:37:28.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8111e-05 2024-09-14T22:37:28.018 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.018 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:28.018 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.019 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:6299 3.0 3:01b6342f:::rbd_data.113c3b74a67f.0000000000000003:4 [read 8386560~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:28.019 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:23.966238+0000", 2024-09-14T22:37:28.019 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 63.960687333999999, 2024-09-14T22:37:28.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047091099999999997, 2024-09-14T22:37:28.019 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:28.019 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:28.019 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:28.020 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:28.020 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:28.020 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:28.020 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6299 2024-09-14T22:37:28.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.020 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:28.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.020 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:28.020 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966238+0000", 2024-09-14T22:37:28.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:28.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966238+0000", 2024-09-14T22:37:28.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:28.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966238+0000", 2024-09-14T22:37:28.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1500000000000005e-07 2024-09-14T22:37:28.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:28.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966241+0000", 2024-09-14T22:37:28.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8270000000000002e-06 2024-09-14T22:37:28.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:28.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966249+0000", 2024-09-14T22:37:28.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0029999999999994e-06 2024-09-14T22:37:28.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:28.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:23.966255+0000", 2024-09-14T22:37:28.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7760000000000003e-06 2024-09-14T22:37:28.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.023 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:28.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013136+0000", 2024-09-14T22:37:28.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046881379000000001 2024-09-14T22:37:28.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.023 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:28.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013223+0000", 2024-09-14T22:37:28.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6681000000000006e-05 2024-09-14T22:37:28.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.023 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:28.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:24.013329+0000", 2024-09-14T22:37:28.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105919 2024-09-14T22:37:28.024 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.024 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:28.024 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.024 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:31370 3.0 3:02f48f58:::rbd_data.113cda0183a0.0000000000000000:6 [read 813056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:28.024 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:37:28.024 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 54.564527241, 2024-09-14T22:37:28.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047519796000000003, 2024-09-14T22:37:28.024 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:28.024 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:28.024 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:28.024 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:28.025 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:28.025 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:28.025 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 31370 2024-09-14T22:37:28.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.025 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:28.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:28.025 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:37:28.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:28.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:37:28.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:28.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:37:28.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0999999999999999e-07 2024-09-14T22:37:28.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:28.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362401+0000", 2024-09-14T22:37:28.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2900000000000001e-06 2024-09-14T22:37:28.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:28.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362406+0000", 2024-09-14T22:37:28.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6960000000000002e-06 2024-09-14T22:37:28.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:28.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362412+0000", 2024-09-14T22:37:28.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.823e-06 2024-09-14T22:37:28.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:28.028 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.400270+0000", 2024-09-14T22:37:28.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037857737000000002 2024-09-14T22:37:28.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:28.028 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.400342+0000", 2024-09-14T22:37:28.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1797000000000004e-05 2024-09-14T22:37:28.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:28.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.409918+0000", 2024-09-14T22:37:28.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0095760429999999994 2024-09-14T22:37:28.029 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.029 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:28.029 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.029 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.029 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:28.029 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:28.031 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:37:28.031 INFO:teuthology.orchestra.run.smithi001.stdout:3895 writesame 0xdf72d6d thru 0xdf76765 data_size 0x33(0x39f9 bytes) 2024-09-14T22:37:28.032 INFO:teuthology.orchestra.run.smithi001.stdout:3897 write 0x37e982b thru 0x37f8d9a (0xf570 bytes) 2024-09-14T22:37:28.032 INFO:teuthology.orchestra.run.smithi001.stdout:3898 read 0x2e1b6ad thru 0x2e2b56a (0xfebe bytes) 2024-09-14T22:37:28.032 INFO:teuthology.orchestra.run.smithi001.stdout:3899 read 0x9f2de67 thru 0x9f3afe3 (0xd17d bytes) 2024-09-14T22:37:28.032 INFO:teuthology.orchestra.run.smithi001.stdout:3900 trunc from 0xdf76766 to 0x8aa21e0 2024-09-14T22:37:28.032 INFO:teuthology.orchestra.run.smithi001.stdout:3901 trunc from 0x8aa21e0 to 0x9e0551a 2024-09-14T22:37:28.032 INFO:teuthology.orchestra.run.smithi001.stdout:3902 clone 25 order 22 su 524288 sc 6 2024-09-14T22:37:28.032 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:28.023+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:28.132 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:28.127+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:28.204 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:28.204 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:37:28.204 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:37:28.204 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:28.219 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:37:28.233 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:28.227+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:28.334 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:28.331+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:28.389 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:28.389 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:37:28.389 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:37:28.389 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:37:28.389 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:28.403 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:37:28.445 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:28.443+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:28.535 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:28.531+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:28.574 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:28.574 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:37:28.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:37:28.574 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:37:28.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.574 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35306 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1425408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:28.574 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507069+0000", 2024-09-14T22:37:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 55.064872489999999, 2024-09-14T22:37:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045725259999999997, 2024-09-14T22:37:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35306 2024-09-14T22:37:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507069+0000", 2024-09-14T22:37:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507069+0000", 2024-09-14T22:37:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507070+0000", 2024-09-14T22:37:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0400000000000002e-07 2024-09-14T22:37:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507072+0000", 2024-09-14T22:37:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.891e-06 2024-09-14T22:37:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507078+0000", 2024-09-14T22:37:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0279999999999997e-06 2024-09-14T22:37:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:28.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507082+0000", 2024-09-14T22:37:28.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4680000000000003e-06 2024-09-14T22:37:28.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:28.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552632+0000", 2024-09-14T22:37:28.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045550144000000001 2024-09-14T22:37:28.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552707+0000", 2024-09-14T22:37:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4148999999999998e-05 2024-09-14T22:37:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552795+0000", 2024-09-14T22:37:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8176000000000004e-05 2024-09-14T22:37:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35307 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1454080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:37:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 55.064853939000002, 2024-09-14T22:37:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045891358, 2024-09-14T22:37:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35307 2024-09-14T22:37:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:37:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:37:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:37:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4700000000000002e-07 2024-09-14T22:37:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507093+0000", 2024-09-14T22:37:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9520000000000002e-06 2024-09-14T22:37:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507099+0000", 2024-09-14T22:37:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0240000000000003e-06 2024-09-14T22:37:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507105+0000", 2024-09-14T22:37:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5520000000000004e-06 2024-09-14T22:37:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552826+0000", 2024-09-14T22:37:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045721494000000001 2024-09-14T22:37:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552896+0000", 2024-09-14T22:37:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9519999999999998e-05 2024-09-14T22:37:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552979+0000", 2024-09-14T22:37:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3468999999999997e-05 2024-09-14T22:37:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35308 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1482752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:37:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 55.064830942999997, 2024-09-14T22:37:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046076533000000003, 2024-09-14T22:37:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35308 2024-09-14T22:37:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:37:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:37:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:37:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:37:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507113+0000", 2024-09-14T22:37:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9099999999999999e-06 2024-09-14T22:37:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507119+0000", 2024-09-14T22:37:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7440000000000004e-06 2024-09-14T22:37:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507124+0000", 2024-09-14T22:37:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2129999999999999e-06 2024-09-14T22:37:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553007+0000", 2024-09-14T22:37:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045882488999999999 2024-09-14T22:37:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553076+0000", 2024-09-14T22:37:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9233000000000006e-05 2024-09-14T22:37:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553188+0000", 2024-09-14T22:37:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111557 2024-09-14T22:37:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35309 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1511424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:37:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 55.064812037999999, 2024-09-14T22:37:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046231869000000002, 2024-09-14T22:37:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35309 2024-09-14T22:37:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:37:28.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:28.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:37:28.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:28.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:37:28.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9099999999999999e-07 2024-09-14T22:37:28.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507132+0000", 2024-09-14T22:37:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8190000000000001e-06 2024-09-14T22:37:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507139+0000", 2024-09-14T22:37:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9419999999999998e-06 2024-09-14T22:37:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507145+0000", 2024-09-14T22:37:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2600000000000002e-06 2024-09-14T22:37:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553225+0000", 2024-09-14T22:37:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046079991000000001 2024-09-14T22:37:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553284+0000", 2024-09-14T22:37:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8638999999999999e-05 2024-09-14T22:37:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553362+0000", 2024-09-14T22:37:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7826999999999999e-05 2024-09-14T22:37:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35310 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1540096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507151+0000", 2024-09-14T22:37:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 55.064790582000001, 2024-09-14T22:37:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046444541999999998, 2024-09-14T22:37:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35310 2024-09-14T22:37:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507151+0000", 2024-09-14T22:37:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507151+0000", 2024-09-14T22:37:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507152+0000", 2024-09-14T22:37:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1699999999999999e-07 2024-09-14T22:37:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507154+0000", 2024-09-14T22:37:28.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.902e-06 2024-09-14T22:37:28.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:28.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507159+0000", 2024-09-14T22:37:28.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.823e-06 2024-09-14T22:37:28.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:28.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507165+0000", 2024-09-14T22:37:28.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.587e-06 2024-09-14T22:37:28.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553395+0000", 2024-09-14T22:37:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046230318999999999 2024-09-14T22:37:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553434+0000", 2024-09-14T22:37:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8680999999999998e-05 2024-09-14T22:37:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553596+0000", 2024-09-14T22:37:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016191299999999999 2024-09-14T22:37:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35311 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1568768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:37:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 55.064771329000003, 2024-09-14T22:37:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046623592999999998, 2024-09-14T22:37:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35311 2024-09-14T22:37:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:37:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:37:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:37:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:37:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507176+0000", 2024-09-14T22:37:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3349999999999997e-06 2024-09-14T22:37:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507200+0000", 2024-09-14T22:37:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4003e-05 2024-09-14T22:37:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507206+0000", 2024-09-14T22:37:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.976e-06 2024-09-14T22:37:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553627+0000", 2024-09-14T22:37:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046420284999999999 2024-09-14T22:37:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553696+0000", 2024-09-14T22:37:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9567999999999995e-05 2024-09-14T22:37:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553794+0000", 2024-09-14T22:37:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8060000000000006e-05 2024-09-14T22:37:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35566 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1484800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514668+0000", 2024-09-14T22:37:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 55.057273551999998, 2024-09-14T22:37:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045644974999999997, 2024-09-14T22:37:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35566 2024-09-14T22:37:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514668+0000", 2024-09-14T22:37:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514668+0000", 2024-09-14T22:37:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514669+0000", 2024-09-14T22:37:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9099999999999999e-07 2024-09-14T22:37:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514671+0000", 2024-09-14T22:37:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3800000000000001e-06 2024-09-14T22:37:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514677+0000", 2024-09-14T22:37:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9660000000000001e-06 2024-09-14T22:37:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514682+0000", 2024-09-14T22:37:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0939999999999998e-06 2024-09-14T22:37:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560133+0000", 2024-09-14T22:37:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045450398000000003 2024-09-14T22:37:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560207+0000", 2024-09-14T22:37:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4196999999999995e-05 2024-09-14T22:37:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560313+0000", 2024-09-14T22:37:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106549 2024-09-14T22:37:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35567 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1513472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:37:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 55.057253013, 2024-09-14T22:37:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045793215999999998, 2024-09-14T22:37:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35567 2024-09-14T22:37:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:37:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:37:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:37:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1899999999999998e-07 2024-09-14T22:37:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514692+0000", 2024-09-14T22:37:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2699999999999999e-06 2024-09-14T22:37:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514698+0000", 2024-09-14T22:37:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0939999999999996e-06 2024-09-14T22:37:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514703+0000", 2024-09-14T22:37:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3870000000000003e-06 2024-09-14T22:37:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560353+0000", 2024-09-14T22:37:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045649914 2024-09-14T22:37:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560416+0000", 2024-09-14T22:37:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3096999999999996e-05 2024-09-14T22:37:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560482+0000", 2024-09-14T22:37:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6035e-05 2024-09-14T22:37:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35568 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1542144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:37:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 55.057228215000002, 2024-09-14T22:37:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045995921000000002, 2024-09-14T22:37:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35568 2024-09-14T22:37:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:37:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:37:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:37:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4099999999999999e-07 2024-09-14T22:37:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514720+0000", 2024-09-14T22:37:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6869999999999999e-06 2024-09-14T22:37:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514727+0000", 2024-09-14T22:37:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3540000000000002e-06 2024-09-14T22:37:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514732+0000", 2024-09-14T22:37:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7690000000000004e-06 2024-09-14T22:37:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560523+0000", 2024-09-14T22:37:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045791373000000003 2024-09-14T22:37:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560606+0000", 2024-09-14T22:37:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2690999999999998e-05 2024-09-14T22:37:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560710+0000", 2024-09-14T22:37:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000103606 2024-09-14T22:37:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35569 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1570816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:37:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 55.057202406000002, 2024-09-14T22:37:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046110630999999999, 2024-09-14T22:37:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35569 2024-09-14T22:37:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:37:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:37:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:37:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.82e-07 2024-09-14T22:37:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514743+0000", 2024-09-14T22:37:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.269e-06 2024-09-14T22:37:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514750+0000", 2024-09-14T22:37:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7340000000000002e-06 2024-09-14T22:37:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514755+0000", 2024-09-14T22:37:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7260000000000002e-06 2024-09-14T22:37:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560731+0000", 2024-09-14T22:37:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045976377999999998 2024-09-14T22:37:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560777+0000", 2024-09-14T22:37:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5868999999999997e-05 2024-09-14T22:37:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560850+0000", 2024-09-14T22:37:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3172999999999999e-05 2024-09-14T22:37:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:86180 3.18 3:1d0c52d8:::rbd_data.113cd62ea8a5.000000000000004b:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:01.356417+0000", 2024-09-14T22:37:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.215525311, 2024-09-14T22:37:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049143245000000002, 2024-09-14T22:37:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 86180 2024-09-14T22:37:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356417+0000", 2024-09-14T22:37:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356417+0000", 2024-09-14T22:37:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356420+0000", 2024-09-14T22:37:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2030000000000001e-06 2024-09-14T22:37:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356435+0000", 2024-09-14T22:37:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5287000000000001e-05 2024-09-14T22:37:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356472+0000", 2024-09-14T22:37:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7311000000000002e-05 2024-09-14T22:37:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356492+0000", 2024-09-14T22:37:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9519999999999999e-05 2024-09-14T22:37:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356670+0000", 2024-09-14T22:37:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000177567 2024-09-14T22:37:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356924+0000", 2024-09-14T22:37:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025399299999999999 2024-09-14T22:37:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 0", 2024-09-14T22:37:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.365052+0000", 2024-09-14T22:37:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0081289380000000005 2024-09-14T22:37:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.405396+0000", 2024-09-14T22:37:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040343228000000002 2024-09-14T22:37:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:37:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.405421+0000", 2024-09-14T22:37:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5763e-05 2024-09-14T22:37:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:37:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.405491+0000", 2024-09-14T22:37:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9721999999999994e-05 2024-09-14T22:37:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.405560+0000", 2024-09-14T22:37:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8713000000000004e-05 2024-09-14T22:37:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98378 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 919552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148183+0000", 2024-09-14T22:37:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.423759033, 2024-09-14T22:37:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045614549999999997, 2024-09-14T22:37:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98378 2024-09-14T22:37:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148183+0000", 2024-09-14T22:37:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148183+0000", 2024-09-14T22:37:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148183+0000", 2024-09-14T22:37:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0800000000000001e-07 2024-09-14T22:37:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148184+0000", 2024-09-14T22:37:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2780000000000001e-06 2024-09-14T22:37:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148189+0000", 2024-09-14T22:37:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2699999999999998e-06 2024-09-14T22:37:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148191+0000", 2024-09-14T22:37:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7659999999999999e-06 2024-09-14T22:37:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193708+0000", 2024-09-14T22:37:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045516562000000003 2024-09-14T22:37:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193751+0000", 2024-09-14T22:37:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2811000000000001e-05 2024-09-14T22:37:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193797+0000", 2024-09-14T22:37:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6655e-05 2024-09-14T22:37:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98379 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 928768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148195+0000", 2024-09-14T22:37:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.423747204000001, 2024-09-14T22:37:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045817405999999998, 2024-09-14T22:37:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98379 2024-09-14T22:37:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148195+0000", 2024-09-14T22:37:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148195+0000", 2024-09-14T22:37:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148195+0000", 2024-09-14T22:37:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3999999999999998e-07 2024-09-14T22:37:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148196+0000", 2024-09-14T22:37:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.341e-06 2024-09-14T22:37:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148200+0000", 2024-09-14T22:37:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9040000000000002e-06 2024-09-14T22:37:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148203+0000", 2024-09-14T22:37:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.959e-06 2024-09-14T22:37:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193841+0000", 2024-09-14T22:37:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045637886000000003 2024-09-14T22:37:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193933+0000", 2024-09-14T22:37:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1823999999999994e-05 2024-09-14T22:37:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194012+0000", 2024-09-14T22:37:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9252000000000006e-05 2024-09-14T22:37:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98380 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 937984~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148206+0000", 2024-09-14T22:37:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.423735566000001, 2024-09-14T22:37:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045937354, 2024-09-14T22:37:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98380 2024-09-14T22:37:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148206+0000", 2024-09-14T22:37:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148206+0000", 2024-09-14T22:37:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148207+0000", 2024-09-14T22:37:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1e-07 2024-09-14T22:37:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148221+0000", 2024-09-14T22:37:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4679e-05 2024-09-14T22:37:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148228+0000", 2024-09-14T22:37:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3820000000000001e-06 2024-09-14T22:37:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148231+0000", 2024-09-14T22:37:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2849999999999999e-06 2024-09-14T22:37:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194054+0000", 2024-09-14T22:37:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045822596 2024-09-14T22:37:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194098+0000", 2024-09-14T22:37:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4604000000000002e-05 2024-09-14T22:37:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194144+0000", 2024-09-14T22:37:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5597999999999999e-05 2024-09-14T22:37:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98381 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 947200~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148236+0000", 2024-09-14T22:37:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.423706073000002, 2024-09-14T22:37:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046111129000000001, 2024-09-14T22:37:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98381 2024-09-14T22:37:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148236+0000", 2024-09-14T22:37:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148236+0000", 2024-09-14T22:37:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148236+0000", 2024-09-14T22:37:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8799999999999998e-07 2024-09-14T22:37:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148238+0000", 2024-09-14T22:37:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6559999999999999e-06 2024-09-14T22:37:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148246+0000", 2024-09-14T22:37:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3440000000000001e-06 2024-09-14T22:37:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148251+0000", 2024-09-14T22:37:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5480000000000003e-06 2024-09-14T22:37:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194187+0000", 2024-09-14T22:37:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045935847000000002 2024-09-14T22:37:28.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:28.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194267+0000", 2024-09-14T22:37:28.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0267999999999998e-05 2024-09-14T22:37:28.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:28.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194347+0000", 2024-09-14T22:37:28.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0178000000000004e-05 2024-09-14T22:37:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98382 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 956416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148254+0000", 2024-09-14T22:37:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.423687700999999, 2024-09-14T22:37:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046224227, 2024-09-14T22:37:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98382 2024-09-14T22:37:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148254+0000", 2024-09-14T22:37:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148254+0000", 2024-09-14T22:37:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148254+0000", 2024-09-14T22:37:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3099999999999999e-07 2024-09-14T22:37:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148256+0000", 2024-09-14T22:37:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5540000000000001e-06 2024-09-14T22:37:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148259+0000", 2024-09-14T22:37:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2499999999999998e-06 2024-09-14T22:37:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148262+0000", 2024-09-14T22:37:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8310000000000001e-06 2024-09-14T22:37:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194387+0000", 2024-09-14T22:37:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046125082999999997 2024-09-14T22:37:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194430+0000", 2024-09-14T22:37:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3074999999999997e-05 2024-09-14T22:37:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194478+0000", 2024-09-14T22:37:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8202999999999999e-05 2024-09-14T22:37:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98383 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 965632~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148265+0000", 2024-09-14T22:37:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.423676586999999, 2024-09-14T22:37:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046409141000000001, 2024-09-14T22:37:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98383 2024-09-14T22:37:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148265+0000", 2024-09-14T22:37:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148265+0000", 2024-09-14T22:37:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148266+0000", 2024-09-14T22:37:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0800000000000001e-07 2024-09-14T22:37:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148267+0000", 2024-09-14T22:37:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1659999999999999e-06 2024-09-14T22:37:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148271+0000", 2024-09-14T22:37:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1289999999999998e-06 2024-09-14T22:37:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148274+0000", 2024-09-14T22:37:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5939999999999998e-06 2024-09-14T22:37:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194523+0000", 2024-09-14T22:37:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046248162000000002 2024-09-14T22:37:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194603+0000", 2024-09-14T22:37:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0161999999999996e-05 2024-09-14T22:37:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194675+0000", 2024-09-14T22:37:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1719999999999998e-05 2024-09-14T22:37:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98384 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 974848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148278+0000", 2024-09-14T22:37:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.423664258999999, 2024-09-14T22:37:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046527369999999998, 2024-09-14T22:37:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98384 2024-09-14T22:37:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148278+0000", 2024-09-14T22:37:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148278+0000", 2024-09-14T22:37:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.149987+0000", 2024-09-14T22:37:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017093430000000001 2024-09-14T22:37:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.149993+0000", 2024-09-14T22:37:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6219999999999997e-06 2024-09-14T22:37:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150006+0000", 2024-09-14T22:37:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3095000000000001e-05 2024-09-14T22:37:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150014+0000", 2024-09-14T22:37:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6340000000000007e-06 2024-09-14T22:37:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194715+0000", 2024-09-14T22:37:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044700349 2024-09-14T22:37:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194759+0000", 2024-09-14T22:37:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3807000000000003e-05 2024-09-14T22:37:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194805+0000", 2024-09-14T22:37:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6520000000000002e-05 2024-09-14T22:37:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.665 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98390 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 1030144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:28.665 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.150152+0000", 2024-09-14T22:37:28.665 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.421789572000002, 2024-09-14T22:37:28.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045642708999999997, 2024-09-14T22:37:28.665 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:28.665 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:28.665 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:28.665 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:28.665 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98390 2024-09-14T22:37:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150152+0000", 2024-09-14T22:37:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150152+0000", 2024-09-14T22:37:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150153+0000", 2024-09-14T22:37:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.08e-07 2024-09-14T22:37:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150155+0000", 2024-09-14T22:37:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2910000000000002e-06 2024-09-14T22:37:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150162+0000", 2024-09-14T22:37:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4099999999999996e-06 2024-09-14T22:37:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150167+0000", 2024-09-14T22:37:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3199999999999999e-06 2024-09-14T22:37:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195704+0000", 2024-09-14T22:37:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045537243999999998 2024-09-14T22:37:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195749+0000", 2024-09-14T22:37:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4866e-05 2024-09-14T22:37:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195795+0000", 2024-09-14T22:37:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.617e-05 2024-09-14T22:37:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98391 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 1039360~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.150175+0000", 2024-09-14T22:37:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.421766735999999, 2024-09-14T22:37:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045825232, 2024-09-14T22:37:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98391 2024-09-14T22:37:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150175+0000", 2024-09-14T22:37:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150175+0000", 2024-09-14T22:37:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150176+0000", 2024-09-14T22:37:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9700000000000002e-07 2024-09-14T22:37:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150178+0000", 2024-09-14T22:37:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.407e-06 2024-09-14T22:37:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150185+0000", 2024-09-14T22:37:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5189999999999998e-06 2024-09-14T22:37:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150190+0000", 2024-09-14T22:37:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2399999999999998e-06 2024-09-14T22:37:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195838+0000", 2024-09-14T22:37:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045648265 2024-09-14T22:37:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195918+0000", 2024-09-14T22:37:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0094999999999998e-05 2024-09-14T22:37:28.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:28.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:28.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:28.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.196000+0000", 2024-09-14T22:37:28.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2309000000000001e-05 2024-09-14T22:37:28.674 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.674 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:28.674 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.674 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:28.674 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:28.674 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:28.676 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:37:28.676 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:28.671+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:28.736 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T22:37:28.777 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:28.771+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:28.814 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:28.814 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:28.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:28.814 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4412.0:112787 3.1c e79/23 3:3831950c:::rbd_directory:head v 79'293, mlcod=78'291)", 2024-09-14T22:37:28.815 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:28.812038+0000", 2024-09-14T22:37:28.815 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.00065662100000000003, 2024-09-14T22:37:28.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00071161800000000002, 2024-09-14T22:37:28.815 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:28.815 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:28.815 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:28.815 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:28.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:28.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:28.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:28.812038+0000", 2024-09-14T22:37:28.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:28.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:28.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:28.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:28.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:28.812038+0000", 2024-09-14T22:37:28.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:28.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:28.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:28.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:28.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:28.812040+0000", 2024-09-14T22:37:28.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7430000000000001e-06 2024-09-14T22:37:28.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:28.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:28.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:28.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:28.812051+0000", 2024-09-14T22:37:28.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1303e-05 2024-09-14T22:37:28.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:28.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:28.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:28.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:28.812077+0000", 2024-09-14T22:37:28.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6256999999999999e-05 2024-09-14T22:37:28.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:28.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:28.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:28.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:28.812091+0000", 2024-09-14T22:37:28.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.314e-05 2024-09-14T22:37:28.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:28.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:28.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:28.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:28.812160+0000", 2024-09-14T22:37:28.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9252999999999996e-05 2024-09-14T22:37:28.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:28.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:28.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:28.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:28.812182+0000", 2024-09-14T22:37:28.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2036e-05 2024-09-14T22:37:28.818 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:28.818 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:28.818 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:28.819 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:28.819 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T22:37:28.819 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 1 2024-09-14T22:37:28.819 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:28.831 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:37:28.859 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone24 from 0x9e0551a (overlap 0x25d4c63) to 0x790f47 2024-09-14T22:37:28.877 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:28.875+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:28.978 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:28.971+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:29.063 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:29.063 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:29.063 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:37:29.063 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:37:29.063 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:29.078 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:29.071+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:29.085 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:37:29.179 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:29.175+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:29.255 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:29.256 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:29.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:29.256 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:29.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.256 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6395 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8386560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:29.256 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:37:29.256 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 65.318967639999997, 2024-09-14T22:37:29.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054534379000000001, 2024-09-14T22:37:29.256 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.256 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.256 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.257 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.257 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.257 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.257 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6395 2024-09-14T22:37:29.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.257 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:37:29.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:37:29.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933833+0000", 2024-09-14T22:37:29.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3859999999999999e-06 2024-09-14T22:37:29.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933838+0000", 2024-09-14T22:37:29.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8709999999999996e-06 2024-09-14T22:37:29.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933854+0000", 2024-09-14T22:37:29.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6189e-05 2024-09-14T22:37:29.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933871+0000", 2024-09-14T22:37:29.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.626e-05 2024-09-14T22:37:29.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988198+0000", 2024-09-14T22:37:29.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054327581999999999 2024-09-14T22:37:29.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988274+0000", 2024-09-14T22:37:29.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5759000000000004e-05 2024-09-14T22:37:29.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988366+0000", 2024-09-14T22:37:29.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2331999999999996e-05 2024-09-14T22:37:29.260 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.260 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.260 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.261 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104944 3.3 3:c3780f3b:::rbd_data.113ca561f7a9.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:29.261 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:37:29.261 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.219365766999999, 2024-09-14T22:37:29.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073041087000000005, 2024-09-14T22:37:29.261 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.261 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.261 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:29.261 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.261 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.261 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.261 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104944 2024-09-14T22:37:29.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.262 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:37:29.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:37:29.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033436+0000", 2024-09-14T22:37:29.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2060000000000001e-06 2024-09-14T22:37:29.263 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.263 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.263 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.263 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033448+0000", 2024-09-14T22:37:29.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.169e-05 2024-09-14T22:37:29.263 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.263 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.263 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.263 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033474+0000", 2024-09-14T22:37:29.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6400000000000001e-05 2024-09-14T22:37:29.263 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.263 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.263 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033489+0000", 2024-09-14T22:37:29.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5067e-05 2024-09-14T22:37:29.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033658+0000", 2024-09-14T22:37:29.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000168565 2024-09-14T22:37:29.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033793+0000", 2024-09-14T22:37:29.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013536900000000001 2024-09-14T22:37:29.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:37:29.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.047559+0000", 2024-09-14T22:37:29.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013766294 2024-09-14T22:37:29.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:29.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106157+0000", 2024-09-14T22:37:29.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058597110000000001 2024-09-14T22:37:29.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:29.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106290+0000", 2024-09-14T22:37:29.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000133244 2024-09-14T22:37:29.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:29.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106396+0000", 2024-09-14T22:37:29.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106246 2024-09-14T22:37:29.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106475+0000", 2024-09-14T22:37:29.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8895999999999998e-05 2024-09-14T22:37:29.266 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.266 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.266 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.267 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:111858 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 10403840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:29.267 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:37:29.267 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.4289126460000001, 2024-09-14T22:37:29.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054398798999999998, 2024-09-14T22:37:29.267 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.267 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.267 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.267 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.267 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.267 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.267 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 111858 2024-09-14T22:37:29.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.267 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.268 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:37:29.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.268 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.268 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:37:29.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.268 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.268 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823888+0000", 2024-09-14T22:37:29.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0200000000000001e-07 2024-09-14T22:37:29.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823927+0000", 2024-09-14T22:37:29.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8940999999999999e-05 2024-09-14T22:37:29.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823940+0000", 2024-09-14T22:37:29.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3682e-05 2024-09-14T22:37:29.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823950+0000", 2024-09-14T22:37:29.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7110000000000007e-06 2024-09-14T22:37:29.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878202+0000", 2024-09-14T22:37:29.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054251969999999997 2024-09-14T22:37:29.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878249+0000", 2024-09-14T22:37:29.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6854999999999998e-05 2024-09-14T22:37:29.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878286+0000", 2024-09-14T22:37:29.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6838000000000002e-05 2024-09-14T22:37:29.271 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.271 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.271 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.271 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:111859 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 10452992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:29.271 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:37:29.271 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.428823527, 2024-09-14T22:37:29.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054435292000000003, 2024-09-14T22:37:29.271 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.271 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.271 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.271 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.271 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.272 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.272 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 111859 2024-09-14T22:37:29.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.272 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.272 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.272 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.272 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:37:29.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.272 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.272 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.272 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:37:29.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823977+0000", 2024-09-14T22:37:29.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3500000000000005e-07 2024-09-14T22:37:29.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823981+0000", 2024-09-14T22:37:29.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2050000000000004e-06 2024-09-14T22:37:29.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824007+0000", 2024-09-14T22:37:29.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.62e-05 2024-09-14T22:37:29.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.274 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824018+0000", 2024-09-14T22:37:29.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0389e-05 2024-09-14T22:37:29.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.274 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878312+0000", 2024-09-14T22:37:29.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054294754000000001 2024-09-14T22:37:29.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878350+0000", 2024-09-14T22:37:29.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7982000000000003e-05 2024-09-14T22:37:29.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878411+0000", 2024-09-14T22:37:29.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0927000000000001e-05 2024-09-14T22:37:29.275 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.275 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.275 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.275 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112030 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6316032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:29.275 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:37:29.276 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.4287691849999999, 2024-09-14T22:37:29.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054597826000000002, 2024-09-14T22:37:29.276 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.276 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.276 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.276 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.276 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.276 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.276 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112030 2024-09-14T22:37:29.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.276 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:37:29.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:37:29.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824031+0000", 2024-09-14T22:37:29.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3600000000000003e-07 2024-09-14T22:37:29.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824036+0000", 2024-09-14T22:37:29.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7350000000000004e-06 2024-09-14T22:37:29.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824049+0000", 2024-09-14T22:37:29.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3329000000000001e-05 2024-09-14T22:37:29.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824057+0000", 2024-09-14T22:37:29.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2460000000000003e-06 2024-09-14T22:37:29.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878464+0000", 2024-09-14T22:37:29.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054406207999999998 2024-09-14T22:37:29.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878554+0000", 2024-09-14T22:37:29.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0199999999999997e-05 2024-09-14T22:37:29.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878628+0000", 2024-09-14T22:37:29.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4371999999999999e-05 2024-09-14T22:37:29.280 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.280 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.280 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.280 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112031 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6365184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:29.280 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:37:29.280 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.4287308670000001, 2024-09-14T22:37:29.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054741868999999999, 2024-09-14T22:37:29.280 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.280 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.280 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.280 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.281 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.281 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.281 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112031 2024-09-14T22:37:29.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.281 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:37:29.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:37:29.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824070+0000", 2024-09-14T22:37:29.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5099999999999998e-07 2024-09-14T22:37:29.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824073+0000", 2024-09-14T22:37:29.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6940000000000001e-06 2024-09-14T22:37:29.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824084+0000", 2024-09-14T22:37:29.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0380000000000001e-05 2024-09-14T22:37:29.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824091+0000", 2024-09-14T22:37:29.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5730000000000002e-06 2024-09-14T22:37:29.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878670+0000", 2024-09-14T22:37:29.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054579176 2024-09-14T22:37:29.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878742+0000", 2024-09-14T22:37:29.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T22:37:29.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878811+0000", 2024-09-14T22:37:29.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8170999999999995e-05 2024-09-14T22:37:29.284 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.284 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.284 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.284 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112032 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6414336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:29.285 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:37:29.285 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.428697001, 2024-09-14T22:37:29.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054931916999999997, 2024-09-14T22:37:29.285 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.285 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.285 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.285 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.285 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.285 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.285 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112032 2024-09-14T22:37:29.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.285 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:37:29.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:37:29.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:37:29.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:37:29.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.287 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824107+0000", 2024-09-14T22:37:29.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5640000000000001e-06 2024-09-14T22:37:29.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.287 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824117+0000", 2024-09-14T22:37:29.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0366e-05 2024-09-14T22:37:29.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.287 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824158+0000", 2024-09-14T22:37:29.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0670000000000002e-05 2024-09-14T22:37:29.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878853+0000", 2024-09-14T22:37:29.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054694853000000002 2024-09-14T22:37:29.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878934+0000", 2024-09-14T22:37:29.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1754000000000004e-05 2024-09-14T22:37:29.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879034+0000", 2024-09-14T22:37:29.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000100115 2024-09-14T22:37:29.289 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.289 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.289 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.289 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112033 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6463488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:29.289 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:37:29.289 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.428628343, 2024-09-14T22:37:29.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054991076999999999, 2024-09-14T22:37:29.289 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.289 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.289 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.289 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.289 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.290 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.290 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112033 2024-09-14T22:37:29.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.290 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:37:29.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:37:29.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824172+0000", 2024-09-14T22:37:29.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9999999999999996e-07 2024-09-14T22:37:29.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824177+0000", 2024-09-14T22:37:29.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5310000000000003e-06 2024-09-14T22:37:29.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824188+0000", 2024-09-14T22:37:29.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1235e-05 2024-09-14T22:37:29.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824197+0000", 2024-09-14T22:37:29.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7069999999999998e-06 2024-09-14T22:37:29.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879075+0000", 2024-09-14T22:37:29.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054878749999999997 2024-09-14T22:37:29.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879118+0000", 2024-09-14T22:37:29.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2749e-05 2024-09-14T22:37:29.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879162+0000", 2024-09-14T22:37:29.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4205000000000001e-05 2024-09-14T22:37:29.293 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.293 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.293 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.293 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112034 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6512640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:29.293 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:37:29.293 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.428591392, 2024-09-14T22:37:29.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055181246000000003, 2024-09-14T22:37:29.294 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.294 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.294 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.294 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.294 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.294 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.294 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112034 2024-09-14T22:37:29.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.294 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:37:29.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:37:29.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824209+0000", 2024-09-14T22:37:29.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5499999999999998e-07 2024-09-14T22:37:29.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824212+0000", 2024-09-14T22:37:29.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6119999999999999e-06 2024-09-14T22:37:29.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.296 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824224+0000", 2024-09-14T22:37:29.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1127999999999999e-05 2024-09-14T22:37:29.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.296 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824232+0000", 2024-09-14T22:37:29.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4700000000000002e-06 2024-09-14T22:37:29.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879209+0000", 2024-09-14T22:37:29.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054977201000000003 2024-09-14T22:37:29.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879308+0000", 2024-09-14T22:37:29.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8419999999999996e-05 2024-09-14T22:37:29.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879389+0000", 2024-09-14T22:37:29.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1760000000000003e-05 2024-09-14T22:37:29.297 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.298 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.298 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.298 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112035 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6561792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:29.298 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:37:29.298 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.428497664, 2024-09-14T22:37:29.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055300522999999997, 2024-09-14T22:37:29.298 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.298 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.298 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.298 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.298 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.299 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.299 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112035 2024-09-14T22:37:29.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.299 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.299 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:37:29.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.299 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:37:29.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.300 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824303+0000", 2024-09-14T22:37:29.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5099999999999998e-07 2024-09-14T22:37:29.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.300 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824308+0000", 2024-09-14T22:37:29.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6689999999999997e-06 2024-09-14T22:37:29.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.300 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824744+0000", 2024-09-14T22:37:29.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00043601600000000001 2024-09-14T22:37:29.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824756+0000", 2024-09-14T22:37:29.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2476999999999999e-05 2024-09-14T22:37:29.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879434+0000", 2024-09-14T22:37:29.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054677630999999997 2024-09-14T22:37:29.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879518+0000", 2024-09-14T22:37:29.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4220000000000003e-05 2024-09-14T22:37:29.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879602+0000", 2024-09-14T22:37:29.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4559000000000001e-05 2024-09-14T22:37:29.302 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.302 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.302 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.302 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112036 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6610944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:29.302 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:37:29.302 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.428005926, 2024-09-14T22:37:29.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055036320999999999, 2024-09-14T22:37:29.303 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.303 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.303 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.303 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.303 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.303 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.303 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112036 2024-09-14T22:37:29.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.303 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.303 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.303 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:37:29.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.304 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:37:29.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.304 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824795+0000", 2024-09-14T22:37:29.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-06 2024-09-14T22:37:29.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824802+0000", 2024-09-14T22:37:29.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6680000000000004e-06 2024-09-14T22:37:29.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824814+0000", 2024-09-14T22:37:29.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2102e-05 2024-09-14T22:37:29.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824823+0000", 2024-09-14T22:37:29.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5550000000000008e-06 2024-09-14T22:37:29.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879648+0000", 2024-09-14T22:37:29.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054825167000000001 2024-09-14T22:37:29.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879747+0000", 2024-09-14T22:37:29.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8968999999999994e-05 2024-09-14T22:37:29.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879830+0000", 2024-09-14T22:37:29.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2633999999999995e-05 2024-09-14T22:37:29.307 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.307 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.307 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.307 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112037 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6660096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:29.307 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:37:29.307 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.4279632170000001, 2024-09-14T22:37:29.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055122926000000003, 2024-09-14T22:37:29.307 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.307 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.307 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.307 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.307 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.307 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.308 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112037 2024-09-14T22:37:29.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.308 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.308 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:37:29.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.308 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:37:29.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824837+0000", 2024-09-14T22:37:29.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5499999999999998e-07 2024-09-14T22:37:29.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824840+0000", 2024-09-14T22:37:29.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0429999999999999e-06 2024-09-14T22:37:29.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824851+0000", 2024-09-14T22:37:29.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1046999999999999e-05 2024-09-14T22:37:29.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824859+0000", 2024-09-14T22:37:29.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1200000000000002e-06 2024-09-14T22:37:29.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879867+0000", 2024-09-14T22:37:29.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055007697000000001 2024-09-14T22:37:29.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879910+0000", 2024-09-14T22:37:29.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3402000000000002e-05 2024-09-14T22:37:29.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879959+0000", 2024-09-14T22:37:29.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9061999999999999e-05 2024-09-14T22:37:29.311 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.311 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.311 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.311 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112038 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6709248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:29.311 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:37:29.311 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.4279285210000001, 2024-09-14T22:37:29.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055312828000000001, 2024-09-14T22:37:29.312 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.312 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.312 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.312 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.312 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.312 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.312 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112038 2024-09-14T22:37:29.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.312 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.312 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:37:29.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.313 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:37:29.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.313 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824872+0000", 2024-09-14T22:37:29.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T22:37:29.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.313 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824875+0000", 2024-09-14T22:37:29.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7519999999999998e-06 2024-09-14T22:37:29.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.314 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824917+0000", 2024-09-14T22:37:29.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1047999999999997e-05 2024-09-14T22:37:29.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.314 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824926+0000", 2024-09-14T22:37:29.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3749999999999992e-06 2024-09-14T22:37:29.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.314 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880006+0000", 2024-09-14T22:37:29.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055080525999999998 2024-09-14T22:37:29.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880102+0000", 2024-09-14T22:37:29.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5995999999999994e-05 2024-09-14T22:37:29.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880184+0000", 2024-09-14T22:37:29.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1478999999999997e-05 2024-09-14T22:37:29.315 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.315 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.316 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.316 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112039 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6758400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:29.316 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:37:29.316 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.427859851, 2024-09-14T22:37:29.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055449709999999999, 2024-09-14T22:37:29.316 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.316 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.316 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.316 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.316 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.316 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.316 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112039 2024-09-14T22:37:29.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.317 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:37:29.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:37:29.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824941+0000", 2024-09-14T22:37:29.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3699999999999999e-07 2024-09-14T22:37:29.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824946+0000", 2024-09-14T22:37:29.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0209999999999999e-06 2024-09-14T22:37:29.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824972+0000", 2024-09-14T22:37:29.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6750999999999998e-05 2024-09-14T22:37:29.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824982+0000", 2024-09-14T22:37:29.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5019999999999998e-06 2024-09-14T22:37:29.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880227+0000", 2024-09-14T22:37:29.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055245257999999998 2024-09-14T22:37:29.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880305+0000", 2024-09-14T22:37:29.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7424999999999998e-05 2024-09-14T22:37:29.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880389+0000", 2024-09-14T22:37:29.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4815999999999996e-05 2024-09-14T22:37:29.320 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.320 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.320 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.320 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112040 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6807552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:29.320 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:37:29.320 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.427803135, 2024-09-14T22:37:29.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055534615000000002, 2024-09-14T22:37:29.320 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.321 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.321 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.321 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.321 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.321 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.321 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112040 2024-09-14T22:37:29.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.321 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:37:29.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:37:29.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824997+0000", 2024-09-14T22:37:29.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7199999999999998e-07 2024-09-14T22:37:29.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825042+0000", 2024-09-14T22:37:29.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5065999999999998e-05 2024-09-14T22:37:29.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825060+0000", 2024-09-14T22:37:29.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7520000000000002e-05 2024-09-14T22:37:29.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825070+0000", 2024-09-14T22:37:29.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8230000000000006e-06 2024-09-14T22:37:29.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880427+0000", 2024-09-14T22:37:29.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055357244 2024-09-14T22:37:29.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880483+0000", 2024-09-14T22:37:29.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5807000000000003e-05 2024-09-14T22:37:29.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880531+0000", 2024-09-14T22:37:29.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8383000000000001e-05 2024-09-14T22:37:29.324 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.324 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.324 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.325 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112041 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6856704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:29.325 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:37:29.325 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.427713961, 2024-09-14T22:37:29.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055573887000000002, 2024-09-14T22:37:29.325 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.325 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.325 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.325 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.325 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.325 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.325 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112041 2024-09-14T22:37:29.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.325 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:37:29.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:37:29.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825087+0000", 2024-09-14T22:37:29.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2399999999999996e-07 2024-09-14T22:37:29.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825091+0000", 2024-09-14T22:37:29.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9729999999999999e-06 2024-09-14T22:37:29.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825101+0000", 2024-09-14T22:37:29.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.093e-05 2024-09-14T22:37:29.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825110+0000", 2024-09-14T22:37:29.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1450000000000008e-06 2024-09-14T22:37:29.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.328 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880559+0000", 2024-09-14T22:37:29.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055449684999999999 2024-09-14T22:37:29.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.328 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880601+0000", 2024-09-14T22:37:29.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1248000000000002e-05 2024-09-14T22:37:29.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.328 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880659+0000", 2024-09-14T22:37:29.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8981999999999999e-05 2024-09-14T22:37:29.329 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.329 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.329 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.329 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112042 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6905856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:29.329 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:37:29.329 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.4276792270000001, 2024-09-14T22:37:29.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055781415000000001, 2024-09-14T22:37:29.329 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.329 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.329 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.329 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.330 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.330 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.330 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112042 2024-09-14T22:37:29.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.330 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:37:29.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:37:29.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825478+0000", 2024-09-14T22:37:29.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035775 2024-09-14T22:37:29.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825494+0000", 2024-09-14T22:37:29.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5994999999999998e-05 2024-09-14T22:37:29.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825520+0000", 2024-09-14T22:37:29.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5738999999999999e-05 2024-09-14T22:37:29.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825547+0000", 2024-09-14T22:37:29.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7667000000000001e-05 2024-09-14T22:37:29.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880725+0000", 2024-09-14T22:37:29.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055177962999999997 2024-09-14T22:37:29.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880816+0000", 2024-09-14T22:37:29.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0315000000000005e-05 2024-09-14T22:37:29.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880902+0000", 2024-09-14T22:37:29.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5986e-05 2024-09-14T22:37:29.333 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.333 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.333 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.333 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112043 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:29.333 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:37:29.334 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.4272312229999999, 2024-09-14T22:37:29.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055537912000000002, 2024-09-14T22:37:29.334 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.334 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.334 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.334 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.334 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.334 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.334 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112043 2024-09-14T22:37:29.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.334 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:37:29.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:37:29.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825911+0000", 2024-09-14T22:37:29.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00034306899999999999 2024-09-14T22:37:29.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825918+0000", 2024-09-14T22:37:29.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.002e-06 2024-09-14T22:37:29.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825931+0000", 2024-09-14T22:37:29.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2401e-05 2024-09-14T22:37:29.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825942+0000", 2024-09-14T22:37:29.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0903e-05 2024-09-14T22:37:29.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880951+0000", 2024-09-14T22:37:29.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055009379999999997 2024-09-14T22:37:29.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881027+0000", 2024-09-14T22:37:29.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5944000000000003e-05 2024-09-14T22:37:29.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881106+0000", 2024-09-14T22:37:29.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9213000000000002e-05 2024-09-14T22:37:29.337 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.337 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.338 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.338 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112044 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 7004160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:29.338 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:37:29.338 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.426837554, 2024-09-14T22:37:29.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055265416999999997, 2024-09-14T22:37:29.338 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.338 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.338 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.338 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.338 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.338 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.338 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112044 2024-09-14T22:37:29.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.339 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:37:29.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:37:29.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826239+0000", 2024-09-14T22:37:29.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027676000000000003 2024-09-14T22:37:29.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826247+0000", 2024-09-14T22:37:29.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7440000000000007e-06 2024-09-14T22:37:29.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826275+0000", 2024-09-14T22:37:29.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8053000000000001e-05 2024-09-14T22:37:29.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826286+0000", 2024-09-14T22:37:29.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1780000000000001e-05 2024-09-14T22:37:29.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881137+0000", 2024-09-14T22:37:29.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054850500000000003 2024-09-14T22:37:29.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881181+0000", 2024-09-14T22:37:29.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4153000000000002e-05 2024-09-14T22:37:29.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881227+0000", 2024-09-14T22:37:29.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6427000000000001e-05 2024-09-14T22:37:29.342 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.342 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.342 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.342 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112045 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 7053312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:29.342 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:37:29.342 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.4264753210000001, 2024-09-14T22:37:29.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055045050999999998, 2024-09-14T22:37:29.342 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.343 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.343 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.343 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.343 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.343 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.343 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112045 2024-09-14T22:37:29.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.343 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:37:29.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:37:29.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830250+0000", 2024-09-14T22:37:29.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0039258670000000004 2024-09-14T22:37:29.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830260+0000", 2024-09-14T22:37:29.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0088000000000001e-05 2024-09-14T22:37:29.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830281+0000", 2024-09-14T22:37:29.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0720000000000002e-05 2024-09-14T22:37:29.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830292+0000", 2024-09-14T22:37:29.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1455e-05 2024-09-14T22:37:29.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881255+0000", 2024-09-14T22:37:29.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.050962247000000002 2024-09-14T22:37:29.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881296+0000", 2024-09-14T22:37:29.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1557999999999998e-05 2024-09-14T22:37:29.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881369+0000", 2024-09-14T22:37:29.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3115999999999996e-05 2024-09-14T22:37:29.346 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.346 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.346 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.347 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.347 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.347 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:29.347 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-14T22:37:29.348 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:37:29.349 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:29.347+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:29.449 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:29.447+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:29.474 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:29.474 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:29.475 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:37:29.475 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:29.488 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:37:29.550 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:29.543+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:29.608 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:29.609 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:29.609 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:37:29.609 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:37:29.609 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:29.623 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:37:29.643 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:37:29.689 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:29.687+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:29.751 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-14T22:37:29.751 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-14T22:37:29.752 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2024-09-14T22:37:29.752 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:29.747+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:29.794 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:29.794 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:29.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:29.794 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:29.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.794 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97197 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 796672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:29.794 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:37:29.794 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.718908502000001, 2024-09-14T22:37:29.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12434473, 2024-09-14T22:37:29.794 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.795 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.795 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.795 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.795 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.795 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.795 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97197 2024-09-14T22:37:29.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.795 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:37:29.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:37:29.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070524+0000", 2024-09-14T22:37:29.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0299999999999998e-07 2024-09-14T22:37:29.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070527+0000", 2024-09-14T22:37:29.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9569999999999998e-06 2024-09-14T22:37:29.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070535+0000", 2024-09-14T22:37:29.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7630000000000006e-06 2024-09-14T22:37:29.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070543+0000", 2024-09-14T22:37:29.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4869999999999996e-06 2024-09-14T22:37:29.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194525+0000", 2024-09-14T22:37:29.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123982096 2024-09-14T22:37:29.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194699+0000", 2024-09-14T22:37:29.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000174105 2024-09-14T22:37:29.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194868+0000", 2024-09-14T22:37:29.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016861900000000001 2024-09-14T22:37:29.798 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.799 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.799 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.799 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97198 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 805888~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:29.799 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:37:29.799 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.718880125999998, 2024-09-14T22:37:29.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.1248088, 2024-09-14T22:37:29.799 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.799 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.799 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.799 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.799 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.800 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.800 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97198 2024-09-14T22:37:29.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.800 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:37:29.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:37:29.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070552+0000", 2024-09-14T22:37:29.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T22:37:29.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070555+0000", 2024-09-14T22:37:29.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.853e-06 2024-09-14T22:37:29.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070565+0000", 2024-09-14T22:37:29.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.048e-05 2024-09-14T22:37:29.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070573+0000", 2024-09-14T22:37:29.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7200000000000006e-06 2024-09-14T22:37:29.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194954+0000", 2024-09-14T22:37:29.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12438089400000001 2024-09-14T22:37:29.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195145+0000", 2024-09-14T22:37:29.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019091299999999999 2024-09-14T22:37:29.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195360+0000", 2024-09-14T22:37:29.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021527300000000001 2024-09-14T22:37:29.803 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.803 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.803 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.803 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97199 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 815104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:29.804 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:37:29.804 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.718849529, 2024-09-14T22:37:29.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125251901, 2024-09-14T22:37:29.804 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.804 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.804 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.804 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.804 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.804 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.804 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97199 2024-09-14T22:37:29.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.804 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:37:29.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:37:29.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070583+0000", 2024-09-14T22:37:29.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6300000000000005e-07 2024-09-14T22:37:29.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070586+0000", 2024-09-14T22:37:29.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8270000000000002e-06 2024-09-14T22:37:29.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070597+0000", 2024-09-14T22:37:29.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1960000000000001e-05 2024-09-14T22:37:29.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070605+0000", 2024-09-14T22:37:29.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6459999999999993e-06 2024-09-14T22:37:29.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195432+0000", 2024-09-14T22:37:29.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124826926 2024-09-14T22:37:29.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195626+0000", 2024-09-14T22:37:29.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019357099999999999 2024-09-14T22:37:29.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195834+0000", 2024-09-14T22:37:29.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020830800000000001 2024-09-14T22:37:29.808 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.808 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.808 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.808 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97200 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 824320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:29.808 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:37:29.808 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.718817482999999, 2024-09-14T22:37:29.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125693477, 2024-09-14T22:37:29.808 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.808 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.808 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.808 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.809 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.809 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.809 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97200 2024-09-14T22:37:29.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.809 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:37:29.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:37:29.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070615+0000", 2024-09-14T22:37:29.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T22:37:29.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070618+0000", 2024-09-14T22:37:29.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.943e-06 2024-09-14T22:37:29.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070627+0000", 2024-09-14T22:37:29.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1099999999999992e-06 2024-09-14T22:37:29.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070634+0000", 2024-09-14T22:37:29.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:37:29.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195908+0000", 2024-09-14T22:37:29.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125273893 2024-09-14T22:37:29.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196097+0000", 2024-09-14T22:37:29.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018953899999999999 2024-09-14T22:37:29.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196308+0000", 2024-09-14T22:37:29.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021027799999999999 2024-09-14T22:37:29.812 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.812 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.812 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.812 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97201 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 833536~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:29.813 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:37:29.813 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.718787295999999, 2024-09-14T22:37:29.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126133826, 2024-09-14T22:37:29.813 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.813 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.813 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.813 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.813 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.813 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.813 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97201 2024-09-14T22:37:29.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.813 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:37:29.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:37:29.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074109+0000", 2024-09-14T22:37:29.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003464834 2024-09-14T22:37:29.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074134+0000", 2024-09-14T22:37:29.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5170000000000001e-05 2024-09-14T22:37:29.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074166+0000", 2024-09-14T22:37:29.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1347000000000003e-05 2024-09-14T22:37:29.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074179+0000", 2024-09-14T22:37:29.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.364e-05 2024-09-14T22:37:29.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196377+0000", 2024-09-14T22:37:29.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12219793900000001 2024-09-14T22:37:29.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196568+0000", 2024-09-14T22:37:29.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019097999999999999 2024-09-14T22:37:29.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196778+0000", 2024-09-14T22:37:29.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020991599999999999 2024-09-14T22:37:29.817 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.817 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.817 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.817 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97205 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 870400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:29.817 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:37:29.817 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.715117733, 2024-09-14T22:37:29.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12436148299999999, 2024-09-14T22:37:29.817 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.817 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.817 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.818 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.818 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.818 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.818 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97205 2024-09-14T22:37:29.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.818 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:37:29.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:37:29.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.819 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.819 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074315+0000", 2024-09-14T22:37:29.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9299999999999997e-07 2024-09-14T22:37:29.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.819 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.819 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074318+0000", 2024-09-14T22:37:29.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.19e-06 2024-09-14T22:37:29.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.819 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074327+0000", 2024-09-14T22:37:29.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8529999999999998e-06 2024-09-14T22:37:29.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074334+0000", 2024-09-14T22:37:29.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3869999999999997e-06 2024-09-14T22:37:29.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198365+0000", 2024-09-14T22:37:29.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124031222 2024-09-14T22:37:29.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198513+0000", 2024-09-14T22:37:29.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014772299999999999 2024-09-14T22:37:29.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198675+0000", 2024-09-14T22:37:29.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016241500000000001 2024-09-14T22:37:29.821 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.821 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.821 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.821 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97206 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 879616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:29.822 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:37:29.822 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.715086277000001, 2024-09-14T22:37:29.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124706284, 2024-09-14T22:37:29.822 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.822 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.822 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.822 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.822 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.822 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.822 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97206 2024-09-14T22:37:29.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.822 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:37:29.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:37:29.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074346+0000", 2024-09-14T22:37:29.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8999999999999996e-07 2024-09-14T22:37:29.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.824 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.824 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074349+0000", 2024-09-14T22:37:29.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.96e-06 2024-09-14T22:37:29.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.824 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.824 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074358+0000", 2024-09-14T22:37:29.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9169999999999995e-06 2024-09-14T22:37:29.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.824 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.824 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074365+0000", 2024-09-14T22:37:29.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3699999999999997e-06 2024-09-14T22:37:29.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.825 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.825 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198761+0000", 2024-09-14T22:37:29.825 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124395743 2024-09-14T22:37:29.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.825 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.825 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198899+0000", 2024-09-14T22:37:29.825 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013763499999999999 2024-09-14T22:37:29.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.825 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.825 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199052+0000", 2024-09-14T22:37:29.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015296900000000001 2024-09-14T22:37:29.826 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.826 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.826 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.826 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97207 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 888832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:29.826 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:37:29.826 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.715057064, 2024-09-14T22:37:29.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124988433, 2024-09-14T22:37:29.826 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.826 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.826 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.826 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.827 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.827 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.827 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97207 2024-09-14T22:37:29.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.827 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:37:29.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:37:29.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074375+0000", 2024-09-14T22:37:29.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:37:29.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074378+0000", 2024-09-14T22:37:29.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1829999999999998e-06 2024-09-14T22:37:29.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074389+0000", 2024-09-14T22:37:29.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0737000000000001e-05 2024-09-14T22:37:29.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074396+0000", 2024-09-14T22:37:29.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2559999999999996e-06 2024-09-14T22:37:29.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199139+0000", 2024-09-14T22:37:29.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12474253 2024-09-14T22:37:29.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199240+0000", 2024-09-14T22:37:29.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101221 2024-09-14T22:37:29.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199363+0000", 2024-09-14T22:37:29.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000122822 2024-09-14T22:37:29.830 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.830 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.830 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.830 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97208 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 898048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:29.831 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:37:29.831 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.71502576, 2024-09-14T22:37:29.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12524854899999999, 2024-09-14T22:37:29.831 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.831 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.831 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.831 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.831 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.831 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.831 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97208 2024-09-14T22:37:29.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.831 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.832 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:37:29.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.832 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.832 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:37:29.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.832 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.832 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:37:29.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:37:29.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074410+0000", 2024-09-14T22:37:29.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.18e-06 2024-09-14T22:37:29.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074435+0000", 2024-09-14T22:37:29.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5273e-05 2024-09-14T22:37:29.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074443+0000", 2024-09-14T22:37:29.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3000000000000002e-06 2024-09-14T22:37:29.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.834 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199433+0000", 2024-09-14T22:37:29.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124989502 2024-09-14T22:37:29.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.834 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199535+0000", 2024-09-14T22:37:29.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102557 2024-09-14T22:37:29.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.834 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199654+0000", 2024-09-14T22:37:29.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119061 2024-09-14T22:37:29.835 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.835 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.835 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.835 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97209 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 907264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:29.835 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:37:29.835 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.714977954999998, 2024-09-14T22:37:29.835 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125487354, 2024-09-14T22:37:29.835 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.835 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.835 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.835 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.835 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.835 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.836 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97209 2024-09-14T22:37:29.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.836 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:37:29.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:37:29.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.837 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.837 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:37:29.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9400000000000005e-07 2024-09-14T22:37:29.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.837 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.837 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074458+0000", 2024-09-14T22:37:29.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3299999999999999e-06 2024-09-14T22:37:29.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.837 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.837 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074467+0000", 2024-09-14T22:37:29.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.346e-06 2024-09-14T22:37:29.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074474+0000", 2024-09-14T22:37:29.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.374e-06 2024-09-14T22:37:29.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199720+0000", 2024-09-14T22:37:29.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12524561200000001 2024-09-14T22:37:29.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199820+0000", 2024-09-14T22:37:29.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010049599999999999 2024-09-14T22:37:29.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199941+0000", 2024-09-14T22:37:29.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012050200000000001 2024-09-14T22:37:29.839 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.839 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.839 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.839 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97210 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:29.839 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:37:29.839 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.714947961, 2024-09-14T22:37:29.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125744942, 2024-09-14T22:37:29.840 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.840 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.840 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.840 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.840 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.840 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.840 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97210 2024-09-14T22:37:29.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.840 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:37:29.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:37:29.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:37:29.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6599999999999996e-07 2024-09-14T22:37:29.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074487+0000", 2024-09-14T22:37:29.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8140000000000001e-06 2024-09-14T22:37:29.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074496+0000", 2024-09-14T22:37:29.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9330000000000007e-06 2024-09-14T22:37:29.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074503+0000", 2024-09-14T22:37:29.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4100000000000002e-06 2024-09-14T22:37:29.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200009+0000", 2024-09-14T22:37:29.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12550539899999999 2024-09-14T22:37:29.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200108+0000", 2024-09-14T22:37:29.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9182e-05 2024-09-14T22:37:29.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200229+0000", 2024-09-14T22:37:29.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120538 2024-09-14T22:37:29.843 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.843 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.843 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.844 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97211 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 925696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:29.844 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:37:29.844 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.714917001, 2024-09-14T22:37:29.844 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12599848299999999, 2024-09-14T22:37:29.844 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.844 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.844 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.844 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.844 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.844 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.844 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97211 2024-09-14T22:37:29.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.844 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:37:29.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:37:29.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074516+0000", 2024-09-14T22:37:29.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.42e-06 2024-09-14T22:37:29.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074521+0000", 2024-09-14T22:37:29.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7960000000000003e-06 2024-09-14T22:37:29.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074538+0000", 2024-09-14T22:37:29.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6900000000000001e-05 2024-09-14T22:37:29.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074555+0000", 2024-09-14T22:37:29.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.696e-05 2024-09-14T22:37:29.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200293+0000", 2024-09-14T22:37:29.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12573864100000001 2024-09-14T22:37:29.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200393+0000", 2024-09-14T22:37:29.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9264000000000004e-05 2024-09-14T22:37:29.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200513+0000", 2024-09-14T22:37:29.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012050200000000001 2024-09-14T22:37:29.848 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.848 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.848 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.848 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97212 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 934912~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:29.848 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:37:29.848 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.714842379, 2024-09-14T22:37:29.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12621127300000001, 2024-09-14T22:37:29.848 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.848 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.848 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.849 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.849 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.849 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.849 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97212 2024-09-14T22:37:29.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.849 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:37:29.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:37:29.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.850 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074590+0000", 2024-09-14T22:37:29.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1969999999999999e-06 2024-09-14T22:37:29.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.850 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074596+0000", 2024-09-14T22:37:29.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7629999999999997e-06 2024-09-14T22:37:29.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.850 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074614+0000", 2024-09-14T22:37:29.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7643e-05 2024-09-14T22:37:29.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074635+0000", 2024-09-14T22:37:29.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.075e-05 2024-09-14T22:37:29.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200580+0000", 2024-09-14T22:37:29.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125945482 2024-09-14T22:37:29.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200683+0000", 2024-09-14T22:37:29.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010267099999999999 2024-09-14T22:37:29.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.852 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200800+0000", 2024-09-14T22:37:29.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117767 2024-09-14T22:37:29.852 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.852 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.852 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.852 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97213 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 944128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:29.852 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:37:29.853 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.714777029, 2024-09-14T22:37:29.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126430709, 2024-09-14T22:37:29.853 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.853 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.853 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.853 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.853 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.853 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.853 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97213 2024-09-14T22:37:29.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.853 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:37:29.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:37:29.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078210+0000", 2024-09-14T22:37:29.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0035556839999999999 2024-09-14T22:37:29.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078231+0000", 2024-09-14T22:37:29.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0716e-05 2024-09-14T22:37:29.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078260+0000", 2024-09-14T22:37:29.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9473000000000001e-05 2024-09-14T22:37:29.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078279+0000", 2024-09-14T22:37:29.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8600000000000001e-05 2024-09-14T22:37:29.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.856 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200864+0000", 2024-09-14T22:37:29.856 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.122585295 2024-09-14T22:37:29.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.856 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.856 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200965+0000", 2024-09-14T22:37:29.856 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101004 2024-09-14T22:37:29.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.856 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.856 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.201085+0000", 2024-09-14T22:37:29.856 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119937 2024-09-14T22:37:29.856 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.856 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.857 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.857 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97219 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 999424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:29.857 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:37:29.857 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.710794751999998, 2024-09-14T22:37:29.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124138558, 2024-09-14T22:37:29.857 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.857 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.857 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.857 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.857 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.857 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.858 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97219 2024-09-14T22:37:29.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.858 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:37:29.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:37:29.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078638+0000", 2024-09-14T22:37:29.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.173e-06 2024-09-14T22:37:29.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.859 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078644+0000", 2024-09-14T22:37:29.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2570000000000004e-06 2024-09-14T22:37:29.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.859 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078672+0000", 2024-09-14T22:37:29.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7322999999999999e-05 2024-09-14T22:37:29.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078687+0000", 2024-09-14T22:37:29.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5639999999999999e-05 2024-09-14T22:37:29.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202567+0000", 2024-09-14T22:37:29.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123880134 2024-09-14T22:37:29.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202670+0000", 2024-09-14T22:37:29.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010273899999999999 2024-09-14T22:37:29.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202775+0000", 2024-09-14T22:37:29.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010529199999999999 2024-09-14T22:37:29.861 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.861 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.861 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.861 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97220 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1008640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:29.861 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:37:29.861 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.710725519, 2024-09-14T22:37:29.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12435937, 2024-09-14T22:37:29.862 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.862 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.862 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.862 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.862 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.862 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.862 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97220 2024-09-14T22:37:29.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.862 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:37:29.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:37:29.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078707+0000", 2024-09-14T22:37:29.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.266e-06 2024-09-14T22:37:29.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078713+0000", 2024-09-14T22:37:29.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7270000000000004e-06 2024-09-14T22:37:29.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.864 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078737+0000", 2024-09-14T22:37:29.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3640000000000001e-05 2024-09-14T22:37:29.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.864 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078751+0000", 2024-09-14T22:37:29.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.45e-05 2024-09-14T22:37:29.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.864 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202841+0000", 2024-09-14T22:37:29.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124089938 2024-09-14T22:37:29.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.865 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.865 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202944+0000", 2024-09-14T22:37:29.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010334 2024-09-14T22:37:29.865 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.865 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.865 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203065+0000", 2024-09-14T22:37:29.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012095900000000001 2024-09-14T22:37:29.865 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.865 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.865 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.865 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.866 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97221 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1017856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:29.866 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:37:29.866 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.710659755999998, 2024-09-14T22:37:29.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124574005, 2024-09-14T22:37:29.866 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.866 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.866 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.866 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.866 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.866 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.866 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97221 2024-09-14T22:37:29.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.866 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:37:29.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:37:29.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078773+0000", 2024-09-14T22:37:29.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.13e-06 2024-09-14T22:37:29.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078779+0000", 2024-09-14T22:37:29.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0260000000000004e-06 2024-09-14T22:37:29.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078797+0000", 2024-09-14T22:37:29.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.806e-05 2024-09-14T22:37:29.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078810+0000", 2024-09-14T22:37:29.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.308e-05 2024-09-14T22:37:29.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.869 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.869 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203131+0000", 2024-09-14T22:37:29.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12432077499999999 2024-09-14T22:37:29.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.869 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.869 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203227+0000", 2024-09-14T22:37:29.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6575000000000003e-05 2024-09-14T22:37:29.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.870 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.870 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203346+0000", 2024-09-14T22:37:29.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118359 2024-09-14T22:37:29.870 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.870 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.870 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.870 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.870 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97222 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:29.870 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:37:29.870 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.710603386999999, 2024-09-14T22:37:29.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12480110799999999, 2024-09-14T22:37:29.870 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.870 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.870 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.871 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.871 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.871 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.871 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97222 2024-09-14T22:37:29.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.871 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:37:29.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:37:29.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078829+0000", 2024-09-14T22:37:29.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.057e-06 2024-09-14T22:37:29.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078836+0000", 2024-09-14T22:37:29.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6159999999999998e-06 2024-09-14T22:37:29.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078856+0000", 2024-09-14T22:37:29.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0100000000000001e-05 2024-09-14T22:37:29.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078869+0000", 2024-09-14T22:37:29.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.272e-05 2024-09-14T22:37:29.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203412+0000", 2024-09-14T22:37:29.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12454283300000001 2024-09-14T22:37:29.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.874 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.874 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203511+0000", 2024-09-14T22:37:29.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9893000000000001e-05 2024-09-14T22:37:29.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.874 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.874 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203629+0000", 2024-09-14T22:37:29.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117889 2024-09-14T22:37:29.874 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.874 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.874 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.874 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97223 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1036288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:29.875 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:37:29.875 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.710231400000001, 2024-09-14T22:37:29.875 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124706279, 2024-09-14T22:37:29.875 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.875 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.875 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.875 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.875 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.875 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.875 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97223 2024-09-14T22:37:29.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.875 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.875 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.875 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:37:29.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:37:29.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079202+0000", 2024-09-14T22:37:29.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8330000000000001e-06 2024-09-14T22:37:29.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079212+0000", 2024-09-14T22:37:29.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0207000000000001e-05 2024-09-14T22:37:29.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079238+0000", 2024-09-14T22:37:29.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5626000000000002e-05 2024-09-14T22:37:29.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079256+0000", 2024-09-14T22:37:29.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8386999999999999e-05 2024-09-14T22:37:29.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203691+0000", 2024-09-14T22:37:29.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124434538 2024-09-14T22:37:29.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203786+0000", 2024-09-14T22:37:29.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5344000000000001e-05 2024-09-14T22:37:29.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203906+0000", 2024-09-14T22:37:29.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120344 2024-09-14T22:37:29.878 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.878 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.878 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.879 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97224 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1045504~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:29.879 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:37:29.879 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.710147580000001, 2024-09-14T22:37:29.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12513321199999999, 2024-09-14T22:37:29.879 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:29.879 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:29.879 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:29.879 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:29.879 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:29.879 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:29.879 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97224 2024-09-14T22:37:29.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.880 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:29.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:29.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:37:29.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:29.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:37:29.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:29.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:29.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082581+0000", 2024-09-14T22:37:29.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0032971530000000001 2024-09-14T22:37:29.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:29.881 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082592+0000", 2024-09-14T22:37:29.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1345999999999999e-05 2024-09-14T22:37:29.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:29.881 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082618+0000", 2024-09-14T22:37:29.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5873000000000001e-05 2024-09-14T22:37:29.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:29.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082638+0000", 2024-09-14T22:37:29.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9565999999999999e-05 2024-09-14T22:37:29.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:29.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204004+0000", 2024-09-14T22:37:29.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.121366038 2024-09-14T22:37:29.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:29.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204188+0000", 2024-09-14T22:37:29.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018427 2024-09-14T22:37:29.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:29.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:29.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:29.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204417+0000", 2024-09-14T22:37:29.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022896599999999999 2024-09-14T22:37:29.883 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.883 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.883 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.883 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:29.883 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:29.883 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:29.884 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:37:29.884 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:37:29.885 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:29.885 INFO:teuthology.orchestra.run.smithi001.stdout:3905 punch from 0x33368bd to 0x33459b2, (0xf0f5 bytes) 2024-09-14T22:37:29.885 INFO:teuthology.orchestra.run.smithi001.stdout:3906 read 0x245a756 thru 0x246075a (0x6005 bytes) 2024-09-14T22:37:29.886 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:37:29.886 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:37:29.886 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:37:29.887 INFO:teuthology.orchestra.run.smithi001.stdout:3907 read 0xd9cf84 thru 0xd9fc88 (0x2d05 bytes) 2024-09-14T22:37:29.887 INFO:teuthology.orchestra.run.smithi001.stdout:3908 read 0x1e7ee00 thru 0x1e8b120 (0xc321 bytes) 2024-09-14T22:37:29.887 INFO:teuthology.orchestra.run.smithi001.stdout:3909 read 0x69f2cd0 thru 0x6a00a1f (0xdd50 bytes) 2024-09-14T22:37:29.887 INFO:teuthology.orchestra.run.smithi001.stdout:3910 write 0x76c77db thru 0x76d14a5 (0x9ccb bytes) 2024-09-14T22:37:29.887 INFO:teuthology.orchestra.run.smithi001.stdout:3911 read 0x8f4ee8a thru 0x8f5ea4b (0xfbc2 bytes) 2024-09-14T22:37:29.888 INFO:teuthology.orchestra.run.smithi001.stdout:3912 trunc from 0x9e0551a to 0xad8a2f6 2024-09-14T22:37:29.888 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:37:29.888 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:37:29.889 INFO:teuthology.orchestra.run.smithi001.stdout:3913 read 0x8ba8751 thru 0x8baff40 (0x77f0 bytes) 2024-09-14T22:37:29.889 INFO:teuthology.orchestra.run.smithi001.stdout:3914 write 0x673c3c2 thru 0x674943d (0xd07c bytes) 2024-09-14T22:37:29.889 INFO:teuthology.orchestra.run.smithi001.stdout:3915 punch from 0x8ef1edd to 0x8f018c9, (0xf9ec bytes) 2024-09-14T22:37:29.889 INFO:teuthology.orchestra.run.smithi001.stdout:3916 trunc from 0xad8a2f6 to 0x378c7f2 2024-09-14T22:37:29.889 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:37:29.889 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:37:29.890 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:37:29.890 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:37:29.890 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:29.883+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:29.896 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:37:29.896 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:37:29.897 INFO:teuthology.orchestra.run.smithi001.stdout:3917 write 0x5e439ac thru 0x5e477bb (0x3e10 bytes) 2024-09-14T22:37:29.901 INFO:teuthology.orchestra.run.smithi001.stdout:3918 read 0x57bef7f thru 0x57befcf (0x51 bytes) 2024-09-14T22:37:29.901 INFO:teuthology.orchestra.run.smithi001.stdout:3919 write 0x3efd563 thru 0x3f0ca6e (0xf50c bytes) 2024-09-14T22:37:29.903 INFO:teuthology.orchestra.run.smithi001.stdout:3920 punch from 0x190bb30 to 0x190f232, (0x3702 bytes) 2024-09-14T22:37:29.915 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:37:29.915 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:37:29.919 INFO:teuthology.orchestra.run.smithi001.stdout:3921 write 0xc2b0705 thru 0xc2b972e (0x902a bytes) 2024-09-14T22:37:29.923 INFO:teuthology.orchestra.run.smithi001.stdout:3922 read 0xdcd504 thru 0xddd465 (0xff62 bytes) 2024-09-14T22:37:29.925 INFO:teuthology.orchestra.run.smithi001.stdout:3923 write 0xaeff21f thru 0xaf0ba63 (0xc845 bytes) 2024-09-14T22:37:29.930 INFO:teuthology.orchestra.run.smithi001.stdout:3924 punch from 0xa0918ac to 0xa09261b, (0xd6f bytes) 2024-09-14T22:37:29.930 INFO:teuthology.orchestra.run.smithi001.stdout:3925 read 0x61d3b0f thru 0x61dfe67 (0xc359 bytes) 2024-09-14T22:37:29.930 INFO:teuthology.orchestra.run.smithi001.stdout:3926 punch from 0xa0a9129 to 0xa0b0303, (0x71da bytes) 2024-09-14T22:37:29.931 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:29.931 INFO:teuthology.orchestra.run.smithi001.stdout:3929 read 0x52221f6 thru 0x522bba1 (0x99ac bytes) 2024-09-14T22:37:29.934 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:37:29.934 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:37:29.939 INFO:teuthology.orchestra.run.smithi001.stdout:3930 writesame 0xce301f3 thru 0xce33822 data_size 0x22(0x3630 bytes) 2024-09-14T22:37:29.939 INFO:teuthology.orchestra.run.smithi001.stdout:3932 write 0x78d5dfb thru 0x78e1226 (0xb42c bytes) 2024-09-14T22:37:29.943 INFO:teuthology.orchestra.run.smithi001.stdout:3935 read 0xc0a77f3 thru 0xc0a79e9 (0x1f7 bytes) 2024-09-14T22:37:29.943 INFO:teuthology.orchestra.run.smithi001.stdout:3936 writesame 0x61e0c72 thru 0x61e2081 data_size 0x18(0x1410 bytes) 2024-09-14T22:37:29.947 INFO:teuthology.orchestra.run.smithi001.stdout:3937 write 0x26e7669 thru 0x26f4e14 (0xd7ac bytes) 2024-09-14T22:37:29.965 INFO:teuthology.orchestra.run.smithi001.stdout:3938 write 0x3953143 thru 0x395885e (0x571c bytes) 2024-09-14T22:37:29.968 INFO:teuthology.orchestra.run.smithi001.stdout:3939 punch from 0x390b4c5 to 0x390cecd, (0x1a08 bytes) 2024-09-14T22:37:29.968 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:29.968 INFO:teuthology.orchestra.run.smithi001.stdout:3942 write 0xb57266d thru 0xb5745d1 (0x1f65 bytes) 2024-09-14T22:37:29.970 INFO:teuthology.orchestra.run.smithi001.stdout:3943 read 0xb9ffffd thru 0xba0413f (0x4143 bytes) 2024-09-14T22:37:29.970 INFO:teuthology.orchestra.run.smithi001.stdout:3944 read 0x5b9a0be thru 0x5b9fa24 (0x5967 bytes) 2024-09-14T22:37:29.970 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:29.970 INFO:teuthology.orchestra.run.smithi001.stdout:3947 write 0xab28944 thru 0xab37bd7 (0xf294 bytes) 2024-09-14T22:37:29.974 INFO:teuthology.orchestra.run.smithi001.stdout:3948 write 0x861baee thru 0x861db6b (0x207e bytes) 2024-09-14T22:37:29.978 INFO:teuthology.orchestra.run.smithi001.stdout:3949 write 0xb382a49 thru 0xb38f91a (0xced2 bytes) 2024-09-14T22:37:29.982 INFO:teuthology.orchestra.run.smithi001.stdout:3950 write 0x98742a2 thru 0x98791d7 (0x4f36 bytes) 2024-09-14T22:37:29.985 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:29.979+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:29.986 INFO:teuthology.orchestra.run.smithi001.stdout:3952 write 0x5c59c0c thru 0x5c64074 (0xa469 bytes) 2024-09-14T22:37:29.999 INFO:teuthology.orchestra.run.smithi001.stdout:3954 write 0xdb34b34 thru 0xdb3cce3 (0x81b0 bytes) 2024-09-14T22:37:30.004 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:30.004 INFO:teuthology.orchestra.run.smithi001.stdout:3956 writesame 0x413a3dc thru 0x4145cbd data_size 0x5(0xb8e2 bytes) 2024-09-14T22:37:30.012 INFO:teuthology.orchestra.run.smithi001.stdout:3957 write 0x51e9211 thru 0x51f5614 (0xc404 bytes) 2024-09-14T22:37:30.016 INFO:teuthology.orchestra.run.smithi001.stdout:3958 write 0xc7b46da thru 0xc7c1ac5 (0xd3ec bytes) 2024-09-14T22:37:30.020 INFO:teuthology.orchestra.run.smithi001.stdout:3959 punch from 0xa0ed8bb to 0xa0f8018, (0xa75d bytes) 2024-09-14T22:37:30.020 INFO:teuthology.orchestra.run.smithi001.stdout:3960 write 0x7e7659d thru 0x7e7681e (0x282 bytes) 2024-09-14T22:37:30.029 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:30.029 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:30.034 INFO:teuthology.orchestra.run.smithi001.stdout:3961 trunc from 0xdb3cce4 to 0x61c0432 2024-09-14T22:37:30.044 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:37:30.051 INFO:teuthology.orchestra.run.smithi001.stdout:3962 punch from 0x29be8bd to 0x29ce2da, (0xfa1d bytes) 2024-09-14T22:37:30.051 INFO:teuthology.orchestra.run.smithi001.stdout:3963 punch from 0x247a81f to 0x247bc08, (0x13e9 bytes) 2024-09-14T22:37:30.065 INFO:teuthology.orchestra.run.smithi001.stdout:3964 write 0xa36540c thru 0xa3680d7 (0x2ccc bytes) 2024-09-14T22:37:30.080 INFO:teuthology.orchestra.run.smithi001.stdout:3965 trunc from 0xa3680d8 to 0x2364d70 2024-09-14T22:37:30.087 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:30.083+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:30.160 INFO:teuthology.orchestra.run.smithi001.stdout:3966 trunc from 0x2364d70 to 0x625a696 2024-09-14T22:37:30.178 INFO:teuthology.orchestra.run.smithi001.stdout:3967 trunc from 0x625a696 to 0xaa27236 2024-09-14T22:37:30.187 INFO:teuthology.orchestra.run.smithi001.stdout:3968 punch from 0xa6f5451 to 0xa70143e, (0xbfed bytes) 2024-09-14T22:37:30.187 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:30.188 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:30.183+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:30.204 INFO:teuthology.orchestra.run.smithi001.stdout:3970 write 0xd02090e thru 0xd02357a (0x2c6d bytes) 2024-09-14T22:37:30.211 INFO:teuthology.orchestra.run.smithi001.stdout:3972 write 0x887b3ba thru 0x88819b7 (0x65fe bytes) 2024-09-14T22:37:30.226 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:30.226 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:30.231 INFO:teuthology.orchestra.run.smithi001.stdout:3973 trunc from 0xd02357b to 0x8739d69 2024-09-14T22:37:30.241 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:37:30.254 INFO:teuthology.orchestra.run.smithi001.stdout:3974 read 0x706f9b7 thru 0x707d951 (0xdf9b bytes) 2024-09-14T22:37:30.269 INFO:teuthology.orchestra.run.smithi001.stdout:3975 trunc from 0x8739d69 to 0x2ab7dc0 2024-09-14T22:37:30.288 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:30.287+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:30.290 INFO:teuthology.orchestra.run.smithi001.stdout:3976 flatten 2024-09-14T22:37:30.389 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:30.389 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:30.390 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:30.387+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:30.404 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:37:30.446 INFO:teuthology.orchestra.run.smithi001.stdout:3977 write 0xd443df2 thru 0xd44c3e7 (0x85f6 bytes) 2024-09-14T22:37:30.453 INFO:teuthology.orchestra.run.smithi001.stdout:3978 write 0xd72678 thru 0xd7c123 (0x9aac bytes) 2024-09-14T22:37:30.457 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:30.464 INFO:teuthology.orchestra.run.smithi001.stdout:3980 trunc from 0xd44c3e8 to 0xa9dad3c 2024-09-14T22:37:30.482 INFO:teuthology.orchestra.run.smithi001.stdout:3981 punch from 0x40bcd6d to 0x40beb00, (0x1d93 bytes) 2024-09-14T22:37:30.482 INFO:teuthology.orchestra.run.smithi001.stdout:3982 write 0x7aa16aa thru 0x7aa5c44 (0x459b bytes) 2024-09-14T22:37:30.488 INFO:teuthology.orchestra.run.smithi001.stdout:3984 read 0x797b24f thru 0x7984ebd (0x9c6f bytes) 2024-09-14T22:37:30.488 INFO:teuthology.orchestra.run.smithi001.stdout:3985 read 0x129d05 thru 0x12ec24 (0x4f20 bytes) 2024-09-14T22:37:30.490 INFO:teuthology.orchestra.run.smithi001.stdout:3986 read 0x1f05a31 thru 0x1f084e4 (0x2ab4 bytes) 2024-09-14T22:37:30.490 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:30.487+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:30.491 INFO:teuthology.orchestra.run.smithi001.stdout:3987 writesame 0x29fb49b thru 0x29ff04e data_size 0x2(0x3bb4 bytes) 2024-09-14T22:37:30.496 INFO:teuthology.orchestra.run.smithi001.stdout:3989 writesame 0xef3f00 thru 0xef713a data_size 0xb(0x323b bytes) 2024-09-14T22:37:30.500 INFO:teuthology.orchestra.run.smithi001.stdout:3990 write 0x4c493f thru 0x4d1e8d (0xd54f bytes) 2024-09-14T22:37:30.503 INFO:teuthology.orchestra.run.smithi001.stdout:3991 read 0x5be4502 thru 0x5beff89 (0xba88 bytes) 2024-09-14T22:37:30.503 INFO:teuthology.orchestra.run.smithi001.stdout:3992 punch from 0x53e4e81 to 0x53f2c52, (0xddd1 bytes) 2024-09-14T22:37:30.503 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:30.503 INFO:teuthology.orchestra.run.smithi001.stdout:3994 read 0xa83ed13 thru 0xa84d3b9 (0xe6a7 bytes) 2024-09-14T22:37:30.524 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:30.524 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:30.529 INFO:teuthology.orchestra.run.smithi001.stdout:3995 trunc from 0xa9dad3c to 0xda89d5 2024-09-14T22:37:30.539 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:37:30.575 INFO:teuthology.orchestra.run.smithi001.stdout:3996 write 0x412f633 thru 0x413bdb8 (0xc786 bytes) 2024-09-14T22:37:30.580 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:30.580 INFO:teuthology.orchestra.run.smithi001.stdout:3999 punch from 0x8bc7aa to 0x8bf17f, (0x29d5 bytes) 2024-09-14T22:37:30.591 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:30.587+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:30.620 INFO:teuthology.orchestra.run.smithi001.stdout:4001 write 0xe607561 thru 0xe6169ca (0xf46a bytes) 2024-09-14T22:37:30.665 INFO:teuthology.orchestra.run.smithi001.stdout:4005 trunc from 0xe6169cb to 0x5aaf6 2024-09-14T22:37:30.669 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:30.669 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:30.686 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:37:30.692 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:30.691+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:30.732 INFO:teuthology.orchestra.run.smithi001.stdout:4007 writesame 0x33f5861 thru 0x33f629c data_size 0xa(0xa3c bytes) 2024-09-14T22:37:30.732 INFO:teuthology.orchestra.run.smithi001.stdout:4008 read 0x32bca15 thru 0x32c4102 (0x76ee bytes) 2024-09-14T22:37:30.755 INFO:teuthology.orchestra.run.smithi001.stdout:4009 trunc from 0x33f629d to 0xbc659f3 2024-09-14T22:37:30.761 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.0 to 1 (10000) 2024-09-14T22:37:30.767 INFO:teuthology.orchestra.run.smithi001.stdout:4010 write 0x159263d thru 0x159b57d (0x8f41 bytes) 2024-09-14T22:37:30.775 INFO:teuthology.orchestra.run.smithi001.stdout:4013 read 0x7f40ae9 thru 0x7f4a532 (0x9a4a bytes) 2024-09-14T22:37:30.775 INFO:teuthology.orchestra.run.smithi001.stdout:4014 punch from 0x531a90a to 0x5325740, (0xae36 bytes) 2024-09-14T22:37:30.776 INFO:teuthology.orchestra.run.smithi001.stdout:4015 trunc from 0xbc659f3 to 0xc88eb79 2024-09-14T22:37:30.784 INFO:teuthology.orchestra.run.smithi001.stdout:4016 punch from 0x3958b47 to 0x396172a, (0x8be3 bytes) 2024-09-14T22:37:30.786 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-14T22:37:30.795 INFO:teuthology.orchestra.run.smithi001.stdout:4017 writesame 0xce3b6dc thru 0xce41b15 data_size 0x1(0x643a bytes) 2024-09-14T22:37:30.795 INFO:teuthology.orchestra.run.smithi001.stdout:4018 write 0x751135e thru 0x7517d97 (0x6a3a bytes) 2024-09-14T22:37:30.795 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:30.791+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:30.800 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:30.801 INFO:teuthology.orchestra.run.smithi001.stdout:4022 write 0x14f00a thru 0x155c0b (0x6c02 bytes) 2024-09-14T22:37:30.806 INFO:teuthology.orchestra.run.smithi001.stdout:4025 read 0x71b16b3 thru 0x71c06cf (0xf01d bytes) 2024-09-14T22:37:30.820 INFO:teuthology.orchestra.run.smithi001.stdout:4026 trunc from 0xce41b16 to 0x57f98b5 2024-09-14T22:37:30.841 INFO:teuthology.orchestra.run.smithi001.stdout:4027 read 0x2fe22ae thru 0x2feffc7 (0xdd1a bytes) 2024-09-14T22:37:30.841 INFO:teuthology.orchestra.run.smithi001.stdout:4028 write 0x5769d0c thru 0x5771980 (0x7c75 bytes) 2024-09-14T22:37:30.848 INFO:teuthology.orchestra.run.smithi001.stdout:4029 read 0x139e2dd thru 0x13aa068 (0xbd8c bytes) 2024-09-14T22:37:30.848 INFO:teuthology.orchestra.run.smithi001.stdout:4030 punch from 0x409e9d to 0x40b3c6, (0x1529 bytes) 2024-09-14T22:37:30.848 INFO:teuthology.orchestra.run.smithi001.stdout:4031 punch from 0x1dd03a3 to 0x1dd8274, (0x7ed1 bytes) 2024-09-14T22:37:30.856 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:30.856 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:30.869 INFO:teuthology.orchestra.run.smithi001.stdout:4032 writesame 0x93eb8a2 thru 0x93ed3c5 data_size 0x12(0x1b24 bytes) 2024-09-14T22:37:30.869 INFO:teuthology.orchestra.run.smithi001.stdout:4033 read 0x75f5e4d thru 0x75fc853 (0x6a07 bytes) 2024-09-14T22:37:30.869 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:30.869 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:30.871 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:37:30.891 INFO:teuthology.orchestra.run.smithi001.stdout:4036 writesame 0xedacaeb thru 0xedb1f4b data_size 0x1(0x5461 bytes) 2024-09-14T22:37:30.891 INFO:teuthology.orchestra.run.smithi001.stdout:4037 read 0xa21bb38 thru 0xa228d3b (0xd204 bytes) 2024-09-14T22:37:30.891 INFO:teuthology.orchestra.run.smithi001.stdout:4038 punch from 0x8732497 to 0x8739049, (0x6bb2 bytes) 2024-09-14T22:37:30.891 INFO:teuthology.orchestra.run.smithi001.stdout:4039 write 0xe68d5fb thru 0xe690f07 (0x390d bytes) 2024-09-14T22:37:30.894 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:30.887+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:30.896 INFO:teuthology.orchestra.run.smithi001.stdout:4040 writesame 0x60a77d3 thru 0x60b7134 data_size 0x89(0xf962 bytes) 2024-09-14T22:37:30.901 INFO:teuthology.orchestra.run.smithi001.stdout:4041 punch from 0xbeccc3d to 0xbed046d, (0x3830 bytes) 2024-09-14T22:37:30.901 INFO:teuthology.orchestra.run.smithi001.stdout:4042 read 0x9ce60ab thru 0x9cf0c21 (0xab77 bytes) 2024-09-14T22:37:30.902 INFO:teuthology.orchestra.run.smithi001.stdout:4044 write 0x5cfce1c thru 0x5d040f4 (0x72d9 bytes) 2024-09-14T22:37:30.906 INFO:teuthology.orchestra.run.smithi001.stdout:4045 writesame 0x3b2b7c4 thru 0x3b2e37c data_size 0x5b(0x2bb9 bytes) 2024-09-14T22:37:30.910 INFO:teuthology.orchestra.run.smithi001.stdout:4046 write 0xd510e91 thru 0xd51e478 (0xd5e8 bytes) 2024-09-14T22:37:30.915 INFO:teuthology.orchestra.run.smithi001.stdout:4048 write 0xe393c08 thru 0xe393fa6 (0x39f bytes) 2024-09-14T22:37:30.917 INFO:teuthology.orchestra.run.smithi001.stdout:4049 write 0xa083a8d thru 0xa092f75 (0xf4e9 bytes) 2024-09-14T22:37:30.922 INFO:teuthology.orchestra.run.smithi001.stdout:4050 write 0x74b9dbb thru 0x74c39e5 (0x9c2b bytes) 2024-09-14T22:37:30.938 INFO:teuthology.orchestra.run.smithi001.stdout:4051 trunc from 0xedb1f4c to 0x75a088e 2024-09-14T22:37:30.954 INFO:teuthology.orchestra.run.smithi001.stdout:4053 read 0x5baf9fc thru 0x5bb8b5e (0x9163 bytes) 2024-09-14T22:37:30.954 INFO:teuthology.orchestra.run.smithi001.stdout:4055 write 0x13017a7 thru 0x1302d97 (0x15f1 bytes) 2024-09-14T22:37:30.956 INFO:teuthology.orchestra.run.smithi001.stdout:4056 punch from 0x55a6312 to 0x55b1808, (0xb4f6 bytes) 2024-09-14T22:37:30.956 INFO:teuthology.orchestra.run.smithi001.stdout:4057 read 0x6327c84 thru 0x6332022 (0xa39f bytes) 2024-09-14T22:37:30.956 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:30.966 INFO:teuthology.orchestra.run.smithi001.stdout:4060 writesame 0x8240b85 thru 0x82509c6 data_size 0x73(0xfe42 bytes) 2024-09-14T22:37:30.966 INFO:teuthology.orchestra.run.smithi001.stdout:4062 read 0x81ba736 thru 0x81c7f76 (0xd841 bytes) 2024-09-14T22:37:30.980 INFO:teuthology.orchestra.run.smithi001.stdout:4063 write 0x9ce06f9 thru 0x9ceb4cd (0xadd5 bytes) 2024-09-14T22:37:30.985 INFO:teuthology.orchestra.run.smithi001.stdout:4064 read 0x65b81ab thru 0x65c58ee (0xd744 bytes) 2024-09-14T22:37:30.986 INFO:teuthology.orchestra.run.smithi001.stdout:4065 punch from 0x15b0120 to 0x15b437e, (0x425e bytes) 2024-09-14T22:37:30.995 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:30.991+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:30.997 INFO:teuthology.orchestra.run.smithi001.stdout:4067 writesame 0xa253ce8 thru 0xa26347c data_size 0x25(0xf795 bytes) 2024-09-14T22:37:30.997 INFO:teuthology.orchestra.run.smithi001.stdout:4069 write 0x7c20a2a thru 0x7c2fa50 (0xf027 bytes) 2024-09-14T22:37:31.003 INFO:teuthology.orchestra.run.smithi001.stdout:4070 read 0x15f4b73 thru 0x15fd9b6 (0x8e44 bytes) 2024-09-14T22:37:31.003 INFO:teuthology.orchestra.run.smithi001.stdout:4071 write 0xb3fa42 thru 0xb477b5 (0x7d74 bytes) 2024-09-14T22:37:31.006 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:31.006 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:31.010 INFO:teuthology.orchestra.run.smithi001.stdout:4072 punch from 0x8116a7c to 0x811ebbd, (0x8141 bytes) 2024-09-14T22:37:31.026 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:37:31.027 INFO:teuthology.orchestra.run.smithi001.stdout:4074 write 0xe9a7332 thru 0xe9a8cef (0x19be bytes) 2024-09-14T22:37:31.034 INFO:teuthology.orchestra.run.smithi001.stdout:4075 trunc from 0xe9a8cf0 to 0xe44db97 2024-09-14T22:37:31.047 INFO:teuthology.orchestra.run.smithi001.stdout:4076 writesame 0x2470d91 thru 0x2478704 data_size 0x6(0x7974 bytes) 2024-09-14T22:37:31.057 INFO:teuthology.orchestra.run.smithi001.stdout:4077 read 0x752c020 thru 0x7539874 (0xd855 bytes) 2024-09-14T22:37:31.057 INFO:teuthology.orchestra.run.smithi001.stdout:4078 read 0x1808974 thru 0x18152f9 (0xc986 bytes) 2024-09-14T22:37:31.058 INFO:teuthology.orchestra.run.smithi001.stdout:4080 read 0xbef1ba5 thru 0xbefadd7 (0x9233 bytes) 2024-09-14T22:37:31.058 INFO:teuthology.orchestra.run.smithi001.stdout:4084 read 0x77ced34 thru 0x77d35f1 (0x48be bytes) 2024-09-14T22:37:31.059 INFO:teuthology.orchestra.run.smithi001.stdout:4085 writesame 0x718c201 thru 0x718e0e0 data_size 0x1a(0x1ee0 bytes) 2024-09-14T22:37:31.074 INFO:teuthology.orchestra.run.smithi001.stdout:4087 trunc from 0xe44db97 to 0x743a15f 2024-09-14T22:37:31.095 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:31.091+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:31.104 INFO:teuthology.orchestra.run.smithi001.stdout:4089 trunc from 0x743a15f to 0x2b7eb60 2024-09-14T22:37:31.131 INFO:teuthology.orchestra.run.smithi001.stdout:4090 read 0x1c0b00e thru 0x1c125da (0x75cd bytes) 2024-09-14T22:37:31.132 INFO:teuthology.orchestra.run.smithi001.stdout:4091 write 0x14f4600 thru 0x14f6d2e (0x272f bytes) 2024-09-14T22:37:31.165 INFO:teuthology.orchestra.run.smithi001.stdout:4092 write 0xacd6558 thru 0xace2f96 (0xca3f bytes) 2024-09-14T22:37:31.172 INFO:teuthology.orchestra.run.smithi001.stdout:4093 write 0xcc422a thru 0xcc4256 (0x2d bytes) 2024-09-14T22:37:31.175 INFO:teuthology.orchestra.run.smithi001.stdout:4094 writesame 0x7127a0 thru 0x719d14 data_size 0x9(0x7575 bytes) 2024-09-14T22:37:31.177 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:31.177 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:31.181 INFO:teuthology.orchestra.run.smithi001.stdout:4095 write 0x84e18b3 thru 0x84e6b45 (0x5293 bytes) 2024-09-14T22:37:31.189 INFO:teuthology.orchestra.run.smithi001.stdout:4096 punch from 0x1d14139 to 0x1d21e19, (0xdce0 bytes) 2024-09-14T22:37:31.189 INFO:teuthology.orchestra.run.smithi001.stdout:4097 writesame 0x7e1003b thru 0x7e1b746 data_size 0x21(0xb70c bytes) 2024-09-14T22:37:31.193 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:37:31.197 INFO:teuthology.orchestra.run.smithi001.stdout:4098 read 0x6a3995f thru 0x6a3ffab (0x664d bytes) 2024-09-14T22:37:31.197 INFO:teuthology.orchestra.run.smithi001.stdout:4102 punch from 0x5692603 to 0x5694a22, (0x241f bytes) 2024-09-14T22:37:31.197 INFO:teuthology.orchestra.run.smithi001.stdout:4103 write 0x595e94f thru 0x595f813 (0xec5 bytes) 2024-09-14T22:37:31.197 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:31.195+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:31.203 INFO:teuthology.orchestra.run.smithi001.stdout:4104 read 0x8b110cb thru 0x8b20523 (0xf459 bytes) 2024-09-14T22:37:31.203 INFO:teuthology.orchestra.run.smithi001.stdout:4105 punch from 0x4320913 to 0x432c0d0, (0xb7bd bytes) 2024-09-14T22:37:31.217 INFO:teuthology.orchestra.run.smithi001.stdout:4107 write 0xb47202b thru 0xb4810ef (0xf0c5 bytes) 2024-09-14T22:37:31.240 INFO:teuthology.orchestra.run.smithi001.stdout:4110 writesame 0xc0e9074 thru 0xc0f7b32 data_size 0x55(0xeabf bytes) 2024-09-14T22:37:31.240 INFO:teuthology.orchestra.run.smithi001.stdout:4111 writesame 0x14129d thru 0x149a11 data_size 0x3(0x8775 bytes) 2024-09-14T22:37:31.257 INFO:teuthology.orchestra.run.smithi001.stdout:4112 write 0x399dfdd thru 0x399e0f2 (0x116 bytes) 2024-09-14T22:37:31.265 INFO:teuthology.orchestra.run.smithi001.stdout:4114 writesame 0x69fb6f3 thru 0x6a03a58 data_size 0x79(0x8366 bytes) 2024-09-14T22:37:31.276 INFO:teuthology.orchestra.run.smithi001.stdout:4116 punch from 0x67139ed to 0x6722c97, (0xf2aa bytes) 2024-09-14T22:37:31.276 INFO:teuthology.orchestra.run.smithi001.stdout:4117 write 0x2896005 thru 0x2896696 (0x692 bytes) 2024-09-14T22:37:31.283 INFO:teuthology.orchestra.run.smithi001.stdout:4118 write 0x883b033 thru 0x88447bb (0x9789 bytes) 2024-09-14T22:37:31.290 INFO:teuthology.orchestra.run.smithi001.stdout:4119 read 0x31f3889 thru 0x31fc2d2 (0x8a4a bytes) 2024-09-14T22:37:31.297 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:31.295+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:31.310 INFO:teuthology.orchestra.run.smithi001.stdout:4120 trunc from 0xc0f7b33 to 0x1967bab 2024-09-14T22:37:31.329 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:31.329 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:31.343 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:37:31.352 INFO:teuthology.orchestra.run.smithi001.stdout:4122 write 0xf4be64 thru 0xf56c3b (0xadd8 bytes) 2024-09-14T22:37:31.356 INFO:teuthology.orchestra.run.smithi001.stdout:4124 read 0xe9a89d thru 0xe9adde (0x542 bytes) 2024-09-14T22:37:31.372 INFO:teuthology.orchestra.run.smithi001.stdout:4125 write 0x2d72d59 thru 0x2d80ca2 (0xdf4a bytes) 2024-09-14T22:37:31.398 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:31.391+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:31.414 INFO:teuthology.orchestra.run.smithi001.stdout:4127 writesame 0xec99437 thru 0xeca4028 data_size 0xd(0xabf2 bytes) 2024-09-14T22:37:31.414 INFO:teuthology.orchestra.run.smithi001.stdout:4128 write 0x4346d5b thru 0x434fdfa (0x90a0 bytes) 2024-09-14T22:37:31.421 INFO:teuthology.orchestra.run.smithi001.stdout:4130 read 0x3f92a7d thru 0x3f999cf (0x6f53 bytes) 2024-09-14T22:37:31.421 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:31.421 INFO:teuthology.orchestra.run.smithi001.stdout:4132 read 0xbabe2e3 thru 0xbabfedb (0x1bf9 bytes) 2024-09-14T22:37:31.422 INFO:teuthology.orchestra.run.smithi001.stdout:4135 read 0xec05ddb thru 0xec06765 (0x98b bytes) 2024-09-14T22:37:31.422 INFO:teuthology.orchestra.run.smithi001.stdout:4136 write 0xec89968 thru 0xec8a4db (0xb74 bytes) 2024-09-14T22:37:31.428 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:31.451 INFO:teuthology.orchestra.run.smithi001.stdout:4138 trunc from 0xeca4029 to 0x488530a 2024-09-14T22:37:31.477 INFO:teuthology.orchestra.run.smithi001.stdout:4139 read 0x228afca thru 0x228b689 (0x6c0 bytes) 2024-09-14T22:37:31.477 INFO:teuthology.orchestra.run.smithi001.stdout:4140 writesame 0x956198 thru 0x95fa98 data_size 0x17(0x9901 bytes) 2024-09-14T22:37:31.481 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:31.481 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:31.498 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:31.491+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:31.499 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:37:31.504 INFO:teuthology.orchestra.run.smithi001.stdout:4142 writesame 0x788c7b7 thru 0x7890c3b data_size 0x3(0x4485 bytes) 2024-09-14T22:37:31.530 INFO:teuthology.orchestra.run.smithi001.stdout:4143 writesame 0xc1ad89d thru 0xc1b19ba data_size 0x5(0x411e bytes) 2024-09-14T22:37:31.530 INFO:teuthology.orchestra.run.smithi001.stdout:4144 writesame 0x670f8bc thru 0x67112cb data_size 0x18(0x1a10 bytes) 2024-09-14T22:37:31.538 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:31.538 INFO:teuthology.orchestra.run.smithi001.stdout:4146 punch from 0x8f707bc to 0x8f79969, (0x91ad bytes) 2024-09-14T22:37:31.538 INFO:teuthology.orchestra.run.smithi001.stdout:4147 punch from 0x5d002ea to 0x5d05a85, (0x579b bytes) 2024-09-14T22:37:31.538 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:31.538 INFO:teuthology.orchestra.run.smithi001.stdout:4154 punch from 0x6115c28 to 0x6124dc3, (0xf19b bytes) 2024-09-14T22:37:31.538 INFO:teuthology.orchestra.run.smithi001.stdout:4155 write 0xa91dc25 thru 0xa929724 (0xbb00 bytes) 2024-09-14T22:37:31.547 INFO:teuthology.orchestra.run.smithi001.stdout:4156 read 0x9b319f4 thru 0x9b40029 (0xe636 bytes) 2024-09-14T22:37:31.547 INFO:teuthology.orchestra.run.smithi001.stdout:4158 write 0xdb917b thru 0xdbf264 (0x60ea bytes) 2024-09-14T22:37:31.554 INFO:teuthology.orchestra.run.smithi001.stdout:4159 trunc from 0xc1b19bb to 0xc8039ba 2024-09-14T22:37:31.586 INFO:teuthology.orchestra.run.smithi001.stdout:4160 writesame 0xeda94e4 thru 0xedb0962 data_size 0x1(0x747f bytes) 2024-09-14T22:37:31.688 INFO:teuthology.orchestra.run.smithi001.stdout:4162 write 0x4f8fe6b thru 0x4f93457 (0x35ed bytes) 2024-09-14T22:37:31.690 INFO:teuthology.orchestra.run.smithi001.stdout:4163 read 0x2df23e thru 0x2e1ef9 (0x2cbc bytes) 2024-09-14T22:37:31.691 INFO:teuthology.orchestra.run.smithi001.stdout:4164 read 0x79e5beb thru 0x79f0f66 (0xb37c bytes) 2024-09-14T22:37:31.691 INFO:teuthology.orchestra.run.smithi001.stdout:4165 write 0x1e9605 thru 0x1f5196 (0xbb92 bytes) 2024-09-14T22:37:31.691 INFO:teuthology.orchestra.run.smithi001.stdout:4166 read 0xe851398 thru 0xe8594ef (0x8158 bytes) 2024-09-14T22:37:31.691 INFO:teuthology.orchestra.run.smithi001.stdout:4167 read 0x2c583eb thru 0x2c6788f (0xf4a5 bytes) 2024-09-14T22:37:31.691 INFO:teuthology.orchestra.run.smithi001.stdout:4169 writesame 0x15ba402 thru 0x15c5d1a data_size 0x75(0xb919 bytes) 2024-09-14T22:37:31.691 INFO:teuthology.orchestra.run.smithi001.stdout:4170 read 0x43f6a80 thru 0x43f71ae (0x72f bytes) 2024-09-14T22:37:31.691 INFO:teuthology.orchestra.run.smithi001.stdout:4171 read 0x558eaf9 thru 0x55981fe (0x9706 bytes) 2024-09-14T22:37:31.691 INFO:teuthology.orchestra.run.smithi001.stdout:4172 read 0x499c1bb thru 0x49a0fde (0x4e24 bytes) 2024-09-14T22:37:31.691 INFO:teuthology.orchestra.run.smithi001.stdout:4173 write 0x532b6c6 thru 0x5334a31 (0x936c bytes) 2024-09-14T22:37:31.691 INFO:teuthology.orchestra.run.smithi001.stdout:4175 punch from 0x5e43bc8 to 0x5e4457c, (0x9b4 bytes) 2024-09-14T22:37:31.691 INFO:teuthology.orchestra.run.smithi001.stdout:4178 writesame 0x8e96a6c thru 0x8ea2c63 data_size 0xc(0xc1f8 bytes) 2024-09-14T22:37:31.691 INFO:teuthology.orchestra.run.smithi001.stdout:4179 writesame 0x4d93deb thru 0x4d9dd45 data_size 0x29(0x9f5b bytes) 2024-09-14T22:37:31.692 INFO:teuthology.orchestra.run.smithi001.stdout:4180 read 0x98ff955 thru 0x99013ec (0x1a98 bytes) 2024-09-14T22:37:31.692 INFO:teuthology.orchestra.run.smithi001.stdout:4181 write 0x914d3c6 thru 0x915a44a (0xd085 bytes) 2024-09-14T22:37:31.692 INFO:teuthology.orchestra.run.smithi001.stdout:4183 read 0x99255e7 thru 0x993260e (0xd028 bytes) 2024-09-14T22:37:31.692 INFO:teuthology.orchestra.run.smithi001.stdout:4186 punch from 0x9aeda53 to 0x9af660f, (0x8bbc bytes) 2024-09-14T22:37:31.692 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:31.692 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:31.692 INFO:teuthology.orchestra.run.smithi001.stdout:4187 writesame 0xe011bf4 thru 0xe01ec4a data_size 0x1(0xd057 bytes) 2024-09-14T22:37:31.693 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:37:31.694 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:31.683+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:31.696 INFO:teuthology.orchestra.run.smithi001.stdout:4190 write 0x1b889a4 thru 0x1b970b8 (0xe715 bytes) 2024-09-14T22:37:31.698 INFO:teuthology.orchestra.run.smithi001.stdout:4191 read 0x467991d thru 0x467f315 (0x59f9 bytes) 2024-09-14T22:37:31.699 INFO:teuthology.orchestra.run.smithi001.stdout:4192 write 0xf509 thru 0x1ba9c (0xc594 bytes) 2024-09-14T22:37:31.703 INFO:teuthology.orchestra.run.smithi001.stdout:4193 punch from 0xb524da2 to 0xb52927d, (0x44db bytes) 2024-09-14T22:37:31.703 INFO:teuthology.orchestra.run.smithi001.stdout:4194 write 0xbaa3a13 thru 0xbaa8f1c (0x550a bytes) 2024-09-14T22:37:31.711 INFO:teuthology.orchestra.run.smithi001.stdout:4195 writesame 0xd21986b thru 0xd224abb data_size 0x1(0xb251 bytes) 2024-09-14T22:37:31.762 INFO:teuthology.orchestra.run.smithi001.stdout:4196 read 0x3dd574a thru 0x3ddf51a (0x9dd1 bytes) 2024-09-14T22:37:31.765 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.1 to 1 (10000) 2024-09-14T22:37:31.778 INFO:teuthology.orchestra.run.smithi001.stdout:4197 trunc from 0xedb0963 to 0x6ab444a 2024-09-14T22:37:31.782 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T22:37:31.796 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-14T22:37:31.797 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:31.791+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:31.819 INFO:teuthology.orchestra.run.smithi001.stdout:4198 trunc from 0x6ab444a to 0x2ce053e 2024-09-14T22:37:31.846 INFO:teuthology.orchestra.run.smithi001.stdout:4201 read 0x299e690 thru 0x299e8d3 (0x244 bytes) 2024-09-14T22:37:31.846 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:31.849 INFO:teuthology.orchestra.run.smithi001.stdout:4203 trunc from 0x2ce053e to 0xe61ef2 2024-09-14T22:37:31.855 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:31.855 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:31.875 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:37:31.893 INFO:teuthology.orchestra.run.smithi001.stdout:4206 trunc from 0xe61ef2 to 0xc473d06 2024-09-14T22:37:31.893 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:31.891+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:31.908 INFO:teuthology.orchestra.run.smithi001.stdout:4207 read 0x2e86278 thru 0x2e94691 (0xe41a bytes) 2024-09-14T22:37:31.908 INFO:teuthology.orchestra.run.smithi001.stdout:4208 read 0x4ffa5f7 thru 0x50014e3 (0x6eed bytes) 2024-09-14T22:37:31.908 INFO:teuthology.orchestra.run.smithi001.stdout:4209 read 0x2593982 thru 0x25a08a0 (0xcf1f bytes) 2024-09-14T22:37:31.908 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:31.908 INFO:teuthology.orchestra.run.smithi001.stdout:4213 read 0x539e22 thru 0x53c0cd (0x22ac bytes) 2024-09-14T22:37:31.909 INFO:teuthology.orchestra.run.smithi001.stdout:4214 write 0x36a85d2 thru 0x36b4fc2 (0xc9f1 bytes) 2024-09-14T22:37:31.933 INFO:teuthology.orchestra.run.smithi001.stdout:4215 trunc from 0xc473d06 to 0x19c9e9a 2024-09-14T22:37:31.956 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:37:31.958 INFO:teuthology.orchestra.run.smithi001.stdout:4217 trunc from 0x19c9e9a to 0x1e0b55a 2024-09-14T22:37:31.992 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:31.987+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:31.997 INFO:teuthology.orchestra.run.smithi001.stdout:4218 write 0xabb3666 thru 0xabb5181 (0x1b1c bytes) 2024-09-14T22:37:32.004 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:32.004 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:32.004 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:32.004 INFO:teuthology.orchestra.run.smithi001.stdout:4224 write 0x8e2b200 thru 0x8e3177e (0x657f bytes) 2024-09-14T22:37:32.011 INFO:teuthology.orchestra.run.smithi001.stdout:4225 write 0x5ce1e20 thru 0x5ce98d6 (0x7ab7 bytes) 2024-09-14T22:37:32.020 INFO:teuthology.orchestra.run.smithi001.stdout:4229 punch from 0x82e86be to 0x82f62be, (0xdc00 bytes) 2024-09-14T22:37:32.020 INFO:teuthology.orchestra.run.smithi001.stdout:4231 write 0x2d41255 thru 0x2d47533 (0x62df bytes) 2024-09-14T22:37:32.025 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:32.025 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:32.027 INFO:teuthology.orchestra.run.smithi001.stdout:4232 read 0x897b29c thru 0x897bbee (0x953 bytes) 2024-09-14T22:37:32.027 INFO:teuthology.orchestra.run.smithi001.stdout:4233 read 0x9421f8d thru 0x9423189 (0x11fd bytes) 2024-09-14T22:37:32.028 INFO:teuthology.orchestra.run.smithi001.stdout:4234 write 0x13d5d40 thru 0x13dfd60 (0xa021 bytes) 2024-09-14T22:37:32.031 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:32.031 INFO:teuthology.orchestra.run.smithi001.stdout:4239 read 0x4d83589 thru 0x4d840c0 (0xb38 bytes) 2024-09-14T22:37:32.037 INFO:teuthology.orchestra.run.smithi001.stdout:4240 trunc from 0xabb5182 to 0x7c0ff49 2024-09-14T22:37:32.044 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:37:32.074 INFO:teuthology.orchestra.run.smithi001.stdout:4241 trunc from 0x7c0ff49 to 0xe315877 2024-09-14T22:37:32.086 INFO:teuthology.orchestra.run.smithi001.stdout:4242 punch from 0x9f97d2d to 0x9f9b23b, (0x350e bytes) 2024-09-14T22:37:32.086 INFO:teuthology.orchestra.run.smithi001.stdout:4243 write 0x3e46c6 thru 0x3ebd63 (0x769e bytes) 2024-09-14T22:37:32.088 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:32.093 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:32.091+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:32.096 INFO:teuthology.orchestra.run.smithi001.stdout:4248 trunc from 0xe315877 to 0x9581d84 2024-09-14T22:37:32.108 INFO:teuthology.orchestra.run.smithi001.stdout:4249 write 0x40ee24e thru 0x40f4da9 (0x6b5c bytes) 2024-09-14T22:37:32.115 INFO:teuthology.orchestra.run.smithi001.stdout:4250 write 0x66ed728 thru 0x66effed (0x28c6 bytes) 2024-09-14T22:37:32.121 INFO:teuthology.orchestra.run.smithi001.stdout:4251 read 0x4487230 thru 0x4493fe1 (0xcdb2 bytes) 2024-09-14T22:37:32.121 INFO:teuthology.orchestra.run.smithi001.stdout:4253 punch from 0x494f3ea to 0x4956a70, (0x7686 bytes) 2024-09-14T22:37:32.121 INFO:teuthology.orchestra.run.smithi001.stdout:4254 read 0x7169558 thru 0x71777c5 (0xe26e bytes) 2024-09-14T22:37:32.121 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:32.126 INFO:teuthology.orchestra.run.smithi001.stdout:4256 trunc from 0x9581d84 to 0x5f41d8b 2024-09-14T22:37:32.134 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:37:32.135 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:37:32.151 INFO:teuthology.orchestra.run.smithi001.stdout:4257 writesame 0x2373106 thru 0x2376eb4 data_size 0x1(0x3daf bytes) 2024-09-14T22:37:32.153 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:37:32.153 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:37:32.184 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:37:32.184 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:37:32.191 INFO:teuthology.orchestra.run.smithi001.stdout:4259 writesame 0x96ec669 thru 0x96f9857 data_size 0x1(0xd1ef bytes) 2024-09-14T22:37:32.191 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:32.191 INFO:teuthology.orchestra.run.smithi001.stdout:4261 read 0x874cc46 thru 0x8757cc8 (0xb083 bytes) 2024-09-14T22:37:32.191 INFO:teuthology.orchestra.run.smithi001.stdout:4262 write 0x53c854 thru 0x543a5f (0x720c bytes) 2024-09-14T22:37:32.193 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:32.187+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:32.194 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:32.200 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:37:32.201 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:37:32.208 INFO:teuthology.orchestra.run.smithi001.stdout:4264 write 0xadc1f85 thru 0xadc522c (0x32a8 bytes) 2024-09-14T22:37:32.215 INFO:teuthology.orchestra.run.smithi001.stdout:4265 writesame 0x3b26e4d thru 0x3b27b55 data_size 0x1(0xd09 bytes) 2024-09-14T22:37:32.219 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:32.219 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:37:32.219 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:37:32.219 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:37:32.220 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:32.222 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:37:32.222 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:37:32.224 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:32.232 INFO:teuthology.orchestra.run.smithi001.stdout:4267 trunc from 0xadc522d to 0x52e7a00 2024-09-14T22:37:32.240 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:37:32.245 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:37:32.245 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:37:32.259 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:32.259 INFO:teuthology.orchestra.run.smithi001.stdout:4270 read 0x2906ced thru 0x2914634 (0xd948 bytes) 2024-09-14T22:37:32.259 INFO:teuthology.orchestra.run.smithi001.stdout:4271 read 0x2841827 thru 0x284c7f6 (0xafd0 bytes) 2024-09-14T22:37:32.260 INFO:teuthology.orchestra.run.smithi001.stdout:4272 punch from 0x15aba7c to 0x15b2941, (0x6ec5 bytes) 2024-09-14T22:37:32.260 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:32.260 INFO:teuthology.orchestra.run.smithi001.stdout:4274 read 0x30434dd thru 0x304a67f (0x71a3 bytes) 2024-09-14T22:37:32.260 INFO:teuthology.orchestra.run.smithi001.stdout:4275 punch from 0x282d9ee to 0x2835ccf, (0x82e1 bytes) 2024-09-14T22:37:32.262 INFO:teuthology.orchestra.run.smithi001.stdout:4276 trunc from 0x52e7a00 to 0x7669208 2024-09-14T22:37:32.267 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:37:32.267 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:37:32.275 INFO:teuthology.orchestra.run.smithi001.stdout:4279 writesame 0x24f453e thru 0x24f7b53 data_size 0x2e(0x3616 bytes) 2024-09-14T22:37:32.281 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:32.282 INFO:teuthology.orchestra.run.smithi001.stdout:4281 write 0x709d82f thru 0x70a08e7 (0x30b9 bytes) 2024-09-14T22:37:32.289 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:37:32.290 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:37:32.291 INFO:teuthology.orchestra.run.smithi001.stdout:4282 write 0x1de2fdc thru 0x1de594d (0x2972 bytes) 2024-09-14T22:37:32.294 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:32.291+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:32.299 INFO:teuthology.orchestra.run.smithi001.stdout:4284 punch from 0xe8b639 to 0xe9357f, (0x7f46 bytes) 2024-09-14T22:37:32.299 INFO:teuthology.orchestra.run.smithi001.stdout:4285 write 0xaa30d thru 0xabc52 (0x1946 bytes) 2024-09-14T22:37:32.301 INFO:teuthology.orchestra.run.smithi001.stdout:4286 read 0x5b309c6 thru 0x5b31422 (0xa5d bytes) 2024-09-14T22:37:32.322 INFO:teuthology.orchestra.run.smithi001.stdout:4287 writesame 0xe3a9e03 thru 0xe3adf9e data_size 0x44(0x419c bytes) 2024-09-14T22:37:32.322 INFO:teuthology.orchestra.run.smithi001.stdout:4288 punch from 0x91052ed to 0x910ed1f, (0x9a32 bytes) 2024-09-14T22:37:32.322 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:32.322 INFO:teuthology.orchestra.run.smithi001.stdout:4290 write 0x667c04f thru 0x6686203 (0xa1b5 bytes) 2024-09-14T22:37:32.333 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:32.333 INFO:teuthology.orchestra.run.smithi001.stdout:4293 write 0x6a05284 thru 0x6a0c474 (0x71f1 bytes) 2024-09-14T22:37:32.340 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:32.353 INFO:teuthology.orchestra.run.smithi001.stdout:4297 trunc from 0xe3adf9f to 0x2a691f6 2024-09-14T22:37:32.363 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:32.363 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:37:32.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:37:32.363 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:37:32.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.363 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:31370 3.0 3:02f48f58:::rbd_data.113cda0183a0.0000000000000000:6 [read 813056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:32.364 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:37:32.364 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 58.999911216000001, 2024-09-14T22:37:32.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047519796000000003, 2024-09-14T22:37:32.364 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.364 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.364 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.364 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.364 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.364 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.364 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 31370 2024-09-14T22:37:32.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.364 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:37:32.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:37:32.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:37:32.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0999999999999999e-07 2024-09-14T22:37:32.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362401+0000", 2024-09-14T22:37:32.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2900000000000001e-06 2024-09-14T22:37:32.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362406+0000", 2024-09-14T22:37:32.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6960000000000002e-06 2024-09-14T22:37:32.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362412+0000", 2024-09-14T22:37:32.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.823e-06 2024-09-14T22:37:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.400270+0000", 2024-09-14T22:37:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037857737000000002 2024-09-14T22:37:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.400342+0000", 2024-09-14T22:37:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1797000000000004e-05 2024-09-14T22:37:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.409918+0000", 2024-09-14T22:37:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0095760429999999994 2024-09-14T22:37:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:113946 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 5554176~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:37:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.923438+0000", 2024-09-14T22:37:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.4388705860000002, 2024-09-14T22:37:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047179154000000001, 2024-09-14T22:37:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.369 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.369 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.369 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.369 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 113946 2024-09-14T22:37:32.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.369 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.923438+0000", 2024-09-14T22:37:32.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.923438+0000", 2024-09-14T22:37:32.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.923439+0000", 2024-09-14T22:37:32.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8100000000000003e-07 2024-09-14T22:37:32.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.923441+0000", 2024-09-14T22:37:32.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.43e-06 2024-09-14T22:37:32.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.371 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.371 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.923446+0000", 2024-09-14T22:37:32.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8250000000000004e-06 2024-09-14T22:37:32.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.371 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.371 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.923449+0000", 2024-09-14T22:37:32.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9840000000000001e-06 2024-09-14T22:37:32.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.371 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.371 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.970508+0000", 2024-09-14T22:37:32.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047058425000000001 2024-09-14T22:37:32.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.970551+0000", 2024-09-14T22:37:32.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3723000000000001e-05 2024-09-14T22:37:32.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.970618+0000", 2024-09-14T22:37:32.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6285999999999995e-05 2024-09-14T22:37:32.372 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.372 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.372 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.373 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:113947 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 5603328~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:37:32.373 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.923453+0000", 2024-09-14T22:37:32.373 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.4388560840000002, 2024-09-14T22:37:32.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047310420999999998, 2024-09-14T22:37:32.373 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.373 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.373 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.373 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.373 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.373 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.373 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 113947 2024-09-14T22:37:32.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.374 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.923453+0000", 2024-09-14T22:37:32.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.923453+0000", 2024-09-14T22:37:32.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926004+0000", 2024-09-14T22:37:32.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002551382 2024-09-14T22:37:32.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926009+0000", 2024-09-14T22:37:32.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5639999999999998e-06 2024-09-14T22:37:32.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926022+0000", 2024-09-14T22:37:32.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2734999999999999e-05 2024-09-14T22:37:32.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926029+0000", 2024-09-14T22:37:32.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2520000000000002e-06 2024-09-14T22:37:32.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.970650+0000", 2024-09-14T22:37:32.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044621277000000001 2024-09-14T22:37:32.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.970700+0000", 2024-09-14T22:37:32.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9713000000000003e-05 2024-09-14T22:37:32.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.970763+0000", 2024-09-14T22:37:32.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3497999999999994e-05 2024-09-14T22:37:32.377 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.377 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.377 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.377 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114136 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3448832~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:37:32.377 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932179+0000", 2024-09-14T22:37:32.378 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.4301297179999999, 2024-09-14T22:37:32.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047236017999999998, 2024-09-14T22:37:32.378 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.378 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.378 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.378 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.378 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.378 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.378 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114136 2024-09-14T22:37:32.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.378 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932179+0000", 2024-09-14T22:37:32.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.379 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932179+0000", 2024-09-14T22:37:32.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.379 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932324+0000", 2024-09-14T22:37:32.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014451000000000001 2024-09-14T22:37:32.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932328+0000", 2024-09-14T22:37:32.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5889999999999996e-06 2024-09-14T22:37:32.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932338+0000", 2024-09-14T22:37:32.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3000000000000007e-06 2024-09-14T22:37:32.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932344+0000", 2024-09-14T22:37:32.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7150000000000001e-06 2024-09-14T22:37:32.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979233+0000", 2024-09-14T22:37:32.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046888711 2024-09-14T22:37:32.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979313+0000", 2024-09-14T22:37:32.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9487000000000007e-05 2024-09-14T22:37:32.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979415+0000", 2024-09-14T22:37:32.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000102706 2024-09-14T22:37:32.382 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.382 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.382 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.382 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114138 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3497984~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:37:32.382 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932356+0000", 2024-09-14T22:37:32.382 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.4299528320000001, 2024-09-14T22:37:32.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047285958000000003, 2024-09-14T22:37:32.382 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.382 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.382 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.383 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.383 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.383 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.383 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114138 2024-09-14T22:37:32.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.383 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932356+0000", 2024-09-14T22:37:32.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932356+0000", 2024-09-14T22:37:32.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932498+0000", 2024-09-14T22:37:32.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000142182 2024-09-14T22:37:32.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932503+0000", 2024-09-14T22:37:32.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.301e-06 2024-09-14T22:37:32.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.385 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932514+0000", 2024-09-14T22:37:32.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1484000000000001e-05 2024-09-14T22:37:32.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.385 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932521+0000", 2024-09-14T22:37:32.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0740000000000004e-06 2024-09-14T22:37:32.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979451+0000", 2024-09-14T22:37:32.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046929379 2024-09-14T22:37:32.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979535+0000", 2024-09-14T22:37:32.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4423000000000001e-05 2024-09-14T22:37:32.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979642+0000", 2024-09-14T22:37:32.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010711499999999999 2024-09-14T22:37:32.387 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.387 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.387 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.387 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114139 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3547136~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:37:32.387 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932554+0000", 2024-09-14T22:37:32.387 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.4297550650000002, 2024-09-14T22:37:32.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047292230999999997, 2024-09-14T22:37:32.387 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.387 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.387 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.387 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.388 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.388 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.388 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114139 2024-09-14T22:37:32.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.388 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932554+0000", 2024-09-14T22:37:32.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932554+0000", 2024-09-14T22:37:32.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932819+0000", 2024-09-14T22:37:32.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026538400000000002 2024-09-14T22:37:32.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932825+0000", 2024-09-14T22:37:32.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3469999999999998e-06 2024-09-14T22:37:32.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932836+0000", 2024-09-14T22:37:32.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1e-05 2024-09-14T22:37:32.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932843+0000", 2024-09-14T22:37:32.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1790000000000002e-06 2024-09-14T22:37:32.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979673+0000", 2024-09-14T22:37:32.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046830389 2024-09-14T22:37:32.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979750+0000", 2024-09-14T22:37:32.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6312999999999999e-05 2024-09-14T22:37:32.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979846+0000", 2024-09-14T22:37:32.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6619000000000004e-05 2024-09-14T22:37:32.391 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.391 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.391 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.391 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114141 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3596288~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:37:32.391 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932855+0000", 2024-09-14T22:37:32.391 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.4294544670000002, 2024-09-14T22:37:32.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047201870999999999, 2024-09-14T22:37:32.392 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.392 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.392 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.392 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.392 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.392 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.392 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114141 2024-09-14T22:37:32.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.392 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932855+0000", 2024-09-14T22:37:32.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932855+0000", 2024-09-14T22:37:32.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934033+0000", 2024-09-14T22:37:32.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001178264 2024-09-14T22:37:32.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934038+0000", 2024-09-14T22:37:32.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4820000000000002e-06 2024-09-14T22:37:32.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934048+0000", 2024-09-14T22:37:32.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9890000000000004e-06 2024-09-14T22:37:32.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934055+0000", 2024-09-14T22:37:32.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8329999999999996e-06 2024-09-14T22:37:32.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979878+0000", 2024-09-14T22:37:32.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045822773999999997 2024-09-14T22:37:32.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979958+0000", 2024-09-14T22:37:32.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0477999999999998e-05 2024-09-14T22:37:32.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.980056+0000", 2024-09-14T22:37:32.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8051e-05 2024-09-14T22:37:32.395 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.395 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.396 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.396 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114159 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3940352~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:37:32.396 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.934179+0000", 2024-09-14T22:37:32.396 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.428130431, 2024-09-14T22:37:32.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047341439999999999, 2024-09-14T22:37:32.396 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.396 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.396 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.396 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.396 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.396 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.396 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114159 2024-09-14T22:37:32.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.397 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.397 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934179+0000", 2024-09-14T22:37:32.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.397 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934179+0000", 2024-09-14T22:37:32.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.397 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935021+0000", 2024-09-14T22:37:32.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00084276800000000003 2024-09-14T22:37:32.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935027+0000", 2024-09-14T22:37:32.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2959999999999998e-06 2024-09-14T22:37:32.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935038+0000", 2024-09-14T22:37:32.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1198e-05 2024-09-14T22:37:32.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935054+0000", 2024-09-14T22:37:32.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5764e-05 2024-09-14T22:37:32.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.981341+0000", 2024-09-14T22:37:32.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.04628782 2024-09-14T22:37:32.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.981415+0000", 2024-09-14T22:37:32.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3349999999999994e-05 2024-09-14T22:37:32.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.981520+0000", 2024-09-14T22:37:32.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105244 2024-09-14T22:37:32.400 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.400 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.400 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.400 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114180 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4136960~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:37:32.400 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:37:32.400 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.4271308949999999, 2024-09-14T22:37:32.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047190621000000002, 2024-09-14T22:37:32.400 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.400 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.401 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.401 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.401 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.401 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.401 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114180 2024-09-14T22:37:32.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.401 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:37:32.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:37:32.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:37:32.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6800000000000002e-07 2024-09-14T22:37:32.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935180+0000", 2024-09-14T22:37:32.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.42e-06 2024-09-14T22:37:32.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935185+0000", 2024-09-14T22:37:32.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1359999999999996e-06 2024-09-14T22:37:32.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935188+0000", 2024-09-14T22:37:32.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.711e-06 2024-09-14T22:37:32.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982201+0000", 2024-09-14T22:37:32.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047013309000000003 2024-09-14T22:37:32.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982278+0000", 2024-09-14T22:37:32.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6705000000000005e-05 2024-09-14T22:37:32.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982369+0000", 2024-09-14T22:37:32.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1071999999999998e-05 2024-09-14T22:37:32.404 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.404 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.404 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.404 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114182 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4186112~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:37:32.405 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:37:32.405 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.427102273, 2024-09-14T22:37:32.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047368990999999999, 2024-09-14T22:37:32.405 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.405 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.405 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.405 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.405 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.405 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.405 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114182 2024-09-14T22:37:32.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.405 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:37:32.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:37:32.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:37:32.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7099999999999998e-07 2024-09-14T22:37:32.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935209+0000", 2024-09-14T22:37:32.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0260000000000002e-06 2024-09-14T22:37:32.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935213+0000", 2024-09-14T22:37:32.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3429999999999998e-06 2024-09-14T22:37:32.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935217+0000", 2024-09-14T22:37:32.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2480000000000001e-06 2024-09-14T22:37:32.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982406+0000", 2024-09-14T22:37:32.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047188920000000002 2024-09-14T22:37:32.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982483+0000", 2024-09-14T22:37:32.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7008999999999994e-05 2024-09-14T22:37:32.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982576+0000", 2024-09-14T22:37:32.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3173999999999999e-05 2024-09-14T22:37:32.409 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.409 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.409 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.409 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114185 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4202496~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:37:32.409 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.935221+0000", 2024-09-14T22:37:32.409 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.427088108, 2024-09-14T22:37:32.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047601128999999999, 2024-09-14T22:37:32.409 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.409 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.409 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.410 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.410 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.410 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.410 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114185 2024-09-14T22:37:32.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.410 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935221+0000", 2024-09-14T22:37:32.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935221+0000", 2024-09-14T22:37:32.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935970+0000", 2024-09-14T22:37:32.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074930899999999998 2024-09-14T22:37:32.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935976+0000", 2024-09-14T22:37:32.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5439999999999998e-06 2024-09-14T22:37:32.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935988+0000", 2024-09-14T22:37:32.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1779e-05 2024-09-14T22:37:32.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935996+0000", 2024-09-14T22:37:32.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.106e-06 2024-09-14T22:37:32.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982607+0000", 2024-09-14T22:37:32.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046611504999999998 2024-09-14T22:37:32.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982686+0000", 2024-09-14T22:37:32.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8805000000000002e-05 2024-09-14T22:37:32.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982822+0000", 2024-09-14T22:37:32.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000136081 2024-09-14T22:37:32.413 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.413 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.413 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.414 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114189 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4300800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:37:32.414 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.936048+0000", 2024-09-14T22:37:32.414 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.426260976, 2024-09-14T22:37:32.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047231539000000003, 2024-09-14T22:37:32.414 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.414 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.414 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.414 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.414 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.414 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.414 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114189 2024-09-14T22:37:32.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.414 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936048+0000", 2024-09-14T22:37:32.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936048+0000", 2024-09-14T22:37:32.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936049+0000", 2024-09-14T22:37:32.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4099999999999999e-07 2024-09-14T22:37:32.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936052+0000", 2024-09-14T22:37:32.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0029999999999999e-06 2024-09-14T22:37:32.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936058+0000", 2024-09-14T22:37:32.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7880000000000001e-06 2024-09-14T22:37:32.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936064+0000", 2024-09-14T22:37:32.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2279999999999998e-06 2024-09-14T22:37:32.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983120+0000", 2024-09-14T22:37:32.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047056384999999999 2024-09-14T22:37:32.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983194+0000", 2024-09-14T22:37:32.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3811000000000002e-05 2024-09-14T22:37:32.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983280+0000", 2024-09-14T22:37:32.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5883000000000004e-05 2024-09-14T22:37:32.418 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.418 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.418 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.418 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114191 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4349952~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:37:32.418 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:37:32.418 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.426237236, 2024-09-14T22:37:32.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047365365, 2024-09-14T22:37:32.418 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.418 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.418 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.418 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.419 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.419 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.419 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114191 2024-09-14T22:37:32.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.419 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:37:32.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:37:32.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:37:32.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0499999999999999e-07 2024-09-14T22:37:32.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936075+0000", 2024-09-14T22:37:32.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.734e-06 2024-09-14T22:37:32.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936082+0000", 2024-09-14T22:37:32.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8360000000000003e-06 2024-09-14T22:37:32.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936087+0000", 2024-09-14T22:37:32.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3129999999999998e-06 2024-09-14T22:37:32.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983319+0000", 2024-09-14T22:37:32.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047232140999999998 2024-09-14T22:37:32.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983365+0000", 2024-09-14T22:37:32.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5776999999999999e-05 2024-09-14T22:37:32.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983437+0000", 2024-09-14T22:37:32.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2158999999999998e-05 2024-09-14T22:37:32.422 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.422 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.422 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.423 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114193 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4399104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:37:32.423 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.936095+0000", 2024-09-14T22:37:32.423 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.4262142999999998, 2024-09-14T22:37:32.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047505915000000003, 2024-09-14T22:37:32.423 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.423 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.423 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.423 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.423 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.423 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.423 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114193 2024-09-14T22:37:32.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.423 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936095+0000", 2024-09-14T22:37:32.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936095+0000", 2024-09-14T22:37:32.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938021+0000", 2024-09-14T22:37:32.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001926178 2024-09-14T22:37:32.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938027+0000", 2024-09-14T22:37:32.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3110000000000002e-06 2024-09-14T22:37:32.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938041+0000", 2024-09-14T22:37:32.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3344e-05 2024-09-14T22:37:32.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938049+0000", 2024-09-14T22:37:32.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2279999999999998e-06 2024-09-14T22:37:32.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983468+0000", 2024-09-14T22:37:32.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045419494999999997 2024-09-14T22:37:32.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983524+0000", 2024-09-14T22:37:32.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5952000000000002e-05 2024-09-14T22:37:32.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983601+0000", 2024-09-14T22:37:32.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6407000000000002e-05 2024-09-14T22:37:32.427 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.427 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.427 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.427 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114710 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4857856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:37:32.427 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.962571+0000", 2024-09-14T22:37:32.427 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.3997380279999998, 2024-09-14T22:37:32.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047253314999999997, 2024-09-14T22:37:32.427 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.427 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.427 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.427 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.427 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.428 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.428 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114710 2024-09-14T22:37:32.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.428 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962571+0000", 2024-09-14T22:37:32.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962571+0000", 2024-09-14T22:37:32.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962572+0000", 2024-09-14T22:37:32.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1399999999999997e-07 2024-09-14T22:37:32.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962574+0000", 2024-09-14T22:37:32.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8679999999999999e-06 2024-09-14T22:37:32.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962583+0000", 2024-09-14T22:37:32.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8310000000000007e-06 2024-09-14T22:37:32.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962589+0000", 2024-09-14T22:37:32.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9529999999999996e-06 2024-09-14T22:37:32.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009692+0000", 2024-09-14T22:37:32.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047103249999999999 2024-09-14T22:37:32.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009767+0000", 2024-09-14T22:37:32.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4672000000000006e-05 2024-09-14T22:37:32.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009824+0000", 2024-09-14T22:37:32.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7226999999999999e-05 2024-09-14T22:37:32.431 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.431 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.431 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.431 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114712 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4907008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:37:32.431 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:37:32.431 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.3997125480000001, 2024-09-14T22:37:32.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047421087000000001, 2024-09-14T22:37:32.432 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.432 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.432 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.432 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.432 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.432 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.432 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114712 2024-09-14T22:37:32.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.432 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:37:32.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:37:32.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:37:32.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.03e-07 2024-09-14T22:37:32.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962599+0000", 2024-09-14T22:37:32.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3199999999999998e-06 2024-09-14T22:37:32.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962606+0000", 2024-09-14T22:37:32.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5289999999999997e-06 2024-09-14T22:37:32.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962612+0000", 2024-09-14T22:37:32.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.947e-06 2024-09-14T22:37:32.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009866+0000", 2024-09-14T22:37:32.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047254681999999999 2024-09-14T22:37:32.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009924+0000", 2024-09-14T22:37:32.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7556000000000003e-05 2024-09-14T22:37:32.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010018+0000", 2024-09-14T22:37:32.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.365e-05 2024-09-14T22:37:32.435 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.435 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.435 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.436 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114714 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4956160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:37:32.436 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.962618+0000", 2024-09-14T22:37:32.436 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.3996910300000001, 2024-09-14T22:37:32.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.04759302, 2024-09-14T22:37:32.436 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.436 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.436 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.436 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.436 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.436 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.436 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114714 2024-09-14T22:37:32.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.436 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962618+0000", 2024-09-14T22:37:32.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962618+0000", 2024-09-14T22:37:32.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966011+0000", 2024-09-14T22:37:32.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003393067 2024-09-14T22:37:32.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966016+0000", 2024-09-14T22:37:32.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3580000000000002e-06 2024-09-14T22:37:32.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966032+0000", 2024-09-14T22:37:32.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5356e-05 2024-09-14T22:37:32.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966040+0000", 2024-09-14T22:37:32.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1980000000000001e-06 2024-09-14T22:37:32.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010055+0000", 2024-09-14T22:37:32.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044014676000000003 2024-09-14T22:37:32.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010127+0000", 2024-09-14T22:37:32.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2206000000000006e-05 2024-09-14T22:37:32.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010211+0000", 2024-09-14T22:37:32.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4159000000000005e-05 2024-09-14T22:37:32.440 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.440 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.440 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.440 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114754 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 6283264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:37:32.440 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:37:32.440 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.3955161170000001, 2024-09-14T22:37:32.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047228150000000003, 2024-09-14T22:37:32.440 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.440 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.441 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.441 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.441 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.441 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.441 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114754 2024-09-14T22:37:32.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.441 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:37:32.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:37:32.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:37:32.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7399999999999999e-07 2024-09-14T22:37:32.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966796+0000", 2024-09-14T22:37:32.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3630000000000001e-06 2024-09-14T22:37:32.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966802+0000", 2024-09-14T22:37:32.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4679999999999997e-06 2024-09-14T22:37:32.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966807+0000", 2024-09-14T22:37:32.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1109999999999999e-06 2024-09-14T22:37:32.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.013929+0000", 2024-09-14T22:37:32.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047121720999999998 2024-09-14T22:37:32.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.013968+0000", 2024-09-14T22:37:32.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8877000000000001e-05 2024-09-14T22:37:32.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014021+0000", 2024-09-14T22:37:32.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3236000000000003e-05 2024-09-14T22:37:32.444 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.444 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.444 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.445 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114789 3.19 3:9836bb25:::rbd_object_map.113c14c67864.000000000000001b:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:37:32.445 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.966813+0000", 2024-09-14T22:37:32.445 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.3954956549999999, 2024-09-14T22:37:32.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05001311, 2024-09-14T22:37:32.445 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.445 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.445 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:32.445 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.445 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.445 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.445 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114789 2024-09-14T22:37:32.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.445 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966813+0000", 2024-09-14T22:37:32.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966813+0000", 2024-09-14T22:37:32.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966814+0000", 2024-09-14T22:37:32.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9200000000000002e-07 2024-09-14T22:37:32.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966817+0000", 2024-09-14T22:37:32.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8380000000000002e-06 2024-09-14T22:37:32.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966831+0000", 2024-09-14T22:37:32.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4209999999999999e-05 2024-09-14T22:37:32.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966845+0000", 2024-09-14T22:37:32.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3799e-05 2024-09-14T22:37:32.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014046+0000", 2024-09-14T22:37:32.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047201604000000001 2024-09-14T22:37:32.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014116+0000", 2024-09-14T22:37:32.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9387000000000005e-05 2024-09-14T22:37:32.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-14T22:37:32.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014469+0000", 2024-09-14T22:37:32.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00035355800000000002 2024-09-14T22:37:32.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:32.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016719+0000", 2024-09-14T22:37:32.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0022498459999999998 2024-09-14T22:37:32.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:37:32.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016735+0000", 2024-09-14T22:37:32.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5792999999999999e-05 2024-09-14T22:37:32.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:37:32.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016790+0000", 2024-09-14T22:37:32.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4715e-05 2024-09-14T22:37:32.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.450 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.450 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016827+0000", 2024-09-14T22:37:32.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6968000000000002e-05 2024-09-14T22:37:32.450 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.450 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.450 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.450 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114803 3.19 3:9836bb25:::rbd_object_map.113c14c67864.000000000000001b:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:37:32.450 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.966853+0000", 2024-09-14T22:37:32.450 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.3954562250000002, 2024-09-14T22:37:32.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050656093999999999, 2024-09-14T22:37:32.451 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.451 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.451 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:32.451 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.451 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.451 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.451 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114803 2024-09-14T22:37:32.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.451 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966853+0000", 2024-09-14T22:37:32.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966853+0000", 2024-09-14T22:37:32.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969930+0000", 2024-09-14T22:37:32.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030769759999999999 2024-09-14T22:37:32.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969936+0000", 2024-09-14T22:37:32.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7150000000000003e-06 2024-09-14T22:37:32.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969949+0000", 2024-09-14T22:37:32.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3166000000000001e-05 2024-09-14T22:37:32.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969957+0000", 2024-09-14T22:37:32.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8269999999999998e-06 2024-09-14T22:37:32.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014996+0000", 2024-09-14T22:37:32.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045039202 2024-09-14T22:37:32.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.015085+0000", 2024-09-14T22:37:32.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8987999999999996e-05 2024-09-14T22:37:32.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-14T22:37:32.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.015318+0000", 2024-09-14T22:37:32.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023284999999999999 2024-09-14T22:37:32.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:32.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016898+0000", 2024-09-14T22:37:32.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0015807379999999999 2024-09-14T22:37:32.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:37:32.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.017455+0000", 2024-09-14T22:37:32.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00055664600000000005 2024-09-14T22:37:32.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:37:32.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.017483+0000", 2024-09-14T22:37:32.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8238e-05 2024-09-14T22:37:32.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.017509+0000", 2024-09-14T22:37:32.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5748000000000001e-05 2024-09-14T22:37:32.456 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.456 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.456 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.456 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.456 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.456 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:32.457 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:37:32.459 INFO:teuthology.orchestra.run.smithi001.stdout:4301 read 0x20e236a thru 0x20e2bd1 (0x868 bytes) 2024-09-14T22:37:32.459 INFO:teuthology.orchestra.run.smithi001.stdout:4302 write 0xcdd65d3 thru 0xcddbe70 (0x589e bytes) 2024-09-14T22:37:32.459 INFO:teuthology.orchestra.run.smithi001.stdout:4303 write 0x2b7a9c9 thru 0x2b7cb56 (0x218e bytes) 2024-09-14T22:37:32.459 INFO:teuthology.orchestra.run.smithi001.stdout:4304 write 0xc27a029 thru 0xc289c78 (0xfc50 bytes) 2024-09-14T22:37:32.459 INFO:teuthology.orchestra.run.smithi001.stdout:4305 write 0xc602a6d thru 0xc604094 (0x1628 bytes) 2024-09-14T22:37:32.459 INFO:teuthology.orchestra.run.smithi001.stdout:4306 read 0x9d926ed thru 0x9d9c72e (0xa042 bytes) 2024-09-14T22:37:32.459 INFO:teuthology.orchestra.run.smithi001.stdout:4307 read 0x876776 thru 0x88301a (0xc8a5 bytes) 2024-09-14T22:37:32.459 INFO:teuthology.orchestra.run.smithi001.stdout:4308 punch from 0x9dc56ac to 0x9dc8a60, (0x33b4 bytes) 2024-09-14T22:37:32.459 INFO:teuthology.orchestra.run.smithi001.stdout:4309 write 0x4d4857e thru 0x4d4fb28 (0x75ab bytes) 2024-09-14T22:37:32.459 INFO:teuthology.orchestra.run.smithi001.stdout:4310 write 0x47b1dff thru 0x47be219 (0xc41b bytes) 2024-09-14T22:37:32.459 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:32.455+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:32.483 INFO:teuthology.orchestra.run.smithi001.stdout:4311 write 0xed0784f thru 0xed16af1 (0xf2a3 bytes) 2024-09-14T22:37:32.491 INFO:teuthology.orchestra.run.smithi001.stdout:4312 read 0xab973ad thru 0xab9a588 (0x31dc bytes) 2024-09-14T22:37:32.491 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:32.515 INFO:teuthology.orchestra.run.smithi001.stdout:4316 trunc from 0xed16af2 to 0x451eea4 2024-09-14T22:37:32.549 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:32.558 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:32.555+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:32.564 INFO:teuthology.orchestra.run.smithi001.stdout:4318 writesame 0x4e92154 thru 0x4e94550 data_size 0x25(0x23fd bytes) 2024-09-14T22:37:32.564 INFO:teuthology.orchestra.run.smithi001.stdout:4319 write 0x3fdceb7 thru 0x3fdd828 (0x972 bytes) 2024-09-14T22:37:32.590 INFO:teuthology.orchestra.run.smithi001.stdout:4320 trunc from 0x4e94551 to 0xd8dd4e8 2024-09-14T22:37:32.608 INFO:teuthology.orchestra.run.smithi001.stdout:4321 read 0xaad960b thru 0xaada2c3 (0xcb9 bytes) 2024-09-14T22:37:32.608 INFO:teuthology.orchestra.run.smithi001.stdout:4322 read 0xb8d0eb5 thru 0xb8d3716 (0x2862 bytes) 2024-09-14T22:37:32.608 INFO:teuthology.orchestra.run.smithi001.stdout:4323 read 0x4926959 thru 0x49282c6 (0x196e bytes) 2024-09-14T22:37:32.608 INFO:teuthology.orchestra.run.smithi001.stdout:4326 read 0x7467d4d thru 0x746d3fe (0x56b2 bytes) 2024-09-14T22:37:32.608 INFO:teuthology.orchestra.run.smithi001.stdout:4327 punch from 0x661d5e6 to 0x662289b, (0x52b5 bytes) 2024-09-14T22:37:32.608 INFO:teuthology.orchestra.run.smithi001.stdout:4328 writesame 0x4adeabe thru 0x4ae4e78 data_size 0xb(0x63bb bytes) 2024-09-14T22:37:32.612 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:32.612 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:37:32.612 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:37:32.612 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:32.617 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:32.617 INFO:teuthology.orchestra.run.smithi001.stdout:4330 read 0xc37d86e thru 0xc3846a1 (0x6e34 bytes) 2024-09-14T22:37:32.617 INFO:teuthology.orchestra.run.smithi001.stdout:4331 write 0x42162e3 thru 0x4218a99 (0x27b7 bytes) 2024-09-14T22:37:32.624 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:37:32.625 INFO:teuthology.orchestra.run.smithi001.stdout:4332 writesame 0x922d027 thru 0x923b225 data_size 0x15(0xe1ff bytes) 2024-09-14T22:37:32.634 INFO:teuthology.orchestra.run.smithi001.stdout:4333 read 0xcc046b1 thru 0xcc0646b (0x1dbb bytes) 2024-09-14T22:37:32.634 INFO:teuthology.orchestra.run.smithi001.stdout:4335 writesame 0x27363c6 thru 0x273c593 data_size 0x6b(0x61ce bytes) 2024-09-14T22:37:32.638 INFO:teuthology.orchestra.run.smithi001.stdout:4336 trunc from 0xd8dd4e8 to 0xe0cc3bd 2024-09-14T22:37:32.651 INFO:teuthology.orchestra.run.smithi001.stdout:4338 writesame 0xa7c2076 thru 0xa7cf0ca data_size 0x13(0xd055 bytes) 2024-09-14T22:37:32.659 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:32.655+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:32.659 INFO:teuthology.orchestra.run.smithi001.stdout:4339 punch from 0x376b451 to 0x376cd98, (0x1947 bytes) 2024-09-14T22:37:32.660 INFO:teuthology.orchestra.run.smithi001.stdout:4340 write 0x3646097 thru 0x3649753 (0x36bd bytes) 2024-09-14T22:37:32.663 INFO:teuthology.orchestra.run.smithi001.stdout:4341 write 0x6d4c5f0 thru 0x6d527c4 (0x61d5 bytes) 2024-09-14T22:37:32.670 INFO:teuthology.orchestra.run.smithi001.stdout:4342 writesame 0x7303bb0 thru 0x730913b data_size 0x64(0x558c bytes) 2024-09-14T22:37:32.674 INFO:teuthology.orchestra.run.smithi001.stdout:4343 writesame 0x4254d17 thru 0x4262241 data_size 0x29(0xd52b bytes) 2024-09-14T22:37:32.678 INFO:teuthology.orchestra.run.smithi001.stdout:4344 write 0x472b51f thru 0x47394a5 (0xdf87 bytes) 2024-09-14T22:37:32.686 INFO:teuthology.orchestra.run.smithi001.stdout:4345 punch from 0xa569df4 to 0xa573dea, (0x9ff6 bytes) 2024-09-14T22:37:32.686 INFO:teuthology.orchestra.run.smithi001.stdout:4346 punch from 0x2862d66 to 0x28663d7, (0x3671 bytes) 2024-09-14T22:37:32.687 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:32.687 INFO:teuthology.orchestra.run.smithi001.stdout:4348 write 0x70fe8e5 thru 0x7109b9e (0xb2ba bytes) 2024-09-14T22:37:32.697 INFO:teuthology.orchestra.run.smithi001.stdout:4349 punch from 0xa46a31f to 0xa4738a0, (0x9581 bytes) 2024-09-14T22:37:32.697 INFO:teuthology.orchestra.run.smithi001.stdout:4352 write 0xe41c69 thru 0xe4cbfa (0xaf92 bytes) 2024-09-14T22:37:32.700 INFO:teuthology.orchestra.run.smithi001.stdout:4353 writesame 0x34d3ab6 thru 0x34d3f8b data_size 0x1(0x4d6 bytes) 2024-09-14T22:37:32.708 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:32.708 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:32.708 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:32.708 INFO:teuthology.orchestra.run.smithi001.stdout:4358 write 0xd482ae thru 0xd5543f (0xd192 bytes) 2024-09-14T22:37:32.711 INFO:teuthology.orchestra.run.smithi001.stdout:4361 write 0xaec96d8 thru 0xaeca97a (0x12a3 bytes) 2024-09-14T22:37:32.718 INFO:teuthology.orchestra.run.smithi001.stdout:4362 writesame 0x8be1df8 thru 0x8bed166 data_size 0x1(0xb36f bytes) 2024-09-14T22:37:32.752 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:32.752 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:37:32.752 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:37:32.752 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:37:32.753 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:32.760 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:32.755+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:32.766 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:37:32.771 INFO:teuthology.orchestra.run.smithi001.stdout:4364 read 0x7c89595 thru 0x7c8b761 (0x21cd bytes) 2024-09-14T22:37:32.773 INFO:teuthology.orchestra.run.smithi001.stdout:4365 write 0x18c7c66 thru 0x18c9eb6 (0x2251 bytes) 2024-09-14T22:37:32.775 INFO:teuthology.orchestra.run.smithi001.stdout:4367 writesame 0xc7a599e thru 0xc7b1855 data_size 0x8(0xbeb8 bytes) 2024-09-14T22:37:32.781 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2024-09-14T22:37:32.785 INFO:teuthology.orchestra.run.smithi001.stdout:4369 punch from 0x763f74d to 0x7641e68, (0x271b bytes) 2024-09-14T22:37:32.785 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.2 to 1 (10000) 2024-09-14T22:37:32.799 INFO:teuthology.orchestra.run.smithi001.stdout:4371 trunc from 0xe0cc3bd to 0x43631a2 2024-09-14T22:37:32.816 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-14T22:37:32.836 INFO:teuthology.orchestra.run.smithi001.stdout:4372 writesame 0x2d3f0d4 thru 0x2d4da73 data_size 0x20(0xe9a0 bytes) 2024-09-14T22:37:32.840 INFO:teuthology.orchestra.run.smithi001.stdout:4373 read 0x41f8861 thru 0x42022ef (0x9a8f bytes) 2024-09-14T22:37:32.860 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:32.855+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:32.864 INFO:teuthology.orchestra.run.smithi001.stdout:4374 writesame 0xb4affdd thru 0xb4b9390 data_size 0x89(0x93b4 bytes) 2024-09-14T22:37:32.864 INFO:teuthology.orchestra.run.smithi001.stdout:4375 punch from 0x8816afd to 0x881fda8, (0x92ab bytes) 2024-09-14T22:37:32.878 INFO:teuthology.orchestra.run.smithi001.stdout:4376 write 0xbea0ead thru 0xbeb0adb (0xfc2f bytes) 2024-09-14T22:37:32.885 INFO:teuthology.orchestra.run.smithi001.stdout:4377 read 0x75a1e0d thru 0x75a3640 (0x1834 bytes) 2024-09-14T22:37:32.898 INFO:teuthology.orchestra.run.smithi001.stdout:4379 write 0xbed4790 thru 0xbede93d (0xa1ae bytes) 2024-09-14T22:37:32.901 INFO:teuthology.orchestra.run.smithi001.stdout:4381 read 0x78403af thru 0x7844fb5 (0x4c07 bytes) 2024-09-14T22:37:32.902 INFO:teuthology.orchestra.run.smithi001.stdout:4383 read 0x6435e53 thru 0x643c9a8 (0x6b56 bytes) 2024-09-14T22:37:32.909 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:32.909 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:37:32.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:37:32.909 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:37:32.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.909 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35306 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1425408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:32.909 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507069+0000", 2024-09-14T22:37:32.909 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.399332127999998, 2024-09-14T22:37:32.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045725259999999997, 2024-09-14T22:37:32.910 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.910 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.910 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.910 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.910 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.910 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.910 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35306 2024-09-14T22:37:32.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.910 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507069+0000", 2024-09-14T22:37:32.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507069+0000", 2024-09-14T22:37:32.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507070+0000", 2024-09-14T22:37:32.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0400000000000002e-07 2024-09-14T22:37:32.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507072+0000", 2024-09-14T22:37:32.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.891e-06 2024-09-14T22:37:32.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507078+0000", 2024-09-14T22:37:32.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0279999999999997e-06 2024-09-14T22:37:32.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507082+0000", 2024-09-14T22:37:32.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4680000000000003e-06 2024-09-14T22:37:32.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552632+0000", 2024-09-14T22:37:32.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045550144000000001 2024-09-14T22:37:32.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552707+0000", 2024-09-14T22:37:32.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4148999999999998e-05 2024-09-14T22:37:32.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552795+0000", 2024-09-14T22:37:32.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8176000000000004e-05 2024-09-14T22:37:32.913 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.913 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.914 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.914 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35307 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1454080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:32.914 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:37:32.914 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.399313577000001, 2024-09-14T22:37:32.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045891358, 2024-09-14T22:37:32.914 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.914 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.914 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.914 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.914 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.914 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.914 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35307 2024-09-14T22:37:32.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.915 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:37:32.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:37:32.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:37:32.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4700000000000002e-07 2024-09-14T22:37:32.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507093+0000", 2024-09-14T22:37:32.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9520000000000002e-06 2024-09-14T22:37:32.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507099+0000", 2024-09-14T22:37:32.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0240000000000003e-06 2024-09-14T22:37:32.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507105+0000", 2024-09-14T22:37:32.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5520000000000004e-06 2024-09-14T22:37:32.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552826+0000", 2024-09-14T22:37:32.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045721494000000001 2024-09-14T22:37:32.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552896+0000", 2024-09-14T22:37:32.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9519999999999998e-05 2024-09-14T22:37:32.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552979+0000", 2024-09-14T22:37:32.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3468999999999997e-05 2024-09-14T22:37:32.918 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.918 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.918 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.918 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35308 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1482752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:32.918 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:37:32.918 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.399290581000002, 2024-09-14T22:37:32.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046076533000000003, 2024-09-14T22:37:32.918 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.918 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.918 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.918 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.919 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.919 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.919 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35308 2024-09-14T22:37:32.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.919 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:37:32.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:37:32.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.920 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:37:32.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:37:32.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.920 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507113+0000", 2024-09-14T22:37:32.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9099999999999999e-06 2024-09-14T22:37:32.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.920 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507119+0000", 2024-09-14T22:37:32.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7440000000000004e-06 2024-09-14T22:37:32.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507124+0000", 2024-09-14T22:37:32.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2129999999999999e-06 2024-09-14T22:37:32.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553007+0000", 2024-09-14T22:37:32.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045882488999999999 2024-09-14T22:37:32.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553076+0000", 2024-09-14T22:37:32.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9233000000000006e-05 2024-09-14T22:37:32.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553188+0000", 2024-09-14T22:37:32.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111557 2024-09-14T22:37:32.922 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.922 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.922 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.922 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35309 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1511424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:32.922 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:37:32.922 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.399271675999998, 2024-09-14T22:37:32.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046231869000000002, 2024-09-14T22:37:32.923 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.923 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.923 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.923 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.923 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.923 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.923 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35309 2024-09-14T22:37:32.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.923 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:37:32.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:37:32.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:37:32.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9099999999999999e-07 2024-09-14T22:37:32.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507132+0000", 2024-09-14T22:37:32.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8190000000000001e-06 2024-09-14T22:37:32.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507139+0000", 2024-09-14T22:37:32.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9419999999999998e-06 2024-09-14T22:37:32.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507145+0000", 2024-09-14T22:37:32.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2600000000000002e-06 2024-09-14T22:37:32.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553225+0000", 2024-09-14T22:37:32.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046079991000000001 2024-09-14T22:37:32.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553284+0000", 2024-09-14T22:37:32.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8638999999999999e-05 2024-09-14T22:37:32.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553362+0000", 2024-09-14T22:37:32.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7826999999999999e-05 2024-09-14T22:37:32.927 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.927 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.927 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.927 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35310 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1540096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:32.927 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507151+0000", 2024-09-14T22:37:32.927 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.399250219999999, 2024-09-14T22:37:32.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046444541999999998, 2024-09-14T22:37:32.927 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.927 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.928 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.928 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.928 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.928 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.928 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35310 2024-09-14T22:37:32.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.928 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507151+0000", 2024-09-14T22:37:32.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507151+0000", 2024-09-14T22:37:32.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507152+0000", 2024-09-14T22:37:32.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1699999999999999e-07 2024-09-14T22:37:32.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507154+0000", 2024-09-14T22:37:32.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.902e-06 2024-09-14T22:37:32.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507159+0000", 2024-09-14T22:37:32.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.823e-06 2024-09-14T22:37:32.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507165+0000", 2024-09-14T22:37:32.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.587e-06 2024-09-14T22:37:32.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553395+0000", 2024-09-14T22:37:32.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046230318999999999 2024-09-14T22:37:32.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553434+0000", 2024-09-14T22:37:32.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8680999999999998e-05 2024-09-14T22:37:32.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553596+0000", 2024-09-14T22:37:32.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016191299999999999 2024-09-14T22:37:32.931 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.931 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.931 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.931 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35311 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1568768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:32.931 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:37:32.931 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.399230967000001, 2024-09-14T22:37:32.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046623592999999998, 2024-09-14T22:37:32.932 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.932 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.932 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.932 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.932 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.932 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.932 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35311 2024-09-14T22:37:32.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.932 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:37:32.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:37:32.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:37:32.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:37:32.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507176+0000", 2024-09-14T22:37:32.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3349999999999997e-06 2024-09-14T22:37:32.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507200+0000", 2024-09-14T22:37:32.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4003e-05 2024-09-14T22:37:32.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507206+0000", 2024-09-14T22:37:32.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.976e-06 2024-09-14T22:37:32.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553627+0000", 2024-09-14T22:37:32.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046420284999999999 2024-09-14T22:37:32.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553696+0000", 2024-09-14T22:37:32.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9567999999999995e-05 2024-09-14T22:37:32.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553794+0000", 2024-09-14T22:37:32.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8060000000000006e-05 2024-09-14T22:37:32.935 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.935 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.936 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.936 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35566 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1484800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:32.936 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514668+0000", 2024-09-14T22:37:32.936 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.391733189999997, 2024-09-14T22:37:32.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045644974999999997, 2024-09-14T22:37:32.936 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.936 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.936 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.936 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.936 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.936 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.936 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35566 2024-09-14T22:37:32.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.937 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514668+0000", 2024-09-14T22:37:32.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514668+0000", 2024-09-14T22:37:32.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514669+0000", 2024-09-14T22:37:32.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9099999999999999e-07 2024-09-14T22:37:32.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514671+0000", 2024-09-14T22:37:32.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3800000000000001e-06 2024-09-14T22:37:32.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514677+0000", 2024-09-14T22:37:32.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9660000000000001e-06 2024-09-14T22:37:32.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514682+0000", 2024-09-14T22:37:32.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0939999999999998e-06 2024-09-14T22:37:32.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560133+0000", 2024-09-14T22:37:32.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045450398000000003 2024-09-14T22:37:32.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560207+0000", 2024-09-14T22:37:32.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4196999999999995e-05 2024-09-14T22:37:32.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560313+0000", 2024-09-14T22:37:32.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106549 2024-09-14T22:37:32.940 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.940 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.940 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.940 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35567 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1513472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:32.940 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:37:32.940 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.391712650999999, 2024-09-14T22:37:32.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045793215999999998, 2024-09-14T22:37:32.941 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.941 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.941 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.941 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.941 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.941 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.941 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35567 2024-09-14T22:37:32.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.941 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:37:32.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:37:32.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:37:32.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1899999999999998e-07 2024-09-14T22:37:32.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514692+0000", 2024-09-14T22:37:32.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2699999999999999e-06 2024-09-14T22:37:32.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514698+0000", 2024-09-14T22:37:32.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0939999999999996e-06 2024-09-14T22:37:32.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514703+0000", 2024-09-14T22:37:32.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3870000000000003e-06 2024-09-14T22:37:32.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560353+0000", 2024-09-14T22:37:32.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045649914 2024-09-14T22:37:32.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560416+0000", 2024-09-14T22:37:32.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3096999999999996e-05 2024-09-14T22:37:32.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560482+0000", 2024-09-14T22:37:32.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6035e-05 2024-09-14T22:37:32.944 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.945 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.945 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.945 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35568 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1542144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:32.945 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:37:32.945 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.391687853000001, 2024-09-14T22:37:32.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045995921000000002, 2024-09-14T22:37:32.945 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.945 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.945 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.945 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.945 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.945 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.946 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35568 2024-09-14T22:37:32.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.946 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:37:32.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:37:32.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:37:32.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4099999999999999e-07 2024-09-14T22:37:32.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514720+0000", 2024-09-14T22:37:32.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6869999999999999e-06 2024-09-14T22:37:32.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514727+0000", 2024-09-14T22:37:32.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3540000000000002e-06 2024-09-14T22:37:32.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514732+0000", 2024-09-14T22:37:32.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7690000000000004e-06 2024-09-14T22:37:32.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560523+0000", 2024-09-14T22:37:32.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045791373000000003 2024-09-14T22:37:32.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560606+0000", 2024-09-14T22:37:32.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2690999999999998e-05 2024-09-14T22:37:32.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560710+0000", 2024-09-14T22:37:32.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000103606 2024-09-14T22:37:32.949 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.949 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.949 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.949 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35569 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1570816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:32.949 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:37:32.949 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.391662044, 2024-09-14T22:37:32.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046110630999999999, 2024-09-14T22:37:32.949 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.949 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.950 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.950 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.950 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.950 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.950 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35569 2024-09-14T22:37:32.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.950 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:37:32.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:37:32.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.951 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.951 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:37:32.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.82e-07 2024-09-14T22:37:32.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.951 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.951 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514743+0000", 2024-09-14T22:37:32.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.269e-06 2024-09-14T22:37:32.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.951 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.951 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514750+0000", 2024-09-14T22:37:32.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7340000000000002e-06 2024-09-14T22:37:32.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514755+0000", 2024-09-14T22:37:32.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7260000000000002e-06 2024-09-14T22:37:32.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560731+0000", 2024-09-14T22:37:32.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045976377999999998 2024-09-14T22:37:32.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560777+0000", 2024-09-14T22:37:32.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5868999999999997e-05 2024-09-14T22:37:32.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560850+0000", 2024-09-14T22:37:32.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3172999999999999e-05 2024-09-14T22:37:32.953 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.953 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.953 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.953 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:86180 3.18 3:1d0c52d8:::rbd_data.113cd62ea8a5.000000000000004b:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:32.953 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:01.356417+0000", 2024-09-14T22:37:32.954 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.549984948999999, 2024-09-14T22:37:32.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049143245000000002, 2024-09-14T22:37:32.954 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.954 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.954 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:32.954 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.954 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.954 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.954 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 86180 2024-09-14T22:37:32.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.954 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.954 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.954 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356417+0000", 2024-09-14T22:37:32.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356417+0000", 2024-09-14T22:37:32.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356420+0000", 2024-09-14T22:37:32.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2030000000000001e-06 2024-09-14T22:37:32.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356435+0000", 2024-09-14T22:37:32.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5287000000000001e-05 2024-09-14T22:37:32.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356472+0000", 2024-09-14T22:37:32.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7311000000000002e-05 2024-09-14T22:37:32.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356492+0000", 2024-09-14T22:37:32.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9519999999999999e-05 2024-09-14T22:37:32.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356670+0000", 2024-09-14T22:37:32.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000177567 2024-09-14T22:37:32.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356924+0000", 2024-09-14T22:37:32.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025399299999999999 2024-09-14T22:37:32.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 0", 2024-09-14T22:37:32.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.365052+0000", 2024-09-14T22:37:32.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0081289380000000005 2024-09-14T22:37:32.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:32.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.405396+0000", 2024-09-14T22:37:32.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040343228000000002 2024-09-14T22:37:32.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:37:32.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.405421+0000", 2024-09-14T22:37:32.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5763e-05 2024-09-14T22:37:32.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:37:32.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.405491+0000", 2024-09-14T22:37:32.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9721999999999994e-05 2024-09-14T22:37:32.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.405560+0000", 2024-09-14T22:37:32.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8713000000000004e-05 2024-09-14T22:37:32.959 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.959 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.959 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.959 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98378 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 919552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:32.959 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148183+0000", 2024-09-14T22:37:32.959 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.758218671000002, 2024-09-14T22:37:32.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045614549999999997, 2024-09-14T22:37:32.959 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.959 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.959 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.960 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.960 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.960 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.960 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98378 2024-09-14T22:37:32.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.960 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148183+0000", 2024-09-14T22:37:32.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148183+0000", 2024-09-14T22:37:32.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148183+0000", 2024-09-14T22:37:32.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0800000000000001e-07 2024-09-14T22:37:32.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148184+0000", 2024-09-14T22:37:32.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2780000000000001e-06 2024-09-14T22:37:32.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148189+0000", 2024-09-14T22:37:32.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2699999999999998e-06 2024-09-14T22:37:32.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148191+0000", 2024-09-14T22:37:32.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7659999999999999e-06 2024-09-14T22:37:32.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193708+0000", 2024-09-14T22:37:32.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045516562000000003 2024-09-14T22:37:32.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193751+0000", 2024-09-14T22:37:32.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2811000000000001e-05 2024-09-14T22:37:32.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193797+0000", 2024-09-14T22:37:32.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6655e-05 2024-09-14T22:37:32.963 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.963 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.963 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.963 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98379 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 928768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:32.963 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148195+0000", 2024-09-14T22:37:32.963 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.758206842, 2024-09-14T22:37:32.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045817405999999998, 2024-09-14T22:37:32.964 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.964 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.964 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.964 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.964 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.964 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.964 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98379 2024-09-14T22:37:32.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.964 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148195+0000", 2024-09-14T22:37:32.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148195+0000", 2024-09-14T22:37:32.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148195+0000", 2024-09-14T22:37:32.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3999999999999998e-07 2024-09-14T22:37:32.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148196+0000", 2024-09-14T22:37:32.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.341e-06 2024-09-14T22:37:32.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148200+0000", 2024-09-14T22:37:32.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9040000000000002e-06 2024-09-14T22:37:32.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148203+0000", 2024-09-14T22:37:32.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.959e-06 2024-09-14T22:37:32.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193841+0000", 2024-09-14T22:37:32.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045637886000000003 2024-09-14T22:37:32.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193933+0000", 2024-09-14T22:37:32.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1823999999999994e-05 2024-09-14T22:37:32.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194012+0000", 2024-09-14T22:37:32.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9252000000000006e-05 2024-09-14T22:37:32.967 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.967 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.967 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.968 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98380 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 937984~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:32.968 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148206+0000", 2024-09-14T22:37:32.968 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.758195204, 2024-09-14T22:37:32.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045937354, 2024-09-14T22:37:32.968 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.968 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.968 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.968 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.968 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.968 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.968 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98380 2024-09-14T22:37:32.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.969 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148206+0000", 2024-09-14T22:37:32.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148206+0000", 2024-09-14T22:37:32.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148207+0000", 2024-09-14T22:37:32.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1e-07 2024-09-14T22:37:32.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148221+0000", 2024-09-14T22:37:32.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4679e-05 2024-09-14T22:37:32.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148228+0000", 2024-09-14T22:37:32.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3820000000000001e-06 2024-09-14T22:37:32.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148231+0000", 2024-09-14T22:37:32.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2849999999999999e-06 2024-09-14T22:37:32.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194054+0000", 2024-09-14T22:37:32.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045822596 2024-09-14T22:37:32.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194098+0000", 2024-09-14T22:37:32.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4604000000000002e-05 2024-09-14T22:37:32.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194144+0000", 2024-09-14T22:37:32.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5597999999999999e-05 2024-09-14T22:37:32.972 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.972 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.972 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.972 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98381 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 947200~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:32.972 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148236+0000", 2024-09-14T22:37:32.972 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.758165711, 2024-09-14T22:37:32.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046111129000000001, 2024-09-14T22:37:32.972 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.973 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.973 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.973 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.973 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.973 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.973 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98381 2024-09-14T22:37:32.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.973 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.973 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.973 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148236+0000", 2024-09-14T22:37:32.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148236+0000", 2024-09-14T22:37:32.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148236+0000", 2024-09-14T22:37:32.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8799999999999998e-07 2024-09-14T22:37:32.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148238+0000", 2024-09-14T22:37:32.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6559999999999999e-06 2024-09-14T22:37:32.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148246+0000", 2024-09-14T22:37:32.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3440000000000001e-06 2024-09-14T22:37:32.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148251+0000", 2024-09-14T22:37:32.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5480000000000003e-06 2024-09-14T22:37:32.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194187+0000", 2024-09-14T22:37:32.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045935847000000002 2024-09-14T22:37:32.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194267+0000", 2024-09-14T22:37:32.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0267999999999998e-05 2024-09-14T22:37:32.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194347+0000", 2024-09-14T22:37:32.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0178000000000004e-05 2024-09-14T22:37:32.976 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.976 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.976 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.976 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98382 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 956416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:32.977 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148254+0000", 2024-09-14T22:37:32.977 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.758147339000001, 2024-09-14T22:37:32.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046224227, 2024-09-14T22:37:32.977 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.977 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.977 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.977 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.977 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.977 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.977 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98382 2024-09-14T22:37:32.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.977 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148254+0000", 2024-09-14T22:37:32.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.978 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148254+0000", 2024-09-14T22:37:32.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.978 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148254+0000", 2024-09-14T22:37:32.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3099999999999999e-07 2024-09-14T22:37:32.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148256+0000", 2024-09-14T22:37:32.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5540000000000001e-06 2024-09-14T22:37:32.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148259+0000", 2024-09-14T22:37:32.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2499999999999998e-06 2024-09-14T22:37:32.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148262+0000", 2024-09-14T22:37:32.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8310000000000001e-06 2024-09-14T22:37:32.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194387+0000", 2024-09-14T22:37:32.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046125082999999997 2024-09-14T22:37:32.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194430+0000", 2024-09-14T22:37:32.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3074999999999997e-05 2024-09-14T22:37:32.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194478+0000", 2024-09-14T22:37:32.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8202999999999999e-05 2024-09-14T22:37:32.980 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.981 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.981 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.981 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98383 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 965632~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:32.981 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148265+0000", 2024-09-14T22:37:32.981 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.758136225000001, 2024-09-14T22:37:32.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046409141000000001, 2024-09-14T22:37:32.981 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.981 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.981 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.981 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.981 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.981 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.982 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98383 2024-09-14T22:37:32.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.982 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148265+0000", 2024-09-14T22:37:32.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148265+0000", 2024-09-14T22:37:32.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148266+0000", 2024-09-14T22:37:32.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0800000000000001e-07 2024-09-14T22:37:32.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148267+0000", 2024-09-14T22:37:32.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1659999999999999e-06 2024-09-14T22:37:32.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148271+0000", 2024-09-14T22:37:32.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1289999999999998e-06 2024-09-14T22:37:32.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148274+0000", 2024-09-14T22:37:32.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5939999999999998e-06 2024-09-14T22:37:32.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194523+0000", 2024-09-14T22:37:32.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046248162000000002 2024-09-14T22:37:32.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194603+0000", 2024-09-14T22:37:32.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0161999999999996e-05 2024-09-14T22:37:32.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194675+0000", 2024-09-14T22:37:32.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1719999999999998e-05 2024-09-14T22:37:32.985 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.985 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.985 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.985 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98384 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 974848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:32.985 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148278+0000", 2024-09-14T22:37:32.985 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.758123897000001, 2024-09-14T22:37:32.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046527369999999998, 2024-09-14T22:37:32.986 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.986 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.986 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.986 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.986 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.986 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.986 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98384 2024-09-14T22:37:32.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.986 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148278+0000", 2024-09-14T22:37:32.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148278+0000", 2024-09-14T22:37:32.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.149987+0000", 2024-09-14T22:37:32.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017093430000000001 2024-09-14T22:37:32.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.149993+0000", 2024-09-14T22:37:32.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6219999999999997e-06 2024-09-14T22:37:32.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150006+0000", 2024-09-14T22:37:32.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3095000000000001e-05 2024-09-14T22:37:32.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150014+0000", 2024-09-14T22:37:32.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6340000000000007e-06 2024-09-14T22:37:32.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194715+0000", 2024-09-14T22:37:32.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044700349 2024-09-14T22:37:32.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194759+0000", 2024-09-14T22:37:32.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3807000000000003e-05 2024-09-14T22:37:32.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194805+0000", 2024-09-14T22:37:32.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6520000000000002e-05 2024-09-14T22:37:32.989 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.989 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.990 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.990 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98390 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 1030144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:32.990 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.150152+0000", 2024-09-14T22:37:32.990 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.75624921, 2024-09-14T22:37:32.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045642708999999997, 2024-09-14T22:37:32.990 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.990 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.990 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.990 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.990 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.990 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.990 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98390 2024-09-14T22:37:32.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.991 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150152+0000", 2024-09-14T22:37:32.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150152+0000", 2024-09-14T22:37:32.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150153+0000", 2024-09-14T22:37:32.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.08e-07 2024-09-14T22:37:32.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150155+0000", 2024-09-14T22:37:32.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2910000000000002e-06 2024-09-14T22:37:32.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150162+0000", 2024-09-14T22:37:32.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4099999999999996e-06 2024-09-14T22:37:32.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150167+0000", 2024-09-14T22:37:32.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3199999999999999e-06 2024-09-14T22:37:32.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195704+0000", 2024-09-14T22:37:32.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045537243999999998 2024-09-14T22:37:32.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195749+0000", 2024-09-14T22:37:32.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4866e-05 2024-09-14T22:37:32.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195795+0000", 2024-09-14T22:37:32.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.617e-05 2024-09-14T22:37:32.994 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.994 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.994 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.994 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98391 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 1039360~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:32.994 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.150175+0000", 2024-09-14T22:37:32.994 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.756226374000001, 2024-09-14T22:37:32.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045825232, 2024-09-14T22:37:32.994 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:32.995 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:32.995 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:32.995 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:32.995 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:32.995 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:32.995 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98391 2024-09-14T22:37:32.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.995 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:32.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:32.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150175+0000", 2024-09-14T22:37:32.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:32.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150175+0000", 2024-09-14T22:37:32.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:32.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:32.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150176+0000", 2024-09-14T22:37:32.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9700000000000002e-07 2024-09-14T22:37:32.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:32.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150178+0000", 2024-09-14T22:37:32.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.407e-06 2024-09-14T22:37:32.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:32.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150185+0000", 2024-09-14T22:37:32.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5189999999999998e-06 2024-09-14T22:37:32.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:32.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150190+0000", 2024-09-14T22:37:32.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2399999999999998e-06 2024-09-14T22:37:32.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:32.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195838+0000", 2024-09-14T22:37:32.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045648265 2024-09-14T22:37:32.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:32.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195918+0000", 2024-09-14T22:37:32.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0094999999999998e-05 2024-09-14T22:37:32.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:32.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:32.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:32.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.196000+0000", 2024-09-14T22:37:32.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2309000000000001e-05 2024-09-14T22:37:32.999 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.999 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.999 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.999 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:32.999 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:32.999 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:33.000 INFO:teuthology.orchestra.run.smithi001.stdout:4385 trunc from 0xbede93e to 0x2bb2b05 2024-09-14T22:37:33.001 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:37:33.001 INFO:teuthology.orchestra.run.smithi001.stdout:4386 read 0x1a7a2a9 thru 0x1a7da54 (0x37ac bytes) 2024-09-14T22:37:33.001 INFO:teuthology.orchestra.run.smithi001.stdout:4388 write 0x85bbea7 thru 0x85c7201 (0xb35b bytes) 2024-09-14T22:37:33.001 INFO:teuthology.orchestra.run.smithi001.stdout:4391 trunc from 0x85c7202 to 0xcea3e9 2024-09-14T22:37:33.002 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:32.995+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:33.016 INFO:teuthology.orchestra.run.smithi001.stdout:4392 punch from 0xcbcccd to 0xcbfd52, (0x3085 bytes) 2024-09-14T22:37:33.016 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:33.016 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:33.016 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:33.016 INFO:teuthology.orchestra.run.smithi001.stdout:4397 punch from 0xccfa4 to 0xd1ee7, (0x4f43 bytes) 2024-09-14T22:37:33.016 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:33.017 INFO:teuthology.orchestra.run.smithi001.stdout:4402 trunc from 0xcea3e9 to 0x1a8075d 2024-09-14T22:37:33.026 INFO:teuthology.orchestra.run.smithi001.stdout:4403 punch from 0x15443f to 0x156c1a, (0x27db bytes) 2024-09-14T22:37:33.045 INFO:teuthology.orchestra.run.smithi001.stdout:4404 write 0x9f738fb thru 0x9f81bd8 (0xe2de bytes) 2024-09-14T22:37:33.053 INFO:teuthology.orchestra.run.smithi001.stdout:4405 write 0x82f7dfb thru 0x82fb866 (0x3a6c bytes) 2024-09-14T22:37:33.058 INFO:teuthology.orchestra.run.smithi001.stdout:4406 writesame 0x8d64ca7 thru 0x8d67e96 data_size 0x22(0x31f0 bytes) 2024-09-14T22:37:33.063 INFO:teuthology.orchestra.run.smithi001.stdout:4408 write 0x39a821 thru 0x39ae81 (0x661 bytes) 2024-09-14T22:37:33.083 INFO:teuthology.orchestra.run.smithi001.stdout:4410 write 0xbd629d1 thru 0xbd6ad45 (0x8375 bytes) 2024-09-14T22:37:33.098 INFO:teuthology.orchestra.run.smithi001.stdout:4411 trunc from 0xbd6ad46 to 0x8355771 2024-09-14T22:37:33.101 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:33.099+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:33.138 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:33.138 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:33.138 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:37:33.138 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:33.153 INFO:teuthology.orchestra.run.smithi001.stdout:4412 writesame 0xe65e1e9 thru 0xe66d12a data_size 0xaf(0xef42 bytes) 2024-09-14T22:37:33.153 INFO:teuthology.orchestra.run.smithi001.stdout:4413 read 0xbb0ea4a thru 0xbb141da (0x5791 bytes) 2024-09-14T22:37:33.153 INFO:teuthology.orchestra.run.smithi001.stdout:4414 punch from 0x74ab6b6 to 0x74ac660, (0xfaa bytes) 2024-09-14T22:37:33.153 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:33.153 INFO:teuthology.orchestra.run.smithi001.stdout:4416 punch from 0x1b100ef to 0x1b13731, (0x3642 bytes) 2024-09-14T22:37:33.153 INFO:teuthology.orchestra.run.smithi001.stdout:4417 write 0xadfe1db thru 0xae02258 (0x407e bytes) 2024-09-14T22:37:33.154 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:37:33.160 INFO:teuthology.orchestra.run.smithi001.stdout:4418 read 0x2ffe5ca thru 0x3005ae7 (0x751e bytes) 2024-09-14T22:37:33.160 INFO:teuthology.orchestra.run.smithi001.stdout:4419 write 0x19c239f thru 0x19d16fb (0xf35d bytes) 2024-09-14T22:37:33.164 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:33.164 INFO:teuthology.orchestra.run.smithi001.stdout:4421 write 0x81ac47f thru 0x81b0612 (0x4194 bytes) 2024-09-14T22:37:33.171 INFO:teuthology.orchestra.run.smithi001.stdout:4422 read 0x8f0c1e7 thru 0x8f1ba9d (0xf8b7 bytes) 2024-09-14T22:37:33.171 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:33.171 INFO:teuthology.orchestra.run.smithi001.stdout:4427 read 0x5e7cbbe thru 0x5e8af2c (0xe36f bytes) 2024-09-14T22:37:33.171 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:33.171 INFO:teuthology.orchestra.run.smithi001.stdout:4431 write 0x8ae0f10 thru 0x8ae9178 (0x8269 bytes) 2024-09-14T22:37:33.173 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:33.173 INFO:teuthology.orchestra.run.smithi001.stdout:4434 read 0x847e984 thru 0x84814f3 (0x2b70 bytes) 2024-09-14T22:37:33.174 INFO:teuthology.orchestra.run.smithi001.stdout:4435 read 0x3a4df74 thru 0x3a5c05a (0xe0e7 bytes) 2024-09-14T22:37:33.174 INFO:teuthology.orchestra.run.smithi001.stdout:4436 write 0xe2a7c98 thru 0xe2ab8d0 (0x3c39 bytes) 2024-09-14T22:37:33.178 INFO:teuthology.orchestra.run.smithi001.stdout:4437 punch from 0x3f57e03 to 0x3f61654, (0x9851 bytes) 2024-09-14T22:37:33.178 INFO:teuthology.orchestra.run.smithi001.stdout:4441 writesame 0x5224d59 thru 0x5226631 data_size 0x1(0x18d9 bytes) 2024-09-14T22:37:33.184 INFO:teuthology.orchestra.run.smithi001.stdout:4442 punch from 0x873dd06 to 0x87445d8, (0x68d2 bytes) 2024-09-14T22:37:33.184 INFO:teuthology.orchestra.run.smithi001.stdout:4443 writesame 0x817a5ed thru 0x8188df0 data_size 0x3e(0xe804 bytes) 2024-09-14T22:37:33.187 INFO:teuthology.orchestra.run.smithi001.stdout:4444 read 0x2ef021d thru 0x2efe815 (0xe5f9 bytes) 2024-09-14T22:37:33.188 INFO:teuthology.orchestra.run.smithi001.stdout:4445 punch from 0x3255388 to 0x3264125, (0xed9d bytes) 2024-09-14T22:37:33.188 INFO:teuthology.orchestra.run.smithi001.stdout:4446 write 0x4595c4a thru 0x459ae64 (0x521b bytes) 2024-09-14T22:37:33.192 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:33.192 INFO:teuthology.orchestra.run.smithi001.stdout:4448 write 0x4aa3f16 thru 0x4ab2e77 (0xef62 bytes) 2024-09-14T22:37:33.197 INFO:teuthology.orchestra.run.smithi001.stdout:4449 read 0xa2fbbd9 thru 0xa30b5b3 (0xf9db bytes) 2024-09-14T22:37:33.197 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:33.197 INFO:teuthology.orchestra.run.smithi001.stdout:4451 write 0x3e6f41a thru 0x3e78b5e (0x9745 bytes) 2024-09-14T22:37:33.201 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:33.195+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:33.211 INFO:teuthology.orchestra.run.smithi001.stdout:4452 write 0xeb64ee2 thru 0xeb66b8f (0x1cae bytes) 2024-09-14T22:37:33.216 INFO:teuthology.orchestra.run.smithi001.stdout:4453 write 0xe13d532 thru 0xe13ffe4 (0x2ab3 bytes) 2024-09-14T22:37:33.220 INFO:teuthology.orchestra.run.smithi001.stdout:4454 write 0xcd30c thru 0xdbbcd (0xe8c2 bytes) 2024-09-14T22:37:33.241 INFO:teuthology.orchestra.run.smithi001.stdout:4457 trunc from 0xeb66b90 to 0x7326ec3 2024-09-14T22:37:33.262 INFO:teuthology.orchestra.run.smithi001.stdout:4458 trunc from 0x7326ec3 to 0x7033fec 2024-09-14T22:37:33.275 INFO:teuthology.orchestra.run.smithi001.stdout:4459 punch from 0x67bc71d to 0x67bdc7f, (0x1562 bytes) 2024-09-14T22:37:33.275 INFO:teuthology.orchestra.run.smithi001.stdout:4460 writesame 0x3aae7b7 thru 0x3ab05e3 data_size 0x19(0x1e2d bytes) 2024-09-14T22:37:33.289 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:33.289 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:33.290 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:37:33.290 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:37:33.290 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:33.297 INFO:teuthology.orchestra.run.smithi001.stdout:4461 write 0x7ea52a8 thru 0x7eb1e9f (0xcbf8 bytes) 2024-09-14T22:37:33.302 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:33.299+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:33.304 INFO:teuthology.orchestra.run.smithi001.stdout:4462 write 0x45136a4 thru 0x4513f72 (0x8cf bytes) 2024-09-14T22:37:33.305 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:37:33.310 INFO:teuthology.orchestra.run.smithi001.stdout:4464 writesame 0x1cc85a1 thru 0x1ccabda data_size 0x15(0x263a bytes) 2024-09-14T22:37:33.313 INFO:teuthology.orchestra.run.smithi001.stdout:4465 read 0x60278c9 thru 0x602abb6 (0x32ee bytes) 2024-09-14T22:37:33.313 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:33.325 INFO:teuthology.orchestra.run.smithi001.stdout:4467 trunc from 0x7eb1ea0 to 0xc60b21b 2024-09-14T22:37:33.338 INFO:teuthology.orchestra.run.smithi001.stdout:4468 read 0x6a318bf thru 0x6a3205c (0x79e bytes) 2024-09-14T22:37:33.339 INFO:teuthology.orchestra.run.smithi001.stdout:4469 punch from 0x218df23 to 0x219a732, (0xc80f bytes) 2024-09-14T22:37:33.339 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:33.339 INFO:teuthology.orchestra.run.smithi001.stdout:4471 punch from 0x6a7486b to 0x6a79277, (0x4a0c bytes) 2024-09-14T22:37:33.339 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:33.355 INFO:teuthology.orchestra.run.smithi001.stdout:4473 write 0xe41f60d thru 0xe42d99f (0xe393 bytes) 2024-09-14T22:37:33.362 INFO:teuthology.orchestra.run.smithi001.stdout:4475 write 0xbc7f9f thru 0xbd599c (0xd9fe bytes) 2024-09-14T22:37:33.366 INFO:teuthology.orchestra.run.smithi001.stdout:4476 writesame 0x8d2f08b thru 0x8d3f06a data_size 0xb2(0xffe0 bytes) 2024-09-14T22:37:33.371 INFO:teuthology.orchestra.run.smithi001.stdout:4478 read 0xa5cd48c thru 0xa5d4d23 (0x7898 bytes) 2024-09-14T22:37:33.371 INFO:teuthology.orchestra.run.smithi001.stdout:4479 punch from 0x763a8fd to 0x763cd0b, (0x240e bytes) 2024-09-14T22:37:33.371 INFO:teuthology.orchestra.run.smithi001.stdout:4480 writesame 0x6de005d thru 0x6dea8b9 data_size 0x3(0xa85d bytes) 2024-09-14T22:37:33.394 INFO:teuthology.orchestra.run.smithi001.stdout:4482 write 0x33d3683 thru 0x33dfc65 (0xc5e3 bytes) 2024-09-14T22:37:33.402 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:33.395+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:33.409 INFO:teuthology.orchestra.run.smithi001.stdout:4483 write 0xebd2bee thru 0xebd6e7a (0x428d bytes) 2024-09-14T22:37:33.414 INFO:teuthology.orchestra.run.smithi001.stdout:4484 write 0xb865e44 thru 0xb86619a (0x357 bytes) 2024-09-14T22:37:33.419 INFO:teuthology.orchestra.run.smithi001.stdout:4486 write 0xc2ddde5 thru 0xc2e5938 (0x7b54 bytes) 2024-09-14T22:37:33.425 INFO:teuthology.orchestra.run.smithi001.stdout:4487 read 0x974d858 thru 0x975d638 (0xfde1 bytes) 2024-09-14T22:37:33.425 INFO:teuthology.orchestra.run.smithi001.stdout:4488 punch from 0xe38b4c9 to 0xe38cd0f, (0x1846 bytes) 2024-09-14T22:37:33.425 INFO:teuthology.orchestra.run.smithi001.stdout:4489 write 0x4c98f78 thru 0x4ca06b5 (0x773e bytes) 2024-09-14T22:37:33.431 INFO:teuthology.orchestra.run.smithi001.stdout:4490 write 0x3da9914 thru 0x3dabb4d (0x223a bytes) 2024-09-14T22:37:33.436 INFO:teuthology.orchestra.run.smithi001.stdout:4491 punch from 0xb21633e to 0xb218048, (0x1d0a bytes) 2024-09-14T22:37:33.436 INFO:teuthology.orchestra.run.smithi001.stdout:4492 write 0xeb9598f thru 0xeba0486 (0xaaf8 bytes) 2024-09-14T22:37:33.439 INFO:teuthology.orchestra.run.smithi001.stdout:4494 read 0x74ba28d thru 0x74c9849 (0xf5bd bytes) 2024-09-14T22:37:33.439 INFO:teuthology.orchestra.run.smithi001.stdout:4496 read 0x81fc904 thru 0x81fd91d (0x101a bytes) 2024-09-14T22:37:33.454 INFO:teuthology.orchestra.run.smithi001.stdout:4498 trunc from 0xebd6e7b to 0x9cd0eb8 2024-09-14T22:37:33.475 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:33.475 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:33.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:33.475 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:33.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.476 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6395 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8386560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:33.476 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:37:33.476 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 69.537240178999994, 2024-09-14T22:37:33.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054534379000000001, 2024-09-14T22:37:33.476 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:33.476 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:33.476 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:33.476 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:33.476 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:33.476 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:33.476 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6395 2024-09-14T22:37:33.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.477 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:33.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:33.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:37:33.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:33.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:37:33.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:33.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933833+0000", 2024-09-14T22:37:33.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3859999999999999e-06 2024-09-14T22:37:33.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:33.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933838+0000", 2024-09-14T22:37:33.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8709999999999996e-06 2024-09-14T22:37:33.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:33.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933854+0000", 2024-09-14T22:37:33.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6189e-05 2024-09-14T22:37:33.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:33.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933871+0000", 2024-09-14T22:37:33.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.626e-05 2024-09-14T22:37:33.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:33.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988198+0000", 2024-09-14T22:37:33.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054327581999999999 2024-09-14T22:37:33.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:33.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988274+0000", 2024-09-14T22:37:33.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5759000000000004e-05 2024-09-14T22:37:33.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.480 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:33.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988366+0000", 2024-09-14T22:37:33.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2331999999999996e-05 2024-09-14T22:37:33.480 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.480 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:33.480 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.480 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104944 3.3 3:c3780f3b:::rbd_data.113ca561f7a9.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:33.480 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:37:33.480 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.437638306, 2024-09-14T22:37:33.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073041087000000005, 2024-09-14T22:37:33.481 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:33.481 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:33.481 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:33.481 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:33.481 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:33.481 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:33.481 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104944 2024-09-14T22:37:33.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.481 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:33.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:33.481 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:37:33.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:33.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:37:33.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:33.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033436+0000", 2024-09-14T22:37:33.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2060000000000001e-06 2024-09-14T22:37:33.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:33.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033448+0000", 2024-09-14T22:37:33.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.169e-05 2024-09-14T22:37:33.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:33.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033474+0000", 2024-09-14T22:37:33.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6400000000000001e-05 2024-09-14T22:37:33.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:33.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033489+0000", 2024-09-14T22:37:33.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5067e-05 2024-09-14T22:37:33.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.484 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:33.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033658+0000", 2024-09-14T22:37:33.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000168565 2024-09-14T22:37:33.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.484 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:33.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033793+0000", 2024-09-14T22:37:33.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013536900000000001 2024-09-14T22:37:33.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.484 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:37:33.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.047559+0000", 2024-09-14T22:37:33.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013766294 2024-09-14T22:37:33.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.485 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:33.485 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106157+0000", 2024-09-14T22:37:33.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058597110000000001 2024-09-14T22:37:33.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.485 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:33.485 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106290+0000", 2024-09-14T22:37:33.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000133244 2024-09-14T22:37:33.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.485 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:33.485 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106396+0000", 2024-09-14T22:37:33.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106246 2024-09-14T22:37:33.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.486 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:33.486 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106475+0000", 2024-09-14T22:37:33.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8895999999999998e-05 2024-09-14T22:37:33.486 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.486 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:33.486 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.486 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:111858 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 10403840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:33.486 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:37:33.486 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6471851849999997, 2024-09-14T22:37:33.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054398798999999998, 2024-09-14T22:37:33.486 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:33.487 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:33.487 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:33.487 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:33.487 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:33.487 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:33.487 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 111858 2024-09-14T22:37:33.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.487 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:33.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:33.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:37:33.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.488 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:33.488 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:37:33.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.488 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:33.488 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823888+0000", 2024-09-14T22:37:33.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0200000000000001e-07 2024-09-14T22:37:33.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.488 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:33.488 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823927+0000", 2024-09-14T22:37:33.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8940999999999999e-05 2024-09-14T22:37:33.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:33.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823940+0000", 2024-09-14T22:37:33.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3682e-05 2024-09-14T22:37:33.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:33.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823950+0000", 2024-09-14T22:37:33.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7110000000000007e-06 2024-09-14T22:37:33.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:33.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878202+0000", 2024-09-14T22:37:33.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054251969999999997 2024-09-14T22:37:33.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:33.490 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878249+0000", 2024-09-14T22:37:33.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6854999999999998e-05 2024-09-14T22:37:33.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:33.490 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878286+0000", 2024-09-14T22:37:33.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6838000000000002e-05 2024-09-14T22:37:33.490 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.490 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:33.490 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.491 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:111859 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 10452992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:33.491 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:37:33.491 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6470960659999996, 2024-09-14T22:37:33.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054435292000000003, 2024-09-14T22:37:33.491 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:33.491 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:33.491 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:33.491 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:33.491 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:33.491 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:33.491 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 111859 2024-09-14T22:37:33.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.491 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:33.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.492 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:33.492 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:37:33.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.492 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:33.492 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:37:33.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.492 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:33.492 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823977+0000", 2024-09-14T22:37:33.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3500000000000005e-07 2024-09-14T22:37:33.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.493 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:33.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823981+0000", 2024-09-14T22:37:33.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2050000000000004e-06 2024-09-14T22:37:33.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.493 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:33.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824007+0000", 2024-09-14T22:37:33.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.62e-05 2024-09-14T22:37:33.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.493 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:33.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824018+0000", 2024-09-14T22:37:33.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0389e-05 2024-09-14T22:37:33.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.494 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:33.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878312+0000", 2024-09-14T22:37:33.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054294754000000001 2024-09-14T22:37:33.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.494 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:33.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878350+0000", 2024-09-14T22:37:33.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7982000000000003e-05 2024-09-14T22:37:33.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.494 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:33.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878411+0000", 2024-09-14T22:37:33.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0927000000000001e-05 2024-09-14T22:37:33.495 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.495 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:33.495 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.495 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.495 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112030 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6316032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:33.495 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:37:33.495 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6470417240000002, 2024-09-14T22:37:33.495 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054597826000000002, 2024-09-14T22:37:33.495 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:33.495 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:33.495 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:33.495 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:33.495 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:33.495 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:33.495 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112030 2024-09-14T22:37:33.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.496 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:33.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.496 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:33.496 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:37:33.496 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.496 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:33.496 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:37:33.496 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.497 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:33.497 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824031+0000", 2024-09-14T22:37:33.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3600000000000003e-07 2024-09-14T22:37:33.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.497 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:33.497 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824036+0000", 2024-09-14T22:37:33.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7350000000000004e-06 2024-09-14T22:37:33.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.497 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:33.497 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824049+0000", 2024-09-14T22:37:33.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3329000000000001e-05 2024-09-14T22:37:33.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:33.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824057+0000", 2024-09-14T22:37:33.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2460000000000003e-06 2024-09-14T22:37:33.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:33.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878464+0000", 2024-09-14T22:37:33.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054406207999999998 2024-09-14T22:37:33.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:33.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878554+0000", 2024-09-14T22:37:33.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0199999999999997e-05 2024-09-14T22:37:33.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:33.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878628+0000", 2024-09-14T22:37:33.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4371999999999999e-05 2024-09-14T22:37:33.499 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.499 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:33.499 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.499 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112031 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6365184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:33.499 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:37:33.499 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6470034059999996, 2024-09-14T22:37:33.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054741868999999999, 2024-09-14T22:37:33.500 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:33.500 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:33.500 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:33.500 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:33.500 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:33.500 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:33.500 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112031 2024-09-14T22:37:33.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.500 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:33.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.500 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:33.500 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:37:33.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.501 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:33.501 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:37:33.501 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.501 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.501 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:33.501 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824070+0000", 2024-09-14T22:37:33.501 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5099999999999998e-07 2024-09-14T22:37:33.501 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.501 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:33.501 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824073+0000", 2024-09-14T22:37:33.501 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6940000000000001e-06 2024-09-14T22:37:33.501 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.502 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:33.502 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824084+0000", 2024-09-14T22:37:33.502 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0380000000000001e-05 2024-09-14T22:37:33.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.502 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:33.502 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824091+0000", 2024-09-14T22:37:33.502 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5730000000000002e-06 2024-09-14T22:37:33.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.502 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:33.502 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878670+0000", 2024-09-14T22:37:33.502 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054579176 2024-09-14T22:37:33.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.503 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:33.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878742+0000", 2024-09-14T22:37:33.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T22:37:33.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.503 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:33.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878811+0000", 2024-09-14T22:37:33.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8170999999999995e-05 2024-09-14T22:37:33.503 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.503 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:33.503 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.503 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112032 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6414336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:33.503 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:37:33.504 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6469695399999997, 2024-09-14T22:37:33.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054931916999999997, 2024-09-14T22:37:33.504 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:33.504 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:33.504 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:33.504 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:33.504 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:33.504 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:33.504 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112032 2024-09-14T22:37:33.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.504 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:33.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:33.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:37:33.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:33.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:37:33.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:33.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:37:33.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:37:33.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:33.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824107+0000", 2024-09-14T22:37:33.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5640000000000001e-06 2024-09-14T22:37:33.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.506 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:33.506 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824117+0000", 2024-09-14T22:37:33.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0366e-05 2024-09-14T22:37:33.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.506 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:33.506 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824158+0000", 2024-09-14T22:37:33.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0670000000000002e-05 2024-09-14T22:37:33.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.506 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:33.506 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878853+0000", 2024-09-14T22:37:33.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054694853000000002 2024-09-14T22:37:33.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.507 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:33.507 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878934+0000", 2024-09-14T22:37:33.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1754000000000004e-05 2024-09-14T22:37:33.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.507 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:33.507 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879034+0000", 2024-09-14T22:37:33.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000100115 2024-09-14T22:37:33.507 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.507 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:33.507 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.508 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112033 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6463488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:33.508 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:37:33.508 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6469008819999997, 2024-09-14T22:37:33.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054991076999999999, 2024-09-14T22:37:33.508 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:33.508 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:33.508 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:33.508 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:33.508 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:33.508 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:33.508 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112033 2024-09-14T22:37:33.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.508 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:33.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:33.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:37:33.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:33.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:37:33.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:33.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824172+0000", 2024-09-14T22:37:33.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9999999999999996e-07 2024-09-14T22:37:33.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:33.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824177+0000", 2024-09-14T22:37:33.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5310000000000003e-06 2024-09-14T22:37:33.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:33.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824188+0000", 2024-09-14T22:37:33.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1235e-05 2024-09-14T22:37:33.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:33.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824197+0000", 2024-09-14T22:37:33.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7069999999999998e-06 2024-09-14T22:37:33.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.511 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:33.511 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879075+0000", 2024-09-14T22:37:33.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054878749999999997 2024-09-14T22:37:33.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.511 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:33.511 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879118+0000", 2024-09-14T22:37:33.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2749e-05 2024-09-14T22:37:33.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.511 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:33.511 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879162+0000", 2024-09-14T22:37:33.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4205000000000001e-05 2024-09-14T22:37:33.511 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.512 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:33.512 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.512 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112034 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6512640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:33.512 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:37:33.512 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6468639310000004, 2024-09-14T22:37:33.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055181246000000003, 2024-09-14T22:37:33.512 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:33.512 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:33.512 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:33.512 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:33.512 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:33.512 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:33.513 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112034 2024-09-14T22:37:33.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.513 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:33.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:33.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:37:33.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:33.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:37:33.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:33.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824209+0000", 2024-09-14T22:37:33.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5499999999999998e-07 2024-09-14T22:37:33.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:33.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824212+0000", 2024-09-14T22:37:33.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6119999999999999e-06 2024-09-14T22:37:33.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:33.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824224+0000", 2024-09-14T22:37:33.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1127999999999999e-05 2024-09-14T22:37:33.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:33.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824232+0000", 2024-09-14T22:37:33.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4700000000000002e-06 2024-09-14T22:37:33.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:33.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879209+0000", 2024-09-14T22:37:33.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054977201000000003 2024-09-14T22:37:33.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:33.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879308+0000", 2024-09-14T22:37:33.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8419999999999996e-05 2024-09-14T22:37:33.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.516 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:33.516 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879389+0000", 2024-09-14T22:37:33.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1760000000000003e-05 2024-09-14T22:37:33.516 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.516 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:33.516 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.516 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112035 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6561792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:33.516 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:37:33.516 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.646770203, 2024-09-14T22:37:33.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055300522999999997, 2024-09-14T22:37:33.516 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:33.517 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:33.517 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:33.517 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:33.517 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:33.517 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:33.517 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112035 2024-09-14T22:37:33.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.517 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:33.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.517 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:33.517 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:37:33.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:33.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:37:33.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:33.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824303+0000", 2024-09-14T22:37:33.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5099999999999998e-07 2024-09-14T22:37:33.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:33.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824308+0000", 2024-09-14T22:37:33.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6689999999999997e-06 2024-09-14T22:37:33.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:33.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824744+0000", 2024-09-14T22:37:33.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00043601600000000001 2024-09-14T22:37:33.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:33.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824756+0000", 2024-09-14T22:37:33.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2476999999999999e-05 2024-09-14T22:37:33.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:33.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879434+0000", 2024-09-14T22:37:33.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054677630999999997 2024-09-14T22:37:33.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:33.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879518+0000", 2024-09-14T22:37:33.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4220000000000003e-05 2024-09-14T22:37:33.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:33.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879602+0000", 2024-09-14T22:37:33.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4559000000000001e-05 2024-09-14T22:37:33.520 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.520 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:33.520 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.521 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112036 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6610944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:33.521 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:37:33.521 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.646278465, 2024-09-14T22:37:33.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055036320999999999, 2024-09-14T22:37:33.521 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:33.521 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:33.521 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:33.521 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:33.521 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:33.521 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:33.521 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112036 2024-09-14T22:37:33.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.521 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:33.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:33.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:37:33.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:33.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:37:33.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:33.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824795+0000", 2024-09-14T22:37:33.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-06 2024-09-14T22:37:33.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:33.523 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824802+0000", 2024-09-14T22:37:33.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6680000000000004e-06 2024-09-14T22:37:33.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:33.523 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824814+0000", 2024-09-14T22:37:33.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2102e-05 2024-09-14T22:37:33.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:33.523 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824823+0000", 2024-09-14T22:37:33.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5550000000000008e-06 2024-09-14T22:37:33.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:33.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879648+0000", 2024-09-14T22:37:33.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054825167000000001 2024-09-14T22:37:33.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:33.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879747+0000", 2024-09-14T22:37:33.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8968999999999994e-05 2024-09-14T22:37:33.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:33.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879830+0000", 2024-09-14T22:37:33.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2633999999999995e-05 2024-09-14T22:37:33.525 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.525 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:33.525 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.525 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112037 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6660096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:33.525 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:37:33.525 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6462357560000003, 2024-09-14T22:37:33.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055122926000000003, 2024-09-14T22:37:33.525 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:33.525 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:33.525 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:33.525 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:33.525 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:33.526 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:33.526 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112037 2024-09-14T22:37:33.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.526 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:33.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.526 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:33.526 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:37:33.526 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.526 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:33.526 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:37:33.526 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:33.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824837+0000", 2024-09-14T22:37:33.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5499999999999998e-07 2024-09-14T22:37:33.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:33.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824840+0000", 2024-09-14T22:37:33.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0429999999999999e-06 2024-09-14T22:37:33.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:33.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824851+0000", 2024-09-14T22:37:33.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1046999999999999e-05 2024-09-14T22:37:33.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.528 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:33.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824859+0000", 2024-09-14T22:37:33.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1200000000000002e-06 2024-09-14T22:37:33.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.528 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:33.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879867+0000", 2024-09-14T22:37:33.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055007697000000001 2024-09-14T22:37:33.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.528 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:33.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879910+0000", 2024-09-14T22:37:33.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3402000000000002e-05 2024-09-14T22:37:33.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:33.529 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879959+0000", 2024-09-14T22:37:33.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9061999999999999e-05 2024-09-14T22:37:33.529 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.529 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:33.529 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.529 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112038 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6709248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:33.529 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:37:33.529 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6462010600000001, 2024-09-14T22:37:33.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055312828000000001, 2024-09-14T22:37:33.530 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:33.530 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:33.530 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:33.530 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:33.530 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:33.530 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:33.530 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112038 2024-09-14T22:37:33.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.530 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:33.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:33.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:37:33.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:33.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:37:33.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:33.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824872+0000", 2024-09-14T22:37:33.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T22:37:33.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:33.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824875+0000", 2024-09-14T22:37:33.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7519999999999998e-06 2024-09-14T22:37:33.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.532 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:33.532 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824917+0000", 2024-09-14T22:37:33.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1047999999999997e-05 2024-09-14T22:37:33.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.532 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:33.532 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824926+0000", 2024-09-14T22:37:33.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3749999999999992e-06 2024-09-14T22:37:33.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.532 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:33.532 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880006+0000", 2024-09-14T22:37:33.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055080525999999998 2024-09-14T22:37:33.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:33.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880102+0000", 2024-09-14T22:37:33.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5995999999999994e-05 2024-09-14T22:37:33.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:33.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880184+0000", 2024-09-14T22:37:33.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1478999999999997e-05 2024-09-14T22:37:33.533 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.533 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:33.533 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.533 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112039 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6758400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:33.534 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:37:33.534 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.64613239, 2024-09-14T22:37:33.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055449709999999999, 2024-09-14T22:37:33.534 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:33.534 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:33.534 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:33.534 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:33.534 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:33.534 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:33.534 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112039 2024-09-14T22:37:33.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.534 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:33.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.534 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:33.534 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:37:33.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:33.535 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:37:33.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:33.535 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824941+0000", 2024-09-14T22:37:33.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3699999999999999e-07 2024-09-14T22:37:33.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:33.535 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824946+0000", 2024-09-14T22:37:33.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0209999999999999e-06 2024-09-14T22:37:33.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:33.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824972+0000", 2024-09-14T22:37:33.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6750999999999998e-05 2024-09-14T22:37:33.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:33.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824982+0000", 2024-09-14T22:37:33.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5019999999999998e-06 2024-09-14T22:37:33.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:33.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880227+0000", 2024-09-14T22:37:33.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055245257999999998 2024-09-14T22:37:33.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:33.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880305+0000", 2024-09-14T22:37:33.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7424999999999998e-05 2024-09-14T22:37:33.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:33.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880389+0000", 2024-09-14T22:37:33.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4815999999999996e-05 2024-09-14T22:37:33.537 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.537 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:33.537 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.538 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112040 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6807552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:33.538 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:37:33.538 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6460756740000004, 2024-09-14T22:37:33.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055534615000000002, 2024-09-14T22:37:33.538 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:33.538 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:33.538 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:33.538 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:33.538 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:33.538 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:33.538 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112040 2024-09-14T22:37:33.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.538 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:33.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:33.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:37:33.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:33.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:37:33.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:33.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824997+0000", 2024-09-14T22:37:33.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7199999999999998e-07 2024-09-14T22:37:33.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:33.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825042+0000", 2024-09-14T22:37:33.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5065999999999998e-05 2024-09-14T22:37:33.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:33.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825060+0000", 2024-09-14T22:37:33.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7520000000000002e-05 2024-09-14T22:37:33.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:33.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825070+0000", 2024-09-14T22:37:33.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8230000000000006e-06 2024-09-14T22:37:33.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:33.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880427+0000", 2024-09-14T22:37:33.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055357244 2024-09-14T22:37:33.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:33.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880483+0000", 2024-09-14T22:37:33.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5807000000000003e-05 2024-09-14T22:37:33.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:33.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880531+0000", 2024-09-14T22:37:33.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8383000000000001e-05 2024-09-14T22:37:33.542 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.542 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:33.542 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.542 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112041 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6856704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:33.542 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:37:33.542 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6459865000000002, 2024-09-14T22:37:33.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055573887000000002, 2024-09-14T22:37:33.542 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:33.543 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:33.543 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:33.543 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:33.543 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:33.543 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:33.543 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112041 2024-09-14T22:37:33.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.543 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:33.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:33.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:37:33.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.544 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:33.544 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:37:33.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.544 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:33.544 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825087+0000", 2024-09-14T22:37:33.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2399999999999996e-07 2024-09-14T22:37:33.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.544 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:33.544 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825091+0000", 2024-09-14T22:37:33.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9729999999999999e-06 2024-09-14T22:37:33.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:33.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825101+0000", 2024-09-14T22:37:33.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.093e-05 2024-09-14T22:37:33.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:33.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825110+0000", 2024-09-14T22:37:33.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1450000000000008e-06 2024-09-14T22:37:33.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:33.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880559+0000", 2024-09-14T22:37:33.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055449684999999999 2024-09-14T22:37:33.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:33.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880601+0000", 2024-09-14T22:37:33.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1248000000000002e-05 2024-09-14T22:37:33.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:33.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880659+0000", 2024-09-14T22:37:33.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8981999999999999e-05 2024-09-14T22:37:33.546 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.547 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:33.547 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.547 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.547 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112042 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6905856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:33.547 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:37:33.547 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6459517659999996, 2024-09-14T22:37:33.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055781415000000001, 2024-09-14T22:37:33.547 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:33.547 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:33.547 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:33.547 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:33.547 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:33.547 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:33.548 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112042 2024-09-14T22:37:33.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.548 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:33.548 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.548 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:33.548 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:37:33.548 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.548 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.548 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:33.548 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:37:33.548 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.548 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.549 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:33.549 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825478+0000", 2024-09-14T22:37:33.549 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035775 2024-09-14T22:37:33.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.549 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:33.549 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825494+0000", 2024-09-14T22:37:33.549 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5994999999999998e-05 2024-09-14T22:37:33.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.549 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:33.549 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825520+0000", 2024-09-14T22:37:33.549 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5738999999999999e-05 2024-09-14T22:37:33.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.550 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:33.550 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825547+0000", 2024-09-14T22:37:33.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7667000000000001e-05 2024-09-14T22:37:33.550 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.550 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:33.550 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880725+0000", 2024-09-14T22:37:33.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055177962999999997 2024-09-14T22:37:33.550 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.550 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:33.550 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880816+0000", 2024-09-14T22:37:33.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0315000000000005e-05 2024-09-14T22:37:33.551 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.551 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:33.551 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880902+0000", 2024-09-14T22:37:33.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5986e-05 2024-09-14T22:37:33.551 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.551 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:33.551 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.551 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.551 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112043 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:33.551 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:37:33.551 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6455037619999997, 2024-09-14T22:37:33.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055537912000000002, 2024-09-14T22:37:33.552 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:33.552 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:33.552 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:33.552 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:33.552 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:33.552 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:33.552 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112043 2024-09-14T22:37:33.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.552 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:33.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.552 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:33.552 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:37:33.552 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.553 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.553 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:33.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:37:33.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.553 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.553 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:33.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825911+0000", 2024-09-14T22:37:33.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00034306899999999999 2024-09-14T22:37:33.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.553 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.553 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:33.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825918+0000", 2024-09-14T22:37:33.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.002e-06 2024-09-14T22:37:33.554 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.554 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:33.554 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825931+0000", 2024-09-14T22:37:33.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2401e-05 2024-09-14T22:37:33.554 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.554 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:33.554 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825942+0000", 2024-09-14T22:37:33.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0903e-05 2024-09-14T22:37:33.554 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.554 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:33.554 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880951+0000", 2024-09-14T22:37:33.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055009379999999997 2024-09-14T22:37:33.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.555 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.555 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:33.555 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881027+0000", 2024-09-14T22:37:33.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5944000000000003e-05 2024-09-14T22:37:33.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.555 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.555 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:33.555 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881106+0000", 2024-09-14T22:37:33.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9213000000000002e-05 2024-09-14T22:37:33.555 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.555 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:33.555 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.556 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.556 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112044 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 7004160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:33.556 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:37:33.556 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6451100929999996, 2024-09-14T22:37:33.556 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055265416999999997, 2024-09-14T22:37:33.556 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:33.556 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:33.556 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:33.556 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:33.556 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:33.556 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:33.556 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112044 2024-09-14T22:37:33.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.557 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:33.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.557 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:33.557 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:37:33.557 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.557 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.557 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:33.557 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:37:33.557 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.557 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.557 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:33.557 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826239+0000", 2024-09-14T22:37:33.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027676000000000003 2024-09-14T22:37:33.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.558 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:33.558 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826247+0000", 2024-09-14T22:37:33.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7440000000000007e-06 2024-09-14T22:37:33.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.558 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:33.558 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826275+0000", 2024-09-14T22:37:33.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8053000000000001e-05 2024-09-14T22:37:33.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.559 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:33.559 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826286+0000", 2024-09-14T22:37:33.559 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1780000000000001e-05 2024-09-14T22:37:33.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.559 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.559 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:33.559 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881137+0000", 2024-09-14T22:37:33.559 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054850500000000003 2024-09-14T22:37:33.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.559 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.559 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:33.559 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881181+0000", 2024-09-14T22:37:33.559 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4153000000000002e-05 2024-09-14T22:37:33.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.560 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.560 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:33.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881227+0000", 2024-09-14T22:37:33.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6427000000000001e-05 2024-09-14T22:37:33.560 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.560 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:33.560 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.560 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.560 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.560 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112045 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 7053312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:33.560 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:37:33.560 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6447478599999998, 2024-09-14T22:37:33.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055045050999999998, 2024-09-14T22:37:33.560 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:33.560 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:33.561 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:33.561 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:33.561 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:33.561 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:33.561 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112045 2024-09-14T22:37:33.561 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.561 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:33.561 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.561 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:33.561 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:37:33.561 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.561 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.561 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.561 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:33.562 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:37:33.562 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:33.562 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.562 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.562 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:33.562 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830250+0000", 2024-09-14T22:37:33.562 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0039258670000000004 2024-09-14T22:37:33.562 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.562 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.562 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:33.562 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830260+0000", 2024-09-14T22:37:33.562 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0088000000000001e-05 2024-09-14T22:37:33.562 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.562 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.563 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:33.563 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830281+0000", 2024-09-14T22:37:33.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0720000000000002e-05 2024-09-14T22:37:33.563 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.563 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:33.563 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830292+0000", 2024-09-14T22:37:33.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1455e-05 2024-09-14T22:37:33.563 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.563 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:33.563 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881255+0000", 2024-09-14T22:37:33.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.050962247000000002 2024-09-14T22:37:33.563 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.564 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:33.564 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881296+0000", 2024-09-14T22:37:33.564 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1557999999999998e-05 2024-09-14T22:37:33.564 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:33.564 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:33.564 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:33.564 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881369+0000", 2024-09-14T22:37:33.564 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3115999999999996e-05 2024-09-14T22:37:33.564 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.564 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:33.564 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.564 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:33.564 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:33.564 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:33.566 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:37:33.567 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:33.559+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:33.567 INFO:teuthology.orchestra.run.smithi001.stdout:4499 write 0xd3b6aa2 thru 0xd3b9a16 (0x2f75 bytes) 2024-09-14T22:37:33.567 INFO:teuthology.orchestra.run.smithi001.stdout:4500 punch from 0xd2f88e0 to 0xd301fb0, (0x96d0 bytes) 2024-09-14T22:37:33.567 INFO:teuthology.orchestra.run.smithi001.stdout:4501 write 0x7ebd219 thru 0x7ec86e5 (0xb4cd bytes) 2024-09-14T22:37:33.567 INFO:teuthology.orchestra.run.smithi001.stdout:4502 trunc from 0xd3b9a17 to 0xb34cd94 2024-09-14T22:37:33.568 INFO:teuthology.orchestra.run.smithi001.stdout:4505 write 0xaae7566 thru 0xaaf3d2a (0xc7c5 bytes) 2024-09-14T22:37:33.568 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:33.568 INFO:teuthology.orchestra.run.smithi001.stdout:4507 write 0x56015d2 thru 0x560ee50 (0xd87f bytes) 2024-09-14T22:37:33.568 INFO:teuthology.orchestra.run.smithi001.stdout:4508 read 0x4cc1cb5 thru 0x4cc43cc (0x2718 bytes) 2024-09-14T22:37:33.568 INFO:teuthology.orchestra.run.smithi001.stdout:4509 read 0x68e230f thru 0x68e30de (0xdd0 bytes) 2024-09-14T22:37:33.568 INFO:teuthology.orchestra.run.smithi001.stdout:4510 read 0x74affeb thru 0x74b5cd8 (0x5cee bytes) 2024-09-14T22:37:33.576 INFO:teuthology.orchestra.run.smithi001.stdout:4512 trunc from 0xb34cd94 to 0x898bf3 2024-09-14T22:37:33.613 INFO:teuthology.orchestra.run.smithi001.stdout:4513 read 0x2bf473 thru 0x2c0372 (0xf00 bytes) 2024-09-14T22:37:33.633 INFO:teuthology.orchestra.run.smithi001.stdout:4515 write 0x2cea9e6 thru 0x2cfa54b (0xfb66 bytes) 2024-09-14T22:37:33.641 INFO:teuthology.orchestra.run.smithi001.stdout:4516 trunc from 0x2cfa54c to 0x366893d 2024-09-14T22:37:33.653 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:33.666 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:33.659+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:33.677 INFO:teuthology.orchestra.run.smithi001.stdout:4521 writesame 0x7f1a772 thru 0x7f274f7 data_size 0x25(0xcd86 bytes) 2024-09-14T22:37:33.678 INFO:teuthology.orchestra.run.smithi001.stdout:4522 read 0xeb5c71 thru 0xec3d0a (0xe09a bytes) 2024-09-14T22:37:33.683 INFO:teuthology.orchestra.run.smithi001.stdout:4525 trunc from 0x7f274f8 to 0x95ded56 2024-09-14T22:37:33.693 INFO:teuthology.orchestra.run.smithi001.stdout:4527 write 0x7b601b thru 0x7b7c44 (0x1c2a bytes) 2024-09-14T22:37:33.698 INFO:teuthology.orchestra.run.smithi001.stdout:4528 trunc from 0x95ded56 to 0x89ff7e6 2024-09-14T22:37:33.709 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:33.722 INFO:teuthology.orchestra.run.smithi001.stdout:4530 writesame 0x8cefd33 thru 0x8cfa56a data_size 0x1c(0xa838 bytes) 2024-09-14T22:37:33.722 INFO:teuthology.orchestra.run.smithi001.stdout:4531 read 0x1e44507 thru 0x1e45778 (0x1272 bytes) 2024-09-14T22:37:33.722 INFO:teuthology.orchestra.run.smithi001.stdout:4532 write 0x24bafa0 thru 0x24c3fc9 (0x902a bytes) 2024-09-14T22:37:33.739 INFO:teuthology.orchestra.run.smithi001.stdout:4533 trunc from 0x8cfa56b to 0xbcd483c 2024-09-14T22:37:33.749 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:33.749 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:33.749 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:37:33.749 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:33.752 INFO:teuthology.orchestra.run.smithi001.stdout:4534 write 0x78ec58d thru 0x78f2024 (0x5a98 bytes) 2024-09-14T22:37:33.764 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:37:33.779 INFO:teuthology.orchestra.run.smithi001.stdout:4535 writesame 0xebd2ac5 thru 0xebe08a0 data_size 0x6(0xdddc bytes) 2024-09-14T22:37:33.792 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.3 to 1 (10000) 2024-09-14T22:37:33.807 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:33.803+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:33.808 INFO:teuthology.orchestra.run.smithi001.stdout:4536 trunc from 0xebe08a1 to 0x2f7ed25 2024-09-14T22:37:33.823 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-14T22:37:33.835 INFO:teuthology.orchestra.run.smithi001.stdout:4537 write 0x2cc7e4a thru 0x2cd0e8f (0x9046 bytes) 2024-09-14T22:37:33.837 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:33.845 INFO:teuthology.orchestra.run.smithi001.stdout:4539 write 0x36fb097 thru 0x3702389 (0x72f3 bytes) 2024-09-14T22:37:33.851 INFO:teuthology.orchestra.run.smithi001.stdout:4540 read 0x1ca8038 thru 0x1caa041 (0x200a bytes) 2024-09-14T22:37:33.851 INFO:teuthology.orchestra.run.smithi001.stdout:4541 read 0x284623f thru 0x2854a60 (0xe822 bytes) 2024-09-14T22:37:33.851 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:33.853 INFO:teuthology.orchestra.run.smithi001.stdout:4546 trunc from 0x370238a to 0x24a4250 2024-09-14T22:37:33.867 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:33.863+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:33.871 INFO:teuthology.orchestra.run.smithi001.stdout:4547 trunc from 0x24a4250 to 0x29ac6a7 2024-09-14T22:37:33.905 INFO:teuthology.orchestra.run.smithi001.stdout:4548 write 0xc1e5057 thru 0xc1e5bcb (0xb75 bytes) 2024-09-14T22:37:33.912 INFO:teuthology.orchestra.run.smithi001.stdout:4549 read 0xb524d4c thru 0xb531d69 (0xd01e bytes) 2024-09-14T22:37:33.912 INFO:teuthology.orchestra.run.smithi001.stdout:4550 punch from 0x4a98f28 to 0x4aa732d, (0xe405 bytes) 2024-09-14T22:37:33.912 INFO:teuthology.orchestra.run.smithi001.stdout:4551 write 0xb2a3859 thru 0xb2ad90f (0xa0b7 bytes) 2024-09-14T22:37:33.918 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:33.918 INFO:teuthology.orchestra.run.smithi001.stdout:4555 write 0x4f8a6a5 thru 0x4f97c43 (0xd59f bytes) 2024-09-14T22:37:33.924 INFO:teuthology.orchestra.run.smithi001.stdout:4556 read 0x1673b0c thru 0x16836c3 (0xfbb8 bytes) 2024-09-14T22:37:33.938 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:33.938 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:33.938 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:37:33.938 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:37:33.938 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:33.952 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:37:33.967 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:33.963+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:34.068 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:34.067+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:34.120 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:34.120 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:34.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:34.120 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:34.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.120 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97197 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 796672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:34.121 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:37:34.121 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.045048294000001, 2024-09-14T22:37:34.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12434473, 2024-09-14T22:37:34.121 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:34.121 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:34.121 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:34.121 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:34.121 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:34.121 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:34.121 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97197 2024-09-14T22:37:34.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.122 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:34.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:34.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:37:34.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:34.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:37:34.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:34.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070524+0000", 2024-09-14T22:37:34.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0299999999999998e-07 2024-09-14T22:37:34.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:34.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070527+0000", 2024-09-14T22:37:34.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9569999999999998e-06 2024-09-14T22:37:34.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:34.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070535+0000", 2024-09-14T22:37:34.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7630000000000006e-06 2024-09-14T22:37:34.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:34.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070543+0000", 2024-09-14T22:37:34.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4869999999999996e-06 2024-09-14T22:37:34.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:34.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194525+0000", 2024-09-14T22:37:34.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123982096 2024-09-14T22:37:34.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:34.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194699+0000", 2024-09-14T22:37:34.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000174105 2024-09-14T22:37:34.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:34.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194868+0000", 2024-09-14T22:37:34.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016861900000000001 2024-09-14T22:37:34.125 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.125 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:34.125 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.125 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97198 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 805888~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:34.125 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:37:34.125 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.045019918000001, 2024-09-14T22:37:34.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.1248088, 2024-09-14T22:37:34.125 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:34.126 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:34.126 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:34.126 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:34.126 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:34.126 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:34.126 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97198 2024-09-14T22:37:34.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.126 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:34.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.126 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:34.126 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:37:34.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:34.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:37:34.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:34.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070552+0000", 2024-09-14T22:37:34.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T22:37:34.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:34.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070555+0000", 2024-09-14T22:37:34.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.853e-06 2024-09-14T22:37:34.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:34.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070565+0000", 2024-09-14T22:37:34.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.048e-05 2024-09-14T22:37:34.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:34.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070573+0000", 2024-09-14T22:37:34.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7200000000000006e-06 2024-09-14T22:37:34.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:34.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194954+0000", 2024-09-14T22:37:34.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12438089400000001 2024-09-14T22:37:34.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:34.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195145+0000", 2024-09-14T22:37:34.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019091299999999999 2024-09-14T22:37:34.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:34.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195360+0000", 2024-09-14T22:37:34.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021527300000000001 2024-09-14T22:37:34.130 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.130 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:34.130 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.130 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97199 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 815104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:34.130 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:37:34.130 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.044989320999999, 2024-09-14T22:37:34.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125251901, 2024-09-14T22:37:34.131 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:34.131 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:34.131 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:34.131 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:34.131 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:34.131 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:34.131 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97199 2024-09-14T22:37:34.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.131 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:34.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:34.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:37:34.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:34.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:37:34.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:34.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070583+0000", 2024-09-14T22:37:34.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6300000000000005e-07 2024-09-14T22:37:34.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:34.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070586+0000", 2024-09-14T22:37:34.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8270000000000002e-06 2024-09-14T22:37:34.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:34.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070597+0000", 2024-09-14T22:37:34.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1960000000000001e-05 2024-09-14T22:37:34.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:34.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070605+0000", 2024-09-14T22:37:34.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6459999999999993e-06 2024-09-14T22:37:34.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:34.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195432+0000", 2024-09-14T22:37:34.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124826926 2024-09-14T22:37:34.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:34.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195626+0000", 2024-09-14T22:37:34.135 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019357099999999999 2024-09-14T22:37:34.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.135 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.135 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:34.135 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195834+0000", 2024-09-14T22:37:34.135 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020830800000000001 2024-09-14T22:37:34.135 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.135 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:34.135 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.135 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.135 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97200 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 824320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:34.135 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:37:34.135 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.044957275000002, 2024-09-14T22:37:34.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125693477, 2024-09-14T22:37:34.136 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:34.136 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:34.136 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:34.136 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:34.136 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:34.136 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:34.136 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97200 2024-09-14T22:37:34.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.136 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:34.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:34.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:37:34.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:34.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:37:34.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:34.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070615+0000", 2024-09-14T22:37:34.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T22:37:34.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:34.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070618+0000", 2024-09-14T22:37:34.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.943e-06 2024-09-14T22:37:34.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:34.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070627+0000", 2024-09-14T22:37:34.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1099999999999992e-06 2024-09-14T22:37:34.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:34.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070634+0000", 2024-09-14T22:37:34.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:37:34.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.139 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:34.139 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195908+0000", 2024-09-14T22:37:34.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125273893 2024-09-14T22:37:34.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.139 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:34.139 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196097+0000", 2024-09-14T22:37:34.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018953899999999999 2024-09-14T22:37:34.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.139 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:34.139 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196308+0000", 2024-09-14T22:37:34.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021027799999999999 2024-09-14T22:37:34.139 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.139 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:34.140 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.140 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97201 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 833536~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:34.140 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:37:34.140 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.044927088000001, 2024-09-14T22:37:34.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126133826, 2024-09-14T22:37:34.140 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:34.140 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:34.140 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:34.140 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:34.140 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:34.140 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:34.141 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97201 2024-09-14T22:37:34.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.141 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:34.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:34.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:37:34.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:34.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:37:34.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:34.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074109+0000", 2024-09-14T22:37:34.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003464834 2024-09-14T22:37:34.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:34.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074134+0000", 2024-09-14T22:37:34.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5170000000000001e-05 2024-09-14T22:37:34.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.143 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:34.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074166+0000", 2024-09-14T22:37:34.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1347000000000003e-05 2024-09-14T22:37:34.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.143 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:34.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074179+0000", 2024-09-14T22:37:34.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.364e-05 2024-09-14T22:37:34.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.143 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:34.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196377+0000", 2024-09-14T22:37:34.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12219793900000001 2024-09-14T22:37:34.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:34.144 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196568+0000", 2024-09-14T22:37:34.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019097999999999999 2024-09-14T22:37:34.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:34.144 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196778+0000", 2024-09-14T22:37:34.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020991599999999999 2024-09-14T22:37:34.144 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.144 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:34.144 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.145 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97205 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 870400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:34.145 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:37:34.145 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.041257524999999, 2024-09-14T22:37:34.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12436148299999999, 2024-09-14T22:37:34.145 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:34.145 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:34.145 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:34.145 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:34.145 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:34.145 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:34.145 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97205 2024-09-14T22:37:34.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.145 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:34.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:34.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:37:34.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:34.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:37:34.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:34.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074315+0000", 2024-09-14T22:37:34.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9299999999999997e-07 2024-09-14T22:37:34.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:34.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074318+0000", 2024-09-14T22:37:34.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.19e-06 2024-09-14T22:37:34.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:34.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074327+0000", 2024-09-14T22:37:34.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8529999999999998e-06 2024-09-14T22:37:34.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:34.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074334+0000", 2024-09-14T22:37:34.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3869999999999997e-06 2024-09-14T22:37:34.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.148 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:34.148 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198365+0000", 2024-09-14T22:37:34.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124031222 2024-09-14T22:37:34.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.148 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:34.148 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198513+0000", 2024-09-14T22:37:34.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014772299999999999 2024-09-14T22:37:34.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.148 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:34.148 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198675+0000", 2024-09-14T22:37:34.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016241500000000001 2024-09-14T22:37:34.149 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.149 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:34.149 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.149 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97206 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 879616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:34.149 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:37:34.149 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.041226069, 2024-09-14T22:37:34.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124706284, 2024-09-14T22:37:34.149 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:34.149 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:34.149 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:34.149 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:34.150 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:34.150 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:34.150 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97206 2024-09-14T22:37:34.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.150 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:34.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:34.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:37:34.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:34.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:37:34.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:34.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074346+0000", 2024-09-14T22:37:34.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8999999999999996e-07 2024-09-14T22:37:34.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:34.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074349+0000", 2024-09-14T22:37:34.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.96e-06 2024-09-14T22:37:34.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:34.152 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074358+0000", 2024-09-14T22:37:34.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9169999999999995e-06 2024-09-14T22:37:34.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.152 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:34.152 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074365+0000", 2024-09-14T22:37:34.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3699999999999997e-06 2024-09-14T22:37:34.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.152 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:34.152 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198761+0000", 2024-09-14T22:37:34.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124395743 2024-09-14T22:37:34.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:34.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198899+0000", 2024-09-14T22:37:34.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013763499999999999 2024-09-14T22:37:34.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:34.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199052+0000", 2024-09-14T22:37:34.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015296900000000001 2024-09-14T22:37:34.153 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.153 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:34.153 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.154 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97207 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 888832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:34.154 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:37:34.154 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.041196855999999, 2024-09-14T22:37:34.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124988433, 2024-09-14T22:37:34.154 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:34.154 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:34.154 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:34.154 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:34.154 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:34.154 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:34.154 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97207 2024-09-14T22:37:34.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.154 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:34.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:34.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:37:34.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:34.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:37:34.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:34.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074375+0000", 2024-09-14T22:37:34.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:37:34.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:34.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074378+0000", 2024-09-14T22:37:34.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1829999999999998e-06 2024-09-14T22:37:34.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:34.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074389+0000", 2024-09-14T22:37:34.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0737000000000001e-05 2024-09-14T22:37:34.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:34.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074396+0000", 2024-09-14T22:37:34.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2559999999999996e-06 2024-09-14T22:37:34.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:34.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199139+0000", 2024-09-14T22:37:34.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12474253 2024-09-14T22:37:34.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:34.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199240+0000", 2024-09-14T22:37:34.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101221 2024-09-14T22:37:34.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:34.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199363+0000", 2024-09-14T22:37:34.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000122822 2024-09-14T22:37:34.158 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.158 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:34.158 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.158 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97208 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 898048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:34.158 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:37:34.158 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.041165551999999, 2024-09-14T22:37:34.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12524854899999999, 2024-09-14T22:37:34.158 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:34.158 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:34.158 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:34.158 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:34.159 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:34.159 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:34.159 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97208 2024-09-14T22:37:34.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.159 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:34.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:34.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:37:34.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:34.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:37:34.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:34.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:37:34.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:37:34.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:34.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074410+0000", 2024-09-14T22:37:34.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.18e-06 2024-09-14T22:37:34.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:34.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074435+0000", 2024-09-14T22:37:34.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5273e-05 2024-09-14T22:37:34.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.161 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:34.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074443+0000", 2024-09-14T22:37:34.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3000000000000002e-06 2024-09-14T22:37:34.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.161 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:34.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199433+0000", 2024-09-14T22:37:34.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124989502 2024-09-14T22:37:34.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.161 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:34.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199535+0000", 2024-09-14T22:37:34.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102557 2024-09-14T22:37:34.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:34.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199654+0000", 2024-09-14T22:37:34.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119061 2024-09-14T22:37:34.162 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.162 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:34.162 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.162 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97209 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 907264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:34.162 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:37:34.163 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.041117747000001, 2024-09-14T22:37:34.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125487354, 2024-09-14T22:37:34.163 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:34.163 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:34.163 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:34.163 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:34.163 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:34.163 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:34.163 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97209 2024-09-14T22:37:34.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.163 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:34.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:34.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:37:34.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:34.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:37:34.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:34.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:37:34.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9400000000000005e-07 2024-09-14T22:37:34.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:34.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074458+0000", 2024-09-14T22:37:34.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3299999999999999e-06 2024-09-14T22:37:34.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:34.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074467+0000", 2024-09-14T22:37:34.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.346e-06 2024-09-14T22:37:34.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:34.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074474+0000", 2024-09-14T22:37:34.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.374e-06 2024-09-14T22:37:34.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:34.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199720+0000", 2024-09-14T22:37:34.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12524561200000001 2024-09-14T22:37:34.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:34.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199820+0000", 2024-09-14T22:37:34.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010049599999999999 2024-09-14T22:37:34.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:34.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199941+0000", 2024-09-14T22:37:34.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012050200000000001 2024-09-14T22:37:34.167 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.167 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:34.167 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.167 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97210 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:34.167 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:37:34.167 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.041087752999999, 2024-09-14T22:37:34.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125744942, 2024-09-14T22:37:34.168 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:34.168 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:34.168 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:34.168 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:34.168 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:34.168 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:34.168 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97210 2024-09-14T22:37:34.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.168 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:34.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:34.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:37:34.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:34.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:37:34.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:34.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:37:34.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6599999999999996e-07 2024-09-14T22:37:34.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:34.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074487+0000", 2024-09-14T22:37:34.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8140000000000001e-06 2024-09-14T22:37:34.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:34.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074496+0000", 2024-09-14T22:37:34.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9330000000000007e-06 2024-09-14T22:37:34.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:34.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074503+0000", 2024-09-14T22:37:34.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4100000000000002e-06 2024-09-14T22:37:34.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:34.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200009+0000", 2024-09-14T22:37:34.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12550539899999999 2024-09-14T22:37:34.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:34.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200108+0000", 2024-09-14T22:37:34.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9182e-05 2024-09-14T22:37:34.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:34.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200229+0000", 2024-09-14T22:37:34.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120538 2024-09-14T22:37:34.171 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.171 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:34.171 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.172 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97211 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 925696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:34.172 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:37:34.172 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.041056792999999, 2024-09-14T22:37:34.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12599848299999999, 2024-09-14T22:37:34.172 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:34.172 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:34.172 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:34.172 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:34.172 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:34.172 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:34.172 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97211 2024-09-14T22:37:34.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.173 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:34.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:34.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:37:34.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:34.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:37:34.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:34.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074516+0000", 2024-09-14T22:37:34.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.42e-06 2024-09-14T22:37:34.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:34.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074521+0000", 2024-09-14T22:37:34.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7960000000000003e-06 2024-09-14T22:37:34.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:34.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074538+0000", 2024-09-14T22:37:34.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6900000000000001e-05 2024-09-14T22:37:34.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:34.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074555+0000", 2024-09-14T22:37:34.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.696e-05 2024-09-14T22:37:34.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:34.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200293+0000", 2024-09-14T22:37:34.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12573864100000001 2024-09-14T22:37:34.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:34.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200393+0000", 2024-09-14T22:37:34.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9264000000000004e-05 2024-09-14T22:37:34.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:34.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200513+0000", 2024-09-14T22:37:34.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012050200000000001 2024-09-14T22:37:34.176 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.176 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:34.176 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.176 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97212 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 934912~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:34.176 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:37:34.176 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.040982171, 2024-09-14T22:37:34.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12621127300000001, 2024-09-14T22:37:34.177 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:34.177 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:34.177 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:34.177 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:34.177 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:34.177 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:34.177 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97212 2024-09-14T22:37:34.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.177 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:34.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:34.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:37:34.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:34.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:37:34.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:34.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074590+0000", 2024-09-14T22:37:34.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1969999999999999e-06 2024-09-14T22:37:34.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:34.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074596+0000", 2024-09-14T22:37:34.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7629999999999997e-06 2024-09-14T22:37:34.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:34.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074614+0000", 2024-09-14T22:37:34.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7643e-05 2024-09-14T22:37:34.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:34.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074635+0000", 2024-09-14T22:37:34.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.075e-05 2024-09-14T22:37:34.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:34.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200580+0000", 2024-09-14T22:37:34.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125945482 2024-09-14T22:37:34.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:34.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200683+0000", 2024-09-14T22:37:34.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010267099999999999 2024-09-14T22:37:34.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:34.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200800+0000", 2024-09-14T22:37:34.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117767 2024-09-14T22:37:34.180 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.180 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:34.181 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.181 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97213 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 944128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:34.181 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:37:34.181 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.040916821, 2024-09-14T22:37:34.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126430709, 2024-09-14T22:37:34.181 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:34.181 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:34.181 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:34.181 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:34.181 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:34.181 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:34.182 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97213 2024-09-14T22:37:34.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.182 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:34.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:34.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:37:34.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:34.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:37:34.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:34.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078210+0000", 2024-09-14T22:37:34.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0035556839999999999 2024-09-14T22:37:34.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:34.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078231+0000", 2024-09-14T22:37:34.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0716e-05 2024-09-14T22:37:34.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:34.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078260+0000", 2024-09-14T22:37:34.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9473000000000001e-05 2024-09-14T22:37:34.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:34.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078279+0000", 2024-09-14T22:37:34.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8600000000000001e-05 2024-09-14T22:37:34.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:34.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200864+0000", 2024-09-14T22:37:34.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.122585295 2024-09-14T22:37:34.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:34.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200965+0000", 2024-09-14T22:37:34.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101004 2024-09-14T22:37:34.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:34.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.201085+0000", 2024-09-14T22:37:34.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119937 2024-09-14T22:37:34.185 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.185 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:34.185 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.185 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97219 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 999424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:34.185 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:37:34.185 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.036934544000001, 2024-09-14T22:37:34.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124138558, 2024-09-14T22:37:34.186 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:34.186 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:34.186 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:34.186 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:34.186 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:34.186 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:34.186 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97219 2024-09-14T22:37:34.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.186 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:34.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:34.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:37:34.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:34.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:37:34.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:34.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078638+0000", 2024-09-14T22:37:34.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.173e-06 2024-09-14T22:37:34.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:34.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078644+0000", 2024-09-14T22:37:34.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2570000000000004e-06 2024-09-14T22:37:34.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:34.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078672+0000", 2024-09-14T22:37:34.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7322999999999999e-05 2024-09-14T22:37:34.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:34.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078687+0000", 2024-09-14T22:37:34.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5639999999999999e-05 2024-09-14T22:37:34.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:34.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202567+0000", 2024-09-14T22:37:34.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123880134 2024-09-14T22:37:34.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:34.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202670+0000", 2024-09-14T22:37:34.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010273899999999999 2024-09-14T22:37:34.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:34.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202775+0000", 2024-09-14T22:37:34.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010529199999999999 2024-09-14T22:37:34.189 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.189 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:34.189 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.190 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97220 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1008640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:34.190 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:37:34.190 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.036865311, 2024-09-14T22:37:34.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12435937, 2024-09-14T22:37:34.190 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:34.190 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:34.190 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:34.190 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:34.190 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:34.190 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:34.190 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97220 2024-09-14T22:37:34.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.191 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:34.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:34.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:37:34.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:34.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:37:34.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:34.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078707+0000", 2024-09-14T22:37:34.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.266e-06 2024-09-14T22:37:34.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:34.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078713+0000", 2024-09-14T22:37:34.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7270000000000004e-06 2024-09-14T22:37:34.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:34.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078737+0000", 2024-09-14T22:37:34.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3640000000000001e-05 2024-09-14T22:37:34.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:34.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078751+0000", 2024-09-14T22:37:34.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.45e-05 2024-09-14T22:37:34.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:34.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202841+0000", 2024-09-14T22:37:34.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124089938 2024-09-14T22:37:34.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:34.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202944+0000", 2024-09-14T22:37:34.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010334 2024-09-14T22:37:34.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:34.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203065+0000", 2024-09-14T22:37:34.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012095900000000001 2024-09-14T22:37:34.194 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.194 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:34.194 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.194 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97221 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1017856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:34.195 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:37:34.195 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.036799548000001, 2024-09-14T22:37:34.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124574005, 2024-09-14T22:37:34.195 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:34.195 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:34.195 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:34.195 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:34.195 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:34.195 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:34.195 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97221 2024-09-14T22:37:34.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.195 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:34.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:34.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:37:34.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:34.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:37:34.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:34.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078773+0000", 2024-09-14T22:37:34.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.13e-06 2024-09-14T22:37:34.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:34.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078779+0000", 2024-09-14T22:37:34.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0260000000000004e-06 2024-09-14T22:37:34.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:34.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078797+0000", 2024-09-14T22:37:34.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.806e-05 2024-09-14T22:37:34.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:34.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078810+0000", 2024-09-14T22:37:34.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.308e-05 2024-09-14T22:37:34.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:34.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203131+0000", 2024-09-14T22:37:34.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12432077499999999 2024-09-14T22:37:34.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:34.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203227+0000", 2024-09-14T22:37:34.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6575000000000003e-05 2024-09-14T22:37:34.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:34.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203346+0000", 2024-09-14T22:37:34.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118359 2024-09-14T22:37:34.199 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.199 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:34.199 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.199 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97222 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:34.199 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:37:34.199 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.036743178999998, 2024-09-14T22:37:34.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12480110799999999, 2024-09-14T22:37:34.199 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:34.199 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:34.199 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:34.200 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:34.200 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:34.200 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:34.200 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97222 2024-09-14T22:37:34.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.200 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:34.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:34.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:37:34.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:34.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:37:34.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:34.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078829+0000", 2024-09-14T22:37:34.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.057e-06 2024-09-14T22:37:34.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:34.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078836+0000", 2024-09-14T22:37:34.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6159999999999998e-06 2024-09-14T22:37:34.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:34.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078856+0000", 2024-09-14T22:37:34.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0100000000000001e-05 2024-09-14T22:37:34.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:34.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078869+0000", 2024-09-14T22:37:34.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.272e-05 2024-09-14T22:37:34.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:34.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203412+0000", 2024-09-14T22:37:34.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12454283300000001 2024-09-14T22:37:34.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:34.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203511+0000", 2024-09-14T22:37:34.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9893000000000001e-05 2024-09-14T22:37:34.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:34.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203629+0000", 2024-09-14T22:37:34.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117889 2024-09-14T22:37:34.203 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.203 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:34.203 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.204 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97223 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1036288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:34.204 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:37:34.204 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.036371192000001, 2024-09-14T22:37:34.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124706279, 2024-09-14T22:37:34.204 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:34.204 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:34.204 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:34.204 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:34.204 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:34.204 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:34.204 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97223 2024-09-14T22:37:34.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.204 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:34.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:34.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:37:34.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:34.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:37:34.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:34.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079202+0000", 2024-09-14T22:37:34.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8330000000000001e-06 2024-09-14T22:37:34.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:34.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079212+0000", 2024-09-14T22:37:34.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0207000000000001e-05 2024-09-14T22:37:34.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:34.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079238+0000", 2024-09-14T22:37:34.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5626000000000002e-05 2024-09-14T22:37:34.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:34.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079256+0000", 2024-09-14T22:37:34.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8386999999999999e-05 2024-09-14T22:37:34.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:34.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203691+0000", 2024-09-14T22:37:34.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124434538 2024-09-14T22:37:34.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:34.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203786+0000", 2024-09-14T22:37:34.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5344000000000001e-05 2024-09-14T22:37:34.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:34.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203906+0000", 2024-09-14T22:37:34.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120344 2024-09-14T22:37:34.208 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.208 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:34.208 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.208 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97224 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1045504~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:34.208 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:37:34.208 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.036287372, 2024-09-14T22:37:34.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12513321199999999, 2024-09-14T22:37:34.208 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:34.208 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:34.208 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:34.208 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:34.209 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:34.209 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:34.209 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97224 2024-09-14T22:37:34.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.209 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:34.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:34.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:37:34.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:34.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:37:34.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:34.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:34.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082581+0000", 2024-09-14T22:37:34.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0032971530000000001 2024-09-14T22:37:34.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:34.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082592+0000", 2024-09-14T22:37:34.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1345999999999999e-05 2024-09-14T22:37:34.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:34.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082618+0000", 2024-09-14T22:37:34.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5873000000000001e-05 2024-09-14T22:37:34.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:34.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082638+0000", 2024-09-14T22:37:34.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9565999999999999e-05 2024-09-14T22:37:34.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:34.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204004+0000", 2024-09-14T22:37:34.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.121366038 2024-09-14T22:37:34.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:34.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204188+0000", 2024-09-14T22:37:34.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018427 2024-09-14T22:37:34.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:34.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:34.212 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:34.212 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204417+0000", 2024-09-14T22:37:34.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022896599999999999 2024-09-14T22:37:34.212 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.212 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:34.212 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.212 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:34.212 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:34.212 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:34.214 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:37:34.214 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:34.211+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:34.310 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:37:34.315 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:34.307+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:34.340 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:34.340 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:34.340 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:37:34.340 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:34.354 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:37:34.439 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:34.435+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:34.464 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:37:34.465 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:37:34.484 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:37:34.484 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:37:34.503 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:37:34.503 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:37:34.516 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:34.516 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:34.516 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:37:34.516 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:37:34.516 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:34.517 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:34.515+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:34.520 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:37:34.520 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:37:34.530 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:37:34.542 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:37:34.542 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:37:34.560 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:37:34.561 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:37:34.581 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:37:34.581 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:37:34.599 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:37:34.600 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:37:34.617 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:34.615+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:34.696 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:34.696 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:34.710 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:37:34.718 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:34.715+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:34.796 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.4 to 1 (10000) 2024-09-14T22:37:34.816 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T22:37:34.818 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:34.811+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:34.822 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-14T22:37:34.834 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:34.834 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:34.848 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:37:34.919 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:34.915+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:35.015 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:35.015 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:35.020 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:35.015+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:35.029 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:37:35.121 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:35.119+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:35.196 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:35.196 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:35.212 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:37:35.222 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:35.215+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:35.322 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:35.315+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:35.342 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:35.342 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:35.356 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:37:35.424 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:35.419+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:35.482 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:35.483 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:35.496 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:37:35.525 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:35.519+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:35.626 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:35.623+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:35.665 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:35.665 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:35.679 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:37:35.764 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:35.763+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:35.805 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2024-09-14T22:37:35.809 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.5 to 1 (10000) 2024-09-14T22:37:35.828 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:35.823+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:35.834 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-14T22:37:35.853 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:35.853 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:35.868 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:37:35.929 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:35.927+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:36.030 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:36.023+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:36.033 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:36.034 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:36.048 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:37:36.130 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:36.127+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:36.176 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:36.176 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:36.189 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:37:36.230 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:36.223+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:36.331 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:36.327+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:36.353 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:36.353 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:36.366 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:37:36.432 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:36.427+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:36.533 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:36.533 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:36.533 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:36.531+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:36.548 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:37:36.595 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:36.595 INFO:teuthology.orchestra.run.smithi001.stdout:4558 read 0x3d4ab0a thru 0x3d5879d (0xdc94 bytes) 2024-09-14T22:37:36.595 INFO:teuthology.orchestra.run.smithi001.stdout:4561 writesame 0x825f1ed thru 0x826bb00 data_size 0x22(0xc914 bytes) 2024-09-14T22:37:36.618 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:37:36.622 INFO:teuthology.orchestra.run.smithi001.stdout:4563 write 0xc955eb3 thru 0xc95b3ce (0x551c bytes) 2024-09-14T22:37:36.633 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:36.627+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:36.643 INFO:teuthology.orchestra.run.smithi001.stdout:4564 writesame 0xe1521a3 thru 0xe15c5b5 data_size 0x27(0xa413 bytes) 2024-09-14T22:37:36.644 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:36.644 INFO:teuthology.orchestra.run.smithi001.stdout:4566 read 0xb08d61d thru 0xb0905b6 (0x2f9a bytes) 2024-09-14T22:37:36.658 INFO:teuthology.orchestra.run.smithi001.stdout:4567 writesame 0xe98f22e thru 0xe99eb7c data_size 0x1(0xf94f bytes) 2024-09-14T22:37:36.658 INFO:teuthology.orchestra.run.smithi001.stdout:4569 read 0xd5cdbda thru 0xd5d455f (0x6986 bytes) 2024-09-14T22:37:36.658 INFO:teuthology.orchestra.run.smithi001.stdout:4570 writesame 0x69a16a3 thru 0x69a711d data_size 0x7(0x5a7b bytes) 2024-09-14T22:37:36.665 INFO:teuthology.orchestra.run.smithi001.stdout:4571 write 0x434e22 thru 0x43b79d (0x697c bytes) 2024-09-14T22:37:36.668 INFO:teuthology.orchestra.run.smithi001.stdout:4572 writesame 0x81889dd thru 0x8197ca1 data_size 0x1(0xf2c5 bytes) 2024-09-14T22:37:36.720 INFO:teuthology.orchestra.run.smithi001.stdout:4573 writesame 0x2bf179f thru 0x2c0133f data_size 0x1(0xfba1 bytes) 2024-09-14T22:37:36.734 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:36.727+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:36.740 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:36.740 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:36.755 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:37:36.781 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:37:36.781 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:37:36.794 INFO:teuthology.orchestra.run.smithi001.stdout:4574 punch from 0x4f7d79c to 0x4f7e742, (0xfa6 bytes) 2024-09-14T22:37:36.794 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:36.798 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:37:36.798 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:37:36.807 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.6 to 1 (10000) 2024-09-14T22:37:36.811 INFO:teuthology.orchestra.run.smithi001.stdout:4578 trunc from 0xe99eb7d to 0x3407bcf 2024-09-14T22:37:36.821 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:37:36.821 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:37:36.834 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-14T22:37:36.835 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:36.831+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:36.840 INFO:teuthology.orchestra.run.smithi001.stdout:4579 punch from 0x26af719 to 0x26bdf01, (0xe7e8 bytes) 2024-09-14T22:37:36.840 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:36.840 INFO:teuthology.orchestra.run.smithi001.stdout:4581 trunc from 0x3407bcf to 0x33f13eb 2024-09-14T22:37:36.841 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:37:36.841 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:37:36.853 INFO:teuthology.orchestra.run.smithi001.stdout:4582 trunc from 0x33f13eb to 0x1ce01b0 2024-09-14T22:37:36.863 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:37:36.863 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:37:36.886 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:37:36.886 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:37:36.902 INFO:teuthology.orchestra.run.smithi001.stdout:4583 write 0xde5afed thru 0xde5d8c8 (0x28dc bytes) 2024-09-14T22:37:36.908 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:37:36.909 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:37:36.909 INFO:teuthology.orchestra.run.smithi001.stdout:4584 write 0xaedbafd thru 0xaee996e (0xde72 bytes) 2024-09-14T22:37:36.915 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:36.915 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:37:36.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:36.916 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4412.0:119520 3.7 e87/27 3:e47edc0d:::rbd_data.113c19a22fb1.000000000000002b:head v 87'334, mlcod=83'332)", 2024-09-14T22:37:36.916 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:36.916128+0000", 2024-09-14T22:37:36.916 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.00058388400000000001, 2024-09-14T22:37:36.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00060653499999999995, 2024-09-14T22:37:36.916 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:36.916 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:36.916 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:36.916 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:36.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:36.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:36.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:36.916128+0000", 2024-09-14T22:37:36.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:36.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:36.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:36.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:36.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:36.916128+0000", 2024-09-14T22:37:36.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:36.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:36.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:36.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:36.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:36.916130+0000", 2024-09-14T22:37:36.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.108e-06 2024-09-14T22:37:36.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:36.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:36.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:36.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:36.916314+0000", 2024-09-14T22:37:36.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000183382 2024-09-14T22:37:36.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:36.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:36.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:36.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:36.916338+0000", 2024-09-14T22:37:36.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4445999999999999e-05 2024-09-14T22:37:36.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:36.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:36.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:36.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:36.916351+0000", 2024-09-14T22:37:36.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2972999999999999e-05 2024-09-14T22:37:36.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:36.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:36.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:36.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:36.916448+0000", 2024-09-14T22:37:36.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6487e-05 2024-09-14T22:37:36.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:36.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:36.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:36.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:36.916467+0000", 2024-09-14T22:37:36.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9735e-05 2024-09-14T22:37:36.919 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:36.919 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:36.919 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:36.919 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:36.919 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:37:36.919 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 1 2024-09-14T22:37:36.919 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:36.920 INFO:teuthology.orchestra.run.smithi001.stdout:4585 read 0xcae83f4 thru 0xcaf64e3 (0xe0f0 bytes) 2024-09-14T22:37:36.920 INFO:teuthology.orchestra.run.smithi001.stdout:4587 punch from 0x3fe0de6 to 0x3fe2a62, (0x1c7c bytes) 2024-09-14T22:37:36.931 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:37:36.931 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:37:36.935 INFO:teuthology.orchestra.run.smithi001.stdout:4588 trunc from 0xde5d8c9 to 0x1d389d0 2024-09-14T22:37:36.936 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:36.931+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:36.942 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:37:36.965 INFO:teuthology.orchestra.run.smithi001.stdout:4589 read 0x144196a thru 0x14444f2 (0x2b89 bytes) 2024-09-14T22:37:36.966 INFO:teuthology.orchestra.run.smithi001.stdout:4590 read 0x156252d thru 0x156c336 (0x9e0a bytes) 2024-09-14T22:37:36.967 INFO:teuthology.orchestra.run.smithi001.stdout:4591 write 0xb6b779 thru 0xb741b9 (0x8a41 bytes) 2024-09-14T22:37:36.986 INFO:teuthology.orchestra.run.smithi001.stdout:4593 trunc from 0x1d389d0 to 0xd6b15d0 2024-09-14T22:37:37.013 INFO:teuthology.orchestra.run.smithi001.stdout:4594 trunc from 0xd6b15d0 to 0x2975495 2024-09-14T22:37:37.034 INFO:teuthology.orchestra.run.smithi001.stdout:4596 punch from 0xcd4a21 to 0xcd6e90, (0x246f bytes) 2024-09-14T22:37:37.034 INFO:teuthology.orchestra.run.smithi001.stdout:4598 read 0x9ac7ef thru 0x9bb711 (0xef23 bytes) 2024-09-14T22:37:37.034 INFO:teuthology.orchestra.run.smithi001.stdout:4599 read 0x208187f thru 0x208e58c (0xcd0e bytes) 2024-09-14T22:37:37.035 INFO:teuthology.orchestra.run.smithi001.stdout:4600 punch from 0x19f5c5c to 0x1a0077c, (0xab20 bytes) 2024-09-14T22:37:37.036 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:37.031+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:37.040 INFO:teuthology.orchestra.run.smithi001.stdout:4602 trunc from 0x2975495 to 0x5b982de 2024-09-14T22:37:37.053 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:37.054 INFO:teuthology.orchestra.run.smithi001.stdout:4605 trunc from 0x5b982de to 0x55c165a 2024-09-14T22:37:37.086 INFO:teuthology.orchestra.run.smithi001.stdout:4607 write 0x851d9ed thru 0x852973c (0xbd50 bytes) 2024-09-14T22:37:37.095 INFO:teuthology.orchestra.run.smithi001.stdout:4608 writesame 0x1848868 thru 0x1849684 data_size 0x1(0xe1d bytes) 2024-09-14T22:37:37.095 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:37.102 INFO:teuthology.orchestra.run.smithi001.stdout:4610 trunc from 0x852973d to 0xc7668aa 2024-09-14T22:37:37.110 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:37.111 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:37:37.111 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:37:37.111 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:37:37.111 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:37.117 INFO:teuthology.orchestra.run.smithi001.stdout:4611 write 0xb4e175c thru 0xb4f081e (0xf0c3 bytes) 2024-09-14T22:37:37.125 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:37:37.126 INFO:teuthology.orchestra.run.smithi001.stdout:4612 read 0x8494f88 thru 0x849a1f5 (0x526e bytes) 2024-09-14T22:37:37.126 INFO:teuthology.orchestra.run.smithi001.stdout:4613 read 0xabe36e4 thru 0xabeeebb (0xb7d8 bytes) 2024-09-14T22:37:37.128 INFO:teuthology.orchestra.run.smithi001.stdout:4614 punch from 0xaa264f1 to 0xaa2c169, (0x5c78 bytes) 2024-09-14T22:37:37.128 INFO:teuthology.orchestra.run.smithi001.stdout:4615 trunc from 0xc7668aa to 0xcb3101e 2024-09-14T22:37:37.141 INFO:teuthology.orchestra.run.smithi001.stdout:4616 writesame 0x4bf8064 thru 0x4bf94b9 data_size 0x13(0x1456 bytes) 2024-09-14T22:37:37.142 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:37.139+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:37.150 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:37.150 INFO:teuthology.orchestra.run.smithi001.stdout:4618 read 0xaf33949 thru 0xaf392a0 (0x5958 bytes) 2024-09-14T22:37:37.150 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:37.150 INFO:teuthology.orchestra.run.smithi001.stdout:4620 read 0x1bd78f1 thru 0x1bd866f (0xd7f bytes) 2024-09-14T22:37:37.151 INFO:teuthology.orchestra.run.smithi001.stdout:4622 write 0x992defa thru 0x993484a (0x6951 bytes) 2024-09-14T22:37:37.158 INFO:teuthology.orchestra.run.smithi001.stdout:4623 clone 26 order 22 su 65536 sc 3 2024-09-14T22:37:37.238 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:37.231+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:37.254 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:37.254 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:37:37.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:37:37.255 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:37:37.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.255 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35306 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1425408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:37.255 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507069+0000", 2024-09-14T22:37:37.255 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 63.745602495999997, 2024-09-14T22:37:37.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045725259999999997, 2024-09-14T22:37:37.255 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:37.255 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:37.255 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:37.255 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:37.255 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:37.255 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.255 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35306 2024-09-14T22:37:37.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.256 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:37.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:37.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507069+0000", 2024-09-14T22:37:37.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:37.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507069+0000", 2024-09-14T22:37:37.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:37.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507070+0000", 2024-09-14T22:37:37.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0400000000000002e-07 2024-09-14T22:37:37.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:37.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507072+0000", 2024-09-14T22:37:37.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.891e-06 2024-09-14T22:37:37.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:37.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507078+0000", 2024-09-14T22:37:37.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0279999999999997e-06 2024-09-14T22:37:37.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507082+0000", 2024-09-14T22:37:37.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4680000000000003e-06 2024-09-14T22:37:37.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:37.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552632+0000", 2024-09-14T22:37:37.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045550144000000001 2024-09-14T22:37:37.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:37.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552707+0000", 2024-09-14T22:37:37.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4148999999999998e-05 2024-09-14T22:37:37.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.259 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:37.259 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552795+0000", 2024-09-14T22:37:37.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8176000000000004e-05 2024-09-14T22:37:37.259 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.259 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:37.259 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.259 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35307 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1454080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:37.259 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:37:37.259 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 63.745583945, 2024-09-14T22:37:37.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045891358, 2024-09-14T22:37:37.260 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:37.260 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:37.260 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:37.260 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:37.260 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:37.260 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.260 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35307 2024-09-14T22:37:37.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.260 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:37.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.260 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:37.261 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:37:37.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.261 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:37.261 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:37:37.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.261 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:37.261 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:37:37.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4700000000000002e-07 2024-09-14T22:37:37.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:37.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507093+0000", 2024-09-14T22:37:37.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9520000000000002e-06 2024-09-14T22:37:37.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:37.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507099+0000", 2024-09-14T22:37:37.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0240000000000003e-06 2024-09-14T22:37:37.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507105+0000", 2024-09-14T22:37:37.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5520000000000004e-06 2024-09-14T22:37:37.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:37.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552826+0000", 2024-09-14T22:37:37.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045721494000000001 2024-09-14T22:37:37.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:37.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552896+0000", 2024-09-14T22:37:37.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9519999999999998e-05 2024-09-14T22:37:37.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:37.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552979+0000", 2024-09-14T22:37:37.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3468999999999997e-05 2024-09-14T22:37:37.264 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.264 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:37.264 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.264 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35308 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1482752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:37.264 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:37:37.264 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 63.745560949000001, 2024-09-14T22:37:37.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046076533000000003, 2024-09-14T22:37:37.264 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:37.264 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:37.265 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:37.265 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:37.265 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:37.265 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.265 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35308 2024-09-14T22:37:37.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.265 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:37.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:37.265 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:37:37.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.266 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:37.266 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:37:37.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.266 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:37.266 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:37:37.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:37:37.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.266 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:37.266 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507113+0000", 2024-09-14T22:37:37.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9099999999999999e-06 2024-09-14T22:37:37.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:37.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507119+0000", 2024-09-14T22:37:37.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7440000000000004e-06 2024-09-14T22:37:37.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507124+0000", 2024-09-14T22:37:37.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2129999999999999e-06 2024-09-14T22:37:37.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:37.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553007+0000", 2024-09-14T22:37:37.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045882488999999999 2024-09-14T22:37:37.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:37.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553076+0000", 2024-09-14T22:37:37.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9233000000000006e-05 2024-09-14T22:37:37.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:37.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553188+0000", 2024-09-14T22:37:37.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111557 2024-09-14T22:37:37.269 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.269 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:37.269 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.269 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35309 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1511424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:37.269 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:37:37.269 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 63.745542043999997, 2024-09-14T22:37:37.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046231869000000002, 2024-09-14T22:37:37.269 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:37.269 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:37.269 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:37.269 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:37.269 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:37.270 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.270 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35309 2024-09-14T22:37:37.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.270 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:37.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:37.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:37:37.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:37.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:37:37.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:37.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:37:37.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9099999999999999e-07 2024-09-14T22:37:37.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:37.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507132+0000", 2024-09-14T22:37:37.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8190000000000001e-06 2024-09-14T22:37:37.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:37.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507139+0000", 2024-09-14T22:37:37.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9419999999999998e-06 2024-09-14T22:37:37.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507145+0000", 2024-09-14T22:37:37.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2600000000000002e-06 2024-09-14T22:37:37.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:37.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553225+0000", 2024-09-14T22:37:37.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046079991000000001 2024-09-14T22:37:37.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:37.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553284+0000", 2024-09-14T22:37:37.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8638999999999999e-05 2024-09-14T22:37:37.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.273 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:37.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553362+0000", 2024-09-14T22:37:37.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7826999999999999e-05 2024-09-14T22:37:37.273 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.273 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:37.273 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.274 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35310 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1540096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:37.274 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507151+0000", 2024-09-14T22:37:37.274 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 63.745520587999998, 2024-09-14T22:37:37.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046444541999999998, 2024-09-14T22:37:37.274 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:37.274 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:37.274 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:37.274 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:37.274 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:37.274 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.274 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35310 2024-09-14T22:37:37.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.275 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:37.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:37.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507151+0000", 2024-09-14T22:37:37.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:37.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507151+0000", 2024-09-14T22:37:37.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:37.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507152+0000", 2024-09-14T22:37:37.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1699999999999999e-07 2024-09-14T22:37:37.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:37.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507154+0000", 2024-09-14T22:37:37.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.902e-06 2024-09-14T22:37:37.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:37.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507159+0000", 2024-09-14T22:37:37.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.823e-06 2024-09-14T22:37:37.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.277 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.277 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507165+0000", 2024-09-14T22:37:37.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.587e-06 2024-09-14T22:37:37.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.277 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:37.277 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553395+0000", 2024-09-14T22:37:37.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046230318999999999 2024-09-14T22:37:37.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.277 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:37.277 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553434+0000", 2024-09-14T22:37:37.278 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8680999999999998e-05 2024-09-14T22:37:37.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.278 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:37.278 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553596+0000", 2024-09-14T22:37:37.278 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016191299999999999 2024-09-14T22:37:37.278 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.278 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:37.278 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.278 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35311 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1568768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:37.278 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:37:37.278 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 63.745501335, 2024-09-14T22:37:37.278 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046623592999999998, 2024-09-14T22:37:37.279 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:37.279 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:37.279 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:37.279 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:37.279 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:37.279 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.279 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35311 2024-09-14T22:37:37.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.279 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:37.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.279 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:37.279 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:37:37.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:37.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:37:37.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:37.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:37:37.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:37:37.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:37.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507176+0000", 2024-09-14T22:37:37.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3349999999999997e-06 2024-09-14T22:37:37.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:37.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507200+0000", 2024-09-14T22:37:37.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4003e-05 2024-09-14T22:37:37.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507206+0000", 2024-09-14T22:37:37.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.976e-06 2024-09-14T22:37:37.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:37.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553627+0000", 2024-09-14T22:37:37.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046420284999999999 2024-09-14T22:37:37.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:37.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553696+0000", 2024-09-14T22:37:37.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9567999999999995e-05 2024-09-14T22:37:37.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:37.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553794+0000", 2024-09-14T22:37:37.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8060000000000006e-05 2024-09-14T22:37:37.282 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.282 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:37.282 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.283 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35566 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1484800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:37.283 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514668+0000", 2024-09-14T22:37:37.283 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 63.738003558000003, 2024-09-14T22:37:37.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045644974999999997, 2024-09-14T22:37:37.283 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:37.283 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:37.283 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:37.283 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:37.283 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:37.283 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.283 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35566 2024-09-14T22:37:37.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.284 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:37.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:37.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514668+0000", 2024-09-14T22:37:37.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:37.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514668+0000", 2024-09-14T22:37:37.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:37.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514669+0000", 2024-09-14T22:37:37.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9099999999999999e-07 2024-09-14T22:37:37.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:37.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514671+0000", 2024-09-14T22:37:37.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3800000000000001e-06 2024-09-14T22:37:37.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:37.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514677+0000", 2024-09-14T22:37:37.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9660000000000001e-06 2024-09-14T22:37:37.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514682+0000", 2024-09-14T22:37:37.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0939999999999998e-06 2024-09-14T22:37:37.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:37.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560133+0000", 2024-09-14T22:37:37.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045450398000000003 2024-09-14T22:37:37.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:37.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560207+0000", 2024-09-14T22:37:37.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4196999999999995e-05 2024-09-14T22:37:37.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:37.287 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560313+0000", 2024-09-14T22:37:37.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106549 2024-09-14T22:37:37.287 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.287 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:37.287 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.287 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35567 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1513472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:37.287 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:37:37.287 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 63.737983018999998, 2024-09-14T22:37:37.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045793215999999998, 2024-09-14T22:37:37.287 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:37.287 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:37.287 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:37.287 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:37.288 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:37.288 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.288 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35567 2024-09-14T22:37:37.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.288 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:37.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.288 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:37.288 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:37:37.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.288 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:37.288 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:37:37.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:37.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:37:37.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1899999999999998e-07 2024-09-14T22:37:37.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:37.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514692+0000", 2024-09-14T22:37:37.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2699999999999999e-06 2024-09-14T22:37:37.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:37.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514698+0000", 2024-09-14T22:37:37.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0939999999999996e-06 2024-09-14T22:37:37.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514703+0000", 2024-09-14T22:37:37.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3870000000000003e-06 2024-09-14T22:37:37.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:37.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560353+0000", 2024-09-14T22:37:37.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045649914 2024-09-14T22:37:37.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:37.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560416+0000", 2024-09-14T22:37:37.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3096999999999996e-05 2024-09-14T22:37:37.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:37.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560482+0000", 2024-09-14T22:37:37.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6035e-05 2024-09-14T22:37:37.291 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.291 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:37.291 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.291 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35568 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1542144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:37.291 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:37:37.291 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 63.737958221, 2024-09-14T22:37:37.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045995921000000002, 2024-09-14T22:37:37.292 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:37.292 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:37.292 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:37.292 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:37.292 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:37.292 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.292 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35568 2024-09-14T22:37:37.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.292 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:37.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.292 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:37.293 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:37:37.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.293 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.293 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:37.293 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:37:37.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.293 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.293 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:37.293 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:37:37.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4099999999999999e-07 2024-09-14T22:37:37.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.293 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:37.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514720+0000", 2024-09-14T22:37:37.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6869999999999999e-06 2024-09-14T22:37:37.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:37.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514727+0000", 2024-09-14T22:37:37.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3540000000000002e-06 2024-09-14T22:37:37.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514732+0000", 2024-09-14T22:37:37.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7690000000000004e-06 2024-09-14T22:37:37.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:37.295 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560523+0000", 2024-09-14T22:37:37.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045791373000000003 2024-09-14T22:37:37.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:37.295 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560606+0000", 2024-09-14T22:37:37.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2690999999999998e-05 2024-09-14T22:37:37.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:37.295 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560710+0000", 2024-09-14T22:37:37.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000103606 2024-09-14T22:37:37.295 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.295 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:37.295 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.296 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35569 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1570816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:37.296 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:37:37.296 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 63.737932411999999, 2024-09-14T22:37:37.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046110630999999999, 2024-09-14T22:37:37.296 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:37.296 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:37.296 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:37.296 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:37.296 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:37.296 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.296 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35569 2024-09-14T22:37:37.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.296 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:37.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:37.297 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:37:37.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:37.297 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:37:37.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:37.297 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:37:37.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.82e-07 2024-09-14T22:37:37.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:37.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514743+0000", 2024-09-14T22:37:37.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.269e-06 2024-09-14T22:37:37.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:37.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514750+0000", 2024-09-14T22:37:37.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7340000000000002e-06 2024-09-14T22:37:37.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514755+0000", 2024-09-14T22:37:37.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7260000000000002e-06 2024-09-14T22:37:37.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:37.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560731+0000", 2024-09-14T22:37:37.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045976377999999998 2024-09-14T22:37:37.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:37.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560777+0000", 2024-09-14T22:37:37.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5868999999999997e-05 2024-09-14T22:37:37.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:37.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560850+0000", 2024-09-14T22:37:37.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3172999999999999e-05 2024-09-14T22:37:37.300 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.300 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:37.300 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.300 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:86180 3.18 3:1d0c52d8:::rbd_data.113cd62ea8a5.000000000000004b:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:37.300 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:01.356417+0000", 2024-09-14T22:37:37.300 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 35.896255316999998, 2024-09-14T22:37:37.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049143245000000002, 2024-09-14T22:37:37.300 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:37.300 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:37.300 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:37.300 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:37.300 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:37.300 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.301 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 86180 2024-09-14T22:37:37.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.301 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:37.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:37.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356417+0000", 2024-09-14T22:37:37.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:37.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356417+0000", 2024-09-14T22:37:37.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:37.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356420+0000", 2024-09-14T22:37:37.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2030000000000001e-06 2024-09-14T22:37:37.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:37.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356435+0000", 2024-09-14T22:37:37.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5287000000000001e-05 2024-09-14T22:37:37.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:37.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356472+0000", 2024-09-14T22:37:37.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7311000000000002e-05 2024-09-14T22:37:37.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356492+0000", 2024-09-14T22:37:37.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9519999999999999e-05 2024-09-14T22:37:37.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:37.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356670+0000", 2024-09-14T22:37:37.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000177567 2024-09-14T22:37:37.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:37.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356924+0000", 2024-09-14T22:37:37.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025399299999999999 2024-09-14T22:37:37.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 0", 2024-09-14T22:37:37.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.365052+0000", 2024-09-14T22:37:37.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0081289380000000005 2024-09-14T22:37:37.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:37.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.405396+0000", 2024-09-14T22:37:37.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040343228000000002 2024-09-14T22:37:37.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:37:37.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.405421+0000", 2024-09-14T22:37:37.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5763e-05 2024-09-14T22:37:37.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:37:37.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.405491+0000", 2024-09-14T22:37:37.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9721999999999994e-05 2024-09-14T22:37:37.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:37.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.405560+0000", 2024-09-14T22:37:37.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8713000000000004e-05 2024-09-14T22:37:37.305 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.306 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:37.306 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.306 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98378 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 919552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:37.306 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148183+0000", 2024-09-14T22:37:37.306 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.104489039000001, 2024-09-14T22:37:37.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045614549999999997, 2024-09-14T22:37:37.306 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:37.306 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:37.306 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:37.306 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:37.306 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:37.306 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.307 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98378 2024-09-14T22:37:37.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.307 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:37.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:37.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148183+0000", 2024-09-14T22:37:37.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:37.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148183+0000", 2024-09-14T22:37:37.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:37.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148183+0000", 2024-09-14T22:37:37.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0800000000000001e-07 2024-09-14T22:37:37.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:37.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148184+0000", 2024-09-14T22:37:37.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2780000000000001e-06 2024-09-14T22:37:37.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:37.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148189+0000", 2024-09-14T22:37:37.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2699999999999998e-06 2024-09-14T22:37:37.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148191+0000", 2024-09-14T22:37:37.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7659999999999999e-06 2024-09-14T22:37:37.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:37.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193708+0000", 2024-09-14T22:37:37.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045516562000000003 2024-09-14T22:37:37.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:37.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193751+0000", 2024-09-14T22:37:37.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2811000000000001e-05 2024-09-14T22:37:37.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:37.311 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193797+0000", 2024-09-14T22:37:37.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6655e-05 2024-09-14T22:37:37.311 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.311 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:37.311 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.311 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98379 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 928768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:37.311 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148195+0000", 2024-09-14T22:37:37.311 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.104477209999999, 2024-09-14T22:37:37.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045817405999999998, 2024-09-14T22:37:37.312 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:37.312 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:37.312 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:37.312 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:37.312 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:37.312 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.312 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98379 2024-09-14T22:37:37.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.312 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:37.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.312 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:37.312 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148195+0000", 2024-09-14T22:37:37.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:37.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148195+0000", 2024-09-14T22:37:37.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:37.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148195+0000", 2024-09-14T22:37:37.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3999999999999998e-07 2024-09-14T22:37:37.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:37.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148196+0000", 2024-09-14T22:37:37.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.341e-06 2024-09-14T22:37:37.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:37.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148200+0000", 2024-09-14T22:37:37.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9040000000000002e-06 2024-09-14T22:37:37.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148203+0000", 2024-09-14T22:37:37.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.959e-06 2024-09-14T22:37:37.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.315 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:37.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193841+0000", 2024-09-14T22:37:37.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045637886000000003 2024-09-14T22:37:37.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.315 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:37.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193933+0000", 2024-09-14T22:37:37.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1823999999999994e-05 2024-09-14T22:37:37.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.315 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:37.316 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194012+0000", 2024-09-14T22:37:37.316 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9252000000000006e-05 2024-09-14T22:37:37.316 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.316 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:37.316 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.316 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98380 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 937984~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:37.316 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148206+0000", 2024-09-14T22:37:37.316 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.104465571999999, 2024-09-14T22:37:37.316 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045937354, 2024-09-14T22:37:37.317 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:37.317 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:37.317 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:37.317 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:37.317 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:37.320 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.320 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98380 2024-09-14T22:37:37.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.320 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:37.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.320 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:37.321 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148206+0000", 2024-09-14T22:37:37.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.321 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:37.321 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148206+0000", 2024-09-14T22:37:37.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.321 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:37.321 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148207+0000", 2024-09-14T22:37:37.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1e-07 2024-09-14T22:37:37.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:37.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148221+0000", 2024-09-14T22:37:37.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4679e-05 2024-09-14T22:37:37.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:37.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148228+0000", 2024-09-14T22:37:37.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3820000000000001e-06 2024-09-14T22:37:37.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148231+0000", 2024-09-14T22:37:37.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2849999999999999e-06 2024-09-14T22:37:37.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:37.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194054+0000", 2024-09-14T22:37:37.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045822596 2024-09-14T22:37:37.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:37.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194098+0000", 2024-09-14T22:37:37.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4604000000000002e-05 2024-09-14T22:37:37.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:37.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194144+0000", 2024-09-14T22:37:37.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5597999999999999e-05 2024-09-14T22:37:37.324 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.325 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:37.325 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.325 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98381 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 947200~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:37.325 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148236+0000", 2024-09-14T22:37:37.325 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.104436078999999, 2024-09-14T22:37:37.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046111129000000001, 2024-09-14T22:37:37.325 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:37.325 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:37.325 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:37.325 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:37.326 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:37.326 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.326 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98381 2024-09-14T22:37:37.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.326 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:37.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:37.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148236+0000", 2024-09-14T22:37:37.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:37.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148236+0000", 2024-09-14T22:37:37.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:37.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148236+0000", 2024-09-14T22:37:37.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8799999999999998e-07 2024-09-14T22:37:37.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:37.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148238+0000", 2024-09-14T22:37:37.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6559999999999999e-06 2024-09-14T22:37:37.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:37.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148246+0000", 2024-09-14T22:37:37.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3440000000000001e-06 2024-09-14T22:37:37.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148251+0000", 2024-09-14T22:37:37.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5480000000000003e-06 2024-09-14T22:37:37.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:37.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194187+0000", 2024-09-14T22:37:37.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045935847000000002 2024-09-14T22:37:37.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:37.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194267+0000", 2024-09-14T22:37:37.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0267999999999998e-05 2024-09-14T22:37:37.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:37.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194347+0000", 2024-09-14T22:37:37.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0178000000000004e-05 2024-09-14T22:37:37.329 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.330 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:37.330 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.330 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98382 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 956416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:37.330 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148254+0000", 2024-09-14T22:37:37.330 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.104417707, 2024-09-14T22:37:37.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046224227, 2024-09-14T22:37:37.330 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:37.330 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:37.330 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:37.330 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:37.330 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:37.330 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.331 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98382 2024-09-14T22:37:37.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.331 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:37.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:37.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148254+0000", 2024-09-14T22:37:37.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:37.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148254+0000", 2024-09-14T22:37:37.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:37.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148254+0000", 2024-09-14T22:37:37.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3099999999999999e-07 2024-09-14T22:37:37.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:37.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148256+0000", 2024-09-14T22:37:37.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5540000000000001e-06 2024-09-14T22:37:37.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:37.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148259+0000", 2024-09-14T22:37:37.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2499999999999998e-06 2024-09-14T22:37:37.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148262+0000", 2024-09-14T22:37:37.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8310000000000001e-06 2024-09-14T22:37:37.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:37.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194387+0000", 2024-09-14T22:37:37.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046125082999999997 2024-09-14T22:37:37.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:37.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194430+0000", 2024-09-14T22:37:37.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3074999999999997e-05 2024-09-14T22:37:37.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:37.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194478+0000", 2024-09-14T22:37:37.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8202999999999999e-05 2024-09-14T22:37:37.334 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.334 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:37.334 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.334 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98383 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 965632~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:37.335 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148265+0000", 2024-09-14T22:37:37.335 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.104406593, 2024-09-14T22:37:37.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046409141000000001, 2024-09-14T22:37:37.335 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:37.335 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:37.335 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:37.335 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:37.335 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:37.335 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.335 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98383 2024-09-14T22:37:37.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.335 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:37.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:37.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148265+0000", 2024-09-14T22:37:37.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:37.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148265+0000", 2024-09-14T22:37:37.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:37.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148266+0000", 2024-09-14T22:37:37.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0800000000000001e-07 2024-09-14T22:37:37.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:37.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148267+0000", 2024-09-14T22:37:37.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1659999999999999e-06 2024-09-14T22:37:37.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:37.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148271+0000", 2024-09-14T22:37:37.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1289999999999998e-06 2024-09-14T22:37:37.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148274+0000", 2024-09-14T22:37:37.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5939999999999998e-06 2024-09-14T22:37:37.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:37.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194523+0000", 2024-09-14T22:37:37.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046248162000000002 2024-09-14T22:37:37.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:37.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194603+0000", 2024-09-14T22:37:37.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0161999999999996e-05 2024-09-14T22:37:37.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:37.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194675+0000", 2024-09-14T22:37:37.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1719999999999998e-05 2024-09-14T22:37:37.339 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.339 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:37.339 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.339 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98384 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 974848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:37.339 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148278+0000", 2024-09-14T22:37:37.339 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.104394265, 2024-09-14T22:37:37.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046527369999999998, 2024-09-14T22:37:37.340 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:37.340 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:37.340 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:37.340 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:37.340 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:37.340 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.340 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98384 2024-09-14T22:37:37.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.340 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:37.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:37.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148278+0000", 2024-09-14T22:37:37.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:37.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148278+0000", 2024-09-14T22:37:37.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:37.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.149987+0000", 2024-09-14T22:37:37.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017093430000000001 2024-09-14T22:37:37.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:37.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.149993+0000", 2024-09-14T22:37:37.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6219999999999997e-06 2024-09-14T22:37:37.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:37.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150006+0000", 2024-09-14T22:37:37.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3095000000000001e-05 2024-09-14T22:37:37.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150014+0000", 2024-09-14T22:37:37.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6340000000000007e-06 2024-09-14T22:37:37.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.343 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:37.343 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194715+0000", 2024-09-14T22:37:37.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044700349 2024-09-14T22:37:37.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.343 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:37.343 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194759+0000", 2024-09-14T22:37:37.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3807000000000003e-05 2024-09-14T22:37:37.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.343 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:37.343 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194805+0000", 2024-09-14T22:37:37.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6520000000000002e-05 2024-09-14T22:37:37.344 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.344 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:37.344 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.344 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98390 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 1030144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:37.344 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.150152+0000", 2024-09-14T22:37:37.344 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.102519577999999, 2024-09-14T22:37:37.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045642708999999997, 2024-09-14T22:37:37.344 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:37.344 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:37.344 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:37.344 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:37.345 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:37.345 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.345 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98390 2024-09-14T22:37:37.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.345 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:37.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:37.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150152+0000", 2024-09-14T22:37:37.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:37.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150152+0000", 2024-09-14T22:37:37.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:37.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150153+0000", 2024-09-14T22:37:37.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.08e-07 2024-09-14T22:37:37.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:37.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150155+0000", 2024-09-14T22:37:37.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2910000000000002e-06 2024-09-14T22:37:37.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:37.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150162+0000", 2024-09-14T22:37:37.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4099999999999996e-06 2024-09-14T22:37:37.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150167+0000", 2024-09-14T22:37:37.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3199999999999999e-06 2024-09-14T22:37:37.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:37.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195704+0000", 2024-09-14T22:37:37.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045537243999999998 2024-09-14T22:37:37.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:37.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195749+0000", 2024-09-14T22:37:37.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4866e-05 2024-09-14T22:37:37.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:37.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195795+0000", 2024-09-14T22:37:37.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.617e-05 2024-09-14T22:37:37.348 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.348 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:37.348 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.348 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98391 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 1039360~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:37.348 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.150175+0000", 2024-09-14T22:37:37.348 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.102496742, 2024-09-14T22:37:37.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045825232, 2024-09-14T22:37:37.349 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:37.349 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:37.349 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:37.349 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:37.349 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:37.349 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.349 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98391 2024-09-14T22:37:37.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.349 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:37.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:37.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150175+0000", 2024-09-14T22:37:37.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:37.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150175+0000", 2024-09-14T22:37:37.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:37.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:37.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150176+0000", 2024-09-14T22:37:37.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9700000000000002e-07 2024-09-14T22:37:37.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:37.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150178+0000", 2024-09-14T22:37:37.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.407e-06 2024-09-14T22:37:37.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:37.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150185+0000", 2024-09-14T22:37:37.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5189999999999998e-06 2024-09-14T22:37:37.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150190+0000", 2024-09-14T22:37:37.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2399999999999998e-06 2024-09-14T22:37:37.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:37.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195838+0000", 2024-09-14T22:37:37.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045648265 2024-09-14T22:37:37.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:37.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195918+0000", 2024-09-14T22:37:37.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0094999999999998e-05 2024-09-14T22:37:37.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:37.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:37.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:37.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.196000+0000", 2024-09-14T22:37:37.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2309000000000001e-05 2024-09-14T22:37:37.353 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.353 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:37.353 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.353 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:37.353 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:37.353 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:37.354 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:37:37.355 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:37.347+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:37.456 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:37.451+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:37.486 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:37.486 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:37.486 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:37:37.486 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:37.500 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:37:37.587 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:37.583+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:37.657 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:37.651+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:37.668 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:37.668 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:37.668 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:37:37.668 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:37:37.668 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:37.682 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:37:37.771 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:37.767+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:37.816 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.7 to 1 (10000) 2024-09-14T22:37:37.832 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T22:37:37.843 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-14T22:37:37.843 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:37:37.858 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:37.851+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:37.862 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:37.862 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:37.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:37.862 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:37.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.862 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6395 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8386560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:37.862 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:37:37.862 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 73.925192248000002, 2024-09-14T22:37:37.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054534379000000001, 2024-09-14T22:37:37.862 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:37.862 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:37.862 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:37.863 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:37.863 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:37.863 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.863 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6395 2024-09-14T22:37:37.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.863 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:37.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:37.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:37:37.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:37.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:37:37.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.864 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:37.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933833+0000", 2024-09-14T22:37:37.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3859999999999999e-06 2024-09-14T22:37:37.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.864 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:37.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933838+0000", 2024-09-14T22:37:37.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8709999999999996e-06 2024-09-14T22:37:37.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.865 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:37.865 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933854+0000", 2024-09-14T22:37:37.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6189e-05 2024-09-14T22:37:37.865 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.865 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.865 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933871+0000", 2024-09-14T22:37:37.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.626e-05 2024-09-14T22:37:37.865 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.865 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:37.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988198+0000", 2024-09-14T22:37:37.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054327581999999999 2024-09-14T22:37:37.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:37.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988274+0000", 2024-09-14T22:37:37.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5759000000000004e-05 2024-09-14T22:37:37.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:37.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988366+0000", 2024-09-14T22:37:37.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2331999999999996e-05 2024-09-14T22:37:37.867 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.867 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:37.867 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.867 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104944 3.3 3:c3780f3b:::rbd_data.113ca561f7a9.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:37.867 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:37:37.867 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.825590375000001, 2024-09-14T22:37:37.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073041087000000005, 2024-09-14T22:37:37.867 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:37.867 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:37.868 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:37.868 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:37.868 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:37.868 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.868 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104944 2024-09-14T22:37:37.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.868 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:37.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:37.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:37:37.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:37.869 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:37:37.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.869 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:37.869 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033436+0000", 2024-09-14T22:37:37.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2060000000000001e-06 2024-09-14T22:37:37.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.869 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:37.869 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033448+0000", 2024-09-14T22:37:37.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.169e-05 2024-09-14T22:37:37.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.870 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:37.870 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033474+0000", 2024-09-14T22:37:37.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6400000000000001e-05 2024-09-14T22:37:37.870 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.870 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.870 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033489+0000", 2024-09-14T22:37:37.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5067e-05 2024-09-14T22:37:37.870 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.870 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:37.870 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033658+0000", 2024-09-14T22:37:37.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000168565 2024-09-14T22:37:37.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:37.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033793+0000", 2024-09-14T22:37:37.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013536900000000001 2024-09-14T22:37:37.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:37:37.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.047559+0000", 2024-09-14T22:37:37.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013766294 2024-09-14T22:37:37.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:37.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106157+0000", 2024-09-14T22:37:37.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058597110000000001 2024-09-14T22:37:37.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:37.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106290+0000", 2024-09-14T22:37:37.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000133244 2024-09-14T22:37:37.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:37.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106396+0000", 2024-09-14T22:37:37.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106246 2024-09-14T22:37:37.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:37.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106475+0000", 2024-09-14T22:37:37.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8895999999999998e-05 2024-09-14T22:37:37.873 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.873 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:37.873 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.873 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:111858 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 10403840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:37.873 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:37:37.873 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.035137254, 2024-09-14T22:37:37.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054398798999999998, 2024-09-14T22:37:37.873 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:37.874 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:37.874 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:37.874 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:37.874 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:37.874 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.874 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 111858 2024-09-14T22:37:37.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.874 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:37.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.874 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:37.874 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:37:37.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.875 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:37.875 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:37:37.875 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.875 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:37.875 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823888+0000", 2024-09-14T22:37:37.875 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0200000000000001e-07 2024-09-14T22:37:37.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.875 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:37.875 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823927+0000", 2024-09-14T22:37:37.875 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8940999999999999e-05 2024-09-14T22:37:37.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:37.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823940+0000", 2024-09-14T22:37:37.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3682e-05 2024-09-14T22:37:37.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823950+0000", 2024-09-14T22:37:37.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7110000000000007e-06 2024-09-14T22:37:37.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:37.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878202+0000", 2024-09-14T22:37:37.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054251969999999997 2024-09-14T22:37:37.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:37.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878249+0000", 2024-09-14T22:37:37.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6854999999999998e-05 2024-09-14T22:37:37.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:37.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878286+0000", 2024-09-14T22:37:37.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6838000000000002e-05 2024-09-14T22:37:37.877 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.878 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:37.878 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.878 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:111859 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 10452992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:37.878 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:37:37.878 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.035048135, 2024-09-14T22:37:37.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054435292000000003, 2024-09-14T22:37:37.878 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:37.878 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:37.878 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:37.878 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:37.878 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:37.878 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.879 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 111859 2024-09-14T22:37:37.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.879 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:37.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.879 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:37.879 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:37:37.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.879 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:37.879 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:37:37.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:37.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823977+0000", 2024-09-14T22:37:37.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3500000000000005e-07 2024-09-14T22:37:37.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:37.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823981+0000", 2024-09-14T22:37:37.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2050000000000004e-06 2024-09-14T22:37:37.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:37.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824007+0000", 2024-09-14T22:37:37.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.62e-05 2024-09-14T22:37:37.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.881 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824018+0000", 2024-09-14T22:37:37.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0389e-05 2024-09-14T22:37:37.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:37.881 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878312+0000", 2024-09-14T22:37:37.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054294754000000001 2024-09-14T22:37:37.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:37.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878350+0000", 2024-09-14T22:37:37.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7982000000000003e-05 2024-09-14T22:37:37.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:37.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878411+0000", 2024-09-14T22:37:37.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0927000000000001e-05 2024-09-14T22:37:37.882 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.882 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:37.882 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.882 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112030 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6316032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:37.882 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:37:37.883 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.034993793, 2024-09-14T22:37:37.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054597826000000002, 2024-09-14T22:37:37.883 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:37.883 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:37.883 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:37.883 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:37.883 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:37.883 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.883 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112030 2024-09-14T22:37:37.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.883 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:37.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.883 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:37.884 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:37:37.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.884 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:37.884 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:37:37.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.884 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:37.884 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824031+0000", 2024-09-14T22:37:37.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3600000000000003e-07 2024-09-14T22:37:37.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:37.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824036+0000", 2024-09-14T22:37:37.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7350000000000004e-06 2024-09-14T22:37:37.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:37.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824049+0000", 2024-09-14T22:37:37.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3329000000000001e-05 2024-09-14T22:37:37.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824057+0000", 2024-09-14T22:37:37.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2460000000000003e-06 2024-09-14T22:37:37.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:37.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878464+0000", 2024-09-14T22:37:37.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054406207999999998 2024-09-14T22:37:37.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:37.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878554+0000", 2024-09-14T22:37:37.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0199999999999997e-05 2024-09-14T22:37:37.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:37.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878628+0000", 2024-09-14T22:37:37.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4371999999999999e-05 2024-09-14T22:37:37.887 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.887 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:37.887 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.887 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112031 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6365184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:37.887 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:37:37.887 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.034955475, 2024-09-14T22:37:37.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054741868999999999, 2024-09-14T22:37:37.887 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:37.887 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:37.887 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:37.887 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:37.888 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:37.888 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.888 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112031 2024-09-14T22:37:37.888 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.888 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:37.888 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.888 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:37.888 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:37:37.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.888 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.888 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.888 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:37.888 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:37:37.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:37.889 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824070+0000", 2024-09-14T22:37:37.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5099999999999998e-07 2024-09-14T22:37:37.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:37.889 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824073+0000", 2024-09-14T22:37:37.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6940000000000001e-06 2024-09-14T22:37:37.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:37.889 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824084+0000", 2024-09-14T22:37:37.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0380000000000001e-05 2024-09-14T22:37:37.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824091+0000", 2024-09-14T22:37:37.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5730000000000002e-06 2024-09-14T22:37:37.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:37.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878670+0000", 2024-09-14T22:37:37.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054579176 2024-09-14T22:37:37.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:37.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878742+0000", 2024-09-14T22:37:37.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T22:37:37.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:37.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878811+0000", 2024-09-14T22:37:37.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8170999999999995e-05 2024-09-14T22:37:37.891 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.891 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:37.891 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.891 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112032 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6414336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:37.892 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:37:37.892 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.034921609, 2024-09-14T22:37:37.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054931916999999997, 2024-09-14T22:37:37.892 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:37.892 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:37.892 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:37.892 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:37.892 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:37.892 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.892 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112032 2024-09-14T22:37:37.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.892 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:37.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:37.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:37:37.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:37.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:37:37.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:37.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:37:37.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:37:37.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.894 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:37.894 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824107+0000", 2024-09-14T22:37:37.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5640000000000001e-06 2024-09-14T22:37:37.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.894 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:37.894 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824117+0000", 2024-09-14T22:37:37.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0366e-05 2024-09-14T22:37:37.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.894 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.894 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824158+0000", 2024-09-14T22:37:37.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0670000000000002e-05 2024-09-14T22:37:37.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.895 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:37.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878853+0000", 2024-09-14T22:37:37.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054694853000000002 2024-09-14T22:37:37.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.895 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:37.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878934+0000", 2024-09-14T22:37:37.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1754000000000004e-05 2024-09-14T22:37:37.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.895 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:37.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879034+0000", 2024-09-14T22:37:37.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000100115 2024-09-14T22:37:37.896 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.896 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:37.896 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.896 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112033 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6463488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:37.896 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:37:37.896 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.034852951, 2024-09-14T22:37:37.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054991076999999999, 2024-09-14T22:37:37.896 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:37.896 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:37.896 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:37.896 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:37.897 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:37.897 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.897 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112033 2024-09-14T22:37:37.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.897 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:37.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:37.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:37:37.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:37.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:37:37.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:37.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824172+0000", 2024-09-14T22:37:37.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9999999999999996e-07 2024-09-14T22:37:37.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:37.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824177+0000", 2024-09-14T22:37:37.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5310000000000003e-06 2024-09-14T22:37:37.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:37.899 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824188+0000", 2024-09-14T22:37:37.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1235e-05 2024-09-14T22:37:37.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.899 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.899 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824197+0000", 2024-09-14T22:37:37.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7069999999999998e-06 2024-09-14T22:37:37.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.899 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:37.899 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879075+0000", 2024-09-14T22:37:37.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054878749999999997 2024-09-14T22:37:37.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.900 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:37.900 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879118+0000", 2024-09-14T22:37:37.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2749e-05 2024-09-14T22:37:37.900 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.900 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:37.900 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879162+0000", 2024-09-14T22:37:37.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4205000000000001e-05 2024-09-14T22:37:37.900 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.900 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:37.900 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.900 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.901 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112034 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6512640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:37.901 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:37:37.901 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.034815999999999, 2024-09-14T22:37:37.901 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055181246000000003, 2024-09-14T22:37:37.901 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:37.901 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:37.901 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:37.901 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:37.901 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:37.901 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.901 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112034 2024-09-14T22:37:37.901 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.901 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:37.901 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:37.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:37:37.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:37.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:37:37.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:37.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824209+0000", 2024-09-14T22:37:37.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5499999999999998e-07 2024-09-14T22:37:37.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:37.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824212+0000", 2024-09-14T22:37:37.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6119999999999999e-06 2024-09-14T22:37:37.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:37.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824224+0000", 2024-09-14T22:37:37.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1127999999999999e-05 2024-09-14T22:37:37.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824232+0000", 2024-09-14T22:37:37.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4700000000000002e-06 2024-09-14T22:37:37.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.904 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:37.904 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879209+0000", 2024-09-14T22:37:37.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054977201000000003 2024-09-14T22:37:37.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.904 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:37.904 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879308+0000", 2024-09-14T22:37:37.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8419999999999996e-05 2024-09-14T22:37:37.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.904 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:37.904 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879389+0000", 2024-09-14T22:37:37.905 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1760000000000003e-05 2024-09-14T22:37:37.905 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.905 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:37.905 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.905 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112035 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6561792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:37.905 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:37:37.905 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.034722272, 2024-09-14T22:37:37.905 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055300522999999997, 2024-09-14T22:37:37.905 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:37.905 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:37.905 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:37.905 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:37.906 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:37.906 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.906 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112035 2024-09-14T22:37:37.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.906 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:37.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.906 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:37.906 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:37:37.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.906 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:37.906 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:37:37.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:37.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824303+0000", 2024-09-14T22:37:37.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5099999999999998e-07 2024-09-14T22:37:37.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:37.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824308+0000", 2024-09-14T22:37:37.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6689999999999997e-06 2024-09-14T22:37:37.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:37.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824744+0000", 2024-09-14T22:37:37.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00043601600000000001 2024-09-14T22:37:37.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824756+0000", 2024-09-14T22:37:37.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2476999999999999e-05 2024-09-14T22:37:37.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:37.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879434+0000", 2024-09-14T22:37:37.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054677630999999997 2024-09-14T22:37:37.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.909 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:37.909 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879518+0000", 2024-09-14T22:37:37.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4220000000000003e-05 2024-09-14T22:37:37.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.909 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:37.909 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879602+0000", 2024-09-14T22:37:37.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4559000000000001e-05 2024-09-14T22:37:37.909 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.909 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:37.909 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.909 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112036 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6610944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:37.910 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:37:37.910 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.034230534000001, 2024-09-14T22:37:37.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055036320999999999, 2024-09-14T22:37:37.910 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:37.910 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:37.910 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:37.910 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:37.910 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:37.910 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.910 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112036 2024-09-14T22:37:37.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.910 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:37.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:37.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:37:37.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:37.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:37:37.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:37.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824795+0000", 2024-09-14T22:37:37.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-06 2024-09-14T22:37:37.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:37.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824802+0000", 2024-09-14T22:37:37.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6680000000000004e-06 2024-09-14T22:37:37.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:37.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824814+0000", 2024-09-14T22:37:37.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2102e-05 2024-09-14T22:37:37.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824823+0000", 2024-09-14T22:37:37.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5550000000000008e-06 2024-09-14T22:37:37.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:37.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879648+0000", 2024-09-14T22:37:37.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054825167000000001 2024-09-14T22:37:37.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:37.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879747+0000", 2024-09-14T22:37:37.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8968999999999994e-05 2024-09-14T22:37:37.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:37.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879830+0000", 2024-09-14T22:37:37.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2633999999999995e-05 2024-09-14T22:37:37.914 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.914 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:37.914 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.914 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112037 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6660096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:37.914 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:37:37.914 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.034187825, 2024-09-14T22:37:37.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055122926000000003, 2024-09-14T22:37:37.914 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:37.915 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:37.915 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:37.915 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:37.915 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:37.915 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.915 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112037 2024-09-14T22:37:37.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.915 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:37.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:37.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:37:37.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:37.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:37:37.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:37.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824837+0000", 2024-09-14T22:37:37.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5499999999999998e-07 2024-09-14T22:37:37.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:37.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824840+0000", 2024-09-14T22:37:37.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0429999999999999e-06 2024-09-14T22:37:37.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:37.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824851+0000", 2024-09-14T22:37:37.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1046999999999999e-05 2024-09-14T22:37:37.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824859+0000", 2024-09-14T22:37:37.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1200000000000002e-06 2024-09-14T22:37:37.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:37.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879867+0000", 2024-09-14T22:37:37.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055007697000000001 2024-09-14T22:37:37.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:37.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879910+0000", 2024-09-14T22:37:37.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3402000000000002e-05 2024-09-14T22:37:37.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:37.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879959+0000", 2024-09-14T22:37:37.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9061999999999999e-05 2024-09-14T22:37:37.918 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.919 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:37.919 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.919 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112038 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6709248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:37.919 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:37:37.919 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.034153129, 2024-09-14T22:37:37.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055312828000000001, 2024-09-14T22:37:37.919 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:37.919 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:37.919 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:37.919 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:37.919 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:37.920 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.920 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112038 2024-09-14T22:37:37.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.920 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:37.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:37.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:37:37.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:37.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:37:37.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:37.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824872+0000", 2024-09-14T22:37:37.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T22:37:37.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:37.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824875+0000", 2024-09-14T22:37:37.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7519999999999998e-06 2024-09-14T22:37:37.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:37.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824917+0000", 2024-09-14T22:37:37.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1047999999999997e-05 2024-09-14T22:37:37.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824926+0000", 2024-09-14T22:37:37.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3749999999999992e-06 2024-09-14T22:37:37.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:37.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880006+0000", 2024-09-14T22:37:37.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055080525999999998 2024-09-14T22:37:37.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:37.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880102+0000", 2024-09-14T22:37:37.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5995999999999994e-05 2024-09-14T22:37:37.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.923 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:37.923 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880184+0000", 2024-09-14T22:37:37.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1478999999999997e-05 2024-09-14T22:37:37.923 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.923 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:37.923 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.923 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112039 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6758400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:37.923 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:37:37.924 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.034084459000001, 2024-09-14T22:37:37.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055449709999999999, 2024-09-14T22:37:37.924 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:37.924 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:37.924 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:37.924 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:37.924 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:37.924 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.924 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112039 2024-09-14T22:37:37.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.924 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:37.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:37.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:37:37.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:37.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:37:37.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:37.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824941+0000", 2024-09-14T22:37:37.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3699999999999999e-07 2024-09-14T22:37:37.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:37.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824946+0000", 2024-09-14T22:37:37.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0209999999999999e-06 2024-09-14T22:37:37.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:37.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824972+0000", 2024-09-14T22:37:37.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6750999999999998e-05 2024-09-14T22:37:37.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824982+0000", 2024-09-14T22:37:37.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5019999999999998e-06 2024-09-14T22:37:37.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:37.927 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880227+0000", 2024-09-14T22:37:37.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055245257999999998 2024-09-14T22:37:37.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:37.927 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880305+0000", 2024-09-14T22:37:37.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7424999999999998e-05 2024-09-14T22:37:37.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:37.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880389+0000", 2024-09-14T22:37:37.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4815999999999996e-05 2024-09-14T22:37:37.928 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.928 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:37.928 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.928 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112040 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6807552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:37.928 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:37:37.928 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.034027742999999, 2024-09-14T22:37:37.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055534615000000002, 2024-09-14T22:37:37.928 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:37.928 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:37.928 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:37.929 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:37.929 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:37.929 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.929 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112040 2024-09-14T22:37:37.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.929 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:37.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:37.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:37:37.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:37.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:37:37.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:37.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824997+0000", 2024-09-14T22:37:37.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7199999999999998e-07 2024-09-14T22:37:37.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:37.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825042+0000", 2024-09-14T22:37:37.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5065999999999998e-05 2024-09-14T22:37:37.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:37.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825060+0000", 2024-09-14T22:37:37.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7520000000000002e-05 2024-09-14T22:37:37.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825070+0000", 2024-09-14T22:37:37.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8230000000000006e-06 2024-09-14T22:37:37.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:37.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880427+0000", 2024-09-14T22:37:37.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055357244 2024-09-14T22:37:37.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.932 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.932 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:37.932 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880483+0000", 2024-09-14T22:37:37.932 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5807000000000003e-05 2024-09-14T22:37:37.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.932 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.932 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:37.932 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880531+0000", 2024-09-14T22:37:37.932 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8383000000000001e-05 2024-09-14T22:37:37.932 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.933 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:37.933 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.933 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112041 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6856704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:37.933 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:37:37.933 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.033938569, 2024-09-14T22:37:37.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055573887000000002, 2024-09-14T22:37:37.933 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:37.933 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:37.933 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:37.933 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:37.933 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:37.933 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.934 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112041 2024-09-14T22:37:37.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.934 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:37.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:37.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:37:37.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:37.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:37:37.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:37.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825087+0000", 2024-09-14T22:37:37.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2399999999999996e-07 2024-09-14T22:37:37.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:37.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825091+0000", 2024-09-14T22:37:37.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9729999999999999e-06 2024-09-14T22:37:37.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:37.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825101+0000", 2024-09-14T22:37:37.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.093e-05 2024-09-14T22:37:37.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.936 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825110+0000", 2024-09-14T22:37:37.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1450000000000008e-06 2024-09-14T22:37:37.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.936 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:37.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880559+0000", 2024-09-14T22:37:37.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055449684999999999 2024-09-14T22:37:37.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.936 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:37.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880601+0000", 2024-09-14T22:37:37.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1248000000000002e-05 2024-09-14T22:37:37.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.937 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.937 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:37.937 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880659+0000", 2024-09-14T22:37:37.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8981999999999999e-05 2024-09-14T22:37:37.937 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.937 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:37.937 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.937 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.937 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112042 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6905856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:37.937 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:37:37.937 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.033903835, 2024-09-14T22:37:37.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055781415000000001, 2024-09-14T22:37:37.938 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:37.938 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:37.938 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:37.938 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:37.938 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:37.938 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.938 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112042 2024-09-14T22:37:37.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.938 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:37.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.938 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:37.938 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:37:37.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:37.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:37:37.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:37.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825478+0000", 2024-09-14T22:37:37.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035775 2024-09-14T22:37:37.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.940 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:37.940 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825494+0000", 2024-09-14T22:37:37.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5994999999999998e-05 2024-09-14T22:37:37.940 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.940 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:37.940 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825520+0000", 2024-09-14T22:37:37.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5738999999999999e-05 2024-09-14T22:37:37.940 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.940 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.940 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825547+0000", 2024-09-14T22:37:37.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7667000000000001e-05 2024-09-14T22:37:37.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.941 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:37.941 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880725+0000", 2024-09-14T22:37:37.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055177962999999997 2024-09-14T22:37:37.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.941 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:37.941 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880816+0000", 2024-09-14T22:37:37.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0315000000000005e-05 2024-09-14T22:37:37.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.941 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:37.942 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880902+0000", 2024-09-14T22:37:37.942 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5986e-05 2024-09-14T22:37:37.942 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.942 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:37.942 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.942 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.942 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.942 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112043 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:37.942 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:37:37.942 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.033455830999999, 2024-09-14T22:37:37.942 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055537912000000002, 2024-09-14T22:37:37.942 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:37.942 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:37.942 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:37.943 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:37.943 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:37.943 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.943 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112043 2024-09-14T22:37:37.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.943 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:37.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:37.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:37:37.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:37.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:37:37.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:37.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825911+0000", 2024-09-14T22:37:37.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00034306899999999999 2024-09-14T22:37:37.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:37.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825918+0000", 2024-09-14T22:37:37.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.002e-06 2024-09-14T22:37:37.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:37.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825931+0000", 2024-09-14T22:37:37.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2401e-05 2024-09-14T22:37:37.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825942+0000", 2024-09-14T22:37:37.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0903e-05 2024-09-14T22:37:37.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:37.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880951+0000", 2024-09-14T22:37:37.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055009379999999997 2024-09-14T22:37:37.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.946 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:37.946 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881027+0000", 2024-09-14T22:37:37.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5944000000000003e-05 2024-09-14T22:37:37.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.946 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:37.946 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881106+0000", 2024-09-14T22:37:37.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9213000000000002e-05 2024-09-14T22:37:37.946 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.946 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:37.946 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.947 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112044 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 7004160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:37.947 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:37:37.947 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.033062162, 2024-09-14T22:37:37.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055265416999999997, 2024-09-14T22:37:37.947 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:37.947 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:37.947 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:37.947 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:37.947 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:37.947 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.947 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112044 2024-09-14T22:37:37.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.947 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:37.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:37.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:37:37.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:37.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:37:37.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:37.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826239+0000", 2024-09-14T22:37:37.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027676000000000003 2024-09-14T22:37:37.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:37.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826247+0000", 2024-09-14T22:37:37.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7440000000000007e-06 2024-09-14T22:37:37.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:37.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826275+0000", 2024-09-14T22:37:37.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8053000000000001e-05 2024-09-14T22:37:37.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826286+0000", 2024-09-14T22:37:37.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1780000000000001e-05 2024-09-14T22:37:37.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.950 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:37.950 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881137+0000", 2024-09-14T22:37:37.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054850500000000003 2024-09-14T22:37:37.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.950 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:37.950 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881181+0000", 2024-09-14T22:37:37.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4153000000000002e-05 2024-09-14T22:37:37.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.950 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:37.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881227+0000", 2024-09-14T22:37:37.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6427000000000001e-05 2024-09-14T22:37:37.951 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.951 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:37.951 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.951 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112045 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 7053312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:37.951 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:37:37.951 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.032699929, 2024-09-14T22:37:37.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055045050999999998, 2024-09-14T22:37:37.951 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:37.951 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:37.951 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:37.951 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:37.952 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:37.952 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:37.952 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112045 2024-09-14T22:37:37.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.952 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:37.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:37.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:37:37.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:37.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:37:37.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:37.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:37.953 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830250+0000", 2024-09-14T22:37:37.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0039258670000000004 2024-09-14T22:37:37.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:37.953 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830260+0000", 2024-09-14T22:37:37.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0088000000000001e-05 2024-09-14T22:37:37.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:37.953 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830281+0000", 2024-09-14T22:37:37.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0720000000000002e-05 2024-09-14T22:37:37.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:37.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830292+0000", 2024-09-14T22:37:37.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1455e-05 2024-09-14T22:37:37.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:37.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881255+0000", 2024-09-14T22:37:37.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.050962247000000002 2024-09-14T22:37:37.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:37.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881296+0000", 2024-09-14T22:37:37.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1557999999999998e-05 2024-09-14T22:37:37.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:37.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:37.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:37.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881369+0000", 2024-09-14T22:37:37.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3115999999999996e-05 2024-09-14T22:37:37.955 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.955 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:37.955 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.955 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:37.955 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:37.955 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:37.957 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:37:37.999 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:37.995+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:38.059 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:38.055+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:38.077 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-14T22:37:38.098 INFO:teuthology.orchestra.run.smithi001.stdout:4624 punch from 0x5c6ae64 to 0x5c75b4d, (0xace9 bytes) 2024-09-14T22:37:38.105 INFO:teuthology.orchestra.run.smithi001.stdout:4627 punch from 0x783ffa6 to 0x784bb11, (0xbb6b bytes) 2024-09-14T22:37:38.120 INFO:teuthology.orchestra.run.smithi001.stdout:4628 trunc from 0xcb3101e to 0x1eb4cfd 2024-09-14T22:37:38.140 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:38.140 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:38.140 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:37:38.140 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:38.153 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:38.153 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:38.153 INFO:teuthology.orchestra.run.smithi001.stdout:4631 read 0x17fb155 thru 0x17fb1d7 (0x83 bytes) 2024-09-14T22:37:38.157 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:37:38.161 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:38.159+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:38.164 INFO:teuthology.orchestra.run.smithi001.stdout:4632 write 0x36c2110 thru 0x36cc88a (0xa77b bytes) 2024-09-14T22:37:38.181 INFO:teuthology.orchestra.run.smithi001.stdout:4635 write 0x63f9643 thru 0x6408a76 (0xf434 bytes) 2024-09-14T22:37:38.188 INFO:teuthology.orchestra.run.smithi001.stdout:4637 flatten 2024-09-14T22:37:38.223 INFO:teuthology.orchestra.run.smithi001.stdout:4639 punch from 0x121c238 to 0x122a0b0, (0xde78 bytes) 2024-09-14T22:37:38.232 INFO:teuthology.orchestra.run.smithi001.stdout:4641 write 0x786755e thru 0x786af32 (0x39d5 bytes) 2024-09-14T22:37:38.236 INFO:teuthology.orchestra.run.smithi001.stdout:4643 read 0x46d9d9c thru 0x46e0b23 (0x6d88 bytes) 2024-09-14T22:37:38.236 INFO:teuthology.orchestra.run.smithi001.stdout:4644 read 0xc456a5 thru 0xc4eaa2 (0x93fe bytes) 2024-09-14T22:37:38.236 INFO:teuthology.orchestra.run.smithi001.stdout:4645 writesame 0x489a54 thru 0x493e92 data_size 0x1(0xa43f bytes) 2024-09-14T22:37:38.261 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:38.255+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:38.265 INFO:teuthology.orchestra.run.smithi001.stdout:4646 writesame 0xa4430d6 thru 0xa44f544 data_size 0x1(0xc46f bytes) 2024-09-14T22:37:38.265 INFO:teuthology.orchestra.run.smithi001.stdout:4647 writesame 0x5c563bb thru 0x5c599b4 data_size 0x31(0x35fa bytes) 2024-09-14T22:37:38.270 INFO:teuthology.orchestra.run.smithi001.stdout:4648 punch from 0x1c12247 to 0x1c1a130, (0x7ee9 bytes) 2024-09-14T22:37:38.281 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:37:38.282 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-14T22:37:38.282 INFO:teuthology.orchestra.run.smithi001.stdout:4649 write 0xd1c0b50 thru 0xd1c0b84 (0x35 bytes) 2024-09-14T22:37:38.287 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:38.287 INFO:teuthology.orchestra.run.smithi001.stdout:4652 write 0x7efe571 thru 0x7f0e44b (0xfedb bytes) 2024-09-14T22:37:38.293 INFO:teuthology.orchestra.run.smithi001.stdout:4653 writesame 0x475353c thru 0x475b1c8 data_size 0x7(0x7c8d bytes) 2024-09-14T22:37:38.298 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":131,"stamp":"2024-09-14T22:37:37.810191+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1111019613,"num_objects":1159,"num_object_clones":320,"num_object_copies":2318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1159,"num_whiteouts":384,"num_read":148364,"num_read_kb":544953,"num_write":16150,"num_write_kb":254760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":536,"num_bytes_recovered":27389785,"num_keys_recovered":22,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12584,"ondisk_log_size":12584,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":346372,"kb_used_data":42640,"kb_used_omap":17,"kb_used_meta":303534,"kb_avail":754628348,"statfs":{"total":773094113280,"available":772739428352,"internally_reserved":0,"allocated":43663360,"data_stored":355993064,"data_compressed":9842758,"data_compressed_allocated":27803648,"data_compressed_original":347747254,"omap_allocated":18180,"internal_metadata":310819068},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.025787"},"pg_stats":[{"pgid":"3.1f","version":"81'242","reported_seq":4120,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.808574+0000","last_change":"2024-09-14T22:37:25.739233+0000","last_active":"2024-09-14T22:37:32.808574+0000","last_peered":"2024-09-14T22:37:32.808574+0000","last_clean":"2024-09-14T22:37:32.808574+0000","last_became_active":"2024-09-14T22:37:25.718906+0000","last_became_peered":"2024-09-14T22:37:25.718906+0000","last_unstale":"2024-09-14T22:37:32.808574+0000","last_undegraded":"2024-09-14T22:37:32.808574+0000","last_fullsized":"2024-09-14T22:37:32.808574+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:55:57.908853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017190879999999999,"stat_sum":{"num_bytes":51625811,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":14,"num_read":3899,"num_read_kb":15185,"num_write":258,"num_write_kb":6027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":3277632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"79'921","reported_seq":4839,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.809140+0000","last_change":"2024-09-14T22:36:17.733296+0000","last_active":"2024-09-14T22:37:32.809140+0000","last_peered":"2024-09-14T22:37:32.809140+0000","last_clean":"2024-09-14T22:37:32.809140+0000","last_became_active":"2024-09-14T22:36:15.709272+0000","last_became_peered":"2024-09-14T22:36:15.709272+0000","last_unstale":"2024-09-14T22:37:32.809140+0000","last_undegraded":"2024-09-14T22:37:32.809140+0000","last_fullsized":"2024-09-14T22:37:32.809140+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":921,"log_dups_size":0,"ondisk_log_size":921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:49:47.937464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001723619,"stat_sum":{"num_bytes":62550087,"num_objects":32,"num_object_clones":9,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":12,"num_read":8781,"num_read_kb":22718,"num_write":1480,"num_write_kb":8170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"79'314","reported_seq":7571,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.809887+0000","last_change":"2024-09-14T22:36:17.702313+0000","last_active":"2024-09-14T22:37:32.809887+0000","last_peered":"2024-09-14T22:37:32.809887+0000","last_clean":"2024-09-14T22:37:32.809887+0000","last_became_active":"2024-09-14T22:36:15.719488+0000","last_became_peered":"2024-09-14T22:36:15.719488+0000","last_unstale":"2024-09-14T22:37:32.809887+0000","last_undegraded":"2024-09-14T22:37:32.809887+0000","last_fullsized":"2024-09-14T22:37:32.809887+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:20:16.819676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053914200000000005,"stat_sum":{"num_bytes":64267617,"num_objects":52,"num_object_clones":19,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":20,"num_read":7454,"num_read_kb":40817,"num_write":291,"num_write_kb":8757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"82'302","reported_seq":3003,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.809149+0000","last_change":"2024-09-14T22:36:17.702413+0000","last_active":"2024-09-14T22:37:32.809149+0000","last_peered":"2024-09-14T22:37:32.809149+0000","last_clean":"2024-09-14T22:37:32.809149+0000","last_became_active":"2024-09-14T22:36:15.719470+0000","last_became_peered":"2024-09-14T22:36:15.719470+0000","last_unstale":"2024-09-14T22:37:32.809149+0000","last_undegraded":"2024-09-14T22:37:32.809149+0000","last_fullsized":"2024-09-14T22:37:32.809149+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:48:58.357123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055079499999999997,"stat_sum":{"num_bytes":26279920,"num_objects":45,"num_object_clones":12,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":19,"num_read":2674,"num_read_kb":14679,"num_write":333,"num_write_kb":7367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'395","reported_seq":4551,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.809439+0000","last_change":"2024-09-14T22:37:25.736579+0000","last_active":"2024-09-14T22:37:32.809439+0000","last_peered":"2024-09-14T22:37:32.809439+0000","last_clean":"2024-09-14T22:37:32.809439+0000","last_became_active":"2024-09-14T22:37:25.718519+0000","last_became_peered":"2024-09-14T22:37:25.718519+0000","last_unstale":"2024-09-14T22:37:32.809439+0000","last_undegraded":"2024-09-14T22:37:32.809439+0000","last_fullsized":"2024-09-14T22:37:32.809439+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:55:01.095884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092291499999999996,"stat_sum":{"num_bytes":25177553,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":11,"num_read":4929,"num_read_kb":28545,"num_write":494,"num_write_kb":8858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":7858104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"82'328","reported_seq":2521,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.876023+0000","last_change":"2024-09-14T22:36:17.711364+0000","last_active":"2024-09-14T22:37:32.876023+0000","last_peered":"2024-09-14T22:37:32.876023+0000","last_clean":"2024-09-14T22:37:32.876023+0000","last_became_active":"2024-09-14T22:36:15.701852+0000","last_became_peered":"2024-09-14T22:36:15.701852+0000","last_unstale":"2024-09-14T22:37:32.876023+0000","last_undegraded":"2024-09-14T22:37:32.876023+0000","last_fullsized":"2024-09-14T22:37:32.876023+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:17:55.480586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012502780000000001,"stat_sum":{"num_bytes":8128485,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":17,"num_read":3159,"num_read_kb":9131,"num_write":350,"num_write_kb":5788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"87'324","reported_seq":4979,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T22:37:36.814271+0000","last_change":"2024-09-14T22:36:17.705423+0000","last_active":"2024-09-14T22:37:36.814271+0000","last_peered":"2024-09-14T22:37:36.814271+0000","last_clean":"2024-09-14T22:37:36.814271+0000","last_became_active":"2024-09-14T22:36:15.700990+0000","last_became_peered":"2024-09-14T22:36:15.700990+0000","last_unstale":"2024-09-14T22:37:36.814271+0000","last_undegraded":"2024-09-14T22:37:36.814271+0000","last_fullsized":"2024-09-14T22:37:36.814271+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:20:52.988019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010520620000000001,"stat_sum":{"num_bytes":20139232,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":11,"num_read":5021,"num_read_kb":15102,"num_write":342,"num_write_kb":5325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"82'221","reported_seq":2728,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.808709+0000","last_change":"2024-09-14T22:37:25.719758+0000","last_active":"2024-09-14T22:37:32.808709+0000","last_peered":"2024-09-14T22:37:32.808709+0000","last_clean":"2024-09-14T22:37:32.808709+0000","last_became_active":"2024-09-14T22:37:25.717354+0000","last_became_peered":"2024-09-14T22:37:25.717354+0000","last_unstale":"2024-09-14T22:37:32.808709+0000","last_undegraded":"2024-09-14T22:37:32.808709+0000","last_fullsized":"2024-09-14T22:37:32.808709+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:58:50.594733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002063484,"stat_sum":{"num_bytes":19040318,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":4,"num_read":2527,"num_read_kb":18581,"num_write":293,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":633727,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"83'219","reported_seq":3144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.822406+0000","last_change":"2024-09-14T22:36:17.721195+0000","last_active":"2024-09-14T22:37:32.822406+0000","last_peered":"2024-09-14T22:37:32.822406+0000","last_clean":"2024-09-14T22:37:32.822406+0000","last_became_active":"2024-09-14T22:36:15.715869+0000","last_became_peered":"2024-09-14T22:36:15.715869+0000","last_unstale":"2024-09-14T22:37:32.822406+0000","last_undegraded":"2024-09-14T22:37:32.822406+0000","last_fullsized":"2024-09-14T22:37:32.822406+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:47:17.519839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001160672,"stat_sum":{"num_bytes":19658672,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":11,"num_read":3182,"num_read_kb":13742,"num_write":255,"num_write_kb":14649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"79'393","reported_seq":3887,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.810514+0000","last_change":"2024-09-14T22:36:17.703048+0000","last_active":"2024-09-14T22:37:32.810514+0000","last_peered":"2024-09-14T22:37:32.810514+0000","last_clean":"2024-09-14T22:37:32.810514+0000","last_became_active":"2024-09-14T22:36:15.725078+0000","last_became_peered":"2024-09-14T22:36:15.725078+0000","last_unstale":"2024-09-14T22:37:32.810514+0000","last_undegraded":"2024-09-14T22:37:32.810514+0000","last_fullsized":"2024-09-14T22:37:32.810514+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:19:54.219161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010500290000000001,"stat_sum":{"num_bytes":59689882,"num_objects":49,"num_object_clones":14,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":15,"num_read":4376,"num_read_kb":28795,"num_write":533,"num_write_kb":11852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"82'228","reported_seq":2888,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.802324+0000","last_change":"2024-09-14T22:36:18.009371+0000","last_active":"2024-09-14T22:37:32.802324+0000","last_peered":"2024-09-14T22:37:32.802324+0000","last_clean":"2024-09-14T22:37:32.802324+0000","last_became_active":"2024-09-14T22:36:15.717341+0000","last_became_peered":"2024-09-14T22:36:15.717341+0000","last_unstale":"2024-09-14T22:37:32.802324+0000","last_undegraded":"2024-09-14T22:37:32.802324+0000","last_fullsized":"2024-09-14T22:37:32.802324+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:40:54.611834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054571999999999997,"stat_sum":{"num_bytes":9371341,"num_objects":24,"num_object_clones":7,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":8,"num_read":2763,"num_read_kb":3328,"num_write":248,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"83'788","reported_seq":6871,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.880568+0000","last_change":"2024-09-14T22:36:17.711384+0000","last_active":"2024-09-14T22:37:32.880568+0000","last_peered":"2024-09-14T22:37:32.880568+0000","last_clean":"2024-09-14T22:37:32.880568+0000","last_became_active":"2024-09-14T22:36:15.723075+0000","last_became_peered":"2024-09-14T22:36:15.723075+0000","last_unstale":"2024-09-14T22:37:32.880568+0000","last_undegraded":"2024-09-14T22:37:32.880568+0000","last_fullsized":"2024-09-14T22:37:32.880568+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":788,"log_dups_size":0,"ondisk_log_size":788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:03:40.527644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010804009999999999,"stat_sum":{"num_bytes":37175321,"num_objects":47,"num_object_clones":13,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":14,"num_read":8660,"num_read_kb":21474,"num_write":1136,"num_write_kb":9408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.807895+0000","last_change":"2024-09-14T22:36:08.042860+0000","last_active":"2024-09-14T22:37:32.807895+0000","last_peered":"2024-09-14T22:37:32.807895+0000","last_clean":"2024-09-14T22:37:32.807895+0000","last_became_active":"2024-09-14T22:36:06.040961+0000","last_became_peered":"2024-09-14T22:36:06.040961+0000","last_unstale":"2024-09-14T22:37:32.807895+0000","last_undegraded":"2024-09-14T22:37:32.807895+0000","last_fullsized":"2024-09-14T22:37:32.807895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:03:48.547834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010780950000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"82'312","reported_seq":1640,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.809269+0000","last_change":"2024-09-14T22:36:17.720897+0000","last_active":"2024-09-14T22:37:32.809269+0000","last_peered":"2024-09-14T22:37:32.809269+0000","last_clean":"2024-09-14T22:37:32.809269+0000","last_became_active":"2024-09-14T22:36:15.700982+0000","last_became_peered":"2024-09-14T22:36:15.700982+0000","last_unstale":"2024-09-14T22:37:32.809269+0000","last_undegraded":"2024-09-14T22:37:32.809269+0000","last_fullsized":"2024-09-14T22:37:32.809269+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:08:15.177998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010495459999999999,"stat_sum":{"num_bytes":20611036,"num_objects":40,"num_object_clones":12,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":13,"num_read":2369,"num_read_kb":8666,"num_write":320,"num_write_kb":7994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.807983+0000","last_change":"2024-09-14T22:36:41.930087+0000","last_active":"2024-09-14T22:37:32.807983+0000","last_peered":"2024-09-14T22:37:32.807983+0000","last_clean":"2024-09-14T22:37:32.807983+0000","last_became_active":"2024-09-14T22:36:41.929522+0000","last_became_peered":"2024-09-14T22:36:41.929522+0000","last_unstale":"2024-09-14T22:37:32.807983+0000","last_undegraded":"2024-09-14T22:37:32.807983+0000","last_fullsized":"2024-09-14T22:37:32.807983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:15:10.589042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"83'332","reported_seq":4470,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.822850+0000","last_change":"2024-09-14T22:36:21.722517+0000","last_active":"2024-09-14T22:37:32.822850+0000","last_peered":"2024-09-14T22:37:32.822850+0000","last_clean":"2024-09-14T22:37:32.822850+0000","last_became_active":"2024-09-14T22:36:21.722085+0000","last_became_peered":"2024-09-14T22:36:21.722085+0000","last_unstale":"2024-09-14T22:37:32.822850+0000","last_undegraded":"2024-09-14T22:37:32.822850+0000","last_fullsized":"2024-09-14T22:37:32.822850+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:53:32.754230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020774109999999999,"stat_sum":{"num_bytes":47655829,"num_objects":32,"num_object_clones":6,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":10,"num_read":5103,"num_read_kb":23318,"num_write":396,"num_write_kb":6442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.802633+0000","last_change":"2024-09-14T22:36:08.054606+0000","last_active":"2024-09-14T22:37:32.802633+0000","last_peered":"2024-09-14T22:37:32.802633+0000","last_clean":"2024-09-14T22:37:32.802633+0000","last_became_active":"2024-09-14T22:36:06.266908+0000","last_became_peered":"2024-09-14T22:36:06.266908+0000","last_unstale":"2024-09-14T22:37:32.802633+0000","last_undegraded":"2024-09-14T22:37:32.802633+0000","last_fullsized":"2024-09-14T22:37:32.802633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:36.738643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098249300000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"83'1119","reported_seq":3766,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T22:37:34.800664+0000","last_change":"2024-09-14T22:37:17.676125+0000","last_active":"2024-09-14T22:37:34.800664+0000","last_peered":"2024-09-14T22:37:34.800664+0000","last_clean":"2024-09-14T22:37:34.800664+0000","last_became_active":"2024-09-14T22:37:17.674322+0000","last_became_peered":"2024-09-14T22:37:17.674322+0000","last_unstale":"2024-09-14T22:37:34.800664+0000","last_undegraded":"2024-09-14T22:37:34.800664+0000","last_fullsized":"2024-09-14T22:37:34.800664+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1119,"log_dups_size":0,"ondisk_log_size":1119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:54:44.017669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012564830000000001,"stat_sum":{"num_bytes":16092273,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":15,"num_read":7482,"num_read_kb":14334,"num_write":1975,"num_write_kb":9318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":4264139,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.808667+0000","last_change":"2024-09-14T22:36:41.930961+0000","last_active":"2024-09-14T22:37:32.808667+0000","last_peered":"2024-09-14T22:37:32.808667+0000","last_clean":"2024-09-14T22:37:32.808667+0000","last_became_active":"2024-09-14T22:36:41.930619+0000","last_became_peered":"2024-09-14T22:36:41.930619+0000","last_unstale":"2024-09-14T22:37:32.808667+0000","last_undegraded":"2024-09-14T22:37:32.808667+0000","last_fullsized":"2024-09-14T22:37:32.808667+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:04:40.582999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"83'380","reported_seq":3044,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.810299+0000","last_change":"2024-09-14T22:36:17.702597+0000","last_active":"2024-09-14T22:37:32.810299+0000","last_peered":"2024-09-14T22:37:32.810299+0000","last_clean":"2024-09-14T22:37:32.810299+0000","last_became_active":"2024-09-14T22:36:15.717186+0000","last_became_peered":"2024-09-14T22:36:15.717186+0000","last_unstale":"2024-09-14T22:37:32.810299+0000","last_undegraded":"2024-09-14T22:37:32.810299+0000","last_fullsized":"2024-09-14T22:37:32.810299+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:47:54.484398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071220099999999998,"stat_sum":{"num_bytes":24964501,"num_objects":34,"num_object_clones":10,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":13,"num_read":4078,"num_read_kb":5907,"num_write":358,"num_write_kb":2187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"79'255","reported_seq":3686,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.784466+0000","last_change":"2024-09-14T22:36:17.711068+0000","last_active":"2024-09-14T22:37:32.784466+0000","last_peered":"2024-09-14T22:37:32.784466+0000","last_clean":"2024-09-14T22:37:32.784466+0000","last_became_active":"2024-09-14T22:36:15.706147+0000","last_became_peered":"2024-09-14T22:36:15.706147+0000","last_unstale":"2024-09-14T22:37:32.784466+0000","last_undegraded":"2024-09-14T22:37:32.784466+0000","last_fullsized":"2024-09-14T22:37:32.784466+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:58:33.555717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082625500000000002,"stat_sum":{"num_bytes":37489798,"num_objects":28,"num_object_clones":9,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":9,"num_read":3360,"num_read_kb":16744,"num_write":244,"num_write_kb":5687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.809222+0000","last_change":"2024-09-14T22:36:08.055938+0000","last_active":"2024-09-14T22:37:32.809222+0000","last_peered":"2024-09-14T22:37:32.809222+0000","last_clean":"2024-09-14T22:37:32.809222+0000","last_became_active":"2024-09-14T22:36:06.039600+0000","last_became_peered":"2024-09-14T22:36:06.039600+0000","last_unstale":"2024-09-14T22:37:32.809222+0000","last_undegraded":"2024-09-14T22:37:32.809222+0000","last_fullsized":"2024-09-14T22:37:32.809222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:49:09.245563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021617870000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"78'739","reported_seq":8115,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T22:37:34.800529+0000","last_change":"2024-09-14T22:37:16.669227+0000","last_active":"2024-09-14T22:37:34.800529+0000","last_peered":"2024-09-14T22:37:34.800529+0000","last_clean":"2024-09-14T22:37:34.800529+0000","last_became_active":"2024-09-14T22:37:16.662230+0000","last_became_peered":"2024-09-14T22:37:16.662230+0000","last_unstale":"2024-09-14T22:37:34.800529+0000","last_undegraded":"2024-09-14T22:37:34.800529+0000","last_fullsized":"2024-09-14T22:37:34.800529+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:18:22.556190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021500769999999998,"stat_sum":{"num_bytes":64266431,"num_objects":44,"num_object_clones":11,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":13,"num_read":9395,"num_read_kb":19599,"num_write":1052,"num_write_kb":9919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":4851034,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":72,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T22:37:34.800562+0000","last_change":"2024-09-14T22:37:13.762987+0000","last_active":"2024-09-14T22:37:34.800562+0000","last_peered":"2024-09-14T22:37:34.800562+0000","last_clean":"2024-09-14T22:37:34.800562+0000","last_became_active":"2024-09-14T22:37:13.762641+0000","last_became_peered":"2024-09-14T22:37:13.762641+0000","last_unstale":"2024-09-14T22:37:34.800562+0000","last_undegraded":"2024-09-14T22:37:34.800562+0000","last_fullsized":"2024-09-14T22:37:34.800562+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:30:59.887867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"83'271","reported_seq":1836,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.808493+0000","last_change":"2024-09-14T22:36:17.720438+0000","last_active":"2024-09-14T22:37:32.808493+0000","last_peered":"2024-09-14T22:37:32.808493+0000","last_clean":"2024-09-14T22:37:32.808493+0000","last_became_active":"2024-09-14T22:36:15.711105+0000","last_became_peered":"2024-09-14T22:36:15.711105+0000","last_unstale":"2024-09-14T22:37:32.808493+0000","last_undegraded":"2024-09-14T22:37:32.808493+0000","last_fullsized":"2024-09-14T22:37:32.808493+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:32:17.419345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085549400000000005,"stat_sum":{"num_bytes":43842060,"num_objects":25,"num_object_clones":8,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":9,"num_read":1628,"num_read_kb":8229,"num_write":284,"num_write_kb":2028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.802169+0000","last_change":"2024-09-14T22:36:08.054759+0000","last_active":"2024-09-14T22:37:32.802169+0000","last_peered":"2024-09-14T22:37:32.802169+0000","last_clean":"2024-09-14T22:37:32.802169+0000","last_became_active":"2024-09-14T22:36:06.039969+0000","last_became_peered":"2024-09-14T22:36:06.039969+0000","last_unstale":"2024-09-14T22:37:32.802169+0000","last_undegraded":"2024-09-14T22:37:32.802169+0000","last_fullsized":"2024-09-14T22:37:32.802169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:03:45.369532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010817520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"83'892","reported_seq":3331,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.807851+0000","last_change":"2024-09-14T22:36:51.718064+0000","last_active":"2024-09-14T22:37:32.807851+0000","last_peered":"2024-09-14T22:37:32.807851+0000","last_clean":"2024-09-14T22:37:32.807851+0000","last_became_active":"2024-09-14T22:36:51.715823+0000","last_became_peered":"2024-09-14T22:36:51.715823+0000","last_unstale":"2024-09-14T22:37:32.807851+0000","last_undegraded":"2024-09-14T22:37:32.807851+0000","last_fullsized":"2024-09-14T22:37:32.807851+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":892,"log_dups_size":0,"ondisk_log_size":892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:21:40.878401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44497460,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":10,"num_read":6633,"num_read_kb":10451,"num_write":1375,"num_write_kb":6132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":173209,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.809159+0000","last_change":"2024-09-14T22:37:13.625072+0000","last_active":"2024-09-14T22:37:32.809159+0000","last_peered":"2024-09-14T22:37:32.809159+0000","last_clean":"2024-09-14T22:37:32.809159+0000","last_became_active":"2024-09-14T22:37:13.624669+0000","last_became_peered":"2024-09-14T22:37:13.624669+0000","last_unstale":"2024-09-14T22:37:32.809159+0000","last_undegraded":"2024-09-14T22:37:32.809159+0000","last_fullsized":"2024-09-14T22:37:32.809159+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:54:20.545098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011401880000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":292,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.802497+0000","last_change":"2024-09-14T22:36:00.402862+0000","last_active":"2024-09-14T22:37:32.802497+0000","last_peered":"2024-09-14T22:37:32.802497+0000","last_clean":"2024-09-14T22:37:32.802497+0000","last_became_active":"2024-09-14T22:36:00.400529+0000","last_became_peered":"2024-09-14T22:36:00.400529+0000","last_unstale":"2024-09-14T22:37:32.802497+0000","last_undegraded":"2024-09-14T22:37:32.802497+0000","last_fullsized":"2024-09-14T22:37:32.802497+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_clean_scrub_stamp":"2024-09-14T22:35:58.989025+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:07:12.951728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"82'374","reported_seq":5190,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.802166+0000","last_change":"2024-09-14T22:37:25.713957+0000","last_active":"2024-09-14T22:37:32.802166+0000","last_peered":"2024-09-14T22:37:32.802166+0000","last_clean":"2024-09-14T22:37:32.802166+0000","last_became_active":"2024-09-14T22:37:25.712476+0000","last_became_peered":"2024-09-14T22:37:25.712476+0000","last_unstale":"2024-09-14T22:37:32.802166+0000","last_undegraded":"2024-09-14T22:37:32.802166+0000","last_fullsized":"2024-09-14T22:37:32.802166+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:11:39.920434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001381511,"stat_sum":{"num_bytes":31660772,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":9,"num_read":5607,"num_read_kb":9967,"num_write":544,"num_write_kb":12023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":2127217,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"80'244","reported_seq":5667,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.784264+0000","last_change":"2024-09-14T22:37:16.658991+0000","last_active":"2024-09-14T22:37:32.784264+0000","last_peered":"2024-09-14T22:37:32.784264+0000","last_clean":"2024-09-14T22:37:32.784264+0000","last_became_active":"2024-09-14T22:37:16.657865+0000","last_became_peered":"2024-09-14T22:37:16.657865+0000","last_unstale":"2024-09-14T22:37:32.784264+0000","last_undegraded":"2024-09-14T22:37:32.784264+0000","last_fullsized":"2024-09-14T22:37:32.784264+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:12:16.803063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060272699999999995,"stat_sum":{"num_bytes":49352262,"num_objects":43,"num_object_clones":14,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":12,"num_read":5875,"num_read_kb":12262,"num_write":236,"num_write_kb":5250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":4204723,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"83'428","reported_seq":2861,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.894440+0000","last_change":"2024-09-14T22:36:17.776247+0000","last_active":"2024-09-14T22:37:32.894440+0000","last_peered":"2024-09-14T22:37:32.894440+0000","last_clean":"2024-09-14T22:37:32.894440+0000","last_became_active":"2024-09-14T22:36:15.718794+0000","last_became_peered":"2024-09-14T22:36:15.718794+0000","last_unstale":"2024-09-14T22:37:32.894440+0000","last_undegraded":"2024-09-14T22:37:32.894440+0000","last_fullsized":"2024-09-14T22:37:32.894440+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:48.357497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088662699999999997,"stat_sum":{"num_bytes":19371023,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":8,"num_read":4234,"num_read_kb":18414,"num_write":417,"num_write_kb":13069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"82'301","reported_seq":1686,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.808739+0000","last_change":"2024-09-14T22:36:17.733002+0000","last_active":"2024-09-14T22:37:32.808739+0000","last_peered":"2024-09-14T22:37:32.808739+0000","last_clean":"2024-09-14T22:37:32.808739+0000","last_became_active":"2024-09-14T22:36:15.702885+0000","last_became_peered":"2024-09-14T22:36:15.702885+0000","last_unstale":"2024-09-14T22:37:32.808739+0000","last_undegraded":"2024-09-14T22:37:32.808739+0000","last_fullsized":"2024-09-14T22:37:32.808739+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:53:21.399927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015180580000000001,"stat_sum":{"num_bytes":34136085,"num_objects":35,"num_object_clones":11,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":12,"num_read":2365,"num_read_kb":8357,"num_write":280,"num_write_kb":6300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"83'274","reported_seq":5326,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.813065+0000","last_change":"2024-09-14T22:36:17.702738+0000","last_active":"2024-09-14T22:37:32.813065+0000","last_peered":"2024-09-14T22:37:32.813065+0000","last_clean":"2024-09-14T22:37:32.813065+0000","last_became_active":"2024-09-14T22:36:15.713452+0000","last_became_peered":"2024-09-14T22:36:15.713452+0000","last_unstale":"2024-09-14T22:37:32.813065+0000","last_undegraded":"2024-09-14T22:37:32.813065+0000","last_fullsized":"2024-09-14T22:37:32.813065+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:40:44.768952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082782099999999996,"stat_sum":{"num_bytes":38038174,"num_objects":36,"num_object_clones":14,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":14,"num_read":5161,"num_read_kb":17038,"num_write":307,"num_write_kb":8786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"83'235","reported_seq":3021,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.802626+0000","last_change":"2024-09-14T22:36:18.009776+0000","last_active":"2024-09-14T22:37:32.802626+0000","last_peered":"2024-09-14T22:37:32.802626+0000","last_clean":"2024-09-14T22:37:32.802626+0000","last_became_active":"2024-09-14T22:36:15.723205+0000","last_became_peered":"2024-09-14T22:36:15.723205+0000","last_unstale":"2024-09-14T22:37:32.802626+0000","last_undegraded":"2024-09-14T22:37:32.802626+0000","last_fullsized":"2024-09-14T22:37:32.802626+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:17:42.504389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087783999999999996,"stat_sum":{"num_bytes":25138871,"num_objects":39,"num_object_clones":10,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":13,"num_read":2724,"num_read_kb":13200,"num_write":248,"num_write_kb":4487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"83'302","reported_seq":1699,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.802774+0000","last_change":"2024-09-14T22:36:18.009666+0000","last_active":"2024-09-14T22:37:32.802774+0000","last_peered":"2024-09-14T22:37:32.802774+0000","last_clean":"2024-09-14T22:37:32.802774+0000","last_became_active":"2024-09-14T22:36:15.715659+0000","last_became_peered":"2024-09-14T22:36:15.715659+0000","last_unstale":"2024-09-14T22:37:32.802774+0000","last_undegraded":"2024-09-14T22:37:32.802774+0000","last_fullsized":"2024-09-14T22:37:32.802774+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:34:50.780979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000776448,"stat_sum":{"num_bytes":31940998,"num_objects":35,"num_object_clones":10,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":10,"num_read":2029,"num_read_kb":7936,"num_write":366,"num_write_kb":3382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"82'228","reported_seq":7001,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.802887+0000","last_change":"2024-09-14T22:36:18.010182+0000","last_active":"2024-09-14T22:37:32.802887+0000","last_peered":"2024-09-14T22:37:32.802887+0000","last_clean":"2024-09-14T22:37:32.802887+0000","last_became_active":"2024-09-14T22:36:15.719693+0000","last_became_peered":"2024-09-14T22:36:15.719693+0000","last_unstale":"2024-09-14T22:37:32.802887+0000","last_undegraded":"2024-09-14T22:37:32.802887+0000","last_fullsized":"2024-09-14T22:37:32.802887+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:16:47.696472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001045011,"stat_sum":{"num_bytes":23908902,"num_objects":38,"num_object_clones":10,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":9,"num_read":6904,"num_read_kb":19887,"num_write":300,"num_write_kb":9182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"83'300","reported_seq":3125,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.808246+0000","last_change":"2024-09-14T22:37:26.718624+0000","last_active":"2024-09-14T22:37:32.808246+0000","last_peered":"2024-09-14T22:37:32.808246+0000","last_clean":"2024-09-14T22:37:32.808246+0000","last_became_active":"2024-09-14T22:37:26.717088+0000","last_became_peered":"2024-09-14T22:37:26.717088+0000","last_unstale":"2024-09-14T22:37:32.808246+0000","last_undegraded":"2024-09-14T22:37:32.808246+0000","last_fullsized":"2024-09-14T22:37:32.808246+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:59:19.875480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094227299999999998,"stat_sum":{"num_bytes":35099769,"num_objects":38,"num_object_clones":12,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":14,"num_read":2958,"num_read_kb":24411,"num_write":360,"num_write_kb":12096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"82'245","reported_seq":1705,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.802219+0000","last_change":"2024-09-14T22:36:18.009347+0000","last_active":"2024-09-14T22:37:32.802219+0000","last_peered":"2024-09-14T22:37:32.802219+0000","last_clean":"2024-09-14T22:37:32.802219+0000","last_became_active":"2024-09-14T22:36:15.715569+0000","last_became_peered":"2024-09-14T22:36:15.715569+0000","last_unstale":"2024-09-14T22:37:32.802219+0000","last_undegraded":"2024-09-14T22:37:32.802219+0000","last_fullsized":"2024-09-14T22:37:32.802219+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:44:30.757450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054921600000000005,"stat_sum":{"num_bytes":29773590,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":11,"num_read":1947,"num_read_kb":10189,"num_write":308,"num_write_kb":9963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"83'247","reported_seq":4705,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.809537+0000","last_change":"2024-09-14T22:36:17.720448+0000","last_active":"2024-09-14T22:37:32.809537+0000","last_peered":"2024-09-14T22:37:32.809537+0000","last_clean":"2024-09-14T22:37:32.809537+0000","last_became_active":"2024-09-14T22:36:15.716696+0000","last_became_peered":"2024-09-14T22:36:15.716696+0000","last_unstale":"2024-09-14T22:37:32.809537+0000","last_undegraded":"2024-09-14T22:37:32.809537+0000","last_fullsized":"2024-09-14T22:37:32.809537+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:30:53.040218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086547600000000001,"stat_sum":{"num_bytes":37561236,"num_objects":38,"num_object_clones":9,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":13,"num_read":5092,"num_read_kb":31570,"num_write":231,"num_write_kb":2117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"82'311","reported_seq":5732,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.802259+0000","last_change":"2024-09-14T22:36:18.010088+0000","last_active":"2024-09-14T22:37:32.802259+0000","last_peered":"2024-09-14T22:37:32.802259+0000","last_clean":"2024-09-14T22:37:32.802259+0000","last_became_active":"2024-09-14T22:36:15.715866+0000","last_became_peered":"2024-09-14T22:36:15.715866+0000","last_unstale":"2024-09-14T22:37:32.802259+0000","last_undegraded":"2024-09-14T22:37:32.802259+0000","last_fullsized":"2024-09-14T22:37:32.802259+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:29:23.921984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010286189999999999,"stat_sum":{"num_bytes":51923917,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":11,"num_read":5879,"num_read_kb":32277,"num_write":375,"num_write_kb":14482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1110429226,"num_objects":1155,"num_object_clones":320,"num_object_copies":2310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1155,"num_whiteouts":384,"num_read":148248,"num_read_kb":544853,"num_write":15989,"num_write_kb":252342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":536,"num_bytes_recovered":27389785,"num_keys_recovered":22,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36093952,"data_stored":351792656,"data_compressed":9938975,"data_compressed_allocated":27869184,"data_compressed_original":347014070,"omap_allocated":4238,"internal_metadata":0},"log_size":12464,"ondisk_log_size":12464,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542166,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50460,"kb_used_data":6796,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94321380,"statfs":{"total":96636764160,"available":96585093120,"internally_reserved":0,"allocated":6959104,"data_stored":54495657,"data_compressed":1865544,"data_compressed_allocated":4472832,"data_compressed_original":53151728,"omap_allocated":1590,"internal_metadata":44693962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542167,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38684,"kb_used_data":3328,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333156,"statfs":{"total":96636764160,"available":96597151744,"internally_reserved":0,"allocated":3407872,"data_stored":29255081,"data_compressed":650805,"data_compressed_allocated":2019328,"data_compressed_original":28582979,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542167,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45588,"kb_used_data":6076,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326252,"statfs":{"total":96636764160,"available":96590082048,"internally_reserved":0,"allocated":6221824,"data_stored":48271090,"data_compressed":1360690,"data_compressed_allocated":4005888,"data_compressed_original":47085710,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542166,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35748,"kb_used_data":8724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336092,"statfs":{"total":96636764160,"available":96600158208,"internally_reserved":0,"allocated":8933376,"data_stored":82247681,"data_compressed":2203556,"data_compressed_allocated":6393856,"data_compressed_original":80917069,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542167,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53268,"kb_used_data":4032,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318572,"statfs":{"total":96636764160,"available":96582217728,"internally_reserved":0,"allocated":4128768,"data_stored":27586773,"data_compressed":1007940,"data_compressed_allocated":2326528,"data_compressed_original":26679302,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":66,"seq":283467841542,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35872,"kb_used_data":3200,"kb_used_omap":6,"kb_used_meta":32633,"kb_avail":94335968,"statfs":{"total":96636764160,"available":96600031232,"internally_reserved":0,"allocated":3276800,"data_stored":24944558,"data_compressed":455862,"data_compressed_allocated":1794048,"data_compressed_original":24168618,"omap_allocated":7025,"internal_metadata":33416335},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53560,"kb_used_data":4328,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318280,"statfs":{"total":96636764160,"available":96581918720,"internally_reserved":0,"allocated":4431872,"data_stored":32764797,"data_compressed":976741,"data_compressed_allocated":2584576,"data_compressed_original":31815514,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542166,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33192,"kb_used_data":6156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338648,"statfs":{"total":96636764160,"available":96602775552,"internally_reserved":0,"allocated":6303744,"data_stored":56427427,"data_compressed":1321620,"data_compressed_allocated":4206592,"data_compressed_original":55346334,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5357568,"data_stored":55564772,"data_compressed":1303342,"data_compressed_allocated":4169728,"data_compressed_original":54887582,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3391488,"data_stored":32217687,"data_compressed":976670,"data_compressed_allocated":2576384,"data_compressed_original":31737690,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2277376,"data_stored":24487279,"data_compressed":455862,"data_compressed_allocated":1794048,"data_compressed_original":24168618,"omap_allocated":4238,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3231744,"data_stored":27266627,"data_compressed":1033526,"data_compressed_allocated":2359296,"data_compressed_original":26781702,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7995392,"data_stored":81821039,"data_compressed":2203556,"data_compressed_allocated":6393856,"data_compressed_original":80917069,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5275648,"data_stored":47882392,"data_compressed":1360703,"data_compressed_allocated":4005888,"data_compressed_original":47126670,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2469888,"data_stored":28900253,"data_compressed":650990,"data_compressed_allocated":2027520,"data_compressed_original":28656707,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":6094848,"data_stored":53652607,"data_compressed":1954326,"data_compressed_allocated":4542464,"data_compressed_original":52738032,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:37:38.300 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:37:38.301 INFO:teuthology.orchestra.run.smithi001.stdout:4654 write 0x4f684f3 thru 0x4f70a45 (0x8553 bytes) 2024-09-14T22:37:38.312 INFO:teuthology.orchestra.run.smithi001.stdout:4655 write 0x72735cd thru 0x7278d07 (0x573b bytes) 2024-09-14T22:37:38.318 INFO:teuthology.orchestra.run.smithi001.stdout:4656 write 0x62dd9c7 thru 0x62e6ea2 (0x94dc bytes) 2024-09-14T22:37:38.320 INFO:teuthology.orchestra.run.smithi001.stdout:4657 writesame 0x49c6e1c thru 0x49cabed data_size 0x29(0x3dd2 bytes) 2024-09-14T22:37:38.322 INFO:teuthology.orchestra.run.smithi001.stdout:4659 writesame 0xaade723 thru 0xaade814 data_size 0x2(0xf2 bytes) 2024-09-14T22:37:38.326 INFO:teuthology.orchestra.run.smithi001.stdout:4660 read 0xbcef275 thru 0xbcfa92e (0xb6ba bytes) 2024-09-14T22:37:38.328 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:38.328 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:38.328 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:37:38.329 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:37:38.329 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:38.345 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:37:38.346 INFO:teuthology.orchestra.run.smithi001.stdout:4662 trunc from 0xd1c0b85 to 0x2750d68 2024-09-14T22:37:38.362 INFO:teuthology.orchestra.run.smithi001.stdout:4663 trunc from 0x2750d68 to 0x26f84fe 2024-09-14T22:37:38.363 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:38.359+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:38.396 INFO:teuthology.orchestra.run.smithi001.stdout:4664 write 0xb381cf4 thru 0xb38dba3 (0xbeb0 bytes) 2024-09-14T22:37:38.402 INFO:teuthology.orchestra.run.smithi001.stdout:4665 write 0x6d2a8b4 thru 0x6d2c723 (0x1e70 bytes) 2024-09-14T22:37:38.407 INFO:teuthology.orchestra.run.smithi001.stdout:4666 clone 27 order 18 su 4096 sc 5 2024-09-14T22:37:38.464 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:38.459+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:38.518 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:38.518 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:38.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:38.518 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:38.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.519 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97197 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 796672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:38.519 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:37:38.519 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.445053262999998, 2024-09-14T22:37:38.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12434473, 2024-09-14T22:37:38.519 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:38.519 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:38.519 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:38.519 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:38.519 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:38.519 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:38.519 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97197 2024-09-14T22:37:38.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.520 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:38.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:38.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:37:38.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:38.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:37:38.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:38.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070524+0000", 2024-09-14T22:37:38.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0299999999999998e-07 2024-09-14T22:37:38.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.521 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:38.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070527+0000", 2024-09-14T22:37:38.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9569999999999998e-06 2024-09-14T22:37:38.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.521 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:38.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070535+0000", 2024-09-14T22:37:38.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7630000000000006e-06 2024-09-14T22:37:38.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:38.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070543+0000", 2024-09-14T22:37:38.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4869999999999996e-06 2024-09-14T22:37:38.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:38.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194525+0000", 2024-09-14T22:37:38.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123982096 2024-09-14T22:37:38.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:38.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194699+0000", 2024-09-14T22:37:38.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000174105 2024-09-14T22:37:38.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:38.523 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194868+0000", 2024-09-14T22:37:38.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016861900000000001 2024-09-14T22:37:38.523 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.523 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:38.523 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.523 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97198 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 805888~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:38.523 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:37:38.523 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.445024886999999, 2024-09-14T22:37:38.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.1248088, 2024-09-14T22:37:38.523 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:38.523 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:38.523 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:38.524 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:38.524 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:38.524 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:38.524 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97198 2024-09-14T22:37:38.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.524 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:38.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:38.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:37:38.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:38.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:37:38.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:38.525 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070552+0000", 2024-09-14T22:37:38.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T22:37:38.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:38.525 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070555+0000", 2024-09-14T22:37:38.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.853e-06 2024-09-14T22:37:38.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:38.525 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070565+0000", 2024-09-14T22:37:38.526 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.048e-05 2024-09-14T22:37:38.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.526 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:38.526 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070573+0000", 2024-09-14T22:37:38.526 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7200000000000006e-06 2024-09-14T22:37:38.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.526 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:38.526 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194954+0000", 2024-09-14T22:37:38.526 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12438089400000001 2024-09-14T22:37:38.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.526 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:38.526 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195145+0000", 2024-09-14T22:37:38.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019091299999999999 2024-09-14T22:37:38.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:38.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195360+0000", 2024-09-14T22:37:38.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021527300000000001 2024-09-14T22:37:38.527 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.527 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:38.527 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.527 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97199 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 815104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:38.527 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:37:38.527 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.44499429, 2024-09-14T22:37:38.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125251901, 2024-09-14T22:37:38.528 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:38.528 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:38.528 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:38.528 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:38.528 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:38.528 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:38.528 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97199 2024-09-14T22:37:38.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.528 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:38.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.528 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:38.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:37:38.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:38.529 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:37:38.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:38.529 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070583+0000", 2024-09-14T22:37:38.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6300000000000005e-07 2024-09-14T22:37:38.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:38.529 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070586+0000", 2024-09-14T22:37:38.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8270000000000002e-06 2024-09-14T22:37:38.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:38.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070597+0000", 2024-09-14T22:37:38.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1960000000000001e-05 2024-09-14T22:37:38.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:38.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070605+0000", 2024-09-14T22:37:38.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6459999999999993e-06 2024-09-14T22:37:38.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:38.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195432+0000", 2024-09-14T22:37:38.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124826926 2024-09-14T22:37:38.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:38.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195626+0000", 2024-09-14T22:37:38.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019357099999999999 2024-09-14T22:37:38.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:38.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195834+0000", 2024-09-14T22:37:38.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020830800000000001 2024-09-14T22:37:38.531 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.532 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:38.532 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.532 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97200 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 824320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:38.532 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:37:38.532 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.444962243999999, 2024-09-14T22:37:38.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125693477, 2024-09-14T22:37:38.532 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:38.532 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:38.532 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:38.533 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:38.533 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:38.533 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:38.533 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97200 2024-09-14T22:37:38.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.533 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:38.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:38.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:37:38.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.534 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:38.534 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:37:38.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.534 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:38.534 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070615+0000", 2024-09-14T22:37:38.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T22:37:38.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.534 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:38.534 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070618+0000", 2024-09-14T22:37:38.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.943e-06 2024-09-14T22:37:38.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:38.535 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070627+0000", 2024-09-14T22:37:38.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1099999999999992e-06 2024-09-14T22:37:38.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:38.535 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070634+0000", 2024-09-14T22:37:38.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:37:38.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:38.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195908+0000", 2024-09-14T22:37:38.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125273893 2024-09-14T22:37:38.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:38.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196097+0000", 2024-09-14T22:37:38.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018953899999999999 2024-09-14T22:37:38.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:38.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196308+0000", 2024-09-14T22:37:38.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021027799999999999 2024-09-14T22:37:38.537 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.537 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:38.537 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.537 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97201 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 833536~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:38.537 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:37:38.537 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.444932056999999, 2024-09-14T22:37:38.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126133826, 2024-09-14T22:37:38.537 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:38.537 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:38.537 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:38.537 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:38.538 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:38.538 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:38.538 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97201 2024-09-14T22:37:38.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.538 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:38.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.538 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:38.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:37:38.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:38.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:37:38.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:38.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074109+0000", 2024-09-14T22:37:38.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003464834 2024-09-14T22:37:38.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:38.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074134+0000", 2024-09-14T22:37:38.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5170000000000001e-05 2024-09-14T22:37:38.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:38.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074166+0000", 2024-09-14T22:37:38.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1347000000000003e-05 2024-09-14T22:37:38.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:38.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074179+0000", 2024-09-14T22:37:38.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.364e-05 2024-09-14T22:37:38.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:38.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196377+0000", 2024-09-14T22:37:38.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12219793900000001 2024-09-14T22:37:38.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:38.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196568+0000", 2024-09-14T22:37:38.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019097999999999999 2024-09-14T22:37:38.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:38.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196778+0000", 2024-09-14T22:37:38.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020991599999999999 2024-09-14T22:37:38.542 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.542 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:38.542 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.542 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97205 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 870400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:38.542 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:37:38.542 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.441262494, 2024-09-14T22:37:38.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12436148299999999, 2024-09-14T22:37:38.542 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:38.542 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:38.543 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:38.543 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:38.543 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:38.543 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:38.543 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97205 2024-09-14T22:37:38.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.543 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:38.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:38.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:37:38.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.544 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:38.544 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:37:38.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.544 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:38.544 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074315+0000", 2024-09-14T22:37:38.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9299999999999997e-07 2024-09-14T22:37:38.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.544 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:38.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074318+0000", 2024-09-14T22:37:38.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.19e-06 2024-09-14T22:37:38.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:38.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074327+0000", 2024-09-14T22:37:38.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8529999999999998e-06 2024-09-14T22:37:38.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:38.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074334+0000", 2024-09-14T22:37:38.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3869999999999997e-06 2024-09-14T22:37:38.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:38.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198365+0000", 2024-09-14T22:37:38.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124031222 2024-09-14T22:37:38.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:38.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198513+0000", 2024-09-14T22:37:38.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014772299999999999 2024-09-14T22:37:38.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:38.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198675+0000", 2024-09-14T22:37:38.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016241500000000001 2024-09-14T22:37:38.547 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.547 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:38.547 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.547 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.547 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97206 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 879616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:38.547 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:37:38.547 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.441231038000002, 2024-09-14T22:37:38.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124706284, 2024-09-14T22:37:38.547 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:38.547 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:38.547 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:38.548 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:38.548 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:38.548 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:38.548 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97206 2024-09-14T22:37:38.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.548 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:38.548 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.548 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:38.548 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:37:38.548 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.549 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:38.549 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:37:38.549 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.549 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:38.549 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074346+0000", 2024-09-14T22:37:38.549 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8999999999999996e-07 2024-09-14T22:37:38.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.549 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:38.549 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074349+0000", 2024-09-14T22:37:38.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.96e-06 2024-09-14T22:37:38.550 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.550 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:38.550 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074358+0000", 2024-09-14T22:37:38.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9169999999999995e-06 2024-09-14T22:37:38.550 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.550 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:38.550 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074365+0000", 2024-09-14T22:37:38.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3699999999999997e-06 2024-09-14T22:37:38.550 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.551 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:38.551 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198761+0000", 2024-09-14T22:37:38.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124395743 2024-09-14T22:37:38.551 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.551 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:38.551 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198899+0000", 2024-09-14T22:37:38.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013763499999999999 2024-09-14T22:37:38.551 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.551 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:38.551 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199052+0000", 2024-09-14T22:37:38.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015296900000000001 2024-09-14T22:37:38.552 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.552 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:38.552 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.552 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97207 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 888832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:38.552 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:37:38.552 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.441201825, 2024-09-14T22:37:38.552 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124988433, 2024-09-14T22:37:38.552 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:38.553 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:38.553 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:38.553 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:38.553 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:38.553 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:38.553 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97207 2024-09-14T22:37:38.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.553 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:38.553 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.553 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:38.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:37:38.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.553 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.554 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:38.554 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:37:38.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.554 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.554 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:38.554 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074375+0000", 2024-09-14T22:37:38.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:37:38.554 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.554 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:38.554 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074378+0000", 2024-09-14T22:37:38.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1829999999999998e-06 2024-09-14T22:37:38.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.555 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.555 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:38.555 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074389+0000", 2024-09-14T22:37:38.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0737000000000001e-05 2024-09-14T22:37:38.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.555 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.555 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:38.555 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074396+0000", 2024-09-14T22:37:38.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2559999999999996e-06 2024-09-14T22:37:38.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.555 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.555 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:38.555 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199139+0000", 2024-09-14T22:37:38.556 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12474253 2024-09-14T22:37:38.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.556 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.556 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:38.556 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199240+0000", 2024-09-14T22:37:38.556 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101221 2024-09-14T22:37:38.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.556 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.556 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:38.556 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199363+0000", 2024-09-14T22:37:38.556 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000122822 2024-09-14T22:37:38.556 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.556 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:38.556 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.557 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.557 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97208 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 898048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:38.557 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:37:38.557 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.441170521, 2024-09-14T22:37:38.557 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12524854899999999, 2024-09-14T22:37:38.557 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:38.557 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:38.557 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:38.557 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:38.557 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:38.557 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:38.558 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97208 2024-09-14T22:37:38.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.558 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:38.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.558 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:38.558 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:37:38.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.558 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:38.558 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:37:38.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.559 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:38.559 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:37:38.559 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:37:38.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.559 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.559 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:38.559 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074410+0000", 2024-09-14T22:37:38.559 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.18e-06 2024-09-14T22:37:38.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.559 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.559 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:38.559 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074435+0000", 2024-09-14T22:37:38.559 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5273e-05 2024-09-14T22:37:38.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.560 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.560 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:38.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074443+0000", 2024-09-14T22:37:38.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3000000000000002e-06 2024-09-14T22:37:38.560 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.560 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.560 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:38.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199433+0000", 2024-09-14T22:37:38.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124989502 2024-09-14T22:37:38.560 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.560 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.560 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:38.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199535+0000", 2024-09-14T22:37:38.561 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102557 2024-09-14T22:37:38.561 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.561 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.561 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:38.561 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199654+0000", 2024-09-14T22:37:38.561 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119061 2024-09-14T22:37:38.561 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.561 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:38.561 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.561 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.561 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.561 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97209 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 907264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:38.561 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:37:38.562 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.441122715999999, 2024-09-14T22:37:38.562 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125487354, 2024-09-14T22:37:38.562 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:38.562 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:38.562 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:38.562 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:38.562 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:38.562 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:38.562 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97209 2024-09-14T22:37:38.562 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.562 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:38.562 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.562 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:38.562 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:37:38.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.563 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.563 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:38.563 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:37:38.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.563 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.563 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:38.563 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:37:38.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9400000000000005e-07 2024-09-14T22:37:38.563 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.563 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:38.564 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074458+0000", 2024-09-14T22:37:38.564 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3299999999999999e-06 2024-09-14T22:37:38.564 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.564 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.564 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:38.564 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074467+0000", 2024-09-14T22:37:38.564 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.346e-06 2024-09-14T22:37:38.564 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.564 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.564 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:38.564 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074474+0000", 2024-09-14T22:37:38.564 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.374e-06 2024-09-14T22:37:38.564 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.565 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.565 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:38.565 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199720+0000", 2024-09-14T22:37:38.565 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12524561200000001 2024-09-14T22:37:38.565 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.565 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.565 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:38.565 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199820+0000", 2024-09-14T22:37:38.565 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010049599999999999 2024-09-14T22:37:38.565 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.565 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.565 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:38.565 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199941+0000", 2024-09-14T22:37:38.566 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012050200000000001 2024-09-14T22:37:38.566 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.566 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:38.566 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.566 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.566 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.566 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97210 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:38.566 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:37:38.566 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.441092722, 2024-09-14T22:37:38.566 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125744942, 2024-09-14T22:37:38.566 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:38.566 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:38.566 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:38.566 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:38.567 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:38.567 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:38.567 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97210 2024-09-14T22:37:38.567 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.567 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:38.567 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.567 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:38.567 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:37:38.567 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.567 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.567 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.568 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:38.568 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:37:38.568 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.568 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.568 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.568 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:38.568 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:37:38.568 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6599999999999996e-07 2024-09-14T22:37:38.568 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.568 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.568 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:38.568 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074487+0000", 2024-09-14T22:37:38.568 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8140000000000001e-06 2024-09-14T22:37:38.569 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.569 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.569 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:38.569 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074496+0000", 2024-09-14T22:37:38.569 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9330000000000007e-06 2024-09-14T22:37:38.569 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.569 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.569 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:38.569 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074503+0000", 2024-09-14T22:37:38.569 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4100000000000002e-06 2024-09-14T22:37:38.569 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.569 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.569 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:38.569 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200009+0000", 2024-09-14T22:37:38.570 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12550539899999999 2024-09-14T22:37:38.570 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.570 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.570 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:38.570 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200108+0000", 2024-09-14T22:37:38.570 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9182e-05 2024-09-14T22:37:38.570 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.570 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.570 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:38.570 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200229+0000", 2024-09-14T22:37:38.570 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120538 2024-09-14T22:37:38.570 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.570 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:38.570 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.570 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.570 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.571 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97211 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 925696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:38.571 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:37:38.571 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.441061762, 2024-09-14T22:37:38.571 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12599848299999999, 2024-09-14T22:37:38.571 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:38.571 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:38.571 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:38.571 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:38.571 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:38.571 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:38.571 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97211 2024-09-14T22:37:38.571 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.571 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:38.572 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.572 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:38.572 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:37:38.572 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.572 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.572 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.572 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:38.572 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:37:38.572 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.572 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.572 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.572 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:38.572 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074516+0000", 2024-09-14T22:37:38.573 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.42e-06 2024-09-14T22:37:38.573 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.573 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.573 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:38.573 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074521+0000", 2024-09-14T22:37:38.573 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7960000000000003e-06 2024-09-14T22:37:38.573 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.573 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.573 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:38.573 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074538+0000", 2024-09-14T22:37:38.573 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6900000000000001e-05 2024-09-14T22:37:38.573 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.573 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.573 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:38.574 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074555+0000", 2024-09-14T22:37:38.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.696e-05 2024-09-14T22:37:38.574 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.574 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.574 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:38.574 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200293+0000", 2024-09-14T22:37:38.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12573864100000001 2024-09-14T22:37:38.574 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.574 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.574 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:38.574 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200393+0000", 2024-09-14T22:37:38.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9264000000000004e-05 2024-09-14T22:37:38.574 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.575 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.575 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:38.575 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200513+0000", 2024-09-14T22:37:38.575 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012050200000000001 2024-09-14T22:37:38.575 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.575 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:38.575 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.575 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.575 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.575 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97212 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 934912~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:38.575 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:37:38.575 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.440987140000001, 2024-09-14T22:37:38.575 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12621127300000001, 2024-09-14T22:37:38.575 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:38.576 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:38.576 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:38.576 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:38.576 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:38.576 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:38.576 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97212 2024-09-14T22:37:38.576 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.576 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:38.576 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.576 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:38.576 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:37:38.576 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.576 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.576 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.577 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:38.577 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:37:38.577 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.577 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.577 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.577 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:38.577 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074590+0000", 2024-09-14T22:37:38.577 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1969999999999999e-06 2024-09-14T22:37:38.577 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.577 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.577 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:38.577 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074596+0000", 2024-09-14T22:37:38.577 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7629999999999997e-06 2024-09-14T22:37:38.578 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.578 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:38.578 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074614+0000", 2024-09-14T22:37:38.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7643e-05 2024-09-14T22:37:38.578 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.578 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:38.578 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074635+0000", 2024-09-14T22:37:38.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.075e-05 2024-09-14T22:37:38.578 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.578 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:38.579 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200580+0000", 2024-09-14T22:37:38.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125945482 2024-09-14T22:37:38.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.579 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:38.579 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200683+0000", 2024-09-14T22:37:38.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010267099999999999 2024-09-14T22:37:38.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.579 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:38.579 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200800+0000", 2024-09-14T22:37:38.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117767 2024-09-14T22:37:38.579 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.580 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:38.580 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.580 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97213 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 944128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:38.580 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:37:38.580 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.440921790000001, 2024-09-14T22:37:38.580 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126430709, 2024-09-14T22:37:38.580 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:38.580 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:38.580 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:38.580 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:38.580 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:38.580 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:38.581 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97213 2024-09-14T22:37:38.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.581 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:38.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.581 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:38.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:37:38.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.581 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:38.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:37:38.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.582 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:38.582 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078210+0000", 2024-09-14T22:37:38.582 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0035556839999999999 2024-09-14T22:37:38.582 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.582 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:38.582 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078231+0000", 2024-09-14T22:37:38.582 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0716e-05 2024-09-14T22:37:38.582 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.582 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:38.582 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078260+0000", 2024-09-14T22:37:38.582 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9473000000000001e-05 2024-09-14T22:37:38.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.583 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:38.583 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078279+0000", 2024-09-14T22:37:38.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8600000000000001e-05 2024-09-14T22:37:38.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.583 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:38.583 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200864+0000", 2024-09-14T22:37:38.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.122585295 2024-09-14T22:37:38.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.583 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:38.583 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200965+0000", 2024-09-14T22:37:38.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101004 2024-09-14T22:37:38.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:38.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.201085+0000", 2024-09-14T22:37:38.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119937 2024-09-14T22:37:38.584 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.584 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:38.584 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.584 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97219 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 999424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:38.585 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:37:38.585 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.436939512999999, 2024-09-14T22:37:38.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124138558, 2024-09-14T22:37:38.585 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:38.585 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:38.585 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:38.585 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:38.585 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:38.585 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:38.585 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97219 2024-09-14T22:37:38.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.585 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:38.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:38.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:37:38.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.586 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:38.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:37:38.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.586 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:38.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078638+0000", 2024-09-14T22:37:38.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.173e-06 2024-09-14T22:37:38.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.587 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:38.587 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078644+0000", 2024-09-14T22:37:38.587 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2570000000000004e-06 2024-09-14T22:37:38.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.587 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:38.587 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078672+0000", 2024-09-14T22:37:38.587 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7322999999999999e-05 2024-09-14T22:37:38.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.587 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:38.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078687+0000", 2024-09-14T22:37:38.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5639999999999999e-05 2024-09-14T22:37:38.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:38.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202567+0000", 2024-09-14T22:37:38.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123880134 2024-09-14T22:37:38.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:38.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202670+0000", 2024-09-14T22:37:38.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010273899999999999 2024-09-14T22:37:38.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:38.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202775+0000", 2024-09-14T22:37:38.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010529199999999999 2024-09-14T22:37:38.589 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.589 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:38.589 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.589 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97220 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1008640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:38.589 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:37:38.590 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.436870280000001, 2024-09-14T22:37:38.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12435937, 2024-09-14T22:37:38.590 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:38.590 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:38.590 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:38.590 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:38.590 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:38.590 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:38.590 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97220 2024-09-14T22:37:38.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.590 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:38.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.591 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:38.591 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:37:38.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.591 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:38.591 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:37:38.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.591 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:38.591 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078707+0000", 2024-09-14T22:37:38.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.266e-06 2024-09-14T22:37:38.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.592 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:38.592 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078713+0000", 2024-09-14T22:37:38.592 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7270000000000004e-06 2024-09-14T22:37:38.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.592 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:38.592 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078737+0000", 2024-09-14T22:37:38.592 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3640000000000001e-05 2024-09-14T22:37:38.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.592 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:38.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078751+0000", 2024-09-14T22:37:38.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.45e-05 2024-09-14T22:37:38.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.593 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:38.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202841+0000", 2024-09-14T22:37:38.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124089938 2024-09-14T22:37:38.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.593 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:38.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202944+0000", 2024-09-14T22:37:38.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010334 2024-09-14T22:37:38.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:38.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203065+0000", 2024-09-14T22:37:38.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012095900000000001 2024-09-14T22:37:38.594 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.594 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:38.594 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.594 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97221 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1017856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:38.595 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:37:38.595 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.436804516999999, 2024-09-14T22:37:38.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124574005, 2024-09-14T22:37:38.595 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:38.595 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:38.595 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:38.595 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:38.595 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:38.595 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:38.596 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97221 2024-09-14T22:37:38.596 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.596 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:38.596 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.596 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:38.596 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:37:38.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.596 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.596 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.596 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:38.596 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:37:38.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.596 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.597 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:38.597 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078773+0000", 2024-09-14T22:37:38.597 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.13e-06 2024-09-14T22:37:38.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.597 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:38.597 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078779+0000", 2024-09-14T22:37:38.597 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0260000000000004e-06 2024-09-14T22:37:38.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:38.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078797+0000", 2024-09-14T22:37:38.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.806e-05 2024-09-14T22:37:38.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:38.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078810+0000", 2024-09-14T22:37:38.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.308e-05 2024-09-14T22:37:38.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:38.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203131+0000", 2024-09-14T22:37:38.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12432077499999999 2024-09-14T22:37:38.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:38.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203227+0000", 2024-09-14T22:37:38.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6575000000000003e-05 2024-09-14T22:37:38.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:38.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203346+0000", 2024-09-14T22:37:38.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118359 2024-09-14T22:37:38.600 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.600 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:38.600 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.600 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97222 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:38.600 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:37:38.600 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.436748148, 2024-09-14T22:37:38.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12480110799999999, 2024-09-14T22:37:38.600 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:38.600 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:38.600 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:38.600 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:38.601 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:38.601 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:38.601 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97222 2024-09-14T22:37:38.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.601 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:38.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:38.601 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:37:38.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:38.601 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:37:38.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.602 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:38.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078829+0000", 2024-09-14T22:37:38.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.057e-06 2024-09-14T22:37:38.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.602 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:38.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078836+0000", 2024-09-14T22:37:38.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6159999999999998e-06 2024-09-14T22:37:38.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:38.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078856+0000", 2024-09-14T22:37:38.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0100000000000001e-05 2024-09-14T22:37:38.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:38.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078869+0000", 2024-09-14T22:37:38.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.272e-05 2024-09-14T22:37:38.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:38.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203412+0000", 2024-09-14T22:37:38.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12454283300000001 2024-09-14T22:37:38.604 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.604 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:38.604 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203511+0000", 2024-09-14T22:37:38.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9893000000000001e-05 2024-09-14T22:37:38.604 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.604 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:38.604 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203629+0000", 2024-09-14T22:37:38.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117889 2024-09-14T22:37:38.604 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.604 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:38.605 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.605 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.605 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.605 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97223 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1036288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:38.605 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:37:38.605 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.436376160999998, 2024-09-14T22:37:38.605 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124706279, 2024-09-14T22:37:38.605 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:38.605 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:38.605 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:38.605 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:38.606 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:38.606 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:38.606 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97223 2024-09-14T22:37:38.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.606 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:38.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.606 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:38.606 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:37:38.606 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.606 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:38.606 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:37:38.606 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:38.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079202+0000", 2024-09-14T22:37:38.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8330000000000001e-06 2024-09-14T22:37:38.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:38.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079212+0000", 2024-09-14T22:37:38.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0207000000000001e-05 2024-09-14T22:37:38.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:38.608 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079238+0000", 2024-09-14T22:37:38.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5626000000000002e-05 2024-09-14T22:37:38.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:38.608 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079256+0000", 2024-09-14T22:37:38.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8386999999999999e-05 2024-09-14T22:37:38.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:38.608 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203691+0000", 2024-09-14T22:37:38.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124434538 2024-09-14T22:37:38.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.609 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.609 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:38.609 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203786+0000", 2024-09-14T22:37:38.609 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5344000000000001e-05 2024-09-14T22:37:38.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.609 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.609 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:38.609 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203906+0000", 2024-09-14T22:37:38.609 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120344 2024-09-14T22:37:38.609 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.609 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:38.609 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.610 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.610 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.610 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97224 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1045504~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:38.610 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:37:38.610 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.436292341000001, 2024-09-14T22:37:38.610 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12513321199999999, 2024-09-14T22:37:38.610 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:38.610 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:38.610 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:38.610 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:38.610 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:38.610 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:38.611 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97224 2024-09-14T22:37:38.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.611 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:38.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.611 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:38.611 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:37:38.611 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.611 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:38.611 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:37:38.611 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:38.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:38.612 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082581+0000", 2024-09-14T22:37:38.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0032971530000000001 2024-09-14T22:37:38.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:38.612 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082592+0000", 2024-09-14T22:37:38.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1345999999999999e-05 2024-09-14T22:37:38.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:38.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082618+0000", 2024-09-14T22:37:38.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5873000000000001e-05 2024-09-14T22:37:38.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.613 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:38.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082638+0000", 2024-09-14T22:37:38.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9565999999999999e-05 2024-09-14T22:37:38.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.613 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:38.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204004+0000", 2024-09-14T22:37:38.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.121366038 2024-09-14T22:37:38.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.614 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:38.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204188+0000", 2024-09-14T22:37:38.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018427 2024-09-14T22:37:38.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:38.614 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:38.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:38.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204417+0000", 2024-09-14T22:37:38.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022896599999999999 2024-09-14T22:37:38.614 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.615 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:38.615 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.615 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:38.615 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:38.615 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:38.617 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:37:38.618 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-14T22:37:38.618 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:37:38.619 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:38.611+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:38.619 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":131,"stamp":"2024-09-14T22:37:37.810191+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1111019613,"num_objects":1159,"num_object_clones":320,"num_object_copies":2318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1159,"num_whiteouts":384,"num_read":148364,"num_read_kb":544953,"num_write":16150,"num_write_kb":254760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":536,"num_bytes_recovered":27389785,"num_keys_recovered":22,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12584,"ondisk_log_size":12584,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":346372,"kb_used_data":42640,"kb_used_omap":17,"kb_used_meta":303534,"kb_avail":754628348,"statfs":{"total":773094113280,"available":772739428352,"internally_reserved":0,"allocated":43663360,"data_stored":355993064,"data_compressed":9842758,"data_compressed_allocated":27803648,"data_compressed_original":347747254,"omap_allocated":18180,"internal_metadata":310819068},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.025787"},"pg_stats":[{"pgid":"3.1f","version":"81'242","reported_seq":4120,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.808574+0000","last_change":"2024-09-14T22:37:25.739233+0000","last_active":"2024-09-14T22:37:32.808574+0000","last_peered":"2024-09-14T22:37:32.808574+0000","last_clean":"2024-09-14T22:37:32.808574+0000","last_became_active":"2024-09-14T22:37:25.718906+0000","last_became_peered":"2024-09-14T22:37:25.718906+0000","last_unstale":"2024-09-14T22:37:32.808574+0000","last_undegraded":"2024-09-14T22:37:32.808574+0000","last_fullsized":"2024-09-14T22:37:32.808574+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:55:57.908853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017190879999999999,"stat_sum":{"num_bytes":51625811,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":14,"num_read":3899,"num_read_kb":15185,"num_write":258,"num_write_kb":6027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":3277632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"79'921","reported_seq":4839,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.809140+0000","last_change":"2024-09-14T22:36:17.733296+0000","last_active":"2024-09-14T22:37:32.809140+0000","last_peered":"2024-09-14T22:37:32.809140+0000","last_clean":"2024-09-14T22:37:32.809140+0000","last_became_active":"2024-09-14T22:36:15.709272+0000","last_became_peered":"2024-09-14T22:36:15.709272+0000","last_unstale":"2024-09-14T22:37:32.809140+0000","last_undegraded":"2024-09-14T22:37:32.809140+0000","last_fullsized":"2024-09-14T22:37:32.809140+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":921,"log_dups_size":0,"ondisk_log_size":921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:49:47.937464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001723619,"stat_sum":{"num_bytes":62550087,"num_objects":32,"num_object_clones":9,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":12,"num_read":8781,"num_read_kb":22718,"num_write":1480,"num_write_kb":8170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"79'314","reported_seq":7571,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.809887+0000","last_change":"2024-09-14T22:36:17.702313+0000","last_active":"2024-09-14T22:37:32.809887+0000","last_peered":"2024-09-14T22:37:32.809887+0000","last_clean":"2024-09-14T22:37:32.809887+0000","last_became_active":"2024-09-14T22:36:15.719488+0000","last_became_peered":"2024-09-14T22:36:15.719488+0000","last_unstale":"2024-09-14T22:37:32.809887+0000","last_undegraded":"2024-09-14T22:37:32.809887+0000","last_fullsized":"2024-09-14T22:37:32.809887+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:20:16.819676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053914200000000005,"stat_sum":{"num_bytes":64267617,"num_objects":52,"num_object_clones":19,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":20,"num_read":7454,"num_read_kb":40817,"num_write":291,"num_write_kb":8757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"82'302","reported_seq":3003,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.809149+0000","last_change":"2024-09-14T22:36:17.702413+0000","last_active":"2024-09-14T22:37:32.809149+0000","last_peered":"2024-09-14T22:37:32.809149+0000","last_clean":"2024-09-14T22:37:32.809149+0000","last_became_active":"2024-09-14T22:36:15.719470+0000","last_became_peered":"2024-09-14T22:36:15.719470+0000","last_unstale":"2024-09-14T22:37:32.809149+0000","last_undegraded":"2024-09-14T22:37:32.809149+0000","last_fullsized":"2024-09-14T22:37:32.809149+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:48:58.357123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055079499999999997,"stat_sum":{"num_bytes":26279920,"num_objects":45,"num_object_clones":12,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":19,"num_read":2674,"num_read_kb":14679,"num_write":333,"num_write_kb":7367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'395","reported_seq":4551,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.809439+0000","last_change":"2024-09-14T22:37:25.736579+0000","last_active":"2024-09-14T22:37:32.809439+0000","last_peered":"2024-09-14T22:37:32.809439+0000","last_clean":"2024-09-14T22:37:32.809439+0000","last_became_active":"2024-09-14T22:37:25.718519+0000","last_became_peered":"2024-09-14T22:37:25.718519+0000","last_unstale":"2024-09-14T22:37:32.809439+0000","last_undegraded":"2024-09-14T22:37:32.809439+0000","last_fullsized":"2024-09-14T22:37:32.809439+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:55:01.095884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092291499999999996,"stat_sum":{"num_bytes":25177553,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":11,"num_read":4929,"num_read_kb":28545,"num_write":494,"num_write_kb":8858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":7858104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"82'328","reported_seq":2521,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.876023+0000","last_change":"2024-09-14T22:36:17.711364+0000","last_active":"2024-09-14T22:37:32.876023+0000","last_peered":"2024-09-14T22:37:32.876023+0000","last_clean":"2024-09-14T22:37:32.876023+0000","last_became_active":"2024-09-14T22:36:15.701852+0000","last_became_peered":"2024-09-14T22:36:15.701852+0000","last_unstale":"2024-09-14T22:37:32.876023+0000","last_undegraded":"2024-09-14T22:37:32.876023+0000","last_fullsized":"2024-09-14T22:37:32.876023+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:17:55.480586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012502780000000001,"stat_sum":{"num_bytes":8128485,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":17,"num_read":3159,"num_read_kb":9131,"num_write":350,"num_write_kb":5788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"87'324","reported_seq":4979,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T22:37:36.814271+0000","last_change":"2024-09-14T22:36:17.705423+0000","last_active":"2024-09-14T22:37:36.814271+0000","last_peered":"2024-09-14T22:37:36.814271+0000","last_clean":"2024-09-14T22:37:36.814271+0000","last_became_active":"2024-09-14T22:36:15.700990+0000","last_became_peered":"2024-09-14T22:36:15.700990+0000","last_unstale":"2024-09-14T22:37:36.814271+0000","last_undegraded":"2024-09-14T22:37:36.814271+0000","last_fullsized":"2024-09-14T22:37:36.814271+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:20:52.988019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010520620000000001,"stat_sum":{"num_bytes":20139232,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":11,"num_read":5021,"num_read_kb":15102,"num_write":342,"num_write_kb":5325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"82'221","reported_seq":2728,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.808709+0000","last_change":"2024-09-14T22:37:25.719758+0000","last_active":"2024-09-14T22:37:32.808709+0000","last_peered":"2024-09-14T22:37:32.808709+0000","last_clean":"2024-09-14T22:37:32.808709+0000","last_became_active":"2024-09-14T22:37:25.717354+0000","last_became_peered":"2024-09-14T22:37:25.717354+0000","last_unstale":"2024-09-14T22:37:32.808709+0000","last_undegraded":"2024-09-14T22:37:32.808709+0000","last_fullsized":"2024-09-14T22:37:32.808709+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:58:50.594733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002063484,"stat_sum":{"num_bytes":19040318,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":4,"num_read":2527,"num_read_kb":18581,"num_write":293,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":633727,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"83'219","reported_seq":3144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.822406+0000","last_change":"2024-09-14T22:36:17.721195+0000","last_active":"2024-09-14T22:37:32.822406+0000","last_peered":"2024-09-14T22:37:32.822406+0000","last_clean":"2024-09-14T22:37:32.822406+0000","last_became_active":"2024-09-14T22:36:15.715869+0000","last_became_peered":"2024-09-14T22:36:15.715869+0000","last_unstale":"2024-09-14T22:37:32.822406+0000","last_undegraded":"2024-09-14T22:37:32.822406+0000","last_fullsized":"2024-09-14T22:37:32.822406+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:47:17.519839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001160672,"stat_sum":{"num_bytes":19658672,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":11,"num_read":3182,"num_read_kb":13742,"num_write":255,"num_write_kb":14649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"79'393","reported_seq":3887,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.810514+0000","last_change":"2024-09-14T22:36:17.703048+0000","last_active":"2024-09-14T22:37:32.810514+0000","last_peered":"2024-09-14T22:37:32.810514+0000","last_clean":"2024-09-14T22:37:32.810514+0000","last_became_active":"2024-09-14T22:36:15.725078+0000","last_became_peered":"2024-09-14T22:36:15.725078+0000","last_unstale":"2024-09-14T22:37:32.810514+0000","last_undegraded":"2024-09-14T22:37:32.810514+0000","last_fullsized":"2024-09-14T22:37:32.810514+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:19:54.219161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010500290000000001,"stat_sum":{"num_bytes":59689882,"num_objects":49,"num_object_clones":14,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":15,"num_read":4376,"num_read_kb":28795,"num_write":533,"num_write_kb":11852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"82'228","reported_seq":2888,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.802324+0000","last_change":"2024-09-14T22:36:18.009371+0000","last_active":"2024-09-14T22:37:32.802324+0000","last_peered":"2024-09-14T22:37:32.802324+0000","last_clean":"2024-09-14T22:37:32.802324+0000","last_became_active":"2024-09-14T22:36:15.717341+0000","last_became_peered":"2024-09-14T22:36:15.717341+0000","last_unstale":"2024-09-14T22:37:32.802324+0000","last_undegraded":"2024-09-14T22:37:32.802324+0000","last_fullsized":"2024-09-14T22:37:32.802324+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:40:54.611834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054571999999999997,"stat_sum":{"num_bytes":9371341,"num_objects":24,"num_object_clones":7,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":8,"num_read":2763,"num_read_kb":3328,"num_write":248,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"83'788","reported_seq":6871,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.880568+0000","last_change":"2024-09-14T22:36:17.711384+0000","last_active":"2024-09-14T22:37:32.880568+0000","last_peered":"2024-09-14T22:37:32.880568+0000","last_clean":"2024-09-14T22:37:32.880568+0000","last_became_active":"2024-09-14T22:36:15.723075+0000","last_became_peered":"2024-09-14T22:36:15.723075+0000","last_unstale":"2024-09-14T22:37:32.880568+0000","last_undegraded":"2024-09-14T22:37:32.880568+0000","last_fullsized":"2024-09-14T22:37:32.880568+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":788,"log_dups_size":0,"ondisk_log_size":788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:03:40.527644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010804009999999999,"stat_sum":{"num_bytes":37175321,"num_objects":47,"num_object_clones":13,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":14,"num_read":8660,"num_read_kb":21474,"num_write":1136,"num_write_kb":9408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.807895+0000","last_change":"2024-09-14T22:36:08.042860+0000","last_active":"2024-09-14T22:37:32.807895+0000","last_peered":"2024-09-14T22:37:32.807895+0000","last_clean":"2024-09-14T22:37:32.807895+0000","last_became_active":"2024-09-14T22:36:06.040961+0000","last_became_peered":"2024-09-14T22:36:06.040961+0000","last_unstale":"2024-09-14T22:37:32.807895+0000","last_undegraded":"2024-09-14T22:37:32.807895+0000","last_fullsized":"2024-09-14T22:37:32.807895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:03:48.547834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010780950000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"82'312","reported_seq":1640,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.809269+0000","last_change":"2024-09-14T22:36:17.720897+0000","last_active":"2024-09-14T22:37:32.809269+0000","last_peered":"2024-09-14T22:37:32.809269+0000","last_clean":"2024-09-14T22:37:32.809269+0000","last_became_active":"2024-09-14T22:36:15.700982+0000","last_became_peered":"2024-09-14T22:36:15.700982+0000","last_unstale":"2024-09-14T22:37:32.809269+0000","last_undegraded":"2024-09-14T22:37:32.809269+0000","last_fullsized":"2024-09-14T22:37:32.809269+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:08:15.177998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010495459999999999,"stat_sum":{"num_bytes":20611036,"num_objects":40,"num_object_clones":12,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":13,"num_read":2369,"num_read_kb":8666,"num_write":320,"num_write_kb":7994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.807983+0000","last_change":"2024-09-14T22:36:41.930087+0000","last_active":"2024-09-14T22:37:32.807983+0000","last_peered":"2024-09-14T22:37:32.807983+0000","last_clean":"2024-09-14T22:37:32.807983+0000","last_became_active":"2024-09-14T22:36:41.929522+0000","last_became_peered":"2024-09-14T22:36:41.929522+0000","last_unstale":"2024-09-14T22:37:32.807983+0000","last_undegraded":"2024-09-14T22:37:32.807983+0000","last_fullsized":"2024-09-14T22:37:32.807983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:15:10.589042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"83'332","reported_seq":4470,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.822850+0000","last_change":"2024-09-14T22:36:21.722517+0000","last_active":"2024-09-14T22:37:32.822850+0000","last_peered":"2024-09-14T22:37:32.822850+0000","last_clean":"2024-09-14T22:37:32.822850+0000","last_became_active":"2024-09-14T22:36:21.722085+0000","last_became_peered":"2024-09-14T22:36:21.722085+0000","last_unstale":"2024-09-14T22:37:32.822850+0000","last_undegraded":"2024-09-14T22:37:32.822850+0000","last_fullsized":"2024-09-14T22:37:32.822850+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:53:32.754230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020774109999999999,"stat_sum":{"num_bytes":47655829,"num_objects":32,"num_object_clones":6,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":10,"num_read":5103,"num_read_kb":23318,"num_write":396,"num_write_kb":6442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.802633+0000","last_change":"2024-09-14T22:36:08.054606+0000","last_active":"2024-09-14T22:37:32.802633+0000","last_peered":"2024-09-14T22:37:32.802633+0000","last_clean":"2024-09-14T22:37:32.802633+0000","last_became_active":"2024-09-14T22:36:06.266908+0000","last_became_peered":"2024-09-14T22:36:06.266908+0000","last_unstale":"2024-09-14T22:37:32.802633+0000","last_undegraded":"2024-09-14T22:37:32.802633+0000","last_fullsized":"2024-09-14T22:37:32.802633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:36.738643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098249300000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"83'1119","reported_seq":3766,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T22:37:34.800664+0000","last_change":"2024-09-14T22:37:17.676125+0000","last_active":"2024-09-14T22:37:34.800664+0000","last_peered":"2024-09-14T22:37:34.800664+0000","last_clean":"2024-09-14T22:37:34.800664+0000","last_became_active":"2024-09-14T22:37:17.674322+0000","last_became_peered":"2024-09-14T22:37:17.674322+0000","last_unstale":"2024-09-14T22:37:34.800664+0000","last_undegraded":"2024-09-14T22:37:34.800664+0000","last_fullsized":"2024-09-14T22:37:34.800664+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1119,"log_dups_size":0,"ondisk_log_size":1119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:54:44.017669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012564830000000001,"stat_sum":{"num_bytes":16092273,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":15,"num_read":7482,"num_read_kb":14334,"num_write":1975,"num_write_kb":9318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":4264139,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.808667+0000","last_change":"2024-09-14T22:36:41.930961+0000","last_active":"2024-09-14T22:37:32.808667+0000","last_peered":"2024-09-14T22:37:32.808667+0000","last_clean":"2024-09-14T22:37:32.808667+0000","last_became_active":"2024-09-14T22:36:41.930619+0000","last_became_peered":"2024-09-14T22:36:41.930619+0000","last_unstale":"2024-09-14T22:37:32.808667+0000","last_undegraded":"2024-09-14T22:37:32.808667+0000","last_fullsized":"2024-09-14T22:37:32.808667+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:04:40.582999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"83'380","reported_seq":3044,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.810299+0000","last_change":"2024-09-14T22:36:17.702597+0000","last_active":"2024-09-14T22:37:32.810299+0000","last_peered":"2024-09-14T22:37:32.810299+0000","last_clean":"2024-09-14T22:37:32.810299+0000","last_became_active":"2024-09-14T22:36:15.717186+0000","last_became_peered":"2024-09-14T22:36:15.717186+0000","last_unstale":"2024-09-14T22:37:32.810299+0000","last_undegraded":"2024-09-14T22:37:32.810299+0000","last_fullsized":"2024-09-14T22:37:32.810299+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:47:54.484398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071220099999999998,"stat_sum":{"num_bytes":24964501,"num_objects":34,"num_object_clones":10,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":13,"num_read":4078,"num_read_kb":5907,"num_write":358,"num_write_kb":2187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"79'255","reported_seq":3686,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.784466+0000","last_change":"2024-09-14T22:36:17.711068+0000","last_active":"2024-09-14T22:37:32.784466+0000","last_peered":"2024-09-14T22:37:32.784466+0000","last_clean":"2024-09-14T22:37:32.784466+0000","last_became_active":"2024-09-14T22:36:15.706147+0000","last_became_peered":"2024-09-14T22:36:15.706147+0000","last_unstale":"2024-09-14T22:37:32.784466+0000","last_undegraded":"2024-09-14T22:37:32.784466+0000","last_fullsized":"2024-09-14T22:37:32.784466+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:58:33.555717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082625500000000002,"stat_sum":{"num_bytes":37489798,"num_objects":28,"num_object_clones":9,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":9,"num_read":3360,"num_read_kb":16744,"num_write":244,"num_write_kb":5687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.809222+0000","last_change":"2024-09-14T22:36:08.055938+0000","last_active":"2024-09-14T22:37:32.809222+0000","last_peered":"2024-09-14T22:37:32.809222+0000","last_clean":"2024-09-14T22:37:32.809222+0000","last_became_active":"2024-09-14T22:36:06.039600+0000","last_became_peered":"2024-09-14T22:36:06.039600+0000","last_unstale":"2024-09-14T22:37:32.809222+0000","last_undegraded":"2024-09-14T22:37:32.809222+0000","last_fullsized":"2024-09-14T22:37:32.809222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:49:09.245563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021617870000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"78'739","reported_seq":8115,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T22:37:34.800529+0000","last_change":"2024-09-14T22:37:16.669227+0000","last_active":"2024-09-14T22:37:34.800529+0000","last_peered":"2024-09-14T22:37:34.800529+0000","last_clean":"2024-09-14T22:37:34.800529+0000","last_became_active":"2024-09-14T22:37:16.662230+0000","last_became_peered":"2024-09-14T22:37:16.662230+0000","last_unstale":"2024-09-14T22:37:34.800529+0000","last_undegraded":"2024-09-14T22:37:34.800529+0000","last_fullsized":"2024-09-14T22:37:34.800529+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:18:22.556190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021500769999999998,"stat_sum":{"num_bytes":64266431,"num_objects":44,"num_object_clones":11,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":13,"num_read":9395,"num_read_kb":19599,"num_write":1052,"num_write_kb":9919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":4851034,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":72,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T22:37:34.800562+0000","last_change":"2024-09-14T22:37:13.762987+0000","last_active":"2024-09-14T22:37:34.800562+0000","last_peered":"2024-09-14T22:37:34.800562+0000","last_clean":"2024-09-14T22:37:34.800562+0000","last_became_active":"2024-09-14T22:37:13.762641+0000","last_became_peered":"2024-09-14T22:37:13.762641+0000","last_unstale":"2024-09-14T22:37:34.800562+0000","last_undegraded":"2024-09-14T22:37:34.800562+0000","last_fullsized":"2024-09-14T22:37:34.800562+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:30:59.887867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"83'271","reported_seq":1836,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.808493+0000","last_change":"2024-09-14T22:36:17.720438+0000","last_active":"2024-09-14T22:37:32.808493+0000","last_peered":"2024-09-14T22:37:32.808493+0000","last_clean":"2024-09-14T22:37:32.808493+0000","last_became_active":"2024-09-14T22:36:15.711105+0000","last_became_peered":"2024-09-14T22:36:15.711105+0000","last_unstale":"2024-09-14T22:37:32.808493+0000","last_undegraded":"2024-09-14T22:37:32.808493+0000","last_fullsized":"2024-09-14T22:37:32.808493+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:32:17.419345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085549400000000005,"stat_sum":{"num_bytes":43842060,"num_objects":25,"num_object_clones":8,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":9,"num_read":1628,"num_read_kb":8229,"num_write":284,"num_write_kb":2028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.802169+0000","last_change":"2024-09-14T22:36:08.054759+0000","last_active":"2024-09-14T22:37:32.802169+0000","last_peered":"2024-09-14T22:37:32.802169+0000","last_clean":"2024-09-14T22:37:32.802169+0000","last_became_active":"2024-09-14T22:36:06.039969+0000","last_became_peered":"2024-09-14T22:36:06.039969+0000","last_unstale":"2024-09-14T22:37:32.802169+0000","last_undegraded":"2024-09-14T22:37:32.802169+0000","last_fullsized":"2024-09-14T22:37:32.802169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:03:45.369532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010817520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"83'892","reported_seq":3331,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.807851+0000","last_change":"2024-09-14T22:36:51.718064+0000","last_active":"2024-09-14T22:37:32.807851+0000","last_peered":"2024-09-14T22:37:32.807851+0000","last_clean":"2024-09-14T22:37:32.807851+0000","last_became_active":"2024-09-14T22:36:51.715823+0000","last_became_peered":"2024-09-14T22:36:51.715823+0000","last_unstale":"2024-09-14T22:37:32.807851+0000","last_undegraded":"2024-09-14T22:37:32.807851+0000","last_fullsized":"2024-09-14T22:37:32.807851+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":892,"log_dups_size":0,"ondisk_log_size":892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:21:40.878401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44497460,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":10,"num_read":6633,"num_read_kb":10451,"num_write":1375,"num_write_kb":6132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":173209,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.809159+0000","last_change":"2024-09-14T22:37:13.625072+0000","last_active":"2024-09-14T22:37:32.809159+0000","last_peered":"2024-09-14T22:37:32.809159+0000","last_clean":"2024-09-14T22:37:32.809159+0000","last_became_active":"2024-09-14T22:37:13.624669+0000","last_became_peered":"2024-09-14T22:37:13.624669+0000","last_unstale":"2024-09-14T22:37:32.809159+0000","last_undegraded":"2024-09-14T22:37:32.809159+0000","last_fullsized":"2024-09-14T22:37:32.809159+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:54:20.545098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011401880000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":292,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.802497+0000","last_change":"2024-09-14T22:36:00.402862+0000","last_active":"2024-09-14T22:37:32.802497+0000","last_peered":"2024-09-14T22:37:32.802497+0000","last_clean":"2024-09-14T22:37:32.802497+0000","last_became_active":"2024-09-14T22:36:00.400529+0000","last_became_peered":"2024-09-14T22:36:00.400529+0000","last_unstale":"2024-09-14T22:37:32.802497+0000","last_undegraded":"2024-09-14T22:37:32.802497+0000","last_fullsized":"2024-09-14T22:37:32.802497+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_clean_scrub_stamp":"2024-09-14T22:35:58.989025+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:07:12.951728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"82'374","reported_seq":5190,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.802166+0000","last_change":"2024-09-14T22:37:25.713957+0000","last_active":"2024-09-14T22:37:32.802166+0000","last_peered":"2024-09-14T22:37:32.802166+0000","last_clean":"2024-09-14T22:37:32.802166+0000","last_became_active":"2024-09-14T22:37:25.712476+0000","last_became_peered":"2024-09-14T22:37:25.712476+0000","last_unstale":"2024-09-14T22:37:32.802166+0000","last_undegraded":"2024-09-14T22:37:32.802166+0000","last_fullsized":"2024-09-14T22:37:32.802166+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:11:39.920434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001381511,"stat_sum":{"num_bytes":31660772,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":9,"num_read":5607,"num_read_kb":9967,"num_write":544,"num_write_kb":12023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":2127217,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"80'244","reported_seq":5667,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.784264+0000","last_change":"2024-09-14T22:37:16.658991+0000","last_active":"2024-09-14T22:37:32.784264+0000","last_peered":"2024-09-14T22:37:32.784264+0000","last_clean":"2024-09-14T22:37:32.784264+0000","last_became_active":"2024-09-14T22:37:16.657865+0000","last_became_peered":"2024-09-14T22:37:16.657865+0000","last_unstale":"2024-09-14T22:37:32.784264+0000","last_undegraded":"2024-09-14T22:37:32.784264+0000","last_fullsized":"2024-09-14T22:37:32.784264+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:12:16.803063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060272699999999995,"stat_sum":{"num_bytes":49352262,"num_objects":43,"num_object_clones":14,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":12,"num_read":5875,"num_read_kb":12262,"num_write":236,"num_write_kb":5250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":4204723,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"83'428","reported_seq":2861,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.894440+0000","last_change":"2024-09-14T22:36:17.776247+0000","last_active":"2024-09-14T22:37:32.894440+0000","last_peered":"2024-09-14T22:37:32.894440+0000","last_clean":"2024-09-14T22:37:32.894440+0000","last_became_active":"2024-09-14T22:36:15.718794+0000","last_became_peered":"2024-09-14T22:36:15.718794+0000","last_unstale":"2024-09-14T22:37:32.894440+0000","last_undegraded":"2024-09-14T22:37:32.894440+0000","last_fullsized":"2024-09-14T22:37:32.894440+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:48.357497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088662699999999997,"stat_sum":{"num_bytes":19371023,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":8,"num_read":4234,"num_read_kb":18414,"num_write":417,"num_write_kb":13069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"82'301","reported_seq":1686,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.808739+0000","last_change":"2024-09-14T22:36:17.733002+0000","last_active":"2024-09-14T22:37:32.808739+0000","last_peered":"2024-09-14T22:37:32.808739+0000","last_clean":"2024-09-14T22:37:32.808739+0000","last_became_active":"2024-09-14T22:36:15.702885+0000","last_became_peered":"2024-09-14T22:36:15.702885+0000","last_unstale":"2024-09-14T22:37:32.808739+0000","last_undegraded":"2024-09-14T22:37:32.808739+0000","last_fullsized":"2024-09-14T22:37:32.808739+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:53:21.399927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015180580000000001,"stat_sum":{"num_bytes":34136085,"num_objects":35,"num_object_clones":11,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":12,"num_read":2365,"num_read_kb":8357,"num_write":280,"num_write_kb":6300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"83'274","reported_seq":5326,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.813065+0000","last_change":"2024-09-14T22:36:17.702738+0000","last_active":"2024-09-14T22:37:32.813065+0000","last_peered":"2024-09-14T22:37:32.813065+0000","last_clean":"2024-09-14T22:37:32.813065+0000","last_became_active":"2024-09-14T22:36:15.713452+0000","last_became_peered":"2024-09-14T22:36:15.713452+0000","last_unstale":"2024-09-14T22:37:32.813065+0000","last_undegraded":"2024-09-14T22:37:32.813065+0000","last_fullsized":"2024-09-14T22:37:32.813065+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:40:44.768952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082782099999999996,"stat_sum":{"num_bytes":38038174,"num_objects":36,"num_object_clones":14,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":14,"num_read":5161,"num_read_kb":17038,"num_write":307,"num_write_kb":8786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"83'235","reported_seq":3021,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.802626+0000","last_change":"2024-09-14T22:36:18.009776+0000","last_active":"2024-09-14T22:37:32.802626+0000","last_peered":"2024-09-14T22:37:32.802626+0000","last_clean":"2024-09-14T22:37:32.802626+0000","last_became_active":"2024-09-14T22:36:15.723205+0000","last_became_peered":"2024-09-14T22:36:15.723205+0000","last_unstale":"2024-09-14T22:37:32.802626+0000","last_undegraded":"2024-09-14T22:37:32.802626+0000","last_fullsized":"2024-09-14T22:37:32.802626+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:17:42.504389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087783999999999996,"stat_sum":{"num_bytes":25138871,"num_objects":39,"num_object_clones":10,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":13,"num_read":2724,"num_read_kb":13200,"num_write":248,"num_write_kb":4487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"83'302","reported_seq":1699,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.802774+0000","last_change":"2024-09-14T22:36:18.009666+0000","last_active":"2024-09-14T22:37:32.802774+0000","last_peered":"2024-09-14T22:37:32.802774+0000","last_clean":"2024-09-14T22:37:32.802774+0000","last_became_active":"2024-09-14T22:36:15.715659+0000","last_became_peered":"2024-09-14T22:36:15.715659+0000","last_unstale":"2024-09-14T22:37:32.802774+0000","last_undegraded":"2024-09-14T22:37:32.802774+0000","last_fullsized":"2024-09-14T22:37:32.802774+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:34:50.780979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000776448,"stat_sum":{"num_bytes":31940998,"num_objects":35,"num_object_clones":10,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":10,"num_read":2029,"num_read_kb":7936,"num_write":366,"num_write_kb":3382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"82'228","reported_seq":7001,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.802887+0000","last_change":"2024-09-14T22:36:18.010182+0000","last_active":"2024-09-14T22:37:32.802887+0000","last_peered":"2024-09-14T22:37:32.802887+0000","last_clean":"2024-09-14T22:37:32.802887+0000","last_became_active":"2024-09-14T22:36:15.719693+0000","last_became_peered":"2024-09-14T22:36:15.719693+0000","last_unstale":"2024-09-14T22:37:32.802887+0000","last_undegraded":"2024-09-14T22:37:32.802887+0000","last_fullsized":"2024-09-14T22:37:32.802887+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:16:47.696472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001045011,"stat_sum":{"num_bytes":23908902,"num_objects":38,"num_object_clones":10,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":9,"num_read":6904,"num_read_kb":19887,"num_write":300,"num_write_kb":9182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"83'300","reported_seq":3125,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.808246+0000","last_change":"2024-09-14T22:37:26.718624+0000","last_active":"2024-09-14T22:37:32.808246+0000","last_peered":"2024-09-14T22:37:32.808246+0000","last_clean":"2024-09-14T22:37:32.808246+0000","last_became_active":"2024-09-14T22:37:26.717088+0000","last_became_peered":"2024-09-14T22:37:26.717088+0000","last_unstale":"2024-09-14T22:37:32.808246+0000","last_undegraded":"2024-09-14T22:37:32.808246+0000","last_fullsized":"2024-09-14T22:37:32.808246+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:59:19.875480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094227299999999998,"stat_sum":{"num_bytes":35099769,"num_objects":38,"num_object_clones":12,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":14,"num_read":2958,"num_read_kb":24411,"num_write":360,"num_write_kb":12096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"82'245","reported_seq":1705,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.802219+0000","last_change":"2024-09-14T22:36:18.009347+0000","last_active":"2024-09-14T22:37:32.802219+0000","last_peered":"2024-09-14T22:37:32.802219+0000","last_clean":"2024-09-14T22:37:32.802219+0000","last_became_active":"2024-09-14T22:36:15.715569+0000","last_became_peered":"2024-09-14T22:36:15.715569+0000","last_unstale":"2024-09-14T22:37:32.802219+0000","last_undegraded":"2024-09-14T22:37:32.802219+0000","last_fullsized":"2024-09-14T22:37:32.802219+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:44:30.757450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054921600000000005,"stat_sum":{"num_bytes":29773590,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":11,"num_read":1947,"num_read_kb":10189,"num_write":308,"num_write_kb":9963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"83'247","reported_seq":4705,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.809537+0000","last_change":"2024-09-14T22:36:17.720448+0000","last_active":"2024-09-14T22:37:32.809537+0000","last_peered":"2024-09-14T22:37:32.809537+0000","last_clean":"2024-09-14T22:37:32.809537+0000","last_became_active":"2024-09-14T22:36:15.716696+0000","last_became_peered":"2024-09-14T22:36:15.716696+0000","last_unstale":"2024-09-14T22:37:32.809537+0000","last_undegraded":"2024-09-14T22:37:32.809537+0000","last_fullsized":"2024-09-14T22:37:32.809537+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:30:53.040218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086547600000000001,"stat_sum":{"num_bytes":37561236,"num_objects":38,"num_object_clones":9,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":13,"num_read":5092,"num_read_kb":31570,"num_write":231,"num_write_kb":2117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"82'311","reported_seq":5732,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T22:37:32.802259+0000","last_change":"2024-09-14T22:36:18.010088+0000","last_active":"2024-09-14T22:37:32.802259+0000","last_peered":"2024-09-14T22:37:32.802259+0000","last_clean":"2024-09-14T22:37:32.802259+0000","last_became_active":"2024-09-14T22:36:15.715866+0000","last_became_peered":"2024-09-14T22:36:15.715866+0000","last_unstale":"2024-09-14T22:37:32.802259+0000","last_undegraded":"2024-09-14T22:37:32.802259+0000","last_fullsized":"2024-09-14T22:37:32.802259+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:29:23.921984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010286189999999999,"stat_sum":{"num_bytes":51923917,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":11,"num_read":5879,"num_read_kb":32277,"num_write":375,"num_write_kb":14482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1110429226,"num_objects":1155,"num_object_clones":320,"num_object_copies":2310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1155,"num_whiteouts":384,"num_read":148248,"num_read_kb":544853,"num_write":15989,"num_write_kb":252342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":536,"num_bytes_recovered":27389785,"num_keys_recovered":22,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36093952,"data_stored":351792656,"data_compressed":9938975,"data_compressed_allocated":27869184,"data_compressed_original":347014070,"omap_allocated":4238,"internal_metadata":0},"log_size":12464,"ondisk_log_size":12464,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542166,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50460,"kb_used_data":6796,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94321380,"statfs":{"total":96636764160,"available":96585093120,"internally_reserved":0,"allocated":6959104,"data_stored":54495657,"data_compressed":1865544,"data_compressed_allocated":4472832,"data_compressed_original":53151728,"omap_allocated":1590,"internal_metadata":44693962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542167,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38684,"kb_used_data":3328,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333156,"statfs":{"total":96636764160,"available":96597151744,"internally_reserved":0,"allocated":3407872,"data_stored":29255081,"data_compressed":650805,"data_compressed_allocated":2019328,"data_compressed_original":28582979,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542167,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45588,"kb_used_data":6076,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326252,"statfs":{"total":96636764160,"available":96590082048,"internally_reserved":0,"allocated":6221824,"data_stored":48271090,"data_compressed":1360690,"data_compressed_allocated":4005888,"data_compressed_original":47085710,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542166,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35748,"kb_used_data":8724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336092,"statfs":{"total":96636764160,"available":96600158208,"internally_reserved":0,"allocated":8933376,"data_stored":82247681,"data_compressed":2203556,"data_compressed_allocated":6393856,"data_compressed_original":80917069,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542167,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53268,"kb_used_data":4032,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318572,"statfs":{"total":96636764160,"available":96582217728,"internally_reserved":0,"allocated":4128768,"data_stored":27586773,"data_compressed":1007940,"data_compressed_allocated":2326528,"data_compressed_original":26679302,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":66,"seq":283467841542,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35872,"kb_used_data":3200,"kb_used_omap":6,"kb_used_meta":32633,"kb_avail":94335968,"statfs":{"total":96636764160,"available":96600031232,"internally_reserved":0,"allocated":3276800,"data_stored":24944558,"data_compressed":455862,"data_compressed_allocated":1794048,"data_compressed_original":24168618,"omap_allocated":7025,"internal_metadata":33416335},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53560,"kb_used_data":4328,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318280,"statfs":{"total":96636764160,"available":96581918720,"internally_reserved":0,"allocated":4431872,"data_stored":32764797,"data_compressed":976741,"data_compressed_allocated":2584576,"data_compressed_original":31815514,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542166,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33192,"kb_used_data":6156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338648,"statfs":{"total":96636764160,"available":96602775552,"internally_reserved":0,"allocated":6303744,"data_stored":56427427,"data_compressed":1321620,"data_compressed_allocated":4206592,"data_compressed_original":55346334,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5357568,"data_stored":55564772,"data_compressed":1303342,"data_compressed_allocated":4169728,"data_compressed_original":54887582,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3391488,"data_stored":32217687,"data_compressed":976670,"data_compressed_allocated":2576384,"data_compressed_original":31737690,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2277376,"data_stored":24487279,"data_compressed":455862,"data_compressed_allocated":1794048,"data_compressed_original":24168618,"omap_allocated":4238,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3231744,"data_stored":27266627,"data_compressed":1033526,"data_compressed_allocated":2359296,"data_compressed_original":26781702,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7995392,"data_stored":81821039,"data_compressed":2203556,"data_compressed_allocated":6393856,"data_compressed_original":80917069,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5275648,"data_stored":47882392,"data_compressed":1360703,"data_compressed_allocated":4005888,"data_compressed_original":47126670,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2469888,"data_stored":28900253,"data_compressed":650990,"data_compressed_allocated":2027520,"data_compressed_original":28656707,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":6094848,"data_stored":53652607,"data_compressed":1954326,"data_compressed_allocated":4542464,"data_compressed_original":52738032,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:37:38.621 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-14T22:37:38.621 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:37:38.622 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T22:37:38.622 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T22:37:38.717 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:38.715+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:38.786 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:38.787 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:38.787 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:37:38.787 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:38.801 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:37:38.818 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:38.815+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:38.825 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2024-09-14T22:37:38.919 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:38.915+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:38.928 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:38.928 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:38.928 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:37:38.928 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:37:38.929 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:38.943 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:37:38.954 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:37:38.976 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:37:38.976 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:38.976 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 89, 2024-09-14T22:37:38.976 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "f07f407a-b17d-42c4-bf73-92dd80114a4f", 2024-09-14T22:37:38.976 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-09-14T22:35:45.983055+0000", 2024-09-14T22:37:38.976 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-09-14T22:37:38.819969+0000", 2024-09-14T22:37:38.976 INFO:teuthology.orchestra.run.smithi001.stdout: "last_up_change": "2024-09-14T22:37:12.616042+0000", 2024-09-14T22:37:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: "last_in_change": "2024-09-14T22:35:48.283160+0000", 2024-09-14T22:37:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T22:37:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_num": 5801984, 2024-09-14T22:37:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_set": [ 2024-09-14T22:37:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: "noscrub", 2024-09-14T22:37:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: "pglog_hardlimit", 2024-09-14T22:37:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: "purged_snapdirs", 2024-09-14T22:37:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: "recovery_deletes", 2024-09-14T22:37:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: "sortbitwise" 2024-09-14T22:37:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:37:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_version": 6, 2024-09-14T22:37:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T22:37:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T22:37:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T22:37:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_snapshot": "", 2024-09-14T22:37:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_max": 3, 2024-09-14T22:37:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd": 8, 2024-09-14T22:37:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: "require_min_compat_client": "luminous", 2024-09-14T22:37:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: "min_compat_client": "luminous", 2024-09-14T22:37:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: "require_osd_release": "squid", 2024-09-14T22:37:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_crimson": false, 2024-09-14T22:37:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: "pools": [ 2024-09-14T22:37:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 1, 2024-09-14T22:37:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": ".mgr", 2024-09-14T22:37:38.979 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-14T22:35:58.938784+0000", 2024-09-14T22:37:38.979 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-14T22:37:38.979 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool", 2024-09-14T22:37:38.979 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-14T22:37:38.979 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-14T22:37:38.979 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-14T22:37:38.980 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-14T22:37:38.980 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:37:38.980 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:37:38.980 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:37:38.980 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:37:38.980 INFO:teuthology.orchestra.run.smithi001.stdout: "is_stretch_pool": false, 2024-09-14T22:37:38.980 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-14T22:37:38.980 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:37:38.980 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 1, 2024-09-14T22:37:38.980 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 1, 2024-09-14T22:37:38.980 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 1, 2024-09-14T22:37:38.981 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 1, 2024-09-14T22:37:38.981 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 1, 2024-09-14T22:37:38.981 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-14T22:37:38.981 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-14T22:37:38.981 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-14T22:37:38.981 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-14T22:37:38.981 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-14T22:37:38.981 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-14T22:37:38.981 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-14T22:37:38.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:38.981 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "18", 2024-09-14T22:37:38.981 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-14T22:37:38.982 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:37:38.982 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:37:38.982 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-14T22:37:38.982 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:37:38.982 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 0, 2024-09-14T22:37:38.982 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 0, 2024-09-14T22:37:38.982 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-14T22:37:38.982 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-14T22:37:38.982 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-14T22:37:38.982 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-14T22:37:38.982 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-14T22:37:38.983 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-14T22:37:38.983 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-14T22:37:38.983 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-14T22:37:38.983 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-14T22:37:38.983 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-14T22:37:38.983 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-14T22:37:38.983 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:37:38.983 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:37:38.983 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:37:38.983 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-14T22:37:38.983 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-14T22:37:38.983 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-14T22:37:38.984 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-14T22:37:38.984 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-14T22:37:38.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:38.984 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-14T22:37:38.984 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-14T22:37:38.984 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-14T22:37:38.984 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:37:38.984 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:37:38.984 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:37:38.984 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:37:38.984 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-14T22:37:38.984 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-14T22:37:38.985 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-14T22:37:38.985 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-14T22:37:38.985 INFO:teuthology.orchestra.run.smithi001.stdout: "options": { 2024-09-14T22:37:38.985 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_max": 32, 2024-09-14T22:37:38.985 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_min": 1 2024-09-14T22:37:38.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:38.985 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-14T22:37:38.985 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": {} 2024-09-14T22:37:38.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:38.985 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-14T22:37:38.985 INFO:teuthology.orchestra.run.smithi001.stdout: "score_type": "Fair distribution", 2024-09-14T22:37:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 8, 2024-09-14T22:37:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 8, 2024-09-14T22:37:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 0.25, 2024-09-14T22:37:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 2, 2024-09-14T22:37:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 2, 2024-09-14T22:37:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 0.87667274475097656, 2024-09-14T22:37:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T22:37:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T22:37:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 2, 2024-09-14T22:37:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "rbd", 2024-09-14T22:37:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-14T22:36:04.304242+0000", 2024-09-14T22:37:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 8193, 2024-09-14T22:37:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T22:37:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-14T22:37:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-14T22:37:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-14T22:37:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-14T22:37:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:37:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:37:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:37:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:37:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: "is_stretch_pool": false, 2024-09-14T22:37:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-14T22:37:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:37:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 8, 2024-09-14T22:37:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 8, 2024-09-14T22:37:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 8, 2024-09-14T22:37:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 8, 2024-09-14T22:37:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 8, 2024-09-14T22:37:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-14T22:37:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-14T22:37:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-14T22:37:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-14T22:37:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-14T22:37:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-14T22:37:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-14T22:37:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "22", 2024-09-14T22:37:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-14T22:37:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:37:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:37:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-14T22:37:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:37:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 2, 2024-09-14T22:37:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 22, 2024-09-14T22:37:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-14T22:37:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-14T22:37:38.990 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-14T22:37:38.990 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-14T22:37:38.990 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-14T22:37:38.990 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-14T22:37:38.990 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-14T22:37:38.990 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-14T22:37:38.990 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-14T22:37:38.990 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-14T22:37:38.990 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-14T22:37:38.991 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:37:38.991 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:37:38.991 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:37:38.991 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-14T22:37:38.991 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-14T22:37:38.991 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-14T22:37:38.991 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-14T22:37:38.991 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-14T22:37:38.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:38.991 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-14T22:37:38.991 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-14T22:37:38.991 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-14T22:37:38.991 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:37:38.992 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:37:38.992 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:37:38.992 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:37:38.992 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-14T22:37:38.992 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-14T22:37:38.992 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-14T22:37:38.992 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-14T22:37:38.992 INFO:teuthology.orchestra.run.smithi001.stdout: "options": {}, 2024-09-14T22:37:38.992 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-14T22:37:38.992 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd": {} 2024-09-14T22:37:38.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-14T22:37:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: "score_type": "Fair distribution", 2024-09-14T22:37:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 1.9900000095367432, 2024-09-14T22:37:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 1.9900000095367432, 2024-09-14T22:37:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 0.87999999523162842, 2024-09-14T22:37:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 1.75, 2024-09-14T22:37:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 1.75, 2024-09-14T22:37:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 0.87667274475097656, 2024-09-14T22:37:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T22:37:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T22:37:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:38.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:38.994 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 3, 2024-09-14T22:37:38.994 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "pool_client.0", 2024-09-14T22:37:38.994 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-14T22:36:14.374249+0000", 2024-09-14T22:37:38.994 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 8193, 2024-09-14T22:37:38.994 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T22:37:38.994 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-14T22:37:38.994 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-14T22:37:38.994 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-14T22:37:38.994 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-14T22:37:38.994 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:37:38.994 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:37:38.994 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:37:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:37:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: "is_stretch_pool": false, 2024-09-14T22:37:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-14T22:37:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:37:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 32, 2024-09-14T22:37:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 32, 2024-09-14T22:37:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 32, 2024-09-14T22:37:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 32, 2024-09-14T22:37:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 32, 2024-09-14T22:37:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-14T22:37:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-14T22:37:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-14T22:37:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-14T22:37:38.996 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-14T22:37:38.996 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-14T22:37:38.996 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-14T22:37:38.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:38.996 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "89", 2024-09-14T22:37:38.996 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-14T22:37:38.996 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:37:38.996 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:37:38.996 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-14T22:37:38.996 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:37:38.997 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 29, 2024-09-14T22:37:38.997 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 89, 2024-09-14T22:37:38.997 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-14T22:37:38.997 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-14T22:37:38.997 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-14T22:37:38.997 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-14T22:37:38.997 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-14T22:37:38.997 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-14T22:37:38.997 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-14T22:37:38.997 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-14T22:37:38.997 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-14T22:37:38.997 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-14T22:37:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-14T22:37:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:37:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:37:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:37:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-14T22:37:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-14T22:37:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-14T22:37:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-14T22:37:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-14T22:37:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-14T22:37:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-14T22:37:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-14T22:37:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:37:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:37:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:37:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:37:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-14T22:37:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-14T22:37:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-14T22:37:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-14T22:37:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: "options": {}, 2024-09-14T22:37:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-14T22:37:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd": {} 2024-09-14T22:37:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.000 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-14T22:37:39.000 INFO:teuthology.orchestra.run.smithi001.stdout: "score_type": "Fair distribution", 2024-09-14T22:37:39.000 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 1.75, 2024-09-14T22:37:39.000 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 1.75, 2024-09-14T22:37:39.000 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 1, 2024-09-14T22:37:39.000 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 1.75, 2024-09-14T22:37:39.000 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 1.75, 2024-09-14T22:37:39.000 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 0.87667274475097656, 2024-09-14T22:37:39.000 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T22:37:39.000 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T22:37:39.001 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.001 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.001 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:37:39.001 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2024-09-14T22:37:39.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.001 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 0, 2024-09-14T22:37:39.001 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "9831fcc5-9040-4a80-99d4-838225e65139", 2024-09-14T22:37:39.001 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:37:39.001 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:37:39.001 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:37:39.002 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:37:39.002 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:37:39.002 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:37:39.002 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:37:39.002 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 76, 2024-09-14T22:37:39.002 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:37:39.002 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:37:39.002 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:37:39.003 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:39.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.003 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:39.003 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6824", 2024-09-14T22:37:39.003 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:37:39.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.004 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:39.004 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6825", 2024-09-14T22:37:39.004 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:37:39.004 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.004 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.004 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:37:39.004 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:39.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.005 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:39.005 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6826", 2024-09-14T22:37:39.005 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:37:39.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.005 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:39.005 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6827", 2024-09-14T22:37:39.006 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:37:39.006 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.006 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.006 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:37:39.006 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:39.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.006 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:39.006 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6830", 2024-09-14T22:37:39.006 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:37:39.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6831", 2024-09-14T22:37:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:37:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:37:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6828", 2024-09-14T22:37:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:37:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.008 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:39.008 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6829", 2024-09-14T22:37:39.008 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:37:39.008 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.008 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.008 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6825/1702929600", 2024-09-14T22:37:39.008 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6827/1702929600", 2024-09-14T22:37:39.008 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6831/1702929600", 2024-09-14T22:37:39.008 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6829/1702929600", 2024-09-14T22:37:39.008 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:37:39.009 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:37:39.009 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:37:39.009 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.009 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 1, 2024-09-14T22:37:39.009 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "7f79abe8-115d-46b2-8677-72fcb0c8ab7d", 2024-09-14T22:37:39.009 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:37:39.009 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:37:39.009 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:37:39.009 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 0.1606292724609375, 2024-09-14T22:37:39.010 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:37:39.010 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:37:39.010 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2024-09-14T22:37:39.010 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 86, 2024-09-14T22:37:39.010 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:37:39.010 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:37:39.010 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:37:39.010 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:39.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.010 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:39.010 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6808", 2024-09-14T22:37:39.010 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:37:39.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.011 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:39.011 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6809", 2024-09-14T22:37:39.011 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:37:39.011 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.011 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.011 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:37:39.011 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:39.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.012 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:39.012 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6810", 2024-09-14T22:37:39.012 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:37:39.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.012 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:39.012 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6811", 2024-09-14T22:37:39.012 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:37:39.012 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.012 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:37:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6814", 2024-09-14T22:37:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:37:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6815", 2024-09-14T22:37:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:37:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:37:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6812", 2024-09-14T22:37:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:37:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:39.015 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6813", 2024-09-14T22:37:39.015 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:37:39.015 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.015 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.015 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6809/719990653", 2024-09-14T22:37:39.015 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6811/719990653", 2024-09-14T22:37:39.015 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6815/719990653", 2024-09-14T22:37:39.015 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6813/719990653", 2024-09-14T22:37:39.015 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:37:39.015 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:37:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:37:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 2, 2024-09-14T22:37:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "1f5768b2-4db4-4b2e-8d5e-5206f8bde28d", 2024-09-14T22:37:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:37:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:37:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:37:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:37:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 14, 2024-09-14T22:37:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 30, 2024-09-14T22:37:39.017 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 66, 2024-09-14T22:37:39.017 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 70, 2024-09-14T22:37:39.017 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 31, 2024-09-14T22:37:39.017 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:37:39.017 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:37:39.017 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:39.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.017 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:39.017 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6816", 2024-09-14T22:37:39.017 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:37:39.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6817", 2024-09-14T22:37:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:37:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:37:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6818", 2024-09-14T22:37:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:37:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6819", 2024-09-14T22:37:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:37:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:37:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6822", 2024-09-14T22:37:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:37:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6823", 2024-09-14T22:37:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:37:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:37:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6820", 2024-09-14T22:37:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:37:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6821", 2024-09-14T22:37:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:37:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6817/1006730601", 2024-09-14T22:37:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6819/1006730601", 2024-09-14T22:37:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6823/1006730601", 2024-09-14T22:37:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6821/1006730601", 2024-09-14T22:37:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:37:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:37:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:37:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 3, 2024-09-14T22:37:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "8103f3c7-7899-4fc3-bb02-dd2273120138", 2024-09-14T22:37:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:37:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:37:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:37:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 0.852752685546875, 2024-09-14T22:37:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:37:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:37:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:37:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 87, 2024-09-14T22:37:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:37:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:37:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:37:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6800", 2024-09-14T22:37:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:37:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6801", 2024-09-14T22:37:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:37:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.025 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:37:39.025 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:39.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.025 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:39.025 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6802", 2024-09-14T22:37:39.025 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:37:39.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.025 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:39.025 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6803", 2024-09-14T22:37:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:37:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:37:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6806", 2024-09-14T22:37:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:37:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:39.027 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6807", 2024-09-14T22:37:39.027 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:37:39.027 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.027 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.027 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:37:39.027 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:39.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.027 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6804", 2024-09-14T22:37:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:37:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6805", 2024-09-14T22:37:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:37:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6801/750403671", 2024-09-14T22:37:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6803/750403671", 2024-09-14T22:37:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6807/750403671", 2024-09-14T22:37:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6805/750403671", 2024-09-14T22:37:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:37:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:37:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:37:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 4, 2024-09-14T22:37:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "23cce5cb-726e-44c2-8edb-c2567a89dfbc", 2024-09-14T22:37:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:37:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:37:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:37:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:37:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:37:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:37:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:37:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 75, 2024-09-14T22:37:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:37:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:37:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:37:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6816", 2024-09-14T22:37:39.031 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:37:39.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.031 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:39.031 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6817", 2024-09-14T22:37:39.031 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:37:39.031 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.031 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.031 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:37:39.031 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:39.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.032 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:39.032 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6818", 2024-09-14T22:37:39.032 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:37:39.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.032 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:39.032 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6819", 2024-09-14T22:37:39.032 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:37:39.032 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.032 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.032 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:37:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6822", 2024-09-14T22:37:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:37:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6823", 2024-09-14T22:37:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:37:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:37:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6820", 2024-09-14T22:37:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:37:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6821", 2024-09-14T22:37:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:37:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.035 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.176:6817/1408471829", 2024-09-14T22:37:39.035 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.176:6819/1408471829", 2024-09-14T22:37:39.035 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.176:6823/1408471829", 2024-09-14T22:37:39.035 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.176:6821/1408471829", 2024-09-14T22:37:39.035 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:37:39.035 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:37:39.035 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:37:39.035 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.035 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 5, 2024-09-14T22:37:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "eb2f187b-d2f1-49ab-ad5e-ea6dc441a28e", 2024-09-14T22:37:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:37:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:37:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:37:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:37:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:37:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:37:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:37:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 75, 2024-09-14T22:37:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:37:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:37:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:37:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6805", 2024-09-14T22:37:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:37:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6808", 2024-09-14T22:37:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:37:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:37:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6810", 2024-09-14T22:37:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:37:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6811", 2024-09-14T22:37:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:37:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:37:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6814", 2024-09-14T22:37:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:37:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6815", 2024-09-14T22:37:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:37:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:37:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6812", 2024-09-14T22:37:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:37:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.041 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:39.041 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6813", 2024-09-14T22:37:39.041 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:37:39.041 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.041 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.041 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.176:6808/1945902326", 2024-09-14T22:37:39.041 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.176:6811/1945902326", 2024-09-14T22:37:39.041 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.176:6815/1945902326", 2024-09-14T22:37:39.041 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.176:6813/1945902326", 2024-09-14T22:37:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:37:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:37:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:37:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 6, 2024-09-14T22:37:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "d371aed8-2ad2-49a5-8ece-4b7fd37b4822", 2024-09-14T22:37:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:37:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:37:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:37:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:37:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:37:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:37:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:37:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 77, 2024-09-14T22:37:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:37:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:37:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:37:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6800", 2024-09-14T22:37:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:37:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6801", 2024-09-14T22:37:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:37:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:37:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6802", 2024-09-14T22:37:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:37:39.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.045 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:39.045 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6803", 2024-09-14T22:37:39.045 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:37:39.045 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.045 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.045 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:37:39.045 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:39.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.046 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:39.046 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6807", 2024-09-14T22:37:39.046 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:37:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6809", 2024-09-14T22:37:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:37:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:37:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6804", 2024-09-14T22:37:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:37:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6806", 2024-09-14T22:37:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:37:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.049 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.176:6801/4116085586", 2024-09-14T22:37:39.049 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.176:6803/4116085586", 2024-09-14T22:37:39.049 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.176:6809/4116085586", 2024-09-14T22:37:39.049 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.176:6806/4116085586", 2024-09-14T22:37:39.049 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:37:39.049 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:37:39.049 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:37:39.049 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.049 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 7, 2024-09-14T22:37:39.049 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "a012c819-8d49-4310-a840-c2752d9314ab", 2024-09-14T22:37:39.050 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:37:39.050 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:37:39.050 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:37:39.050 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:37:39.050 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:37:39.050 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:37:39.050 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:37:39.050 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 76, 2024-09-14T22:37:39.050 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:37:39.050 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:37:39.050 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:37:39.050 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:39.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.051 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:39.051 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6824", 2024-09-14T22:37:39.051 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:37:39.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.051 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:39.051 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6825", 2024-09-14T22:37:39.051 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:37:39.051 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.051 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.052 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:37:39.052 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:39.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.052 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:39.052 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6826", 2024-09-14T22:37:39.052 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:37:39.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.052 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:39.052 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6827", 2024-09-14T22:37:39.052 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:37:39.052 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.053 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.053 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:37:39.053 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:39.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.053 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:39.053 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6830", 2024-09-14T22:37:39.053 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:37:39.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.053 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:39.053 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6831", 2024-09-14T22:37:39.054 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:37:39.054 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.054 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.054 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:37:39.054 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:39.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.054 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:39.054 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6828", 2024-09-14T22:37:39.054 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:37:39.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.055 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:39.055 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6829", 2024-09-14T22:37:39.055 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:37:39.055 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.055 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.055 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.176:6825/877760244", 2024-09-14T22:37:39.055 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.176:6827/877760244", 2024-09-14T22:37:39.055 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.176:6831/877760244", 2024-09-14T22:37:39.055 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.176:6829/877760244", 2024-09-14T22:37:39.055 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:37:39.055 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:37:39.056 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:37:39.056 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.056 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.056 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:37:39.056 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_xinfo": [ 2024-09-14T22:37:39.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.056 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 0, 2024-09-14T22:37:39.056 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:37:39.056 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:37:39.056 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:37:39.056 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:37:39.057 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:37:39.057 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:55.511688+0000", 2024-09-14T22:37:39.057 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:37:39.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.057 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 1, 2024-09-14T22:37:39.057 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:37:39.057 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:37:39.057 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:37:39.057 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:37:39.057 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:37:39.057 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.742414+0000", 2024-09-14T22:37:39.058 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:37:39.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.058 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 2, 2024-09-14T22:37:39.058 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:37:39.058 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:37:39.058 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:37:39.058 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:37:39.058 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:37:39.058 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.871355+0000", 2024-09-14T22:37:39.058 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 31 2024-09-14T22:37:39.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.059 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 3, 2024-09-14T22:37:39.059 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:37:39.059 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:37:39.059 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:37:39.059 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:37:39.059 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:37:39.059 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.504553+0000", 2024-09-14T22:37:39.059 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:37:39.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.059 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 4, 2024-09-14T22:37:39.060 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:37:39.060 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:37:39.060 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:37:39.060 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:37:39.060 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:37:39.060 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.589640+0000", 2024-09-14T22:37:39.060 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:37:39.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.060 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 5, 2024-09-14T22:37:39.060 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:37:39.060 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:37:39.061 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:37:39.061 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:37:39.061 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:37:39.061 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.518845+0000", 2024-09-14T22:37:39.061 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:37:39.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.061 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.061 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 6, 2024-09-14T22:37:39.061 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:37:39.061 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:37:39.061 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:37:39.061 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:37:39.062 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:37:39.062 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.731632+0000", 2024-09-14T22:37:39.062 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:37:39.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.062 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 7, 2024-09-14T22:37:39.062 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:37:39.062 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:37:39.062 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:37:39.062 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:37:39.062 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:37:39.062 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.952634+0000", 2024-09-14T22:37:39.063 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:37:39.063 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.063 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:37:39.063 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap": [], 2024-09-14T22:37:39.063 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap_items": [ 2024-09-14T22:37:39.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.063 INFO:teuthology.orchestra.run.smithi001.stdout: "pgid": "3.2", 2024-09-14T22:37:39.063 INFO:teuthology.orchestra.run.smithi001.stdout: "mappings": [ 2024-09-14T22:37:39.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:39.063 INFO:teuthology.orchestra.run.smithi001.stdout: "from": 3, 2024-09-14T22:37:39.063 INFO:teuthology.orchestra.run.smithi001.stdout: "to": 6 2024-09-14T22:37:39.063 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.063 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:39.064 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.064 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:37:39.064 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap_primaries": [], 2024-09-14T22:37:39.064 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_temp": [], 2024-09-14T22:37:39.064 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_temp": [], 2024-09-14T22:37:39.064 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklist": {}, 2024-09-14T22:37:39.064 INFO:teuthology.orchestra.run.smithi001.stdout: "range_blocklist": {}, 2024-09-14T22:37:39.064 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profiles": { 2024-09-14T22:37:39.064 INFO:teuthology.orchestra.run.smithi001.stdout: "default": { 2024-09-14T22:37:39.064 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-failure-domain": "osd", 2024-09-14T22:37:39.065 INFO:teuthology.orchestra.run.smithi001.stdout: "k": "2", 2024-09-14T22:37:39.065 INFO:teuthology.orchestra.run.smithi001.stdout: "m": "1", 2024-09-14T22:37:39.065 INFO:teuthology.orchestra.run.smithi001.stdout: "plugin": "jerasure", 2024-09-14T22:37:39.065 INFO:teuthology.orchestra.run.smithi001.stdout: "technique": "reed_sol_van" 2024-09-14T22:37:39.065 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:39.065 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps_queue": [], 2024-09-14T22:37:39.065 INFO:teuthology.orchestra.run.smithi001.stdout: "new_removed_snaps": [], 2024-09-14T22:37:39.065 INFO:teuthology.orchestra.run.smithi001.stdout: "new_purged_snaps": [], 2024-09-14T22:37:39.065 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_node_flags": {}, 2024-09-14T22:37:39.066 INFO:teuthology.orchestra.run.smithi001.stdout: "device_class_flags": {}, 2024-09-14T22:37:39.066 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": { 2024-09-14T22:37:39.066 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode_enabled": false, 2024-09-14T22:37:39.066 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_bucket_count": 0, 2024-09-14T22:37:39.066 INFO:teuthology.orchestra.run.smithi001.stdout: "degraded_stretch_mode": 0, 2024-09-14T22:37:39.066 INFO:teuthology.orchestra.run.smithi001.stdout: "recovering_stretch_mode": 0, 2024-09-14T22:37:39.066 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode_bucket": 0 2024-09-14T22:37:39.066 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:39.066 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:39.068 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:39.068 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:39.068 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:39.068 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:39.068 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.068 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:72252 3.2 3:442c6d72:::rbd_data.113cb4c83968.0000000000000015:head [zero 4456448~393216] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-14T22:37:39.069 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:37:39.069 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.360610309000002, 2024-09-14T22:37:39.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0176591500000001, 2024-09-14T22:37:39.069 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:39.069 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:39.069 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T22:37:39.069 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:39.069 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:39.069 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:39.069 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 72252 2024-09-14T22:37:39.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.070 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:39.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:39.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:37:39.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:39.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:37:39.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.071 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:39.071 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700101+0000", 2024-09-14T22:37:39.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4570000000000001e-06 2024-09-14T22:37:39.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.071 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:39.071 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700114+0000", 2024-09-14T22:37:39.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2799999999999999e-05 2024-09-14T22:37:39.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.072 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.072 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:39.072 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700146+0000", 2024-09-14T22:37:39.072 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2116000000000002e-05 2024-09-14T22:37:39.072 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.072 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.072 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:39.072 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700170+0000", 2024-09-14T22:37:39.072 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3770000000000001e-05 2024-09-14T22:37:39.072 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.073 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:39.073 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700285+0000", 2024-09-14T22:37:39.073 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000115228 2024-09-14T22:37:39.073 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.073 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op must wait for map", 2024-09-14T22:37:39.073 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700297+0000", 2024-09-14T22:37:39.073 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1681999999999999e-05 2024-09-14T22:37:39.073 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.074 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:39.074 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.703312+0000", 2024-09-14T22:37:39.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003015046 2024-09-14T22:37:39.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.074 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:37:39.074 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.703327+0000", 2024-09-14T22:37:39.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5481999999999998e-05 2024-09-14T22:37:39.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.074 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:39.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:51.717566+0000", 2024-09-14T22:37:39.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0142383509999999 2024-09-14T22:37:39.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.075 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.075 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:39.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:51.717757+0000", 2024-09-14T22:37:39.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000191218 2024-09-14T22:37:39.075 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.075 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:39.075 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.075 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.076 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:74156 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T22:37:39.076 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:37:39.076 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.546141599000002, 2024-09-14T22:37:39.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062073521, 2024-09-14T22:37:39.076 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:39.076 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:39.076 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:39.076 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:39.076 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:39.076 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:39.076 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74156 2024-09-14T22:37:39.076 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.077 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:39.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.077 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:39.077 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:37:39.077 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.077 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.077 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:39.077 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:37:39.077 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.077 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.078 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.078 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:39.078 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514569+0000", 2024-09-14T22:37:39.078 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0229999999999999e-06 2024-09-14T22:37:39.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.078 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.078 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:39.078 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514596+0000", 2024-09-14T22:37:39.078 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6903e-05 2024-09-14T22:37:39.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.079 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.079 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:39.079 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514623+0000", 2024-09-14T22:37:39.079 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7206e-05 2024-09-14T22:37:39.079 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.079 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.079 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:39.079 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514637+0000", 2024-09-14T22:37:39.079 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:37:39.079 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.079 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.079 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:39.080 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514781+0000", 2024-09-14T22:37:39.080 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014394899999999999 2024-09-14T22:37:39.080 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.080 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.080 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:39.080 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.515005+0000", 2024-09-14T22:37:39.080 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022458499999999999 2024-09-14T22:37:39.080 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.080 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.080 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:37:39.081 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.527785+0000", 2024-09-14T22:37:39.081 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012780112 2024-09-14T22:37:39.081 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.081 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.081 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:39.081 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576209+0000", 2024-09-14T22:37:39.081 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.048423992999999999 2024-09-14T22:37:39.081 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.081 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.081 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:39.081 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576346+0000", 2024-09-14T22:37:39.082 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000136909 2024-09-14T22:37:39.082 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.082 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.082 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:39.082 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576517+0000", 2024-09-14T22:37:39.082 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000170848 2024-09-14T22:37:39.082 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.082 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.082 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:39.082 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576640+0000", 2024-09-14T22:37:39.082 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000122953 2024-09-14T22:37:39.083 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.083 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:39.083 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.083 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.083 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.083 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82513 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 557056~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:39.083 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:37:39.083 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.64150978, 2024-09-14T22:37:39.083 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059345486000000003, 2024-09-14T22:37:39.083 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:39.083 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:39.083 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:39.084 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:39.084 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:39.084 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:39.084 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82513 2024-09-14T22:37:39.084 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.084 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:39.084 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.084 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:39.084 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:37:39.084 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.084 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.085 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.085 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:39.085 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:37:39.085 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.085 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.085 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.085 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:39.085 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419200+0000", 2024-09-14T22:37:39.085 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.184e-06 2024-09-14T22:37:39.085 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.085 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:39.086 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419207+0000", 2024-09-14T22:37:39.086 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2690000000000002e-06 2024-09-14T22:37:39.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:39.086 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419224+0000", 2024-09-14T22:37:39.086 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6623e-05 2024-09-14T22:37:39.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:39.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419236+0000", 2024-09-14T22:37:39.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2367e-05 2024-09-14T22:37:39.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:39.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478294+0000", 2024-09-14T22:37:39.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059057927000000003 2024-09-14T22:37:39.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:39.088 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478402+0000", 2024-09-14T22:37:39.088 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107921 2024-09-14T22:37:39.088 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.088 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.088 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:39.088 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478544+0000", 2024-09-14T22:37:39.088 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014219499999999999 2024-09-14T22:37:39.088 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.088 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:39.088 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.088 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.089 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.089 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82514 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 704512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:39.089 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:37:39.089 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.641450677000002, 2024-09-14T22:37:39.089 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059580793, 2024-09-14T22:37:39.089 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:39.089 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:39.089 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:39.089 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:39.089 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:39.089 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:39.089 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82514 2024-09-14T22:37:39.090 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.090 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:39.090 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.090 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:39.090 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:37:39.090 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.090 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.091 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:39.091 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:37:39.091 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.091 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:39.091 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419259+0000", 2024-09-14T22:37:39.091 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.353e-06 2024-09-14T22:37:39.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.092 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.092 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:39.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419276+0000", 2024-09-14T22:37:39.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7567e-05 2024-09-14T22:37:39.092 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.092 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.092 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:39.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419294+0000", 2024-09-14T22:37:39.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.713e-05 2024-09-14T22:37:39.092 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.093 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.093 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:39.093 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419306+0000", 2024-09-14T22:37:39.093 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.279e-05 2024-09-14T22:37:39.093 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.093 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.093 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:39.093 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478589+0000", 2024-09-14T22:37:39.093 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059282978 2024-09-14T22:37:39.093 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.093 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.094 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:39.094 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478694+0000", 2024-09-14T22:37:39.094 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000104561 2024-09-14T22:37:39.094 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.094 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.094 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:39.094 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478838+0000", 2024-09-14T22:37:39.094 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014441399999999999 2024-09-14T22:37:39.094 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.094 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:39.095 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.095 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.095 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.095 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82515 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:39.095 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:37:39.095 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.641385526999997, 2024-09-14T22:37:39.095 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059815123999999997, 2024-09-14T22:37:39.095 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:39.095 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:39.095 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:39.096 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:39.096 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:39.096 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:39.096 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82515 2024-09-14T22:37:39.096 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.096 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:39.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.096 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:39.096 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:37:39.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.097 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:39.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:37:39.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.097 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:39.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419324+0000", 2024-09-14T22:37:39.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2130000000000001e-06 2024-09-14T22:37:39.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.098 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.098 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:39.098 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419331+0000", 2024-09-14T22:37:39.098 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4730000000000003e-06 2024-09-14T22:37:39.098 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.098 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.098 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:39.098 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419348+0000", 2024-09-14T22:37:39.098 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6269999999999998e-05 2024-09-14T22:37:39.098 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.099 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.099 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:39.099 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419360+0000", 2024-09-14T22:37:39.099 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2243e-05 2024-09-14T22:37:39.099 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.099 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.099 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:39.099 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478884+0000", 2024-09-14T22:37:39.099 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059524369000000001 2024-09-14T22:37:39.099 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.099 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.100 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:39.100 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478990+0000", 2024-09-14T22:37:39.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010569 2024-09-14T22:37:39.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.100 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.100 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:39.100 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479138+0000", 2024-09-14T22:37:39.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000147866 2024-09-14T22:37:39.100 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.100 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:39.100 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.101 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.101 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82516 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 999424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:39.101 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:37:39.101 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.641333021999998, 2024-09-14T22:37:39.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061979595999999998, 2024-09-14T22:37:39.101 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:39.101 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:39.101 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:39.101 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:39.101 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:39.102 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:39.102 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82516 2024-09-14T22:37:39.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.102 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:39.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:39.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:37:39.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:39.103 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:37:39.103 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.103 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.103 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.103 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:39.103 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419376+0000", 2024-09-14T22:37:39.103 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.237e-06 2024-09-14T22:37:39.103 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.103 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.103 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:39.104 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419383+0000", 2024-09-14T22:37:39.104 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8170000000000001e-06 2024-09-14T22:37:39.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.104 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.104 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:39.104 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419400+0000", 2024-09-14T22:37:39.104 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6636000000000001e-05 2024-09-14T22:37:39.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.104 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.104 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:39.104 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419412+0000", 2024-09-14T22:37:39.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2357e-05 2024-09-14T22:37:39.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:39.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481080+0000", 2024-09-14T22:37:39.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061668032999999997 2024-09-14T22:37:39.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:39.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481193+0000", 2024-09-14T22:37:39.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112335 2024-09-14T22:37:39.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:39.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481355+0000", 2024-09-14T22:37:39.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016218100000000001 2024-09-14T22:37:39.106 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.106 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:39.106 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.106 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82538 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1097728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:39.106 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:37:39.107 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.641280952000002, 2024-09-14T22:37:39.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059971705, 2024-09-14T22:37:39.107 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:39.107 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:39.107 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:39.107 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:39.107 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:39.107 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:39.107 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82538 2024-09-14T22:37:39.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.107 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:39.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:39.108 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:37:39.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:39.108 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:37:39.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:39.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419429+0000", 2024-09-14T22:37:39.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.497e-06 2024-09-14T22:37:39.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.109 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:39.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419436+0000", 2024-09-14T22:37:39.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2099999999999996e-06 2024-09-14T22:37:39.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.109 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:39.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419453+0000", 2024-09-14T22:37:39.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6840000000000001e-05 2024-09-14T22:37:39.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:39.110 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419465+0000", 2024-09-14T22:37:39.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2283000000000001e-05 2024-09-14T22:37:39.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:39.110 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479182+0000", 2024-09-14T22:37:39.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059717339000000001 2024-09-14T22:37:39.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:39.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479297+0000", 2024-09-14T22:37:39.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000114259 2024-09-14T22:37:39.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:39.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479399+0000", 2024-09-14T22:37:39.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102277 2024-09-14T22:37:39.111 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.111 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:39.112 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.112 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82539 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:39.112 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:37:39.112 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.641228101999999, 2024-09-14T22:37:39.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060166935999999997, 2024-09-14T22:37:39.112 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:39.112 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:39.112 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:39.112 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:39.112 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:39.113 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:39.113 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82539 2024-09-14T22:37:39.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.113 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:39.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:39.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:37:39.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:39.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:37:39.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:39.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419482+0000", 2024-09-14T22:37:39.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4130000000000001e-06 2024-09-14T22:37:39.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:39.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419488+0000", 2024-09-14T22:37:39.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8460000000000002e-06 2024-09-14T22:37:39.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:39.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419505+0000", 2024-09-14T22:37:39.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6477000000000002e-05 2024-09-14T22:37:39.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:39.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419518+0000", 2024-09-14T22:37:39.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2797e-05 2024-09-14T22:37:39.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:39.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479438+0000", 2024-09-14T22:37:39.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059919897 2024-09-14T22:37:39.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:39.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479547+0000", 2024-09-14T22:37:39.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109807 2024-09-14T22:37:39.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:39.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479647+0000", 2024-09-14T22:37:39.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9698999999999995e-05 2024-09-14T22:37:39.117 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.117 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:39.117 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.117 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82542 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:39.117 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:37:39.117 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.641175658999998, 2024-09-14T22:37:39.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060367430999999999, 2024-09-14T22:37:39.117 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:39.117 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:39.118 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:39.118 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:39.118 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:39.118 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:39.118 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82542 2024-09-14T22:37:39.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.118 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:39.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:39.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:37:39.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:39.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:37:39.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:39.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419534+0000", 2024-09-14T22:37:39.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T22:37:39.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:39.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419542+0000", 2024-09-14T22:37:39.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8199999999999997e-06 2024-09-14T22:37:39.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:39.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419559+0000", 2024-09-14T22:37:39.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6852999999999999e-05 2024-09-14T22:37:39.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.121 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:39.121 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419571+0000", 2024-09-14T22:37:39.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2364000000000001e-05 2024-09-14T22:37:39.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.121 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:39.121 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479686+0000", 2024-09-14T22:37:39.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060115522999999997 2024-09-14T22:37:39.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.121 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:39.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479795+0000", 2024-09-14T22:37:39.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010831399999999999 2024-09-14T22:37:39.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:39.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479900+0000", 2024-09-14T22:37:39.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105284 2024-09-14T22:37:39.122 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.122 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:39.122 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.123 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82544 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:39.123 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:37:39.123 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.641121923, 2024-09-14T22:37:39.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060590515999999997, 2024-09-14T22:37:39.123 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:39.123 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:39.123 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:39.123 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:39.123 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:39.123 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:39.123 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82544 2024-09-14T22:37:39.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.123 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:39.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:39.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:37:39.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:39.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:37:39.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:39.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419588+0000", 2024-09-14T22:37:39.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.35e-06 2024-09-14T22:37:39.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:39.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419595+0000", 2024-09-14T22:37:39.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4030000000000001e-06 2024-09-14T22:37:39.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:39.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419612+0000", 2024-09-14T22:37:39.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6670000000000001e-05 2024-09-14T22:37:39.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.126 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:39.126 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419624+0000", 2024-09-14T22:37:39.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-05 2024-09-14T22:37:39.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.126 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:39.126 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479957+0000", 2024-09-14T22:37:39.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060333287999999999 2024-09-14T22:37:39.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:39.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480074+0000", 2024-09-14T22:37:39.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116443 2024-09-14T22:37:39.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:39.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480177+0000", 2024-09-14T22:37:39.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000103102 2024-09-14T22:37:39.127 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.128 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:39.128 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.128 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82546 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:39.128 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:37:39.128 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.641069004000002, 2024-09-14T22:37:39.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060795428999999998, 2024-09-14T22:37:39.128 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:39.128 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:39.128 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:39.128 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:39.129 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:39.129 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:39.129 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82546 2024-09-14T22:37:39.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.129 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:39.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:39.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:37:39.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:39.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:37:39.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:39.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419640+0000", 2024-09-14T22:37:39.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.254e-06 2024-09-14T22:37:39.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:39.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419647+0000", 2024-09-14T22:37:39.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9e-06 2024-09-14T22:37:39.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:39.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419664+0000", 2024-09-14T22:37:39.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6345999999999999e-05 2024-09-14T22:37:39.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:39.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419676+0000", 2024-09-14T22:37:39.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.224e-05 2024-09-14T22:37:39.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:39.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480218+0000", 2024-09-14T22:37:39.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060542133999999997 2024-09-14T22:37:39.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:39.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480336+0000", 2024-09-14T22:37:39.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117472 2024-09-14T22:37:39.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:39.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480435+0000", 2024-09-14T22:37:39.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9083e-05 2024-09-14T22:37:39.132 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.133 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:39.133 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.133 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82548 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:39.133 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:37:39.133 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.641016993999997, 2024-09-14T22:37:39.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061011636000000001, 2024-09-14T22:37:39.133 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:39.133 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:39.133 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:39.133 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:39.133 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:39.134 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:39.134 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82548 2024-09-14T22:37:39.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.134 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:39.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:39.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:37:39.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:39.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:37:39.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.135 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.135 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:39.135 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419693+0000", 2024-09-14T22:37:39.135 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.387e-06 2024-09-14T22:37:39.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.135 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.135 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:39.135 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419699+0000", 2024-09-14T22:37:39.135 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8730000000000001e-06 2024-09-14T22:37:39.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.135 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.135 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:39.135 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419716+0000", 2024-09-14T22:37:39.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6416e-05 2024-09-14T22:37:39.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:39.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419728+0000", 2024-09-14T22:37:39.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2597e-05 2024-09-14T22:37:39.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:39.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480476+0000", 2024-09-14T22:37:39.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060747989000000002 2024-09-14T22:37:39.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:39.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480583+0000", 2024-09-14T22:37:39.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010638500000000001 2024-09-14T22:37:39.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:39.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480703+0000", 2024-09-14T22:37:39.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011998900000000001 2024-09-14T22:37:39.137 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.137 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:39.137 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.137 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82550 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1982464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:39.138 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:37:39.138 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.640964240999999, 2024-09-14T22:37:39.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061295483999999997, 2024-09-14T22:37:39.138 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:39.138 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:39.138 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:39.138 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:39.138 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:39.138 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:39.138 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82550 2024-09-14T22:37:39.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.138 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:39.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.139 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:39.139 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:37:39.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.139 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:39.139 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:37:39.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.139 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:39.139 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419745+0000", 2024-09-14T22:37:39.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.406e-06 2024-09-14T22:37:39.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.140 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:39.140 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419753+0000", 2024-09-14T22:37:39.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8140000000000001e-06 2024-09-14T22:37:39.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.140 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:39.140 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419770+0000", 2024-09-14T22:37:39.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6719999999999999e-05 2024-09-14T22:37:39.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.140 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:39.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419782+0000", 2024-09-14T22:37:39.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2252999999999999e-05 2024-09-14T22:37:39.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:39.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480745+0000", 2024-09-14T22:37:39.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060962385000000001 2024-09-14T22:37:39.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:39.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480900+0000", 2024-09-14T22:37:39.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015513899999999999 2024-09-14T22:37:39.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:39.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481039+0000", 2024-09-14T22:37:39.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013976700000000001 2024-09-14T22:37:39.142 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.142 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:39.142 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.142 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82588 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1064960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:39.143 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:37:39.143 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.640910351000002, 2024-09-14T22:37:39.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061863732999999997, 2024-09-14T22:37:39.143 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:39.143 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:39.143 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:39.143 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:39.143 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:39.143 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:39.143 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82588 2024-09-14T22:37:39.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.143 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:39.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:39.144 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:37:39.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:39.144 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:37:39.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:39.144 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419799+0000", 2024-09-14T22:37:39.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4160000000000001e-06 2024-09-14T22:37:39.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:39.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419807+0000", 2024-09-14T22:37:39.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2169999999999994e-06 2024-09-14T22:37:39.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:39.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419824+0000", 2024-09-14T22:37:39.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6776e-05 2024-09-14T22:37:39.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:39.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419837+0000", 2024-09-14T22:37:39.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2680000000000001e-05 2024-09-14T22:37:39.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:39.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481395+0000", 2024-09-14T22:37:39.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061558416999999997 2024-09-14T22:37:39.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:39.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481532+0000", 2024-09-14T22:37:39.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000136861 2024-09-14T22:37:39.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:39.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481662+0000", 2024-09-14T22:37:39.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012936600000000001 2024-09-14T22:37:39.147 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.147 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:39.147 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.147 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82589 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:39.147 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:37:39.148 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.640855756000001, 2024-09-14T22:37:39.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062064592000000002, 2024-09-14T22:37:39.148 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:39.148 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:39.148 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:39.148 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:39.148 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:39.148 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:39.148 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82589 2024-09-14T22:37:39.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.148 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:39.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.149 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:39.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:37:39.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.149 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:39.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:37:39.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.149 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:39.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419854+0000", 2024-09-14T22:37:39.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.367e-06 2024-09-14T22:37:39.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:39.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419862+0000", 2024-09-14T22:37:39.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8930000000000005e-06 2024-09-14T22:37:39.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:39.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419896+0000", 2024-09-14T22:37:39.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4134000000000002e-05 2024-09-14T22:37:39.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:39.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419909+0000", 2024-09-14T22:37:39.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3332999999999999e-05 2024-09-14T22:37:39.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:39.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481702+0000", 2024-09-14T22:37:39.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061792947000000001 2024-09-14T22:37:39.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.152 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:39.152 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481808+0000", 2024-09-14T22:37:39.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105655 2024-09-14T22:37:39.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.152 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:39.152 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481917+0000", 2024-09-14T22:37:39.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109263 2024-09-14T22:37:39.152 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.152 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:39.152 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.153 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82590 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1359872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:39.153 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:37:39.153 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.640782149000003, 2024-09-14T22:37:39.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062362895000000002, 2024-09-14T22:37:39.153 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:39.153 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:39.153 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:39.153 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:39.153 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:39.153 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:39.153 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82590 2024-09-14T22:37:39.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.154 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:39.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:39.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:37:39.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:39.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:37:39.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:39.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419928+0000", 2024-09-14T22:37:39.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4470000000000001e-06 2024-09-14T22:37:39.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:39.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419942+0000", 2024-09-14T22:37:39.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4229e-05 2024-09-14T22:37:39.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:39.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419972+0000", 2024-09-14T22:37:39.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.065e-05 2024-09-14T22:37:39.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:39.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419985+0000", 2024-09-14T22:37:39.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2903e-05 2024-09-14T22:37:39.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:39.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481960+0000", 2024-09-14T22:37:39.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061974435000000001 2024-09-14T22:37:39.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:39.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482106+0000", 2024-09-14T22:37:39.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014643800000000001 2024-09-14T22:37:39.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:39.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482289+0000", 2024-09-14T22:37:39.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000182793 2024-09-14T22:37:39.157 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.157 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:39.157 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.158 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82592 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:39.158 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:37:39.158 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.640705926999999, 2024-09-14T22:37:39.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062631507000000003, 2024-09-14T22:37:39.158 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:39.158 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:39.158 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:39.158 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:39.158 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:39.158 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:39.158 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82592 2024-09-14T22:37:39.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.159 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:39.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:39.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:37:39.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:39.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:37:39.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:39.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420017+0000", 2024-09-14T22:37:39.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4744e-05 2024-09-14T22:37:39.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:39.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420036+0000", 2024-09-14T22:37:39.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8680000000000001e-05 2024-09-14T22:37:39.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:39.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420054+0000", 2024-09-14T22:37:39.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7799999999999999e-05 2024-09-14T22:37:39.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.161 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:39.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420067+0000", 2024-09-14T22:37:39.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3013000000000001e-05 2024-09-14T22:37:39.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.161 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:39.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482358+0000", 2024-09-14T22:37:39.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062291445000000001 2024-09-14T22:37:39.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:39.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482469+0000", 2024-09-14T22:37:39.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110608 2024-09-14T22:37:39.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:39.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482634+0000", 2024-09-14T22:37:39.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000165217 2024-09-14T22:37:39.162 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.162 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:39.163 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.163 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82594 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1654784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:39.163 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:37:39.163 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.640607951, 2024-09-14T22:37:39.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062893849000000002, 2024-09-14T22:37:39.163 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:39.163 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:39.163 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:39.163 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:39.163 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:39.164 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:39.164 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82594 2024-09-14T22:37:39.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.164 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:39.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:39.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:37:39.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:39.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:37:39.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:39.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420102+0000", 2024-09-14T22:37:39.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5e-06 2024-09-14T22:37:39.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:39.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420115+0000", 2024-09-14T22:37:39.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.324e-05 2024-09-14T22:37:39.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:39.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420133+0000", 2024-09-14T22:37:39.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8213e-05 2024-09-14T22:37:39.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:39.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420147+0000", 2024-09-14T22:37:39.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3284e-05 2024-09-14T22:37:39.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:39.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482688+0000", 2024-09-14T22:37:39.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062541425999999997 2024-09-14T22:37:39.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:39.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482794+0000", 2024-09-14T22:37:39.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010575500000000001 2024-09-14T22:37:39.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:39.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482994+0000", 2024-09-14T22:37:39.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020043099999999999 2024-09-14T22:37:39.168 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.168 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:39.168 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.168 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82596 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:39.168 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:37:39.168 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.640544650999999, 2024-09-14T22:37:39.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063187238000000007, 2024-09-14T22:37:39.168 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:39.168 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:39.169 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:39.169 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:39.169 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:39.169 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:39.169 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82596 2024-09-14T22:37:39.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.169 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:39.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:39.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:37:39.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:39.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:37:39.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:39.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420165+0000", 2024-09-14T22:37:39.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4100000000000001e-06 2024-09-14T22:37:39.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:39.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420173+0000", 2024-09-14T22:37:39.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8129999999999996e-06 2024-09-14T22:37:39.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:39.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420190+0000", 2024-09-14T22:37:39.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7076999999999999e-05 2024-09-14T22:37:39.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:39.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420203+0000", 2024-09-14T22:37:39.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2683e-05 2024-09-14T22:37:39.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:39.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483060+0000", 2024-09-14T22:37:39.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062857114000000006 2024-09-14T22:37:39.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:39.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483167+0000", 2024-09-14T22:37:39.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010713999999999999 2024-09-14T22:37:39.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:39.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483351+0000", 2024-09-14T22:37:39.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018400099999999999 2024-09-14T22:37:39.173 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.173 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:39.173 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.174 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82597 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1949696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:39.174 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:37:39.174 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.640487825000001, 2024-09-14T22:37:39.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063466513000000002, 2024-09-14T22:37:39.174 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:39.174 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:39.174 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:39.174 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:39.174 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:39.174 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:39.175 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82597 2024-09-14T22:37:39.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.175 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:39.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:39.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:37:39.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:39.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:37:39.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:39.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:39.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420221+0000", 2024-09-14T22:37:39.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.043e-06 2024-09-14T22:37:39.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:39.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420229+0000", 2024-09-14T22:37:39.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0469999999999994e-06 2024-09-14T22:37:39.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:39.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420246+0000", 2024-09-14T22:37:39.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6339999999999999e-05 2024-09-14T22:37:39.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:39.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420258+0000", 2024-09-14T22:37:39.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2296000000000001e-05 2024-09-14T22:37:39.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:39.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483402+0000", 2024-09-14T22:37:39.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063144318000000005 2024-09-14T22:37:39.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:39.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483530+0000", 2024-09-14T22:37:39.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012713199999999999 2024-09-14T22:37:39.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:39.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:39.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:39.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483687+0000", 2024-09-14T22:37:39.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015733699999999999 2024-09-14T22:37:39.179 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.179 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:39.179 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.179 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:39.179 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:39.179 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:39.179 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:39.063+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:39.181 INFO:tasks.thrashosds.thrasher:j is {'epoch': 89, 'fsid': 'f07f407a-b17d-42c4-bf73-92dd80114a4f', 'created': '2024-09-14T22:35:45.983055+0000', 'modified': '2024-09-14T22:37:38.819969+0000', 'last_up_change': '2024-09-14T22:37:12.616042+0000', 'last_in_change': '2024-09-14T22:35:48.283160+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T22:35:58.938784+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8766727447509766, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T22:36:04.304242+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.8766727447509766, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T22:36:14.374249+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 29, 'snap_epoch': 89, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.8766727447509766, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': '9831fcc5-9040-4a80-99d4-838225e65139', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 76, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6824', 'nonce': 1702929600}, {'type': 'v1', 'addr': '172.21.15.1:6825', 'nonce': 1702929600}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6826', 'nonce': 1702929600}, {'type': 'v1', 'addr': '172.21.15.1:6827', 'nonce': 1702929600}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6830', 'nonce': 1702929600}, {'type': 'v1', 'addr': '172.21.15.1:6831', 'nonce': 1702929600}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6828', 'nonce': 1702929600}, {'type': 'v1', 'addr': '172.21.15.1:6829', 'nonce': 1702929600}]}, 'public_addr': '172.21.15.1:6825/1702929600', 'cluster_addr': '172.21.15.1:6827/1702929600', 'heartbeat_back_addr': '172.21.15.1:6831/1702929600', 'heartbeat_front_addr': '172.21.15.1:6829/1702929600', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '7f79abe8-115d-46b2-8677-72fcb0c8ab7d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.1606292724609375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 86, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6808', 'nonce': 719990653}, {'type': 'v1', 'addr': '172.21.15.1:6809', 'nonce': 719990653}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6810', 'nonce': 719990653}, {'type': 'v1', 'addr': '172.21.15.1:6811', 'nonce': 719990653}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6814', 'nonce': 719990653}, {'type': 'v1', 'addr': '172.21.15.1:6815', 'nonce': 719990653}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6812', 'nonce': 719990653}, {'type': 'v1', 'addr': '172.21.15.1:6813', 'nonce': 719990653}]}, 'public_addr': '172.21.15.1:6809/719990653', 'cluster_addr': '172.21.15.1:6811/719990653', 'heartbeat_back_addr': '172.21.15.1:6815/719990653', 'heartbeat_front_addr': '172.21.15.1:6813/719990653', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '1f5768b2-4db4-4b2e-8d5e-5206f8bde28d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 14, 'last_clean_end': 30, 'up_from': 66, 'up_thru': 70, 'down_at': 31, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6816', 'nonce': 1006730601}, {'type': 'v1', 'addr': '172.21.15.1:6817', 'nonce': 1006730601}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6818', 'nonce': 1006730601}, {'type': 'v1', 'addr': '172.21.15.1:6819', 'nonce': 1006730601}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6822', 'nonce': 1006730601}, {'type': 'v1', 'addr': '172.21.15.1:6823', 'nonce': 1006730601}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6820', 'nonce': 1006730601}, {'type': 'v1', 'addr': '172.21.15.1:6821', 'nonce': 1006730601}]}, 'public_addr': '172.21.15.1:6817/1006730601', 'cluster_addr': '172.21.15.1:6819/1006730601', 'heartbeat_back_addr': '172.21.15.1:6823/1006730601', 'heartbeat_front_addr': '172.21.15.1:6821/1006730601', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '8103f3c7-7899-4fc3-bb02-dd2273120138', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.852752685546875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 87, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6800', 'nonce': 750403671}, {'type': 'v1', 'addr': '172.21.15.1:6801', 'nonce': 750403671}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6802', 'nonce': 750403671}, {'type': 'v1', 'addr': '172.21.15.1:6803', 'nonce': 750403671}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6806', 'nonce': 750403671}, {'type': 'v1', 'addr': '172.21.15.1:6807', 'nonce': 750403671}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6804', 'nonce': 750403671}, {'type': 'v1', 'addr': '172.21.15.1:6805', 'nonce': 750403671}]}, 'public_addr': '172.21.15.1:6801/750403671', 'cluster_addr': '172.21.15.1:6803/750403671', 'heartbeat_back_addr': '172.21.15.1:6807/750403671', 'heartbeat_front_addr': '172.21.15.1:6805/750403671', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '23cce5cb-726e-44c2-8edb-c2567a89dfbc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6816', 'nonce': 1408471829}, {'type': 'v1', 'addr': '172.21.15.176:6817', 'nonce': 1408471829}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6818', 'nonce': 1408471829}, {'type': 'v1', 'addr': '172.21.15.176:6819', 'nonce': 1408471829}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6822', 'nonce': 1408471829}, {'type': 'v1', 'addr': '172.21.15.176:6823', 'nonce': 1408471829}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6820', 'nonce': 1408471829}, {'type': 'v1', 'addr': '172.21.15.176:6821', 'nonce': 1408471829}]}, 'public_addr': '172.21.15.176:6817/1408471829', 'cluster_addr': '172.21.15.176:6819/1408471829', 'heartbeat_back_addr': '172.21.15.176:6823/1408471829', 'heartbeat_front_addr': '172.21.15.176:6821/1408471829', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'eb2f187b-d2f1-49ab-ad5e-ea6dc441a28e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6805', 'nonce': 1945902326}, {'type': 'v1', 'addr': '172.21.15.176:6808', 'nonce': 1945902326}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6810', 'nonce': 1945902326}, {'type': 'v1', 'addr': '172.21.15.176:6811', 'nonce': 1945902326}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6814', 'nonce': 1945902326}, {'type': 'v1', 'addr': '172.21.15.176:6815', 'nonce': 1945902326}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6812', 'nonce': 1945902326}, {'type': 'v1', 'addr': '172.21.15.176:6813', 'nonce': 1945902326}]}, 'public_addr': '172.21.15.176:6808/1945902326', 'cluster_addr': '172.21.15.176:6811/1945902326', 'heartbeat_back_addr': '172.21.15.176:6815/1945902326', 'heartbeat_front_addr': '172.21.15.176:6813/1945902326', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'd371aed8-2ad2-49a5-8ece-4b7fd37b4822', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 77, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6800', 'nonce': 4116085586}, {'type': 'v1', 'addr': '172.21.15.176:6801', 'nonce': 4116085586}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6802', 'nonce': 4116085586}, {'type': 'v1', 'addr': '172.21.15.176:6803', 'nonce': 4116085586}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6807', 'nonce': 4116085586}, {'type': 'v1', 'addr': '172.21.15.176:6809', 'nonce': 4116085586}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6804', 'nonce': 4116085586}, {'type': 'v1', 'addr': '172.21.15.176:6806', 'nonce': 4116085586}]}, 'public_addr': '172.21.15.176:6801/4116085586', 'cluster_addr': '172.21.15.176:6803/4116085586', 'heartbeat_back_addr': '172.21.15.176:6809/4116085586', 'heartbeat_front_addr': '172.21.15.176:6806/4116085586', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'a012c819-8d49-4310-a840-c2752d9314ab', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 76, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6824', 'nonce': 877760244}, {'type': 'v1', 'addr': '172.21.15.176:6825', 'nonce': 877760244}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6826', 'nonce': 877760244}, {'type': 'v1', 'addr': '172.21.15.176:6827', 'nonce': 877760244}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6830', 'nonce': 877760244}, {'type': 'v1', 'addr': '172.21.15.176:6831', 'nonce': 877760244}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6828', 'nonce': 877760244}, {'type': 'v1', 'addr': '172.21.15.176:6829', 'nonce': 877760244}]}, 'public_addr': '172.21.15.176:6825/877760244', 'cluster_addr': '172.21.15.176:6827/877760244', 'heartbeat_back_addr': '172.21.15.176:6831/877760244', 'heartbeat_front_addr': '172.21.15.176:6829/877760244', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:55.511688+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.742414+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.871355+0000', 'dead_epoch': 31}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.504553+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.589640+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.518845+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.731632+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.952634+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.2', 'mappings': [{'from': 3, 'to': 6}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T22:37:39.181 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:37:39.182 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-14T22:37:39.182 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:37:39.183 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:37:39.183 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:37:39.184 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:37:39.184 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:37:39.184 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:37:39.185 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:37:39.185 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:37:39.185 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:37:39.186 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:37:39.186 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:37:39.206 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:37:39.207 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:37:39.227 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:39.223+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:39.240 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:37:39.240 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:37:39.261 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:37:39.261 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:37:39.320 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:39.319+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:39.356 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:39.357 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:39.373 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:37:39.423 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:39.419+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:39.469 INFO:teuthology.orchestra.run.smithi001.stdout:4667 writesame 0xd2ccbdc thru 0xd2cd2b8 data_size 0x1(0x6dd bytes) 2024-09-14T22:37:39.469 INFO:teuthology.orchestra.run.smithi001.stdout:4668 read 0x11cc609 thru 0x11d6af4 (0xa4ec bytes) 2024-09-14T22:37:39.469 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:39.469 INFO:teuthology.orchestra.run.smithi001.stdout:4670 write 0x930a23d thru 0x930a40d (0x1d1 bytes) 2024-09-14T22:37:39.488 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:39.483+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:39.492 INFO:teuthology.orchestra.run.smithi001.stdout:4673 trunc from 0xd2cd2b9 to 0x1da0625 2024-09-14T22:37:39.498 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:39.498 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:39.515 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:37:39.524 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:37:39.525 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-14T22:37:39.543 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":133,"stamp":"2024-09-14T22:37:38.885849+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1090088397,"num_objects":1161,"num_object_clones":320,"num_object_copies":2322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1161,"num_whiteouts":384,"num_read":149969,"num_read_kb":546427,"num_write":16546,"num_write_kb":257009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":612,"num_bytes_recovered":31733554,"num_keys_recovered":22,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12910,"ondisk_log_size":12910,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":330208,"kb_used_data":43120,"kb_used_omap":17,"kb_used_meta":286894,"kb_avail":754644512,"statfs":{"total":773094113280,"available":772755980288,"internally_reserved":0,"allocated":44154880,"data_stored":356305721,"data_compressed":9970239,"data_compressed_allocated":27910144,"data_compressed_original":347886518,"omap_allocated":18180,"internal_metadata":293779708},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20931216,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1605,"num_read_kb":1474,"num_write":407,"num_write_kb":2301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":4343769,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001137"},"pg_stats":[{"pgid":"3.1f","version":"81'242","reported_seq":4131,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.829197+0000","last_change":"2024-09-14T22:37:25.739233+0000","last_active":"2024-09-14T22:37:37.829197+0000","last_peered":"2024-09-14T22:37:37.829197+0000","last_clean":"2024-09-14T22:37:37.829197+0000","last_became_active":"2024-09-14T22:37:25.718906+0000","last_became_peered":"2024-09-14T22:37:25.718906+0000","last_unstale":"2024-09-14T22:37:37.829197+0000","last_undegraded":"2024-09-14T22:37:37.829197+0000","last_fullsized":"2024-09-14T22:37:37.829197+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:09:54.383459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017190879999999999,"stat_sum":{"num_bytes":51625811,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":14,"num_read":3899,"num_read_kb":15185,"num_write":258,"num_write_kb":6027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":3277632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"79'921","reported_seq":4850,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.815785+0000","last_change":"2024-09-14T22:36:17.733296+0000","last_active":"2024-09-14T22:37:37.815785+0000","last_peered":"2024-09-14T22:37:37.815785+0000","last_clean":"2024-09-14T22:37:37.815785+0000","last_became_active":"2024-09-14T22:36:15.709272+0000","last_became_peered":"2024-09-14T22:36:15.709272+0000","last_unstale":"2024-09-14T22:37:37.815785+0000","last_undegraded":"2024-09-14T22:37:37.815785+0000","last_fullsized":"2024-09-14T22:37:37.815785+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":921,"log_dups_size":0,"ondisk_log_size":921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:55:25.381951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001723619,"stat_sum":{"num_bytes":62550087,"num_objects":32,"num_object_clones":9,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":12,"num_read":8781,"num_read_kb":22718,"num_write":1480,"num_write_kb":8170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"79'314","reported_seq":7582,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.845846+0000","last_change":"2024-09-14T22:36:17.702313+0000","last_active":"2024-09-14T22:37:37.845846+0000","last_peered":"2024-09-14T22:37:37.845846+0000","last_clean":"2024-09-14T22:37:37.845846+0000","last_became_active":"2024-09-14T22:36:15.719488+0000","last_became_peered":"2024-09-14T22:36:15.719488+0000","last_unstale":"2024-09-14T22:37:37.845846+0000","last_undegraded":"2024-09-14T22:37:37.845846+0000","last_fullsized":"2024-09-14T22:37:37.845846+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:34:41.068274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053914200000000005,"stat_sum":{"num_bytes":64267617,"num_objects":52,"num_object_clones":19,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":20,"num_read":7454,"num_read_kb":40817,"num_write":291,"num_write_kb":8757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"88'312","reported_seq":3027,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.855304+0000","last_change":"2024-09-14T22:36:17.702413+0000","last_active":"2024-09-14T22:37:37.855304+0000","last_peered":"2024-09-14T22:37:37.855304+0000","last_clean":"2024-09-14T22:37:37.855304+0000","last_became_active":"2024-09-14T22:36:15.719470+0000","last_became_peered":"2024-09-14T22:36:15.719470+0000","last_unstale":"2024-09-14T22:37:37.855304+0000","last_undegraded":"2024-09-14T22:37:37.855304+0000","last_fullsized":"2024-09-14T22:37:37.855304+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:02:43.365673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055079499999999997,"stat_sum":{"num_bytes":25755632,"num_objects":45,"num_object_clones":12,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":19,"num_read":2679,"num_read_kb":14735,"num_write":343,"num_write_kb":7468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'395","reported_seq":4562,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.829237+0000","last_change":"2024-09-14T22:37:25.736579+0000","last_active":"2024-09-14T22:37:37.829237+0000","last_peered":"2024-09-14T22:37:37.829237+0000","last_clean":"2024-09-14T22:37:37.829237+0000","last_became_active":"2024-09-14T22:37:25.718519+0000","last_became_peered":"2024-09-14T22:37:25.718519+0000","last_unstale":"2024-09-14T22:37:37.829237+0000","last_undegraded":"2024-09-14T22:37:37.829237+0000","last_fullsized":"2024-09-14T22:37:37.829237+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:47:04.824272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092291499999999996,"stat_sum":{"num_bytes":25177553,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":11,"num_read":4929,"num_read_kb":28545,"num_write":494,"num_write_kb":8858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":7858104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"83'332","reported_seq":2589,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.885614+0000","last_change":"2024-09-14T22:36:17.711364+0000","last_active":"2024-09-14T22:37:37.885614+0000","last_peered":"2024-09-14T22:37:37.885614+0000","last_clean":"2024-09-14T22:37:37.885614+0000","last_became_active":"2024-09-14T22:36:15.701852+0000","last_became_peered":"2024-09-14T22:36:15.701852+0000","last_unstale":"2024-09-14T22:37:37.885614+0000","last_undegraded":"2024-09-14T22:37:37.885614+0000","last_fullsized":"2024-09-14T22:37:37.885614+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:42:54.258305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012502780000000001,"stat_sum":{"num_bytes":8128485,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":17,"num_read":3212,"num_read_kb":9184,"num_write":354,"num_write_kb":5846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"87'324","reported_seq":4979,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T22:37:36.814271+0000","last_change":"2024-09-14T22:36:17.705423+0000","last_active":"2024-09-14T22:37:36.814271+0000","last_peered":"2024-09-14T22:37:36.814271+0000","last_clean":"2024-09-14T22:37:36.814271+0000","last_became_active":"2024-09-14T22:36:15.700990+0000","last_became_peered":"2024-09-14T22:36:15.700990+0000","last_unstale":"2024-09-14T22:37:36.814271+0000","last_undegraded":"2024-09-14T22:37:36.814271+0000","last_fullsized":"2024-09-14T22:37:36.814271+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:20:52.988019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010520620000000001,"stat_sum":{"num_bytes":20139232,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":11,"num_read":5021,"num_read_kb":15102,"num_write":342,"num_write_kb":5325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"83'225","reported_seq":2744,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.815152+0000","last_change":"2024-09-14T22:37:25.719758+0000","last_active":"2024-09-14T22:37:37.815152+0000","last_peered":"2024-09-14T22:37:37.815152+0000","last_clean":"2024-09-14T22:37:37.815152+0000","last_became_active":"2024-09-14T22:37:25.717354+0000","last_became_peered":"2024-09-14T22:37:25.717354+0000","last_unstale":"2024-09-14T22:37:37.815152+0000","last_undegraded":"2024-09-14T22:37:37.815152+0000","last_fullsized":"2024-09-14T22:37:37.815152+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:09:53.466611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002063484,"stat_sum":{"num_bytes":17991742,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":4,"num_read":2528,"num_read_kb":18581,"num_write":297,"num_write_kb":10190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":633727,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"87'243","reported_seq":3182,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.829282+0000","last_change":"2024-09-14T22:36:17.721195+0000","last_active":"2024-09-14T22:37:37.829282+0000","last_peered":"2024-09-14T22:37:37.829282+0000","last_clean":"2024-09-14T22:37:37.829282+0000","last_became_active":"2024-09-14T22:36:15.715869+0000","last_became_peered":"2024-09-14T22:36:15.715869+0000","last_unstale":"2024-09-14T22:37:37.829282+0000","last_undegraded":"2024-09-14T22:37:37.829282+0000","last_fullsized":"2024-09-14T22:37:37.829282+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:04:50.912894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001160672,"stat_sum":{"num_bytes":15041443,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":11,"num_read":3185,"num_read_kb":13746,"num_write":279,"num_write_kb":14932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"79'393","reported_seq":3898,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.845946+0000","last_change":"2024-09-14T22:36:17.703048+0000","last_active":"2024-09-14T22:37:37.845946+0000","last_peered":"2024-09-14T22:37:37.845946+0000","last_clean":"2024-09-14T22:37:37.845946+0000","last_became_active":"2024-09-14T22:36:15.725078+0000","last_became_peered":"2024-09-14T22:36:15.725078+0000","last_unstale":"2024-09-14T22:37:37.845946+0000","last_undegraded":"2024-09-14T22:37:37.845946+0000","last_fullsized":"2024-09-14T22:37:37.845946+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:52.048783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010500290000000001,"stat_sum":{"num_bytes":59689882,"num_objects":49,"num_object_clones":14,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":15,"num_read":4376,"num_read_kb":28795,"num_write":533,"num_write_kb":11852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"87'233","reported_seq":2905,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.829560+0000","last_change":"2024-09-14T22:36:18.009371+0000","last_active":"2024-09-14T22:37:37.829560+0000","last_peered":"2024-09-14T22:37:37.829560+0000","last_clean":"2024-09-14T22:37:37.829560+0000","last_became_active":"2024-09-14T22:36:15.717341+0000","last_became_peered":"2024-09-14T22:36:15.717341+0000","last_unstale":"2024-09-14T22:37:37.829560+0000","last_undegraded":"2024-09-14T22:37:37.829560+0000","last_fullsized":"2024-09-14T22:37:37.829560+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:02:39.822372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054571999999999997,"stat_sum":{"num_bytes":8921493,"num_objects":24,"num_object_clones":7,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":8,"num_read":2764,"num_read_kb":3328,"num_write":253,"num_write_kb":5258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"88'913","reported_seq":7008,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.823205+0000","last_change":"2024-09-14T22:36:17.711384+0000","last_active":"2024-09-14T22:37:37.823205+0000","last_peered":"2024-09-14T22:37:37.823205+0000","last_clean":"2024-09-14T22:37:37.823205+0000","last_became_active":"2024-09-14T22:36:15.723075+0000","last_became_peered":"2024-09-14T22:36:15.723075+0000","last_unstale":"2024-09-14T22:37:37.823205+0000","last_undegraded":"2024-09-14T22:37:37.823205+0000","last_fullsized":"2024-09-14T22:37:37.823205+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":913,"log_dups_size":0,"ondisk_log_size":913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:48:29.925119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010804009999999999,"stat_sum":{"num_bytes":37175323,"num_objects":47,"num_object_clones":13,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":14,"num_read":9237,"num_read_kb":21931,"num_write":1333,"num_write_kb":9660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":151,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.818357+0000","last_change":"2024-09-14T22:36:08.042860+0000","last_active":"2024-09-14T22:37:37.818357+0000","last_peered":"2024-09-14T22:37:37.818357+0000","last_clean":"2024-09-14T22:37:37.818357+0000","last_became_active":"2024-09-14T22:36:06.040961+0000","last_became_peered":"2024-09-14T22:36:06.040961+0000","last_unstale":"2024-09-14T22:37:37.818357+0000","last_undegraded":"2024-09-14T22:37:37.818357+0000","last_fullsized":"2024-09-14T22:37:37.818357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:03:48.547834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010780950000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"86'325","reported_seq":1666,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.829102+0000","last_change":"2024-09-14T22:36:17.720897+0000","last_active":"2024-09-14T22:37:37.829102+0000","last_peered":"2024-09-14T22:37:37.829102+0000","last_clean":"2024-09-14T22:37:37.829102+0000","last_became_active":"2024-09-14T22:36:15.700982+0000","last_became_peered":"2024-09-14T22:36:15.700982+0000","last_unstale":"2024-09-14T22:37:37.829102+0000","last_undegraded":"2024-09-14T22:37:37.829102+0000","last_fullsized":"2024-09-14T22:37:37.829102+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:09:53.473707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010495459999999999,"stat_sum":{"num_bytes":20611036,"num_objects":40,"num_object_clones":12,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":13,"num_read":2371,"num_read_kb":8672,"num_write":333,"num_write_kb":8270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":101,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.818485+0000","last_change":"2024-09-14T22:36:41.930087+0000","last_active":"2024-09-14T22:37:37.818485+0000","last_peered":"2024-09-14T22:37:37.818485+0000","last_clean":"2024-09-14T22:37:37.818485+0000","last_became_active":"2024-09-14T22:36:41.929522+0000","last_became_peered":"2024-09-14T22:36:41.929522+0000","last_unstale":"2024-09-14T22:37:37.818485+0000","last_undegraded":"2024-09-14T22:37:37.818485+0000","last_fullsized":"2024-09-14T22:37:37.818485+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:15:10.589042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"87'336","reported_seq":4486,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.829837+0000","last_change":"2024-09-14T22:36:21.722517+0000","last_active":"2024-09-14T22:37:37.829837+0000","last_peered":"2024-09-14T22:37:37.829837+0000","last_clean":"2024-09-14T22:37:37.829837+0000","last_became_active":"2024-09-14T22:36:21.722085+0000","last_became_peered":"2024-09-14T22:36:21.722085+0000","last_unstale":"2024-09-14T22:37:37.829837+0000","last_undegraded":"2024-09-14T22:37:37.829837+0000","last_fullsized":"2024-09-14T22:37:37.829837+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:52:26.553766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020774109999999999,"stat_sum":{"num_bytes":47068084,"num_objects":32,"num_object_clones":6,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":10,"num_read":5104,"num_read_kb":23364,"num_write":400,"num_write_kb":6559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":151,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.829848+0000","last_change":"2024-09-14T22:36:08.054606+0000","last_active":"2024-09-14T22:37:37.829848+0000","last_peered":"2024-09-14T22:37:37.829848+0000","last_clean":"2024-09-14T22:37:37.829848+0000","last_became_active":"2024-09-14T22:36:06.266908+0000","last_became_peered":"2024-09-14T22:36:06.266908+0000","last_unstale":"2024-09-14T22:37:37.829848+0000","last_undegraded":"2024-09-14T22:37:37.829848+0000","last_fullsized":"2024-09-14T22:37:37.829848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:36.738643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098249300000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"83'1119","reported_seq":3766,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T22:37:34.800664+0000","last_change":"2024-09-14T22:37:17.676125+0000","last_active":"2024-09-14T22:37:34.800664+0000","last_peered":"2024-09-14T22:37:34.800664+0000","last_clean":"2024-09-14T22:37:34.800664+0000","last_became_active":"2024-09-14T22:37:17.674322+0000","last_became_peered":"2024-09-14T22:37:17.674322+0000","last_unstale":"2024-09-14T22:37:34.800664+0000","last_undegraded":"2024-09-14T22:37:34.800664+0000","last_fullsized":"2024-09-14T22:37:34.800664+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1119,"log_dups_size":0,"ondisk_log_size":1119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:54:44.017669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012564830000000001,"stat_sum":{"num_bytes":16092273,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":15,"num_read":7482,"num_read_kb":14334,"num_write":1975,"num_write_kb":9318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":4264139,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":101,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.829284+0000","last_change":"2024-09-14T22:36:41.930961+0000","last_active":"2024-09-14T22:37:37.829284+0000","last_peered":"2024-09-14T22:37:37.829284+0000","last_clean":"2024-09-14T22:37:37.829284+0000","last_became_active":"2024-09-14T22:36:41.930619+0000","last_became_peered":"2024-09-14T22:36:41.930619+0000","last_unstale":"2024-09-14T22:37:37.829284+0000","last_undegraded":"2024-09-14T22:37:37.829284+0000","last_fullsized":"2024-09-14T22:37:37.829284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:04:40.582999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"83'384","reported_seq":3059,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.845920+0000","last_change":"2024-09-14T22:36:17.702597+0000","last_active":"2024-09-14T22:37:37.845920+0000","last_peered":"2024-09-14T22:37:37.845920+0000","last_clean":"2024-09-14T22:37:37.845920+0000","last_became_active":"2024-09-14T22:36:15.717186+0000","last_became_peered":"2024-09-14T22:36:15.717186+0000","last_unstale":"2024-09-14T22:37:37.845920+0000","last_undegraded":"2024-09-14T22:37:37.845920+0000","last_fullsized":"2024-09-14T22:37:37.845920+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:42:55.836711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071220099999999998,"stat_sum":{"num_bytes":21818773,"num_objects":33,"num_object_clones":10,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":13,"num_read":4078,"num_read_kb":5907,"num_write":362,"num_write_kb":2204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"79'255","reported_seq":3697,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.813380+0000","last_change":"2024-09-14T22:36:17.711068+0000","last_active":"2024-09-14T22:37:37.813380+0000","last_peered":"2024-09-14T22:37:37.813380+0000","last_clean":"2024-09-14T22:37:37.813380+0000","last_became_active":"2024-09-14T22:36:15.706147+0000","last_became_peered":"2024-09-14T22:36:15.706147+0000","last_unstale":"2024-09-14T22:37:37.813380+0000","last_undegraded":"2024-09-14T22:37:37.813380+0000","last_fullsized":"2024-09-14T22:37:37.813380+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:44:53.470473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082625500000000002,"stat_sum":{"num_bytes":37489798,"num_objects":28,"num_object_clones":9,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":9,"num_read":3360,"num_read_kb":16744,"num_write":244,"num_write_kb":5687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":153,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.845817+0000","last_change":"2024-09-14T22:36:08.055938+0000","last_active":"2024-09-14T22:37:37.845817+0000","last_peered":"2024-09-14T22:37:37.845817+0000","last_clean":"2024-09-14T22:37:37.845817+0000","last_became_active":"2024-09-14T22:36:06.039600+0000","last_became_peered":"2024-09-14T22:36:06.039600+0000","last_unstale":"2024-09-14T22:37:37.845817+0000","last_undegraded":"2024-09-14T22:37:37.845817+0000","last_fullsized":"2024-09-14T22:37:37.845817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:49:09.245563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021617870000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"78'739","reported_seq":8115,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T22:37:34.800529+0000","last_change":"2024-09-14T22:37:16.669227+0000","last_active":"2024-09-14T22:37:34.800529+0000","last_peered":"2024-09-14T22:37:34.800529+0000","last_clean":"2024-09-14T22:37:34.800529+0000","last_became_active":"2024-09-14T22:37:16.662230+0000","last_became_peered":"2024-09-14T22:37:16.662230+0000","last_unstale":"2024-09-14T22:37:34.800529+0000","last_undegraded":"2024-09-14T22:37:34.800529+0000","last_fullsized":"2024-09-14T22:37:34.800529+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:18:22.556190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021500769999999998,"stat_sum":{"num_bytes":64266431,"num_objects":44,"num_object_clones":11,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":13,"num_read":9395,"num_read_kb":19599,"num_write":1052,"num_write_kb":9919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":4851034,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":72,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T22:37:34.800562+0000","last_change":"2024-09-14T22:37:13.762987+0000","last_active":"2024-09-14T22:37:34.800562+0000","last_peered":"2024-09-14T22:37:34.800562+0000","last_clean":"2024-09-14T22:37:34.800562+0000","last_became_active":"2024-09-14T22:37:13.762641+0000","last_became_peered":"2024-09-14T22:37:13.762641+0000","last_unstale":"2024-09-14T22:37:34.800562+0000","last_undegraded":"2024-09-14T22:37:34.800562+0000","last_fullsized":"2024-09-14T22:37:34.800562+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:30:59.887867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"83'273","reported_seq":1849,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.829202+0000","last_change":"2024-09-14T22:36:17.720438+0000","last_active":"2024-09-14T22:37:37.829202+0000","last_peered":"2024-09-14T22:37:37.829202+0000","last_clean":"2024-09-14T22:37:37.829202+0000","last_became_active":"2024-09-14T22:36:15.711105+0000","last_became_peered":"2024-09-14T22:36:15.711105+0000","last_unstale":"2024-09-14T22:37:37.829202+0000","last_undegraded":"2024-09-14T22:37:37.829202+0000","last_fullsized":"2024-09-14T22:37:37.829202+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:38:25.829117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085549400000000005,"stat_sum":{"num_bytes":43842060,"num_objects":25,"num_object_clones":8,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":9,"num_read":1628,"num_read_kb":8229,"num_write":286,"num_write_kb":2071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":151,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.829351+0000","last_change":"2024-09-14T22:36:08.054759+0000","last_active":"2024-09-14T22:37:37.829351+0000","last_peered":"2024-09-14T22:37:37.829351+0000","last_clean":"2024-09-14T22:37:37.829351+0000","last_became_active":"2024-09-14T22:36:06.039969+0000","last_became_peered":"2024-09-14T22:36:06.039969+0000","last_unstale":"2024-09-14T22:37:37.829351+0000","last_undegraded":"2024-09-14T22:37:37.829351+0000","last_fullsized":"2024-09-14T22:37:37.829351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:03:45.369532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010817520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"87'901","reported_seq":3352,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.818596+0000","last_change":"2024-09-14T22:36:51.718064+0000","last_active":"2024-09-14T22:37:37.818596+0000","last_peered":"2024-09-14T22:37:37.818596+0000","last_clean":"2024-09-14T22:37:37.818596+0000","last_became_active":"2024-09-14T22:36:51.715823+0000","last_became_peered":"2024-09-14T22:36:51.715823+0000","last_unstale":"2024-09-14T22:37:37.818596+0000","last_undegraded":"2024-09-14T22:37:37.818596+0000","last_fullsized":"2024-09-14T22:37:37.818596+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:49:20.197282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47643188,"num_objects":41,"num_object_clones":10,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":10,"num_read":6634,"num_read_kb":10451,"num_write":1384,"num_write_kb":6322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":173209,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":159,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.845738+0000","last_change":"2024-09-14T22:37:13.625072+0000","last_active":"2024-09-14T22:37:37.845738+0000","last_peered":"2024-09-14T22:37:37.845738+0000","last_clean":"2024-09-14T22:37:37.845738+0000","last_became_active":"2024-09-14T22:37:13.624669+0000","last_became_peered":"2024-09-14T22:37:13.624669+0000","last_unstale":"2024-09-14T22:37:37.845738+0000","last_undegraded":"2024-09-14T22:37:37.845738+0000","last_fullsized":"2024-09-14T22:37:37.845738+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:54:20.545098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011401880000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":302,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.829445+0000","last_change":"2024-09-14T22:36:00.402862+0000","last_active":"2024-09-14T22:37:37.829445+0000","last_peered":"2024-09-14T22:37:37.829445+0000","last_clean":"2024-09-14T22:37:37.829445+0000","last_became_active":"2024-09-14T22:36:00.400529+0000","last_became_peered":"2024-09-14T22:36:00.400529+0000","last_unstale":"2024-09-14T22:37:37.829445+0000","last_undegraded":"2024-09-14T22:37:37.829445+0000","last_fullsized":"2024-09-14T22:37:37.829445+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_clean_scrub_stamp":"2024-09-14T22:35:58.989025+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:07:12.951728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"86'377","reported_seq":5296,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.824201+0000","last_change":"2024-09-14T22:37:37.823251+0000","last_active":"2024-09-14T22:37:37.824201+0000","last_peered":"2024-09-14T22:37:37.824201+0000","last_clean":"2024-09-14T22:37:37.824201+0000","last_became_active":"2024-09-14T22:37:37.820975+0000","last_became_peered":"2024-09-14T22:37:37.820975+0000","last_unstale":"2024-09-14T22:37:37.824201+0000","last_undegraded":"2024-09-14T22:37:37.824201+0000","last_fullsized":"2024-09-14T22:37:37.824201+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:34:15.247344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001381511,"stat_sum":{"num_bytes":30612196,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":9,"num_read":5608,"num_read_kb":9967,"num_write":547,"num_write_kb":12050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":6470986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"87'245","reported_seq":5679,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.813008+0000","last_change":"2024-09-14T22:37:16.658991+0000","last_active":"2024-09-14T22:37:37.813008+0000","last_peered":"2024-09-14T22:37:37.813008+0000","last_clean":"2024-09-14T22:37:37.813008+0000","last_became_active":"2024-09-14T22:37:16.657865+0000","last_became_peered":"2024-09-14T22:37:16.657865+0000","last_unstale":"2024-09-14T22:37:37.813008+0000","last_undegraded":"2024-09-14T22:37:37.813008+0000","last_fullsized":"2024-09-14T22:37:37.813008+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:50:15.524704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060272699999999995,"stat_sum":{"num_bytes":50373376,"num_objects":44,"num_object_clones":14,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":12,"num_read":5875,"num_read_kb":12262,"num_write":237,"num_write_kb":5256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":4204723,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"88'480","reported_seq":3237,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.893673+0000","last_change":"2024-09-14T22:36:17.776247+0000","last_active":"2024-09-14T22:37:37.893673+0000","last_peered":"2024-09-14T22:37:37.893673+0000","last_clean":"2024-09-14T22:37:37.893673+0000","last_became_active":"2024-09-14T22:36:15.718794+0000","last_became_peered":"2024-09-14T22:36:15.718794+0000","last_unstale":"2024-09-14T22:37:37.893673+0000","last_undegraded":"2024-09-14T22:37:37.893673+0000","last_fullsized":"2024-09-14T22:37:37.893673+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:27:39.448131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088662699999999997,"stat_sum":{"num_bytes":19371023,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":8,"num_read":5143,"num_read_kb":19215,"num_write":468,"num_write_kb":13120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"88'309","reported_seq":1708,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.866294+0000","last_change":"2024-09-14T22:36:17.733002+0000","last_active":"2024-09-14T22:37:37.866294+0000","last_peered":"2024-09-14T22:37:37.866294+0000","last_clean":"2024-09-14T22:37:37.866294+0000","last_became_active":"2024-09-14T22:36:15.702885+0000","last_became_peered":"2024-09-14T22:36:15.702885+0000","last_unstale":"2024-09-14T22:37:37.866294+0000","last_undegraded":"2024-09-14T22:37:37.866294+0000","last_fullsized":"2024-09-14T22:37:37.866294+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:47:35.197333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015180580000000001,"stat_sum":{"num_bytes":34136133,"num_objects":36,"num_object_clones":11,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":12,"num_read":2370,"num_read_kb":8363,"num_write":288,"num_write_kb":6338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"87'282","reported_seq":5346,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.846144+0000","last_change":"2024-09-14T22:36:17.702738+0000","last_active":"2024-09-14T22:37:37.846144+0000","last_peered":"2024-09-14T22:37:37.846144+0000","last_clean":"2024-09-14T22:37:37.846144+0000","last_became_active":"2024-09-14T22:36:15.713452+0000","last_became_peered":"2024-09-14T22:36:15.713452+0000","last_unstale":"2024-09-14T22:37:37.846144+0000","last_undegraded":"2024-09-14T22:37:37.846144+0000","last_fullsized":"2024-09-14T22:37:37.846144+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:19:59.961264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082782099999999996,"stat_sum":{"num_bytes":34168770,"num_objects":35,"num_object_clones":14,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":14,"num_read":5162,"num_read_kb":17038,"num_write":315,"num_write_kb":8952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"83'238","reported_seq":3035,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.829792+0000","last_change":"2024-09-14T22:36:18.009776+0000","last_active":"2024-09-14T22:37:37.829792+0000","last_peered":"2024-09-14T22:37:37.829792+0000","last_clean":"2024-09-14T22:37:37.829792+0000","last_became_active":"2024-09-14T22:36:15.723205+0000","last_became_peered":"2024-09-14T22:36:15.723205+0000","last_unstale":"2024-09-14T22:37:37.829792+0000","last_undegraded":"2024-09-14T22:37:37.829792+0000","last_fullsized":"2024-09-14T22:37:37.829792+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:43:56.748832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087783999999999996,"stat_sum":{"num_bytes":21993143,"num_objects":38,"num_object_clones":10,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":13,"num_read":2724,"num_read_kb":13200,"num_write":251,"num_write_kb":4525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"88'309","reported_seq":1717,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.830054+0000","last_change":"2024-09-14T22:36:18.009666+0000","last_active":"2024-09-14T22:37:37.830054+0000","last_peered":"2024-09-14T22:37:37.830054+0000","last_clean":"2024-09-14T22:37:37.830054+0000","last_became_active":"2024-09-14T22:36:15.715659+0000","last_became_peered":"2024-09-14T22:36:15.715659+0000","last_unstale":"2024-09-14T22:37:37.830054+0000","last_undegraded":"2024-09-14T22:37:37.830054+0000","last_fullsized":"2024-09-14T22:37:37.830054+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:26:19.604793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000776448,"stat_sum":{"num_bytes":28271029,"num_objects":35,"num_object_clones":10,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":10,"num_read":2030,"num_read_kb":7936,"num_write":373,"num_write_kb":3474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"88'242","reported_seq":7039,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.888769+0000","last_change":"2024-09-14T22:36:18.010182+0000","last_active":"2024-09-14T22:37:37.888769+0000","last_peered":"2024-09-14T22:37:37.888769+0000","last_clean":"2024-09-14T22:37:37.888769+0000","last_became_active":"2024-09-14T22:36:15.719693+0000","last_became_peered":"2024-09-14T22:36:15.719693+0000","last_unstale":"2024-09-14T22:37:37.888769+0000","last_undegraded":"2024-09-14T22:37:37.888769+0000","last_fullsized":"2024-09-14T22:37:37.888769+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:30:16.781866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001045011,"stat_sum":{"num_bytes":22816458,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":9,"num_read":6947,"num_read_kb":19932,"num_write":313,"num_write_kb":9192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"87'312","reported_seq":3148,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.818551+0000","last_change":"2024-09-14T22:37:26.718624+0000","last_active":"2024-09-14T22:37:37.818551+0000","last_peered":"2024-09-14T22:37:37.818551+0000","last_clean":"2024-09-14T22:37:37.818551+0000","last_became_active":"2024-09-14T22:37:26.717088+0000","last_became_peered":"2024-09-14T22:37:26.717088+0000","last_unstale":"2024-09-14T22:37:37.818551+0000","last_undegraded":"2024-09-14T22:37:37.818551+0000","last_fullsized":"2024-09-14T22:37:37.818551+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:17:13.520881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094227299999999998,"stat_sum":{"num_bytes":35166605,"num_objects":39,"num_object_clones":12,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":14,"num_read":2958,"num_read_kb":24411,"num_write":372,"num_write_kb":12314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"83'249","reported_seq":1720,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.829442+0000","last_change":"2024-09-14T22:36:18.009347+0000","last_active":"2024-09-14T22:37:37.829442+0000","last_peered":"2024-09-14T22:37:37.829442+0000","last_clean":"2024-09-14T22:37:37.829442+0000","last_became_active":"2024-09-14T22:36:15.715569+0000","last_became_peered":"2024-09-14T22:36:15.715569+0000","last_unstale":"2024-09-14T22:37:37.829442+0000","last_undegraded":"2024-09-14T22:37:37.829442+0000","last_fullsized":"2024-09-14T22:37:37.829442+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:54:26.666871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054921600000000005,"stat_sum":{"num_bytes":29773590,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":11,"num_read":1947,"num_read_kb":10189,"num_write":312,"num_write_kb":9971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"86'254","reported_seq":4723,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.828943+0000","last_change":"2024-09-14T22:36:17.720448+0000","last_active":"2024-09-14T22:37:37.828943+0000","last_peered":"2024-09-14T22:37:37.828943+0000","last_clean":"2024-09-14T22:37:37.828943+0000","last_became_active":"2024-09-14T22:36:15.716696+0000","last_became_peered":"2024-09-14T22:36:15.716696+0000","last_unstale":"2024-09-14T22:37:37.828943+0000","last_undegraded":"2024-09-14T22:37:37.828943+0000","last_fullsized":"2024-09-14T22:37:37.828943+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:19:24.905274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086547600000000001,"stat_sum":{"num_bytes":37561236,"num_objects":38,"num_object_clones":9,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":13,"num_read":5092,"num_read_kb":31570,"num_write":238,"num_write_kb":2216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"83'314","reported_seq":5747,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T22:37:37.829447+0000","last_change":"2024-09-14T22:36:18.010088+0000","last_active":"2024-09-14T22:37:37.829447+0000","last_peered":"2024-09-14T22:37:37.829447+0000","last_clean":"2024-09-14T22:37:37.829447+0000","last_became_active":"2024-09-14T22:36:15.715866+0000","last_became_peered":"2024-09-14T22:36:15.715866+0000","last_unstale":"2024-09-14T22:37:37.829447+0000","last_undegraded":"2024-09-14T22:37:37.829447+0000","last_fullsized":"2024-09-14T22:37:37.829447+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:07:34.726674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010286189999999999,"stat_sum":{"num_bytes":49958508,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":11,"num_read":5880,"num_read_kb":32277,"num_write":378,"num_write_kb":14490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1089498010,"num_objects":1157,"num_object_clones":320,"num_object_copies":2314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1157,"num_whiteouts":384,"num_read":149853,"num_read_kb":546327,"num_write":16385,"num_write_kb":254591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":612,"num_bytes_recovered":31733554,"num_keys_recovered":22,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":35880960,"data_stored":351327358,"data_compressed":9973902,"data_compressed_allocated":27844608,"data_compressed_original":346686390,"omap_allocated":4238,"internal_metadata":0},"log_size":12790,"ondisk_log_size":12790,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542167,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33332,"kb_used_data":6304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338508,"statfs":{"total":96636764160,"available":96602632192,"internally_reserved":0,"allocated":6455296,"data_stored":48432814,"data_compressed":1673381,"data_compressed_allocated":4046848,"data_compressed_original":47167472,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542168,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38752,"kb_used_data":3400,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333088,"statfs":{"total":96636764160,"available":96597082112,"internally_reserved":0,"allocated":3481600,"data_stored":29332896,"data_compressed":650841,"data_compressed_allocated":2023424,"data_compressed_original":28628035,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542168,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45596,"kb_used_data":6084,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326244,"statfs":{"total":96636764160,"available":96590073856,"internally_reserved":0,"allocated":6230016,"data_stored":48196278,"data_compressed":1360590,"data_compressed_allocated":3997696,"data_compressed_original":47011982,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542167,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35744,"kb_used_data":8720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336096,"statfs":{"total":96636764160,"available":96600162304,"internally_reserved":0,"allocated":8929280,"data_stored":82168691,"data_compressed":2167670,"data_compressed_allocated":6344704,"data_compressed_original":80818765,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542168,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54040,"kb_used_data":4804,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94317800,"statfs":{"total":96636764160,"available":96581427200,"internally_reserved":0,"allocated":4919296,"data_stored":33891151,"data_compressed":1327754,"data_compressed_allocated":2871296,"data_compressed_original":32827398,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":66,"seq":283467841542,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35872,"kb_used_data":3200,"kb_used_omap":6,"kb_used_meta":32633,"kb_avail":94335968,"statfs":{"total":96636764160,"available":96600031232,"internally_reserved":0,"allocated":3276800,"data_stored":24944558,"data_compressed":455862,"data_compressed_allocated":1794048,"data_compressed_original":24168618,"omap_allocated":7025,"internal_metadata":33416335},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53560,"kb_used_data":4328,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318280,"statfs":{"total":96636764160,"available":96581918720,"internally_reserved":0,"allocated":4431872,"data_stored":32764797,"data_compressed":976741,"data_compressed_allocated":2584576,"data_compressed_original":31815514,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542167,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33312,"kb_used_data":6280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338528,"statfs":{"total":96636764160,"available":96602652672,"internally_reserved":0,"allocated":6430720,"data_stored":56574536,"data_compressed":1357400,"data_compressed_allocated":4247552,"data_compressed_original":55448734,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5353472,"data_stored":55515191,"data_compressed":1351591,"data_compressed_allocated":4210688,"data_compressed_original":54875294,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3391488,"data_stored":32217687,"data_compressed":976670,"data_compressed_allocated":2576384,"data_compressed_original":31737690,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2277376,"data_stored":24487279,"data_compressed":455862,"data_compressed_allocated":1794048,"data_compressed_original":24168618,"omap_allocated":4238,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3899392,"data_stored":33422174,"data_compressed":1327754,"data_compressed_allocated":2871296,"data_compressed_original":32827398,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7913472,"data_stored":81699728,"data_compressed":2167670,"data_compressed_allocated":6344704,"data_compressed_original":80818765,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5201920,"data_stored":47727234,"data_compressed":1360590,"data_compressed_allocated":3997696,"data_compressed_original":47011982,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2482176,"data_stored":28890832,"data_compressed":666193,"data_compressed_allocated":2039808,"data_compressed_original":28652611,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5361664,"data_stored":47367233,"data_compressed":1667572,"data_compressed_allocated":4009984,"data_compressed_original":46594032,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:37:39.545 INFO:tasks.thrashosds.thrasher:Setting 2.0 to [5, 0, 1, 4] 2024-09-14T22:37:39.545 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.0', '5', '0', '1', '4'] 2024-09-14T22:37:39.545 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.0 5 0 1 4 2024-09-14T22:37:39.588 INFO:teuthology.orchestra.run.smithi001.stdout:4675 trunc from 0x1da0625 to 0x91f6ce8 2024-09-14T22:37:39.588 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:39.587+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:39.595 INFO:teuthology.orchestra.run.smithi001.stdout:4676 write 0x88ccbb9 thru 0x88d7a3b (0xae83 bytes) 2024-09-14T22:37:39.604 INFO:teuthology.orchestra.run.smithi001.stdout:4678 writesame 0x1cdd1d8 thru 0x1ceb0e3 data_size 0x32(0xdf0c bytes) 2024-09-14T22:37:39.616 INFO:teuthology.orchestra.run.smithi001.stdout:4679 write 0xa6a9d25 thru 0xa6b72fe (0xd5da bytes) 2024-09-14T22:37:39.627 INFO:teuthology.orchestra.run.smithi001.stdout:4680 writesame 0x241d354 thru 0x242a7b5 data_size 0x5(0xd462 bytes) 2024-09-14T22:37:39.645 INFO:teuthology.orchestra.run.smithi001.stdout:4681 trunc from 0xa6b72ff to 0xec5d7f9 2024-09-14T22:37:39.652 INFO:teuthology.orchestra.run.smithi001.stdout:4682 read 0xcd7bb49 thru 0xcd82546 (0x69fe bytes) 2024-09-14T22:37:39.652 INFO:teuthology.orchestra.run.smithi001.stdout:4683 read 0x72462e5 thru 0x7249c08 (0x3924 bytes) 2024-09-14T22:37:39.652 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:39.652 INFO:teuthology.orchestra.run.smithi001.stdout:4685 writesame 0x3554719 thru 0x3562970 data_size 0x4(0xe258 bytes) 2024-09-14T22:37:39.665 INFO:teuthology.orchestra.run.smithi001.stdout:4686 punch from 0x6ca937f to 0x6cae0ce, (0x4d4f bytes) 2024-09-14T22:37:39.665 INFO:teuthology.orchestra.run.smithi001.stdout:4687 write 0xb2aaf0 thru 0xb2f99d (0x4eae bytes) 2024-09-14T22:37:39.689 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:39.687+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:39.695 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:39.695 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:39.701 INFO:teuthology.orchestra.run.smithi001.stdout:4689 read 0xa627d6f thru 0xa629343 (0x15d5 bytes) 2024-09-14T22:37:39.701 INFO:teuthology.orchestra.run.smithi001.stdout:4693 write 0x53d7c73 thru 0x53e71b1 (0xf53f bytes) 2024-09-14T22:37:39.712 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:37:39.717 INFO:teuthology.orchestra.run.smithi001.stdout:4694 punch from 0x887be02 to 0x887be06, (0x4 bytes) 2024-09-14T22:37:39.717 INFO:teuthology.orchestra.run.smithi001.stdout:4695 write 0x98ca2ec thru 0x98cdc62 (0x3977 bytes) 2024-09-14T22:37:39.728 INFO:teuthology.orchestra.run.smithi001.stdout:4697 punch from 0x4bf8422 to 0x4bfd4ce, (0x50ac bytes) 2024-09-14T22:37:39.728 INFO:teuthology.orchestra.run.smithi001.stdout:4698 writesame 0x170a223 thru 0x171409a data_size 0x2c(0x9e78 bytes) 2024-09-14T22:37:39.739 INFO:teuthology.orchestra.run.smithi001.stdout:4699 read 0x4b422fe thru 0x4b499df (0x76e2 bytes) 2024-09-14T22:37:39.739 INFO:teuthology.orchestra.run.smithi001.stdout:4700 write 0x7d389cb thru 0x7d40ae9 (0x811f bytes) 2024-09-14T22:37:39.752 INFO:teuthology.orchestra.run.smithi001.stdout:4701 writesame 0x31cb6f7 thru 0x31d8db7 data_size 0x1(0xd6c1 bytes) 2024-09-14T22:37:39.771 INFO:teuthology.orchestra.run.smithi001.stdout:4702 write 0x56d61e5 thru 0x56daf1f (0x4d3b bytes) 2024-09-14T22:37:39.784 INFO:teuthology.orchestra.run.smithi001.stdout:4703 write 0x9388322 thru 0x939219b (0x9e7a bytes) 2024-09-14T22:37:39.790 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:39.783+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:39.793 INFO:teuthology.orchestra.run.smithi001.stdout:4704 writesame 0x84460d2 thru 0x8450483 data_size 0x17(0xa3b2 bytes) 2024-09-14T22:37:39.802 INFO:teuthology.orchestra.run.smithi001.stdout:4706 read 0x1629b64 thru 0x1632081 (0x851e bytes) 2024-09-14T22:37:39.803 INFO:teuthology.orchestra.run.smithi001.stdout:4707 writesame 0x5e83b62 thru 0x5e86e9d data_size 0x6(0x333c bytes) 2024-09-14T22:37:39.816 INFO:teuthology.orchestra.run.smithi001.stdout:4710 read 0xa70df9a thru 0xa712dfe (0x4e65 bytes) 2024-09-14T22:37:39.817 INFO:teuthology.orchestra.run.smithi001.stdout:4711 read 0x508a4d thru 0x50a5e4 (0x1b98 bytes) 2024-09-14T22:37:39.818 INFO:teuthology.orchestra.run.smithi001.stdout:4712 write 0x9b9be80 thru 0x9baaee9 (0xf06a bytes) 2024-09-14T22:37:39.830 INFO:teuthology.orchestra.run.smithi001.stdout:4713 write 0x2879cda thru 0x2880661 (0x6988 bytes) 2024-09-14T22:37:39.844 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:39.844 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:39.848 INFO:teuthology.orchestra.run.smithi001.stdout:4714 trunc from 0xec5d7f9 to 0xa19ccaf 2024-09-14T22:37:39.861 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:37:39.864 INFO:teuthology.orchestra.run.smithi001.stdout:4715 read 0x6500aa1 thru 0x6509180 (0x86e0 bytes) 2024-09-14T22:37:39.864 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:39.864 INFO:teuthology.orchestra.run.smithi001.stdout:4717 read 0x8ba56df thru 0x8bb54b9 (0xfddb bytes) 2024-09-14T22:37:39.873 INFO:teuthology.orchestra.run.smithi001.stdout:4718 trunc from 0xa19ccaf to 0x20b131b 2024-09-14T22:37:39.890 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:39.887+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:39.910 INFO:teuthology.orchestra.run.smithi001.stdout:4719 trunc from 0x20b131b to 0x6aa852d 2024-09-14T22:37:39.917 INFO:teuthology.orchestra.run.smithi001.stdout:4722 write 0x31c29a8 thru 0x31cac1c (0x8275 bytes) 2024-09-14T22:37:39.929 INFO:teuthology.orchestra.run.smithi001.stdout:4723 write 0x51e3029 thru 0x51f2d5d (0xfd35 bytes) 2024-09-14T22:37:39.941 INFO:teuthology.orchestra.run.smithi001.stdout:4724 read 0x57b2cb2 thru 0x57bd7ca (0xab19 bytes) 2024-09-14T22:37:39.956 INFO:teuthology.orchestra.run.smithi001.stdout:4727 write 0xe059c4a thru 0xe069357 (0xf70e bytes) 2024-09-14T22:37:39.967 INFO:teuthology.orchestra.run.smithi001.stdout:4728 read 0x71c77de thru 0x71d2a29 (0xb24c bytes) 2024-09-14T22:37:39.967 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:39.968 INFO:teuthology.orchestra.run.smithi001.stdout:4731 read 0x95fc63 thru 0x9695ce (0x996c bytes) 2024-09-14T22:37:39.980 INFO:teuthology.orchestra.run.smithi001.stdout:4736 trunc from 0xe069358 to 0x4f2e83e 2024-09-14T22:37:39.991 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:39.987+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:39.995 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:39.995 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:40.014 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:37:40.015 INFO:teuthology.orchestra.run.smithi001.stdout:4737 write 0xaf0c315 thru 0xaf1607d (0x9d69 bytes) 2024-09-14T22:37:40.030 INFO:teuthology.orchestra.run.smithi001.stdout:4738 writesame 0x30f960c thru 0x30f9d57 data_size 0x2(0x74c bytes) 2024-09-14T22:37:40.032 INFO:teuthology.orchestra.run.smithi001.stdout:4739 writesame 0x29fb2d3 thru 0x2a04344 data_size 0x3(0x9072 bytes) 2024-09-14T22:37:40.049 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:40.050 INFO:teuthology.orchestra.run.smithi001.stdout:4742 read 0x7f9aa16 thru 0x7fa4d69 (0xa354 bytes) 2024-09-14T22:37:40.050 INFO:teuthology.orchestra.run.smithi001.stdout:4743 read 0x4f00fea thru 0x4f055a6 (0x45bd bytes) 2024-09-14T22:37:40.061 INFO:teuthology.orchestra.run.smithi001.stdout:4744 trunc from 0xaf1607e to 0x2b0fdbc 2024-09-14T22:37:40.092 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:40.087+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:40.097 INFO:teuthology.orchestra.run.smithi001.stdout:4745 writesame 0x9f1fe0a thru 0x9f2c84c data_size 0xd(0xca43 bytes) 2024-09-14T22:37:40.110 INFO:teuthology.orchestra.run.smithi001.stdout:4749 writesame 0xd30a027 thru 0xd30afd4 data_size 0x9(0xfae bytes) 2024-09-14T22:37:40.111 INFO:teuthology.orchestra.run.smithi001.stdout:4751 punch from 0x10d0e2d to 0x10d60d7, (0x52aa bytes) 2024-09-14T22:37:40.120 INFO:teuthology.orchestra.run.smithi001.stdout:4752 trunc from 0xd30afd5 to 0x64c2e32 2024-09-14T22:37:40.129 INFO:teuthology.orchestra.run.smithi001.stdout:4755 read 0x4133269 thru 0x413d4ca (0xa262 bytes) 2024-09-14T22:37:40.129 INFO:teuthology.orchestra.run.smithi001.stdout:4756 read 0x11b7372 thru 0x11c09c9 (0x9658 bytes) 2024-09-14T22:37:40.129 INFO:teuthology.orchestra.run.smithi001.stdout:4757 read 0x2fa8b9b thru 0x2faaa72 (0x1ed8 bytes) 2024-09-14T22:37:40.129 INFO:teuthology.orchestra.run.smithi001.stdout:4758 read 0x54af38 thru 0x54f630 (0x46f9 bytes) 2024-09-14T22:37:40.130 INFO:teuthology.orchestra.run.smithi001.stdout:4759 read 0x40c35d7 thru 0x40d16a3 (0xe0cd bytes) 2024-09-14T22:37:40.130 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:40.130 INFO:teuthology.orchestra.run.smithi001.stdout:4761 read 0x3ede52f thru 0x3eeaabc (0xc58e bytes) 2024-09-14T22:37:40.139 INFO:teuthology.orchestra.run.smithi001.stdout:4762 trunc from 0x64c2e32 to 0xae5ad7b 2024-09-14T22:37:40.149 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:40.149 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:40.149 INFO:teuthology.orchestra.run.smithi001.stdout:4763 read 0x126d324 thru 0x12710fa (0x3dd7 bytes) 2024-09-14T22:37:40.149 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:40.166 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:37:40.168 INFO:teuthology.orchestra.run.smithi001.stdout:4767 writesame 0xe59eb28 thru 0xe5a457a data_size 0x1(0x5a53 bytes) 2024-09-14T22:37:40.168 INFO:teuthology.orchestra.run.smithi001.stdout:4768 read 0x9dd2c46 thru 0x9dd6cf9 (0x40b4 bytes) 2024-09-14T22:37:40.183 INFO:teuthology.orchestra.run.smithi001.stdout:4769 trunc from 0xe5a457b to 0xa8576f 2024-09-14T22:37:40.193 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:40.191+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:40.212 INFO:teuthology.orchestra.run.smithi001.stdout:4770 read 0x12152d thru 0x1223ff (0xed3 bytes) 2024-09-14T22:37:40.213 INFO:teuthology.orchestra.run.smithi001.stdout:4771 read 0x1aa122 thru 0x1b5402 (0xb2e1 bytes) 2024-09-14T22:37:40.216 INFO:teuthology.orchestra.run.smithi001.stdout:4772 read 0x233a9c thru 0x24089d (0xce02 bytes) 2024-09-14T22:37:40.232 INFO:teuthology.orchestra.run.smithi001.stdout:4773 write 0x30d6e53 thru 0x30dc1d9 (0x5387 bytes) 2024-09-14T22:37:40.263 INFO:teuthology.orchestra.run.smithi001.stdout:4775 writesame 0x673a6b9 thru 0x673fab7 data_size 0x1(0x53ff bytes) 2024-09-14T22:37:40.263 INFO:teuthology.orchestra.run.smithi001.stdout:4776 punch from 0x447b3bb to 0x447e4b6, (0x30fb bytes) 2024-09-14T22:37:40.263 INFO:teuthology.orchestra.run.smithi001.stdout:4777 read 0x147cdbe thru 0x147ead6 (0x1d19 bytes) 2024-09-14T22:37:40.263 INFO:teuthology.orchestra.run.smithi001.stdout:4778 write 0x2c01adf thru 0x2c01bbd (0xdf bytes) 2024-09-14T22:37:40.289 INFO:teuthology.orchestra.run.smithi001.stdout:4779 write 0x9f94e2a thru 0x9f99b81 (0x4d58 bytes) 2024-09-14T22:37:40.292 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:40.293 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:40.295 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:40.290+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:40.306 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:37:40.308 INFO:teuthology.orchestra.run.smithi001.stdout:4780 write 0x40b90e1 thru 0x40c71cb (0xe0eb bytes) 2024-09-14T22:37:40.345 INFO:teuthology.orchestra.run.smithi001.stdout:4781 write 0xe124e1a thru 0xe134509 (0xf6f0 bytes) 2024-09-14T22:37:40.365 INFO:teuthology.orchestra.run.smithi001.stdout:4784 write 0x769cf0b thru 0x76a4ced (0x7de3 bytes) 2024-09-14T22:37:40.381 INFO:teuthology.orchestra.run.smithi001.stdout:4785 read 0xb21eb4e thru 0xb22c95b (0xde0e bytes) 2024-09-14T22:37:40.381 INFO:teuthology.orchestra.run.smithi001.stdout:4786 read 0x30821e7 thru 0x30828a1 (0x6bb bytes) 2024-09-14T22:37:40.381 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:40.382 INFO:teuthology.orchestra.run.smithi001.stdout:4788 read 0x710611b thru 0x7112493 (0xc379 bytes) 2024-09-14T22:37:40.392 INFO:teuthology.orchestra.run.smithi001.stdout:4789 trunc from 0xe13450a to 0x5ac9807 2024-09-14T22:37:40.395 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:40.390+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:40.415 INFO:teuthology.orchestra.run.smithi001.stdout:4792 write 0x240f62a thru 0x2415db1 (0x6788 bytes) 2024-09-14T22:37:40.436 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:40.436 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:40.448 INFO:teuthology.orchestra.run.smithi001.stdout:4793 writesame 0x9c4654e thru 0x9c4f0b7 data_size 0x2b(0x8b6a bytes) 2024-09-14T22:37:40.448 INFO:teuthology.orchestra.run.smithi001.stdout:4794 write 0x5cbf8af thru 0x5ccd563 (0xdcb5 bytes) 2024-09-14T22:37:40.453 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:37:40.465 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:40.465 INFO:teuthology.orchestra.run.smithi001.stdout:4796 write 0x1cd1e07 thru 0x1cdb5f3 (0x97ed bytes) 2024-09-14T22:37:40.495 INFO:teuthology.orchestra.run.smithi001.stdout:4797 write 0xec0a535 thru 0xec1918e (0xec5a bytes) 2024-09-14T22:37:40.496 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:40.490+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:40.512 INFO:teuthology.orchestra.run.smithi001.stdout:4798 read 0x4eb85c2 thru 0x4ec3e62 (0xb8a1 bytes) 2024-09-14T22:37:40.512 INFO:teuthology.orchestra.run.smithi001.stdout:4800 read 0x47c3b7f thru 0x47c4328 (0x7aa bytes) 2024-09-14T22:37:40.512 INFO:teuthology.orchestra.run.smithi001.stdout:4801 clone 28 order 19 su 65536 sc 15 2024-09-14T22:37:40.586 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:40.586 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:40.597 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:40.595+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:40.600 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:37:40.697 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:40.690+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:40.773 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:40.773 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:40.788 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:37:40.798 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:40.790+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:40.845 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T22:37:40.846 INFO:teuthology.orchestra.run.smithi001.stderr:set 2.0 pg_upmap_items mapping to [5->0,1->4] 2024-09-14T22:37:40.916 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:40.915+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:40.929 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:40.929 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:40.946 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:37:40.951 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone27 from 0xec1918f (overlap 0xa8576f) to 0x9781e3 2024-09-14T22:37:41.000 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:40.994+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:41.090 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:41.090 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:41.101 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:41.094+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:41.106 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:37:41.113 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-14T22:37:41.202 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:41.199+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:41.232 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:41.232 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:41.249 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:37:41.280 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:37:41.340 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:41.339+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:41.404 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:41.403+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:41.418 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:41.419 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:41.435 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:37:41.436 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:41.436 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:41.436 INFO:teuthology.orchestra.run.smithi001.stdout:4805 punch from 0x92e0908 to 0x92e5c53, (0x534b bytes) 2024-09-14T22:37:41.438 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:37:41.438 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:37:41.442 INFO:teuthology.orchestra.run.smithi001.stdout:4807 writesame 0xa88a1a0 thru 0xa8966ed data_size 0x5(0xc54e bytes) 2024-09-14T22:37:41.452 INFO:teuthology.orchestra.run.smithi001.stdout:4808 read 0x606c5c9 thru 0x6079e75 (0xd8ad bytes) 2024-09-14T22:37:41.452 INFO:teuthology.orchestra.run.smithi001.stdout:4809 read 0x37e8452 thru 0x37f1540 (0x90ef bytes) 2024-09-14T22:37:41.452 INFO:teuthology.orchestra.run.smithi001.stdout:4810 trunc from 0xec1918f to 0xe550a7b 2024-09-14T22:37:41.460 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:37:41.461 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:37:41.483 INFO:teuthology.orchestra.run.smithi001.stdout:4811 write 0xd593819 thru 0xd59cd8e (0x9576 bytes) 2024-09-14T22:37:41.489 INFO:teuthology.orchestra.run.smithi001.stdout:4812 write 0x53f5181 thru 0x53fda29 (0x88a9 bytes) 2024-09-14T22:37:41.494 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:37:41.495 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:37:41.495 INFO:teuthology.orchestra.run.smithi001.stdout:4813 read 0x411b275 thru 0x411ddec (0x2b78 bytes) 2024-09-14T22:37:41.496 INFO:teuthology.orchestra.run.smithi001.stdout:4815 trunc from 0xe550a7b to 0xd77995b 2024-09-14T22:37:41.505 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:41.503+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:41.520 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:37:41.520 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:37:41.535 INFO:teuthology.orchestra.run.smithi001.stdout:4817 punch from 0x62717fc to 0x627d990, (0xc194 bytes) 2024-09-14T22:37:41.536 INFO:teuthology.orchestra.run.smithi001.stdout:4818 read 0x44f2298 thru 0x4500343 (0xe0ac bytes) 2024-09-14T22:37:41.536 INFO:teuthology.orchestra.run.smithi001.stdout:4819 writesame 0xc556d6a thru 0xc5596fb data_size 0x11(0x2992 bytes) 2024-09-14T22:37:41.542 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:41.542 INFO:teuthology.orchestra.run.smithi001.stdout:4821 writesame 0x323f90c thru 0x3245214 data_size 0x1(0x5909 bytes) 2024-09-14T22:37:41.545 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:37:41.546 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:37:41.568 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:37:41.568 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:37:41.578 INFO:teuthology.orchestra.run.smithi001.stdout:4822 punch from 0x85fe691 to 0x860e191, (0xfb00 bytes) 2024-09-14T22:37:41.578 INFO:teuthology.orchestra.run.smithi001.stdout:4823 read 0x5b952c7 thru 0x5ba4d27 (0xfa61 bytes) 2024-09-14T22:37:41.578 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:41.583 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:41.584 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:37:41.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:37:41.584 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:37:41.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.584 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35306 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1425408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:41.584 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507069+0000", 2024-09-14T22:37:41.584 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.076323957, 2024-09-14T22:37:41.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045725259999999997, 2024-09-14T22:37:41.584 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:41.584 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:41.585 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:41.585 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:41.585 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:41.585 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:41.585 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35306 2024-09-14T22:37:41.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.585 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:41.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:41.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507069+0000", 2024-09-14T22:37:41.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:41.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507069+0000", 2024-09-14T22:37:41.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:41.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507070+0000", 2024-09-14T22:37:41.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0400000000000002e-07 2024-09-14T22:37:41.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:41.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507072+0000", 2024-09-14T22:37:41.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.891e-06 2024-09-14T22:37:41.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:41.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507078+0000", 2024-09-14T22:37:41.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0279999999999997e-06 2024-09-14T22:37:41.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:41.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507082+0000", 2024-09-14T22:37:41.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4680000000000003e-06 2024-09-14T22:37:41.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:41.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552632+0000", 2024-09-14T22:37:41.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045550144000000001 2024-09-14T22:37:41.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:41.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552707+0000", 2024-09-14T22:37:41.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4148999999999998e-05 2024-09-14T22:37:41.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:41.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552795+0000", 2024-09-14T22:37:41.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8176000000000004e-05 2024-09-14T22:37:41.589 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.589 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:41.589 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.589 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35307 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1454080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:41.589 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:37:41.589 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.076305406000003, 2024-09-14T22:37:41.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045891358, 2024-09-14T22:37:41.590 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:41.590 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:41.590 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:41.590 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:41.590 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:41.590 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:41.590 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35307 2024-09-14T22:37:41.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.590 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:41.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:41.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:37:41.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:41.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:37:41.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:41.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507088+0000", 2024-09-14T22:37:41.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4700000000000002e-07 2024-09-14T22:37:41.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:41.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507093+0000", 2024-09-14T22:37:41.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9520000000000002e-06 2024-09-14T22:37:41.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:41.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507099+0000", 2024-09-14T22:37:41.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0240000000000003e-06 2024-09-14T22:37:41.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:41.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507105+0000", 2024-09-14T22:37:41.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5520000000000004e-06 2024-09-14T22:37:41.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:41.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552826+0000", 2024-09-14T22:37:41.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045721494000000001 2024-09-14T22:37:41.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:41.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552896+0000", 2024-09-14T22:37:41.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9519999999999998e-05 2024-09-14T22:37:41.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:41.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.552979+0000", 2024-09-14T22:37:41.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3468999999999997e-05 2024-09-14T22:37:41.594 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.594 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:41.594 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.594 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35308 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1482752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:41.594 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:37:41.594 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.076282410000005, 2024-09-14T22:37:41.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046076533000000003, 2024-09-14T22:37:41.594 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:41.594 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:41.594 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:41.595 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:41.595 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:41.595 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:41.595 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35308 2024-09-14T22:37:41.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.595 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:41.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:41.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:37:41.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:41.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:37:41.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:41.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507111+0000", 2024-09-14T22:37:41.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:37:41.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:41.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507113+0000", 2024-09-14T22:37:41.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9099999999999999e-06 2024-09-14T22:37:41.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:41.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507119+0000", 2024-09-14T22:37:41.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7440000000000004e-06 2024-09-14T22:37:41.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:41.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507124+0000", 2024-09-14T22:37:41.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2129999999999999e-06 2024-09-14T22:37:41.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:41.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553007+0000", 2024-09-14T22:37:41.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045882488999999999 2024-09-14T22:37:41.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:41.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553076+0000", 2024-09-14T22:37:41.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9233000000000006e-05 2024-09-14T22:37:41.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:41.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553188+0000", 2024-09-14T22:37:41.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111557 2024-09-14T22:37:41.598 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.600 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:41.600 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.600 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35309 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1511424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:41.600 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:37:41.600 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.076263505, 2024-09-14T22:37:41.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046231869000000002, 2024-09-14T22:37:41.601 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:41.601 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:41.601 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:41.601 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:41.601 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:41.601 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:41.601 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35309 2024-09-14T22:37:41.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.601 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:41.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:41.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:37:41.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:41.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:37:41.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:41.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507130+0000", 2024-09-14T22:37:41.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9099999999999999e-07 2024-09-14T22:37:41.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:41.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507132+0000", 2024-09-14T22:37:41.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8190000000000001e-06 2024-09-14T22:37:41.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:41.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507139+0000", 2024-09-14T22:37:41.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9419999999999998e-06 2024-09-14T22:37:41.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:41.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507145+0000", 2024-09-14T22:37:41.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2600000000000002e-06 2024-09-14T22:37:41.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:41.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553225+0000", 2024-09-14T22:37:41.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046079991000000001 2024-09-14T22:37:41.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:41.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553284+0000", 2024-09-14T22:37:41.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8638999999999999e-05 2024-09-14T22:37:41.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:41.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553362+0000", 2024-09-14T22:37:41.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7826999999999999e-05 2024-09-14T22:37:41.605 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.605 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:41.605 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.606 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35310 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1540096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:41.606 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507151+0000", 2024-09-14T22:37:41.606 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.076242049000001, 2024-09-14T22:37:41.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046444541999999998, 2024-09-14T22:37:41.606 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:41.606 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:41.606 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:41.606 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:41.606 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:41.606 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:41.606 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35310 2024-09-14T22:37:41.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.607 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:41.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:41.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507151+0000", 2024-09-14T22:37:41.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:41.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507151+0000", 2024-09-14T22:37:41.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:41.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507152+0000", 2024-09-14T22:37:41.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1699999999999999e-07 2024-09-14T22:37:41.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:41.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507154+0000", 2024-09-14T22:37:41.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.902e-06 2024-09-14T22:37:41.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:41.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507159+0000", 2024-09-14T22:37:41.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.823e-06 2024-09-14T22:37:41.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:41.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507165+0000", 2024-09-14T22:37:41.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.587e-06 2024-09-14T22:37:41.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:41.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553395+0000", 2024-09-14T22:37:41.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046230318999999999 2024-09-14T22:37:41.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:41.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553434+0000", 2024-09-14T22:37:41.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8680999999999998e-05 2024-09-14T22:37:41.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:41.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553596+0000", 2024-09-14T22:37:41.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016191299999999999 2024-09-14T22:37:41.610 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.610 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:41.610 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.610 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35311 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1568768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:41.610 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:37:41.610 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.076222795999996, 2024-09-14T22:37:41.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046623592999999998, 2024-09-14T22:37:41.611 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:41.611 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:41.611 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:41.611 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:41.611 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:41.611 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:41.611 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35311 2024-09-14T22:37:41.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.611 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:41.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:41.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:37:41.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:41.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:37:41.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:41.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507171+0000", 2024-09-14T22:37:41.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:37:41.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:41.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507176+0000", 2024-09-14T22:37:41.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3349999999999997e-06 2024-09-14T22:37:41.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:41.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507200+0000", 2024-09-14T22:37:41.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4003e-05 2024-09-14T22:37:41.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:41.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.507206+0000", 2024-09-14T22:37:41.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.976e-06 2024-09-14T22:37:41.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:41.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553627+0000", 2024-09-14T22:37:41.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046420284999999999 2024-09-14T22:37:41.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:41.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553696+0000", 2024-09-14T22:37:41.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9567999999999995e-05 2024-09-14T22:37:41.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:41.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.553794+0000", 2024-09-14T22:37:41.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8060000000000006e-05 2024-09-14T22:37:41.615 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.615 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:41.615 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.615 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35566 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1484800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:41.615 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514668+0000", 2024-09-14T22:37:41.615 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.068725018999999, 2024-09-14T22:37:41.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045644974999999997, 2024-09-14T22:37:41.615 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:41.615 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:41.615 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:41.615 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:41.615 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:41.616 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:41.616 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35566 2024-09-14T22:37:41.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.616 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:41.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.616 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:41.616 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514668+0000", 2024-09-14T22:37:41.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:41.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514668+0000", 2024-09-14T22:37:41.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:41.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514669+0000", 2024-09-14T22:37:41.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9099999999999999e-07 2024-09-14T22:37:41.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:41.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514671+0000", 2024-09-14T22:37:41.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3800000000000001e-06 2024-09-14T22:37:41.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:41.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514677+0000", 2024-09-14T22:37:41.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9660000000000001e-06 2024-09-14T22:37:41.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:41.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514682+0000", 2024-09-14T22:37:41.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0939999999999998e-06 2024-09-14T22:37:41.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:41.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560133+0000", 2024-09-14T22:37:41.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045450398000000003 2024-09-14T22:37:41.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:41.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560207+0000", 2024-09-14T22:37:41.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4196999999999995e-05 2024-09-14T22:37:41.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:41.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560313+0000", 2024-09-14T22:37:41.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106549 2024-09-14T22:37:41.620 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.620 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:41.620 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.620 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35567 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1513472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:41.620 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:37:41.620 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.068704479999994, 2024-09-14T22:37:41.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045793215999999998, 2024-09-14T22:37:41.620 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:41.620 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:41.620 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:41.621 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:41.621 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:41.621 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:41.621 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35567 2024-09-14T22:37:41.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.621 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:41.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:41.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:37:41.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:41.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:37:41.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:41.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514689+0000", 2024-09-14T22:37:41.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1899999999999998e-07 2024-09-14T22:37:41.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:41.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514692+0000", 2024-09-14T22:37:41.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2699999999999999e-06 2024-09-14T22:37:41.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:41.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514698+0000", 2024-09-14T22:37:41.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0939999999999996e-06 2024-09-14T22:37:41.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:41.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514703+0000", 2024-09-14T22:37:41.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3870000000000003e-06 2024-09-14T22:37:41.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:41.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560353+0000", 2024-09-14T22:37:41.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045649914 2024-09-14T22:37:41.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:41.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560416+0000", 2024-09-14T22:37:41.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3096999999999996e-05 2024-09-14T22:37:41.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:41.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560482+0000", 2024-09-14T22:37:41.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6035e-05 2024-09-14T22:37:41.624 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.624 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:41.624 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.625 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35568 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1542144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:41.625 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:37:41.625 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.068679681999996, 2024-09-14T22:37:41.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045995921000000002, 2024-09-14T22:37:41.625 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:41.625 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:41.625 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:41.625 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:41.625 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:41.625 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:41.625 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35568 2024-09-14T22:37:41.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.626 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:41.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:41.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:37:41.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:41.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:37:41.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:41.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514714+0000", 2024-09-14T22:37:41.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4099999999999999e-07 2024-09-14T22:37:41.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:41.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514720+0000", 2024-09-14T22:37:41.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6869999999999999e-06 2024-09-14T22:37:41.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:41.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514727+0000", 2024-09-14T22:37:41.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3540000000000002e-06 2024-09-14T22:37:41.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:41.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514732+0000", 2024-09-14T22:37:41.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7690000000000004e-06 2024-09-14T22:37:41.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:41.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560523+0000", 2024-09-14T22:37:41.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045791373000000003 2024-09-14T22:37:41.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:41.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560606+0000", 2024-09-14T22:37:41.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2690999999999998e-05 2024-09-14T22:37:41.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.629 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:41.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560710+0000", 2024-09-14T22:37:41.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000103606 2024-09-14T22:37:41.629 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.629 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:41.629 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.629 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:35569 3.8 3:12e9cfac:::rbd_data.113cda0183a0.0000000000000002:6 [read 1570816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:37:41.629 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:37:41.629 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.068653873000002, 2024-09-14T22:37:41.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046110630999999999, 2024-09-14T22:37:41.629 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:41.629 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:41.630 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:41.630 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:41.630 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:41.630 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:41.630 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 35569 2024-09-14T22:37:41.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.630 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:41.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:41.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:37:41.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:41.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:37:41.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:41.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514740+0000", 2024-09-14T22:37:41.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.82e-07 2024-09-14T22:37:41.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:41.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514743+0000", 2024-09-14T22:37:41.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.269e-06 2024-09-14T22:37:41.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:41.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514750+0000", 2024-09-14T22:37:41.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7340000000000002e-06 2024-09-14T22:37:41.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:41.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.514755+0000", 2024-09-14T22:37:41.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7260000000000002e-06 2024-09-14T22:37:41.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:41.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560731+0000", 2024-09-14T22:37:41.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045976377999999998 2024-09-14T22:37:41.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:41.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560777+0000", 2024-09-14T22:37:41.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5868999999999997e-05 2024-09-14T22:37:41.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:41.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.560850+0000", 2024-09-14T22:37:41.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3172999999999999e-05 2024-09-14T22:37:41.633 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.633 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:41.633 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.634 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:86180 3.18 3:1d0c52d8:::rbd_data.113cd62ea8a5.000000000000004b:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-14T22:37:41.634 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:01.356417+0000", 2024-09-14T22:37:41.634 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 40.226976778000001, 2024-09-14T22:37:41.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049143245000000002, 2024-09-14T22:37:41.634 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:41.634 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:41.634 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:41.634 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:41.634 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:41.634 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:41.634 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 86180 2024-09-14T22:37:41.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.634 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:41.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:41.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356417+0000", 2024-09-14T22:37:41.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:41.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356417+0000", 2024-09-14T22:37:41.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:41.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356420+0000", 2024-09-14T22:37:41.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2030000000000001e-06 2024-09-14T22:37:41.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:41.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356435+0000", 2024-09-14T22:37:41.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5287000000000001e-05 2024-09-14T22:37:41.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:41.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356472+0000", 2024-09-14T22:37:41.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7311000000000002e-05 2024-09-14T22:37:41.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:41.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356492+0000", 2024-09-14T22:37:41.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9519999999999999e-05 2024-09-14T22:37:41.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:41.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356670+0000", 2024-09-14T22:37:41.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000177567 2024-09-14T22:37:41.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:41.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.356924+0000", 2024-09-14T22:37:41.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025399299999999999 2024-09-14T22:37:41.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 0", 2024-09-14T22:37:41.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.365052+0000", 2024-09-14T22:37:41.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0081289380000000005 2024-09-14T22:37:41.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:41.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.405396+0000", 2024-09-14T22:37:41.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040343228000000002 2024-09-14T22:37:41.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:37:41.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.405421+0000", 2024-09-14T22:37:41.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5763e-05 2024-09-14T22:37:41.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:37:41.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.405491+0000", 2024-09-14T22:37:41.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9721999999999994e-05 2024-09-14T22:37:41.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:41.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:01.405560+0000", 2024-09-14T22:37:41.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8713000000000004e-05 2024-09-14T22:37:41.639 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.640 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:41.640 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.640 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98378 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 919552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:41.640 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148183+0000", 2024-09-14T22:37:41.640 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.4352105, 2024-09-14T22:37:41.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045614549999999997, 2024-09-14T22:37:41.640 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:41.640 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:41.640 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:41.640 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:41.640 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:41.640 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:41.641 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98378 2024-09-14T22:37:41.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.641 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:41.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:41.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148183+0000", 2024-09-14T22:37:41.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:41.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148183+0000", 2024-09-14T22:37:41.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:41.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148183+0000", 2024-09-14T22:37:41.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0800000000000001e-07 2024-09-14T22:37:41.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:41.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148184+0000", 2024-09-14T22:37:41.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2780000000000001e-06 2024-09-14T22:37:41.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:41.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148189+0000", 2024-09-14T22:37:41.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2699999999999998e-06 2024-09-14T22:37:41.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:41.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148191+0000", 2024-09-14T22:37:41.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7659999999999999e-06 2024-09-14T22:37:41.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:41.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193708+0000", 2024-09-14T22:37:41.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045516562000000003 2024-09-14T22:37:41.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:41.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193751+0000", 2024-09-14T22:37:41.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2811000000000001e-05 2024-09-14T22:37:41.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.644 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:41.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193797+0000", 2024-09-14T22:37:41.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6655e-05 2024-09-14T22:37:41.644 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.644 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:41.644 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.644 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98379 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 928768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:41.644 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148195+0000", 2024-09-14T22:37:41.644 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.435198670999998, 2024-09-14T22:37:41.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045817405999999998, 2024-09-14T22:37:41.645 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:41.645 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:41.645 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:41.645 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:41.645 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:41.645 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:41.645 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98379 2024-09-14T22:37:41.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.645 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:41.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:41.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148195+0000", 2024-09-14T22:37:41.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:41.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148195+0000", 2024-09-14T22:37:41.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:41.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148195+0000", 2024-09-14T22:37:41.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3999999999999998e-07 2024-09-14T22:37:41.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:41.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148196+0000", 2024-09-14T22:37:41.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.341e-06 2024-09-14T22:37:41.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:41.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148200+0000", 2024-09-14T22:37:41.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9040000000000002e-06 2024-09-14T22:37:41.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:41.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148203+0000", 2024-09-14T22:37:41.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.959e-06 2024-09-14T22:37:41.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:41.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193841+0000", 2024-09-14T22:37:41.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045637886000000003 2024-09-14T22:37:41.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:41.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.193933+0000", 2024-09-14T22:37:41.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1823999999999994e-05 2024-09-14T22:37:41.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:41.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194012+0000", 2024-09-14T22:37:41.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9252000000000006e-05 2024-09-14T22:37:41.649 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.649 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:41.649 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.649 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98380 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 937984~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:41.649 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148206+0000", 2024-09-14T22:37:41.649 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.435187032999998, 2024-09-14T22:37:41.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045937354, 2024-09-14T22:37:41.649 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:41.649 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:41.649 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:41.649 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:41.650 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:41.650 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:41.650 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98380 2024-09-14T22:37:41.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.650 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:41.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:41.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148206+0000", 2024-09-14T22:37:41.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:41.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148206+0000", 2024-09-14T22:37:41.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:41.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148207+0000", 2024-09-14T22:37:41.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1e-07 2024-09-14T22:37:41.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:41.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148221+0000", 2024-09-14T22:37:41.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4679e-05 2024-09-14T22:37:41.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:41.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148228+0000", 2024-09-14T22:37:41.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3820000000000001e-06 2024-09-14T22:37:41.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:41.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148231+0000", 2024-09-14T22:37:41.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2849999999999999e-06 2024-09-14T22:37:41.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:41.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194054+0000", 2024-09-14T22:37:41.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045822596 2024-09-14T22:37:41.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:41.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194098+0000", 2024-09-14T22:37:41.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4604000000000002e-05 2024-09-14T22:37:41.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:41.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194144+0000", 2024-09-14T22:37:41.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5597999999999999e-05 2024-09-14T22:37:41.654 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.654 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:41.654 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.654 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98381 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 947200~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:41.654 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148236+0000", 2024-09-14T22:37:41.654 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.435157539999999, 2024-09-14T22:37:41.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046111129000000001, 2024-09-14T22:37:41.654 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:41.654 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:41.654 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:41.655 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:41.655 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:41.655 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:41.655 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98381 2024-09-14T22:37:41.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.655 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:41.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:41.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148236+0000", 2024-09-14T22:37:41.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:41.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148236+0000", 2024-09-14T22:37:41.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:41.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148236+0000", 2024-09-14T22:37:41.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8799999999999998e-07 2024-09-14T22:37:41.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:41.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148238+0000", 2024-09-14T22:37:41.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6559999999999999e-06 2024-09-14T22:37:41.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:41.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148246+0000", 2024-09-14T22:37:41.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3440000000000001e-06 2024-09-14T22:37:41.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:41.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148251+0000", 2024-09-14T22:37:41.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5480000000000003e-06 2024-09-14T22:37:41.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:41.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194187+0000", 2024-09-14T22:37:41.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045935847000000002 2024-09-14T22:37:41.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:41.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194267+0000", 2024-09-14T22:37:41.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0267999999999998e-05 2024-09-14T22:37:41.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:41.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194347+0000", 2024-09-14T22:37:41.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0178000000000004e-05 2024-09-14T22:37:41.658 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.659 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:41.659 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.659 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98382 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 956416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:41.659 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148254+0000", 2024-09-14T22:37:41.659 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.435139167999999, 2024-09-14T22:37:41.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046224227, 2024-09-14T22:37:41.659 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:41.659 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:41.659 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:41.659 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:41.659 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:41.659 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:41.659 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98382 2024-09-14T22:37:41.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.660 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:41.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:41.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148254+0000", 2024-09-14T22:37:41.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:41.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148254+0000", 2024-09-14T22:37:41.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:41.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148254+0000", 2024-09-14T22:37:41.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3099999999999999e-07 2024-09-14T22:37:41.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:41.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148256+0000", 2024-09-14T22:37:41.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5540000000000001e-06 2024-09-14T22:37:41.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:41.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148259+0000", 2024-09-14T22:37:41.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2499999999999998e-06 2024-09-14T22:37:41.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:41.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148262+0000", 2024-09-14T22:37:41.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8310000000000001e-06 2024-09-14T22:37:41.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:41.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194387+0000", 2024-09-14T22:37:41.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046125082999999997 2024-09-14T22:37:41.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:41.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194430+0000", 2024-09-14T22:37:41.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3074999999999997e-05 2024-09-14T22:37:41.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:41.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194478+0000", 2024-09-14T22:37:41.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8202999999999999e-05 2024-09-14T22:37:41.663 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.663 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:41.663 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.663 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98383 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 965632~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:41.663 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148265+0000", 2024-09-14T22:37:41.663 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.435128054, 2024-09-14T22:37:41.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046409141000000001, 2024-09-14T22:37:41.664 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:41.664 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:41.664 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:41.664 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:41.664 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:41.664 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:41.664 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98383 2024-09-14T22:37:41.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.664 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:41.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:41.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148265+0000", 2024-09-14T22:37:41.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:41.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148265+0000", 2024-09-14T22:37:41.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:41.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148266+0000", 2024-09-14T22:37:41.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0800000000000001e-07 2024-09-14T22:37:41.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:41.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148267+0000", 2024-09-14T22:37:41.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1659999999999999e-06 2024-09-14T22:37:41.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.666 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:41.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148271+0000", 2024-09-14T22:37:41.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1289999999999998e-06 2024-09-14T22:37:41.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.666 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:41.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148274+0000", 2024-09-14T22:37:41.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5939999999999998e-06 2024-09-14T22:37:41.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:41.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194523+0000", 2024-09-14T22:37:41.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046248162000000002 2024-09-14T22:37:41.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:41.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194603+0000", 2024-09-14T22:37:41.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0161999999999996e-05 2024-09-14T22:37:41.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:41.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194675+0000", 2024-09-14T22:37:41.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1719999999999998e-05 2024-09-14T22:37:41.668 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.668 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:41.668 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.668 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98384 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 974848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:41.668 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.148278+0000", 2024-09-14T22:37:41.668 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.435115725999999, 2024-09-14T22:37:41.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046527369999999998, 2024-09-14T22:37:41.668 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:41.668 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:41.668 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:41.668 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:41.669 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:41.669 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:41.669 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98384 2024-09-14T22:37:41.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.669 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:41.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:41.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148278+0000", 2024-09-14T22:37:41.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:41.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.148278+0000", 2024-09-14T22:37:41.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:41.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.149987+0000", 2024-09-14T22:37:41.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017093430000000001 2024-09-14T22:37:41.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:41.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.149993+0000", 2024-09-14T22:37:41.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6219999999999997e-06 2024-09-14T22:37:41.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:41.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150006+0000", 2024-09-14T22:37:41.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3095000000000001e-05 2024-09-14T22:37:41.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:41.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150014+0000", 2024-09-14T22:37:41.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6340000000000007e-06 2024-09-14T22:37:41.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:41.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194715+0000", 2024-09-14T22:37:41.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044700349 2024-09-14T22:37:41.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:41.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194759+0000", 2024-09-14T22:37:41.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3807000000000003e-05 2024-09-14T22:37:41.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:41.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.194805+0000", 2024-09-14T22:37:41.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6520000000000002e-05 2024-09-14T22:37:41.672 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.672 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:41.672 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.672 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98390 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 1030144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:41.672 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.150152+0000", 2024-09-14T22:37:41.673 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.433241038999999, 2024-09-14T22:37:41.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045642708999999997, 2024-09-14T22:37:41.673 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:41.673 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:41.673 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:41.673 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:41.673 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:41.673 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:41.673 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98390 2024-09-14T22:37:41.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.673 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:41.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:41.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150152+0000", 2024-09-14T22:37:41.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:41.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150152+0000", 2024-09-14T22:37:41.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:41.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150153+0000", 2024-09-14T22:37:41.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.08e-07 2024-09-14T22:37:41.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:41.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150155+0000", 2024-09-14T22:37:41.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2910000000000002e-06 2024-09-14T22:37:41.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:41.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150162+0000", 2024-09-14T22:37:41.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4099999999999996e-06 2024-09-14T22:37:41.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:41.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150167+0000", 2024-09-14T22:37:41.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3199999999999999e-06 2024-09-14T22:37:41.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:41.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195704+0000", 2024-09-14T22:37:41.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045537243999999998 2024-09-14T22:37:41.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:41.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195749+0000", 2024-09-14T22:37:41.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4866e-05 2024-09-14T22:37:41.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:41.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195795+0000", 2024-09-14T22:37:41.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.617e-05 2024-09-14T22:37:41.676 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.677 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:41.677 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.677 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:98391 3.8 3:1147016a:::rbd_data.113c5a3f285d.0000000000000000:15 [read 1039360~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:41.677 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:10.150175+0000", 2024-09-14T22:37:41.677 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.433218202999999, 2024-09-14T22:37:41.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045825232, 2024-09-14T22:37:41.677 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:37:41.677 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:37:41.677 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:37:41.677 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:37:41.677 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:37:41.678 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:41.678 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 98391 2024-09-14T22:37:41.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.678 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:37:41.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:37:41.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150175+0000", 2024-09-14T22:37:41.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:37:41.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150175+0000", 2024-09-14T22:37:41.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:37:41.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:37:41.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150176+0000", 2024-09-14T22:37:41.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9700000000000002e-07 2024-09-14T22:37:41.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:37:41.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150178+0000", 2024-09-14T22:37:41.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.407e-06 2024-09-14T22:37:41.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:37:41.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150185+0000", 2024-09-14T22:37:41.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5189999999999998e-06 2024-09-14T22:37:41.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:37:41.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.150190+0000", 2024-09-14T22:37:41.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2399999999999998e-06 2024-09-14T22:37:41.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:37:41.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195838+0000", 2024-09-14T22:37:41.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045648265 2024-09-14T22:37:41.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:37:41.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.195918+0000", 2024-09-14T22:37:41.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0094999999999998e-05 2024-09-14T22:37:41.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:41.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:41.681 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:37:41.681 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:10.196000+0000", 2024-09-14T22:37:41.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2309000000000001e-05 2024-09-14T22:37:41.681 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.681 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:41.681 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.681 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:41.681 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:41.681 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:41.682 INFO:teuthology.orchestra.run.smithi001.stdout:4825 trunc from 0xd77995b to 0x75e312d 2024-09-14T22:37:41.682 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:37:41.683 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:37:41.684 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:37:41.684 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:37:41.684 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:37:41.685 INFO:teuthology.orchestra.run.smithi001.stdout:4826 clone 29 order 20 su 16384 sc 11 2024-09-14T22:37:41.685 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:41.679+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:41.784 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:41.783+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:41.817 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:41.817 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:41.817 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:37:41.817 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:41.830 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:37:41.847 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2024-09-14T22:37:41.885 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:41.878+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:41.957 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:41.958 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:41.958 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:37:41.958 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:37:41.958 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:41.972 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:37:41.988 INFO:teuthology.orchestra.run.smithi001.stdout:flattening image image_client.0-clone28 2024-09-14T22:37:42.019 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:42.014+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:42.088 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:42.083+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:42.161 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:42.162 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:42.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:42.162 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:42.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.162 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6395 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8386560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:42.162 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:37:42.162 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 78.224750622000002, 2024-09-14T22:37:42.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054534379000000001, 2024-09-14T22:37:42.162 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.162 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.162 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.162 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.162 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.163 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.163 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6395 2024-09-14T22:37:42.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.163 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:37:42.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:37:42.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933833+0000", 2024-09-14T22:37:42.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3859999999999999e-06 2024-09-14T22:37:42.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933838+0000", 2024-09-14T22:37:42.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8709999999999996e-06 2024-09-14T22:37:42.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933854+0000", 2024-09-14T22:37:42.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6189e-05 2024-09-14T22:37:42.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933871+0000", 2024-09-14T22:37:42.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.626e-05 2024-09-14T22:37:42.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988198+0000", 2024-09-14T22:37:42.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054327581999999999 2024-09-14T22:37:42.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988274+0000", 2024-09-14T22:37:42.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5759000000000004e-05 2024-09-14T22:37:42.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988366+0000", 2024-09-14T22:37:42.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2331999999999996e-05 2024-09-14T22:37:42.166 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.166 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.166 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.166 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104944 3.3 3:c3780f3b:::rbd_data.113ca561f7a9.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:42.166 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:37:42.167 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.125148749000001, 2024-09-14T22:37:42.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073041087000000005, 2024-09-14T22:37:42.167 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.167 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.167 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:42.167 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.167 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.167 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.167 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104944 2024-09-14T22:37:42.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.167 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:37:42.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:37:42.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033436+0000", 2024-09-14T22:37:42.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2060000000000001e-06 2024-09-14T22:37:42.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033448+0000", 2024-09-14T22:37:42.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.169e-05 2024-09-14T22:37:42.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033474+0000", 2024-09-14T22:37:42.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6400000000000001e-05 2024-09-14T22:37:42.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033489+0000", 2024-09-14T22:37:42.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5067e-05 2024-09-14T22:37:42.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033658+0000", 2024-09-14T22:37:42.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000168565 2024-09-14T22:37:42.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033793+0000", 2024-09-14T22:37:42.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013536900000000001 2024-09-14T22:37:42.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:37:42.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.047559+0000", 2024-09-14T22:37:42.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013766294 2024-09-14T22:37:42.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:42.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106157+0000", 2024-09-14T22:37:42.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058597110000000001 2024-09-14T22:37:42.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:42.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106290+0000", 2024-09-14T22:37:42.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000133244 2024-09-14T22:37:42.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:42.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106396+0000", 2024-09-14T22:37:42.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106246 2024-09-14T22:37:42.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106475+0000", 2024-09-14T22:37:42.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8895999999999998e-05 2024-09-14T22:37:42.172 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.172 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.172 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.172 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:111858 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 10403840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:42.173 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:37:42.173 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.334695628, 2024-09-14T22:37:42.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054398798999999998, 2024-09-14T22:37:42.173 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.173 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.173 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.173 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.173 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.173 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.173 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 111858 2024-09-14T22:37:42.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.173 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:37:42.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:37:42.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823888+0000", 2024-09-14T22:37:42.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0200000000000001e-07 2024-09-14T22:37:42.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823927+0000", 2024-09-14T22:37:42.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8940999999999999e-05 2024-09-14T22:37:42.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823940+0000", 2024-09-14T22:37:42.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3682e-05 2024-09-14T22:37:42.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823950+0000", 2024-09-14T22:37:42.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7110000000000007e-06 2024-09-14T22:37:42.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878202+0000", 2024-09-14T22:37:42.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054251969999999997 2024-09-14T22:37:42.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878249+0000", 2024-09-14T22:37:42.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6854999999999998e-05 2024-09-14T22:37:42.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878286+0000", 2024-09-14T22:37:42.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6838000000000002e-05 2024-09-14T22:37:42.177 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.177 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.177 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.177 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:111859 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 10452992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:42.177 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:37:42.177 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.334606509, 2024-09-14T22:37:42.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054435292000000003, 2024-09-14T22:37:42.177 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.177 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.178 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.178 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.178 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.178 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.178 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 111859 2024-09-14T22:37:42.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.178 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:37:42.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:37:42.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823977+0000", 2024-09-14T22:37:42.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3500000000000005e-07 2024-09-14T22:37:42.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823981+0000", 2024-09-14T22:37:42.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2050000000000004e-06 2024-09-14T22:37:42.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824007+0000", 2024-09-14T22:37:42.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.62e-05 2024-09-14T22:37:42.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824018+0000", 2024-09-14T22:37:42.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0389e-05 2024-09-14T22:37:42.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878312+0000", 2024-09-14T22:37:42.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054294754000000001 2024-09-14T22:37:42.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878350+0000", 2024-09-14T22:37:42.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7982000000000003e-05 2024-09-14T22:37:42.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878411+0000", 2024-09-14T22:37:42.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0927000000000001e-05 2024-09-14T22:37:42.181 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.181 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.181 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.182 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112030 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6316032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:42.182 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:37:42.182 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.334552167, 2024-09-14T22:37:42.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054597826000000002, 2024-09-14T22:37:42.182 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.182 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.182 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.182 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.182 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.182 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.182 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112030 2024-09-14T22:37:42.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.183 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:37:42.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:37:42.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824031+0000", 2024-09-14T22:37:42.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3600000000000003e-07 2024-09-14T22:37:42.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824036+0000", 2024-09-14T22:37:42.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7350000000000004e-06 2024-09-14T22:37:42.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824049+0000", 2024-09-14T22:37:42.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3329000000000001e-05 2024-09-14T22:37:42.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824057+0000", 2024-09-14T22:37:42.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2460000000000003e-06 2024-09-14T22:37:42.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878464+0000", 2024-09-14T22:37:42.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054406207999999998 2024-09-14T22:37:42.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878554+0000", 2024-09-14T22:37:42.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0199999999999997e-05 2024-09-14T22:37:42.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878628+0000", 2024-09-14T22:37:42.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4371999999999999e-05 2024-09-14T22:37:42.186 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.186 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.186 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.186 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112031 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6365184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:42.186 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:37:42.186 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.334513849, 2024-09-14T22:37:42.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054741868999999999, 2024-09-14T22:37:42.187 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.187 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.187 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.187 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.187 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.187 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.187 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112031 2024-09-14T22:37:42.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.187 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:37:42.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:37:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824070+0000", 2024-09-14T22:37:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5099999999999998e-07 2024-09-14T22:37:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824073+0000", 2024-09-14T22:37:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6940000000000001e-06 2024-09-14T22:37:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824084+0000", 2024-09-14T22:37:42.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0380000000000001e-05 2024-09-14T22:37:42.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824091+0000", 2024-09-14T22:37:42.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5730000000000002e-06 2024-09-14T22:37:42.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878670+0000", 2024-09-14T22:37:42.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054579176 2024-09-14T22:37:42.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878742+0000", 2024-09-14T22:37:42.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T22:37:42.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878811+0000", 2024-09-14T22:37:42.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8170999999999995e-05 2024-09-14T22:37:42.190 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.190 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.190 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.191 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112032 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6414336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:42.191 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:37:42.191 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.334479983, 2024-09-14T22:37:42.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054931916999999997, 2024-09-14T22:37:42.191 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.191 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.191 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.191 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.191 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.191 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.191 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112032 2024-09-14T22:37:42.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.191 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:37:42.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:37:42.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:37:42.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:37:42.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824107+0000", 2024-09-14T22:37:42.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5640000000000001e-06 2024-09-14T22:37:42.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824117+0000", 2024-09-14T22:37:42.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0366e-05 2024-09-14T22:37:42.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824158+0000", 2024-09-14T22:37:42.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0670000000000002e-05 2024-09-14T22:37:42.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878853+0000", 2024-09-14T22:37:42.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054694853000000002 2024-09-14T22:37:42.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878934+0000", 2024-09-14T22:37:42.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1754000000000004e-05 2024-09-14T22:37:42.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879034+0000", 2024-09-14T22:37:42.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000100115 2024-09-14T22:37:42.195 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.195 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.195 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.195 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112033 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6463488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:42.195 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:37:42.195 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.334411325, 2024-09-14T22:37:42.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054991076999999999, 2024-09-14T22:37:42.195 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.196 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.196 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.196 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.196 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.196 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.196 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112033 2024-09-14T22:37:42.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.196 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:37:42.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:37:42.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824172+0000", 2024-09-14T22:37:42.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9999999999999996e-07 2024-09-14T22:37:42.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824177+0000", 2024-09-14T22:37:42.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5310000000000003e-06 2024-09-14T22:37:42.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824188+0000", 2024-09-14T22:37:42.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1235e-05 2024-09-14T22:37:42.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824197+0000", 2024-09-14T22:37:42.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7069999999999998e-06 2024-09-14T22:37:42.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879075+0000", 2024-09-14T22:37:42.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054878749999999997 2024-09-14T22:37:42.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879118+0000", 2024-09-14T22:37:42.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2749e-05 2024-09-14T22:37:42.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879162+0000", 2024-09-14T22:37:42.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4205000000000001e-05 2024-09-14T22:37:42.199 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.199 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.199 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.200 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112034 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6512640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:42.200 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:37:42.200 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.334374373999999, 2024-09-14T22:37:42.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055181246000000003, 2024-09-14T22:37:42.200 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.200 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.200 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.200 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.200 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.200 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.200 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112034 2024-09-14T22:37:42.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.201 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:37:42.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:37:42.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824209+0000", 2024-09-14T22:37:42.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5499999999999998e-07 2024-09-14T22:37:42.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824212+0000", 2024-09-14T22:37:42.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6119999999999999e-06 2024-09-14T22:37:42.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824224+0000", 2024-09-14T22:37:42.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1127999999999999e-05 2024-09-14T22:37:42.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824232+0000", 2024-09-14T22:37:42.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4700000000000002e-06 2024-09-14T22:37:42.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879209+0000", 2024-09-14T22:37:42.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054977201000000003 2024-09-14T22:37:42.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879308+0000", 2024-09-14T22:37:42.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8419999999999996e-05 2024-09-14T22:37:42.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879389+0000", 2024-09-14T22:37:42.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1760000000000003e-05 2024-09-14T22:37:42.204 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.204 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.204 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.204 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112035 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6561792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:42.204 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:37:42.204 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.334280646, 2024-09-14T22:37:42.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055300522999999997, 2024-09-14T22:37:42.204 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.205 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.205 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.205 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.205 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.205 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.205 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112035 2024-09-14T22:37:42.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.205 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:37:42.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:37:42.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824303+0000", 2024-09-14T22:37:42.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5099999999999998e-07 2024-09-14T22:37:42.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824308+0000", 2024-09-14T22:37:42.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6689999999999997e-06 2024-09-14T22:37:42.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824744+0000", 2024-09-14T22:37:42.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00043601600000000001 2024-09-14T22:37:42.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824756+0000", 2024-09-14T22:37:42.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2476999999999999e-05 2024-09-14T22:37:42.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879434+0000", 2024-09-14T22:37:42.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054677630999999997 2024-09-14T22:37:42.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.208 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879518+0000", 2024-09-14T22:37:42.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4220000000000003e-05 2024-09-14T22:37:42.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.208 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879602+0000", 2024-09-14T22:37:42.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4559000000000001e-05 2024-09-14T22:37:42.208 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.208 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.208 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.209 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112036 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6610944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:42.209 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:37:42.209 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.333788908000001, 2024-09-14T22:37:42.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055036320999999999, 2024-09-14T22:37:42.209 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.209 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.209 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.209 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.209 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.209 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.209 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112036 2024-09-14T22:37:42.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.209 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:37:42.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:37:42.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824795+0000", 2024-09-14T22:37:42.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-06 2024-09-14T22:37:42.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824802+0000", 2024-09-14T22:37:42.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6680000000000004e-06 2024-09-14T22:37:42.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824814+0000", 2024-09-14T22:37:42.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2102e-05 2024-09-14T22:37:42.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.212 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824823+0000", 2024-09-14T22:37:42.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5550000000000008e-06 2024-09-14T22:37:42.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.212 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.212 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879648+0000", 2024-09-14T22:37:42.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054825167000000001 2024-09-14T22:37:42.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.212 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.212 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879747+0000", 2024-09-14T22:37:42.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8968999999999994e-05 2024-09-14T22:37:42.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879830+0000", 2024-09-14T22:37:42.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2633999999999995e-05 2024-09-14T22:37:42.213 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.213 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.213 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.213 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112037 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6660096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:42.213 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:37:42.213 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.333746199, 2024-09-14T22:37:42.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055122926000000003, 2024-09-14T22:37:42.213 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.213 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.213 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.214 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.214 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.214 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.214 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112037 2024-09-14T22:37:42.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.214 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:37:42.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:37:42.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824837+0000", 2024-09-14T22:37:42.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5499999999999998e-07 2024-09-14T22:37:42.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824840+0000", 2024-09-14T22:37:42.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0429999999999999e-06 2024-09-14T22:37:42.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824851+0000", 2024-09-14T22:37:42.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1046999999999999e-05 2024-09-14T22:37:42.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824859+0000", 2024-09-14T22:37:42.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1200000000000002e-06 2024-09-14T22:37:42.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879867+0000", 2024-09-14T22:37:42.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055007697000000001 2024-09-14T22:37:42.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879910+0000", 2024-09-14T22:37:42.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3402000000000002e-05 2024-09-14T22:37:42.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879959+0000", 2024-09-14T22:37:42.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9061999999999999e-05 2024-09-14T22:37:42.217 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.217 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.217 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.217 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112038 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6709248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:42.217 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:37:42.217 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.333711503, 2024-09-14T22:37:42.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055312828000000001, 2024-09-14T22:37:42.218 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.218 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.218 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.218 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.218 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.218 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.218 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112038 2024-09-14T22:37:42.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.218 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:37:42.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:37:42.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824872+0000", 2024-09-14T22:37:42.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T22:37:42.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824875+0000", 2024-09-14T22:37:42.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7519999999999998e-06 2024-09-14T22:37:42.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824917+0000", 2024-09-14T22:37:42.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1047999999999997e-05 2024-09-14T22:37:42.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824926+0000", 2024-09-14T22:37:42.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3749999999999992e-06 2024-09-14T22:37:42.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880006+0000", 2024-09-14T22:37:42.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055080525999999998 2024-09-14T22:37:42.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880102+0000", 2024-09-14T22:37:42.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5995999999999994e-05 2024-09-14T22:37:42.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880184+0000", 2024-09-14T22:37:42.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1478999999999997e-05 2024-09-14T22:37:42.222 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.222 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.222 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.222 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112039 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6758400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:42.222 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:37:42.222 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.333642833000001, 2024-09-14T22:37:42.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055449709999999999, 2024-09-14T22:37:42.222 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.222 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.222 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.222 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.223 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.223 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.223 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112039 2024-09-14T22:37:42.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.223 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:37:42.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:37:42.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824941+0000", 2024-09-14T22:37:42.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3699999999999999e-07 2024-09-14T22:37:42.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824946+0000", 2024-09-14T22:37:42.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0209999999999999e-06 2024-09-14T22:37:42.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824972+0000", 2024-09-14T22:37:42.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6750999999999998e-05 2024-09-14T22:37:42.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824982+0000", 2024-09-14T22:37:42.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5019999999999998e-06 2024-09-14T22:37:42.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880227+0000", 2024-09-14T22:37:42.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055245257999999998 2024-09-14T22:37:42.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880305+0000", 2024-09-14T22:37:42.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7424999999999998e-05 2024-09-14T22:37:42.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880389+0000", 2024-09-14T22:37:42.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4815999999999996e-05 2024-09-14T22:37:42.226 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.226 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.226 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.227 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112040 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6807552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:42.227 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:37:42.227 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.333586116999999, 2024-09-14T22:37:42.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055534615000000002, 2024-09-14T22:37:42.227 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.227 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.227 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.227 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.227 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.227 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.227 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112040 2024-09-14T22:37:42.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.227 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:37:42.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:37:42.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824997+0000", 2024-09-14T22:37:42.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7199999999999998e-07 2024-09-14T22:37:42.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825042+0000", 2024-09-14T22:37:42.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5065999999999998e-05 2024-09-14T22:37:42.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825060+0000", 2024-09-14T22:37:42.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7520000000000002e-05 2024-09-14T22:37:42.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825070+0000", 2024-09-14T22:37:42.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8230000000000006e-06 2024-09-14T22:37:42.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880427+0000", 2024-09-14T22:37:42.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055357244 2024-09-14T22:37:42.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880483+0000", 2024-09-14T22:37:42.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5807000000000003e-05 2024-09-14T22:37:42.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880531+0000", 2024-09-14T22:37:42.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8383000000000001e-05 2024-09-14T22:37:42.232 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.232 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.232 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.233 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112041 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6856704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:42.233 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:37:42.233 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.333496943, 2024-09-14T22:37:42.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055573887000000002, 2024-09-14T22:37:42.233 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.233 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.233 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.233 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.233 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.233 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.233 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112041 2024-09-14T22:37:42.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.233 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:37:42.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:37:42.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825087+0000", 2024-09-14T22:37:42.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2399999999999996e-07 2024-09-14T22:37:42.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825091+0000", 2024-09-14T22:37:42.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9729999999999999e-06 2024-09-14T22:37:42.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825101+0000", 2024-09-14T22:37:42.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.093e-05 2024-09-14T22:37:42.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825110+0000", 2024-09-14T22:37:42.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1450000000000008e-06 2024-09-14T22:37:42.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880559+0000", 2024-09-14T22:37:42.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055449684999999999 2024-09-14T22:37:42.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880601+0000", 2024-09-14T22:37:42.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1248000000000002e-05 2024-09-14T22:37:42.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880659+0000", 2024-09-14T22:37:42.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8981999999999999e-05 2024-09-14T22:37:42.237 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.237 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.237 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.237 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112042 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6905856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:42.237 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:37:42.237 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.333462209, 2024-09-14T22:37:42.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055781415000000001, 2024-09-14T22:37:42.237 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.238 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.238 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.238 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.238 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.238 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.238 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112042 2024-09-14T22:37:42.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.238 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:37:42.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:37:42.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825478+0000", 2024-09-14T22:37:42.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035775 2024-09-14T22:37:42.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825494+0000", 2024-09-14T22:37:42.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5994999999999998e-05 2024-09-14T22:37:42.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825520+0000", 2024-09-14T22:37:42.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5738999999999999e-05 2024-09-14T22:37:42.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825547+0000", 2024-09-14T22:37:42.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7667000000000001e-05 2024-09-14T22:37:42.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880725+0000", 2024-09-14T22:37:42.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055177962999999997 2024-09-14T22:37:42.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880816+0000", 2024-09-14T22:37:42.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0315000000000005e-05 2024-09-14T22:37:42.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880902+0000", 2024-09-14T22:37:42.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5986e-05 2024-09-14T22:37:42.241 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.241 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.241 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.242 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112043 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:42.242 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:37:42.242 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.333014205, 2024-09-14T22:37:42.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055537912000000002, 2024-09-14T22:37:42.242 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.242 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.242 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.242 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.242 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.242 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.242 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112043 2024-09-14T22:37:42.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.243 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:37:42.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:37:42.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825911+0000", 2024-09-14T22:37:42.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00034306899999999999 2024-09-14T22:37:42.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.244 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825918+0000", 2024-09-14T22:37:42.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.002e-06 2024-09-14T22:37:42.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.244 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825931+0000", 2024-09-14T22:37:42.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2401e-05 2024-09-14T22:37:42.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.244 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825942+0000", 2024-09-14T22:37:42.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0903e-05 2024-09-14T22:37:42.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880951+0000", 2024-09-14T22:37:42.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055009379999999997 2024-09-14T22:37:42.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881027+0000", 2024-09-14T22:37:42.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5944000000000003e-05 2024-09-14T22:37:42.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881106+0000", 2024-09-14T22:37:42.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9213000000000002e-05 2024-09-14T22:37:42.246 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.246 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.246 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.246 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112044 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 7004160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:42.246 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:37:42.246 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.332620536, 2024-09-14T22:37:42.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055265416999999997, 2024-09-14T22:37:42.246 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.246 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.246 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.247 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.247 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.247 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.247 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112044 2024-09-14T22:37:42.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.247 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:37:42.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:37:42.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.248 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.248 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826239+0000", 2024-09-14T22:37:42.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027676000000000003 2024-09-14T22:37:42.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.248 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.248 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826247+0000", 2024-09-14T22:37:42.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7440000000000007e-06 2024-09-14T22:37:42.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.248 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.249 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826275+0000", 2024-09-14T22:37:42.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8053000000000001e-05 2024-09-14T22:37:42.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.249 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.249 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826286+0000", 2024-09-14T22:37:42.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1780000000000001e-05 2024-09-14T22:37:42.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.249 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.249 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881137+0000", 2024-09-14T22:37:42.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054850500000000003 2024-09-14T22:37:42.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881181+0000", 2024-09-14T22:37:42.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4153000000000002e-05 2024-09-14T22:37:42.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881227+0000", 2024-09-14T22:37:42.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6427000000000001e-05 2024-09-14T22:37:42.250 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.250 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.250 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.251 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112045 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 7053312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:42.251 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:37:42.251 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.332258303, 2024-09-14T22:37:42.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055045050999999998, 2024-09-14T22:37:42.251 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.251 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.251 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.251 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.251 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.251 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.251 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112045 2024-09-14T22:37:42.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.252 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:37:42.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:37:42.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830250+0000", 2024-09-14T22:37:42.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0039258670000000004 2024-09-14T22:37:42.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830260+0000", 2024-09-14T22:37:42.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0088000000000001e-05 2024-09-14T22:37:42.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830281+0000", 2024-09-14T22:37:42.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0720000000000002e-05 2024-09-14T22:37:42.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.254 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.254 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830292+0000", 2024-09-14T22:37:42.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1455e-05 2024-09-14T22:37:42.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.254 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.254 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881255+0000", 2024-09-14T22:37:42.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.050962247000000002 2024-09-14T22:37:42.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.254 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.254 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881296+0000", 2024-09-14T22:37:42.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1557999999999998e-05 2024-09-14T22:37:42.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881369+0000", 2024-09-14T22:37:42.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3115999999999996e-05 2024-09-14T22:37:42.255 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.255 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.255 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.255 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.255 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.255 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:42.257 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:37:42.257 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:42.255+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:42.357 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:42.355+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:42.374 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:42.374 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:42.375 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:37:42.375 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:42.389 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:37:42.408 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-14T22:37:42.458 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:42.455+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:42.532 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:42.532 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:42.532 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:37:42.532 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:37:42.532 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:42.547 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:37:42.591 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:42.586+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:42.639 INFO:teuthology.orchestra.run.smithi001.stdout:4827 read 0x5d771e1 thru 0x5d7bae1 (0x4901 bytes) 2024-09-14T22:37:42.641 INFO:teuthology.orchestra.run.smithi001.stdout:4828 read 0x47340ae thru 0x4738a51 (0x49a4 bytes) 2024-09-14T22:37:42.642 INFO:teuthology.orchestra.run.smithi001.stdout:4829 writesame 0x67f3a7a thru 0x6800310 data_size 0x1(0xc897 bytes) 2024-09-14T22:37:42.659 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:42.655+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:42.688 INFO:teuthology.orchestra.run.smithi001.stdout:4830 trunc from 0x75e312d to 0x4bc32e3 2024-09-14T22:37:42.696 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:42.696 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:42.696 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:42.696 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:42.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.696 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97197 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 796672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:42.696 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:37:42.696 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.620088656999997, 2024-09-14T22:37:42.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12434473, 2024-09-14T22:37:42.697 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.697 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.697 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.697 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.697 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.697 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.697 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97197 2024-09-14T22:37:42.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.697 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:37:42.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:37:42.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070524+0000", 2024-09-14T22:37:42.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0299999999999998e-07 2024-09-14T22:37:42.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070527+0000", 2024-09-14T22:37:42.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9569999999999998e-06 2024-09-14T22:37:42.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070535+0000", 2024-09-14T22:37:42.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7630000000000006e-06 2024-09-14T22:37:42.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070543+0000", 2024-09-14T22:37:42.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4869999999999996e-06 2024-09-14T22:37:42.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194525+0000", 2024-09-14T22:37:42.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123982096 2024-09-14T22:37:42.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.700 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.700 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194699+0000", 2024-09-14T22:37:42.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000174105 2024-09-14T22:37:42.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.700 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.700 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194868+0000", 2024-09-14T22:37:42.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016861900000000001 2024-09-14T22:37:42.700 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.700 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.700 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.701 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97198 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 805888~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:42.701 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:37:42.701 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.620060281000001, 2024-09-14T22:37:42.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.1248088, 2024-09-14T22:37:42.701 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.701 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.701 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.701 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.701 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.701 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.702 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97198 2024-09-14T22:37:42.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.702 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:37:42.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:37:42.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070552+0000", 2024-09-14T22:37:42.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T22:37:42.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070555+0000", 2024-09-14T22:37:42.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.853e-06 2024-09-14T22:37:42.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070565+0000", 2024-09-14T22:37:42.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.048e-05 2024-09-14T22:37:42.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070573+0000", 2024-09-14T22:37:42.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7200000000000006e-06 2024-09-14T22:37:42.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194954+0000", 2024-09-14T22:37:42.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12438089400000001 2024-09-14T22:37:42.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195145+0000", 2024-09-14T22:37:42.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019091299999999999 2024-09-14T22:37:42.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195360+0000", 2024-09-14T22:37:42.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021527300000000001 2024-09-14T22:37:42.705 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.705 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.705 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.705 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97199 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 815104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:42.705 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:37:42.705 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.620029684000002, 2024-09-14T22:37:42.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125251901, 2024-09-14T22:37:42.706 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.706 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.706 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.706 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.706 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.706 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.706 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97199 2024-09-14T22:37:42.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.706 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.706 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:37:42.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:37:42.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070583+0000", 2024-09-14T22:37:42.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6300000000000005e-07 2024-09-14T22:37:42.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070586+0000", 2024-09-14T22:37:42.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8270000000000002e-06 2024-09-14T22:37:42.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070597+0000", 2024-09-14T22:37:42.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1960000000000001e-05 2024-09-14T22:37:42.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070605+0000", 2024-09-14T22:37:42.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6459999999999993e-06 2024-09-14T22:37:42.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195432+0000", 2024-09-14T22:37:42.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124826926 2024-09-14T22:37:42.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195626+0000", 2024-09-14T22:37:42.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019357099999999999 2024-09-14T22:37:42.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195834+0000", 2024-09-14T22:37:42.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020830800000000001 2024-09-14T22:37:42.709 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.709 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.709 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.710 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97200 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 824320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:42.710 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:37:42.710 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.619997638000001, 2024-09-14T22:37:42.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125693477, 2024-09-14T22:37:42.710 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.710 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.710 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.710 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.710 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.710 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.710 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97200 2024-09-14T22:37:42.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.711 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:37:42.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:37:42.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070615+0000", 2024-09-14T22:37:42.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T22:37:42.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070618+0000", 2024-09-14T22:37:42.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.943e-06 2024-09-14T22:37:42.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070627+0000", 2024-09-14T22:37:42.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1099999999999992e-06 2024-09-14T22:37:42.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070634+0000", 2024-09-14T22:37:42.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:37:42.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195908+0000", 2024-09-14T22:37:42.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125273893 2024-09-14T22:37:42.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196097+0000", 2024-09-14T22:37:42.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018953899999999999 2024-09-14T22:37:42.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196308+0000", 2024-09-14T22:37:42.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021027799999999999 2024-09-14T22:37:42.714 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.714 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.714 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.714 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97201 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 833536~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:42.714 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:37:42.714 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.619967451000001, 2024-09-14T22:37:42.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126133826, 2024-09-14T22:37:42.714 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.714 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.714 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.714 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.715 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.715 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.715 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97201 2024-09-14T22:37:42.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.715 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:37:42.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:37:42.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.716 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.716 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074109+0000", 2024-09-14T22:37:42.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003464834 2024-09-14T22:37:42.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.716 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.716 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074134+0000", 2024-09-14T22:37:42.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5170000000000001e-05 2024-09-14T22:37:42.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.717 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.717 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074166+0000", 2024-09-14T22:37:42.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1347000000000003e-05 2024-09-14T22:37:42.717 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.717 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.717 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.717 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074179+0000", 2024-09-14T22:37:42.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.364e-05 2024-09-14T22:37:42.717 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.717 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.717 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.717 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196377+0000", 2024-09-14T22:37:42.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12219793900000001 2024-09-14T22:37:42.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196568+0000", 2024-09-14T22:37:42.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019097999999999999 2024-09-14T22:37:42.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196778+0000", 2024-09-14T22:37:42.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020991599999999999 2024-09-14T22:37:42.718 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.718 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.719 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.719 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97205 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 870400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:42.719 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:37:42.719 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.616297887999998, 2024-09-14T22:37:42.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12436148299999999, 2024-09-14T22:37:42.719 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.719 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.719 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.719 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.719 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.719 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.720 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97205 2024-09-14T22:37:42.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.720 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:37:42.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:37:42.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.721 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.721 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074315+0000", 2024-09-14T22:37:42.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9299999999999997e-07 2024-09-14T22:37:42.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.721 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.721 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074318+0000", 2024-09-14T22:37:42.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.19e-06 2024-09-14T22:37:42.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.721 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.721 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074327+0000", 2024-09-14T22:37:42.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8529999999999998e-06 2024-09-14T22:37:42.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.722 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.722 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074334+0000", 2024-09-14T22:37:42.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3869999999999997e-06 2024-09-14T22:37:42.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.722 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.722 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198365+0000", 2024-09-14T22:37:42.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124031222 2024-09-14T22:37:42.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.722 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.722 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198513+0000", 2024-09-14T22:37:42.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014772299999999999 2024-09-14T22:37:42.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198675+0000", 2024-09-14T22:37:42.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016241500000000001 2024-09-14T22:37:42.723 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.723 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.723 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.723 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97206 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 879616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:42.723 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:37:42.723 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.616266432000003, 2024-09-14T22:37:42.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124706284, 2024-09-14T22:37:42.724 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.724 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.724 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.724 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.724 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.724 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.724 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97206 2024-09-14T22:37:42.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.724 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:37:42.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:37:42.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074346+0000", 2024-09-14T22:37:42.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8999999999999996e-07 2024-09-14T22:37:42.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074349+0000", 2024-09-14T22:37:42.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.96e-06 2024-09-14T22:37:42.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.726 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.726 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074358+0000", 2024-09-14T22:37:42.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9169999999999995e-06 2024-09-14T22:37:42.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.726 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.726 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074365+0000", 2024-09-14T22:37:42.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3699999999999997e-06 2024-09-14T22:37:42.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.726 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198761+0000", 2024-09-14T22:37:42.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124395743 2024-09-14T22:37:42.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198899+0000", 2024-09-14T22:37:42.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013763499999999999 2024-09-14T22:37:42.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199052+0000", 2024-09-14T22:37:42.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015296900000000001 2024-09-14T22:37:42.727 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.727 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.728 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.728 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97207 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 888832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:42.728 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:37:42.728 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.616237218999999, 2024-09-14T22:37:42.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124988433, 2024-09-14T22:37:42.728 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.728 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.728 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.728 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.728 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.728 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.728 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97207 2024-09-14T22:37:42.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.729 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:37:42.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:37:42.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074375+0000", 2024-09-14T22:37:42.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:37:42.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074378+0000", 2024-09-14T22:37:42.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1829999999999998e-06 2024-09-14T22:37:42.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074389+0000", 2024-09-14T22:37:42.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0737000000000001e-05 2024-09-14T22:37:42.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074396+0000", 2024-09-14T22:37:42.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2559999999999996e-06 2024-09-14T22:37:42.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199139+0000", 2024-09-14T22:37:42.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12474253 2024-09-14T22:37:42.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199240+0000", 2024-09-14T22:37:42.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101221 2024-09-14T22:37:42.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199363+0000", 2024-09-14T22:37:42.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000122822 2024-09-14T22:37:42.732 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.732 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.732 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.732 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97208 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 898048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:42.732 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:37:42.732 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.616205915000002, 2024-09-14T22:37:42.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12524854899999999, 2024-09-14T22:37:42.733 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.733 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.733 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.733 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.733 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.733 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.733 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97208 2024-09-14T22:37:42.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.733 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:37:42.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:37:42.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:37:42.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:37:42.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074410+0000", 2024-09-14T22:37:42.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.18e-06 2024-09-14T22:37:42.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074435+0000", 2024-09-14T22:37:42.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5273e-05 2024-09-14T22:37:42.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074443+0000", 2024-09-14T22:37:42.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3000000000000002e-06 2024-09-14T22:37:42.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199433+0000", 2024-09-14T22:37:42.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124989502 2024-09-14T22:37:42.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199535+0000", 2024-09-14T22:37:42.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102557 2024-09-14T22:37:42.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199654+0000", 2024-09-14T22:37:42.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119061 2024-09-14T22:37:42.737 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.737 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.737 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.737 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97209 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 907264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:42.737 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:37:42.737 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.616158110000001, 2024-09-14T22:37:42.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125487354, 2024-09-14T22:37:42.737 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.737 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.737 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.737 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.737 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.738 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.738 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97209 2024-09-14T22:37:42.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.738 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:37:42.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:37:42.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:37:42.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9400000000000005e-07 2024-09-14T22:37:42.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074458+0000", 2024-09-14T22:37:42.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3299999999999999e-06 2024-09-14T22:37:42.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074467+0000", 2024-09-14T22:37:42.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.346e-06 2024-09-14T22:37:42.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074474+0000", 2024-09-14T22:37:42.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.374e-06 2024-09-14T22:37:42.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199720+0000", 2024-09-14T22:37:42.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12524561200000001 2024-09-14T22:37:42.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199820+0000", 2024-09-14T22:37:42.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010049599999999999 2024-09-14T22:37:42.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199941+0000", 2024-09-14T22:37:42.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012050200000000001 2024-09-14T22:37:42.741 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.741 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.741 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.741 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97210 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:42.741 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:37:42.741 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.616128115999999, 2024-09-14T22:37:42.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125744942, 2024-09-14T22:37:42.742 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.742 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.742 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.742 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.742 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.742 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.742 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97210 2024-09-14T22:37:42.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.742 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:37:42.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.743 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:37:42.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.743 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:37:42.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6599999999999996e-07 2024-09-14T22:37:42.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.743 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074487+0000", 2024-09-14T22:37:42.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8140000000000001e-06 2024-09-14T22:37:42.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.744 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.744 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074496+0000", 2024-09-14T22:37:42.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9330000000000007e-06 2024-09-14T22:37:42.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.744 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.744 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074503+0000", 2024-09-14T22:37:42.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4100000000000002e-06 2024-09-14T22:37:42.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200009+0000", 2024-09-14T22:37:42.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12550539899999999 2024-09-14T22:37:42.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200108+0000", 2024-09-14T22:37:42.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9182e-05 2024-09-14T22:37:42.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200229+0000", 2024-09-14T22:37:42.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120538 2024-09-14T22:37:42.746 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.746 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.746 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.746 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97211 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 925696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:42.746 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:37:42.746 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.616097156000002, 2024-09-14T22:37:42.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12599848299999999, 2024-09-14T22:37:42.746 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.746 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.746 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.747 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.747 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.747 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.747 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97211 2024-09-14T22:37:42.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.747 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:37:42.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:37:42.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074516+0000", 2024-09-14T22:37:42.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.42e-06 2024-09-14T22:37:42.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074521+0000", 2024-09-14T22:37:42.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7960000000000003e-06 2024-09-14T22:37:42.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074538+0000", 2024-09-14T22:37:42.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6900000000000001e-05 2024-09-14T22:37:42.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074555+0000", 2024-09-14T22:37:42.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.696e-05 2024-09-14T22:37:42.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200293+0000", 2024-09-14T22:37:42.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12573864100000001 2024-09-14T22:37:42.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200393+0000", 2024-09-14T22:37:42.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9264000000000004e-05 2024-09-14T22:37:42.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200513+0000", 2024-09-14T22:37:42.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012050200000000001 2024-09-14T22:37:42.750 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.750 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.750 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.750 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97212 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 934912~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:42.751 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:37:42.751 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.616022534000003, 2024-09-14T22:37:42.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12621127300000001, 2024-09-14T22:37:42.751 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.751 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.751 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.751 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.751 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.751 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.751 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97212 2024-09-14T22:37:42.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.751 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:37:42.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:37:42.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074590+0000", 2024-09-14T22:37:42.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1969999999999999e-06 2024-09-14T22:37:42.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074596+0000", 2024-09-14T22:37:42.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7629999999999997e-06 2024-09-14T22:37:42.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.753 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.753 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074614+0000", 2024-09-14T22:37:42.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7643e-05 2024-09-14T22:37:42.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.753 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.753 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074635+0000", 2024-09-14T22:37:42.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.075e-05 2024-09-14T22:37:42.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.753 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.753 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200580+0000", 2024-09-14T22:37:42.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125945482 2024-09-14T22:37:42.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200683+0000", 2024-09-14T22:37:42.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010267099999999999 2024-09-14T22:37:42.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200800+0000", 2024-09-14T22:37:42.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117767 2024-09-14T22:37:42.754 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.754 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.755 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.755 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97213 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 944128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:42.755 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:37:42.755 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.615957184000003, 2024-09-14T22:37:42.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126430709, 2024-09-14T22:37:42.755 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.755 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.755 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.755 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.755 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.755 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.756 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97213 2024-09-14T22:37:42.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.756 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:37:42.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:37:42.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078210+0000", 2024-09-14T22:37:42.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0035556839999999999 2024-09-14T22:37:42.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078231+0000", 2024-09-14T22:37:42.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0716e-05 2024-09-14T22:37:42.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078260+0000", 2024-09-14T22:37:42.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9473000000000001e-05 2024-09-14T22:37:42.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.758 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078279+0000", 2024-09-14T22:37:42.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8600000000000001e-05 2024-09-14T22:37:42.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.758 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200864+0000", 2024-09-14T22:37:42.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.122585295 2024-09-14T22:37:42.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.758 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200965+0000", 2024-09-14T22:37:42.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101004 2024-09-14T22:37:42.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.201085+0000", 2024-09-14T22:37:42.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119937 2024-09-14T22:37:42.759 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.759 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.759 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.759 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97219 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 999424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:42.759 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:37:42.759 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.611974906999997, 2024-09-14T22:37:42.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124138558, 2024-09-14T22:37:42.760 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.760 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.760 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.760 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.760 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.760 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.760 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97219 2024-09-14T22:37:42.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.760 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:37:42.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:37:42.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078638+0000", 2024-09-14T22:37:42.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.173e-06 2024-09-14T22:37:42.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078644+0000", 2024-09-14T22:37:42.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2570000000000004e-06 2024-09-14T22:37:42.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078672+0000", 2024-09-14T22:37:42.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7322999999999999e-05 2024-09-14T22:37:42.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078687+0000", 2024-09-14T22:37:42.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5639999999999999e-05 2024-09-14T22:37:42.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202567+0000", 2024-09-14T22:37:42.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123880134 2024-09-14T22:37:42.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202670+0000", 2024-09-14T22:37:42.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010273899999999999 2024-09-14T22:37:42.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202775+0000", 2024-09-14T22:37:42.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010529199999999999 2024-09-14T22:37:42.763 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.763 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.763 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.764 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97220 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1008640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:42.764 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:37:42.764 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.611905673999999, 2024-09-14T22:37:42.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12435937, 2024-09-14T22:37:42.764 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.764 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.764 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.764 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.764 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.764 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.764 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97220 2024-09-14T22:37:42.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.764 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:37:42.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:37:42.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078707+0000", 2024-09-14T22:37:42.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.266e-06 2024-09-14T22:37:42.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078713+0000", 2024-09-14T22:37:42.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7270000000000004e-06 2024-09-14T22:37:42.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078737+0000", 2024-09-14T22:37:42.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3640000000000001e-05 2024-09-14T22:37:42.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078751+0000", 2024-09-14T22:37:42.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.45e-05 2024-09-14T22:37:42.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202841+0000", 2024-09-14T22:37:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124089938 2024-09-14T22:37:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202944+0000", 2024-09-14T22:37:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010334 2024-09-14T22:37:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203065+0000", 2024-09-14T22:37:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012095900000000001 2024-09-14T22:37:42.768 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.768 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.768 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.768 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97221 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1017856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:42.768 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:37:42.768 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.611839910999997, 2024-09-14T22:37:42.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124574005, 2024-09-14T22:37:42.768 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.768 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.768 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.768 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.768 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.769 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.769 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97221 2024-09-14T22:37:42.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.769 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:37:42.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:37:42.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078773+0000", 2024-09-14T22:37:42.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.13e-06 2024-09-14T22:37:42.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078779+0000", 2024-09-14T22:37:42.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0260000000000004e-06 2024-09-14T22:37:42.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078797+0000", 2024-09-14T22:37:42.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.806e-05 2024-09-14T22:37:42.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078810+0000", 2024-09-14T22:37:42.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.308e-05 2024-09-14T22:37:42.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203131+0000", 2024-09-14T22:37:42.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12432077499999999 2024-09-14T22:37:42.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203227+0000", 2024-09-14T22:37:42.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6575000000000003e-05 2024-09-14T22:37:42.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203346+0000", 2024-09-14T22:37:42.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118359 2024-09-14T22:37:42.772 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.772 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.772 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.773 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97222 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:42.773 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:37:42.773 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.611783541999998, 2024-09-14T22:37:42.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12480110799999999, 2024-09-14T22:37:42.773 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.773 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.773 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.773 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.773 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.773 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.773 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97222 2024-09-14T22:37:42.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.773 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:37:42.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:37:42.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078829+0000", 2024-09-14T22:37:42.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.057e-06 2024-09-14T22:37:42.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078836+0000", 2024-09-14T22:37:42.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6159999999999998e-06 2024-09-14T22:37:42.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078856+0000", 2024-09-14T22:37:42.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0100000000000001e-05 2024-09-14T22:37:42.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078869+0000", 2024-09-14T22:37:42.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.272e-05 2024-09-14T22:37:42.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.776 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203412+0000", 2024-09-14T22:37:42.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12454283300000001 2024-09-14T22:37:42.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.776 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203511+0000", 2024-09-14T22:37:42.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9893000000000001e-05 2024-09-14T22:37:42.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.776 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203629+0000", 2024-09-14T22:37:42.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117889 2024-09-14T22:37:42.777 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.777 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.777 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.777 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97223 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1036288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:42.777 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:37:42.777 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.611411554999997, 2024-09-14T22:37:42.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124706279, 2024-09-14T22:37:42.777 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.777 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.777 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.777 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.778 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.778 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.778 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97223 2024-09-14T22:37:42.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.778 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:37:42.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:37:42.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079202+0000", 2024-09-14T22:37:42.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8330000000000001e-06 2024-09-14T22:37:42.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079212+0000", 2024-09-14T22:37:42.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0207000000000001e-05 2024-09-14T22:37:42.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079238+0000", 2024-09-14T22:37:42.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5626000000000002e-05 2024-09-14T22:37:42.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079256+0000", 2024-09-14T22:37:42.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8386999999999999e-05 2024-09-14T22:37:42.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203691+0000", 2024-09-14T22:37:42.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124434538 2024-09-14T22:37:42.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203786+0000", 2024-09-14T22:37:42.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5344000000000001e-05 2024-09-14T22:37:42.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203906+0000", 2024-09-14T22:37:42.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120344 2024-09-14T22:37:42.781 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.781 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.781 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.782 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97224 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1045504~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:42.782 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:37:42.782 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.611327735000003, 2024-09-14T22:37:42.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12513321199999999, 2024-09-14T22:37:42.782 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:42.782 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:42.782 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:42.782 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:42.782 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:42.782 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:42.782 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97224 2024-09-14T22:37:42.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.782 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:42.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:42.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:37:42.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:42.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:37:42.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:42.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:42.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082581+0000", 2024-09-14T22:37:42.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0032971530000000001 2024-09-14T22:37:42.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:42.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082592+0000", 2024-09-14T22:37:42.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1345999999999999e-05 2024-09-14T22:37:42.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:42.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082618+0000", 2024-09-14T22:37:42.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5873000000000001e-05 2024-09-14T22:37:42.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:42.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082638+0000", 2024-09-14T22:37:42.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9565999999999999e-05 2024-09-14T22:37:42.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:42.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204004+0000", 2024-09-14T22:37:42.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.121366038 2024-09-14T22:37:42.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:42.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204188+0000", 2024-09-14T22:37:42.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018427 2024-09-14T22:37:42.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:42.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:42.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:42.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204417+0000", 2024-09-14T22:37:42.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022896599999999999 2024-09-14T22:37:42.786 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.786 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.786 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.786 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:42.786 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:42.786 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:42.787 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:42.787 INFO:teuthology.orchestra.run.smithi001.stdout:4832 read 0x1241278 thru 0x124820e (0x6f97 bytes) 2024-09-14T22:37:42.787 INFO:teuthology.orchestra.run.smithi001.stdout:4835 write 0x7dd0806 thru 0x7dd819e (0x7999 bytes) 2024-09-14T22:37:42.788 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:42.788 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:42.788 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:42.788 INFO:teuthology.orchestra.run.smithi001.stdout:4839 write 0x63c1e4b thru 0x63c94ec (0x76a2 bytes) 2024-09-14T22:37:42.788 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:37:42.789 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:42.782+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:42.796 INFO:teuthology.orchestra.run.smithi001.stdout:4842 read 0x2cd96d0 thru 0x2cdf7ab (0x60dc bytes) 2024-09-14T22:37:42.796 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:42.821 INFO:teuthology.orchestra.run.smithi001.stdout:4845 write 0xda8503f thru 0xda90b5a (0xbb1c bytes) 2024-09-14T22:37:42.832 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:42.832 INFO:teuthology.orchestra.run.smithi001.stdout:4847 read 0x4662fa2 thru 0x4666ec2 (0x3f21 bytes) 2024-09-14T22:37:42.849 INFO:teuthology.orchestra.run.smithi001.stdout:4848 trunc from 0xda90b5b to 0x6543eef 2024-09-14T22:37:42.876 INFO:teuthology.orchestra.run.smithi001.stdout:4849 write 0x4d743ad thru 0x4d7821a (0x3e6e bytes) 2024-09-14T22:37:42.888 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:42.882+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:42.915 INFO:teuthology.orchestra.run.smithi001.stdout:4850 write 0xd1fbae6 thru 0xd20158a (0x5aa5 bytes) 2024-09-14T22:37:42.926 INFO:teuthology.orchestra.run.smithi001.stdout:4852 punch from 0xa28ac59 to 0xa28b8b4, (0xc5b bytes) 2024-09-14T22:37:42.927 INFO:teuthology.orchestra.run.smithi001.stdout:4853 writesame 0xbf2fe4e thru 0xbf3c2c3 data_size 0x1(0xc476 bytes) 2024-09-14T22:37:42.970 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:42.970 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:42.971 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:37:42.971 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:42.977 INFO:teuthology.orchestra.run.smithi001.stdout:4854 write 0xe0d8a53 thru 0xe0dcc52 (0x4200 bytes) 2024-09-14T22:37:42.985 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:37:42.986 INFO:teuthology.orchestra.run.smithi001.stdout:4855 read 0xb6b24e9 thru 0xb6b6d9f (0x48b7 bytes) 2024-09-14T22:37:42.986 INFO:teuthology.orchestra.run.smithi001.stdout:4856 write 0xc38b91f thru 0xc38e82c (0x2f0e bytes) 2024-09-14T22:37:42.993 INFO:teuthology.orchestra.run.smithi001.stdout:4857 write 0x2f9486f thru 0x2f9735d (0x2aef bytes) 2024-09-14T22:37:43.027 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:43.022+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:43.036 INFO:teuthology.orchestra.run.smithi001.stdout:4861 trunc from 0xe0dcc53 to 0x3790d 2024-09-14T22:37:43.090 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:43.082+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:43.167 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:43.167 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:43.167 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:37:43.167 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:37:43.168 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:43.181 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:37:43.190 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:43.187+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:43.193 INFO:teuthology.orchestra.run.smithi001.stdout:4862 writesame 0xbc03ee8 thru 0xbc0d03b data_size 0x4(0x9154 bytes) 2024-09-14T22:37:43.207 INFO:teuthology.orchestra.run.smithi001.stdout:4863 writesame 0xbf39cf9 thru 0xbf496ec data_size 0x22(0xf9f4 bytes) 2024-09-14T22:37:43.207 INFO:teuthology.orchestra.run.smithi001.stdout:4865 write 0x26dcfa9 thru 0x26e0200 (0x3258 bytes) 2024-09-14T22:37:43.216 INFO:teuthology.orchestra.run.smithi001.stdout:4866 read 0x6a9f309 thru 0x6aae4e4 (0xf1dc bytes) 2024-09-14T22:37:43.216 INFO:teuthology.orchestra.run.smithi001.stdout:4869 read 0x9feaaa7 thru 0x9fed66c (0x2bc6 bytes) 2024-09-14T22:37:43.216 INFO:teuthology.orchestra.run.smithi001.stdout:4871 writesame 0x533bd87 thru 0x53499cb data_size 0x1f(0xdc45 bytes) 2024-09-14T22:37:43.236 INFO:teuthology.orchestra.run.smithi001.stdout:4872 read 0x421566 thru 0x42e3a5 (0xce40 bytes) 2024-09-14T22:37:43.238 INFO:teuthology.orchestra.run.smithi001.stdout:4873 punch from 0xb59d00e to 0xb5a88ec, (0xb8de bytes) 2024-09-14T22:37:43.238 INFO:teuthology.orchestra.run.smithi001.stdout:4874 punch from 0x1f89ec3 to 0x1f8d18f, (0x32cc bytes) 2024-09-14T22:37:43.238 INFO:teuthology.orchestra.run.smithi001.stdout:4875 punch from 0x5e5578d to 0x5e5e227, (0x8a9a bytes) 2024-09-14T22:37:43.238 INFO:teuthology.orchestra.run.smithi001.stdout:4876 writesame 0x41079e3 thru 0x4113e6a data_size 0x26(0xc488 bytes) 2024-09-14T22:37:43.251 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:43.252 INFO:teuthology.orchestra.run.smithi001.stdout:4879 write 0xb71fc5a thru 0xb72cc7b (0xd022 bytes) 2024-09-14T22:37:43.267 INFO:teuthology.orchestra.run.smithi001.stdout:4880 writesame 0x57d57d thru 0x586b8b data_size 0x41(0x960f bytes) 2024-09-14T22:37:43.273 INFO:teuthology.orchestra.run.smithi001.stdout:4881 punch from 0x1016fb7 to 0x1016fca, (0x13 bytes) 2024-09-14T22:37:43.273 INFO:teuthology.orchestra.run.smithi001.stdout:4882 punch from 0xb0300cc to 0xb0361f5, (0x6129 bytes) 2024-09-14T22:37:43.273 INFO:teuthology.orchestra.run.smithi001.stdout:4883 punch from 0x4d4d66f to 0x4d53cb1, (0x6642 bytes) 2024-09-14T22:37:43.273 INFO:teuthology.orchestra.run.smithi001.stdout:4884 read 0x819b7f1 thru 0x819e32c (0x2b3c bytes) 2024-09-14T22:37:43.273 INFO:teuthology.orchestra.run.smithi001.stdout:4885 write 0x21050df thru 0x210ac5c (0x5b7e bytes) 2024-09-14T22:37:43.278 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:43.278 INFO:teuthology.orchestra.run.smithi001.stdout:4888 read 0x6b7e37d thru 0x6b8e261 (0xfee5 bytes) 2024-09-14T22:37:43.279 INFO:teuthology.orchestra.run.smithi001.stdout:4889 write 0x17b3546 thru 0x17bbf8f (0x8a4a bytes) 2024-09-14T22:37:43.291 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:43.286+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:43.292 INFO:teuthology.orchestra.run.smithi001.stdout:4890 trunc from 0xbf496ed to 0xc7419c1 2024-09-14T22:37:43.304 INFO:teuthology.orchestra.run.smithi001.stdout:4892 write 0x487f8c8 thru 0x4889155 (0x988e bytes) 2024-09-14T22:37:43.319 INFO:teuthology.orchestra.run.smithi001.stdout:4893 read 0x49ebe76 thru 0x49f75c0 (0xb74b bytes) 2024-09-14T22:37:43.320 INFO:teuthology.orchestra.run.smithi001.stdout:4894 clone 30 order 20 su 8192 sc 11 2024-09-14T22:37:43.354 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:43.354 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:43.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:43.354 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:43.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.354 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:72252 3.2 3:442c6d72:::rbd_data.113cb4c83968.0000000000000015:head [zero 4456448~393216] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-14T22:37:43.354 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:37:43.354 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 52.651296037999998, 2024-09-14T22:37:43.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0176591500000001, 2024-09-14T22:37:43.355 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:43.355 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:43.355 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T22:37:43.355 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:43.355 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:43.355 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:43.355 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 72252 2024-09-14T22:37:43.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.355 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:43.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:43.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:37:43.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.356 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:43.356 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:37:43.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.356 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:43.356 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700101+0000", 2024-09-14T22:37:43.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4570000000000001e-06 2024-09-14T22:37:43.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.356 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:43.356 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700114+0000", 2024-09-14T22:37:43.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2799999999999999e-05 2024-09-14T22:37:43.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:43.357 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700146+0000", 2024-09-14T22:37:43.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2116000000000002e-05 2024-09-14T22:37:43.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:43.357 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700170+0000", 2024-09-14T22:37:43.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3770000000000001e-05 2024-09-14T22:37:43.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.357 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700285+0000", 2024-09-14T22:37:43.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000115228 2024-09-14T22:37:43.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op must wait for map", 2024-09-14T22:37:43.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700297+0000", 2024-09-14T22:37:43.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1681999999999999e-05 2024-09-14T22:37:43.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.703312+0000", 2024-09-14T22:37:43.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003015046 2024-09-14T22:37:43.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:37:43.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.703327+0000", 2024-09-14T22:37:43.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5481999999999998e-05 2024-09-14T22:37:43.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:51.717566+0000", 2024-09-14T22:37:43.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0142383509999999 2024-09-14T22:37:43.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:43.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:51.717757+0000", 2024-09-14T22:37:43.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000191218 2024-09-14T22:37:43.359 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.359 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:43.360 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.360 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:74156 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T22:37:43.360 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:37:43.360 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.836827327999998, 2024-09-14T22:37:43.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062073521, 2024-09-14T22:37:43.360 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:43.360 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:43.360 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:43.360 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:43.360 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:43.360 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:43.360 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74156 2024-09-14T22:37:43.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.361 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:43.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:43.361 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:37:43.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:43.361 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:37:43.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:43.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514569+0000", 2024-09-14T22:37:43.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0229999999999999e-06 2024-09-14T22:37:43.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:43.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514596+0000", 2024-09-14T22:37:43.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6903e-05 2024-09-14T22:37:43.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:43.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514623+0000", 2024-09-14T22:37:43.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7206e-05 2024-09-14T22:37:43.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:43.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514637+0000", 2024-09-14T22:37:43.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:37:43.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514781+0000", 2024-09-14T22:37:43.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014394899999999999 2024-09-14T22:37:43.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:43.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.515005+0000", 2024-09-14T22:37:43.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022458499999999999 2024-09-14T22:37:43.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:37:43.364 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.527785+0000", 2024-09-14T22:37:43.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012780112 2024-09-14T22:37:43.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:43.364 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576209+0000", 2024-09-14T22:37:43.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.048423992999999999 2024-09-14T22:37:43.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:43.364 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576346+0000", 2024-09-14T22:37:43.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000136909 2024-09-14T22:37:43.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:43.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576517+0000", 2024-09-14T22:37:43.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000170848 2024-09-14T22:37:43.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:43.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576640+0000", 2024-09-14T22:37:43.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000122953 2024-09-14T22:37:43.365 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.365 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:43.365 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.366 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82513 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 557056~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:43.366 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:37:43.366 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.932195509000003, 2024-09-14T22:37:43.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059345486000000003, 2024-09-14T22:37:43.366 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:43.366 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:43.366 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:43.366 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:43.366 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:43.366 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:43.366 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82513 2024-09-14T22:37:43.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.366 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:43.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:43.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:37:43.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:43.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:37:43.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:43.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419200+0000", 2024-09-14T22:37:43.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.184e-06 2024-09-14T22:37:43.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:43.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419207+0000", 2024-09-14T22:37:43.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2690000000000002e-06 2024-09-14T22:37:43.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:43.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419224+0000", 2024-09-14T22:37:43.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6623e-05 2024-09-14T22:37:43.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:43.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419236+0000", 2024-09-14T22:37:43.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2367e-05 2024-09-14T22:37:43.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478294+0000", 2024-09-14T22:37:43.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059057927000000003 2024-09-14T22:37:43.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:43.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478402+0000", 2024-09-14T22:37:43.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107921 2024-09-14T22:37:43.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:43.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478544+0000", 2024-09-14T22:37:43.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014219499999999999 2024-09-14T22:37:43.369 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.369 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:43.370 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.370 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82514 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 704512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:43.370 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:37:43.370 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.932136405999998, 2024-09-14T22:37:43.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059580793, 2024-09-14T22:37:43.370 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:43.370 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:43.370 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:43.370 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:43.370 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:43.370 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:43.370 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82514 2024-09-14T22:37:43.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.371 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:43.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:43.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:37:43.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:43.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:37:43.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:43.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419259+0000", 2024-09-14T22:37:43.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.353e-06 2024-09-14T22:37:43.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:43.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419276+0000", 2024-09-14T22:37:43.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7567e-05 2024-09-14T22:37:43.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:43.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419294+0000", 2024-09-14T22:37:43.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.713e-05 2024-09-14T22:37:43.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:43.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419306+0000", 2024-09-14T22:37:43.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.279e-05 2024-09-14T22:37:43.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478589+0000", 2024-09-14T22:37:43.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059282978 2024-09-14T22:37:43.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:43.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478694+0000", 2024-09-14T22:37:43.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000104561 2024-09-14T22:37:43.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:43.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478838+0000", 2024-09-14T22:37:43.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014441399999999999 2024-09-14T22:37:43.374 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.374 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:43.374 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.374 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82515 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:43.374 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:37:43.374 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.932071256, 2024-09-14T22:37:43.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059815123999999997, 2024-09-14T22:37:43.374 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:43.374 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:43.374 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:43.374 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:43.375 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:43.375 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:43.375 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82515 2024-09-14T22:37:43.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.375 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:43.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:43.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:37:43.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:43.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:37:43.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:43.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419324+0000", 2024-09-14T22:37:43.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2130000000000001e-06 2024-09-14T22:37:43.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:43.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419331+0000", 2024-09-14T22:37:43.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4730000000000003e-06 2024-09-14T22:37:43.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:43.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419348+0000", 2024-09-14T22:37:43.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6269999999999998e-05 2024-09-14T22:37:43.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:43.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419360+0000", 2024-09-14T22:37:43.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2243e-05 2024-09-14T22:37:43.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478884+0000", 2024-09-14T22:37:43.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059524369000000001 2024-09-14T22:37:43.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:43.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478990+0000", 2024-09-14T22:37:43.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010569 2024-09-14T22:37:43.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:43.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479138+0000", 2024-09-14T22:37:43.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000147866 2024-09-14T22:37:43.378 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.378 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:43.378 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.378 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82516 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 999424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:43.378 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:37:43.378 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.932018751000001, 2024-09-14T22:37:43.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061979595999999998, 2024-09-14T22:37:43.378 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:43.378 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:43.379 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:43.379 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:43.379 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:43.379 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:43.379 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82516 2024-09-14T22:37:43.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.379 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:43.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.379 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:43.379 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:37:43.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.379 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:43.379 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:37:43.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:43.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419376+0000", 2024-09-14T22:37:43.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.237e-06 2024-09-14T22:37:43.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:43.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419383+0000", 2024-09-14T22:37:43.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8170000000000001e-06 2024-09-14T22:37:43.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:43.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419400+0000", 2024-09-14T22:37:43.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6636000000000001e-05 2024-09-14T22:37:43.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:43.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419412+0000", 2024-09-14T22:37:43.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2357e-05 2024-09-14T22:37:43.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481080+0000", 2024-09-14T22:37:43.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061668032999999997 2024-09-14T22:37:43.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:43.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481193+0000", 2024-09-14T22:37:43.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112335 2024-09-14T22:37:43.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:43.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481355+0000", 2024-09-14T22:37:43.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016218100000000001 2024-09-14T22:37:43.382 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.382 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:43.382 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.382 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82538 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1097728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:43.383 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:37:43.383 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.931966680999999, 2024-09-14T22:37:43.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059971705, 2024-09-14T22:37:43.383 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:43.383 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:43.383 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:43.383 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:43.383 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:43.383 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:43.383 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82538 2024-09-14T22:37:43.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.383 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:43.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:43.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:37:43.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:43.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:37:43.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:43.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419429+0000", 2024-09-14T22:37:43.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.497e-06 2024-09-14T22:37:43.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:43.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419436+0000", 2024-09-14T22:37:43.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2099999999999996e-06 2024-09-14T22:37:43.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:43.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419453+0000", 2024-09-14T22:37:43.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6840000000000001e-05 2024-09-14T22:37:43.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:43.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419465+0000", 2024-09-14T22:37:43.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2283000000000001e-05 2024-09-14T22:37:43.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479182+0000", 2024-09-14T22:37:43.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059717339000000001 2024-09-14T22:37:43.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:43.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479297+0000", 2024-09-14T22:37:43.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000114259 2024-09-14T22:37:43.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:43.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479399+0000", 2024-09-14T22:37:43.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102277 2024-09-14T22:37:43.386 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.386 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:43.386 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.387 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82539 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:43.387 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:37:43.387 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.931913831000003, 2024-09-14T22:37:43.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060166935999999997, 2024-09-14T22:37:43.387 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:43.387 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:43.387 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:43.387 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:43.387 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:43.387 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:43.387 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82539 2024-09-14T22:37:43.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.387 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:43.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:43.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:37:43.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:43.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:37:43.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:43.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419482+0000", 2024-09-14T22:37:43.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4130000000000001e-06 2024-09-14T22:37:43.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:43.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419488+0000", 2024-09-14T22:37:43.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8460000000000002e-06 2024-09-14T22:37:43.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:43.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419505+0000", 2024-09-14T22:37:43.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6477000000000002e-05 2024-09-14T22:37:43.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:43.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419518+0000", 2024-09-14T22:37:43.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2797e-05 2024-09-14T22:37:43.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479438+0000", 2024-09-14T22:37:43.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059919897 2024-09-14T22:37:43.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:43.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479547+0000", 2024-09-14T22:37:43.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109807 2024-09-14T22:37:43.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:43.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479647+0000", 2024-09-14T22:37:43.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9698999999999995e-05 2024-09-14T22:37:43.391 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.391 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:43.391 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.391 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82542 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:43.391 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:37:43.391 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.931861388000002, 2024-09-14T22:37:43.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060367430999999999, 2024-09-14T22:37:43.391 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:43.391 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:43.391 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:43.392 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:43.392 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:43.392 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:43.392 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82542 2024-09-14T22:37:43.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.392 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:43.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:43.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:37:43.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:43.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:37:43.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:43.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419534+0000", 2024-09-14T22:37:43.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T22:37:43.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:43.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419542+0000", 2024-09-14T22:37:43.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8199999999999997e-06 2024-09-14T22:37:43.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:43.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419559+0000", 2024-09-14T22:37:43.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6852999999999999e-05 2024-09-14T22:37:43.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:43.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419571+0000", 2024-09-14T22:37:43.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2364000000000001e-05 2024-09-14T22:37:43.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479686+0000", 2024-09-14T22:37:43.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060115522999999997 2024-09-14T22:37:43.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:43.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479795+0000", 2024-09-14T22:37:43.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010831399999999999 2024-09-14T22:37:43.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:43.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479900+0000", 2024-09-14T22:37:43.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105284 2024-09-14T22:37:43.395 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.395 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:43.395 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.395 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82544 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:43.395 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:37:43.396 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.931807652000003, 2024-09-14T22:37:43.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060590515999999997, 2024-09-14T22:37:43.396 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:43.396 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:43.396 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:43.396 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:43.396 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:43.396 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:43.396 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82544 2024-09-14T22:37:43.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.396 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:43.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:43.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:37:43.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:43.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:37:43.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:43.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419588+0000", 2024-09-14T22:37:43.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.35e-06 2024-09-14T22:37:43.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:43.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419595+0000", 2024-09-14T22:37:43.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4030000000000001e-06 2024-09-14T22:37:43.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:43.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419612+0000", 2024-09-14T22:37:43.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6670000000000001e-05 2024-09-14T22:37:43.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:43.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419624+0000", 2024-09-14T22:37:43.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-05 2024-09-14T22:37:43.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479957+0000", 2024-09-14T22:37:43.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060333287999999999 2024-09-14T22:37:43.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:43.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480074+0000", 2024-09-14T22:37:43.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116443 2024-09-14T22:37:43.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:43.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480177+0000", 2024-09-14T22:37:43.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000103102 2024-09-14T22:37:43.400 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.400 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:43.400 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.400 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82546 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:43.400 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:37:43.400 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.931754732999998, 2024-09-14T22:37:43.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060795428999999998, 2024-09-14T22:37:43.400 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:43.400 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:43.400 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:43.400 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:43.400 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:43.401 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:43.401 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82546 2024-09-14T22:37:43.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.401 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:43.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:43.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:37:43.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:43.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:37:43.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:43.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419640+0000", 2024-09-14T22:37:43.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.254e-06 2024-09-14T22:37:43.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:43.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419647+0000", 2024-09-14T22:37:43.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9e-06 2024-09-14T22:37:43.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:43.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419664+0000", 2024-09-14T22:37:43.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6345999999999999e-05 2024-09-14T22:37:43.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:43.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419676+0000", 2024-09-14T22:37:43.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.224e-05 2024-09-14T22:37:43.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480218+0000", 2024-09-14T22:37:43.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060542133999999997 2024-09-14T22:37:43.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:43.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480336+0000", 2024-09-14T22:37:43.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117472 2024-09-14T22:37:43.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:43.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480435+0000", 2024-09-14T22:37:43.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9083e-05 2024-09-14T22:37:43.404 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.404 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:43.404 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.404 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82548 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:43.404 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:37:43.405 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.931702723000001, 2024-09-14T22:37:43.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061011636000000001, 2024-09-14T22:37:43.405 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:43.405 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:43.405 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:43.405 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:43.405 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:43.405 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:43.405 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82548 2024-09-14T22:37:43.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.405 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:43.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:43.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:37:43.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:43.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:37:43.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:43.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419693+0000", 2024-09-14T22:37:43.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.387e-06 2024-09-14T22:37:43.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:43.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419699+0000", 2024-09-14T22:37:43.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8730000000000001e-06 2024-09-14T22:37:43.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:43.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419716+0000", 2024-09-14T22:37:43.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6416e-05 2024-09-14T22:37:43.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:43.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419728+0000", 2024-09-14T22:37:43.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2597e-05 2024-09-14T22:37:43.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480476+0000", 2024-09-14T22:37:43.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060747989000000002 2024-09-14T22:37:43.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:43.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480583+0000", 2024-09-14T22:37:43.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010638500000000001 2024-09-14T22:37:43.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:43.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480703+0000", 2024-09-14T22:37:43.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011998900000000001 2024-09-14T22:37:43.408 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.408 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:43.408 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.409 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82550 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1982464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:43.409 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:37:43.409 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.931649970000002, 2024-09-14T22:37:43.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061295483999999997, 2024-09-14T22:37:43.409 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:43.409 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:43.409 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:43.409 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:43.409 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:43.409 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:43.409 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82550 2024-09-14T22:37:43.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.410 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:43.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:43.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:37:43.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:43.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:37:43.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:43.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419745+0000", 2024-09-14T22:37:43.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.406e-06 2024-09-14T22:37:43.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:43.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419753+0000", 2024-09-14T22:37:43.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8140000000000001e-06 2024-09-14T22:37:43.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:43.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419770+0000", 2024-09-14T22:37:43.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6719999999999999e-05 2024-09-14T22:37:43.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:43.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419782+0000", 2024-09-14T22:37:43.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2252999999999999e-05 2024-09-14T22:37:43.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480745+0000", 2024-09-14T22:37:43.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060962385000000001 2024-09-14T22:37:43.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:43.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480900+0000", 2024-09-14T22:37:43.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015513899999999999 2024-09-14T22:37:43.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:43.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481039+0000", 2024-09-14T22:37:43.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013976700000000001 2024-09-14T22:37:43.413 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.413 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:43.413 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.413 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82588 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1064960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:43.413 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:37:43.413 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.931596079999998, 2024-09-14T22:37:43.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061863732999999997, 2024-09-14T22:37:43.413 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:43.413 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:43.414 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:43.414 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:43.414 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:43.414 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:43.414 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82588 2024-09-14T22:37:43.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.414 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:43.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:43.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:37:43.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:43.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:37:43.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:43.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419799+0000", 2024-09-14T22:37:43.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4160000000000001e-06 2024-09-14T22:37:43.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:43.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419807+0000", 2024-09-14T22:37:43.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2169999999999994e-06 2024-09-14T22:37:43.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:43.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419824+0000", 2024-09-14T22:37:43.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6776e-05 2024-09-14T22:37:43.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:43.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419837+0000", 2024-09-14T22:37:43.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2680000000000001e-05 2024-09-14T22:37:43.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481395+0000", 2024-09-14T22:37:43.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061558416999999997 2024-09-14T22:37:43.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:43.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481532+0000", 2024-09-14T22:37:43.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000136861 2024-09-14T22:37:43.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:43.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481662+0000", 2024-09-14T22:37:43.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012936600000000001 2024-09-14T22:37:43.417 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.417 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:43.417 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.418 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82589 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:43.418 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:37:43.418 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.931541484999997, 2024-09-14T22:37:43.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062064592000000002, 2024-09-14T22:37:43.418 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:43.418 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:43.418 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:43.418 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:43.418 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:43.418 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:43.418 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82589 2024-09-14T22:37:43.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.418 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:43.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:43.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:37:43.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:43.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:37:43.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:43.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419854+0000", 2024-09-14T22:37:43.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.367e-06 2024-09-14T22:37:43.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:43.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419862+0000", 2024-09-14T22:37:43.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8930000000000005e-06 2024-09-14T22:37:43.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:43.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419896+0000", 2024-09-14T22:37:43.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4134000000000002e-05 2024-09-14T22:37:43.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:43.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419909+0000", 2024-09-14T22:37:43.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3332999999999999e-05 2024-09-14T22:37:43.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481702+0000", 2024-09-14T22:37:43.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061792947000000001 2024-09-14T22:37:43.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:43.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481808+0000", 2024-09-14T22:37:43.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105655 2024-09-14T22:37:43.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:43.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481917+0000", 2024-09-14T22:37:43.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109263 2024-09-14T22:37:43.422 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.422 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:43.422 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.422 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82590 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1359872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:43.422 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:37:43.422 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.931467877999999, 2024-09-14T22:37:43.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062362895000000002, 2024-09-14T22:37:43.422 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:43.422 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:43.422 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:43.422 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:43.423 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:43.423 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:43.423 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82590 2024-09-14T22:37:43.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.423 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:43.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:43.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:37:43.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:43.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:37:43.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:43.424 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419928+0000", 2024-09-14T22:37:43.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4470000000000001e-06 2024-09-14T22:37:43.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:43.424 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419942+0000", 2024-09-14T22:37:43.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4229e-05 2024-09-14T22:37:43.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:43.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419972+0000", 2024-09-14T22:37:43.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.065e-05 2024-09-14T22:37:43.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:43.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419985+0000", 2024-09-14T22:37:43.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2903e-05 2024-09-14T22:37:43.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481960+0000", 2024-09-14T22:37:43.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061974435000000001 2024-09-14T22:37:43.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:43.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482106+0000", 2024-09-14T22:37:43.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014643800000000001 2024-09-14T22:37:43.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:43.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482289+0000", 2024-09-14T22:37:43.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000182793 2024-09-14T22:37:43.426 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.426 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:43.426 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.427 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82592 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:43.427 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:37:43.427 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.931391656000002, 2024-09-14T22:37:43.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062631507000000003, 2024-09-14T22:37:43.427 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:43.427 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:43.427 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:43.427 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:43.427 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:43.427 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:43.427 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82592 2024-09-14T22:37:43.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.427 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:43.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:43.428 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:37:43.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.428 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.428 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:43.428 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:37:43.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.428 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.428 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:43.428 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420017+0000", 2024-09-14T22:37:43.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4744e-05 2024-09-14T22:37:43.428 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:43.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420036+0000", 2024-09-14T22:37:43.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8680000000000001e-05 2024-09-14T22:37:43.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:43.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420054+0000", 2024-09-14T22:37:43.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7799999999999999e-05 2024-09-14T22:37:43.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:43.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420067+0000", 2024-09-14T22:37:43.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3013000000000001e-05 2024-09-14T22:37:43.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482358+0000", 2024-09-14T22:37:43.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062291445000000001 2024-09-14T22:37:43.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:43.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482469+0000", 2024-09-14T22:37:43.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110608 2024-09-14T22:37:43.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:43.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482634+0000", 2024-09-14T22:37:43.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000165217 2024-09-14T22:37:43.431 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.431 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:43.431 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.431 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82594 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1654784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:43.431 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:37:43.431 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.931293680000003, 2024-09-14T22:37:43.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062893849000000002, 2024-09-14T22:37:43.431 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:43.431 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:43.431 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:43.431 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:43.432 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:43.432 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:43.432 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82594 2024-09-14T22:37:43.432 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.432 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:43.432 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.432 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:43.432 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:37:43.432 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.432 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.432 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.432 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:43.432 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:37:43.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.433 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.433 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:43.433 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420102+0000", 2024-09-14T22:37:43.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5e-06 2024-09-14T22:37:43.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.433 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.433 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:43.433 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420115+0000", 2024-09-14T22:37:43.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.324e-05 2024-09-14T22:37:43.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.433 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.433 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:43.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420133+0000", 2024-09-14T22:37:43.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8213e-05 2024-09-14T22:37:43.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:43.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420147+0000", 2024-09-14T22:37:43.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3284e-05 2024-09-14T22:37:43.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482688+0000", 2024-09-14T22:37:43.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062541425999999997 2024-09-14T22:37:43.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.435 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:43.435 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482794+0000", 2024-09-14T22:37:43.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010575500000000001 2024-09-14T22:37:43.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.435 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:43.435 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482994+0000", 2024-09-14T22:37:43.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020043099999999999 2024-09-14T22:37:43.435 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.435 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:43.435 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.435 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82596 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:43.436 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:37:43.436 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.931230380000002, 2024-09-14T22:37:43.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063187238000000007, 2024-09-14T22:37:43.436 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:43.436 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:43.436 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:43.436 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:43.436 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:43.436 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:43.436 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82596 2024-09-14T22:37:43.436 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.436 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:43.436 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.437 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:43.437 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:37:43.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.437 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.437 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:43.437 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:37:43.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.437 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.437 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:43.437 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420165+0000", 2024-09-14T22:37:43.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4100000000000001e-06 2024-09-14T22:37:43.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:43.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420173+0000", 2024-09-14T22:37:43.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8129999999999996e-06 2024-09-14T22:37:43.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:43.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420190+0000", 2024-09-14T22:37:43.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7076999999999999e-05 2024-09-14T22:37:43.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:43.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420203+0000", 2024-09-14T22:37:43.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2683e-05 2024-09-14T22:37:43.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483060+0000", 2024-09-14T22:37:43.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062857114000000006 2024-09-14T22:37:43.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:43.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483167+0000", 2024-09-14T22:37:43.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010713999999999999 2024-09-14T22:37:43.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:43.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483351+0000", 2024-09-14T22:37:43.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018400099999999999 2024-09-14T22:37:43.440 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.440 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:43.440 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.440 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82597 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1949696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:43.440 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:37:43.440 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.931173553999997, 2024-09-14T22:37:43.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063466513000000002, 2024-09-14T22:37:43.440 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:43.440 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:43.440 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:43.440 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:43.441 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:43.441 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:43.441 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82597 2024-09-14T22:37:43.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.441 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:43.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:43.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:37:43.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:43.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:37:43.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:43.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420221+0000", 2024-09-14T22:37:43.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.043e-06 2024-09-14T22:37:43.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:43.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420229+0000", 2024-09-14T22:37:43.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0469999999999994e-06 2024-09-14T22:37:43.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:43.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420246+0000", 2024-09-14T22:37:43.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6339999999999999e-05 2024-09-14T22:37:43.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.443 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:43.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420258+0000", 2024-09-14T22:37:43.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2296000000000001e-05 2024-09-14T22:37:43.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.443 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483402+0000", 2024-09-14T22:37:43.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063144318000000005 2024-09-14T22:37:43.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.443 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:43.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483530+0000", 2024-09-14T22:37:43.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012713199999999999 2024-09-14T22:37:43.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.444 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:43.444 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483687+0000", 2024-09-14T22:37:43.444 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015733699999999999 2024-09-14T22:37:43.444 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.444 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:43.444 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.444 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.444 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:43.444 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:43.446 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:37:43.446 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:43.443+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:43.546 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:43.538+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:43.615 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:43.616 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:43.616 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:37:43.616 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:43.628 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:37:43.671 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:43.666+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:43.686 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:37:43.748 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:43.747+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:43.793 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:43.794 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:43.794 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:37:43.794 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:37:43.794 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:43.808 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:37:43.837 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:37:43.837 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:37:43.851 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:43.846+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:43.856 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:37:43.856 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:37:43.875 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:37:43.875 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:37:43.877 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T22:37:43.894 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:37:43.894 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:37:43.913 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:37:43.913 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:37:43.937 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:37:43.937 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:37:43.943 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:43.943 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:43.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:43.943 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:43.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.943 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:37:43.943 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:37:43.943 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 103.93221665900001, 2024-09-14T22:37:43.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.398029152, 2024-09-14T22:37:43.944 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:43.944 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:43.944 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:43.944 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:43.944 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:37:43.944 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:37:43.944 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 1 2024-09-14T22:37:43.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.944 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:43.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:43.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:37:43.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:43.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:37:43.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:43.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006183+0000", 2024-09-14T22:37:43.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9059999999999998e-06 2024-09-14T22:37:43.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:43.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006198+0000", 2024-09-14T22:37:43.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.523e-05 2024-09-14T22:37:43.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.946 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:43.946 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006219+0000", 2024-09-14T22:37:43.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1342999999999999e-05 2024-09-14T22:37:43.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.946 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:43.946 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006240+0000", 2024-09-14T22:37:43.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1186999999999999e-05 2024-09-14T22:37:43.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.946 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006363+0000", 2024-09-14T22:37:43.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012258200000000001 2024-09-14T22:37:43.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:37:43.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006392+0000", 2024-09-14T22:37:43.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9607e-05 2024-09-14T22:37:43.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401084+0000", 2024-09-14T22:37:43.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.39469142699999998 2024-09-14T22:37:43.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:43.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401394+0000", 2024-09-14T22:37:43.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030987799999999998 2024-09-14T22:37:43.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:37:43.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401860+0000", 2024-09-14T22:37:43.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00046654299999999998 2024-09-14T22:37:43.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:43.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.403386+0000", 2024-09-14T22:37:43.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001525322 2024-09-14T22:37:43.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:43.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404042+0000", 2024-09-14T22:37:43.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065676099999999996 2024-09-14T22:37:43.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:43.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404136+0000", 2024-09-14T22:37:43.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4032999999999999e-05 2024-09-14T22:37:43.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:43.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404209+0000", 2024-09-14T22:37:43.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2333e-05 2024-09-14T22:37:43.950 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.950 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:43.950 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.950 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104182 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14352384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:43.950 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:37:43.950 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.449362673, 2024-09-14T22:37:43.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071642496999999999, 2024-09-14T22:37:43.950 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:43.950 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:43.950 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:43.950 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:43.951 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:43.951 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:43.951 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104182 2024-09-14T22:37:43.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.951 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:43.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:43.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:37:43.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:43.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:37:43.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:43.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489035+0000", 2024-09-14T22:37:43.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3430000000000001e-06 2024-09-14T22:37:43.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:43.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489041+0000", 2024-09-14T22:37:43.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8139999999999997e-06 2024-09-14T22:37:43.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:43.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489059+0000", 2024-09-14T22:37:43.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8193e-05 2024-09-14T22:37:43.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:43.953 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489073+0000", 2024-09-14T22:37:43.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:37:43.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.953 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560434+0000", 2024-09-14T22:37:43.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071361463999999999 2024-09-14T22:37:43.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:43.953 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560545+0000", 2024-09-14T22:37:43.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110263 2024-09-14T22:37:43.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:43.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560676+0000", 2024-09-14T22:37:43.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013138000000000001 2024-09-14T22:37:43.954 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.954 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:43.954 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.954 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104183 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14483456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:43.954 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:37:43.954 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.449239353999999, 2024-09-14T22:37:43.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071781943000000001, 2024-09-14T22:37:43.955 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:43.955 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:43.955 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:43.955 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:43.955 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:43.955 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:43.955 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104183 2024-09-14T22:37:43.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.955 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:43.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:43.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:37:43.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:43.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:37:43.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:43.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489159+0000", 2024-09-14T22:37:43.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5629999999999999e-06 2024-09-14T22:37:43.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:43.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489165+0000", 2024-09-14T22:37:43.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7070000000000003e-06 2024-09-14T22:37:43.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.957 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:43.957 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489186+0000", 2024-09-14T22:37:43.957 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.088e-05 2024-09-14T22:37:43.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.957 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:43.957 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489201+0000", 2024-09-14T22:37:43.957 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5203e-05 2024-09-14T22:37:43.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.957 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.958 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560713+0000", 2024-09-14T22:37:43.958 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071511447000000006 2024-09-14T22:37:43.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.958 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.958 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:43.958 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560819+0000", 2024-09-14T22:37:43.958 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106579 2024-09-14T22:37:43.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.958 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.958 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:43.958 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560939+0000", 2024-09-14T22:37:43.958 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119564 2024-09-14T22:37:43.958 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.958 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:43.959 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.959 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.959 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.959 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104184 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:43.959 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:37:43.959 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.449174911, 2024-09-14T22:37:43.959 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071970107000000005, 2024-09-14T22:37:43.959 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:43.959 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:43.959 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:43.959 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:43.959 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:43.959 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:43.959 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104184 2024-09-14T22:37:43.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.960 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:43.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:43.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:37:43.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:43.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:37:43.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:43.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489223+0000", 2024-09-14T22:37:43.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4470000000000001e-06 2024-09-14T22:37:43.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.961 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:43.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489229+0000", 2024-09-14T22:37:43.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8200000000000002e-06 2024-09-14T22:37:43.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.961 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:43.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489247+0000", 2024-09-14T22:37:43.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8150000000000001e-05 2024-09-14T22:37:43.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:43.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489260+0000", 2024-09-14T22:37:43.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3570000000000001e-05 2024-09-14T22:37:43.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560979+0000", 2024-09-14T22:37:43.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071718831999999996 2024-09-14T22:37:43.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:43.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561075+0000", 2024-09-14T22:37:43.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5848999999999997e-05 2024-09-14T22:37:43.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.963 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:43.963 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561191+0000", 2024-09-14T22:37:43.963 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116439 2024-09-14T22:37:43.963 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.963 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:43.963 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.963 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.963 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.963 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104185 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14745600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:43.963 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:37:43.963 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.449115328000001, 2024-09-14T22:37:43.963 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072173750999999994, 2024-09-14T22:37:43.963 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:43.963 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:43.963 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:43.964 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:43.964 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:43.964 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:43.964 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104185 2024-09-14T22:37:43.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.964 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:43.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:43.964 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:37:43.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:43.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:37:43.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:43.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489282+0000", 2024-09-14T22:37:43.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2500000000000001e-06 2024-09-14T22:37:43.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:43.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489287+0000", 2024-09-14T22:37:43.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1359999999999996e-06 2024-09-14T22:37:43.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:43.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489304+0000", 2024-09-14T22:37:43.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7033000000000001e-05 2024-09-14T22:37:43.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:43.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489318+0000", 2024-09-14T22:37:43.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3934000000000001e-05 2024-09-14T22:37:43.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561232+0000", 2024-09-14T22:37:43.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071913759999999993 2024-09-14T22:37:43.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:43.967 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561340+0000", 2024-09-14T22:37:43.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107921 2024-09-14T22:37:43.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.967 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:43.967 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561455+0000", 2024-09-14T22:37:43.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011471700000000001 2024-09-14T22:37:43.967 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.967 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:43.967 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.967 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104186 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14876672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:43.967 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:37:43.967 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.449059419000001, 2024-09-14T22:37:43.968 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072393374999999996, 2024-09-14T22:37:43.968 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:43.968 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:43.968 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:43.968 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:43.968 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:43.968 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:43.968 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104186 2024-09-14T22:37:43.968 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.968 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:43.968 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.968 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:43.968 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:37:43.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:43.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:37:43.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:43.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489338+0000", 2024-09-14T22:37:43.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.48e-06 2024-09-14T22:37:43.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:43.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489366+0000", 2024-09-14T22:37:43.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7307000000000001e-05 2024-09-14T22:37:43.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:43.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489385+0000", 2024-09-14T22:37:43.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9283000000000002e-05 2024-09-14T22:37:43.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:43.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489399+0000", 2024-09-14T22:37:43.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.452e-05 2024-09-14T22:37:43.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561494+0000", 2024-09-14T22:37:43.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072094885999999997 2024-09-14T22:37:43.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:43.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561604+0000", 2024-09-14T22:37:43.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010925900000000001 2024-09-14T22:37:43.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:43.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561730+0000", 2024-09-14T22:37:43.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012663999999999999 2024-09-14T22:37:43.971 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.971 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:43.971 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.972 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104187 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15007744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:43.972 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:37:43.972 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.448976049999999, 2024-09-14T22:37:43.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072599143000000005, 2024-09-14T22:37:43.972 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:43.972 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:43.972 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:43.972 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:43.972 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:43.972 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:43.972 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104187 2024-09-14T22:37:43.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.972 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:43.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:43.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:37:43.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:43.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:37:43.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:43.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489422+0000", 2024-09-14T22:37:43.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5340000000000001e-06 2024-09-14T22:37:43.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:43.974 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489428+0000", 2024-09-14T22:37:43.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1430000000000003e-06 2024-09-14T22:37:43.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:43.974 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489445+0000", 2024-09-14T22:37:43.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7042999999999999e-05 2024-09-14T22:37:43.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:43.974 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489458+0000", 2024-09-14T22:37:43.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3474e-05 2024-09-14T22:37:43.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561771+0000", 2024-09-14T22:37:43.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072312600000000005 2024-09-14T22:37:43.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:43.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561885+0000", 2024-09-14T22:37:43.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011411799999999999 2024-09-14T22:37:43.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:43.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562019+0000", 2024-09-14T22:37:43.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013423100000000001 2024-09-14T22:37:43.975 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.975 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:43.976 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.976 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104188 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15138816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:43.976 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:37:43.976 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.448916126, 2024-09-14T22:37:43.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072824624000000004, 2024-09-14T22:37:43.976 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:43.976 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:43.976 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:43.976 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:43.976 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:43.976 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:43.976 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104188 2024-09-14T22:37:43.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.977 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:43.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:43.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:37:43.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:43.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:37:43.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:43.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489481+0000", 2024-09-14T22:37:43.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.22e-06 2024-09-14T22:37:43.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:43.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489487+0000", 2024-09-14T22:37:43.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6930000000000004e-06 2024-09-14T22:37:43.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:43.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489508+0000", 2024-09-14T22:37:43.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1135999999999999e-05 2024-09-14T22:37:43.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:43.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489522+0000", 2024-09-14T22:37:43.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3817e-05 2024-09-14T22:37:43.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.979 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.979 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562066+0000", 2024-09-14T22:37:43.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072543813999999998 2024-09-14T22:37:43.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.979 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:43.979 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562177+0000", 2024-09-14T22:37:43.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000111264 2024-09-14T22:37:43.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.979 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:43.979 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562305+0000", 2024-09-14T22:37:43.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012768 2024-09-14T22:37:43.980 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.980 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:43.980 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.980 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104189 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15269888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:43.980 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:37:43.980 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.448855677000001, 2024-09-14T22:37:43.980 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073061300999999995, 2024-09-14T22:37:43.980 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:43.980 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:43.980 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:43.980 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:43.980 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:43.980 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:43.981 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104189 2024-09-14T22:37:43.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.981 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:43.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.981 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:43.981 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:37:43.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.981 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:43.981 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:37:43.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.981 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:43.981 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489542+0000", 2024-09-14T22:37:43.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1659999999999999e-06 2024-09-14T22:37:43.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:43.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489561+0000", 2024-09-14T22:37:43.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.908e-05 2024-09-14T22:37:43.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:43.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489579+0000", 2024-09-14T22:37:43.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8536999999999999e-05 2024-09-14T22:37:43.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:43.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489593+0000", 2024-09-14T22:37:43.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3977e-05 2024-09-14T22:37:43.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562347+0000", 2024-09-14T22:37:43.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072754092000000006 2024-09-14T22:37:43.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:43.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562458+0000", 2024-09-14T22:37:43.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110722 2024-09-14T22:37:43.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:43.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562602+0000", 2024-09-14T22:37:43.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014372699999999999 2024-09-14T22:37:43.983 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.984 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:43.984 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.984 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.984 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104190 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:43.984 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:37:43.984 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.448783380999998, 2024-09-14T22:37:43.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.07326133, 2024-09-14T22:37:43.984 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:43.984 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:43.984 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:43.984 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:43.984 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:43.984 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:43.984 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104190 2024-09-14T22:37:43.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.985 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:43.985 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.985 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:43.985 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:37:43.985 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.985 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.985 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:43.985 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:37:43.985 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.985 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.985 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:43.985 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489614+0000", 2024-09-14T22:37:43.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1999999999999999e-06 2024-09-14T22:37:43.986 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.986 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:43.986 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489620+0000", 2024-09-14T22:37:43.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.524e-06 2024-09-14T22:37:43.986 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.986 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:43.986 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489637+0000", 2024-09-14T22:37:43.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7825999999999998e-05 2024-09-14T22:37:43.986 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.986 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:43.986 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489671+0000", 2024-09-14T22:37:43.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3726000000000002e-05 2024-09-14T22:37:43.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562644+0000", 2024-09-14T22:37:43.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072973089000000005 2024-09-14T22:37:43.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:43.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562762+0000", 2024-09-14T22:37:43.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011743600000000001 2024-09-14T22:37:43.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:43.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562874+0000", 2024-09-14T22:37:43.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011252900000000001 2024-09-14T22:37:43.988 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.988 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:43.988 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.988 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104191 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15532032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:43.988 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:37:43.988 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.448674958000002, 2024-09-14T22:37:43.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073311166999999997, 2024-09-14T22:37:43.988 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:43.988 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:43.988 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:43.988 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:43.988 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:43.989 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:43.989 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104191 2024-09-14T22:37:43.989 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.989 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:43.989 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.989 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:43.989 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:37:43.989 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.989 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.989 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.989 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:43.989 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:37:43.989 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.989 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.990 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:43.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489723+0000", 2024-09-14T22:37:43.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6330000000000001e-06 2024-09-14T22:37:43.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.990 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:43.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489730+0000", 2024-09-14T22:37:43.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2300000000000002e-06 2024-09-14T22:37:43.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.990 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:43.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489751+0000", 2024-09-14T22:37:43.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0849999999999999e-05 2024-09-14T22:37:43.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.991 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:43.991 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489766+0000", 2024-09-14T22:37:43.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.524e-05 2024-09-14T22:37:43.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.991 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.991 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.991 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562896+0000", 2024-09-14T22:37:43.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073129662999999998 2024-09-14T22:37:43.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.991 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.991 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:43.991 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562960+0000", 2024-09-14T22:37:43.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3689999999999995e-05 2024-09-14T22:37:43.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.991 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.991 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:43.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563033+0000", 2024-09-14T22:37:43.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2861000000000006e-05 2024-09-14T22:37:43.992 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.992 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:43.992 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.992 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104192 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:43.992 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:37:43.992 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.448579153000001, 2024-09-14T22:37:43.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073463632000000001, 2024-09-14T22:37:43.992 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:43.992 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:43.992 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:43.993 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:43.993 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:43.993 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:43.993 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104192 2024-09-14T22:37:43.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.993 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:43.993 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.993 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:43.993 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:37:43.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.993 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.993 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:43.993 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:37:43.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:43.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489819+0000", 2024-09-14T22:37:43.994 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.424e-06 2024-09-14T22:37:43.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:43.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489825+0000", 2024-09-14T22:37:43.994 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.02e-06 2024-09-14T22:37:43.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:43.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489844+0000", 2024-09-14T22:37:43.994 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9760000000000001e-05 2024-09-14T22:37:43.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.995 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:43.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489860+0000", 2024-09-14T22:37:43.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5563e-05 2024-09-14T22:37:43.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.995 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563070+0000", 2024-09-14T22:37:43.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073209724000000004 2024-09-14T22:37:43.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.995 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:43.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563175+0000", 2024-09-14T22:37:43.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105183 2024-09-14T22:37:43.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.996 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:43.996 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563281+0000", 2024-09-14T22:37:43.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105958 2024-09-14T22:37:43.996 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.996 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:43.996 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:43.996 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.996 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104193 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15794176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:43.996 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:37:43.996 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.448516986000001, 2024-09-14T22:37:43.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073600040000000005, 2024-09-14T22:37:43.996 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:43.997 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:43.997 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:43.997 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:43.997 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:43.997 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:43.997 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104193 2024-09-14T22:37:43.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.997 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:43.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.997 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:43.997 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:37:43.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:43.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:37:43.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:43.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:43.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489881+0000", 2024-09-14T22:37:43.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5460000000000001e-06 2024-09-14T22:37:43.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:43.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489886+0000", 2024-09-14T22:37:43.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1739999999999999e-06 2024-09-14T22:37:43.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:43.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489912+0000", 2024-09-14T22:37:43.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6461999999999998e-05 2024-09-14T22:37:43.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:43.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489927+0000", 2024-09-14T22:37:43.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:37:43.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:43.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:43.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:43.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563310+0000", 2024-09-14T22:37:43.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073382977000000002 2024-09-14T22:37:43.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.000 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:44.000 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563385+0000", 2024-09-14T22:37:44.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5218999999999999e-05 2024-09-14T22:37:44.000 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.000 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:44.000 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563479+0000", 2024-09-14T22:37:44.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4622000000000003e-05 2024-09-14T22:37:44.000 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:44.000 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:44.000 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:44.000 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.001 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104194 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15925248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:44.001 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:37:44.001 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.448450393999998, 2024-09-14T22:37:44.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073745434999999998, 2024-09-14T22:37:44.001 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:44.001 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:44.001 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:44.001 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:44.001 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:44.001 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:44.001 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104194 2024-09-14T22:37:44.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.001 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:44.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:44.002 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:37:44.002 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:44.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.002 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.002 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:44.002 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:37:44.002 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:44.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.002 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.002 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:44.002 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489947+0000", 2024-09-14T22:37:44.002 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.327e-06 2024-09-14T22:37:44.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.002 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:44.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489953+0000", 2024-09-14T22:37:44.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9329999999999998e-06 2024-09-14T22:37:44.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:44.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489971+0000", 2024-09-14T22:37:44.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7994000000000001e-05 2024-09-14T22:37:44.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:44.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489985+0000", 2024-09-14T22:37:44.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4226000000000001e-05 2024-09-14T22:37:44.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:44.004 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563510+0000", 2024-09-14T22:37:44.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073524667000000002 2024-09-14T22:37:44.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:44.004 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563597+0000", 2024-09-14T22:37:44.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6550000000000003e-05 2024-09-14T22:37:44.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:44.004 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563691+0000", 2024-09-14T22:37:44.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4738e-05 2024-09-14T22:37:44.004 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:44.004 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:44.005 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:44.005 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.005 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.005 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104195 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:44.005 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:37:44.005 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.448392197, 2024-09-14T22:37:44.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073876598000000002, 2024-09-14T22:37:44.005 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:44.005 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:44.005 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:44.005 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:44.005 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:44.005 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:44.005 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104195 2024-09-14T22:37:44.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.006 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:44.006 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.006 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:44.006 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:37:44.006 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:44.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.006 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.006 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:44.006 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:37:44.006 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:44.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.006 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.006 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:44.006 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490006+0000", 2024-09-14T22:37:44.006 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4699999999999999e-06 2024-09-14T22:37:44.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.007 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:44.007 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490011+0000", 2024-09-14T22:37:44.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.366e-06 2024-09-14T22:37:44.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.007 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:44.007 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490046+0000", 2024-09-14T22:37:44.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5496e-05 2024-09-14T22:37:44.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.007 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:44.007 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490061+0000", 2024-09-14T22:37:44.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5054e-05 2024-09-14T22:37:44.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:44.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563719+0000", 2024-09-14T22:37:44.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073657412000000005 2024-09-14T22:37:44.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:44.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563792+0000", 2024-09-14T22:37:44.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3319999999999996e-05 2024-09-14T22:37:44.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:44.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563881+0000", 2024-09-14T22:37:44.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8480000000000007e-05 2024-09-14T22:37:44.008 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:44.009 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:44.009 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:44.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.009 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104196 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16187392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:44.009 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:37:44.009 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.448315048000001, 2024-09-14T22:37:44.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073997496999999995, 2024-09-14T22:37:44.009 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:44.009 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:44.009 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:44.009 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:44.009 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:44.009 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:44.010 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104196 2024-09-14T22:37:44.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.010 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:44.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.010 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:44.010 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:37:44.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:44.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.010 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:44.010 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:37:44.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:44.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.011 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:44.011 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490083+0000", 2024-09-14T22:37:44.011 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.347e-06 2024-09-14T22:37:44.011 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.011 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.011 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:44.011 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490089+0000", 2024-09-14T22:37:44.011 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9630000000000003e-06 2024-09-14T22:37:44.011 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.011 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.011 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:44.011 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490107+0000", 2024-09-14T22:37:44.011 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8859999999999999e-05 2024-09-14T22:37:44.011 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.011 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:44.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490122+0000", 2024-09-14T22:37:44.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.432e-05 2024-09-14T22:37:44.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:44.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563910+0000", 2024-09-14T22:37:44.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073788753999999998 2024-09-14T22:37:44.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:44.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563988+0000", 2024-09-14T22:37:44.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7862999999999996e-05 2024-09-14T22:37:44.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:44.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564079+0000", 2024-09-14T22:37:44.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0389999999999993e-05 2024-09-14T22:37:44.013 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:44.013 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:44.013 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:44.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.013 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104197 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16318464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:44.013 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:37:44.013 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.448223419000001, 2024-09-14T22:37:44.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074103547000000006, 2024-09-14T22:37:44.013 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:44.014 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:44.014 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:44.014 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:44.014 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:44.014 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:44.014 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104197 2024-09-14T22:37:44.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.014 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:44.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:44.014 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:37:44.014 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:44.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:44.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:37:44.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:44.015 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.015 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.015 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:44.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490174+0000", 2024-09-14T22:37:44.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.37e-06 2024-09-14T22:37:44.015 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.015 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.015 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:44.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490197+0000", 2024-09-14T22:37:44.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3017e-05 2024-09-14T22:37:44.015 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.015 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.015 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:44.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490219+0000", 2024-09-14T22:37:44.016 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.181e-05 2024-09-14T22:37:44.016 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.016 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.016 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:44.016 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490240+0000", 2024-09-14T22:37:44.016 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0786000000000001e-05 2024-09-14T22:37:44.016 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.016 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.016 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:44.016 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564109+0000", 2024-09-14T22:37:44.016 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073869031000000002 2024-09-14T22:37:44.016 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.016 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.016 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:44.016 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564186+0000", 2024-09-14T22:37:44.016 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6823999999999995e-05 2024-09-14T22:37:44.017 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:44.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564276+0000", 2024-09-14T22:37:44.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0709000000000002e-05 2024-09-14T22:37:44.017 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:44.017 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:44.017 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:44.017 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.017 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104198 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16449536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:44.017 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:37:44.017 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.448131183000001, 2024-09-14T22:37:44.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074167693000000007, 2024-09-14T22:37:44.017 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:44.018 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:44.018 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:44.018 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:44.018 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:44.018 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:44.018 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104198 2024-09-14T22:37:44.018 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.018 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:44.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.018 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:44.018 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:37:44.018 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:44.018 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.018 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:44.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:37:44.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:44.019 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.019 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.019 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:44.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490266+0000", 2024-09-14T22:37:44.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3599999999999999e-06 2024-09-14T22:37:44.019 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.019 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.019 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:44.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490272+0000", 2024-09-14T22:37:44.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9599999999999997e-06 2024-09-14T22:37:44.019 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.019 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.019 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:44.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490291+0000", 2024-09-14T22:37:44.020 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8485999999999999e-05 2024-09-14T22:37:44.020 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.020 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.020 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:44.020 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490305+0000", 2024-09-14T22:37:44.020 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4327e-05 2024-09-14T22:37:44.020 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.020 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.020 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:44.020 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564306+0000", 2024-09-14T22:37:44.020 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074000745000000007 2024-09-14T22:37:44.020 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.020 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.020 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:44.020 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564367+0000", 2024-09-14T22:37:44.021 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1501999999999995e-05 2024-09-14T22:37:44.021 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.021 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.021 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:44.021 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564433+0000", 2024-09-14T22:37:44.021 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5313000000000003e-05 2024-09-14T22:37:44.021 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:44.021 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:44.021 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:44.021 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.021 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.021 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104199 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16580608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:44.021 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:37:44.021 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.44804963, 2024-09-14T22:37:44.021 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074231631000000006, 2024-09-14T22:37:44.021 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:44.022 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:44.022 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:44.022 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:44.022 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:44.022 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:44.022 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104199 2024-09-14T22:37:44.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.022 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:44.022 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:44.022 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:37:44.022 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:44.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.022 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:44.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:37:44.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:44.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:44.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490349+0000", 2024-09-14T22:37:44.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9429999999999999e-06 2024-09-14T22:37:44.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:44.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490355+0000", 2024-09-14T22:37:44.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1670000000000004e-06 2024-09-14T22:37:44.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:44.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490374+0000", 2024-09-14T22:37:44.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9613e-05 2024-09-14T22:37:44.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.024 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:44.024 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490405+0000", 2024-09-14T22:37:44.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0936000000000003e-05 2024-09-14T22:37:44.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.024 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:44.024 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564451+0000", 2024-09-14T22:37:44.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074045266999999998 2024-09-14T22:37:44.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.024 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:44.024 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564494+0000", 2024-09-14T22:37:44.025 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3133000000000002e-05 2024-09-14T22:37:44.025 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.025 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.025 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:44.025 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564578+0000", 2024-09-14T22:37:44.025 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4572000000000003e-05 2024-09-14T22:37:44.025 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:44.025 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:44.025 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:44.025 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.025 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.025 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104200 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:44.025 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:37:44.026 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.446254402000001, 2024-09-14T22:37:44.026 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072582435000000001, 2024-09-14T22:37:44.026 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:44.026 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:44.026 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:44.026 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:44.026 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:44.026 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:44.026 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104200 2024-09-14T22:37:44.026 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.026 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:44.026 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.026 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:44.026 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:37:44.026 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:44.027 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.027 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:44.027 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:37:44.027 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:44.027 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.027 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:44.027 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492144+0000", 2024-09-14T22:37:44.027 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3070000000000001e-06 2024-09-14T22:37:44.027 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.027 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:44.027 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492155+0000", 2024-09-14T22:37:44.028 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0573e-05 2024-09-14T22:37:44.028 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.028 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.028 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:44.028 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492185+0000", 2024-09-14T22:37:44.028 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.008e-05 2024-09-14T22:37:44.028 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.028 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.028 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:44.028 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492205+0000", 2024-09-14T22:37:44.028 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9882999999999999e-05 2024-09-14T22:37:44.028 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.028 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.029 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:44.029 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564609+0000", 2024-09-14T22:37:44.029 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072404181999999997 2024-09-14T22:37:44.029 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.029 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.029 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:44.029 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564667+0000", 2024-09-14T22:37:44.029 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8102e-05 2024-09-14T22:37:44.029 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:44.029 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:44.029 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:44.029 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564724+0000", 2024-09-14T22:37:44.029 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7308000000000001e-05 2024-09-14T22:37:44.029 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:44.030 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:44.030 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:44.030 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:44.030 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:44.030 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:44.031 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone29 from 0xc7419c1 (overlap 0x3790d) to 0x0 2024-09-14T22:37:44.031 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:37:44.032 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:37:44.032 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:37:44.033 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:37:44.033 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:44.026+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:44.034 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:37:44.034 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-14T22:37:44.133 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:44.131+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:44.200 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:44.201 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:44.217 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:37:44.234 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:44.226+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:44.246 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:44.246 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:44.264 INFO:teuthology.orchestra.run.smithi001.stdout:4898 write 0xc9f94ce thru 0xca028aa (0x93dd bytes) 2024-09-14T22:37:44.283 INFO:teuthology.orchestra.run.smithi001.stdout:4899 writesame 0xc32c852 thru 0xc339a9e data_size 0x1(0xd24d bytes) 2024-09-14T22:37:44.320 INFO:teuthology.orchestra.run.smithi001.stdout:4900 read 0x787336c thru 0x787e111 (0xada6 bytes) 2024-09-14T22:37:44.320 INFO:teuthology.orchestra.run.smithi001.stdout:4901 read 0xb30bfdb thru 0xb318db1 (0xcdd7 bytes) 2024-09-14T22:37:44.321 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:44.329 INFO:teuthology.orchestra.run.smithi001.stdout:4903 trunc from 0xca028ab to 0x7611b94 2024-09-14T22:37:44.334 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:44.326+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:44.378 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:44.378 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:44.382 INFO:teuthology.orchestra.run.smithi001.stdout:4904 write 0x55cfddc thru 0x55d97c2 (0x99e7 bytes) 2024-09-14T22:37:44.397 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:37:44.440 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:44.439+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:44.492 INFO:teuthology.orchestra.run.smithi001.stdout:4905 write 0xdc34b07 thru 0xdc3d7df (0x8cd9 bytes) 2024-09-14T22:37:44.509 INFO:teuthology.orchestra.run.smithi001.stdout:4906 read 0x9122a53 thru 0x913236b (0xf919 bytes) 2024-09-14T22:37:44.509 INFO:teuthology.orchestra.run.smithi001.stdout:4908 punch from 0x57df3bd to 0x57ed039, (0xdc7c bytes) 2024-09-14T22:37:44.509 INFO:teuthology.orchestra.run.smithi001.stdout:4910 read 0x766f8d thru 0x76f4d4 (0x8548 bytes) 2024-09-14T22:37:44.510 INFO:teuthology.orchestra.run.smithi001.stdout:4911 write 0xb1815b3 thru 0xb18c2c8 (0xad16 bytes) 2024-09-14T22:37:44.529 INFO:teuthology.orchestra.run.smithi001.stdout:4913 write 0x2273fc3 thru 0x227a2fe (0x633c bytes) 2024-09-14T22:37:44.536 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:44.531+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:44.555 INFO:teuthology.orchestra.run.smithi001.stdout:4914 read 0x89ea4ba thru 0x89eaa2c (0x573 bytes) 2024-09-14T22:37:44.555 INFO:teuthology.orchestra.run.smithi001.stdout:4915 write 0x5443ddc thru 0x544a894 (0x6ab9 bytes) 2024-09-14T22:37:44.559 INFO:teuthology.orchestra.run.smithi001.stdout:4918 read 0xd07fb23 thru 0xd080499 (0x977 bytes) 2024-09-14T22:37:44.560 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:44.560 INFO:teuthology.orchestra.run.smithi001.stdout:4920 writesame 0x7cb64f thru 0x7d8ca5 data_size 0x1(0xd657 bytes) 2024-09-14T22:37:44.596 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:44.596 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:44.616 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:37:44.637 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:44.630+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:44.738 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:44.735+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:44.776 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:44.776 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:44.790 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:37:44.828 INFO:teuthology.orchestra.run.smithi001.stdout:4921 punch from 0x16765de to 0x168050b, (0x9f2d bytes) 2024-09-14T22:37:44.828 INFO:teuthology.orchestra.run.smithi001.stdout:4923 read 0x4b93b2a thru 0x4b9bcb9 (0x8190 bytes) 2024-09-14T22:37:44.839 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:44.830+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:44.848 INFO:teuthology.orchestra.run.smithi001.stdout:4924 trunc from 0xdc3d7e0 to 0x3cd016c 2024-09-14T22:37:44.863 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2024-09-14T22:37:44.939 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:44.934+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:44.970 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:44.971 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:44.987 INFO:teuthology.orchestra.run.smithi001.stdout:4925 read 0x1a56795 thru 0x1a63a21 (0xd28d bytes) 2024-09-14T22:37:44.987 INFO:teuthology.orchestra.run.smithi001.stdout:4926 writesame 0xb16d9a thru 0xb1741b data_size 0x11(0x682 bytes) 2024-09-14T22:37:44.988 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:37:45.006 INFO:teuthology.orchestra.run.smithi001.stdout:4928 writesame 0x8f8e635 thru 0x8f98fed data_size 0x7(0xa9b9 bytes) 2024-09-14T22:37:45.006 INFO:teuthology.orchestra.run.smithi001.stdout:4929 read 0x142a46c thru 0x1436610 (0xc1a5 bytes) 2024-09-14T22:37:45.018 INFO:teuthology.orchestra.run.smithi001.stdout:4931 writesame 0xb16ff5c thru 0xb17cd7b data_size 0x44(0xce20 bytes) 2024-09-14T22:37:45.018 INFO:teuthology.orchestra.run.smithi001.stdout:4934 write 0x5ced49a thru 0x5cf61eb (0x8d52 bytes) 2024-09-14T22:37:45.040 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:45.034+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:45.046 INFO:teuthology.orchestra.run.smithi001.stdout:4935 write 0xe922a21 thru 0xe92cc5c (0xa23c bytes) 2024-09-14T22:37:45.059 INFO:teuthology.orchestra.run.smithi001.stdout:4936 read 0x8785ef6 thru 0x878ac1a (0x4d25 bytes) 2024-09-14T22:37:45.060 INFO:teuthology.orchestra.run.smithi001.stdout:4938 read 0x1372ab0 thru 0x137cfd1 (0xa522 bytes) 2024-09-14T22:37:45.060 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:45.060 INFO:teuthology.orchestra.run.smithi001.stdout:4941 writesame 0x545664c thru 0x545eba9 data_size 0x2b(0x855e bytes) 2024-09-14T22:37:45.073 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:45.073 INFO:teuthology.orchestra.run.smithi001.stdout:4943 write 0x86deab5 thru 0x86e21e0 (0x372c bytes) 2024-09-14T22:37:45.083 INFO:teuthology.orchestra.run.smithi001.stdout:4944 writesame 0xda75f68 thru 0xda82fdc data_size 0xd(0xd075 bytes) 2024-09-14T22:37:45.102 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:45.102 INFO:teuthology.orchestra.run.smithi001.stdout:4946 write 0x98b84e1 thru 0x98bc7e7 (0x4307 bytes) 2024-09-14T22:37:45.118 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:45.119 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:45.120 INFO:teuthology.orchestra.run.smithi001.stdout:4947 trunc from 0xe92cc5d to 0x9db78be 2024-09-14T22:37:45.137 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:37:45.144 INFO:teuthology.orchestra.run.smithi001.stdout:4948 write 0x27a45a2 thru 0x27af2d8 (0xad37 bytes) 2024-09-14T22:37:45.144 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:45.143+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:45.182 INFO:teuthology.orchestra.run.smithi001.stdout:4949 write 0x64e8be9 thru 0x64f7edc (0xf2f4 bytes) 2024-09-14T22:37:45.200 INFO:teuthology.orchestra.run.smithi001.stdout:4950 read 0x7c05662 thru 0x7c0889a (0x3239 bytes) 2024-09-14T22:37:45.216 INFO:teuthology.orchestra.run.smithi001.stdout:4952 write 0xee4d318 thru 0xee5355f (0x6248 bytes) 2024-09-14T22:37:45.227 INFO:teuthology.orchestra.run.smithi001.stdout:4953 write 0xbf8f978 thru 0xbf9c20f (0xc898 bytes) 2024-09-14T22:37:45.242 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:45.234+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:45.249 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:45.249 INFO:teuthology.orchestra.run.smithi001.stdout:4957 read 0x985deb thru 0x98641e (0x634 bytes) 2024-09-14T22:37:45.250 INFO:teuthology.orchestra.run.smithi001.stdout:4958 read 0x87a1cfd thru 0x87a75bd (0x58c1 bytes) 2024-09-14T22:37:45.250 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:45.250 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:45.268 INFO:teuthology.orchestra.run.smithi001.stdout:4963 trunc from 0xee53560 to 0x739a047 2024-09-14T22:37:45.284 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:45.284 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:45.294 INFO:teuthology.orchestra.run.smithi001.stdout:4964 clone 31 order 24 su 131072 sc 8 2024-09-14T22:37:45.300 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:37:45.343 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:45.334+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:45.443 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:45.438+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:45.475 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:45.475 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:45.491 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:37:45.544 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:45.543+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:45.621 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:45.621 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:45.636 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:37:45.645 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:45.638+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:45.747 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:45.743+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:45.807 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:45.807 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:45.822 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:37:45.865 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:45.863+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:45.873 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-14T22:37:45.873 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:37:45.873 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-14T22:37:45.873 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T22:37:45.948 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:45.942+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:45.997 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:45.997 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:46.015 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:37:46.018 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone30 from 0x739a047 (overlap 0x3cd016c) to 0x36bb47a 2024-09-14T22:37:46.035 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:37:46.049 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:46.042+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:46.149 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:46.150 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:46.150 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:46.147+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:46.169 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:37:46.195 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-14T22:37:46.197 INFO:teuthology.orchestra.run.smithi001.stdout:4965 write 0x587a39a thru 0x5880609 (0x6270 bytes) 2024-09-14T22:37:46.221 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:37:46.221 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:37:46.250 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:46.242+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:46.251 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:37:46.251 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:37:46.284 INFO:teuthology.orchestra.run.smithi001.stdout:4966 read 0x30c1c1d thru 0x30cf4dd (0xd8c1 bytes) 2024-09-14T22:37:46.284 INFO:teuthology.orchestra.run.smithi001.stdout:4967 read 0x540dadd thru 0x541970a (0xbc2e bytes) 2024-09-14T22:37:46.285 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:37:46.286 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:37:46.294 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:37:46.294 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:46.294 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 94, 2024-09-14T22:37:46.294 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "f07f407a-b17d-42c4-bf73-92dd80114a4f", 2024-09-14T22:37:46.294 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-09-14T22:35:45.983055+0000", 2024-09-14T22:37:46.295 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-09-14T22:37:45.868688+0000", 2024-09-14T22:37:46.295 INFO:teuthology.orchestra.run.smithi001.stdout: "last_up_change": "2024-09-14T22:37:12.616042+0000", 2024-09-14T22:37:46.295 INFO:teuthology.orchestra.run.smithi001.stdout: "last_in_change": "2024-09-14T22:35:48.283160+0000", 2024-09-14T22:37:46.295 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T22:37:46.295 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_num": 5804032, 2024-09-14T22:37:46.295 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_set": [ 2024-09-14T22:37:46.295 INFO:teuthology.orchestra.run.smithi001.stdout: "nodeep-scrub", 2024-09-14T22:37:46.295 INFO:teuthology.orchestra.run.smithi001.stdout: "pglog_hardlimit", 2024-09-14T22:37:46.295 INFO:teuthology.orchestra.run.smithi001.stdout: "purged_snapdirs", 2024-09-14T22:37:46.295 INFO:teuthology.orchestra.run.smithi001.stdout: "recovery_deletes", 2024-09-14T22:37:46.295 INFO:teuthology.orchestra.run.smithi001.stdout: "sortbitwise" 2024-09-14T22:37:46.295 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:37:46.296 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_version": 6, 2024-09-14T22:37:46.296 INFO:teuthology.orchestra.run.smithi001.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T22:37:46.296 INFO:teuthology.orchestra.run.smithi001.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T22:37:46.296 INFO:teuthology.orchestra.run.smithi001.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T22:37:46.296 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_snapshot": "", 2024-09-14T22:37:46.296 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_max": 3, 2024-09-14T22:37:46.296 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd": 8, 2024-09-14T22:37:46.296 INFO:teuthology.orchestra.run.smithi001.stdout: "require_min_compat_client": "luminous", 2024-09-14T22:37:46.296 INFO:teuthology.orchestra.run.smithi001.stdout: "min_compat_client": "luminous", 2024-09-14T22:37:46.296 INFO:teuthology.orchestra.run.smithi001.stdout: "require_osd_release": "squid", 2024-09-14T22:37:46.296 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_crimson": false, 2024-09-14T22:37:46.296 INFO:teuthology.orchestra.run.smithi001.stdout: "pools": [ 2024-09-14T22:37:46.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.297 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 1, 2024-09-14T22:37:46.297 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": ".mgr", 2024-09-14T22:37:46.297 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-14T22:35:58.938784+0000", 2024-09-14T22:37:46.297 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-14T22:37:46.297 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool", 2024-09-14T22:37:46.297 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-14T22:37:46.297 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-14T22:37:46.297 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-14T22:37:46.297 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-14T22:37:46.297 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:37:46.297 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:37:46.297 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:37:46.297 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:37:46.298 INFO:teuthology.orchestra.run.smithi001.stdout: "is_stretch_pool": false, 2024-09-14T22:37:46.298 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-14T22:37:46.298 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:37:46.298 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 1, 2024-09-14T22:37:46.298 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 1, 2024-09-14T22:37:46.298 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 1, 2024-09-14T22:37:46.298 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 1, 2024-09-14T22:37:46.298 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 1, 2024-09-14T22:37:46.298 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-14T22:37:46.298 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-14T22:37:46.298 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-14T22:37:46.298 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-14T22:37:46.298 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-14T22:37:46.298 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-14T22:37:46.298 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-14T22:37:46.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.299 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "18", 2024-09-14T22:37:46.299 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-14T22:37:46.299 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:37:46.299 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:37:46.299 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-14T22:37:46.299 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:37:46.299 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 0, 2024-09-14T22:37:46.299 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 0, 2024-09-14T22:37:46.299 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-14T22:37:46.299 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-14T22:37:46.299 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-14T22:37:46.300 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-14T22:37:46.300 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-14T22:37:46.300 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-14T22:37:46.300 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-14T22:37:46.300 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-14T22:37:46.300 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-14T22:37:46.300 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-14T22:37:46.300 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-14T22:37:46.300 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:37:46.300 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:37:46.300 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:37:46.300 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-14T22:37:46.301 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-14T22:37:46.301 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-14T22:37:46.301 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-14T22:37:46.301 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-14T22:37:46.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.301 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-14T22:37:46.301 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-14T22:37:46.301 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-14T22:37:46.301 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:37:46.301 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:37:46.301 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:37:46.302 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:37:46.302 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-14T22:37:46.302 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-14T22:37:46.302 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-14T22:37:46.302 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-14T22:37:46.302 INFO:teuthology.orchestra.run.smithi001.stdout: "options": { 2024-09-14T22:37:46.302 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_max": 32, 2024-09-14T22:37:46.302 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_min": 1 2024-09-14T22:37:46.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.302 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-14T22:37:46.302 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": {} 2024-09-14T22:37:46.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.302 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-14T22:37:46.303 INFO:teuthology.orchestra.run.smithi001.stdout: "score_type": "Fair distribution", 2024-09-14T22:37:46.303 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 8, 2024-09-14T22:37:46.303 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 8, 2024-09-14T22:37:46.303 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 0.25, 2024-09-14T22:37:46.303 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 2, 2024-09-14T22:37:46.303 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 2, 2024-09-14T22:37:46.303 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 0.87667274475097656, 2024-09-14T22:37:46.303 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T22:37:46.303 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T22:37:46.303 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.304 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 2, 2024-09-14T22:37:46.304 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "rbd", 2024-09-14T22:37:46.304 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-14T22:36:04.304242+0000", 2024-09-14T22:37:46.304 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 8193, 2024-09-14T22:37:46.304 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T22:37:46.304 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-14T22:37:46.304 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-14T22:37:46.304 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-14T22:37:46.304 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-14T22:37:46.304 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:37:46.304 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:37:46.304 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:37:46.304 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:37:46.304 INFO:teuthology.orchestra.run.smithi001.stdout: "is_stretch_pool": false, 2024-09-14T22:37:46.305 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-14T22:37:46.305 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:37:46.305 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 8, 2024-09-14T22:37:46.305 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 8, 2024-09-14T22:37:46.305 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 8, 2024-09-14T22:37:46.305 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 8, 2024-09-14T22:37:46.305 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 8, 2024-09-14T22:37:46.305 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-14T22:37:46.305 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-14T22:37:46.305 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-14T22:37:46.305 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-14T22:37:46.305 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-14T22:37:46.305 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-14T22:37:46.306 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-14T22:37:46.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.306 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "22", 2024-09-14T22:37:46.306 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-14T22:37:46.306 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:37:46.306 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:37:46.306 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-14T22:37:46.306 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:37:46.306 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 2, 2024-09-14T22:37:46.306 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 22, 2024-09-14T22:37:46.306 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-14T22:37:46.306 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-14T22:37:46.306 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-14T22:37:46.306 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-14T22:37:46.306 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-14T22:37:46.307 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-14T22:37:46.307 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-14T22:37:46.307 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-14T22:37:46.307 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-14T22:37:46.307 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-14T22:37:46.307 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-14T22:37:46.307 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:37:46.307 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:37:46.307 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:37:46.307 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-14T22:37:46.307 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-14T22:37:46.307 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-14T22:37:46.307 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-14T22:37:46.307 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-14T22:37:46.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.308 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-14T22:37:46.308 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-14T22:37:46.308 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-14T22:37:46.308 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:37:46.308 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:37:46.308 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:37:46.308 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:37:46.308 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-14T22:37:46.308 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-14T22:37:46.308 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-14T22:37:46.308 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-14T22:37:46.308 INFO:teuthology.orchestra.run.smithi001.stdout: "options": {}, 2024-09-14T22:37:46.308 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-14T22:37:46.309 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd": {} 2024-09-14T22:37:46.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.309 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-14T22:37:46.309 INFO:teuthology.orchestra.run.smithi001.stdout: "score_type": "Fair distribution", 2024-09-14T22:37:46.309 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 2, 2024-09-14T22:37:46.309 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 2, 2024-09-14T22:37:46.309 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 1, 2024-09-14T22:37:46.309 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 2, 2024-09-14T22:37:46.309 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 2, 2024-09-14T22:37:46.309 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 0.87667274475097656, 2024-09-14T22:37:46.309 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T22:37:46.309 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T22:37:46.309 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.310 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 3, 2024-09-14T22:37:46.310 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "pool_client.0", 2024-09-14T22:37:46.310 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-14T22:36:14.374249+0000", 2024-09-14T22:37:46.310 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 8193, 2024-09-14T22:37:46.310 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T22:37:46.310 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-14T22:37:46.310 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-14T22:37:46.310 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-14T22:37:46.310 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-14T22:37:46.310 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:37:46.310 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:37:46.310 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:37:46.311 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:37:46.311 INFO:teuthology.orchestra.run.smithi001.stdout: "is_stretch_pool": false, 2024-09-14T22:37:46.311 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-14T22:37:46.311 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:37:46.311 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 32, 2024-09-14T22:37:46.311 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 32, 2024-09-14T22:37:46.311 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 32, 2024-09-14T22:37:46.311 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 32, 2024-09-14T22:37:46.311 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 32, 2024-09-14T22:37:46.311 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-14T22:37:46.311 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-14T22:37:46.311 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-14T22:37:46.311 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-14T22:37:46.311 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-14T22:37:46.312 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-14T22:37:46.312 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-14T22:37:46.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.312 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "94", 2024-09-14T22:37:46.312 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-14T22:37:46.312 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:37:46.312 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:37:46.312 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-14T22:37:46.312 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:37:46.312 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 33, 2024-09-14T22:37:46.312 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 94, 2024-09-14T22:37:46.312 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-14T22:37:46.312 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-14T22:37:46.313 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-14T22:37:46.313 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-14T22:37:46.313 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-14T22:37:46.313 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-14T22:37:46.313 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-14T22:37:46.313 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-14T22:37:46.313 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-14T22:37:46.313 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-14T22:37:46.313 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-14T22:37:46.313 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:37:46.313 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:37:46.314 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:37:46.314 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-14T22:37:46.314 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-14T22:37:46.314 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-14T22:37:46.314 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-14T22:37:46.314 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-14T22:37:46.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.314 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-14T22:37:46.314 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-14T22:37:46.314 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-14T22:37:46.314 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:37:46.314 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:37:46.314 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:37:46.315 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:37:46.315 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-14T22:37:46.315 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-14T22:37:46.315 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-14T22:37:46.315 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-14T22:37:46.315 INFO:teuthology.orchestra.run.smithi001.stdout: "options": {}, 2024-09-14T22:37:46.315 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-14T22:37:46.315 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd": {} 2024-09-14T22:37:46.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.315 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-14T22:37:46.315 INFO:teuthology.orchestra.run.smithi001.stdout: "score_type": "Fair distribution", 2024-09-14T22:37:46.315 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 1.75, 2024-09-14T22:37:46.315 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 1.75, 2024-09-14T22:37:46.316 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 1, 2024-09-14T22:37:46.316 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 1.75, 2024-09-14T22:37:46.316 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 1.75, 2024-09-14T22:37:46.316 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 0.87667274475097656, 2024-09-14T22:37:46.316 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T22:37:46.316 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T22:37:46.316 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.316 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.316 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:37:46.316 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2024-09-14T22:37:46.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.316 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 0, 2024-09-14T22:37:46.316 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "9831fcc5-9040-4a80-99d4-838225e65139", 2024-09-14T22:37:46.316 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:37:46.317 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:37:46.317 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:37:46.317 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:37:46.317 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:37:46.317 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:37:46.317 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:37:46.317 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 76, 2024-09-14T22:37:46.317 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:37:46.317 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:37:46.317 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:37:46.317 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:46.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.317 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:46.318 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6824", 2024-09-14T22:37:46.318 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:37:46.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.318 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:46.318 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6825", 2024-09-14T22:37:46.318 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:37:46.318 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.318 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.318 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:37:46.318 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:46.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.319 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:46.319 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6826", 2024-09-14T22:37:46.319 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:37:46.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.319 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:46.319 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6827", 2024-09-14T22:37:46.319 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:37:46.319 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.319 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.319 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:37:46.319 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:46.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.319 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:46.320 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6830", 2024-09-14T22:37:46.320 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:37:46.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.320 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:46.320 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6831", 2024-09-14T22:37:46.320 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:37:46.320 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.320 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.320 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:37:46.320 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:46.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.321 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:46.321 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6828", 2024-09-14T22:37:46.321 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:37:46.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.321 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:46.321 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6829", 2024-09-14T22:37:46.321 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:37:46.321 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.321 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.322 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6825/1702929600", 2024-09-14T22:37:46.322 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6827/1702929600", 2024-09-14T22:37:46.322 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6831/1702929600", 2024-09-14T22:37:46.322 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6829/1702929600", 2024-09-14T22:37:46.322 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:37:46.322 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:37:46.322 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:37:46.322 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.322 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 1, 2024-09-14T22:37:46.322 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "7f79abe8-115d-46b2-8677-72fcb0c8ab7d", 2024-09-14T22:37:46.322 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:37:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:37:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:37:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 0.1606292724609375, 2024-09-14T22:37:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:37:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:37:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2024-09-14T22:37:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 86, 2024-09-14T22:37:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:37:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:37:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:37:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.323 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6808", 2024-09-14T22:37:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:37:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6809", 2024-09-14T22:37:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:37:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:37:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.324 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6810", 2024-09-14T22:37:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:37:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6811", 2024-09-14T22:37:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:37:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:37:46.325 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:46.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.326 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:46.326 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6814", 2024-09-14T22:37:46.326 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:37:46.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.326 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:46.326 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6815", 2024-09-14T22:37:46.326 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:37:46.326 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.326 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.326 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:37:46.326 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:46.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.327 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:46.327 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6812", 2024-09-14T22:37:46.327 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:37:46.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.327 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:46.327 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6813", 2024-09-14T22:37:46.327 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:37:46.327 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.327 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.327 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6809/719990653", 2024-09-14T22:37:46.327 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6811/719990653", 2024-09-14T22:37:46.328 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6815/719990653", 2024-09-14T22:37:46.328 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6813/719990653", 2024-09-14T22:37:46.328 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:37:46.328 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:37:46.328 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:37:46.328 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.328 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 2, 2024-09-14T22:37:46.328 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "1f5768b2-4db4-4b2e-8d5e-5206f8bde28d", 2024-09-14T22:37:46.328 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:37:46.328 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:37:46.328 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:37:46.329 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:37:46.329 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 14, 2024-09-14T22:37:46.329 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 30, 2024-09-14T22:37:46.329 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 66, 2024-09-14T22:37:46.329 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 70, 2024-09-14T22:37:46.329 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 31, 2024-09-14T22:37:46.329 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:37:46.329 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:37:46.329 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:46.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.329 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:46.329 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6816", 2024-09-14T22:37:46.329 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:37:46.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.330 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:46.330 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6817", 2024-09-14T22:37:46.330 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:37:46.330 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.330 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.330 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:37:46.330 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:46.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.330 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:46.330 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6818", 2024-09-14T22:37:46.330 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:37:46.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.331 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:46.331 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6819", 2024-09-14T22:37:46.331 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:37:46.331 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.331 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.331 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:37:46.331 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:46.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.331 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:46.331 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6822", 2024-09-14T22:37:46.331 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:37:46.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.332 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:46.332 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6823", 2024-09-14T22:37:46.332 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:37:46.332 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.332 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.332 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:37:46.332 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:46.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.332 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:46.332 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6820", 2024-09-14T22:37:46.332 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:37:46.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.333 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:46.333 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6821", 2024-09-14T22:37:46.333 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:37:46.333 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.333 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.333 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6817/1006730601", 2024-09-14T22:37:46.333 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6819/1006730601", 2024-09-14T22:37:46.333 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6823/1006730601", 2024-09-14T22:37:46.333 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6821/1006730601", 2024-09-14T22:37:46.333 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:37:46.333 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:37:46.334 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:37:46.334 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.334 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 3, 2024-09-14T22:37:46.334 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "8103f3c7-7899-4fc3-bb02-dd2273120138", 2024-09-14T22:37:46.334 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:37:46.334 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:37:46.334 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:37:46.334 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 0.852752685546875, 2024-09-14T22:37:46.334 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:37:46.334 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:37:46.334 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:37:46.335 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 87, 2024-09-14T22:37:46.335 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:37:46.335 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:37:46.335 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:37:46.335 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:46.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.335 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:46.335 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6800", 2024-09-14T22:37:46.335 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:37:46.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.335 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:46.335 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6801", 2024-09-14T22:37:46.335 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:37:46.335 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.336 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.336 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:37:46.336 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:46.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.336 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:46.336 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6802", 2024-09-14T22:37:46.336 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:37:46.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.336 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:46.336 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6803", 2024-09-14T22:37:46.336 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:37:46.336 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.337 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.337 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:37:46.337 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:46.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.337 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:46.337 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6806", 2024-09-14T22:37:46.337 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:37:46.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.337 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:46.337 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6807", 2024-09-14T22:37:46.337 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:37:46.338 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.338 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.338 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:37:46.338 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:46.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.338 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:46.338 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6804", 2024-09-14T22:37:46.338 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:37:46.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.338 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:46.338 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6805", 2024-09-14T22:37:46.338 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:37:46.339 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.339 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.339 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6801/750403671", 2024-09-14T22:37:46.339 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6803/750403671", 2024-09-14T22:37:46.339 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6807/750403671", 2024-09-14T22:37:46.339 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6805/750403671", 2024-09-14T22:37:46.339 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:37:46.339 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:37:46.339 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:37:46.339 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.340 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 4, 2024-09-14T22:37:46.340 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "23cce5cb-726e-44c2-8edb-c2567a89dfbc", 2024-09-14T22:37:46.340 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:37:46.340 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:37:46.340 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:37:46.340 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:37:46.340 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:37:46.340 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:37:46.340 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:37:46.340 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 75, 2024-09-14T22:37:46.340 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:37:46.340 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:37:46.340 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:37:46.340 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:46.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.341 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:46.341 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6816", 2024-09-14T22:37:46.341 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:37:46.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.341 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:46.341 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6817", 2024-09-14T22:37:46.341 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:37:46.341 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.341 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.341 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:37:46.341 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:46.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.342 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:46.342 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6818", 2024-09-14T22:37:46.342 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:37:46.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.342 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:46.342 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6819", 2024-09-14T22:37:46.342 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:37:46.342 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.342 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.342 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:37:46.342 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:46.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.343 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:46.343 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6822", 2024-09-14T22:37:46.343 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:37:46.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.343 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:46.343 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6823", 2024-09-14T22:37:46.343 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:37:46.343 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.343 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.343 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:37:46.343 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:46.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.343 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:46.344 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6820", 2024-09-14T22:37:46.344 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:37:46.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.344 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:46.344 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6821", 2024-09-14T22:37:46.344 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:37:46.344 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.344 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.344 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.176:6817/1408471829", 2024-09-14T22:37:46.344 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.176:6819/1408471829", 2024-09-14T22:37:46.344 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.176:6823/1408471829", 2024-09-14T22:37:46.344 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.176:6821/1408471829", 2024-09-14T22:37:46.345 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:37:46.345 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:37:46.345 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:37:46.345 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.345 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 5, 2024-09-14T22:37:46.345 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "eb2f187b-d2f1-49ab-ad5e-ea6dc441a28e", 2024-09-14T22:37:46.345 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:37:46.345 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:37:46.345 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:37:46.345 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:37:46.345 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:37:46.345 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:37:46.345 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:37:46.346 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 75, 2024-09-14T22:37:46.346 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:37:46.346 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:37:46.346 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:37:46.346 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:46.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.346 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:46.346 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6805", 2024-09-14T22:37:46.346 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:37:46.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.346 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:46.346 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6808", 2024-09-14T22:37:46.347 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:37:46.347 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.347 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.347 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:37:46.347 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:46.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.347 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:46.347 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6810", 2024-09-14T22:37:46.347 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:37:46.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.348 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:46.348 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6811", 2024-09-14T22:37:46.348 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:37:46.348 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.348 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.348 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:37:46.348 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:46.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.348 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:46.348 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6814", 2024-09-14T22:37:46.348 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:37:46.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.349 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:46.349 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6815", 2024-09-14T22:37:46.349 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:37:46.349 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.349 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.349 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:37:46.349 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:46.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.349 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:46.349 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6812", 2024-09-14T22:37:46.349 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:37:46.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.350 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:46.350 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6813", 2024-09-14T22:37:46.350 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:37:46.350 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.350 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.350 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.176:6808/1945902326", 2024-09-14T22:37:46.350 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.176:6811/1945902326", 2024-09-14T22:37:46.350 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.176:6815/1945902326", 2024-09-14T22:37:46.350 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.176:6813/1945902326", 2024-09-14T22:37:46.350 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:37:46.350 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:37:46.351 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:37:46.351 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.351 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 6, 2024-09-14T22:37:46.351 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "d371aed8-2ad2-49a5-8ece-4b7fd37b4822", 2024-09-14T22:37:46.351 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:37:46.351 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:37:46.351 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:37:46.351 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:37:46.351 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:37:46.351 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:37:46.351 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:37:46.351 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 77, 2024-09-14T22:37:46.351 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:37:46.352 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:37:46.352 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:37:46.352 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:46.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.352 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:46.352 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6800", 2024-09-14T22:37:46.352 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:37:46.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.352 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:46.352 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6801", 2024-09-14T22:37:46.352 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:37:46.352 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.353 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.353 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:37:46.353 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:46.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.353 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:46.353 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6802", 2024-09-14T22:37:46.353 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:37:46.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.353 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:46.353 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6803", 2024-09-14T22:37:46.353 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:37:46.354 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.354 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.354 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:37:46.354 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:46.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.354 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:46.354 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6807", 2024-09-14T22:37:46.354 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:37:46.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.354 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:46.354 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6809", 2024-09-14T22:37:46.355 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:37:46.355 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.355 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.355 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:37:46.355 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:46.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.355 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:46.355 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6804", 2024-09-14T22:37:46.355 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:37:46.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.355 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:46.355 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6806", 2024-09-14T22:37:46.356 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:37:46.356 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.356 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.356 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.176:6801/4116085586", 2024-09-14T22:37:46.356 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.176:6803/4116085586", 2024-09-14T22:37:46.356 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.176:6809/4116085586", 2024-09-14T22:37:46.356 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.176:6806/4116085586", 2024-09-14T22:37:46.356 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:37:46.356 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:37:46.356 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:37:46.356 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.357 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 7, 2024-09-14T22:37:46.357 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "a012c819-8d49-4310-a840-c2752d9314ab", 2024-09-14T22:37:46.357 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:37:46.357 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:37:46.357 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:37:46.357 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:37:46.357 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:37:46.357 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:37:46.357 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:37:46.357 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 90, 2024-09-14T22:37:46.357 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:37:46.357 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:37:46.357 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:37:46.358 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:46.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.358 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:46.358 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6824", 2024-09-14T22:37:46.358 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:37:46.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.358 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:46.358 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6825", 2024-09-14T22:37:46.358 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:37:46.358 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.358 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.359 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:37:46.359 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:46.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.359 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:46.359 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6826", 2024-09-14T22:37:46.359 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:37:46.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.359 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:46.359 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6827", 2024-09-14T22:37:46.359 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:37:46.359 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.359 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.360 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:37:46.360 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:46.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.360 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:46.360 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6830", 2024-09-14T22:37:46.360 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:37:46.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.360 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:46.360 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6831", 2024-09-14T22:37:46.360 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:37:46.360 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.361 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.361 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:37:46.361 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:37:46.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.361 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:37:46.361 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6828", 2024-09-14T22:37:46.361 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:37:46.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.361 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:37:46.361 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6829", 2024-09-14T22:37:46.361 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:37:46.361 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.362 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.362 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.176:6825/877760244", 2024-09-14T22:37:46.362 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.176:6827/877760244", 2024-09-14T22:37:46.362 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.176:6831/877760244", 2024-09-14T22:37:46.362 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.176:6829/877760244", 2024-09-14T22:37:46.362 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:37:46.362 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:37:46.362 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:37:46.362 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.362 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.362 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:37:46.362 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_xinfo": [ 2024-09-14T22:37:46.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.362 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 0, 2024-09-14T22:37:46.363 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:37:46.363 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:37:46.363 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:37:46.363 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:37:46.363 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:37:46.363 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:55.511688+0000", 2024-09-14T22:37:46.363 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:37:46.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.363 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 1, 2024-09-14T22:37:46.363 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:37:46.363 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:37:46.363 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:37:46.363 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:37:46.363 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:37:46.364 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.742414+0000", 2024-09-14T22:37:46.364 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:37:46.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.364 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 2, 2024-09-14T22:37:46.364 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:37:46.364 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:37:46.364 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:37:46.364 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:37:46.364 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:37:46.364 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.871355+0000", 2024-09-14T22:37:46.364 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 31 2024-09-14T22:37:46.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.365 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 3, 2024-09-14T22:37:46.365 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:37:46.365 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:37:46.365 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:37:46.365 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:37:46.365 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:37:46.365 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.504553+0000", 2024-09-14T22:37:46.365 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:37:46.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.365 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 4, 2024-09-14T22:37:46.365 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:37:46.365 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:37:46.365 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:37:46.366 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:37:46.366 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:37:46.366 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.589640+0000", 2024-09-14T22:37:46.366 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:37:46.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.366 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 5, 2024-09-14T22:37:46.366 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:37:46.366 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:37:46.366 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:37:46.366 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:37:46.366 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:37:46.366 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.518845+0000", 2024-09-14T22:37:46.366 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:37:46.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.367 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 6, 2024-09-14T22:37:46.367 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:37:46.367 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:37:46.367 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:37:46.367 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:37:46.367 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:37:46.367 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.731632+0000", 2024-09-14T22:37:46.367 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:37:46.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.367 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 7, 2024-09-14T22:37:46.368 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:37:46.368 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:37:46.368 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:37:46.368 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:37:46.368 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:37:46.368 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.952634+0000", 2024-09-14T22:37:46.368 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:37:46.368 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.368 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:37:46.368 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap": [], 2024-09-14T22:37:46.368 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap_items": [ 2024-09-14T22:37:46.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.368 INFO:teuthology.orchestra.run.smithi001.stdout: "pgid": "2.0", 2024-09-14T22:37:46.368 INFO:teuthology.orchestra.run.smithi001.stdout: "mappings": [ 2024-09-14T22:37:46.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.369 INFO:teuthology.orchestra.run.smithi001.stdout: "from": 1, 2024-09-14T22:37:46.369 INFO:teuthology.orchestra.run.smithi001.stdout: "to": 4 2024-09-14T22:37:46.369 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.369 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.369 INFO:teuthology.orchestra.run.smithi001.stdout: "pgid": "3.2", 2024-09-14T22:37:46.369 INFO:teuthology.orchestra.run.smithi001.stdout: "mappings": [ 2024-09-14T22:37:46.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:37:46.369 INFO:teuthology.orchestra.run.smithi001.stdout: "from": 3, 2024-09-14T22:37:46.369 INFO:teuthology.orchestra.run.smithi001.stdout: "to": 6 2024-09-14T22:37:46.369 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.369 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:37:46.370 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.370 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:37:46.370 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap_primaries": [], 2024-09-14T22:37:46.370 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_temp": [], 2024-09-14T22:37:46.370 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_temp": [], 2024-09-14T22:37:46.370 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklist": {}, 2024-09-14T22:37:46.370 INFO:teuthology.orchestra.run.smithi001.stdout: "range_blocklist": {}, 2024-09-14T22:37:46.370 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profiles": { 2024-09-14T22:37:46.370 INFO:teuthology.orchestra.run.smithi001.stdout: "default": { 2024-09-14T22:37:46.370 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-failure-domain": "osd", 2024-09-14T22:37:46.370 INFO:teuthology.orchestra.run.smithi001.stdout: "k": "2", 2024-09-14T22:37:46.370 INFO:teuthology.orchestra.run.smithi001.stdout: "m": "1", 2024-09-14T22:37:46.370 INFO:teuthology.orchestra.run.smithi001.stdout: "plugin": "jerasure", 2024-09-14T22:37:46.371 INFO:teuthology.orchestra.run.smithi001.stdout: "technique": "reed_sol_van" 2024-09-14T22:37:46.371 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:37:46.371 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps_queue": [], 2024-09-14T22:37:46.371 INFO:teuthology.orchestra.run.smithi001.stdout: "new_removed_snaps": [], 2024-09-14T22:37:46.371 INFO:teuthology.orchestra.run.smithi001.stdout: "new_purged_snaps": [], 2024-09-14T22:37:46.371 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_node_flags": {}, 2024-09-14T22:37:46.371 INFO:teuthology.orchestra.run.smithi001.stdout: "device_class_flags": {}, 2024-09-14T22:37:46.371 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": { 2024-09-14T22:37:46.371 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode_enabled": false, 2024-09-14T22:37:46.371 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_bucket_count": 0, 2024-09-14T22:37:46.371 INFO:teuthology.orchestra.run.smithi001.stdout: "degraded_stretch_mode": 0, 2024-09-14T22:37:46.371 INFO:teuthology.orchestra.run.smithi001.stdout: "recovering_stretch_mode": 0, 2024-09-14T22:37:46.371 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode_bucket": 0 2024-09-14T22:37:46.372 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:37:46.372 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:46.373 INFO:teuthology.orchestra.run.smithi001.stdout:4969 write 0xcf00826 thru 0xcf0fa55 (0xf230 bytes) 2024-09-14T22:37:46.373 INFO:teuthology.orchestra.run.smithi001.stdout:4970 write 0x250291 thru 0x2536d3 (0x3443 bytes) 2024-09-14T22:37:46.373 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:37:46.374 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:37:46.374 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:46.374 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:46.375 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:37:46.376 INFO:tasks.thrashosds.thrasher:j is {'epoch': 94, 'fsid': 'f07f407a-b17d-42c4-bf73-92dd80114a4f', 'created': '2024-09-14T22:35:45.983055+0000', 'modified': '2024-09-14T22:37:45.868688+0000', 'last_up_change': '2024-09-14T22:37:12.616042+0000', 'last_in_change': '2024-09-14T22:35:48.283160+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T22:35:58.938784+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8766727447509766, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T22:36:04.304242+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8766727447509766, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T22:36:14.374249+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 33, 'snap_epoch': 94, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.8766727447509766, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': '9831fcc5-9040-4a80-99d4-838225e65139', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 76, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6824', 'nonce': 1702929600}, {'type': 'v1', 'addr': '172.21.15.1:6825', 'nonce': 1702929600}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6826', 'nonce': 1702929600}, {'type': 'v1', 'addr': '172.21.15.1:6827', 'nonce': 1702929600}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6830', 'nonce': 1702929600}, {'type': 'v1', 'addr': '172.21.15.1:6831', 'nonce': 1702929600}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6828', 'nonce': 1702929600}, {'type': 'v1', 'addr': '172.21.15.1:6829', 'nonce': 1702929600}]}, 'public_addr': '172.21.15.1:6825/1702929600', 'cluster_addr': '172.21.15.1:6827/1702929600', 'heartbeat_back_addr': '172.21.15.1:6831/1702929600', 'heartbeat_front_addr': '172.21.15.1:6829/1702929600', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '7f79abe8-115d-46b2-8677-72fcb0c8ab7d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.1606292724609375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 86, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6808', 'nonce': 719990653}, {'type': 'v1', 'addr': '172.21.15.1:6809', 'nonce': 719990653}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6810', 'nonce': 719990653}, {'type': 'v1', 'addr': '172.21.15.1:6811', 'nonce': 719990653}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6814', 'nonce': 719990653}, {'type': 'v1', 'addr': '172.21.15.1:6815', 'nonce': 719990653}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6812', 'nonce': 719990653}, {'type': 'v1', 'addr': '172.21.15.1:6813', 'nonce': 719990653}]}, 'public_addr': '172.21.15.1:6809/719990653', 'cluster_addr': '172.21.15.1:6811/719990653', 'heartbeat_back_addr': '172.21.15.1:6815/719990653', 'heartbeat_front_addr': '172.21.15.1:6813/719990653', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '1f5768b2-4db4-4b2e-8d5e-5206f8bde28d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 14, 'last_clean_end': 30, 'up_from': 66, 'up_thru': 70, 'down_at': 31, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6816', 'nonce': 1006730601}, {'type': 'v1', 'addr': '172.21.15.1:6817', 'nonce': 1006730601}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6818', 'nonce': 1006730601}, {'type': 'v1', 'addr': '172.21.15.1:6819', 'nonce': 1006730601}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6822', 'nonce': 1006730601}, {'type': 'v1', 'addr': '172.21.15.1:6823', 'nonce': 1006730601}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6820', 'nonce': 1006730601}, {'type': 'v1', 'addr': '172.21.15.1:6821', 'nonce': 1006730601}]}, 'public_addr': '172.21.15.1:6817/1006730601', 'cluster_addr': '172.21.15.1:6819/1006730601', 'heartbeat_back_addr': '172.21.15.1:6823/1006730601', 'heartbeat_front_addr': '172.21.15.1:6821/1006730601', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '8103f3c7-7899-4fc3-bb02-dd2273120138', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.852752685546875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 87, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6800', 'nonce': 750403671}, {'type': 'v1', 'addr': '172.21.15.1:6801', 'nonce': 750403671}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6802', 'nonce': 750403671}, {'type': 'v1', 'addr': '172.21.15.1:6803', 'nonce': 750403671}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6806', 'nonce': 750403671}, {'type': 'v1', 'addr': '172.21.15.1:6807', 'nonce': 750403671}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6804', 'nonce': 750403671}, {'type': 'v1', 'addr': '172.21.15.1:6805', 'nonce': 750403671}]}, 'public_addr': '172.21.15.1:6801/750403671', 'cluster_addr': '172.21.15.1:6803/750403671', 'heartbeat_back_addr': '172.21.15.1:6807/750403671', 'heartbeat_front_addr': '172.21.15.1:6805/750403671', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '23cce5cb-726e-44c2-8edb-c2567a89dfbc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6816', 'nonce': 1408471829}, {'type': 'v1', 'addr': '172.21.15.176:6817', 'nonce': 1408471829}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6818', 'nonce': 1408471829}, {'type': 'v1', 'addr': '172.21.15.176:6819', 'nonce': 1408471829}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6822', 'nonce': 1408471829}, {'type': 'v1', 'addr': '172.21.15.176:6823', 'nonce': 1408471829}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6820', 'nonce': 1408471829}, {'type': 'v1', 'addr': '172.21.15.176:6821', 'nonce': 1408471829}]}, 'public_addr': '172.21.15.176:6817/1408471829', 'cluster_addr': '172.21.15.176:6819/1408471829', 'heartbeat_back_addr': '172.21.15.176:6823/1408471829', 'heartbeat_front_addr': '172.21.15.176:6821/1408471829', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'eb2f187b-d2f1-49ab-ad5e-ea6dc441a28e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6805', 'nonce': 1945902326}, {'type': 'v1', 'addr': '172.21.15.176:6808', 'nonce': 1945902326}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6810', 'nonce': 1945902326}, {'type': 'v1', 'addr': '172.21.15.176:6811', 'nonce': 1945902326}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6814', 'nonce': 1945902326}, {'type': 'v1', 'addr': '172.21.15.176:6815', 'nonce': 1945902326}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6812', 'nonce': 1945902326}, {'type': 'v1', 'addr': '172.21.15.176:6813', 'nonce': 1945902326}]}, 'public_addr': '172.21.15.176:6808/1945902326', 'cluster_addr': '172.21.15.176:6811/1945902326', 'heartbeat_back_addr': '172.21.15.176:6815/1945902326', 'heartbeat_front_addr': '172.21.15.176:6813/1945902326', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'd371aed8-2ad2-49a5-8ece-4b7fd37b4822', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 77, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6800', 'nonce': 4116085586}, {'type': 'v1', 'addr': '172.21.15.176:6801', 'nonce': 4116085586}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6802', 'nonce': 4116085586}, {'type': 'v1', 'addr': '172.21.15.176:6803', 'nonce': 4116085586}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6807', 'nonce': 4116085586}, {'type': 'v1', 'addr': '172.21.15.176:6809', 'nonce': 4116085586}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6804', 'nonce': 4116085586}, {'type': 'v1', 'addr': '172.21.15.176:6806', 'nonce': 4116085586}]}, 'public_addr': '172.21.15.176:6801/4116085586', 'cluster_addr': '172.21.15.176:6803/4116085586', 'heartbeat_back_addr': '172.21.15.176:6809/4116085586', 'heartbeat_front_addr': '172.21.15.176:6806/4116085586', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'a012c819-8d49-4310-a840-c2752d9314ab', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 90, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6824', 'nonce': 877760244}, {'type': 'v1', 'addr': '172.21.15.176:6825', 'nonce': 877760244}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6826', 'nonce': 877760244}, {'type': 'v1', 'addr': '172.21.15.176:6827', 'nonce': 877760244}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6830', 'nonce': 877760244}, {'type': 'v1', 'addr': '172.21.15.176:6831', 'nonce': 877760244}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6828', 'nonce': 877760244}, {'type': 'v1', 'addr': '172.21.15.176:6829', 'nonce': 877760244}]}, 'public_addr': '172.21.15.176:6825/877760244', 'cluster_addr': '172.21.15.176:6827/877760244', 'heartbeat_back_addr': '172.21.15.176:6831/877760244', 'heartbeat_front_addr': '172.21.15.176:6829/877760244', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:55.511688+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.742414+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.871355+0000', 'dead_epoch': 31}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.504553+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.589640+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.518845+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.731632+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.952634+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '2.0', 'mappings': [{'from': 1, 'to': 4}]}, {'pgid': '3.2', 'mappings': [{'from': 3, 'to': 6}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T22:37:46.376 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:37:46.377 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:37:46.377 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:37:46.377 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:37:46.378 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:37:46.378 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:46.371+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:46.388 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:37:46.388 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:37:46.391 INFO:teuthology.orchestra.run.smithi001.stdout:4971 read 0x77d5cd8 thru 0x77de7b7 (0x8ae0 bytes) 2024-09-14T22:37:46.400 INFO:teuthology.orchestra.run.smithi001.stdout:4972 write 0xcfbf672 thru 0xcfc6ebc (0x784b bytes) 2024-09-14T22:37:46.405 INFO:teuthology.orchestra.run.smithi001.stdout:4973 writesame 0x20c631e thru 0x20cc8da data_size 0x1(0x65bd bytes) 2024-09-14T22:37:46.411 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:37:46.411 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:37:46.473 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:46.466+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:46.473 INFO:teuthology.orchestra.run.smithi001.stdout:4974 writesame 0x69deba5 thru 0x69ecc90 data_size 0xa(0xe0ec bytes) 2024-09-14T22:37:46.533 INFO:teuthology.orchestra.run.smithi001.stdout:4975 read 0x58dc4bb thru 0x58e8efa (0xca40 bytes) 2024-09-14T22:37:46.534 INFO:teuthology.orchestra.run.smithi001.stdout:4977 write 0x664df11 thru 0x665c5f7 (0xe6e7 bytes) 2024-09-14T22:37:46.537 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:46.543 INFO:teuthology.orchestra.run.smithi001.stdout:4980 trunc from 0xcfc6ebd to 0x900fa9f 2024-09-14T22:37:46.544 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:46.544 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:46.544 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:37:46.544 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:37:46.545 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:46.555 INFO:teuthology.orchestra.run.smithi001.stdout:4981 write 0x3f827f thru 0x3f8fb1 (0xd33 bytes) 2024-09-14T22:37:46.557 INFO:teuthology.orchestra.run.smithi001.stdout:4982 write 0x7b65c94 thru 0x7b66f55 (0x12c2 bytes) 2024-09-14T22:37:46.559 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:46.560 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:37:46.562 INFO:teuthology.orchestra.run.smithi001.stdout:4985 trunc from 0x900fa9f to 0xb1feb14 2024-09-14T22:37:46.570 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:46.570 INFO:teuthology.orchestra.run.smithi001.stdout:4987 punch from 0x597eea6 to 0x598c37f, (0xd4d9 bytes) 2024-09-14T22:37:46.570 INFO:teuthology.orchestra.run.smithi001.stdout:4988 write 0x7ac7d39 thru 0x7ad5d76 (0xe03e bytes) 2024-09-14T22:37:46.572 INFO:teuthology.orchestra.run.smithi001.stdout:4989 write 0x1e93800 thru 0x1e97e22 (0x4623 bytes) 2024-09-14T22:37:46.590 INFO:teuthology.orchestra.run.smithi001.stdout:4990 trunc from 0xb1feb14 to 0x1577249 2024-09-14T22:37:46.603 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:46.598+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:46.674 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:46.671+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:46.691 INFO:teuthology.orchestra.run.smithi001.stdout:4991 writesame 0x1225bfb thru 0x1235b22 data_size 0xb8(0xff28 bytes) 2024-09-14T22:37:46.701 INFO:teuthology.orchestra.run.smithi001.stdout:4992 write 0x16f9f12 thru 0x17016c0 (0x77af bytes) 2024-09-14T22:37:46.702 INFO:teuthology.orchestra.run.smithi001.stdout:4993 read 0x132db6b thru 0x1337989 (0x9e1f bytes) 2024-09-14T22:37:46.703 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:46.725 INFO:teuthology.orchestra.run.smithi001.stdout:4996 write 0xc8dca20 thru 0xc8eb4a8 (0xea89 bytes) 2024-09-14T22:37:46.729 INFO:teuthology.orchestra.run.smithi001.stdout:5000 punch from 0x8d03261 to 0x8d07495, (0x4234 bytes) 2024-09-14T22:37:46.729 INFO:teuthology.orchestra.run.smithi001.stdout:5002 write 0x1238381 thru 0x1240f67 (0x8be7 bytes) 2024-09-14T22:37:46.732 INFO:teuthology.orchestra.run.smithi001.stdout:5003 read 0x84b6792 thru 0x84c3058 (0xc8c7 bytes) 2024-09-14T22:37:46.732 INFO:teuthology.orchestra.run.smithi001.stdout:5004 read 0x2d570df thru 0x2d596de (0x2600 bytes) 2024-09-14T22:37:46.733 INFO:teuthology.orchestra.run.smithi001.stdout:5005 write 0xc865072 thru 0xc86e2dd (0x926c bytes) 2024-09-14T22:37:46.740 INFO:teuthology.orchestra.run.smithi001.stdout:5006 punch from 0x6af8cf2 to 0x6b073f2, (0xe700 bytes) 2024-09-14T22:37:46.740 INFO:teuthology.orchestra.run.smithi001.stdout:5009 read 0x617b397 thru 0x618b33d (0xffa7 bytes) 2024-09-14T22:37:46.740 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:46.744 INFO:teuthology.orchestra.run.smithi001.stdout:5011 trunc from 0xc8eb4a9 to 0x6e748b5 2024-09-14T22:37:46.753 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:46.753 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:46.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:46.753 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:46.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.753 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6395 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8386560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:46.753 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:37:46.754 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 82.816107243999994, 2024-09-14T22:37:46.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054534379000000001, 2024-09-14T22:37:46.754 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:46.754 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:46.754 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:46.754 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:46.754 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:46.754 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:46.754 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6395 2024-09-14T22:37:46.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.754 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:46.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:46.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:37:46.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:46.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:37:46.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:46.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933833+0000", 2024-09-14T22:37:46.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3859999999999999e-06 2024-09-14T22:37:46.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:46.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933838+0000", 2024-09-14T22:37:46.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8709999999999996e-06 2024-09-14T22:37:46.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:46.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933854+0000", 2024-09-14T22:37:46.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6189e-05 2024-09-14T22:37:46.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:46.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933871+0000", 2024-09-14T22:37:46.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.626e-05 2024-09-14T22:37:46.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:46.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988198+0000", 2024-09-14T22:37:46.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054327581999999999 2024-09-14T22:37:46.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:46.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988274+0000", 2024-09-14T22:37:46.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5759000000000004e-05 2024-09-14T22:37:46.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:46.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988366+0000", 2024-09-14T22:37:46.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2331999999999996e-05 2024-09-14T22:37:46.757 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.757 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:46.757 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.758 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104944 3.3 3:c3780f3b:::rbd_data.113ca561f7a9.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:46.758 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:37:46.758 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.716505371, 2024-09-14T22:37:46.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073041087000000005, 2024-09-14T22:37:46.758 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:46.758 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:46.758 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:46.758 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:46.758 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:46.758 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:46.758 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104944 2024-09-14T22:37:46.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.759 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:46.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:46.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:37:46.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:46.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:37:46.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:46.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033436+0000", 2024-09-14T22:37:46.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2060000000000001e-06 2024-09-14T22:37:46.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:46.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033448+0000", 2024-09-14T22:37:46.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.169e-05 2024-09-14T22:37:46.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:46.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033474+0000", 2024-09-14T22:37:46.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6400000000000001e-05 2024-09-14T22:37:46.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:46.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033489+0000", 2024-09-14T22:37:46.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5067e-05 2024-09-14T22:37:46.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:46.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033658+0000", 2024-09-14T22:37:46.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000168565 2024-09-14T22:37:46.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:46.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033793+0000", 2024-09-14T22:37:46.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013536900000000001 2024-09-14T22:37:46.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:37:46.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.047559+0000", 2024-09-14T22:37:46.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013766294 2024-09-14T22:37:46.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:46.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106157+0000", 2024-09-14T22:37:46.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058597110000000001 2024-09-14T22:37:46.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:46.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106290+0000", 2024-09-14T22:37:46.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000133244 2024-09-14T22:37:46.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:46.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106396+0000", 2024-09-14T22:37:46.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106246 2024-09-14T22:37:46.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:46.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106475+0000", 2024-09-14T22:37:46.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8895999999999998e-05 2024-09-14T22:37:46.763 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.763 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:46.763 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.764 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:111858 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 10403840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:46.764 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:37:46.764 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.926052250000001, 2024-09-14T22:37:46.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054398798999999998, 2024-09-14T22:37:46.764 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:46.764 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:46.764 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:46.764 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:46.764 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:46.764 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:46.764 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 111858 2024-09-14T22:37:46.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.764 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:46.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:46.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:37:46.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:46.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:37:46.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:46.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823888+0000", 2024-09-14T22:37:46.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0200000000000001e-07 2024-09-14T22:37:46.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:46.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823927+0000", 2024-09-14T22:37:46.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8940999999999999e-05 2024-09-14T22:37:46.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:46.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823940+0000", 2024-09-14T22:37:46.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3682e-05 2024-09-14T22:37:46.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:46.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823950+0000", 2024-09-14T22:37:46.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7110000000000007e-06 2024-09-14T22:37:46.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:46.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878202+0000", 2024-09-14T22:37:46.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054251969999999997 2024-09-14T22:37:46.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:46.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878249+0000", 2024-09-14T22:37:46.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6854999999999998e-05 2024-09-14T22:37:46.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:46.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878286+0000", 2024-09-14T22:37:46.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6838000000000002e-05 2024-09-14T22:37:46.768 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.768 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:46.768 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.768 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:111859 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 10452992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:46.768 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:37:46.768 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.925963131, 2024-09-14T22:37:46.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054435292000000003, 2024-09-14T22:37:46.768 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:46.768 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:46.768 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:46.768 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:46.769 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:46.769 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:46.769 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 111859 2024-09-14T22:37:46.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.769 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:46.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:46.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:37:46.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:46.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:37:46.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:46.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823977+0000", 2024-09-14T22:37:46.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3500000000000005e-07 2024-09-14T22:37:46.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:46.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823981+0000", 2024-09-14T22:37:46.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2050000000000004e-06 2024-09-14T22:37:46.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:46.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824007+0000", 2024-09-14T22:37:46.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.62e-05 2024-09-14T22:37:46.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:46.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824018+0000", 2024-09-14T22:37:46.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0389e-05 2024-09-14T22:37:46.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:46.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878312+0000", 2024-09-14T22:37:46.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054294754000000001 2024-09-14T22:37:46.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:46.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878350+0000", 2024-09-14T22:37:46.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7982000000000003e-05 2024-09-14T22:37:46.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:46.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878411+0000", 2024-09-14T22:37:46.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0927000000000001e-05 2024-09-14T22:37:46.773 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.773 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:46.773 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.773 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112030 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6316032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:46.773 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:37:46.773 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.925908789000001, 2024-09-14T22:37:46.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054597826000000002, 2024-09-14T22:37:46.773 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:46.773 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:46.773 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:46.773 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:46.774 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:46.774 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:46.774 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112030 2024-09-14T22:37:46.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.774 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:46.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:46.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:37:46.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:46.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:37:46.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:46.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824031+0000", 2024-09-14T22:37:46.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3600000000000003e-07 2024-09-14T22:37:46.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:46.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824036+0000", 2024-09-14T22:37:46.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7350000000000004e-06 2024-09-14T22:37:46.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:46.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824049+0000", 2024-09-14T22:37:46.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3329000000000001e-05 2024-09-14T22:37:46.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.776 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:46.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824057+0000", 2024-09-14T22:37:46.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2460000000000003e-06 2024-09-14T22:37:46.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.776 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:46.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878464+0000", 2024-09-14T22:37:46.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054406207999999998 2024-09-14T22:37:46.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:46.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878554+0000", 2024-09-14T22:37:46.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0199999999999997e-05 2024-09-14T22:37:46.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:46.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878628+0000", 2024-09-14T22:37:46.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4371999999999999e-05 2024-09-14T22:37:46.777 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.777 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:46.777 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.778 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112031 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6365184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:46.778 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:37:46.778 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.925870471, 2024-09-14T22:37:46.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054741868999999999, 2024-09-14T22:37:46.778 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:46.778 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:46.778 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:46.778 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:46.779 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:46.779 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:46.779 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112031 2024-09-14T22:37:46.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.779 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:46.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:46.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:37:46.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:46.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:37:46.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:46.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824070+0000", 2024-09-14T22:37:46.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5099999999999998e-07 2024-09-14T22:37:46.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:46.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824073+0000", 2024-09-14T22:37:46.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6940000000000001e-06 2024-09-14T22:37:46.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:46.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824084+0000", 2024-09-14T22:37:46.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0380000000000001e-05 2024-09-14T22:37:46.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:46.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824091+0000", 2024-09-14T22:37:46.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5730000000000002e-06 2024-09-14T22:37:46.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:46.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878670+0000", 2024-09-14T22:37:46.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054579176 2024-09-14T22:37:46.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:46.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878742+0000", 2024-09-14T22:37:46.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T22:37:46.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:46.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878811+0000", 2024-09-14T22:37:46.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8170999999999995e-05 2024-09-14T22:37:46.782 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.782 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:46.782 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.782 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112032 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6414336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:46.783 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:37:46.783 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.925836605000001, 2024-09-14T22:37:46.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054931916999999997, 2024-09-14T22:37:46.783 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:46.783 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:46.783 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:46.783 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:46.783 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:46.783 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:46.783 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112032 2024-09-14T22:37:46.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.783 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:46.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:46.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:37:46.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:46.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:37:46.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:46.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:37:46.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:37:46.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:46.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824107+0000", 2024-09-14T22:37:46.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5640000000000001e-06 2024-09-14T22:37:46.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:46.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824117+0000", 2024-09-14T22:37:46.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0366e-05 2024-09-14T22:37:46.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:46.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824158+0000", 2024-09-14T22:37:46.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0670000000000002e-05 2024-09-14T22:37:46.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:46.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878853+0000", 2024-09-14T22:37:46.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054694853000000002 2024-09-14T22:37:46.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:46.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878934+0000", 2024-09-14T22:37:46.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1754000000000004e-05 2024-09-14T22:37:46.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:46.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879034+0000", 2024-09-14T22:37:46.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000100115 2024-09-14T22:37:46.787 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.787 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:46.787 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.787 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112033 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6463488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:46.787 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:37:46.787 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.925767947000001, 2024-09-14T22:37:46.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054991076999999999, 2024-09-14T22:37:46.787 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:46.787 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:46.788 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:46.788 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:46.788 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:46.788 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:46.788 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112033 2024-09-14T22:37:46.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.788 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:46.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:46.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:37:46.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:46.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:37:46.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:46.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824172+0000", 2024-09-14T22:37:46.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9999999999999996e-07 2024-09-14T22:37:46.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:46.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824177+0000", 2024-09-14T22:37:46.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5310000000000003e-06 2024-09-14T22:37:46.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:46.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824188+0000", 2024-09-14T22:37:46.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1235e-05 2024-09-14T22:37:46.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:46.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824197+0000", 2024-09-14T22:37:46.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7069999999999998e-06 2024-09-14T22:37:46.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:46.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879075+0000", 2024-09-14T22:37:46.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054878749999999997 2024-09-14T22:37:46.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:46.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879118+0000", 2024-09-14T22:37:46.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2749e-05 2024-09-14T22:37:46.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:46.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879162+0000", 2024-09-14T22:37:46.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4205000000000001e-05 2024-09-14T22:37:46.791 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.791 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:46.791 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.792 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112034 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6512640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:46.792 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:37:46.792 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.925730995999999, 2024-09-14T22:37:46.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055181246000000003, 2024-09-14T22:37:46.792 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:46.792 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:46.792 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:46.792 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:46.792 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:46.792 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:46.792 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112034 2024-09-14T22:37:46.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.792 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:46.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:46.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:37:46.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:46.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:37:46.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:46.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824209+0000", 2024-09-14T22:37:46.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5499999999999998e-07 2024-09-14T22:37:46.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:46.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824212+0000", 2024-09-14T22:37:46.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6119999999999999e-06 2024-09-14T22:37:46.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:46.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824224+0000", 2024-09-14T22:37:46.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1127999999999999e-05 2024-09-14T22:37:46.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:46.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824232+0000", 2024-09-14T22:37:46.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4700000000000002e-06 2024-09-14T22:37:46.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:46.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879209+0000", 2024-09-14T22:37:46.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054977201000000003 2024-09-14T22:37:46.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:46.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879308+0000", 2024-09-14T22:37:46.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8419999999999996e-05 2024-09-14T22:37:46.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:46.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879389+0000", 2024-09-14T22:37:46.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1760000000000003e-05 2024-09-14T22:37:46.795 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.795 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:46.795 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.796 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112035 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6561792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:46.796 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:37:46.796 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.925637267999999, 2024-09-14T22:37:46.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055300522999999997, 2024-09-14T22:37:46.796 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:46.796 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:46.796 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:46.796 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:46.796 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:46.796 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:46.797 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112035 2024-09-14T22:37:46.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.797 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:46.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:46.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:37:46.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:46.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:37:46.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:46.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824303+0000", 2024-09-14T22:37:46.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5099999999999998e-07 2024-09-14T22:37:46.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:46.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824308+0000", 2024-09-14T22:37:46.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6689999999999997e-06 2024-09-14T22:37:46.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:46.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824744+0000", 2024-09-14T22:37:46.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00043601600000000001 2024-09-14T22:37:46.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:46.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824756+0000", 2024-09-14T22:37:46.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2476999999999999e-05 2024-09-14T22:37:46.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:46.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879434+0000", 2024-09-14T22:37:46.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054677630999999997 2024-09-14T22:37:46.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:46.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879518+0000", 2024-09-14T22:37:46.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4220000000000003e-05 2024-09-14T22:37:46.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:46.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879602+0000", 2024-09-14T22:37:46.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4559000000000001e-05 2024-09-14T22:37:46.800 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.800 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:46.800 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.800 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112036 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6610944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:46.800 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:37:46.800 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.925145530000002, 2024-09-14T22:37:46.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055036320999999999, 2024-09-14T22:37:46.800 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:46.800 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:46.800 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:46.801 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:46.801 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:46.801 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:46.801 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112036 2024-09-14T22:37:46.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.801 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:46.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:46.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:37:46.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:46.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:37:46.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:46.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824795+0000", 2024-09-14T22:37:46.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-06 2024-09-14T22:37:46.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:46.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824802+0000", 2024-09-14T22:37:46.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6680000000000004e-06 2024-09-14T22:37:46.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:46.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824814+0000", 2024-09-14T22:37:46.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2102e-05 2024-09-14T22:37:46.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:46.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824823+0000", 2024-09-14T22:37:46.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5550000000000008e-06 2024-09-14T22:37:46.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:46.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879648+0000", 2024-09-14T22:37:46.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054825167000000001 2024-09-14T22:37:46.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:46.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879747+0000", 2024-09-14T22:37:46.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8968999999999994e-05 2024-09-14T22:37:46.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:46.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879830+0000", 2024-09-14T22:37:46.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2633999999999995e-05 2024-09-14T22:37:46.804 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.804 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:46.804 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.804 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112037 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6660096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:46.804 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:37:46.804 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.925102820999999, 2024-09-14T22:37:46.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055122926000000003, 2024-09-14T22:37:46.804 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:46.804 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:46.805 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:46.805 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:46.805 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:46.805 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:46.805 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112037 2024-09-14T22:37:46.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.805 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:46.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:46.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:37:46.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:46.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:37:46.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:46.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824837+0000", 2024-09-14T22:37:46.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5499999999999998e-07 2024-09-14T22:37:46.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:46.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824840+0000", 2024-09-14T22:37:46.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0429999999999999e-06 2024-09-14T22:37:46.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:46.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824851+0000", 2024-09-14T22:37:46.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1046999999999999e-05 2024-09-14T22:37:46.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:46.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824859+0000", 2024-09-14T22:37:46.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1200000000000002e-06 2024-09-14T22:37:46.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:46.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879867+0000", 2024-09-14T22:37:46.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055007697000000001 2024-09-14T22:37:46.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:46.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879910+0000", 2024-09-14T22:37:46.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3402000000000002e-05 2024-09-14T22:37:46.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:46.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879959+0000", 2024-09-14T22:37:46.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9061999999999999e-05 2024-09-14T22:37:46.808 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.808 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:46.808 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.808 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112038 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6709248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:46.809 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:37:46.809 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.925068124999999, 2024-09-14T22:37:46.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055312828000000001, 2024-09-14T22:37:46.809 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:46.809 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:46.809 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:46.809 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:46.809 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:46.809 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:46.809 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112038 2024-09-14T22:37:46.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.809 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:46.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:46.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:37:46.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:46.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:37:46.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:46.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824872+0000", 2024-09-14T22:37:46.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T22:37:46.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:46.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824875+0000", 2024-09-14T22:37:46.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7519999999999998e-06 2024-09-14T22:37:46.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:46.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824917+0000", 2024-09-14T22:37:46.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1047999999999997e-05 2024-09-14T22:37:46.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:46.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824926+0000", 2024-09-14T22:37:46.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3749999999999992e-06 2024-09-14T22:37:46.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:46.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880006+0000", 2024-09-14T22:37:46.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055080525999999998 2024-09-14T22:37:46.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:46.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880102+0000", 2024-09-14T22:37:46.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5995999999999994e-05 2024-09-14T22:37:46.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:46.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880184+0000", 2024-09-14T22:37:46.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1478999999999997e-05 2024-09-14T22:37:46.813 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.813 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:46.813 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.813 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112039 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6758400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:46.813 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:37:46.813 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.924999454999998, 2024-09-14T22:37:46.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055449709999999999, 2024-09-14T22:37:46.813 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:46.813 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:46.813 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:46.813 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:46.814 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:46.814 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:46.814 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112039 2024-09-14T22:37:46.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.814 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:46.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:46.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:37:46.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:46.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:37:46.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:46.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824941+0000", 2024-09-14T22:37:46.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3699999999999999e-07 2024-09-14T22:37:46.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:46.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824946+0000", 2024-09-14T22:37:46.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0209999999999999e-06 2024-09-14T22:37:46.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:46.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824972+0000", 2024-09-14T22:37:46.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6750999999999998e-05 2024-09-14T22:37:46.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:46.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824982+0000", 2024-09-14T22:37:46.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5019999999999998e-06 2024-09-14T22:37:46.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:46.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880227+0000", 2024-09-14T22:37:46.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055245257999999998 2024-09-14T22:37:46.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:46.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880305+0000", 2024-09-14T22:37:46.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7424999999999998e-05 2024-09-14T22:37:46.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:46.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880389+0000", 2024-09-14T22:37:46.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4815999999999996e-05 2024-09-14T22:37:46.817 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.817 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:46.817 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.817 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112040 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6807552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:46.817 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:37:46.817 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.924942738999999, 2024-09-14T22:37:46.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055534615000000002, 2024-09-14T22:37:46.818 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:46.818 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:46.818 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:46.818 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:46.818 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:46.818 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:46.818 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112040 2024-09-14T22:37:46.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.818 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:46.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:46.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:37:46.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:46.819 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:37:46.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.819 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:46.819 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824997+0000", 2024-09-14T22:37:46.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7199999999999998e-07 2024-09-14T22:37:46.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.819 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:46.819 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825042+0000", 2024-09-14T22:37:46.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5065999999999998e-05 2024-09-14T22:37:46.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.819 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:46.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825060+0000", 2024-09-14T22:37:46.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7520000000000002e-05 2024-09-14T22:37:46.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:46.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825070+0000", 2024-09-14T22:37:46.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8230000000000006e-06 2024-09-14T22:37:46.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:46.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880427+0000", 2024-09-14T22:37:46.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055357244 2024-09-14T22:37:46.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:46.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880483+0000", 2024-09-14T22:37:46.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5807000000000003e-05 2024-09-14T22:37:46.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:46.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880531+0000", 2024-09-14T22:37:46.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8383000000000001e-05 2024-09-14T22:37:46.821 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.821 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:46.821 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.822 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112041 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6856704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:46.822 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:37:46.822 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.924853564999999, 2024-09-14T22:37:46.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055573887000000002, 2024-09-14T22:37:46.822 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:46.822 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:46.822 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:46.822 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:46.822 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:46.822 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:46.822 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112041 2024-09-14T22:37:46.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.822 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:46.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:46.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:37:46.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:46.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:37:46.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:46.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825087+0000", 2024-09-14T22:37:46.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2399999999999996e-07 2024-09-14T22:37:46.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:46.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825091+0000", 2024-09-14T22:37:46.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9729999999999999e-06 2024-09-14T22:37:46.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.824 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:46.824 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825101+0000", 2024-09-14T22:37:46.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.093e-05 2024-09-14T22:37:46.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.824 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:46.824 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825110+0000", 2024-09-14T22:37:46.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1450000000000008e-06 2024-09-14T22:37:46.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.824 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:46.825 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880559+0000", 2024-09-14T22:37:46.825 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055449684999999999 2024-09-14T22:37:46.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.825 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:46.825 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880601+0000", 2024-09-14T22:37:46.825 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1248000000000002e-05 2024-09-14T22:37:46.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.825 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:46.825 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880659+0000", 2024-09-14T22:37:46.825 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8981999999999999e-05 2024-09-14T22:37:46.825 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.825 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:46.825 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.826 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112042 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6905856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:46.826 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:37:46.826 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.924818831, 2024-09-14T22:37:46.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055781415000000001, 2024-09-14T22:37:46.826 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:46.826 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:46.826 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:46.826 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:46.826 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:46.826 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:46.826 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112042 2024-09-14T22:37:46.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.826 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:46.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:46.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:37:46.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:46.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:37:46.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:46.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825478+0000", 2024-09-14T22:37:46.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035775 2024-09-14T22:37:46.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:46.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825494+0000", 2024-09-14T22:37:46.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5994999999999998e-05 2024-09-14T22:37:46.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:46.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825520+0000", 2024-09-14T22:37:46.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5738999999999999e-05 2024-09-14T22:37:46.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:46.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825547+0000", 2024-09-14T22:37:46.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7667000000000001e-05 2024-09-14T22:37:46.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:46.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880725+0000", 2024-09-14T22:37:46.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055177962999999997 2024-09-14T22:37:46.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:46.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880816+0000", 2024-09-14T22:37:46.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0315000000000005e-05 2024-09-14T22:37:46.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:46.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880902+0000", 2024-09-14T22:37:46.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5986e-05 2024-09-14T22:37:46.830 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.830 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:46.830 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.830 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112043 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:46.830 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:37:46.830 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.924370827000001, 2024-09-14T22:37:46.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055537912000000002, 2024-09-14T22:37:46.830 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:46.830 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:46.830 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:46.831 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:46.831 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:46.831 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:46.831 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112043 2024-09-14T22:37:46.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.831 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:46.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:46.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:37:46.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:46.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:37:46.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.832 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:46.832 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825911+0000", 2024-09-14T22:37:46.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00034306899999999999 2024-09-14T22:37:46.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.832 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:46.832 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825918+0000", 2024-09-14T22:37:46.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.002e-06 2024-09-14T22:37:46.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.832 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:46.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825931+0000", 2024-09-14T22:37:46.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2401e-05 2024-09-14T22:37:46.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:46.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825942+0000", 2024-09-14T22:37:46.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0903e-05 2024-09-14T22:37:46.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:46.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880951+0000", 2024-09-14T22:37:46.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055009379999999997 2024-09-14T22:37:46.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.834 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:46.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881027+0000", 2024-09-14T22:37:46.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5944000000000003e-05 2024-09-14T22:37:46.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.834 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:46.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881106+0000", 2024-09-14T22:37:46.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9213000000000002e-05 2024-09-14T22:37:46.834 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.834 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:46.834 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.834 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112044 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 7004160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:46.835 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:37:46.835 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.923977158, 2024-09-14T22:37:46.835 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055265416999999997, 2024-09-14T22:37:46.835 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:46.835 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:46.835 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:46.835 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:46.835 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:46.835 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:46.835 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112044 2024-09-14T22:37:46.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.835 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:46.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.835 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:46.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:37:46.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:46.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:37:46.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:46.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826239+0000", 2024-09-14T22:37:46.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027676000000000003 2024-09-14T22:37:46.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:46.837 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826247+0000", 2024-09-14T22:37:46.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7440000000000007e-06 2024-09-14T22:37:46.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.837 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:46.837 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826275+0000", 2024-09-14T22:37:46.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8053000000000001e-05 2024-09-14T22:37:46.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.837 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:46.837 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826286+0000", 2024-09-14T22:37:46.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1780000000000001e-05 2024-09-14T22:37:46.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.837 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:46.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881137+0000", 2024-09-14T22:37:46.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054850500000000003 2024-09-14T22:37:46.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:46.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881181+0000", 2024-09-14T22:37:46.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4153000000000002e-05 2024-09-14T22:37:46.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:46.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881227+0000", 2024-09-14T22:37:46.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6427000000000001e-05 2024-09-14T22:37:46.838 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.838 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:46.838 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.839 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112045 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 7053312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:46.839 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:37:46.839 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.923614924999999, 2024-09-14T22:37:46.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055045050999999998, 2024-09-14T22:37:46.839 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:46.839 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:46.839 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:46.839 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:46.839 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:46.839 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:46.839 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112045 2024-09-14T22:37:46.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.839 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:46.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:46.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:37:46.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:46.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:37:46.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:46.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:46.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830250+0000", 2024-09-14T22:37:46.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0039258670000000004 2024-09-14T22:37:46.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:46.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830260+0000", 2024-09-14T22:37:46.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0088000000000001e-05 2024-09-14T22:37:46.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:46.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830281+0000", 2024-09-14T22:37:46.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0720000000000002e-05 2024-09-14T22:37:46.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:46.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830292+0000", 2024-09-14T22:37:46.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1455e-05 2024-09-14T22:37:46.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:46.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881255+0000", 2024-09-14T22:37:46.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.050962247000000002 2024-09-14T22:37:46.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:46.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881296+0000", 2024-09-14T22:37:46.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1557999999999998e-05 2024-09-14T22:37:46.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:46.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:46.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:46.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881369+0000", 2024-09-14T22:37:46.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3115999999999996e-05 2024-09-14T22:37:46.842 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.842 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:46.842 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.843 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:46.843 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:46.843 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:46.844 INFO:teuthology.orchestra.run.smithi001.stdout:5012 read 0x1d1855e thru 0x1d25657 (0xd0fa bytes) 2024-09-14T22:37:46.844 INFO:teuthology.orchestra.run.smithi001.stdout:5014 read 0x1d740fa thru 0x1d825c6 (0xe4cd bytes) 2024-09-14T22:37:46.844 INFO:teuthology.orchestra.run.smithi001.stdout:5017 trunc from 0x6e748b5 to 0xe047989 2024-09-14T22:37:46.845 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:37:46.846 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:37:46.846 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-14T22:37:46.848 INFO:teuthology.orchestra.run.smithi001.stdout:5018 punch from 0x5b9eedc to 0x5ba70c2, (0x81e6 bytes) 2024-09-14T22:37:46.848 INFO:teuthology.orchestra.run.smithi001.stdout:5019 read 0x4b21329 thru 0x4b27679 (0x6351 bytes) 2024-09-14T22:37:46.848 INFO:teuthology.orchestra.run.smithi001.stdout:5020 read 0x373c949 thru 0x3746650 (0x9d08 bytes) 2024-09-14T22:37:46.848 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:46.848 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:46.848 INFO:teuthology.orchestra.run.smithi001.stdout:5023 write 0xb823a82 thru 0xb82886d (0x4dec bytes) 2024-09-14T22:37:46.848 INFO:teuthology.orchestra.run.smithi001.stdout:5024 writesame 0x6f12348 thru 0x6f1b73b data_size 0x22(0x93f4 bytes) 2024-09-14T22:37:46.848 INFO:teuthology.orchestra.run.smithi001.stdout:5025 writesame 0xcd1dffe thru 0xcd2266a data_size 0xb(0x466d bytes) 2024-09-14T22:37:46.848 INFO:teuthology.orchestra.run.smithi001.stdout:5026 punch from 0x7a083be to 0x7a17fbc, (0xfbfe bytes) 2024-09-14T22:37:46.849 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-09-14T22:37:45.875126+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1143317549,"num_objects":1333,"num_object_clones":377,"num_object_copies":2666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1333,"num_whiteouts":420,"num_read":160995,"num_read_kb":582212,"num_write":17848,"num_write_kb":267465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":612,"num_bytes_recovered":31733554,"num_keys_recovered":22,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13952,"ondisk_log_size":13952,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":346732,"kb_used_data":45744,"kb_used_omap":17,"kb_used_meta":300782,"kb_avail":754627988,"statfs":{"total":773094113280,"available":772739059712,"internally_reserved":0,"allocated":46841856,"data_stored":375902727,"data_compressed":11175216,"data_compressed_allocated":29474816,"data_compressed_original":366814134,"omap_allocated":18180,"internal_metadata":308001020},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":53229152,"num_objects":172,"num_object_clones":57,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":36,"num_read":11026,"num_read_kb":35785,"num_write":1302,"num_write_kb":10456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.988859"},"pg_stats":[{"pgid":"3.1f","version":"91'258","reported_seq":4174,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:42.832734+0000","last_change":"2024-09-14T22:37:25.739233+0000","last_active":"2024-09-14T22:37:42.832734+0000","last_peered":"2024-09-14T22:37:42.832734+0000","last_clean":"2024-09-14T22:37:42.832734+0000","last_became_active":"2024-09-14T22:37:25.718906+0000","last_became_peered":"2024-09-14T22:37:25.718906+0000","last_unstale":"2024-09-14T22:37:42.832734+0000","last_undegraded":"2024-09-14T22:37:42.832734+0000","last_fullsized":"2024-09-14T22:37:42.832734+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:56:17.951536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017190879999999999,"stat_sum":{"num_bytes":52972300,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":16,"num_read":3921,"num_read_kb":15245,"num_write":272,"num_write_kb":6134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":3277632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"91'951","reported_seq":4997,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:42.378174+0000","last_change":"2024-09-14T22:36:17.733296+0000","last_active":"2024-09-14T22:37:42.378174+0000","last_peered":"2024-09-14T22:37:42.378174+0000","last_clean":"2024-09-14T22:37:42.378174+0000","last_became_active":"2024-09-14T22:36:15.709272+0000","last_became_peered":"2024-09-14T22:36:15.709272+0000","last_unstale":"2024-09-14T22:37:42.378174+0000","last_undegraded":"2024-09-14T22:37:42.378174+0000","last_fullsized":"2024-09-14T22:37:42.378174+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":951,"log_dups_size":0,"ondisk_log_size":951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:19:51.940839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001723619,"stat_sum":{"num_bytes":63686853,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":12,"num_read":9124,"num_read_kb":22999,"num_write":1511,"num_write_kb":8420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"90'527","reported_seq":7806,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:41.865410+0000","last_change":"2024-09-14T22:36:17.702313+0000","last_active":"2024-09-14T22:37:41.865410+0000","last_peered":"2024-09-14T22:37:41.865410+0000","last_clean":"2024-09-14T22:37:41.865410+0000","last_became_active":"2024-09-14T22:36:15.719488+0000","last_became_peered":"2024-09-14T22:36:15.719488+0000","last_unstale":"2024-09-14T22:37:41.865410+0000","last_undegraded":"2024-09-14T22:37:41.865410+0000","last_fullsized":"2024-09-14T22:37:41.865410+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:04:52.705899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053914200000000005,"stat_sum":{"num_bytes":64660877,"num_objects":54,"num_object_clones":19,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":20,"num_read":8600,"num_read_kb":41754,"num_write":680,"num_write_kb":9149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"91'331","reported_seq":4971,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:42.620679+0000","last_change":"2024-09-14T22:36:17.702413+0000","last_active":"2024-09-14T22:37:42.620679+0000","last_peered":"2024-09-14T22:37:42.620679+0000","last_clean":"2024-09-14T22:37:42.620679+0000","last_became_active":"2024-09-14T22:36:15.719470+0000","last_became_peered":"2024-09-14T22:36:15.719470+0000","last_unstale":"2024-09-14T22:37:42.620679+0000","last_undegraded":"2024-09-14T22:37:42.620679+0000","last_fullsized":"2024-09-14T22:37:42.620679+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:03:07.737441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055079499999999997,"stat_sum":{"num_bytes":29978608,"num_objects":48,"num_object_clones":13,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":20,"num_read":4603,"num_read_kb":22592,"num_write":368,"num_write_kb":7766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"91'428","reported_seq":4664,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:42.707529+0000","last_change":"2024-09-14T22:37:25.736579+0000","last_active":"2024-09-14T22:37:42.707529+0000","last_peered":"2024-09-14T22:37:42.707529+0000","last_clean":"2024-09-14T22:37:42.707529+0000","last_became_active":"2024-09-14T22:37:25.718519+0000","last_became_peered":"2024-09-14T22:37:25.718519+0000","last_unstale":"2024-09-14T22:37:42.707529+0000","last_undegraded":"2024-09-14T22:37:42.707529+0000","last_fullsized":"2024-09-14T22:37:42.707529+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:29:58.427859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092291499999999996,"stat_sum":{"num_bytes":26187613,"num_objects":40,"num_object_clones":13,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":14,"num_read":4998,"num_read_kb":28800,"num_write":523,"num_write_kb":9068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":7858104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"91'368","reported_seq":2773,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:42.818780+0000","last_change":"2024-09-14T22:36:17.711364+0000","last_active":"2024-09-14T22:37:42.818780+0000","last_peered":"2024-09-14T22:37:42.818780+0000","last_clean":"2024-09-14T22:37:42.818780+0000","last_became_active":"2024-09-14T22:36:15.701852+0000","last_became_peered":"2024-09-14T22:36:15.701852+0000","last_unstale":"2024-09-14T22:37:42.818780+0000","last_undegraded":"2024-09-14T22:37:42.818780+0000","last_fullsized":"2024-09-14T22:37:42.818780+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:38:30.655859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012502780000000001,"stat_sum":{"num_bytes":9354713,"num_objects":51,"num_object_clones":16,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":20,"num_read":3362,"num_read_kb":9524,"num_write":385,"num_write_kb":6504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"90'333","reported_seq":4998,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:41.851350+0000","last_change":"2024-09-14T22:36:17.705423+0000","last_active":"2024-09-14T22:37:41.851350+0000","last_peered":"2024-09-14T22:37:41.851350+0000","last_clean":"2024-09-14T22:37:41.851350+0000","last_became_active":"2024-09-14T22:36:15.700990+0000","last_became_peered":"2024-09-14T22:36:15.700990+0000","last_unstale":"2024-09-14T22:37:41.851350+0000","last_undegraded":"2024-09-14T22:37:41.851350+0000","last_fullsized":"2024-09-14T22:37:41.851350+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:12:54.070421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010520620000000001,"stat_sum":{"num_bytes":20479200,"num_objects":35,"num_object_clones":10,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":13,"num_read":5025,"num_read_kb":15102,"num_write":349,"num_write_kb":5341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"91'248","reported_seq":2858,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:42.696758+0000","last_change":"2024-09-14T22:37:25.719758+0000","last_active":"2024-09-14T22:37:42.696758+0000","last_peered":"2024-09-14T22:37:42.696758+0000","last_clean":"2024-09-14T22:37:42.696758+0000","last_became_active":"2024-09-14T22:37:25.717354+0000","last_became_peered":"2024-09-14T22:37:25.717354+0000","last_unstale":"2024-09-14T22:37:42.696758+0000","last_undegraded":"2024-09-14T22:37:42.696758+0000","last_fullsized":"2024-09-14T22:37:42.696758+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:10:54.913067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002063484,"stat_sum":{"num_bytes":20564030,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":5,"num_read":2620,"num_read_kb":20131,"num_write":316,"num_write_kb":11771,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":633727,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"90'255","reported_seq":3288,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:42.208923+0000","last_change":"2024-09-14T22:36:17.721195+0000","last_active":"2024-09-14T22:37:42.208923+0000","last_peered":"2024-09-14T22:37:42.208923+0000","last_clean":"2024-09-14T22:37:42.208923+0000","last_became_active":"2024-09-14T22:36:15.715869+0000","last_became_peered":"2024-09-14T22:36:15.715869+0000","last_unstale":"2024-09-14T22:37:42.208923+0000","last_undegraded":"2024-09-14T22:37:42.208923+0000","last_fullsized":"2024-09-14T22:37:42.208923+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:20:53.130283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001160672,"stat_sum":{"num_bytes":15152035,"num_objects":32,"num_object_clones":9,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":12,"num_read":3276,"num_read_kb":13854,"num_write":290,"num_write_kb":15006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"91'403","reported_seq":3935,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:42.245853+0000","last_change":"2024-09-14T22:36:17.703048+0000","last_active":"2024-09-14T22:37:42.245853+0000","last_peered":"2024-09-14T22:37:42.245853+0000","last_clean":"2024-09-14T22:37:42.245853+0000","last_became_active":"2024-09-14T22:36:15.725078+0000","last_became_peered":"2024-09-14T22:36:15.725078+0000","last_unstale":"2024-09-14T22:37:42.245853+0000","last_undegraded":"2024-09-14T22:37:42.245853+0000","last_fullsized":"2024-09-14T22:37:42.245853+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:17:44.645112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010500290000000001,"stat_sum":{"num_bytes":61634392,"num_objects":55,"num_object_clones":16,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":16,"num_read":4400,"num_read_kb":29007,"num_write":541,"num_write_kb":11868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"91'244","reported_seq":3105,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:42.830831+0000","last_change":"2024-09-14T22:36:18.009371+0000","last_active":"2024-09-14T22:37:42.830831+0000","last_peered":"2024-09-14T22:37:42.830831+0000","last_clean":"2024-09-14T22:37:42.830831+0000","last_became_active":"2024-09-14T22:36:15.717341+0000","last_became_peered":"2024-09-14T22:36:15.717341+0000","last_unstale":"2024-09-14T22:37:42.830831+0000","last_undegraded":"2024-09-14T22:37:42.830831+0000","last_fullsized":"2024-09-14T22:37:42.830831+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:14:41.195262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054571999999999997,"stat_sum":{"num_bytes":11870613,"num_objects":29,"num_object_clones":9,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":9,"num_read":2948,"num_read_kb":3905,"num_write":262,"num_write_kb":5802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"91'933","reported_seq":7055,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:42.227368+0000","last_change":"2024-09-14T22:36:17.711384+0000","last_active":"2024-09-14T22:37:42.227368+0000","last_peered":"2024-09-14T22:37:42.227368+0000","last_clean":"2024-09-14T22:37:42.227368+0000","last_became_active":"2024-09-14T22:36:15.723075+0000","last_became_peered":"2024-09-14T22:36:15.723075+0000","last_unstale":"2024-09-14T22:37:42.227368+0000","last_undegraded":"2024-09-14T22:37:42.227368+0000","last_fullsized":"2024-09-14T22:37:42.227368+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":933,"log_dups_size":0,"ondisk_log_size":933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:02:40.441795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010804009999999999,"stat_sum":{"num_bytes":38277680,"num_objects":52,"num_object_clones":15,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":15,"num_read":9260,"num_read_kb":22156,"num_write":1351,"num_write_kb":10253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":157,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:41.859922+0000","last_change":"2024-09-14T22:36:08.042860+0000","last_active":"2024-09-14T22:37:41.859922+0000","last_peered":"2024-09-14T22:37:41.859922+0000","last_clean":"2024-09-14T22:37:41.859922+0000","last_became_active":"2024-09-14T22:36:06.040961+0000","last_became_peered":"2024-09-14T22:36:06.040961+0000","last_unstale":"2024-09-14T22:37:41.859922+0000","last_undegraded":"2024-09-14T22:37:41.859922+0000","last_fullsized":"2024-09-14T22:37:41.859922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:03:48.547834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010780950000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"91'400","reported_seq":2019,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:42.728242+0000","last_change":"2024-09-14T22:36:17.720897+0000","last_active":"2024-09-14T22:37:42.728242+0000","last_peered":"2024-09-14T22:37:42.728242+0000","last_clean":"2024-09-14T22:37:42.728242+0000","last_became_active":"2024-09-14T22:36:15.700982+0000","last_became_peered":"2024-09-14T22:36:15.700982+0000","last_unstale":"2024-09-14T22:37:42.728242+0000","last_undegraded":"2024-09-14T22:37:42.728242+0000","last_fullsized":"2024-09-14T22:37:42.728242+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:41.826193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010495459999999999,"stat_sum":{"num_bytes":22290396,"num_objects":47,"num_object_clones":14,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":14,"num_read":3116,"num_read_kb":10608,"num_write":402,"num_write_kb":8558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":107,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:41.859836+0000","last_change":"2024-09-14T22:36:41.930087+0000","last_active":"2024-09-14T22:37:41.859836+0000","last_peered":"2024-09-14T22:37:41.859836+0000","last_clean":"2024-09-14T22:37:41.859836+0000","last_became_active":"2024-09-14T22:36:41.929522+0000","last_became_peered":"2024-09-14T22:36:41.929522+0000","last_unstale":"2024-09-14T22:37:41.859836+0000","last_undegraded":"2024-09-14T22:37:41.859836+0000","last_fullsized":"2024-09-14T22:37:41.859836+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:15:10.589042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"91'360","reported_seq":4539,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:42.863032+0000","last_change":"2024-09-14T22:36:21.722517+0000","last_active":"2024-09-14T22:37:42.863032+0000","last_peered":"2024-09-14T22:37:42.863032+0000","last_clean":"2024-09-14T22:37:42.863032+0000","last_became_active":"2024-09-14T22:36:21.722085+0000","last_became_peered":"2024-09-14T22:36:21.722085+0000","last_unstale":"2024-09-14T22:37:42.863032+0000","last_undegraded":"2024-09-14T22:37:42.863032+0000","last_fullsized":"2024-09-14T22:37:42.863032+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:01:13.584587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020774109999999999,"stat_sum":{"num_bytes":49890702,"num_objects":40,"num_object_clones":8,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":11,"num_read":5129,"num_read_kb":23446,"num_write":422,"num_write_kb":7153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":157,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:41.851994+0000","last_change":"2024-09-14T22:36:08.054606+0000","last_active":"2024-09-14T22:37:41.851994+0000","last_peered":"2024-09-14T22:37:41.851994+0000","last_clean":"2024-09-14T22:37:41.851994+0000","last_became_active":"2024-09-14T22:36:06.266908+0000","last_became_peered":"2024-09-14T22:36:06.266908+0000","last_unstale":"2024-09-14T22:37:41.851994+0000","last_undegraded":"2024-09-14T22:37:41.851994+0000","last_fullsized":"2024-09-14T22:37:41.851994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:36.738643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098249300000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"90'1194","reported_seq":3853,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-14T22:37:41.626327+0000","last_change":"2024-09-14T22:37:17.676125+0000","last_active":"2024-09-14T22:37:41.626327+0000","last_peered":"2024-09-14T22:37:41.626327+0000","last_clean":"2024-09-14T22:37:41.626327+0000","last_became_active":"2024-09-14T22:37:17.674322+0000","last_became_peered":"2024-09-14T22:37:17.674322+0000","last_unstale":"2024-09-14T22:37:41.626327+0000","last_undegraded":"2024-09-14T22:37:41.626327+0000","last_fullsized":"2024-09-14T22:37:41.626327+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1194,"log_dups_size":0,"ondisk_log_size":1194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:41:25.349631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012564830000000001,"stat_sum":{"num_bytes":16528446,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":16,"num_read":7830,"num_read_kb":14616,"num_write":2104,"num_write_kb":9478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":4264139,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":107,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:41.906026+0000","last_change":"2024-09-14T22:36:41.930961+0000","last_active":"2024-09-14T22:37:41.906026+0000","last_peered":"2024-09-14T22:37:41.906026+0000","last_clean":"2024-09-14T22:37:41.906026+0000","last_became_active":"2024-09-14T22:36:41.930619+0000","last_became_peered":"2024-09-14T22:36:41.930619+0000","last_unstale":"2024-09-14T22:37:41.906026+0000","last_undegraded":"2024-09-14T22:37:41.906026+0000","last_fullsized":"2024-09-14T22:37:41.906026+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:04:40.582999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"91'403","reported_seq":3111,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:42.854471+0000","last_change":"2024-09-14T22:36:17.702597+0000","last_active":"2024-09-14T22:37:42.854471+0000","last_peered":"2024-09-14T22:37:42.854471+0000","last_clean":"2024-09-14T22:37:42.854471+0000","last_became_active":"2024-09-14T22:36:15.717186+0000","last_became_peered":"2024-09-14T22:36:15.717186+0000","last_unstale":"2024-09-14T22:37:42.854471+0000","last_undegraded":"2024-09-14T22:37:42.854471+0000","last_fullsized":"2024-09-14T22:37:42.854471+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:36:03.641785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071220099999999998,"stat_sum":{"num_bytes":23002533,"num_objects":40,"num_object_clones":12,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":14,"num_read":4111,"num_read_kb":6122,"num_write":379,"num_write_kb":2787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"90'267","reported_seq":3718,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:41.854780+0000","last_change":"2024-09-14T22:36:17.711068+0000","last_active":"2024-09-14T22:37:41.854780+0000","last_peered":"2024-09-14T22:37:41.854780+0000","last_clean":"2024-09-14T22:37:41.854780+0000","last_became_active":"2024-09-14T22:36:15.706147+0000","last_became_peered":"2024-09-14T22:36:15.706147+0000","last_unstale":"2024-09-14T22:37:41.854780+0000","last_undegraded":"2024-09-14T22:37:41.854780+0000","last_fullsized":"2024-09-14T22:37:41.854780+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:35:05.725994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082625500000000002,"stat_sum":{"num_bytes":37489798,"num_objects":28,"num_object_clones":9,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":9,"num_read":3361,"num_read_kb":16744,"num_write":256,"num_write_kb":5734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":159,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:41.865059+0000","last_change":"2024-09-14T22:36:08.055938+0000","last_active":"2024-09-14T22:37:41.865059+0000","last_peered":"2024-09-14T22:37:41.865059+0000","last_clean":"2024-09-14T22:37:41.865059+0000","last_became_active":"2024-09-14T22:36:06.039600+0000","last_became_peered":"2024-09-14T22:36:06.039600+0000","last_unstale":"2024-09-14T22:37:41.865059+0000","last_undegraded":"2024-09-14T22:37:41.865059+0000","last_fullsized":"2024-09-14T22:37:41.865059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:49:09.245563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021617870000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"90'748","reported_seq":8137,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-14T22:37:41.502802+0000","last_change":"2024-09-14T22:37:16.669227+0000","last_active":"2024-09-14T22:37:41.502802+0000","last_peered":"2024-09-14T22:37:41.502802+0000","last_clean":"2024-09-14T22:37:41.502802+0000","last_became_active":"2024-09-14T22:37:16.662230+0000","last_became_peered":"2024-09-14T22:37:16.662230+0000","last_unstale":"2024-09-14T22:37:41.502802+0000","last_undegraded":"2024-09-14T22:37:41.502802+0000","last_fullsized":"2024-09-14T22:37:41.502802+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":748,"log_dups_size":0,"ondisk_log_size":748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:05:06.078317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021500769999999998,"stat_sum":{"num_bytes":65255395,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":13,"num_read":9399,"num_read_kb":19607,"num_write":1061,"num_write_kb":9939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":4851034,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":81,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-14T22:37:40.842914+0000","last_change":"2024-09-14T22:37:13.762987+0000","last_active":"2024-09-14T22:37:40.842914+0000","last_peered":"2024-09-14T22:37:40.842914+0000","last_clean":"2024-09-14T22:37:40.842914+0000","last_became_active":"2024-09-14T22:37:13.762641+0000","last_became_peered":"2024-09-14T22:37:13.762641+0000","last_unstale":"2024-09-14T22:37:40.842914+0000","last_undegraded":"2024-09-14T22:37:40.842914+0000","last_fullsized":"2024-09-14T22:37:40.842914+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:30:59.887867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"91'297","reported_seq":3767,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:42.796336+0000","last_change":"2024-09-14T22:36:17.720438+0000","last_active":"2024-09-14T22:37:42.796336+0000","last_peered":"2024-09-14T22:37:42.796336+0000","last_clean":"2024-09-14T22:37:42.796336+0000","last_became_active":"2024-09-14T22:36:15.711105+0000","last_became_peered":"2024-09-14T22:36:15.711105+0000","last_unstale":"2024-09-14T22:37:42.796336+0000","last_undegraded":"2024-09-14T22:37:42.796336+0000","last_fullsized":"2024-09-14T22:37:42.796336+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:58:02.935971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085549400000000005,"stat_sum":{"num_bytes":53204842,"num_objects":33,"num_object_clones":10,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":10,"num_read":3519,"num_read_kb":16057,"num_write":316,"num_write_kb":2571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":162,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:41.856074+0000","last_change":"2024-09-14T22:37:41.856074+0000","last_active":"2024-09-14T22:37:41.856074+0000","last_peered":"2024-09-14T22:37:41.856074+0000","last_clean":"2024-09-14T22:37:41.856074+0000","last_became_active":"2024-09-14T22:37:41.855775+0000","last_became_peered":"2024-09-14T22:37:41.855775+0000","last_unstale":"2024-09-14T22:37:41.856074+0000","last_undegraded":"2024-09-14T22:37:41.856074+0000","last_fullsized":"2024-09-14T22:37:41.856074+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:43:12.528016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010817520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'950","reported_seq":3494,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:42.411318+0000","last_change":"2024-09-14T22:36:51.718064+0000","last_active":"2024-09-14T22:37:42.411318+0000","last_peered":"2024-09-14T22:37:42.411318+0000","last_clean":"2024-09-14T22:37:42.411318+0000","last_became_active":"2024-09-14T22:36:51.715823+0000","last_became_peered":"2024-09-14T22:36:51.715823+0000","last_unstale":"2024-09-14T22:37:42.411318+0000","last_undegraded":"2024-09-14T22:37:42.411318+0000","last_fullsized":"2024-09-14T22:37:42.411318+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":950,"log_dups_size":0,"ondisk_log_size":950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:20:32.332879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48794304,"num_objects":50,"num_object_clones":12,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":12,"num_read":6845,"num_read_kb":10993,"num_write":1453,"num_write_kb":6440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":173209,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":165,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:41.865087+0000","last_change":"2024-09-14T22:37:13.625072+0000","last_active":"2024-09-14T22:37:41.865087+0000","last_peered":"2024-09-14T22:37:41.865087+0000","last_clean":"2024-09-14T22:37:41.865087+0000","last_became_active":"2024-09-14T22:37:13.624669+0000","last_became_peered":"2024-09-14T22:37:13.624669+0000","last_unstale":"2024-09-14T22:37:41.865087+0000","last_undegraded":"2024-09-14T22:37:41.865087+0000","last_fullsized":"2024-09-14T22:37:41.865087+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:54:20.545098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011401880000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":308,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:41.851789+0000","last_change":"2024-09-14T22:36:00.402862+0000","last_active":"2024-09-14T22:37:41.851789+0000","last_peered":"2024-09-14T22:37:41.851789+0000","last_clean":"2024-09-14T22:37:41.851789+0000","last_became_active":"2024-09-14T22:36:00.400529+0000","last_became_peered":"2024-09-14T22:36:00.400529+0000","last_unstale":"2024-09-14T22:37:41.851789+0000","last_undegraded":"2024-09-14T22:37:41.851789+0000","last_fullsized":"2024-09-14T22:37:41.851789+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_clean_scrub_stamp":"2024-09-14T22:35:58.989025+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:07:12.951728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"91'437","reported_seq":5475,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:42.881836+0000","last_change":"2024-09-14T22:37:37.823251+0000","last_active":"2024-09-14T22:37:42.881836+0000","last_peered":"2024-09-14T22:37:42.881836+0000","last_clean":"2024-09-14T22:37:42.881836+0000","last_became_active":"2024-09-14T22:37:37.820975+0000","last_became_peered":"2024-09-14T22:37:37.820975+0000","last_unstale":"2024-09-14T22:37:42.881836+0000","last_undegraded":"2024-09-14T22:37:42.881836+0000","last_fullsized":"2024-09-14T22:37:42.881836+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:36:17.103526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001381511,"stat_sum":{"num_bytes":30784306,"num_objects":41,"num_object_clones":10,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":11,"num_read":5938,"num_read_kb":11838,"num_write":637,"num_write_kb":12197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":6470986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"91'270","reported_seq":5869,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:42.351307+0000","last_change":"2024-09-14T22:37:16.658991+0000","last_active":"2024-09-14T22:37:42.351307+0000","last_peered":"2024-09-14T22:37:42.351307+0000","last_clean":"2024-09-14T22:37:42.351307+0000","last_became_active":"2024-09-14T22:37:16.657865+0000","last_became_peered":"2024-09-14T22:37:16.657865+0000","last_unstale":"2024-09-14T22:37:42.351307+0000","last_undegraded":"2024-09-14T22:37:42.351307+0000","last_fullsized":"2024-09-14T22:37:42.351307+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:35:46.389832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060272699999999995,"stat_sum":{"num_bytes":53691152,"num_objects":52,"num_object_clones":16,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":14,"num_read":6040,"num_read_kb":12950,"num_write":260,"num_write_kb":5348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":4204723,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"91'507","reported_seq":3300,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:42.263043+0000","last_change":"2024-09-14T22:36:17.776247+0000","last_active":"2024-09-14T22:37:42.263043+0000","last_peered":"2024-09-14T22:37:42.263043+0000","last_clean":"2024-09-14T22:37:42.263043+0000","last_became_active":"2024-09-14T22:36:15.718794+0000","last_became_peered":"2024-09-14T22:36:15.718794+0000","last_unstale":"2024-09-14T22:37:42.263043+0000","last_undegraded":"2024-09-14T22:37:42.263043+0000","last_fullsized":"2024-09-14T22:37:42.263043+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:13:31.998994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088662699999999997,"stat_sum":{"num_bytes":20998515,"num_objects":38,"num_object_clones":12,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":8,"num_read":5250,"num_read_kb":21884,"num_write":489,"num_write_kb":14235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"90'335","reported_seq":1815,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:42.236100+0000","last_change":"2024-09-14T22:36:17.733002+0000","last_active":"2024-09-14T22:37:42.236100+0000","last_peered":"2024-09-14T22:37:42.236100+0000","last_clean":"2024-09-14T22:37:42.236100+0000","last_became_active":"2024-09-14T22:36:15.702885+0000","last_became_peered":"2024-09-14T22:36:15.702885+0000","last_unstale":"2024-09-14T22:37:42.236100+0000","last_undegraded":"2024-09-14T22:37:42.236100+0000","last_fullsized":"2024-09-14T22:37:42.236100+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:13:36.985168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015180580000000001,"stat_sum":{"num_bytes":34442243,"num_objects":42,"num_object_clones":14,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":15,"num_read":2450,"num_read_kb":9727,"num_write":311,"num_write_kb":6448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"90'299","reported_seq":5372,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:41.865835+0000","last_change":"2024-09-14T22:36:17.702738+0000","last_active":"2024-09-14T22:37:41.865835+0000","last_peered":"2024-09-14T22:37:41.865835+0000","last_clean":"2024-09-14T22:37:41.865835+0000","last_became_active":"2024-09-14T22:36:15.713452+0000","last_became_peered":"2024-09-14T22:36:15.713452+0000","last_unstale":"2024-09-14T22:37:41.865835+0000","last_undegraded":"2024-09-14T22:37:41.865835+0000","last_fullsized":"2024-09-14T22:37:41.865835+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:40:01.781986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082782099999999996,"stat_sum":{"num_bytes":34902255,"num_objects":37,"num_object_clones":14,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":14,"num_read":5168,"num_read_kb":17038,"num_write":332,"num_write_kb":9038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"91'259","reported_seq":3176,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:42.675279+0000","last_change":"2024-09-14T22:36:18.009776+0000","last_active":"2024-09-14T22:37:42.675279+0000","last_peered":"2024-09-14T22:37:42.675279+0000","last_clean":"2024-09-14T22:37:42.675279+0000","last_became_active":"2024-09-14T22:36:15.723205+0000","last_became_peered":"2024-09-14T22:36:15.723205+0000","last_unstale":"2024-09-14T22:37:42.675279+0000","last_undegraded":"2024-09-14T22:37:42.675279+0000","last_fullsized":"2024-09-14T22:37:42.675279+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:00:39.176173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087783999999999996,"stat_sum":{"num_bytes":25837926,"num_objects":46,"num_object_clones":13,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":13,"num_read":2846,"num_read_kb":13649,"num_write":269,"num_write_kb":5214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"90'358","reported_seq":1779,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:41.851656+0000","last_change":"2024-09-14T22:36:18.009666+0000","last_active":"2024-09-14T22:37:41.851656+0000","last_peered":"2024-09-14T22:37:41.851656+0000","last_clean":"2024-09-14T22:37:41.851656+0000","last_became_active":"2024-09-14T22:36:15.715659+0000","last_became_peered":"2024-09-14T22:36:15.715659+0000","last_unstale":"2024-09-14T22:37:41.851656+0000","last_undegraded":"2024-09-14T22:37:41.851656+0000","last_fullsized":"2024-09-14T22:37:41.851656+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:23:29.533969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000776448,"stat_sum":{"num_bytes":28271028,"num_objects":35,"num_object_clones":10,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":10,"num_read":2201,"num_read_kb":8070,"num_write":444,"num_write_kb":3580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"91'276","reported_seq":7375,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:42.719643+0000","last_change":"2024-09-14T22:36:18.010182+0000","last_active":"2024-09-14T22:37:42.719643+0000","last_peered":"2024-09-14T22:37:42.719643+0000","last_clean":"2024-09-14T22:37:42.719643+0000","last_became_active":"2024-09-14T22:36:15.719693+0000","last_became_peered":"2024-09-14T22:36:15.719693+0000","last_unstale":"2024-09-14T22:37:42.719643+0000","last_undegraded":"2024-09-14T22:37:42.719643+0000","last_fullsized":"2024-09-14T22:37:42.719643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:03:55.439843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001045011,"stat_sum":{"num_bytes":23774922,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":10,"num_read":7607,"num_read_kb":21143,"num_write":343,"num_write_kb":9261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"91'333","reported_seq":3217,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:42.659273+0000","last_change":"2024-09-14T22:37:26.718624+0000","last_active":"2024-09-14T22:37:42.659273+0000","last_peered":"2024-09-14T22:37:42.659273+0000","last_clean":"2024-09-14T22:37:42.659273+0000","last_became_active":"2024-09-14T22:37:26.717088+0000","last_became_peered":"2024-09-14T22:37:26.717088+0000","last_unstale":"2024-09-14T22:37:42.659273+0000","last_undegraded":"2024-09-14T22:37:42.659273+0000","last_fullsized":"2024-09-14T22:37:42.659273+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:39:58.595235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094227299999999998,"stat_sum":{"num_bytes":36333375,"num_objects":46,"num_object_clones":15,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":16,"num_read":3005,"num_read_kb":24730,"num_write":390,"num_write_kb":12886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"91'263","reported_seq":1764,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:42.693579+0000","last_change":"2024-09-14T22:36:18.009347+0000","last_active":"2024-09-14T22:37:42.693579+0000","last_peered":"2024-09-14T22:37:42.693579+0000","last_clean":"2024-09-14T22:37:42.693579+0000","last_became_active":"2024-09-14T22:36:15.715569+0000","last_became_peered":"2024-09-14T22:36:15.715569+0000","last_unstale":"2024-09-14T22:37:42.693579+0000","last_undegraded":"2024-09-14T22:37:42.693579+0000","last_fullsized":"2024-09-14T22:37:42.693579+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:50:54.587533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054921600000000005,"stat_sum":{"num_bytes":30169208,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":12,"num_read":1973,"num_read_kb":10251,"num_write":325,"num_write_kb":10025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"89'267","reported_seq":6609,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:42.566555+0000","last_change":"2024-09-14T22:36:17.720448+0000","last_active":"2024-09-14T22:37:42.566555+0000","last_peered":"2024-09-14T22:37:42.566555+0000","last_clean":"2024-09-14T22:37:42.566555+0000","last_became_active":"2024-09-14T22:36:15.716696+0000","last_became_peered":"2024-09-14T22:36:15.716696+0000","last_unstale":"2024-09-14T22:37:42.566555+0000","last_undegraded":"2024-09-14T22:37:42.566555+0000","last_fullsized":"2024-09-14T22:37:42.566555+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:00.692077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086547600000000001,"stat_sum":{"num_bytes":39150119,"num_objects":39,"num_object_clones":9,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":13,"num_read":6981,"num_read_kb":35079,"num_write":254,"num_write_kb":2543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"91'330","reported_seq":5855,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T22:37:42.726356+0000","last_change":"2024-09-14T22:36:18.010088+0000","last_active":"2024-09-14T22:37:42.726356+0000","last_peered":"2024-09-14T22:37:42.726356+0000","last_clean":"2024-09-14T22:37:42.726356+0000","last_became_active":"2024-09-14T22:36:15.715866+0000","last_became_peered":"2024-09-14T22:36:15.715866+0000","last_unstale":"2024-09-14T22:37:42.726356+0000","last_undegraded":"2024-09-14T22:37:42.726356+0000","last_fullsized":"2024-09-14T22:37:42.726356+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:03:44.055091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010286189999999999,"stat_sum":{"num_bytes":53096783,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":13,"num_read":5973,"num_read_kb":32491,"num_write":392,"num_write_kb":14530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1142727162,"num_objects":1329,"num_object_clones":377,"num_object_copies":2658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1329,"num_whiteouts":420,"num_read":160879,"num_read_kb":582112,"num_write":17687,"num_write_kb":265047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":612,"num_bytes_recovered":31733554,"num_keys_recovered":22,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":38830080,"data_stored":380782230,"data_compressed":11699338,"data_compressed_allocated":29986816,"data_compressed_original":375604150,"omap_allocated":4238,"internal_metadata":0},"log_size":13832,"ondisk_log_size":13832,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542168,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33604,"kb_used_data":6552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338236,"statfs":{"total":96636764160,"available":96602353664,"internally_reserved":0,"allocated":6709248,"data_stored":48787779,"data_compressed":1796324,"data_compressed_allocated":4194304,"data_compressed_original":47458288,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542169,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38860,"kb_used_data":3512,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332980,"statfs":{"total":96636764160,"available":96596971520,"internally_reserved":0,"allocated":3596288,"data_stored":29410833,"data_compressed":666193,"data_compressed_allocated":2039808,"data_compressed_original":28652611,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542169,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33492,"kb_used_data":6468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338348,"statfs":{"total":96636764160,"available":96602468352,"internally_reserved":0,"allocated":6623232,"data_stored":50971367,"data_compressed":1588616,"data_compressed_allocated":4247552,"data_compressed_original":49698958,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542168,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58540,"kb_used_data":9304,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94313300,"statfs":{"total":96636764160,"available":96576819200,"internally_reserved":0,"allocated":9527296,"data_stored":86025270,"data_compressed":2425894,"data_compressed_allocated":6680576,"data_compressed_original":84509261,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542169,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58608,"kb_used_data":5208,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313232,"statfs":{"total":96636764160,"available":96576749568,"internally_reserved":0,"allocated":5332992,"data_stored":40730294,"data_compressed":1504949,"data_compressed_allocated":3178496,"data_compressed_original":39614470,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":66,"seq":283467841543,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35964,"kb_used_data":3300,"kb_used_omap":6,"kb_used_meta":32633,"kb_avail":94335876,"statfs":{"total":96636764160,"available":96599937024,"internally_reserved":0,"allocated":3379200,"data_stored":25004205,"data_compressed":455862,"data_compressed_allocated":1794048,"data_compressed_original":24168618,"omap_allocated":7025,"internal_metadata":33416335},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53648,"kb_used_data":4412,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318192,"statfs":{"total":96636764160,"available":96581828608,"internally_reserved":0,"allocated":4517888,"data_stored":32764308,"data_compressed":984439,"data_compressed_allocated":2588672,"data_compressed_original":31762266,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542168,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34016,"kb_used_data":6988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337824,"statfs":{"total":96636764160,"available":96601931776,"internally_reserved":0,"allocated":7155712,"data_stored":62208671,"data_compressed":1752939,"data_compressed_allocated":4751360,"data_compressed_original":60949662,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6037504,"data_stored":61140863,"data_compressed":1757379,"data_compressed_allocated":4726784,"data_compressed_original":60392606,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3452928,"data_stored":32276614,"data_compressed":984439,"data_compressed_allocated":2588672,"data_compressed_original":31762266,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2318336,"data_stored":24516530,"data_compressed":455862,"data_compressed_allocated":1794048,"data_compressed_original":24168618,"omap_allocated":4238,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4292608,"data_stored":40265896,"data_compressed":1512673,"data_compressed_allocated":3186688,"data_compressed_original":39626758,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8613888,"data_stored":87250841,"data_compressed":2597247,"data_compressed_allocated":6823936,"data_compressed_original":86221389,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5558272,"data_stored":50492967,"data_compressed":1596340,"data_compressed_allocated":4255744,"data_compressed_original":49711246,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2740224,"data_stored":32609975,"data_compressed":858413,"data_compressed_allocated":2248704,"data_compressed_original":32339011,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5816320,"data_stored":52228544,"data_compressed":1936985,"data_compressed_allocated":4362240,"data_compressed_original":51382256,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:37:46.852 INFO:tasks.thrashosds.thrasher:Setting 3.1d to [1, 2] 2024-09-14T22:37:46.852 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.1d', '1', '2'] 2024-09-14T22:37:46.852 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.1d 1 2 2024-09-14T22:37:46.852 INFO:teuthology.orchestra.run.smithi001.stdout:5027 trunc from 0xe047989 to 0x6242760 2024-09-14T22:37:46.853 INFO:teuthology.orchestra.run.smithi001.stdout:5028 trunc from 0x6242760 to 0x38fa372 2024-09-14T22:37:46.853 INFO:teuthology.orchestra.run.smithi001.stdout:5029 clone 32 order 22 su 524288 sc 15 2024-09-14T22:37:46.853 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:46.839+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:46.896 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T22:37:46.979 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:46.979 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:46.979 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:37:46.979 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:46.980 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:46.979+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:46.996 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:37:47.000 INFO:teuthology.orchestra.run.smithi001.stdout:leaving image image_client.0-clone31 intact 2024-09-14T22:37:47.046 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:47.043+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:47.109 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-14T22:37:47.121 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:47.121 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:47.121 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:37:47.121 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:37:47.121 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:47.136 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:37:47.148 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:47.143+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:47.247 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:47.238+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:47.263 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:47.263 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:47.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:47.263 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:47.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.264 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97197 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 796672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:47.264 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:37:47.264 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.188720939, 2024-09-14T22:37:47.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12434473, 2024-09-14T22:37:47.264 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.264 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.264 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.264 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.264 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.264 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.264 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97197 2024-09-14T22:37:47.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.264 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:37:47.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:37:47.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070524+0000", 2024-09-14T22:37:47.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0299999999999998e-07 2024-09-14T22:37:47.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070527+0000", 2024-09-14T22:37:47.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9569999999999998e-06 2024-09-14T22:37:47.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070535+0000", 2024-09-14T22:37:47.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7630000000000006e-06 2024-09-14T22:37:47.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070543+0000", 2024-09-14T22:37:47.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4869999999999996e-06 2024-09-14T22:37:47.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194525+0000", 2024-09-14T22:37:47.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123982096 2024-09-14T22:37:47.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194699+0000", 2024-09-14T22:37:47.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000174105 2024-09-14T22:37:47.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194868+0000", 2024-09-14T22:37:47.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016861900000000001 2024-09-14T22:37:47.267 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.267 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.268 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.268 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97198 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 805888~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:47.268 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:37:47.268 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.188692562999996, 2024-09-14T22:37:47.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.1248088, 2024-09-14T22:37:47.268 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.268 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.268 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.268 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.268 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.268 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.268 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97198 2024-09-14T22:37:47.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.268 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:37:47.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:37:47.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070552+0000", 2024-09-14T22:37:47.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T22:37:47.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070555+0000", 2024-09-14T22:37:47.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.853e-06 2024-09-14T22:37:47.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070565+0000", 2024-09-14T22:37:47.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.048e-05 2024-09-14T22:37:47.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070573+0000", 2024-09-14T22:37:47.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7200000000000006e-06 2024-09-14T22:37:47.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194954+0000", 2024-09-14T22:37:47.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12438089400000001 2024-09-14T22:37:47.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195145+0000", 2024-09-14T22:37:47.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019091299999999999 2024-09-14T22:37:47.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.272 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.272 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.272 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195360+0000", 2024-09-14T22:37:47.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021527300000000001 2024-09-14T22:37:47.272 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.272 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.272 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.272 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.272 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97199 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 815104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:47.272 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:37:47.272 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.188661965999998, 2024-09-14T22:37:47.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125251901, 2024-09-14T22:37:47.273 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.273 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.273 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.273 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.273 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.273 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.273 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97199 2024-09-14T22:37:47.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.273 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:37:47.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.274 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:37:47.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.274 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070583+0000", 2024-09-14T22:37:47.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6300000000000005e-07 2024-09-14T22:37:47.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.274 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070586+0000", 2024-09-14T22:37:47.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8270000000000002e-06 2024-09-14T22:37:47.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070597+0000", 2024-09-14T22:37:47.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1960000000000001e-05 2024-09-14T22:37:47.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070605+0000", 2024-09-14T22:37:47.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6459999999999993e-06 2024-09-14T22:37:47.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195432+0000", 2024-09-14T22:37:47.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124826926 2024-09-14T22:37:47.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195626+0000", 2024-09-14T22:37:47.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019357099999999999 2024-09-14T22:37:47.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195834+0000", 2024-09-14T22:37:47.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020830800000000001 2024-09-14T22:37:47.276 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.276 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.276 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.277 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97200 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 824320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:47.277 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:37:47.277 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.188629919999997, 2024-09-14T22:37:47.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125693477, 2024-09-14T22:37:47.277 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.277 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.277 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.277 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.277 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.277 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.277 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97200 2024-09-14T22:37:47.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.278 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:37:47.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:37:47.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070615+0000", 2024-09-14T22:37:47.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T22:37:47.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070618+0000", 2024-09-14T22:37:47.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.943e-06 2024-09-14T22:37:47.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070627+0000", 2024-09-14T22:37:47.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1099999999999992e-06 2024-09-14T22:37:47.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.280 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.280 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070634+0000", 2024-09-14T22:37:47.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:37:47.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.280 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.280 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195908+0000", 2024-09-14T22:37:47.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125273893 2024-09-14T22:37:47.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.280 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196097+0000", 2024-09-14T22:37:47.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018953899999999999 2024-09-14T22:37:47.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196308+0000", 2024-09-14T22:37:47.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021027799999999999 2024-09-14T22:37:47.281 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.281 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.281 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.281 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97201 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 833536~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:47.282 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:37:47.282 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.188599732999997, 2024-09-14T22:37:47.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126133826, 2024-09-14T22:37:47.282 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.282 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.282 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.282 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.282 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.282 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.282 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97201 2024-09-14T22:37:47.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.282 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:37:47.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:37:47.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074109+0000", 2024-09-14T22:37:47.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003464834 2024-09-14T22:37:47.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074134+0000", 2024-09-14T22:37:47.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5170000000000001e-05 2024-09-14T22:37:47.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074166+0000", 2024-09-14T22:37:47.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1347000000000003e-05 2024-09-14T22:37:47.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.285 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.285 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074179+0000", 2024-09-14T22:37:47.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.364e-05 2024-09-14T22:37:47.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.285 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.285 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196377+0000", 2024-09-14T22:37:47.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12219793900000001 2024-09-14T22:37:47.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.285 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.285 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196568+0000", 2024-09-14T22:37:47.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019097999999999999 2024-09-14T22:37:47.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196778+0000", 2024-09-14T22:37:47.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020991599999999999 2024-09-14T22:37:47.286 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.286 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.286 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.286 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97205 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 870400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:47.286 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:37:47.286 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.184930170000001, 2024-09-14T22:37:47.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12436148299999999, 2024-09-14T22:37:47.286 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.287 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.287 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.287 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.287 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.287 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.287 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97205 2024-09-14T22:37:47.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.287 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.287 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:37:47.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:37:47.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074315+0000", 2024-09-14T22:37:47.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9299999999999997e-07 2024-09-14T22:37:47.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074318+0000", 2024-09-14T22:37:47.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.19e-06 2024-09-14T22:37:47.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074327+0000", 2024-09-14T22:37:47.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8529999999999998e-06 2024-09-14T22:37:47.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074334+0000", 2024-09-14T22:37:47.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3869999999999997e-06 2024-09-14T22:37:47.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198365+0000", 2024-09-14T22:37:47.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124031222 2024-09-14T22:37:47.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198513+0000", 2024-09-14T22:37:47.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014772299999999999 2024-09-14T22:37:47.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198675+0000", 2024-09-14T22:37:47.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016241500000000001 2024-09-14T22:37:47.290 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.290 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.290 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.291 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97206 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 879616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:47.291 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:37:47.291 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.184898713999999, 2024-09-14T22:37:47.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124706284, 2024-09-14T22:37:47.291 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.291 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.291 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.291 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.291 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.291 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.291 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97206 2024-09-14T22:37:47.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.292 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:37:47.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:37:47.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074346+0000", 2024-09-14T22:37:47.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8999999999999996e-07 2024-09-14T22:37:47.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074349+0000", 2024-09-14T22:37:47.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.96e-06 2024-09-14T22:37:47.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074358+0000", 2024-09-14T22:37:47.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9169999999999995e-06 2024-09-14T22:37:47.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074365+0000", 2024-09-14T22:37:47.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3699999999999997e-06 2024-09-14T22:37:47.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198761+0000", 2024-09-14T22:37:47.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124395743 2024-09-14T22:37:47.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198899+0000", 2024-09-14T22:37:47.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013763499999999999 2024-09-14T22:37:47.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199052+0000", 2024-09-14T22:37:47.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015296900000000001 2024-09-14T22:37:47.295 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.295 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.295 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.295 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97207 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 888832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:47.295 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:37:47.295 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.184869501000001, 2024-09-14T22:37:47.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124988433, 2024-09-14T22:37:47.295 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.295 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.295 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.295 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.296 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.296 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.296 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97207 2024-09-14T22:37:47.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.296 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.296 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:37:47.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:37:47.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074375+0000", 2024-09-14T22:37:47.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:37:47.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074378+0000", 2024-09-14T22:37:47.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1829999999999998e-06 2024-09-14T22:37:47.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074389+0000", 2024-09-14T22:37:47.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0737000000000001e-05 2024-09-14T22:37:47.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074396+0000", 2024-09-14T22:37:47.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2559999999999996e-06 2024-09-14T22:37:47.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199139+0000", 2024-09-14T22:37:47.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12474253 2024-09-14T22:37:47.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199240+0000", 2024-09-14T22:37:47.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101221 2024-09-14T22:37:47.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.299 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199363+0000", 2024-09-14T22:37:47.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000122822 2024-09-14T22:37:47.299 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.299 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.299 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.299 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97208 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 898048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:47.299 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:37:47.300 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.184838196999998, 2024-09-14T22:37:47.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12524854899999999, 2024-09-14T22:37:47.300 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.300 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.300 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.300 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.300 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.300 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.300 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97208 2024-09-14T22:37:47.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.300 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.300 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:37:47.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:37:47.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:37:47.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:37:47.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074410+0000", 2024-09-14T22:37:47.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.18e-06 2024-09-14T22:37:47.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074435+0000", 2024-09-14T22:37:47.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5273e-05 2024-09-14T22:37:47.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074443+0000", 2024-09-14T22:37:47.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3000000000000002e-06 2024-09-14T22:37:47.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.303 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199433+0000", 2024-09-14T22:37:47.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124989502 2024-09-14T22:37:47.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.303 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.303 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199535+0000", 2024-09-14T22:37:47.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102557 2024-09-14T22:37:47.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.303 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.303 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199654+0000", 2024-09-14T22:37:47.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119061 2024-09-14T22:37:47.303 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.303 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.303 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.304 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97209 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 907264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:47.304 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:37:47.304 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.184790391999996, 2024-09-14T22:37:47.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125487354, 2024-09-14T22:37:47.304 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.304 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.304 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.304 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.304 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.304 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.304 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97209 2024-09-14T22:37:47.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.304 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:37:47.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:37:47.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:37:47.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9400000000000005e-07 2024-09-14T22:37:47.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074458+0000", 2024-09-14T22:37:47.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3299999999999999e-06 2024-09-14T22:37:47.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074467+0000", 2024-09-14T22:37:47.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.346e-06 2024-09-14T22:37:47.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074474+0000", 2024-09-14T22:37:47.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.374e-06 2024-09-14T22:37:47.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199720+0000", 2024-09-14T22:37:47.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12524561200000001 2024-09-14T22:37:47.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199820+0000", 2024-09-14T22:37:47.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010049599999999999 2024-09-14T22:37:47.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199941+0000", 2024-09-14T22:37:47.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012050200000000001 2024-09-14T22:37:47.307 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.307 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.307 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.308 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97210 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:47.308 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:37:47.308 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.184760398000002, 2024-09-14T22:37:47.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125744942, 2024-09-14T22:37:47.308 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.308 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.308 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.308 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.308 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.308 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.308 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97210 2024-09-14T22:37:47.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.309 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:37:47.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:37:47.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:37:47.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6599999999999996e-07 2024-09-14T22:37:47.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074487+0000", 2024-09-14T22:37:47.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8140000000000001e-06 2024-09-14T22:37:47.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074496+0000", 2024-09-14T22:37:47.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9330000000000007e-06 2024-09-14T22:37:47.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074503+0000", 2024-09-14T22:37:47.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4100000000000002e-06 2024-09-14T22:37:47.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200009+0000", 2024-09-14T22:37:47.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12550539899999999 2024-09-14T22:37:47.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200108+0000", 2024-09-14T22:37:47.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9182e-05 2024-09-14T22:37:47.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200229+0000", 2024-09-14T22:37:47.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120538 2024-09-14T22:37:47.312 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.312 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.312 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.312 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97211 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 925696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:47.312 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:37:47.312 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.184729437999998, 2024-09-14T22:37:47.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12599848299999999, 2024-09-14T22:37:47.312 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.312 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.312 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.313 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.313 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.313 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.313 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97211 2024-09-14T22:37:47.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.313 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.313 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:37:47.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.313 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:37:47.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.314 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074516+0000", 2024-09-14T22:37:47.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.42e-06 2024-09-14T22:37:47.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.314 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074521+0000", 2024-09-14T22:37:47.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7960000000000003e-06 2024-09-14T22:37:47.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.314 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074538+0000", 2024-09-14T22:37:47.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6900000000000001e-05 2024-09-14T22:37:47.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074555+0000", 2024-09-14T22:37:47.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.696e-05 2024-09-14T22:37:47.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200293+0000", 2024-09-14T22:37:47.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12573864100000001 2024-09-14T22:37:47.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200393+0000", 2024-09-14T22:37:47.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9264000000000004e-05 2024-09-14T22:37:47.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200513+0000", 2024-09-14T22:37:47.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012050200000000001 2024-09-14T22:37:47.316 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.316 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.316 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.316 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97212 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 934912~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:47.316 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:37:47.316 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.184654815999998, 2024-09-14T22:37:47.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12621127300000001, 2024-09-14T22:37:47.317 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.317 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.317 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.317 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.317 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.317 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.317 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97212 2024-09-14T22:37:47.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.317 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:37:47.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:37:47.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074590+0000", 2024-09-14T22:37:47.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1969999999999999e-06 2024-09-14T22:37:47.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074596+0000", 2024-09-14T22:37:47.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7629999999999997e-06 2024-09-14T22:37:47.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074614+0000", 2024-09-14T22:37:47.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7643e-05 2024-09-14T22:37:47.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074635+0000", 2024-09-14T22:37:47.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.075e-05 2024-09-14T22:37:47.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200580+0000", 2024-09-14T22:37:47.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125945482 2024-09-14T22:37:47.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200683+0000", 2024-09-14T22:37:47.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010267099999999999 2024-09-14T22:37:47.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200800+0000", 2024-09-14T22:37:47.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117767 2024-09-14T22:37:47.321 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.321 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.321 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.321 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97213 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 944128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:47.321 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:37:47.321 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.184589465999998, 2024-09-14T22:37:47.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126430709, 2024-09-14T22:37:47.321 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.321 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.321 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.321 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.322 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.322 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.322 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97213 2024-09-14T22:37:47.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.322 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:37:47.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:37:47.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078210+0000", 2024-09-14T22:37:47.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0035556839999999999 2024-09-14T22:37:47.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078231+0000", 2024-09-14T22:37:47.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0716e-05 2024-09-14T22:37:47.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078260+0000", 2024-09-14T22:37:47.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9473000000000001e-05 2024-09-14T22:37:47.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078279+0000", 2024-09-14T22:37:47.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8600000000000001e-05 2024-09-14T22:37:47.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200864+0000", 2024-09-14T22:37:47.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.122585295 2024-09-14T22:37:47.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200965+0000", 2024-09-14T22:37:47.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101004 2024-09-14T22:37:47.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.201085+0000", 2024-09-14T22:37:47.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119937 2024-09-14T22:37:47.325 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.325 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.325 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.325 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97219 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 999424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:47.326 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:37:47.326 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.180607189, 2024-09-14T22:37:47.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124138558, 2024-09-14T22:37:47.326 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.326 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.326 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.326 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.326 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.326 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.326 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97219 2024-09-14T22:37:47.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.326 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:37:47.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:37:47.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078638+0000", 2024-09-14T22:37:47.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.173e-06 2024-09-14T22:37:47.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078644+0000", 2024-09-14T22:37:47.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2570000000000004e-06 2024-09-14T22:37:47.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.328 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078672+0000", 2024-09-14T22:37:47.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7322999999999999e-05 2024-09-14T22:37:47.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.328 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078687+0000", 2024-09-14T22:37:47.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5639999999999999e-05 2024-09-14T22:37:47.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202567+0000", 2024-09-14T22:37:47.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123880134 2024-09-14T22:37:47.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202670+0000", 2024-09-14T22:37:47.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010273899999999999 2024-09-14T22:37:47.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202775+0000", 2024-09-14T22:37:47.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010529199999999999 2024-09-14T22:37:47.329 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.329 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.330 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.330 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97220 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1008640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:47.330 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:37:47.330 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.180537956000002, 2024-09-14T22:37:47.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12435937, 2024-09-14T22:37:47.330 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.330 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.330 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.330 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.330 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.330 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.330 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97220 2024-09-14T22:37:47.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.331 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:37:47.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:37:47.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078707+0000", 2024-09-14T22:37:47.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.266e-06 2024-09-14T22:37:47.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078713+0000", 2024-09-14T22:37:47.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7270000000000004e-06 2024-09-14T22:37:47.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078737+0000", 2024-09-14T22:37:47.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3640000000000001e-05 2024-09-14T22:37:47.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078751+0000", 2024-09-14T22:37:47.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.45e-05 2024-09-14T22:37:47.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202841+0000", 2024-09-14T22:37:47.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124089938 2024-09-14T22:37:47.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202944+0000", 2024-09-14T22:37:47.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010334 2024-09-14T22:37:47.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203065+0000", 2024-09-14T22:37:47.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012095900000000001 2024-09-14T22:37:47.334 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.334 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.334 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.334 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97221 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1017856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:47.334 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:37:47.334 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.180472193, 2024-09-14T22:37:47.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124574005, 2024-09-14T22:37:47.334 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.335 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.335 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.335 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.335 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.335 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.335 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97221 2024-09-14T22:37:47.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.335 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:37:47.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:37:47.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078773+0000", 2024-09-14T22:37:47.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.13e-06 2024-09-14T22:37:47.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078779+0000", 2024-09-14T22:37:47.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0260000000000004e-06 2024-09-14T22:37:47.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078797+0000", 2024-09-14T22:37:47.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.806e-05 2024-09-14T22:37:47.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078810+0000", 2024-09-14T22:37:47.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.308e-05 2024-09-14T22:37:47.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203131+0000", 2024-09-14T22:37:47.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12432077499999999 2024-09-14T22:37:47.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203227+0000", 2024-09-14T22:37:47.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6575000000000003e-05 2024-09-14T22:37:47.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203346+0000", 2024-09-14T22:37:47.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118359 2024-09-14T22:37:47.338 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.338 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.338 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.339 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97222 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:47.339 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:37:47.339 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.180415824000001, 2024-09-14T22:37:47.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12480110799999999, 2024-09-14T22:37:47.339 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.339 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.339 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.339 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.339 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.339 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.339 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97222 2024-09-14T22:37:47.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.339 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:37:47.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:37:47.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078829+0000", 2024-09-14T22:37:47.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.057e-06 2024-09-14T22:37:47.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078836+0000", 2024-09-14T22:37:47.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6159999999999998e-06 2024-09-14T22:37:47.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078856+0000", 2024-09-14T22:37:47.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0100000000000001e-05 2024-09-14T22:37:47.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078869+0000", 2024-09-14T22:37:47.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.272e-05 2024-09-14T22:37:47.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203412+0000", 2024-09-14T22:37:47.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12454283300000001 2024-09-14T22:37:47.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203511+0000", 2024-09-14T22:37:47.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9893000000000001e-05 2024-09-14T22:37:47.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203629+0000", 2024-09-14T22:37:47.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117889 2024-09-14T22:37:47.342 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.342 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.343 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.343 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97223 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1036288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:47.343 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:37:47.343 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.180043836999999, 2024-09-14T22:37:47.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124706279, 2024-09-14T22:37:47.343 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.343 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.343 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.343 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.343 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.343 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.343 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97223 2024-09-14T22:37:47.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.344 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:37:47.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:37:47.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079202+0000", 2024-09-14T22:37:47.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8330000000000001e-06 2024-09-14T22:37:47.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079212+0000", 2024-09-14T22:37:47.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0207000000000001e-05 2024-09-14T22:37:47.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079238+0000", 2024-09-14T22:37:47.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5626000000000002e-05 2024-09-14T22:37:47.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079256+0000", 2024-09-14T22:37:47.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8386999999999999e-05 2024-09-14T22:37:47.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203691+0000", 2024-09-14T22:37:47.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124434538 2024-09-14T22:37:47.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203786+0000", 2024-09-14T22:37:47.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5344000000000001e-05 2024-09-14T22:37:47.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203906+0000", 2024-09-14T22:37:47.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120344 2024-09-14T22:37:47.347 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.347 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.347 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.347 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97224 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1045504~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:47.347 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:37:47.347 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.179960016999999, 2024-09-14T22:37:47.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12513321199999999, 2024-09-14T22:37:47.347 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.347 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.347 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.347 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.347 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.348 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.348 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97224 2024-09-14T22:37:47.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.348 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:37:47.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:37:47.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082581+0000", 2024-09-14T22:37:47.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0032971530000000001 2024-09-14T22:37:47.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082592+0000", 2024-09-14T22:37:47.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1345999999999999e-05 2024-09-14T22:37:47.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082618+0000", 2024-09-14T22:37:47.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5873000000000001e-05 2024-09-14T22:37:47.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082638+0000", 2024-09-14T22:37:47.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9565999999999999e-05 2024-09-14T22:37:47.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204004+0000", 2024-09-14T22:37:47.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.121366038 2024-09-14T22:37:47.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204188+0000", 2024-09-14T22:37:47.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018427 2024-09-14T22:37:47.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204417+0000", 2024-09-14T22:37:47.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022896599999999999 2024-09-14T22:37:47.352 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.352 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.352 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.352 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.352 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.352 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:47.354 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:37:47.355 INFO:teuthology.orchestra.run.smithi001.stdout:5030 read 0x1ea11ea thru 0x1eafd93 (0xebaa bytes) 2024-09-14T22:37:47.355 INFO:teuthology.orchestra.run.smithi001.stdout:5031 read 0x2d21042 thru 0x2d289ca (0x7989 bytes) 2024-09-14T22:37:47.355 INFO:teuthology.orchestra.run.smithi001.stdout:5032 write 0x2f95f5a thru 0x2f9a009 (0x40b0 bytes) 2024-09-14T22:37:47.355 INFO:teuthology.orchestra.run.smithi001.stdout:5034 read 0x340ba98 thru 0x341b243 (0xf7ac bytes) 2024-09-14T22:37:47.355 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:47.355 INFO:teuthology.orchestra.run.smithi001.stdout:5036 punch from 0x384fa9 to 0x394452, (0xf4a9 bytes) 2024-09-14T22:37:47.355 INFO:teuthology.orchestra.run.smithi001.stdout:5037 read 0x4e6930 thru 0x4eb13e (0x480f bytes) 2024-09-14T22:37:47.355 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:47.355 INFO:teuthology.orchestra.run.smithi001.stdout:5042 write 0xa485d41 thru 0xa48d939 (0x7bf9 bytes) 2024-09-14T22:37:47.355 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:47.351+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:47.359 INFO:teuthology.orchestra.run.smithi001.stdout:5043 read 0x82494fe thru 0x824b598 (0x209b bytes) 2024-09-14T22:37:47.359 INFO:teuthology.orchestra.run.smithi001.stdout:5046 read 0x22e8d2a thru 0x22efd09 (0x6fe0 bytes) 2024-09-14T22:37:47.359 INFO:teuthology.orchestra.run.smithi001.stdout:5047 read 0x9b5bb24 thru 0x9b632a5 (0x7782 bytes) 2024-09-14T22:37:47.360 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:47.374 INFO:teuthology.orchestra.run.smithi001.stdout:5049 write 0xbdc5050 thru 0xbdd0f96 (0xbf47 bytes) 2024-09-14T22:37:47.382 INFO:teuthology.orchestra.run.smithi001.stdout:5050 write 0x234413 thru 0x23e666 (0xa254 bytes) 2024-09-14T22:37:47.427 INFO:teuthology.orchestra.run.smithi001.stdout:5051 trunc from 0xbdd0f97 to 0x2845c64 2024-09-14T22:37:47.455 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:47.451+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:47.518 INFO:teuthology.orchestra.run.smithi001.stdout:5052 read 0x60709a thru 0x6089f2 (0x1959 bytes) 2024-09-14T22:37:47.530 INFO:teuthology.orchestra.run.smithi001.stdout:5053 write 0x38777ee thru 0x38806dd (0x8ef0 bytes) 2024-09-14T22:37:47.532 INFO:teuthology.orchestra.run.smithi001.stdout:5054 read 0x173c118 thru 0x173ce66 (0xd4f bytes) 2024-09-14T22:37:47.533 INFO:teuthology.orchestra.run.smithi001.stdout:5055 flatten 2024-09-14T22:37:47.543 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:47.543 INFO:teuthology.orchestra.run.smithi001.stdout:5057 punch from 0x6783f to 0x74645, (0xce06 bytes) 2024-09-14T22:37:47.548 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:47.548 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:47.548 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:37:47.548 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:47.557 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:47.555+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:47.563 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:37:47.574 INFO:teuthology.orchestra.run.smithi001.stdout:5058 writesame 0xd3a69d9 thru 0xd3aed77 data_size 0x17(0x839f bytes) 2024-09-14T22:37:47.574 INFO:teuthology.orchestra.run.smithi001.stdout:5060 writesame 0x10e1fd3 thru 0x10edf93 data_size 0x1(0xbfc1 bytes) 2024-09-14T22:37:47.618 INFO:teuthology.orchestra.run.smithi001.stdout:5063 trunc from 0xd3aed78 to 0xe07925f 2024-09-14T22:37:47.630 INFO:teuthology.orchestra.run.smithi001.stdout:5064 write 0xd5a3f12 thru 0xd5a5220 (0x130f bytes) 2024-09-14T22:37:47.635 INFO:teuthology.orchestra.run.smithi001.stdout:5065 read 0xcf6ecd6 thru 0xcf7bc01 (0xcf2c bytes) 2024-09-14T22:37:47.635 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:47.635 INFO:teuthology.orchestra.run.smithi001.stdout:5068 read 0xa615fae thru 0xa622633 (0xc686 bytes) 2024-09-14T22:37:47.637 INFO:teuthology.orchestra.run.smithi001.stdout:5069 trunc from 0xe07925f to 0xd4f5900 2024-09-14T22:37:47.658 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:47.655+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:47.671 INFO:teuthology.orchestra.run.smithi001.stdout:5071 trunc from 0xd4f5900 to 0x66694a1 2024-09-14T22:37:47.689 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:47.689 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:47.689 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:37:47.689 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:37:47.689 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:47.695 INFO:teuthology.orchestra.run.smithi001.stdout:5073 read 0x5849abf thru 0x5853fa0 (0xa4e2 bytes) 2024-09-14T22:37:47.695 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:47.695 INFO:teuthology.orchestra.run.smithi001.stdout:5075 punch from 0x57a8952 to 0x57ad1c4, (0x4872 bytes) 2024-09-14T22:37:47.695 INFO:teuthology.orchestra.run.smithi001.stdout:5076 clone 33 order 24 su 2097152 sc 2 2024-09-14T22:37:47.703 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:37:47.759 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:47.754+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:47.827 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:47.828 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:47.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:47.828 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:47.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.828 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:72252 3.2 3:442c6d72:::rbd_data.113cb4c83968.0000000000000015:head [zero 4456448~393216] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-14T22:37:47.828 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:37:47.828 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 57.123082371999999, 2024-09-14T22:37:47.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0176591500000001, 2024-09-14T22:37:47.828 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.828 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.828 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T22:37:47.828 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.829 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.829 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.829 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 72252 2024-09-14T22:37:47.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.829 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:37:47.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:37:47.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700101+0000", 2024-09-14T22:37:47.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4570000000000001e-06 2024-09-14T22:37:47.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700114+0000", 2024-09-14T22:37:47.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2799999999999999e-05 2024-09-14T22:37:47.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700146+0000", 2024-09-14T22:37:47.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2116000000000002e-05 2024-09-14T22:37:47.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700170+0000", 2024-09-14T22:37:47.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3770000000000001e-05 2024-09-14T22:37:47.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700285+0000", 2024-09-14T22:37:47.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000115228 2024-09-14T22:37:47.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.832 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op must wait for map", 2024-09-14T22:37:47.832 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700297+0000", 2024-09-14T22:37:47.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1681999999999999e-05 2024-09-14T22:37:47.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.832 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.832 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.703312+0000", 2024-09-14T22:37:47.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003015046 2024-09-14T22:37:47.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.832 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:37:47.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.703327+0000", 2024-09-14T22:37:47.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5481999999999998e-05 2024-09-14T22:37:47.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:51.717566+0000", 2024-09-14T22:37:47.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0142383509999999 2024-09-14T22:37:47.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:51.717757+0000", 2024-09-14T22:37:47.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000191218 2024-09-14T22:37:47.833 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.834 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.834 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.834 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:74156 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T22:37:47.834 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:37:47.834 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 52.308613661999999, 2024-09-14T22:37:47.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062073521, 2024-09-14T22:37:47.834 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.834 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.834 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:47.834 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.834 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.835 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.835 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74156 2024-09-14T22:37:47.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.835 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.835 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.835 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:37:47.835 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.835 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.835 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:37:47.835 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514569+0000", 2024-09-14T22:37:47.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0229999999999999e-06 2024-09-14T22:37:47.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514596+0000", 2024-09-14T22:37:47.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6903e-05 2024-09-14T22:37:47.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.837 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514623+0000", 2024-09-14T22:37:47.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7206e-05 2024-09-14T22:37:47.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.837 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.837 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514637+0000", 2024-09-14T22:37:47.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:37:47.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.837 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.837 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514781+0000", 2024-09-14T22:37:47.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014394899999999999 2024-09-14T22:37:47.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.515005+0000", 2024-09-14T22:37:47.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022458499999999999 2024-09-14T22:37:47.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:37:47.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.527785+0000", 2024-09-14T22:37:47.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012780112 2024-09-14T22:37:47.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:47.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576209+0000", 2024-09-14T22:37:47.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.048423992999999999 2024-09-14T22:37:47.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:47.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576346+0000", 2024-09-14T22:37:47.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000136909 2024-09-14T22:37:47.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:47.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576517+0000", 2024-09-14T22:37:47.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000170848 2024-09-14T22:37:47.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576640+0000", 2024-09-14T22:37:47.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000122953 2024-09-14T22:37:47.840 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.840 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.840 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.840 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82513 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 557056~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:47.840 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:37:47.840 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.403981842999997, 2024-09-14T22:37:47.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059345486000000003, 2024-09-14T22:37:47.840 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.840 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.840 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.841 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.841 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.841 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.841 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82513 2024-09-14T22:37:47.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.841 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:37:47.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:37:47.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419200+0000", 2024-09-14T22:37:47.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.184e-06 2024-09-14T22:37:47.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419207+0000", 2024-09-14T22:37:47.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2690000000000002e-06 2024-09-14T22:37:47.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419224+0000", 2024-09-14T22:37:47.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6623e-05 2024-09-14T22:37:47.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419236+0000", 2024-09-14T22:37:47.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2367e-05 2024-09-14T22:37:47.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478294+0000", 2024-09-14T22:37:47.844 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059057927000000003 2024-09-14T22:37:47.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.844 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.844 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478402+0000", 2024-09-14T22:37:47.844 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107921 2024-09-14T22:37:47.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.844 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.844 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478544+0000", 2024-09-14T22:37:47.844 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014219499999999999 2024-09-14T22:37:47.844 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.845 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.845 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.845 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82514 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 704512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:47.845 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:37:47.845 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.403922739999999, 2024-09-14T22:37:47.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059580793, 2024-09-14T22:37:47.845 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.845 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.845 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.845 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.846 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.846 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.846 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82514 2024-09-14T22:37:47.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.846 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:37:47.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:37:47.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419259+0000", 2024-09-14T22:37:47.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.353e-06 2024-09-14T22:37:47.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419276+0000", 2024-09-14T22:37:47.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7567e-05 2024-09-14T22:37:47.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419294+0000", 2024-09-14T22:37:47.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.713e-05 2024-09-14T22:37:47.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419306+0000", 2024-09-14T22:37:47.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.279e-05 2024-09-14T22:37:47.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478589+0000", 2024-09-14T22:37:47.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059282978 2024-09-14T22:37:47.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478694+0000", 2024-09-14T22:37:47.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000104561 2024-09-14T22:37:47.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478838+0000", 2024-09-14T22:37:47.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014441399999999999 2024-09-14T22:37:47.849 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.849 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.849 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.850 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82515 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:47.850 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:37:47.850 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.403857590000001, 2024-09-14T22:37:47.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059815123999999997, 2024-09-14T22:37:47.850 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.850 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.850 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.850 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.850 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.850 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.850 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82515 2024-09-14T22:37:47.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.851 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:37:47.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:37:47.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419324+0000", 2024-09-14T22:37:47.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2130000000000001e-06 2024-09-14T22:37:47.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.852 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419331+0000", 2024-09-14T22:37:47.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4730000000000003e-06 2024-09-14T22:37:47.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.852 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419348+0000", 2024-09-14T22:37:47.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6269999999999998e-05 2024-09-14T22:37:47.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419360+0000", 2024-09-14T22:37:47.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2243e-05 2024-09-14T22:37:47.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478884+0000", 2024-09-14T22:37:47.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059524369000000001 2024-09-14T22:37:47.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478990+0000", 2024-09-14T22:37:47.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010569 2024-09-14T22:37:47.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479138+0000", 2024-09-14T22:37:47.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000147866 2024-09-14T22:37:47.854 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.854 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.854 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.854 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82516 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 999424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:47.854 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:37:47.854 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.403805085000002, 2024-09-14T22:37:47.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061979595999999998, 2024-09-14T22:37:47.855 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.855 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.855 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.855 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.855 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.855 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.855 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82516 2024-09-14T22:37:47.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.855 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:37:47.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.856 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.856 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:37:47.856 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.856 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.856 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419376+0000", 2024-09-14T22:37:47.856 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.237e-06 2024-09-14T22:37:47.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.856 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.856 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419383+0000", 2024-09-14T22:37:47.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8170000000000001e-06 2024-09-14T22:37:47.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.857 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419400+0000", 2024-09-14T22:37:47.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6636000000000001e-05 2024-09-14T22:37:47.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.857 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419412+0000", 2024-09-14T22:37:47.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2357e-05 2024-09-14T22:37:47.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481080+0000", 2024-09-14T22:37:47.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061668032999999997 2024-09-14T22:37:47.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481193+0000", 2024-09-14T22:37:47.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112335 2024-09-14T22:37:47.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481355+0000", 2024-09-14T22:37:47.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016218100000000001 2024-09-14T22:37:47.858 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.858 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.859 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.859 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82538 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1097728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:47.859 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:37:47.859 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.403753014999999, 2024-09-14T22:37:47.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059971705, 2024-09-14T22:37:47.859 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.859 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.859 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.859 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.859 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.859 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.859 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82538 2024-09-14T22:37:47.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.860 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:37:47.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:37:47.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419429+0000", 2024-09-14T22:37:47.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.497e-06 2024-09-14T22:37:47.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419436+0000", 2024-09-14T22:37:47.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2099999999999996e-06 2024-09-14T22:37:47.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419453+0000", 2024-09-14T22:37:47.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6840000000000001e-05 2024-09-14T22:37:47.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419465+0000", 2024-09-14T22:37:47.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2283000000000001e-05 2024-09-14T22:37:47.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479182+0000", 2024-09-14T22:37:47.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059717339000000001 2024-09-14T22:37:47.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479297+0000", 2024-09-14T22:37:47.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000114259 2024-09-14T22:37:47.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479399+0000", 2024-09-14T22:37:47.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102277 2024-09-14T22:37:47.863 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.863 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.863 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.863 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82539 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:47.863 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:37:47.863 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.403700164999997, 2024-09-14T22:37:47.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060166935999999997, 2024-09-14T22:37:47.864 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.864 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.864 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.864 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.864 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.864 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.864 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82539 2024-09-14T22:37:47.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.864 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.864 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:37:47.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.865 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.865 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:37:47.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.865 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.865 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.865 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419482+0000", 2024-09-14T22:37:47.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4130000000000001e-06 2024-09-14T22:37:47.865 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.865 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.865 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419488+0000", 2024-09-14T22:37:47.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8460000000000002e-06 2024-09-14T22:37:47.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419505+0000", 2024-09-14T22:37:47.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6477000000000002e-05 2024-09-14T22:37:47.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419518+0000", 2024-09-14T22:37:47.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2797e-05 2024-09-14T22:37:47.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479438+0000", 2024-09-14T22:37:47.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059919897 2024-09-14T22:37:47.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479547+0000", 2024-09-14T22:37:47.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109807 2024-09-14T22:37:47.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479647+0000", 2024-09-14T22:37:47.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9698999999999995e-05 2024-09-14T22:37:47.867 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.867 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.868 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.868 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82542 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:47.868 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:37:47.868 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.403647722000002, 2024-09-14T22:37:47.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060367430999999999, 2024-09-14T22:37:47.868 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.868 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.868 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.868 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.868 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.868 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.869 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82542 2024-09-14T22:37:47.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.869 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.869 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.869 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:37:47.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.869 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.869 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:37:47.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.870 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.870 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419534+0000", 2024-09-14T22:37:47.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T22:37:47.870 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.870 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.870 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419542+0000", 2024-09-14T22:37:47.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8199999999999997e-06 2024-09-14T22:37:47.870 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.870 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.870 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419559+0000", 2024-09-14T22:37:47.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6852999999999999e-05 2024-09-14T22:37:47.870 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419571+0000", 2024-09-14T22:37:47.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2364000000000001e-05 2024-09-14T22:37:47.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479686+0000", 2024-09-14T22:37:47.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060115522999999997 2024-09-14T22:37:47.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479795+0000", 2024-09-14T22:37:47.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010831399999999999 2024-09-14T22:37:47.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479900+0000", 2024-09-14T22:37:47.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105284 2024-09-14T22:37:47.872 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.872 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.872 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.872 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82544 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:47.872 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:37:47.872 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.403593985999997, 2024-09-14T22:37:47.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060590515999999997, 2024-09-14T22:37:47.872 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.873 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.873 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.873 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.873 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.873 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.873 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82544 2024-09-14T22:37:47.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.873 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:37:47.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.874 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:37:47.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.874 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.874 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419588+0000", 2024-09-14T22:37:47.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.35e-06 2024-09-14T22:37:47.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.874 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.874 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419595+0000", 2024-09-14T22:37:47.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4030000000000001e-06 2024-09-14T22:37:47.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.875 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.875 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419612+0000", 2024-09-14T22:37:47.875 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6670000000000001e-05 2024-09-14T22:37:47.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.875 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.875 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419624+0000", 2024-09-14T22:37:47.875 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-05 2024-09-14T22:37:47.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.875 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.875 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479957+0000", 2024-09-14T22:37:47.875 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060333287999999999 2024-09-14T22:37:47.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480074+0000", 2024-09-14T22:37:47.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116443 2024-09-14T22:37:47.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480177+0000", 2024-09-14T22:37:47.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000103102 2024-09-14T22:37:47.876 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.876 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.876 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.877 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82546 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:47.877 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:37:47.877 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.403541066999999, 2024-09-14T22:37:47.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060795428999999998, 2024-09-14T22:37:47.877 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.877 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.877 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.877 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.877 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.877 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.877 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82546 2024-09-14T22:37:47.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.877 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:37:47.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:37:47.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419640+0000", 2024-09-14T22:37:47.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.254e-06 2024-09-14T22:37:47.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.879 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.879 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419647+0000", 2024-09-14T22:37:47.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9e-06 2024-09-14T22:37:47.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.879 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.879 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419664+0000", 2024-09-14T22:37:47.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6345999999999999e-05 2024-09-14T22:37:47.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.879 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.879 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419676+0000", 2024-09-14T22:37:47.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.224e-05 2024-09-14T22:37:47.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480218+0000", 2024-09-14T22:37:47.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060542133999999997 2024-09-14T22:37:47.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480336+0000", 2024-09-14T22:37:47.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117472 2024-09-14T22:37:47.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480435+0000", 2024-09-14T22:37:47.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9083e-05 2024-09-14T22:37:47.881 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.881 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.881 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.881 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82548 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:47.881 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:37:47.881 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.403489057000002, 2024-09-14T22:37:47.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061011636000000001, 2024-09-14T22:37:47.881 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.881 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.881 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.881 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.881 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.882 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.882 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82548 2024-09-14T22:37:47.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.882 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:37:47.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:37:47.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.883 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.883 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419693+0000", 2024-09-14T22:37:47.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.387e-06 2024-09-14T22:37:47.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.883 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.883 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419699+0000", 2024-09-14T22:37:47.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8730000000000001e-06 2024-09-14T22:37:47.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.883 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.883 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419716+0000", 2024-09-14T22:37:47.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6416e-05 2024-09-14T22:37:47.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.884 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.884 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419728+0000", 2024-09-14T22:37:47.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2597e-05 2024-09-14T22:37:47.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.884 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.884 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480476+0000", 2024-09-14T22:37:47.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060747989000000002 2024-09-14T22:37:47.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.884 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.884 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480583+0000", 2024-09-14T22:37:47.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010638500000000001 2024-09-14T22:37:47.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480703+0000", 2024-09-14T22:37:47.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011998900000000001 2024-09-14T22:37:47.885 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.885 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.885 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.885 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82550 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1982464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:47.886 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:37:47.886 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.403436304000003, 2024-09-14T22:37:47.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061295483999999997, 2024-09-14T22:37:47.886 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.886 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.886 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.886 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.886 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.886 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.886 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82550 2024-09-14T22:37:47.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.886 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:37:47.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.887 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.887 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:37:47.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.887 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.887 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419745+0000", 2024-09-14T22:37:47.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.406e-06 2024-09-14T22:37:47.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.888 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.888 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419753+0000", 2024-09-14T22:37:47.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8140000000000001e-06 2024-09-14T22:37:47.888 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.888 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.888 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.888 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419770+0000", 2024-09-14T22:37:47.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6719999999999999e-05 2024-09-14T22:37:47.888 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.888 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.888 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.888 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419782+0000", 2024-09-14T22:37:47.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2252999999999999e-05 2024-09-14T22:37:47.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.889 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480745+0000", 2024-09-14T22:37:47.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060962385000000001 2024-09-14T22:37:47.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.889 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480900+0000", 2024-09-14T22:37:47.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015513899999999999 2024-09-14T22:37:47.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.889 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481039+0000", 2024-09-14T22:37:47.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013976700000000001 2024-09-14T22:37:47.890 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.890 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.890 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.890 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82588 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1064960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:47.890 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:37:47.890 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.403382413999999, 2024-09-14T22:37:47.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061863732999999997, 2024-09-14T22:37:47.890 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.890 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.890 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.891 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.891 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.891 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.891 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82588 2024-09-14T22:37:47.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.891 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:37:47.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:37:47.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.892 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.892 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419799+0000", 2024-09-14T22:37:47.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4160000000000001e-06 2024-09-14T22:37:47.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.892 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.892 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419807+0000", 2024-09-14T22:37:47.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2169999999999994e-06 2024-09-14T22:37:47.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.892 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419824+0000", 2024-09-14T22:37:47.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6776e-05 2024-09-14T22:37:47.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419837+0000", 2024-09-14T22:37:47.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2680000000000001e-05 2024-09-14T22:37:47.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481395+0000", 2024-09-14T22:37:47.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061558416999999997 2024-09-14T22:37:47.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.894 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.894 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481532+0000", 2024-09-14T22:37:47.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000136861 2024-09-14T22:37:47.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.894 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.894 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481662+0000", 2024-09-14T22:37:47.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012936600000000001 2024-09-14T22:37:47.894 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.894 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.894 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.895 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82589 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:47.895 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:37:47.895 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.403327818999998, 2024-09-14T22:37:47.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062064592000000002, 2024-09-14T22:37:47.895 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.895 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.895 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.895 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.895 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.895 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.895 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82589 2024-09-14T22:37:47.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.896 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.896 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.896 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:37:47.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.896 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.896 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:37:47.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.896 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.896 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419854+0000", 2024-09-14T22:37:47.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.367e-06 2024-09-14T22:37:47.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419862+0000", 2024-09-14T22:37:47.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8930000000000005e-06 2024-09-14T22:37:47.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419896+0000", 2024-09-14T22:37:47.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4134000000000002e-05 2024-09-14T22:37:47.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419909+0000", 2024-09-14T22:37:47.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3332999999999999e-05 2024-09-14T22:37:47.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481702+0000", 2024-09-14T22:37:47.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061792947000000001 2024-09-14T22:37:47.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481808+0000", 2024-09-14T22:37:47.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105655 2024-09-14T22:37:47.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.899 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481917+0000", 2024-09-14T22:37:47.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109263 2024-09-14T22:37:47.899 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.899 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.899 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.899 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82590 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1359872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:47.899 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:37:47.899 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.403254212, 2024-09-14T22:37:47.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062362895000000002, 2024-09-14T22:37:47.899 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.899 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.899 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.900 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.900 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.900 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.900 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82590 2024-09-14T22:37:47.900 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.900 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.900 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.900 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:37:47.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.900 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.901 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.901 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:37:47.901 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.901 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.901 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.901 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.901 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419928+0000", 2024-09-14T22:37:47.901 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4470000000000001e-06 2024-09-14T22:37:47.901 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.901 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.901 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.901 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419942+0000", 2024-09-14T22:37:47.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4229e-05 2024-09-14T22:37:47.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419972+0000", 2024-09-14T22:37:47.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.065e-05 2024-09-14T22:37:47.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419985+0000", 2024-09-14T22:37:47.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2903e-05 2024-09-14T22:37:47.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481960+0000", 2024-09-14T22:37:47.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061974435000000001 2024-09-14T22:37:47.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482106+0000", 2024-09-14T22:37:47.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014643800000000001 2024-09-14T22:37:47.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482289+0000", 2024-09-14T22:37:47.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000182793 2024-09-14T22:37:47.903 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.904 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.904 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.904 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82592 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:47.904 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:37:47.904 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.403177990000003, 2024-09-14T22:37:47.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062631507000000003, 2024-09-14T22:37:47.904 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.904 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.904 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.904 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.905 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.905 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.905 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82592 2024-09-14T22:37:47.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.905 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.905 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.905 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:37:47.905 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.905 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.906 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:37:47.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.906 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.906 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420017+0000", 2024-09-14T22:37:47.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4744e-05 2024-09-14T22:37:47.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.906 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.906 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420036+0000", 2024-09-14T22:37:47.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8680000000000001e-05 2024-09-14T22:37:47.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.906 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420054+0000", 2024-09-14T22:37:47.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7799999999999999e-05 2024-09-14T22:37:47.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420067+0000", 2024-09-14T22:37:47.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3013000000000001e-05 2024-09-14T22:37:47.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482358+0000", 2024-09-14T22:37:47.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062291445000000001 2024-09-14T22:37:47.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482469+0000", 2024-09-14T22:37:47.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110608 2024-09-14T22:37:47.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482634+0000", 2024-09-14T22:37:47.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000165217 2024-09-14T22:37:47.908 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.908 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.908 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.908 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82594 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1654784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:47.908 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:37:47.909 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.403080013999997, 2024-09-14T22:37:47.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062893849000000002, 2024-09-14T22:37:47.909 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.909 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.909 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.909 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.909 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.909 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.909 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82594 2024-09-14T22:37:47.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.909 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.909 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.909 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:37:47.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.910 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:37:47.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.910 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420102+0000", 2024-09-14T22:37:47.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5e-06 2024-09-14T22:37:47.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420115+0000", 2024-09-14T22:37:47.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.324e-05 2024-09-14T22:37:47.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420133+0000", 2024-09-14T22:37:47.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8213e-05 2024-09-14T22:37:47.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420147+0000", 2024-09-14T22:37:47.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3284e-05 2024-09-14T22:37:47.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482688+0000", 2024-09-14T22:37:47.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062541425999999997 2024-09-14T22:37:47.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482794+0000", 2024-09-14T22:37:47.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010575500000000001 2024-09-14T22:37:47.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482994+0000", 2024-09-14T22:37:47.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020043099999999999 2024-09-14T22:37:47.913 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.913 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.913 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.913 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82596 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:47.913 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:37:47.913 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.403016714000003, 2024-09-14T22:37:47.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063187238000000007, 2024-09-14T22:37:47.913 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.913 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.913 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.913 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.913 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.914 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.914 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82596 2024-09-14T22:37:47.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.914 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.914 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:37:47.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.914 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:37:47.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420165+0000", 2024-09-14T22:37:47.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4100000000000001e-06 2024-09-14T22:37:47.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420173+0000", 2024-09-14T22:37:47.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8129999999999996e-06 2024-09-14T22:37:47.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420190+0000", 2024-09-14T22:37:47.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7076999999999999e-05 2024-09-14T22:37:47.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420203+0000", 2024-09-14T22:37:47.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2683e-05 2024-09-14T22:37:47.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483060+0000", 2024-09-14T22:37:47.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062857114000000006 2024-09-14T22:37:47.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483167+0000", 2024-09-14T22:37:47.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010713999999999999 2024-09-14T22:37:47.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483351+0000", 2024-09-14T22:37:47.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018400099999999999 2024-09-14T22:37:47.917 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.917 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.917 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.918 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82597 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1949696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:47.918 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:37:47.918 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.402959887999998, 2024-09-14T22:37:47.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063466513000000002, 2024-09-14T22:37:47.918 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:47.918 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:47.918 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:47.918 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:47.918 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:47.918 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:47.918 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82597 2024-09-14T22:37:47.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.919 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:47.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:47.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:37:47.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:47.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:37:47.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:47.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:47.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420221+0000", 2024-09-14T22:37:47.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.043e-06 2024-09-14T22:37:47.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:47.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420229+0000", 2024-09-14T22:37:47.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0469999999999994e-06 2024-09-14T22:37:47.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:47.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420246+0000", 2024-09-14T22:37:47.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6339999999999999e-05 2024-09-14T22:37:47.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:47.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420258+0000", 2024-09-14T22:37:47.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2296000000000001e-05 2024-09-14T22:37:47.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:47.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483402+0000", 2024-09-14T22:37:47.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063144318000000005 2024-09-14T22:37:47.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:47.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483530+0000", 2024-09-14T22:37:47.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012713199999999999 2024-09-14T22:37:47.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:47.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:47.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:47.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483687+0000", 2024-09-14T22:37:47.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015733699999999999 2024-09-14T22:37:47.922 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.922 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.922 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.922 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:47.922 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:47.922 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:47.923 INFO:teuthology.orchestra.run.smithi001.stderr:set 3.1d pg_upmap mapping to [1,2] 2024-09-14T22:37:47.924 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2024-09-14T22:37:47.925 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:37:47.925 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:47.918+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:48.024 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:48.018+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:48.099 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:48.099 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:48.100 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:37:48.100 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:48.113 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:37:48.126 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:48.123+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:48.156 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-14T22:37:48.226 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:48.218+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:48.278 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:48.278 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:48.278 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:37:48.279 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:37:48.279 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:48.292 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:37:48.322 INFO:teuthology.orchestra.run.smithi001.stdout:5077 read 0x38ce265 thru 0x38d6168 (0x7f04 bytes) 2024-09-14T22:37:48.325 INFO:teuthology.orchestra.run.smithi001.stdout:5079 read 0x11dd763 thru 0x11e6c9c (0x953a bytes) 2024-09-14T22:37:48.326 INFO:teuthology.orchestra.run.smithi001.stdout:5080 write 0x2525d49 thru 0x2535537 (0xf7ef bytes) 2024-09-14T22:37:48.327 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:48.323+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:48.342 INFO:teuthology.orchestra.run.smithi001.stdout:5082 read 0x537e89 thru 0x53924c (0x13c4 bytes) 2024-09-14T22:37:48.342 INFO:teuthology.orchestra.run.smithi001.stdout:5083 write 0x427eaa1 thru 0x4280c1a (0x217a bytes) 2024-09-14T22:37:48.349 INFO:teuthology.orchestra.run.smithi001.stdout:5084 read 0x3888974 thru 0x388b929 (0x2fb6 bytes) 2024-09-14T22:37:48.377 INFO:teuthology.orchestra.run.smithi001.stdout:5085 writesame 0xe3b16d7 thru 0xe3b9c41 data_size 0xf(0x856b bytes) 2024-09-14T22:37:48.377 INFO:teuthology.orchestra.run.smithi001.stdout:5086 writesame 0x8d4ea3f thru 0x8d54ac7 data_size 0x1(0x6089 bytes) 2024-09-14T22:37:48.399 INFO:teuthology.orchestra.run.smithi001.stdout:5087 read 0x1e841a3 thru 0x1e88abd (0x491b bytes) 2024-09-14T22:37:48.400 INFO:teuthology.orchestra.run.smithi001.stdout:5090 write 0xe37360f thru 0xe37ea28 (0xb41a bytes) 2024-09-14T22:37:48.406 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:48.406 INFO:teuthology.orchestra.run.smithi001.stdout:5092 writesame 0x4d7ff07 thru 0x4d81041 data_size 0x1(0x113b bytes) 2024-09-14T22:37:48.417 INFO:teuthology.orchestra.run.smithi001.stdout:5093 clone 34 order 18 su 16384 sc 3 2024-09-14T22:37:48.428 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:48.423+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:48.451 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:37:48.462 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:48.462 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:48.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:48.462 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:48.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.462 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:37:48.462 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:37:48.462 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 108.45084549800001, 2024-09-14T22:37:48.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.398029152, 2024-09-14T22:37:48.462 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:48.462 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:48.463 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:48.463 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:48.463 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:37:48.463 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:37:48.463 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 1 2024-09-14T22:37:48.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.463 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:48.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:48.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:37:48.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:48.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:37:48.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:48.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006183+0000", 2024-09-14T22:37:48.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9059999999999998e-06 2024-09-14T22:37:48.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:48.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006198+0000", 2024-09-14T22:37:48.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.523e-05 2024-09-14T22:37:48.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.465 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:48.465 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006219+0000", 2024-09-14T22:37:48.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1342999999999999e-05 2024-09-14T22:37:48.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.465 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:48.465 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006240+0000", 2024-09-14T22:37:48.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1186999999999999e-05 2024-09-14T22:37:48.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.465 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:48.465 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006363+0000", 2024-09-14T22:37:48.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012258200000000001 2024-09-14T22:37:48.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:37:48.466 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006392+0000", 2024-09-14T22:37:48.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9607e-05 2024-09-14T22:37:48.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:48.466 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401084+0000", 2024-09-14T22:37:48.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.39469142699999998 2024-09-14T22:37:48.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:48.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401394+0000", 2024-09-14T22:37:48.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030987799999999998 2024-09-14T22:37:48.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:37:48.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401860+0000", 2024-09-14T22:37:48.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00046654299999999998 2024-09-14T22:37:48.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:48.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.403386+0000", 2024-09-14T22:37:48.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001525322 2024-09-14T22:37:48.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:48.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404042+0000", 2024-09-14T22:37:48.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065676099999999996 2024-09-14T22:37:48.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:48.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404136+0000", 2024-09-14T22:37:48.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4032999999999999e-05 2024-09-14T22:37:48.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:48.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404209+0000", 2024-09-14T22:37:48.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2333e-05 2024-09-14T22:37:48.469 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.469 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:48.469 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.469 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104182 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14352384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:48.469 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:37:48.469 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.967991512000001, 2024-09-14T22:37:48.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071642496999999999, 2024-09-14T22:37:48.469 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:48.470 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:48.470 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:48.470 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:48.470 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:48.470 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:48.470 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104182 2024-09-14T22:37:48.470 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.470 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:48.470 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.470 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:48.470 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:37:48.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.470 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:48.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:37:48.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:48.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489035+0000", 2024-09-14T22:37:48.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3430000000000001e-06 2024-09-14T22:37:48.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:48.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489041+0000", 2024-09-14T22:37:48.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8139999999999997e-06 2024-09-14T22:37:48.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:48.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489059+0000", 2024-09-14T22:37:48.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8193e-05 2024-09-14T22:37:48.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:48.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489073+0000", 2024-09-14T22:37:48.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:37:48.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:48.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560434+0000", 2024-09-14T22:37:48.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071361463999999999 2024-09-14T22:37:48.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:48.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560545+0000", 2024-09-14T22:37:48.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110263 2024-09-14T22:37:48.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:48.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560676+0000", 2024-09-14T22:37:48.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013138000000000001 2024-09-14T22:37:48.474 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.474 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:48.474 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.474 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104183 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14483456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:48.474 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:37:48.474 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.967868193000001, 2024-09-14T22:37:48.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071781943000000001, 2024-09-14T22:37:48.474 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:48.474 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:48.474 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:48.475 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:48.475 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:48.475 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:48.475 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104183 2024-09-14T22:37:48.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.475 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:48.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.475 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:48.475 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:37:48.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.475 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:48.475 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:37:48.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:48.476 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489159+0000", 2024-09-14T22:37:48.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5629999999999999e-06 2024-09-14T22:37:48.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:48.476 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489165+0000", 2024-09-14T22:37:48.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7070000000000003e-06 2024-09-14T22:37:48.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:48.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489186+0000", 2024-09-14T22:37:48.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.088e-05 2024-09-14T22:37:48.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:48.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489201+0000", 2024-09-14T22:37:48.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5203e-05 2024-09-14T22:37:48.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:48.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560713+0000", 2024-09-14T22:37:48.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071511447000000006 2024-09-14T22:37:48.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:48.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560819+0000", 2024-09-14T22:37:48.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106579 2024-09-14T22:37:48.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:48.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560939+0000", 2024-09-14T22:37:48.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119564 2024-09-14T22:37:48.478 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.478 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:48.478 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.479 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104184 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:48.479 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:37:48.479 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.967803750000002, 2024-09-14T22:37:48.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071970107000000005, 2024-09-14T22:37:48.479 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:48.479 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:48.479 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:48.479 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:48.479 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:48.479 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:48.480 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104184 2024-09-14T22:37:48.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.480 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:48.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.480 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:48.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:37:48.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.480 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:48.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:37:48.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:48.481 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489223+0000", 2024-09-14T22:37:48.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4470000000000001e-06 2024-09-14T22:37:48.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:48.481 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489229+0000", 2024-09-14T22:37:48.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8200000000000002e-06 2024-09-14T22:37:48.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:48.481 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489247+0000", 2024-09-14T22:37:48.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8150000000000001e-05 2024-09-14T22:37:48.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:48.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489260+0000", 2024-09-14T22:37:48.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3570000000000001e-05 2024-09-14T22:37:48.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:48.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560979+0000", 2024-09-14T22:37:48.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071718831999999996 2024-09-14T22:37:48.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:48.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561075+0000", 2024-09-14T22:37:48.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5848999999999997e-05 2024-09-14T22:37:48.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:48.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561191+0000", 2024-09-14T22:37:48.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116439 2024-09-14T22:37:48.483 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.483 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:48.483 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.483 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104185 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14745600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:48.484 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:37:48.484 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.967744166999999, 2024-09-14T22:37:48.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072173750999999994, 2024-09-14T22:37:48.484 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:48.484 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:48.484 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:48.484 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:48.484 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:48.484 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:48.484 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104185 2024-09-14T22:37:48.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.484 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:48.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.484 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:48.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:37:48.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.485 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:48.485 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:37:48.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.485 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:48.485 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489282+0000", 2024-09-14T22:37:48.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2500000000000001e-06 2024-09-14T22:37:48.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.485 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:48.486 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489287+0000", 2024-09-14T22:37:48.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1359999999999996e-06 2024-09-14T22:37:48.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.486 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:48.486 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489304+0000", 2024-09-14T22:37:48.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7033000000000001e-05 2024-09-14T22:37:48.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.486 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:48.486 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489318+0000", 2024-09-14T22:37:48.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3934000000000001e-05 2024-09-14T22:37:48.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:48.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561232+0000", 2024-09-14T22:37:48.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071913759999999993 2024-09-14T22:37:48.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:48.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561340+0000", 2024-09-14T22:37:48.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107921 2024-09-14T22:37:48.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:48.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561455+0000", 2024-09-14T22:37:48.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011471700000000001 2024-09-14T22:37:48.487 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.488 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:48.488 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.488 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104186 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14876672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:48.488 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:37:48.488 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.967688257999999, 2024-09-14T22:37:48.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072393374999999996, 2024-09-14T22:37:48.488 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:48.488 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:48.488 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:48.488 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:48.488 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:48.489 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:48.489 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104186 2024-09-14T22:37:48.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.489 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:48.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:48.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:37:48.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:48.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:37:48.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:48.490 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489338+0000", 2024-09-14T22:37:48.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.48e-06 2024-09-14T22:37:48.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:48.490 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489366+0000", 2024-09-14T22:37:48.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7307000000000001e-05 2024-09-14T22:37:48.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:48.490 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489385+0000", 2024-09-14T22:37:48.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9283000000000002e-05 2024-09-14T22:37:48.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:48.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489399+0000", 2024-09-14T22:37:48.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.452e-05 2024-09-14T22:37:48.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:48.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561494+0000", 2024-09-14T22:37:48.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072094885999999997 2024-09-14T22:37:48.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:48.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561604+0000", 2024-09-14T22:37:48.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010925900000000001 2024-09-14T22:37:48.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:48.492 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561730+0000", 2024-09-14T22:37:48.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012663999999999999 2024-09-14T22:37:48.492 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.492 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:48.492 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.492 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104187 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15007744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:48.492 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:37:48.492 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.967604889, 2024-09-14T22:37:48.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072599143000000005, 2024-09-14T22:37:48.492 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:48.492 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:48.493 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:48.493 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:48.493 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:48.493 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:48.493 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104187 2024-09-14T22:37:48.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.493 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:48.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.493 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:48.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:37:48.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.493 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:48.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:37:48.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.494 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:48.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489422+0000", 2024-09-14T22:37:48.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5340000000000001e-06 2024-09-14T22:37:48.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.494 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:48.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489428+0000", 2024-09-14T22:37:48.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1430000000000003e-06 2024-09-14T22:37:48.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.495 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:48.495 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489445+0000", 2024-09-14T22:37:48.495 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7042999999999999e-05 2024-09-14T22:37:48.495 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.495 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:48.495 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489458+0000", 2024-09-14T22:37:48.495 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3474e-05 2024-09-14T22:37:48.495 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.495 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:48.495 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561771+0000", 2024-09-14T22:37:48.495 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072312600000000005 2024-09-14T22:37:48.495 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.496 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:48.496 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561885+0000", 2024-09-14T22:37:48.496 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011411799999999999 2024-09-14T22:37:48.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.496 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:48.496 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562019+0000", 2024-09-14T22:37:48.496 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013423100000000001 2024-09-14T22:37:48.496 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.496 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:48.496 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.497 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104188 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15138816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:48.497 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:37:48.497 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.967544964999998, 2024-09-14T22:37:48.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072824624000000004, 2024-09-14T22:37:48.497 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:48.497 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:48.497 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:48.497 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:48.497 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:48.497 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:48.497 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104188 2024-09-14T22:37:48.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.497 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:48.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.497 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:48.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:37:48.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:48.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:37:48.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:48.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489481+0000", 2024-09-14T22:37:48.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.22e-06 2024-09-14T22:37:48.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:48.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489487+0000", 2024-09-14T22:37:48.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6930000000000004e-06 2024-09-14T22:37:48.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:48.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489508+0000", 2024-09-14T22:37:48.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1135999999999999e-05 2024-09-14T22:37:48.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:48.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489522+0000", 2024-09-14T22:37:48.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3817e-05 2024-09-14T22:37:48.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:48.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562066+0000", 2024-09-14T22:37:48.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072543813999999998 2024-09-14T22:37:48.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.500 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:48.500 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562177+0000", 2024-09-14T22:37:48.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000111264 2024-09-14T22:37:48.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.500 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:48.500 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562305+0000", 2024-09-14T22:37:48.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012768 2024-09-14T22:37:48.500 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.500 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:48.500 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.501 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104189 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15269888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:48.501 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:37:48.501 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.967484515999999, 2024-09-14T22:37:48.501 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073061300999999995, 2024-09-14T22:37:48.501 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:48.501 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:48.501 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:48.501 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:48.501 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:48.501 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:48.501 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104189 2024-09-14T22:37:48.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.502 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:48.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.502 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:48.502 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:37:48.502 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.502 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:48.502 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:37:48.502 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.503 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:48.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489542+0000", 2024-09-14T22:37:48.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1659999999999999e-06 2024-09-14T22:37:48.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.503 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:48.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489561+0000", 2024-09-14T22:37:48.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.908e-05 2024-09-14T22:37:48.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.503 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:48.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489579+0000", 2024-09-14T22:37:48.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8536999999999999e-05 2024-09-14T22:37:48.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:48.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489593+0000", 2024-09-14T22:37:48.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3977e-05 2024-09-14T22:37:48.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:48.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562347+0000", 2024-09-14T22:37:48.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072754092000000006 2024-09-14T22:37:48.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:48.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562458+0000", 2024-09-14T22:37:48.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110722 2024-09-14T22:37:48.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:48.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562602+0000", 2024-09-14T22:37:48.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014372699999999999 2024-09-14T22:37:48.505 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.505 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:48.505 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.505 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104190 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:48.505 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:37:48.506 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.96741222, 2024-09-14T22:37:48.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.07326133, 2024-09-14T22:37:48.506 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:48.506 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:48.506 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:48.506 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:48.506 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:48.506 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:48.506 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104190 2024-09-14T22:37:48.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.506 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:48.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.506 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:48.507 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:37:48.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.507 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:48.507 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:37:48.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.507 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:48.507 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489614+0000", 2024-09-14T22:37:48.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1999999999999999e-06 2024-09-14T22:37:48.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:48.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489620+0000", 2024-09-14T22:37:48.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.524e-06 2024-09-14T22:37:48.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:48.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489637+0000", 2024-09-14T22:37:48.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7825999999999998e-05 2024-09-14T22:37:48.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:48.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489671+0000", 2024-09-14T22:37:48.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3726000000000002e-05 2024-09-14T22:37:48.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:48.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562644+0000", 2024-09-14T22:37:48.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072973089000000005 2024-09-14T22:37:48.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:48.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562762+0000", 2024-09-14T22:37:48.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011743600000000001 2024-09-14T22:37:48.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:48.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562874+0000", 2024-09-14T22:37:48.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011252900000000001 2024-09-14T22:37:48.510 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.510 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:48.510 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.510 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104191 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15532032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:48.510 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:37:48.510 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.967303797, 2024-09-14T22:37:48.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073311166999999997, 2024-09-14T22:37:48.510 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:48.511 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:48.511 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:48.511 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:48.511 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:48.511 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:48.511 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104191 2024-09-14T22:37:48.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.511 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:48.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.511 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:48.511 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:37:48.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:48.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:37:48.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:48.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489723+0000", 2024-09-14T22:37:48.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6330000000000001e-06 2024-09-14T22:37:48.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:48.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489730+0000", 2024-09-14T22:37:48.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2300000000000002e-06 2024-09-14T22:37:48.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:48.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489751+0000", 2024-09-14T22:37:48.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0849999999999999e-05 2024-09-14T22:37:48.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:48.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489766+0000", 2024-09-14T22:37:48.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.524e-05 2024-09-14T22:37:48.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:48.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562896+0000", 2024-09-14T22:37:48.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073129662999999998 2024-09-14T22:37:48.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:48.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562960+0000", 2024-09-14T22:37:48.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3689999999999995e-05 2024-09-14T22:37:48.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:48.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563033+0000", 2024-09-14T22:37:48.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2861000000000006e-05 2024-09-14T22:37:48.514 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.514 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:48.514 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.515 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104192 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:48.515 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:37:48.515 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.967207991999999, 2024-09-14T22:37:48.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073463632000000001, 2024-09-14T22:37:48.515 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:48.515 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:48.515 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:48.515 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:48.515 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:48.515 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:48.515 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104192 2024-09-14T22:37:48.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.515 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:48.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.516 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:48.516 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:37:48.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.516 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:48.516 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:37:48.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.516 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:48.516 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489819+0000", 2024-09-14T22:37:48.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.424e-06 2024-09-14T22:37:48.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.517 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:48.517 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489825+0000", 2024-09-14T22:37:48.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.02e-06 2024-09-14T22:37:48.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.517 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:48.517 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489844+0000", 2024-09-14T22:37:48.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9760000000000001e-05 2024-09-14T22:37:48.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.517 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:48.517 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489860+0000", 2024-09-14T22:37:48.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5563e-05 2024-09-14T22:37:48.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:48.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563070+0000", 2024-09-14T22:37:48.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073209724000000004 2024-09-14T22:37:48.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:48.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563175+0000", 2024-09-14T22:37:48.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105183 2024-09-14T22:37:48.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:48.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563281+0000", 2024-09-14T22:37:48.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105958 2024-09-14T22:37:48.519 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.519 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:48.519 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.519 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104193 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15794176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:48.519 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:37:48.519 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.967145824999999, 2024-09-14T22:37:48.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073600040000000005, 2024-09-14T22:37:48.519 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:48.519 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:48.520 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:48.520 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:48.520 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:48.520 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:48.520 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104193 2024-09-14T22:37:48.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.520 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:48.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.521 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:48.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:37:48.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.521 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:48.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:37:48.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.521 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:48.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489881+0000", 2024-09-14T22:37:48.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5460000000000001e-06 2024-09-14T22:37:48.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:48.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489886+0000", 2024-09-14T22:37:48.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1739999999999999e-06 2024-09-14T22:37:48.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:48.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489912+0000", 2024-09-14T22:37:48.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6461999999999998e-05 2024-09-14T22:37:48.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:48.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489927+0000", 2024-09-14T22:37:48.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:37:48.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:48.523 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563310+0000", 2024-09-14T22:37:48.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073382977000000002 2024-09-14T22:37:48.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:48.523 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563385+0000", 2024-09-14T22:37:48.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5218999999999999e-05 2024-09-14T22:37:48.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:48.523 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563479+0000", 2024-09-14T22:37:48.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4622000000000003e-05 2024-09-14T22:37:48.524 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.524 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:48.524 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.524 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104194 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15925248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:48.524 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:37:48.524 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.967079233, 2024-09-14T22:37:48.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073745434999999998, 2024-09-14T22:37:48.524 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:48.524 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:48.524 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:48.525 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:48.525 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:48.525 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:48.525 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104194 2024-09-14T22:37:48.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.525 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:48.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:48.525 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:37:48.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:48.525 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:37:48.526 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.526 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:48.526 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489947+0000", 2024-09-14T22:37:48.526 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.327e-06 2024-09-14T22:37:48.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.526 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:48.526 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489953+0000", 2024-09-14T22:37:48.526 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9329999999999998e-06 2024-09-14T22:37:48.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.526 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:48.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489971+0000", 2024-09-14T22:37:48.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7994000000000001e-05 2024-09-14T22:37:48.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:48.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489985+0000", 2024-09-14T22:37:48.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4226000000000001e-05 2024-09-14T22:37:48.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:48.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563510+0000", 2024-09-14T22:37:48.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073524667000000002 2024-09-14T22:37:48.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.528 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:48.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563597+0000", 2024-09-14T22:37:48.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6550000000000003e-05 2024-09-14T22:37:48.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.528 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:48.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563691+0000", 2024-09-14T22:37:48.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4738e-05 2024-09-14T22:37:48.528 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.528 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:48.528 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.529 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104195 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:48.529 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:37:48.529 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.967021035999998, 2024-09-14T22:37:48.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073876598000000002, 2024-09-14T22:37:48.529 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:48.529 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:48.529 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:48.529 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:48.529 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:48.529 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:48.529 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104195 2024-09-14T22:37:48.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.529 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:48.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:48.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:37:48.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:48.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:37:48.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:48.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490006+0000", 2024-09-14T22:37:48.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4699999999999999e-06 2024-09-14T22:37:48.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:48.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490011+0000", 2024-09-14T22:37:48.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.366e-06 2024-09-14T22:37:48.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:48.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490046+0000", 2024-09-14T22:37:48.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5496e-05 2024-09-14T22:37:48.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:48.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490061+0000", 2024-09-14T22:37:48.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5054e-05 2024-09-14T22:37:48.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.532 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:48.532 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563719+0000", 2024-09-14T22:37:48.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073657412000000005 2024-09-14T22:37:48.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.532 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:48.532 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563792+0000", 2024-09-14T22:37:48.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3319999999999996e-05 2024-09-14T22:37:48.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.532 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:48.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563881+0000", 2024-09-14T22:37:48.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8480000000000007e-05 2024-09-14T22:37:48.533 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.533 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:48.533 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.533 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104196 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16187392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:48.533 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:37:48.533 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.966943886999999, 2024-09-14T22:37:48.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073997496999999995, 2024-09-14T22:37:48.534 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:48.534 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:48.534 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:48.534 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:48.534 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:48.534 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:48.534 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104196 2024-09-14T22:37:48.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.534 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:48.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.534 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:48.534 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:37:48.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:48.535 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:37:48.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:48.535 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490083+0000", 2024-09-14T22:37:48.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.347e-06 2024-09-14T22:37:48.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:48.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490089+0000", 2024-09-14T22:37:48.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9630000000000003e-06 2024-09-14T22:37:48.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:48.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490107+0000", 2024-09-14T22:37:48.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8859999999999999e-05 2024-09-14T22:37:48.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:48.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490122+0000", 2024-09-14T22:37:48.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.432e-05 2024-09-14T22:37:48.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:48.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563910+0000", 2024-09-14T22:37:48.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073788753999999998 2024-09-14T22:37:48.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:48.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563988+0000", 2024-09-14T22:37:48.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7862999999999996e-05 2024-09-14T22:37:48.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:48.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564079+0000", 2024-09-14T22:37:48.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0389999999999993e-05 2024-09-14T22:37:48.538 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.538 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:48.538 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.538 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104197 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16318464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:48.538 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:37:48.538 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.966852257999999, 2024-09-14T22:37:48.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074103547000000006, 2024-09-14T22:37:48.538 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:48.538 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:48.538 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:48.538 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:48.538 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:48.539 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:48.539 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104197 2024-09-14T22:37:48.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.539 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:48.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:48.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:37:48.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:48.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:37:48.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:48.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490174+0000", 2024-09-14T22:37:48.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.37e-06 2024-09-14T22:37:48.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:48.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490197+0000", 2024-09-14T22:37:48.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3017e-05 2024-09-14T22:37:48.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:48.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490219+0000", 2024-09-14T22:37:48.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.181e-05 2024-09-14T22:37:48.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:48.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490240+0000", 2024-09-14T22:37:48.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0786000000000001e-05 2024-09-14T22:37:48.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:48.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564109+0000", 2024-09-14T22:37:48.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073869031000000002 2024-09-14T22:37:48.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:48.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564186+0000", 2024-09-14T22:37:48.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6823999999999995e-05 2024-09-14T22:37:48.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:48.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564276+0000", 2024-09-14T22:37:48.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0709000000000002e-05 2024-09-14T22:37:48.542 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.542 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:48.542 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.542 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104198 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16449536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:48.543 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:37:48.543 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.966760021999999, 2024-09-14T22:37:48.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074167693000000007, 2024-09-14T22:37:48.543 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:48.543 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:48.543 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:48.543 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:48.543 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:48.543 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:48.543 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104198 2024-09-14T22:37:48.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.543 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:48.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.544 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:48.544 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:37:48.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.544 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:48.544 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:37:48.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.544 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:48.544 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490266+0000", 2024-09-14T22:37:48.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3599999999999999e-06 2024-09-14T22:37:48.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:48.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490272+0000", 2024-09-14T22:37:48.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9599999999999997e-06 2024-09-14T22:37:48.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:48.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490291+0000", 2024-09-14T22:37:48.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8485999999999999e-05 2024-09-14T22:37:48.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:48.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490305+0000", 2024-09-14T22:37:48.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4327e-05 2024-09-14T22:37:48.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:48.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564306+0000", 2024-09-14T22:37:48.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074000745000000007 2024-09-14T22:37:48.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:48.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564367+0000", 2024-09-14T22:37:48.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1501999999999995e-05 2024-09-14T22:37:48.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.547 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:48.547 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564433+0000", 2024-09-14T22:37:48.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5313000000000003e-05 2024-09-14T22:37:48.547 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.547 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:48.547 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.547 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.547 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104199 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16580608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:48.547 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:37:48.547 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.966678469000001, 2024-09-14T22:37:48.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074231631000000006, 2024-09-14T22:37:48.547 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:48.547 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:48.548 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:48.548 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:48.548 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:48.548 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:48.548 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104199 2024-09-14T22:37:48.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.548 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:48.548 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.548 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:48.548 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:37:48.548 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.548 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.548 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:48.548 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:37:48.549 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.549 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:48.549 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490349+0000", 2024-09-14T22:37:48.549 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9429999999999999e-06 2024-09-14T22:37:48.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.549 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:48.549 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490355+0000", 2024-09-14T22:37:48.549 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1670000000000004e-06 2024-09-14T22:37:48.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.549 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:48.549 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490374+0000", 2024-09-14T22:37:48.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9613e-05 2024-09-14T22:37:48.550 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.550 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:48.550 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490405+0000", 2024-09-14T22:37:48.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0936000000000003e-05 2024-09-14T22:37:48.550 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.550 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:48.550 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564451+0000", 2024-09-14T22:37:48.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074045266999999998 2024-09-14T22:37:48.550 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.550 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:48.551 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564494+0000", 2024-09-14T22:37:48.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3133000000000002e-05 2024-09-14T22:37:48.551 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.551 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:48.551 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564578+0000", 2024-09-14T22:37:48.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4572000000000003e-05 2024-09-14T22:37:48.551 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.551 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:48.551 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.551 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.551 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104200 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:48.551 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:37:48.552 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.964883240999999, 2024-09-14T22:37:48.552 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072582435000000001, 2024-09-14T22:37:48.552 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:48.552 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:48.552 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:48.552 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:48.552 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:48.552 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:48.552 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104200 2024-09-14T22:37:48.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.552 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:48.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.553 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:48.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:37:48.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.553 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.553 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:48.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:37:48.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:48.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.553 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.553 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:48.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492144+0000", 2024-09-14T22:37:48.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3070000000000001e-06 2024-09-14T22:37:48.554 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.554 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:48.554 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492155+0000", 2024-09-14T22:37:48.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0573e-05 2024-09-14T22:37:48.554 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.554 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:48.554 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492185+0000", 2024-09-14T22:37:48.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.008e-05 2024-09-14T22:37:48.554 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.554 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:48.554 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492205+0000", 2024-09-14T22:37:48.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9882999999999999e-05 2024-09-14T22:37:48.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.555 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.555 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:48.555 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564609+0000", 2024-09-14T22:37:48.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072404181999999997 2024-09-14T22:37:48.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.555 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.555 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:48.555 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564667+0000", 2024-09-14T22:37:48.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8102e-05 2024-09-14T22:37:48.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:48.555 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:48.555 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:48.555 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564724+0000", 2024-09-14T22:37:48.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7308000000000001e-05 2024-09-14T22:37:48.556 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.556 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:48.556 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.556 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:48.556 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:48.556 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:48.557 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:37:48.558 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:48.550+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:48.609 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:37:48.609 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:37:48.626 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:37:48.626 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:37:48.643 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:37:48.643 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:37:48.658 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:48.650+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:48.660 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:37:48.660 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:37:48.681 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:37:48.681 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:37:48.682 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:48.682 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:48.696 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:37:48.702 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:37:48.702 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:37:48.724 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:37:48.724 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:37:48.748 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:37:48.748 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:37:48.759 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:48.754+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:48.827 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:48.827 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:48.841 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:37:48.889 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:48.887+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:48.960 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:48.955+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:48.979 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:48.979 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:48.988 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone33 from 0xe3b9c42 (overlap 0x66694a1) to 0x0 2024-09-14T22:37:48.998 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:37:49.061 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:49.054+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:49.162 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:49.158+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:49.171 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:49.171 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:49.185 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:37:49.263 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:49.259+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:49.313 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:49.313 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:49.327 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:37:49.364 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:49.362+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:49.465 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:49.463+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:49.498 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:49.498 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:49.512 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:37:49.605 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:49.603+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:49.667 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:49.663+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:49.674 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:49.675 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:49.689 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:37:49.777 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:49.775+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:49.858 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:49.858 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:49.868 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:49.862+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:49.873 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:37:49.927 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T22:37:49.970 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:49.962+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:50.064 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:50.064 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:50.070 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:50.067+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:50.084 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:37:50.113 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-14T22:37:50.172 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:50.166+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:50.267 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:50.267 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:50.272 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:50.267+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:50.293 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:37:50.333 INFO:teuthology.orchestra.run.smithi001.stdout:5094 trunc from 0xe3b9c42 to 0x428a51a 2024-09-14T22:37:50.377 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:50.375+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:50.379 INFO:teuthology.orchestra.run.smithi001.stdout:5095 clone 35 order 22 su 16384 sc 2 2024-09-14T22:37:50.459 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:50.459 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:50.472 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:37:50.517 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:50.515+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:50.575 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:50.566+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:50.641 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:50.642 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:50.656 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:37:50.703 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:50.698+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:50.714 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2024-09-14T22:37:50.769 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:37:50.793 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:50.794 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:50.809 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:37:50.813 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:50.811+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:50.826 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone34 from 0x428a51a (overlap 0x428a51a) to 0x6a76e9 2024-09-14T22:37:50.895 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:50.890+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:50.932 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:37:50.932 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:37:50.949 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:37:50.949 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:37:50.966 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:37:50.967 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:37:50.973 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:50.973 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:50.977 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:50.975+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:50.985 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:37:50.985 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:37:50.987 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:37:51.003 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:37:51.004 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:37:51.023 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:37:51.023 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:37:51.043 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:37:51.043 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:37:51.064 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:37:51.064 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:37:51.078 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:51.075+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:51.160 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:51.160 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:51.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:51.160 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:51.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.160 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6395 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8386560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:51.161 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:37:51.161 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 87.221605448999995, 2024-09-14T22:37:51.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054534379000000001, 2024-09-14T22:37:51.161 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.161 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.161 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.161 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.161 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.161 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.161 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6395 2024-09-14T22:37:51.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.161 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:37:51.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:37:51.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933833+0000", 2024-09-14T22:37:51.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3859999999999999e-06 2024-09-14T22:37:51.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933838+0000", 2024-09-14T22:37:51.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8709999999999996e-06 2024-09-14T22:37:51.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933854+0000", 2024-09-14T22:37:51.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6189e-05 2024-09-14T22:37:51.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933871+0000", 2024-09-14T22:37:51.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.626e-05 2024-09-14T22:37:51.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988198+0000", 2024-09-14T22:37:51.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054327581999999999 2024-09-14T22:37:51.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988274+0000", 2024-09-14T22:37:51.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5759000000000004e-05 2024-09-14T22:37:51.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988366+0000", 2024-09-14T22:37:51.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2331999999999996e-05 2024-09-14T22:37:51.165 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.165 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.165 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.165 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104944 3.3 3:c3780f3b:::rbd_data.113ca561f7a9.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:51.165 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:37:51.165 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.122003575999997, 2024-09-14T22:37:51.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073041087000000005, 2024-09-14T22:37:51.166 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.166 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.166 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:51.166 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.166 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.166 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.166 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104944 2024-09-14T22:37:51.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.166 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:37:51.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:37:51.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033436+0000", 2024-09-14T22:37:51.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2060000000000001e-06 2024-09-14T22:37:51.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033448+0000", 2024-09-14T22:37:51.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.169e-05 2024-09-14T22:37:51.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033474+0000", 2024-09-14T22:37:51.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6400000000000001e-05 2024-09-14T22:37:51.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033489+0000", 2024-09-14T22:37:51.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5067e-05 2024-09-14T22:37:51.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033658+0000", 2024-09-14T22:37:51.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000168565 2024-09-14T22:37:51.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033793+0000", 2024-09-14T22:37:51.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013536900000000001 2024-09-14T22:37:51.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:37:51.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.047559+0000", 2024-09-14T22:37:51.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013766294 2024-09-14T22:37:51.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:51.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106157+0000", 2024-09-14T22:37:51.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058597110000000001 2024-09-14T22:37:51.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:51.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106290+0000", 2024-09-14T22:37:51.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000133244 2024-09-14T22:37:51.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:51.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106396+0000", 2024-09-14T22:37:51.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106246 2024-09-14T22:37:51.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106475+0000", 2024-09-14T22:37:51.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8895999999999998e-05 2024-09-14T22:37:51.171 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.171 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.171 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.172 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:111858 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 10403840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:51.172 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:37:51.172 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.331550454999999, 2024-09-14T22:37:51.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054398798999999998, 2024-09-14T22:37:51.172 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.172 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.172 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.172 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.172 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.172 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.172 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 111858 2024-09-14T22:37:51.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.173 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:37:51.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:37:51.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823888+0000", 2024-09-14T22:37:51.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0200000000000001e-07 2024-09-14T22:37:51.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823927+0000", 2024-09-14T22:37:51.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8940999999999999e-05 2024-09-14T22:37:51.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823940+0000", 2024-09-14T22:37:51.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3682e-05 2024-09-14T22:37:51.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823950+0000", 2024-09-14T22:37:51.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7110000000000007e-06 2024-09-14T22:37:51.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878202+0000", 2024-09-14T22:37:51.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054251969999999997 2024-09-14T22:37:51.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878249+0000", 2024-09-14T22:37:51.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6854999999999998e-05 2024-09-14T22:37:51.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878286+0000", 2024-09-14T22:37:51.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6838000000000002e-05 2024-09-14T22:37:51.176 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.176 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.176 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.176 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:111859 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 10452992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:51.176 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:37:51.176 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.331461336, 2024-09-14T22:37:51.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054435292000000003, 2024-09-14T22:37:51.177 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.177 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.177 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.177 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.177 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.177 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.177 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 111859 2024-09-14T22:37:51.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.177 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:37:51.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:37:51.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823977+0000", 2024-09-14T22:37:51.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3500000000000005e-07 2024-09-14T22:37:51.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823981+0000", 2024-09-14T22:37:51.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2050000000000004e-06 2024-09-14T22:37:51.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824007+0000", 2024-09-14T22:37:51.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.62e-05 2024-09-14T22:37:51.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824018+0000", 2024-09-14T22:37:51.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0389e-05 2024-09-14T22:37:51.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878312+0000", 2024-09-14T22:37:51.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054294754000000001 2024-09-14T22:37:51.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878350+0000", 2024-09-14T22:37:51.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7982000000000003e-05 2024-09-14T22:37:51.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878411+0000", 2024-09-14T22:37:51.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0927000000000001e-05 2024-09-14T22:37:51.181 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.181 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.181 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.181 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112030 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6316032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:51.181 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:37:51.181 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.331406994000002, 2024-09-14T22:37:51.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054597826000000002, 2024-09-14T22:37:51.181 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.181 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.181 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.181 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.182 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.182 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.182 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112030 2024-09-14T22:37:51.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.182 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:37:51.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:37:51.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824031+0000", 2024-09-14T22:37:51.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3600000000000003e-07 2024-09-14T22:37:51.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824036+0000", 2024-09-14T22:37:51.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7350000000000004e-06 2024-09-14T22:37:51.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824049+0000", 2024-09-14T22:37:51.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3329000000000001e-05 2024-09-14T22:37:51.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824057+0000", 2024-09-14T22:37:51.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2460000000000003e-06 2024-09-14T22:37:51.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878464+0000", 2024-09-14T22:37:51.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054406207999999998 2024-09-14T22:37:51.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878554+0000", 2024-09-14T22:37:51.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0199999999999997e-05 2024-09-14T22:37:51.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878628+0000", 2024-09-14T22:37:51.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4371999999999999e-05 2024-09-14T22:37:51.185 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.185 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.185 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.185 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112031 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6365184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:51.186 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:37:51.186 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.331368676, 2024-09-14T22:37:51.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054741868999999999, 2024-09-14T22:37:51.186 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.186 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.186 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.186 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.186 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.186 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.186 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112031 2024-09-14T22:37:51.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.186 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:37:51.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:37:51.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824070+0000", 2024-09-14T22:37:51.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5099999999999998e-07 2024-09-14T22:37:51.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824073+0000", 2024-09-14T22:37:51.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6940000000000001e-06 2024-09-14T22:37:51.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824084+0000", 2024-09-14T22:37:51.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0380000000000001e-05 2024-09-14T22:37:51.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824091+0000", 2024-09-14T22:37:51.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5730000000000002e-06 2024-09-14T22:37:51.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878670+0000", 2024-09-14T22:37:51.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054579176 2024-09-14T22:37:51.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878742+0000", 2024-09-14T22:37:51.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T22:37:51.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878811+0000", 2024-09-14T22:37:51.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8170999999999995e-05 2024-09-14T22:37:51.190 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.190 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.190 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.191 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112032 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6414336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:51.191 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:37:51.191 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.331334810000001, 2024-09-14T22:37:51.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054931916999999997, 2024-09-14T22:37:51.191 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.191 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.191 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.191 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.191 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.191 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.191 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112032 2024-09-14T22:37:51.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.191 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:37:51.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:37:51.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:37:51.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:37:51.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824107+0000", 2024-09-14T22:37:51.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5640000000000001e-06 2024-09-14T22:37:51.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824117+0000", 2024-09-14T22:37:51.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0366e-05 2024-09-14T22:37:51.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824158+0000", 2024-09-14T22:37:51.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0670000000000002e-05 2024-09-14T22:37:51.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878853+0000", 2024-09-14T22:37:51.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054694853000000002 2024-09-14T22:37:51.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878934+0000", 2024-09-14T22:37:51.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1754000000000004e-05 2024-09-14T22:37:51.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879034+0000", 2024-09-14T22:37:51.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000100115 2024-09-14T22:37:51.195 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.195 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.195 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.195 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112033 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6463488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:51.195 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:37:51.195 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.331266152000001, 2024-09-14T22:37:51.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054991076999999999, 2024-09-14T22:37:51.195 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.195 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.196 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.196 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.196 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.196 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.196 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112033 2024-09-14T22:37:51.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.196 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:37:51.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:37:51.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824172+0000", 2024-09-14T22:37:51.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9999999999999996e-07 2024-09-14T22:37:51.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824177+0000", 2024-09-14T22:37:51.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5310000000000003e-06 2024-09-14T22:37:51.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824188+0000", 2024-09-14T22:37:51.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1235e-05 2024-09-14T22:37:51.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824197+0000", 2024-09-14T22:37:51.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7069999999999998e-06 2024-09-14T22:37:51.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879075+0000", 2024-09-14T22:37:51.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054878749999999997 2024-09-14T22:37:51.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879118+0000", 2024-09-14T22:37:51.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2749e-05 2024-09-14T22:37:51.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879162+0000", 2024-09-14T22:37:51.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4205000000000001e-05 2024-09-14T22:37:51.199 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.199 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.200 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.200 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112034 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6512640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:51.200 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:37:51.200 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.331229200999999, 2024-09-14T22:37:51.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055181246000000003, 2024-09-14T22:37:51.200 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.200 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.200 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.200 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.200 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.200 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.200 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112034 2024-09-14T22:37:51.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.201 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:37:51.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:37:51.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824209+0000", 2024-09-14T22:37:51.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5499999999999998e-07 2024-09-14T22:37:51.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824212+0000", 2024-09-14T22:37:51.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6119999999999999e-06 2024-09-14T22:37:51.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824224+0000", 2024-09-14T22:37:51.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1127999999999999e-05 2024-09-14T22:37:51.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824232+0000", 2024-09-14T22:37:51.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4700000000000002e-06 2024-09-14T22:37:51.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879209+0000", 2024-09-14T22:37:51.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054977201000000003 2024-09-14T22:37:51.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879308+0000", 2024-09-14T22:37:51.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8419999999999996e-05 2024-09-14T22:37:51.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879389+0000", 2024-09-14T22:37:51.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1760000000000003e-05 2024-09-14T22:37:51.204 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.204 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.204 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.205 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112035 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6561792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:51.205 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:37:51.205 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.331135473, 2024-09-14T22:37:51.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055300522999999997, 2024-09-14T22:37:51.205 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.205 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.205 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.205 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.205 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.205 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.205 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112035 2024-09-14T22:37:51.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.205 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:37:51.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:37:51.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824303+0000", 2024-09-14T22:37:51.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5099999999999998e-07 2024-09-14T22:37:51.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824308+0000", 2024-09-14T22:37:51.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6689999999999997e-06 2024-09-14T22:37:51.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824744+0000", 2024-09-14T22:37:51.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00043601600000000001 2024-09-14T22:37:51.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824756+0000", 2024-09-14T22:37:51.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2476999999999999e-05 2024-09-14T22:37:51.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.208 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879434+0000", 2024-09-14T22:37:51.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054677630999999997 2024-09-14T22:37:51.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.208 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879518+0000", 2024-09-14T22:37:51.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4220000000000003e-05 2024-09-14T22:37:51.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879602+0000", 2024-09-14T22:37:51.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4559000000000001e-05 2024-09-14T22:37:51.209 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.209 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.209 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.209 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112036 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6610944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:51.209 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:37:51.209 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.330643734999999, 2024-09-14T22:37:51.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055036320999999999, 2024-09-14T22:37:51.209 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.209 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.210 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.210 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.210 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.210 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.210 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112036 2024-09-14T22:37:51.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.210 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:37:51.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:37:51.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824795+0000", 2024-09-14T22:37:51.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-06 2024-09-14T22:37:51.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824802+0000", 2024-09-14T22:37:51.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6680000000000004e-06 2024-09-14T22:37:51.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.212 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.212 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824814+0000", 2024-09-14T22:37:51.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2102e-05 2024-09-14T22:37:51.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.212 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.212 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824823+0000", 2024-09-14T22:37:51.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5550000000000008e-06 2024-09-14T22:37:51.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.212 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.212 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879648+0000", 2024-09-14T22:37:51.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054825167000000001 2024-09-14T22:37:51.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879747+0000", 2024-09-14T22:37:51.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8968999999999994e-05 2024-09-14T22:37:51.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879830+0000", 2024-09-14T22:37:51.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2633999999999995e-05 2024-09-14T22:37:51.213 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.213 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.213 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.214 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112037 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6660096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:51.214 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:37:51.214 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.330601026, 2024-09-14T22:37:51.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055122926000000003, 2024-09-14T22:37:51.214 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.214 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.214 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.214 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.214 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.214 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.214 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112037 2024-09-14T22:37:51.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.215 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:37:51.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:37:51.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824837+0000", 2024-09-14T22:37:51.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5499999999999998e-07 2024-09-14T22:37:51.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824840+0000", 2024-09-14T22:37:51.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0429999999999999e-06 2024-09-14T22:37:51.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824851+0000", 2024-09-14T22:37:51.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1046999999999999e-05 2024-09-14T22:37:51.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824859+0000", 2024-09-14T22:37:51.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1200000000000002e-06 2024-09-14T22:37:51.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879867+0000", 2024-09-14T22:37:51.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055007697000000001 2024-09-14T22:37:51.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879910+0000", 2024-09-14T22:37:51.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3402000000000002e-05 2024-09-14T22:37:51.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879959+0000", 2024-09-14T22:37:51.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9061999999999999e-05 2024-09-14T22:37:51.218 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.218 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.218 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.218 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112038 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6709248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:51.219 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:37:51.219 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.33056633, 2024-09-14T22:37:51.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055312828000000001, 2024-09-14T22:37:51.219 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.219 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.219 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.219 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.219 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.219 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.219 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112038 2024-09-14T22:37:51.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.219 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:37:51.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:37:51.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824872+0000", 2024-09-14T22:37:51.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T22:37:51.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824875+0000", 2024-09-14T22:37:51.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7519999999999998e-06 2024-09-14T22:37:51.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824917+0000", 2024-09-14T22:37:51.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1047999999999997e-05 2024-09-14T22:37:51.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824926+0000", 2024-09-14T22:37:51.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3749999999999992e-06 2024-09-14T22:37:51.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880006+0000", 2024-09-14T22:37:51.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055080525999999998 2024-09-14T22:37:51.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880102+0000", 2024-09-14T22:37:51.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5995999999999994e-05 2024-09-14T22:37:51.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880184+0000", 2024-09-14T22:37:51.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1478999999999997e-05 2024-09-14T22:37:51.223 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.223 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.223 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.223 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112039 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6758400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:51.223 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:37:51.224 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.330497659999999, 2024-09-14T22:37:51.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055449709999999999, 2024-09-14T22:37:51.224 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.224 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.224 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.224 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.224 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.224 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.224 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112039 2024-09-14T22:37:51.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.224 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:37:51.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:37:51.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824941+0000", 2024-09-14T22:37:51.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3699999999999999e-07 2024-09-14T22:37:51.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824946+0000", 2024-09-14T22:37:51.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0209999999999999e-06 2024-09-14T22:37:51.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824972+0000", 2024-09-14T22:37:51.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6750999999999998e-05 2024-09-14T22:37:51.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824982+0000", 2024-09-14T22:37:51.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5019999999999998e-06 2024-09-14T22:37:51.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880227+0000", 2024-09-14T22:37:51.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055245257999999998 2024-09-14T22:37:51.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880305+0000", 2024-09-14T22:37:51.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7424999999999998e-05 2024-09-14T22:37:51.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880389+0000", 2024-09-14T22:37:51.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4815999999999996e-05 2024-09-14T22:37:51.228 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.228 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.228 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.228 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112040 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6807552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:51.228 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:37:51.228 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.330440943999999, 2024-09-14T22:37:51.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055534615000000002, 2024-09-14T22:37:51.228 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.228 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.228 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.228 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.229 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.229 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.229 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112040 2024-09-14T22:37:51.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.229 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:37:51.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:37:51.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824997+0000", 2024-09-14T22:37:51.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7199999999999998e-07 2024-09-14T22:37:51.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825042+0000", 2024-09-14T22:37:51.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5065999999999998e-05 2024-09-14T22:37:51.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825060+0000", 2024-09-14T22:37:51.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7520000000000002e-05 2024-09-14T22:37:51.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825070+0000", 2024-09-14T22:37:51.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8230000000000006e-06 2024-09-14T22:37:51.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880427+0000", 2024-09-14T22:37:51.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055357244 2024-09-14T22:37:51.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880483+0000", 2024-09-14T22:37:51.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5807000000000003e-05 2024-09-14T22:37:51.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880531+0000", 2024-09-14T22:37:51.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8383000000000001e-05 2024-09-14T22:37:51.232 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.232 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.232 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.233 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112041 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6856704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:51.233 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:37:51.233 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.33035177, 2024-09-14T22:37:51.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055573887000000002, 2024-09-14T22:37:51.233 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.233 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.233 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.233 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.233 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.233 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.233 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112041 2024-09-14T22:37:51.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.233 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:37:51.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:37:51.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825087+0000", 2024-09-14T22:37:51.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2399999999999996e-07 2024-09-14T22:37:51.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825091+0000", 2024-09-14T22:37:51.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9729999999999999e-06 2024-09-14T22:37:51.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825101+0000", 2024-09-14T22:37:51.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.093e-05 2024-09-14T22:37:51.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825110+0000", 2024-09-14T22:37:51.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1450000000000008e-06 2024-09-14T22:37:51.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880559+0000", 2024-09-14T22:37:51.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055449684999999999 2024-09-14T22:37:51.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880601+0000", 2024-09-14T22:37:51.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1248000000000002e-05 2024-09-14T22:37:51.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880659+0000", 2024-09-14T22:37:51.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8981999999999999e-05 2024-09-14T22:37:51.239 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.239 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.239 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.239 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112042 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6905856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:51.239 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:37:51.239 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.330317036, 2024-09-14T22:37:51.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055781415000000001, 2024-09-14T22:37:51.239 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.239 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.239 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.239 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.240 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.240 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.240 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112042 2024-09-14T22:37:51.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.240 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:37:51.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:37:51.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825478+0000", 2024-09-14T22:37:51.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035775 2024-09-14T22:37:51.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825494+0000", 2024-09-14T22:37:51.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5994999999999998e-05 2024-09-14T22:37:51.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825520+0000", 2024-09-14T22:37:51.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5738999999999999e-05 2024-09-14T22:37:51.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825547+0000", 2024-09-14T22:37:51.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7667000000000001e-05 2024-09-14T22:37:51.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880725+0000", 2024-09-14T22:37:51.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055177962999999997 2024-09-14T22:37:51.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880816+0000", 2024-09-14T22:37:51.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0315000000000005e-05 2024-09-14T22:37:51.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880902+0000", 2024-09-14T22:37:51.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5986e-05 2024-09-14T22:37:51.243 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.243 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.243 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.243 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112043 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:51.243 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:37:51.244 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.329869032000001, 2024-09-14T22:37:51.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055537912000000002, 2024-09-14T22:37:51.244 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.244 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.244 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.244 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.244 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.244 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.244 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112043 2024-09-14T22:37:51.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.244 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.244 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:37:51.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:37:51.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825911+0000", 2024-09-14T22:37:51.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00034306899999999999 2024-09-14T22:37:51.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825918+0000", 2024-09-14T22:37:51.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.002e-06 2024-09-14T22:37:51.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825931+0000", 2024-09-14T22:37:51.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2401e-05 2024-09-14T22:37:51.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825942+0000", 2024-09-14T22:37:51.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0903e-05 2024-09-14T22:37:51.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880951+0000", 2024-09-14T22:37:51.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055009379999999997 2024-09-14T22:37:51.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881027+0000", 2024-09-14T22:37:51.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5944000000000003e-05 2024-09-14T22:37:51.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.248 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.248 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881106+0000", 2024-09-14T22:37:51.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9213000000000002e-05 2024-09-14T22:37:51.248 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.248 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.248 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.248 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112044 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 7004160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:51.248 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:37:51.248 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.329475363, 2024-09-14T22:37:51.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055265416999999997, 2024-09-14T22:37:51.248 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.248 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.249 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.249 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.249 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.249 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.249 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112044 2024-09-14T22:37:51.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.249 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.249 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.249 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:37:51.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:37:51.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826239+0000", 2024-09-14T22:37:51.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027676000000000003 2024-09-14T22:37:51.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826247+0000", 2024-09-14T22:37:51.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7440000000000007e-06 2024-09-14T22:37:51.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826275+0000", 2024-09-14T22:37:51.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8053000000000001e-05 2024-09-14T22:37:51.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826286+0000", 2024-09-14T22:37:51.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1780000000000001e-05 2024-09-14T22:37:51.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881137+0000", 2024-09-14T22:37:51.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054850500000000003 2024-09-14T22:37:51.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881181+0000", 2024-09-14T22:37:51.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4153000000000002e-05 2024-09-14T22:37:51.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881227+0000", 2024-09-14T22:37:51.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6427000000000001e-05 2024-09-14T22:37:51.252 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.253 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.253 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.253 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112045 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 7053312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:51.253 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:37:51.253 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.32911313, 2024-09-14T22:37:51.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055045050999999998, 2024-09-14T22:37:51.253 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.253 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.253 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.253 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.253 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.253 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.254 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112045 2024-09-14T22:37:51.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.254 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.254 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.254 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:37:51.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.254 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.254 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:37:51.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830250+0000", 2024-09-14T22:37:51.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0039258670000000004 2024-09-14T22:37:51.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830260+0000", 2024-09-14T22:37:51.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0088000000000001e-05 2024-09-14T22:37:51.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830281+0000", 2024-09-14T22:37:51.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0720000000000002e-05 2024-09-14T22:37:51.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830292+0000", 2024-09-14T22:37:51.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1455e-05 2024-09-14T22:37:51.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881255+0000", 2024-09-14T22:37:51.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.050962247000000002 2024-09-14T22:37:51.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881296+0000", 2024-09-14T22:37:51.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1557999999999998e-05 2024-09-14T22:37:51.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881369+0000", 2024-09-14T22:37:51.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3115999999999996e-05 2024-09-14T22:37:51.257 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.257 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.257 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.257 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.257 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.257 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:51.259 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:37:51.259 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:51.255+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:51.360 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:51.354+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:51.386 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:51.386 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:51.386 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:37:51.386 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:51.399 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:37:51.461 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:51.459+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:51.562 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:51.554+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:51.565 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:51.565 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:51.565 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:37:51.565 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:37:51.565 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:51.579 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:37:51.667 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:51.662+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:51.759 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:51.759 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:51.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:51.760 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:51.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.760 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97197 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 796672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:51.760 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:37:51.760 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 41.684762298000003, 2024-09-14T22:37:51.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12434473, 2024-09-14T22:37:51.760 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.760 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.760 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.760 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.760 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.760 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.761 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97197 2024-09-14T22:37:51.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.761 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:37:51.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:37:51.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070524+0000", 2024-09-14T22:37:51.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0299999999999998e-07 2024-09-14T22:37:51.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070527+0000", 2024-09-14T22:37:51.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9569999999999998e-06 2024-09-14T22:37:51.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070535+0000", 2024-09-14T22:37:51.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7630000000000006e-06 2024-09-14T22:37:51.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070543+0000", 2024-09-14T22:37:51.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4869999999999996e-06 2024-09-14T22:37:51.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194525+0000", 2024-09-14T22:37:51.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123982096 2024-09-14T22:37:51.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.764 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.764 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194699+0000", 2024-09-14T22:37:51.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000174105 2024-09-14T22:37:51.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194868+0000", 2024-09-14T22:37:51.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016861900000000001 2024-09-14T22:37:51.765 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.765 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.765 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.765 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97198 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 805888~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:51.765 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:37:51.765 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 41.684733921999999, 2024-09-14T22:37:51.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.1248088, 2024-09-14T22:37:51.765 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.765 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.766 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.766 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.766 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.766 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.766 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97198 2024-09-14T22:37:51.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.766 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:37:51.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:37:51.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070552+0000", 2024-09-14T22:37:51.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T22:37:51.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070555+0000", 2024-09-14T22:37:51.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.853e-06 2024-09-14T22:37:51.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070565+0000", 2024-09-14T22:37:51.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.048e-05 2024-09-14T22:37:51.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070573+0000", 2024-09-14T22:37:51.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7200000000000006e-06 2024-09-14T22:37:51.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194954+0000", 2024-09-14T22:37:51.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12438089400000001 2024-09-14T22:37:51.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195145+0000", 2024-09-14T22:37:51.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019091299999999999 2024-09-14T22:37:51.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195360+0000", 2024-09-14T22:37:51.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021527300000000001 2024-09-14T22:37:51.769 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.770 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.770 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.770 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97199 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 815104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:51.770 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:37:51.770 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 41.684703325000001, 2024-09-14T22:37:51.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125251901, 2024-09-14T22:37:51.770 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.770 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.770 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.770 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.770 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.770 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.770 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97199 2024-09-14T22:37:51.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.771 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:37:51.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:37:51.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070583+0000", 2024-09-14T22:37:51.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6300000000000005e-07 2024-09-14T22:37:51.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070586+0000", 2024-09-14T22:37:51.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8270000000000002e-06 2024-09-14T22:37:51.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070597+0000", 2024-09-14T22:37:51.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1960000000000001e-05 2024-09-14T22:37:51.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070605+0000", 2024-09-14T22:37:51.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6459999999999993e-06 2024-09-14T22:37:51.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195432+0000", 2024-09-14T22:37:51.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124826926 2024-09-14T22:37:51.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195626+0000", 2024-09-14T22:37:51.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019357099999999999 2024-09-14T22:37:51.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195834+0000", 2024-09-14T22:37:51.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020830800000000001 2024-09-14T22:37:51.774 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.774 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.774 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.774 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97200 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 824320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:51.775 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:37:51.775 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 41.684671279, 2024-09-14T22:37:51.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125693477, 2024-09-14T22:37:51.775 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.775 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.775 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.775 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.775 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.775 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.775 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97200 2024-09-14T22:37:51.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.775 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.776 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:37:51.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.776 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:37:51.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.776 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070615+0000", 2024-09-14T22:37:51.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T22:37:51.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070618+0000", 2024-09-14T22:37:51.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.943e-06 2024-09-14T22:37:51.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070627+0000", 2024-09-14T22:37:51.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1099999999999992e-06 2024-09-14T22:37:51.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070634+0000", 2024-09-14T22:37:51.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:37:51.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195908+0000", 2024-09-14T22:37:51.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125273893 2024-09-14T22:37:51.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196097+0000", 2024-09-14T22:37:51.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018953899999999999 2024-09-14T22:37:51.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196308+0000", 2024-09-14T22:37:51.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021027799999999999 2024-09-14T22:37:51.779 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.779 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.779 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.779 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97201 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 833536~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:51.779 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:37:51.779 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 41.684641092, 2024-09-14T22:37:51.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126133826, 2024-09-14T22:37:51.779 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.780 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.780 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.780 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.780 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.780 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.780 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97201 2024-09-14T22:37:51.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.780 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:37:51.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:37:51.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074109+0000", 2024-09-14T22:37:51.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003464834 2024-09-14T22:37:51.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074134+0000", 2024-09-14T22:37:51.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5170000000000001e-05 2024-09-14T22:37:51.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074166+0000", 2024-09-14T22:37:51.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1347000000000003e-05 2024-09-14T22:37:51.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074179+0000", 2024-09-14T22:37:51.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.364e-05 2024-09-14T22:37:51.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196377+0000", 2024-09-14T22:37:51.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12219793900000001 2024-09-14T22:37:51.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196568+0000", 2024-09-14T22:37:51.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019097999999999999 2024-09-14T22:37:51.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196778+0000", 2024-09-14T22:37:51.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020991599999999999 2024-09-14T22:37:51.784 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.784 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.784 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.784 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97205 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 870400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:51.784 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:37:51.784 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 41.680971528999997, 2024-09-14T22:37:51.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12436148299999999, 2024-09-14T22:37:51.784 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.784 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.784 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.784 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.784 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.785 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.785 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97205 2024-09-14T22:37:51.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.785 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:37:51.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:37:51.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074315+0000", 2024-09-14T22:37:51.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9299999999999997e-07 2024-09-14T22:37:51.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074318+0000", 2024-09-14T22:37:51.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.19e-06 2024-09-14T22:37:51.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074327+0000", 2024-09-14T22:37:51.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8529999999999998e-06 2024-09-14T22:37:51.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074334+0000", 2024-09-14T22:37:51.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3869999999999997e-06 2024-09-14T22:37:51.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198365+0000", 2024-09-14T22:37:51.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124031222 2024-09-14T22:37:51.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198513+0000", 2024-09-14T22:37:51.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014772299999999999 2024-09-14T22:37:51.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198675+0000", 2024-09-14T22:37:51.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016241500000000001 2024-09-14T22:37:51.788 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.788 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.788 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.789 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97206 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 879616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:51.789 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:37:51.789 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 41.680940073000002, 2024-09-14T22:37:51.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124706284, 2024-09-14T22:37:51.789 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.789 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.789 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.789 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.789 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.789 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.789 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97206 2024-09-14T22:37:51.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.789 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:37:51.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:37:51.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074346+0000", 2024-09-14T22:37:51.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8999999999999996e-07 2024-09-14T22:37:51.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074349+0000", 2024-09-14T22:37:51.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.96e-06 2024-09-14T22:37:51.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074358+0000", 2024-09-14T22:37:51.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9169999999999995e-06 2024-09-14T22:37:51.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074365+0000", 2024-09-14T22:37:51.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3699999999999997e-06 2024-09-14T22:37:51.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198761+0000", 2024-09-14T22:37:51.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124395743 2024-09-14T22:37:51.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198899+0000", 2024-09-14T22:37:51.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013763499999999999 2024-09-14T22:37:51.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199052+0000", 2024-09-14T22:37:51.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015296900000000001 2024-09-14T22:37:51.793 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.793 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.793 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.793 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97207 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 888832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:51.794 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:37:51.794 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 41.680910859999997, 2024-09-14T22:37:51.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124988433, 2024-09-14T22:37:51.794 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.794 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.794 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.794 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.794 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.794 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.794 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97207 2024-09-14T22:37:51.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.794 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:37:51.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:37:51.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074375+0000", 2024-09-14T22:37:51.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:37:51.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074378+0000", 2024-09-14T22:37:51.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1829999999999998e-06 2024-09-14T22:37:51.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074389+0000", 2024-09-14T22:37:51.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0737000000000001e-05 2024-09-14T22:37:51.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074396+0000", 2024-09-14T22:37:51.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2559999999999996e-06 2024-09-14T22:37:51.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199139+0000", 2024-09-14T22:37:51.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12474253 2024-09-14T22:37:51.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199240+0000", 2024-09-14T22:37:51.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101221 2024-09-14T22:37:51.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199363+0000", 2024-09-14T22:37:51.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000122822 2024-09-14T22:37:51.798 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.798 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.798 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.798 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97208 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 898048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:51.798 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:37:51.798 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 41.680879556000001, 2024-09-14T22:37:51.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12524854899999999, 2024-09-14T22:37:51.798 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.798 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.799 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.799 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.799 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.799 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.799 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97208 2024-09-14T22:37:51.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.799 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:37:51.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:37:51.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:37:51.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:37:51.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074410+0000", 2024-09-14T22:37:51.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.18e-06 2024-09-14T22:37:51.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074435+0000", 2024-09-14T22:37:51.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5273e-05 2024-09-14T22:37:51.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074443+0000", 2024-09-14T22:37:51.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3000000000000002e-06 2024-09-14T22:37:51.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199433+0000", 2024-09-14T22:37:51.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124989502 2024-09-14T22:37:51.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199535+0000", 2024-09-14T22:37:51.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102557 2024-09-14T22:37:51.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199654+0000", 2024-09-14T22:37:51.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119061 2024-09-14T22:37:51.802 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.802 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.802 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.803 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97209 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 907264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:51.803 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:37:51.803 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 41.680831750999999, 2024-09-14T22:37:51.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125487354, 2024-09-14T22:37:51.803 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.803 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.803 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.803 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.803 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.803 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.803 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97209 2024-09-14T22:37:51.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.803 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:37:51.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:37:51.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:37:51.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9400000000000005e-07 2024-09-14T22:37:51.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074458+0000", 2024-09-14T22:37:51.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3299999999999999e-06 2024-09-14T22:37:51.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074467+0000", 2024-09-14T22:37:51.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.346e-06 2024-09-14T22:37:51.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074474+0000", 2024-09-14T22:37:51.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.374e-06 2024-09-14T22:37:51.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199720+0000", 2024-09-14T22:37:51.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12524561200000001 2024-09-14T22:37:51.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199820+0000", 2024-09-14T22:37:51.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010049599999999999 2024-09-14T22:37:51.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199941+0000", 2024-09-14T22:37:51.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012050200000000001 2024-09-14T22:37:51.807 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.807 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.807 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.807 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97210 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:51.807 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:37:51.807 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 41.680801756999998, 2024-09-14T22:37:51.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125744942, 2024-09-14T22:37:51.807 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.807 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.807 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.808 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.808 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.808 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.808 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97210 2024-09-14T22:37:51.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.808 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:37:51.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:37:51.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:37:51.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6599999999999996e-07 2024-09-14T22:37:51.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074487+0000", 2024-09-14T22:37:51.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8140000000000001e-06 2024-09-14T22:37:51.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074496+0000", 2024-09-14T22:37:51.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9330000000000007e-06 2024-09-14T22:37:51.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074503+0000", 2024-09-14T22:37:51.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4100000000000002e-06 2024-09-14T22:37:51.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200009+0000", 2024-09-14T22:37:51.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12550539899999999 2024-09-14T22:37:51.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200108+0000", 2024-09-14T22:37:51.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9182e-05 2024-09-14T22:37:51.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200229+0000", 2024-09-14T22:37:51.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120538 2024-09-14T22:37:51.811 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.811 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.811 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.812 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97211 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 925696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:51.812 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:37:51.812 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 41.680770797000001, 2024-09-14T22:37:51.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12599848299999999, 2024-09-14T22:37:51.812 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.812 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.812 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.812 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.812 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.812 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.813 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97211 2024-09-14T22:37:51.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.813 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:37:51.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:37:51.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074516+0000", 2024-09-14T22:37:51.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.42e-06 2024-09-14T22:37:51.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074521+0000", 2024-09-14T22:37:51.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7960000000000003e-06 2024-09-14T22:37:51.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074538+0000", 2024-09-14T22:37:51.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6900000000000001e-05 2024-09-14T22:37:51.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074555+0000", 2024-09-14T22:37:51.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.696e-05 2024-09-14T22:37:51.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200293+0000", 2024-09-14T22:37:51.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12573864100000001 2024-09-14T22:37:51.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200393+0000", 2024-09-14T22:37:51.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9264000000000004e-05 2024-09-14T22:37:51.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200513+0000", 2024-09-14T22:37:51.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012050200000000001 2024-09-14T22:37:51.816 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.816 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.816 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.817 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97212 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 934912~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:51.817 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:37:51.817 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 41.680696175000001, 2024-09-14T22:37:51.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12621127300000001, 2024-09-14T22:37:51.817 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.817 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.817 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.817 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.817 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.817 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.817 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97212 2024-09-14T22:37:51.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.818 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:37:51.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:37:51.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.819 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.819 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074590+0000", 2024-09-14T22:37:51.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1969999999999999e-06 2024-09-14T22:37:51.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.819 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.819 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074596+0000", 2024-09-14T22:37:51.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7629999999999997e-06 2024-09-14T22:37:51.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074614+0000", 2024-09-14T22:37:51.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7643e-05 2024-09-14T22:37:51.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074635+0000", 2024-09-14T22:37:51.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.075e-05 2024-09-14T22:37:51.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200580+0000", 2024-09-14T22:37:51.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125945482 2024-09-14T22:37:51.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200683+0000", 2024-09-14T22:37:51.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010267099999999999 2024-09-14T22:37:51.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200800+0000", 2024-09-14T22:37:51.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117767 2024-09-14T22:37:51.821 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.821 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.822 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.822 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97213 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 944128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:51.822 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:37:51.822 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 41.680630825000001, 2024-09-14T22:37:51.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126430709, 2024-09-14T22:37:51.822 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.822 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.822 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.822 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.822 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.823 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.823 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97213 2024-09-14T22:37:51.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.823 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:37:51.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:37:51.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.824 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.824 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078210+0000", 2024-09-14T22:37:51.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0035556839999999999 2024-09-14T22:37:51.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.824 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.824 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078231+0000", 2024-09-14T22:37:51.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0716e-05 2024-09-14T22:37:51.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.824 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.824 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078260+0000", 2024-09-14T22:37:51.825 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9473000000000001e-05 2024-09-14T22:37:51.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.825 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.825 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078279+0000", 2024-09-14T22:37:51.825 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8600000000000001e-05 2024-09-14T22:37:51.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.825 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.826 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200864+0000", 2024-09-14T22:37:51.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.122585295 2024-09-14T22:37:51.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.826 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.826 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200965+0000", 2024-09-14T22:37:51.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101004 2024-09-14T22:37:51.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.826 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.826 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.201085+0000", 2024-09-14T22:37:51.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119937 2024-09-14T22:37:51.826 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.826 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.827 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.827 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97219 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 999424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:51.827 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:37:51.827 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 41.676648548000003, 2024-09-14T22:37:51.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124138558, 2024-09-14T22:37:51.827 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.827 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.827 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.827 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.827 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.828 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.828 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97219 2024-09-14T22:37:51.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.828 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:37:51.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:37:51.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078638+0000", 2024-09-14T22:37:51.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.173e-06 2024-09-14T22:37:51.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078644+0000", 2024-09-14T22:37:51.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2570000000000004e-06 2024-09-14T22:37:51.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078672+0000", 2024-09-14T22:37:51.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7322999999999999e-05 2024-09-14T22:37:51.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078687+0000", 2024-09-14T22:37:51.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5639999999999999e-05 2024-09-14T22:37:51.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202567+0000", 2024-09-14T22:37:51.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123880134 2024-09-14T22:37:51.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202670+0000", 2024-09-14T22:37:51.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010273899999999999 2024-09-14T22:37:51.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202775+0000", 2024-09-14T22:37:51.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010529199999999999 2024-09-14T22:37:51.831 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.831 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.831 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.832 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97220 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1008640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:51.832 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:37:51.832 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 41.676579314999998, 2024-09-14T22:37:51.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12435937, 2024-09-14T22:37:51.832 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.832 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.832 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.832 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.832 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.832 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.832 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97220 2024-09-14T22:37:51.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.833 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:37:51.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:37:51.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.834 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078707+0000", 2024-09-14T22:37:51.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.266e-06 2024-09-14T22:37:51.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.834 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078713+0000", 2024-09-14T22:37:51.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7270000000000004e-06 2024-09-14T22:37:51.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.834 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078737+0000", 2024-09-14T22:37:51.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3640000000000001e-05 2024-09-14T22:37:51.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.835 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.835 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078751+0000", 2024-09-14T22:37:51.835 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.45e-05 2024-09-14T22:37:51.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.835 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.835 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202841+0000", 2024-09-14T22:37:51.835 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124089938 2024-09-14T22:37:51.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.835 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.835 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202944+0000", 2024-09-14T22:37:51.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010334 2024-09-14T22:37:51.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203065+0000", 2024-09-14T22:37:51.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012095900000000001 2024-09-14T22:37:51.836 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.836 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.836 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.836 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97221 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1017856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:51.836 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:37:51.836 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 41.676513552000003, 2024-09-14T22:37:51.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124574005, 2024-09-14T22:37:51.837 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.837 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.837 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.837 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.837 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.837 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.837 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97221 2024-09-14T22:37:51.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.837 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.837 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.837 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:37:51.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:37:51.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078773+0000", 2024-09-14T22:37:51.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.13e-06 2024-09-14T22:37:51.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078779+0000", 2024-09-14T22:37:51.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0260000000000004e-06 2024-09-14T22:37:51.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078797+0000", 2024-09-14T22:37:51.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.806e-05 2024-09-14T22:37:51.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078810+0000", 2024-09-14T22:37:51.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.308e-05 2024-09-14T22:37:51.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203131+0000", 2024-09-14T22:37:51.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12432077499999999 2024-09-14T22:37:51.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203227+0000", 2024-09-14T22:37:51.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6575000000000003e-05 2024-09-14T22:37:51.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203346+0000", 2024-09-14T22:37:51.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118359 2024-09-14T22:37:51.840 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.840 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.841 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.841 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97222 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:51.841 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:37:51.841 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 41.676457182999997, 2024-09-14T22:37:51.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12480110799999999, 2024-09-14T22:37:51.841 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.841 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.841 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.841 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.841 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.841 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.842 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97222 2024-09-14T22:37:51.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.842 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:37:51.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:37:51.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078829+0000", 2024-09-14T22:37:51.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.057e-06 2024-09-14T22:37:51.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078836+0000", 2024-09-14T22:37:51.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6159999999999998e-06 2024-09-14T22:37:51.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078856+0000", 2024-09-14T22:37:51.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0100000000000001e-05 2024-09-14T22:37:51.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.844 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.844 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078869+0000", 2024-09-14T22:37:51.844 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.272e-05 2024-09-14T22:37:51.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.844 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.844 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203412+0000", 2024-09-14T22:37:51.844 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12454283300000001 2024-09-14T22:37:51.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.844 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.844 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203511+0000", 2024-09-14T22:37:51.844 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9893000000000001e-05 2024-09-14T22:37:51.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203629+0000", 2024-09-14T22:37:51.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117889 2024-09-14T22:37:51.845 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.845 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.845 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.845 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97223 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1036288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:51.845 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:37:51.845 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 41.676085196000002, 2024-09-14T22:37:51.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124706279, 2024-09-14T22:37:51.846 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.846 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.846 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.846 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.846 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.846 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.846 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97223 2024-09-14T22:37:51.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.846 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:37:51.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:37:51.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079202+0000", 2024-09-14T22:37:51.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8330000000000001e-06 2024-09-14T22:37:51.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079212+0000", 2024-09-14T22:37:51.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0207000000000001e-05 2024-09-14T22:37:51.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079238+0000", 2024-09-14T22:37:51.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5626000000000002e-05 2024-09-14T22:37:51.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079256+0000", 2024-09-14T22:37:51.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8386999999999999e-05 2024-09-14T22:37:51.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203691+0000", 2024-09-14T22:37:51.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124434538 2024-09-14T22:37:51.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203786+0000", 2024-09-14T22:37:51.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5344000000000001e-05 2024-09-14T22:37:51.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203906+0000", 2024-09-14T22:37:51.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120344 2024-09-14T22:37:51.849 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.850 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.850 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.850 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97224 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1045504~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:51.850 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:37:51.850 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 41.676001376000002, 2024-09-14T22:37:51.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12513321199999999, 2024-09-14T22:37:51.850 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:51.850 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:51.850 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:51.850 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:51.850 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:51.851 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:51.851 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97224 2024-09-14T22:37:51.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.851 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:51.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:51.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:37:51.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:51.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:37:51.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:51.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.852 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:51.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082581+0000", 2024-09-14T22:37:51.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0032971530000000001 2024-09-14T22:37:51.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.852 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:51.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082592+0000", 2024-09-14T22:37:51.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1345999999999999e-05 2024-09-14T22:37:51.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.852 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:51.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082618+0000", 2024-09-14T22:37:51.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5873000000000001e-05 2024-09-14T22:37:51.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:51.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082638+0000", 2024-09-14T22:37:51.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9565999999999999e-05 2024-09-14T22:37:51.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:51.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204004+0000", 2024-09-14T22:37:51.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.121366038 2024-09-14T22:37:51.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:51.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204188+0000", 2024-09-14T22:37:51.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018427 2024-09-14T22:37:51.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:51.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:51.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:51.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204417+0000", 2024-09-14T22:37:51.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022896599999999999 2024-09-14T22:37:51.854 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.854 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.854 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.854 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:51.854 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:51.854 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:51.856 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:37:51.856 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:51.851+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:51.961 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:51.950+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:52.058 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:52.058 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:52.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.058 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:142470 3.2 3.7c019542 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:37:52.058 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:52.055929+0000", 2024-09-14T22:37:52.058 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.00020336400000000001, 2024-09-14T22:37:52.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022211299999999999, 2024-09-14T22:37:52.058 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:52.058 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:52.058 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T22:37:52.058 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:52.059 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:52.059 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:52.059 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 142470 2024-09-14T22:37:52.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.059 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:52.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.059 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:52.059 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.055929+0000", 2024-09-14T22:37:52.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.059 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:52.059 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.055929+0000", 2024-09-14T22:37:52.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:52.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.055930+0000", 2024-09-14T22:37:52.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1600000000000001e-07 2024-09-14T22:37:52.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:52.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.055933+0000", 2024-09-14T22:37:52.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.101e-06 2024-09-14T22:37:52.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:52.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.055940+0000", 2024-09-14T22:37:52.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1420000000000004e-06 2024-09-14T22:37:52.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:52.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.055945+0000", 2024-09-14T22:37:52.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2349999999999999e-06 2024-09-14T22:37:52.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:52.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.056138+0000", 2024-09-14T22:37:52.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019245800000000001 2024-09-14T22:37:52.115 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.115 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:52.115 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.116 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:142471 3.2 3.7c019542 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:37:52.116 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:52.055957+0000", 2024-09-14T22:37:52.116 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.00017523699999999999, 2024-09-14T22:37:52.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00023439899999999999, 2024-09-14T22:37:52.116 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:52.116 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:52.116 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:37:52.116 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:52.116 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:52.116 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:52.116 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 142471 2024-09-14T22:37:52.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.116 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:52.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.117 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:52.117 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.055957+0000", 2024-09-14T22:37:52.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.117 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:52.117 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.055957+0000", 2024-09-14T22:37:52.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.117 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:52.117 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.055958+0000", 2024-09-14T22:37:52.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9999999999999997e-07 2024-09-14T22:37:52.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:52.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.055961+0000", 2024-09-14T22:37:52.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3819999999999998e-06 2024-09-14T22:37:52.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:52.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.055967+0000", 2024-09-14T22:37:52.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4389999999999997e-06 2024-09-14T22:37:52.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:52.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.055972+0000", 2024-09-14T22:37:52.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4659999999999998e-06 2024-09-14T22:37:52.119 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.119 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:52.119 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.119 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:142472 3.2 3.7c019542 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:37:52.119 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:52.055979+0000", 2024-09-14T22:37:52.119 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.00015351899999999999, 2024-09-14T22:37:52.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024301300000000001, 2024-09-14T22:37:52.119 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:52.119 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:52.119 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:37:52.119 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:52.119 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:52.120 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:52.120 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 142472 2024-09-14T22:37:52.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.120 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:52.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:52.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.055979+0000", 2024-09-14T22:37:52.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:52.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.055979+0000", 2024-09-14T22:37:52.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.121 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:52.121 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.055980+0000", 2024-09-14T22:37:52.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1699999999999999e-07 2024-09-14T22:37:52.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.121 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:52.121 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.055981+0000", 2024-09-14T22:37:52.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.553e-06 2024-09-14T22:37:52.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.121 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:52.121 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.055985+0000", 2024-09-14T22:37:52.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.636e-06 2024-09-14T22:37:52.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:52.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.055988+0000", 2024-09-14T22:37:52.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9330000000000001e-06 2024-09-14T22:37:52.122 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.122 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:52.122 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.122 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:142473 3.2 3.7c019542 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:37:52.122 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:52.055991+0000", 2024-09-14T22:37:52.122 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.00014127200000000001, 2024-09-14T22:37:52.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00025980399999999998, 2024-09-14T22:37:52.123 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:52.123 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:52.123 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T22:37:52.123 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:52.123 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:52.123 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:52.123 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 142473 2024-09-14T22:37:52.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.123 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:52.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:52.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.055991+0000", 2024-09-14T22:37:52.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:52.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.055991+0000", 2024-09-14T22:37:52.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:52.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.055992+0000", 2024-09-14T22:37:52.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1799999999999999e-07 2024-09-14T22:37:52.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:52.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.055993+0000", 2024-09-14T22:37:52.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.539e-06 2024-09-14T22:37:52.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:52.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.056002+0000", 2024-09-14T22:37:52.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2820000000000002e-06 2024-09-14T22:37:52.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:52.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.056006+0000", 2024-09-14T22:37:52.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1860000000000001e-06 2024-09-14T22:37:52.126 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.126 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:52.126 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.126 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.126 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T22:37:52.126 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 4 2024-09-14T22:37:52.126 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:52.126 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:52.050+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:52.127 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:37:52.171 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:52.166+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:52.259 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:52.254+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:52.318 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:52.318 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:52.318 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:37:52.318 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:37:52.318 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:52.338 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:37:52.383 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:52.378+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:52.461 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:52.454+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:52.468 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:52.469 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:52.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:52.469 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:52.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.469 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:72252 3.2 3:442c6d72:::rbd_data.113cb4c83968.0000000000000015:head [zero 4456448~393216] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-14T22:37:52.469 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:37:52.469 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 61.765115205999997, 2024-09-14T22:37:52.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0176591500000001, 2024-09-14T22:37:52.469 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:52.469 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:52.469 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T22:37:52.470 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:52.470 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:52.470 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:52.470 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 72252 2024-09-14T22:37:52.470 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.470 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:52.470 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.470 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:52.470 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:37:52.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.470 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.470 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:52.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:37:52.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:52.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700101+0000", 2024-09-14T22:37:52.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4570000000000001e-06 2024-09-14T22:37:52.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:52.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700114+0000", 2024-09-14T22:37:52.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2799999999999999e-05 2024-09-14T22:37:52.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:52.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700146+0000", 2024-09-14T22:37:52.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2116000000000002e-05 2024-09-14T22:37:52.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:52.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700170+0000", 2024-09-14T22:37:52.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3770000000000001e-05 2024-09-14T22:37:52.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:52.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700285+0000", 2024-09-14T22:37:52.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000115228 2024-09-14T22:37:52.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op must wait for map", 2024-09-14T22:37:52.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700297+0000", 2024-09-14T22:37:52.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1681999999999999e-05 2024-09-14T22:37:52.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:52.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.703312+0000", 2024-09-14T22:37:52.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003015046 2024-09-14T22:37:52.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:37:52.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.703327+0000", 2024-09-14T22:37:52.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5481999999999998e-05 2024-09-14T22:37:52.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:52.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:51.717566+0000", 2024-09-14T22:37:52.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0142383509999999 2024-09-14T22:37:52.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.475 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:52.475 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:51.717757+0000", 2024-09-14T22:37:52.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000191218 2024-09-14T22:37:52.475 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.475 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:52.475 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.476 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:74156 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T22:37:52.476 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:37:52.476 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.950646495999997, 2024-09-14T22:37:52.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062073521, 2024-09-14T22:37:52.476 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:52.476 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:52.476 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:52.476 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:52.476 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:52.476 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:52.477 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74156 2024-09-14T22:37:52.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.477 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:52.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:52.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:37:52.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:52.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:37:52.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:52.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514569+0000", 2024-09-14T22:37:52.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0229999999999999e-06 2024-09-14T22:37:52.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:52.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514596+0000", 2024-09-14T22:37:52.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6903e-05 2024-09-14T22:37:52.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:52.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514623+0000", 2024-09-14T22:37:52.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7206e-05 2024-09-14T22:37:52.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:52.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514637+0000", 2024-09-14T22:37:52.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:37:52.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:52.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514781+0000", 2024-09-14T22:37:52.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014394899999999999 2024-09-14T22:37:52.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.480 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:52.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.515005+0000", 2024-09-14T22:37:52.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022458499999999999 2024-09-14T22:37:52.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.480 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:37:52.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.527785+0000", 2024-09-14T22:37:52.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012780112 2024-09-14T22:37:52.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:52.481 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576209+0000", 2024-09-14T22:37:52.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.048423992999999999 2024-09-14T22:37:52.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:52.481 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576346+0000", 2024-09-14T22:37:52.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000136909 2024-09-14T22:37:52.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:52.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576517+0000", 2024-09-14T22:37:52.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000170848 2024-09-14T22:37:52.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:52.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576640+0000", 2024-09-14T22:37:52.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000122953 2024-09-14T22:37:52.482 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.482 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:52.482 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.483 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82513 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 557056~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:52.483 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:37:52.483 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.046014677000002, 2024-09-14T22:37:52.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059345486000000003, 2024-09-14T22:37:52.483 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:52.483 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:52.483 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:52.483 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:52.483 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:52.483 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:52.484 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82513 2024-09-14T22:37:52.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.484 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:52.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.484 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:52.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:37:52.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.484 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:52.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:37:52.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.485 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:52.485 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419200+0000", 2024-09-14T22:37:52.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.184e-06 2024-09-14T22:37:52.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.485 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:52.485 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419207+0000", 2024-09-14T22:37:52.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2690000000000002e-06 2024-09-14T22:37:52.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.486 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:52.486 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419224+0000", 2024-09-14T22:37:52.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6623e-05 2024-09-14T22:37:52.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.486 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:52.486 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419236+0000", 2024-09-14T22:37:52.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2367e-05 2024-09-14T22:37:52.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.486 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:52.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478294+0000", 2024-09-14T22:37:52.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059057927000000003 2024-09-14T22:37:52.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:52.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478402+0000", 2024-09-14T22:37:52.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107921 2024-09-14T22:37:52.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:52.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478544+0000", 2024-09-14T22:37:52.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014219499999999999 2024-09-14T22:37:52.488 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.488 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:52.488 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.488 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82514 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 704512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:52.488 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:37:52.488 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.045955573999997, 2024-09-14T22:37:52.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059580793, 2024-09-14T22:37:52.488 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:52.489 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:52.489 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:52.489 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:52.489 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:52.489 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:52.489 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82514 2024-09-14T22:37:52.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.489 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:52.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:52.490 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:37:52.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:52.490 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:37:52.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:52.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419259+0000", 2024-09-14T22:37:52.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.353e-06 2024-09-14T22:37:52.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:52.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419276+0000", 2024-09-14T22:37:52.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7567e-05 2024-09-14T22:37:52.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:52.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419294+0000", 2024-09-14T22:37:52.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.713e-05 2024-09-14T22:37:52.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.492 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:52.492 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419306+0000", 2024-09-14T22:37:52.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.279e-05 2024-09-14T22:37:52.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.492 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:52.492 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478589+0000", 2024-09-14T22:37:52.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059282978 2024-09-14T22:37:52.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.493 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:52.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478694+0000", 2024-09-14T22:37:52.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000104561 2024-09-14T22:37:52.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.493 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:52.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478838+0000", 2024-09-14T22:37:52.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014441399999999999 2024-09-14T22:37:52.493 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.494 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:52.494 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.494 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82515 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:52.494 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:37:52.494 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.045890424, 2024-09-14T22:37:52.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059815123999999997, 2024-09-14T22:37:52.495 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:52.495 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:52.495 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:52.495 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:52.495 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:52.495 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:52.495 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82515 2024-09-14T22:37:52.495 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.495 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:52.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.495 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:52.495 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:37:52.496 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.496 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:52.496 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:37:52.496 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.496 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:52.496 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419324+0000", 2024-09-14T22:37:52.496 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2130000000000001e-06 2024-09-14T22:37:52.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.497 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:52.497 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419331+0000", 2024-09-14T22:37:52.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4730000000000003e-06 2024-09-14T22:37:52.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.497 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:52.497 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419348+0000", 2024-09-14T22:37:52.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6269999999999998e-05 2024-09-14T22:37:52.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:52.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419360+0000", 2024-09-14T22:37:52.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2243e-05 2024-09-14T22:37:52.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:52.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478884+0000", 2024-09-14T22:37:52.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059524369000000001 2024-09-14T22:37:52.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:52.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478990+0000", 2024-09-14T22:37:52.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010569 2024-09-14T22:37:52.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:52.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479138+0000", 2024-09-14T22:37:52.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000147866 2024-09-14T22:37:52.499 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.499 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:52.499 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.499 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82516 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 999424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:52.500 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:37:52.500 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.045837919, 2024-09-14T22:37:52.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061979595999999998, 2024-09-14T22:37:52.500 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:52.500 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:52.500 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:52.500 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:52.500 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:52.500 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:52.500 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82516 2024-09-14T22:37:52.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.501 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:52.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.501 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:52.501 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:37:52.501 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.501 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.501 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:52.501 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:37:52.501 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.501 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.502 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:52.502 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419376+0000", 2024-09-14T22:37:52.502 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.237e-06 2024-09-14T22:37:52.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.502 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:52.502 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419383+0000", 2024-09-14T22:37:52.502 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8170000000000001e-06 2024-09-14T22:37:52.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.502 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:52.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419400+0000", 2024-09-14T22:37:52.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6636000000000001e-05 2024-09-14T22:37:52.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.503 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:52.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419412+0000", 2024-09-14T22:37:52.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2357e-05 2024-09-14T22:37:52.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.503 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:52.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481080+0000", 2024-09-14T22:37:52.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061668032999999997 2024-09-14T22:37:52.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:52.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481193+0000", 2024-09-14T22:37:52.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112335 2024-09-14T22:37:52.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:52.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481355+0000", 2024-09-14T22:37:52.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016218100000000001 2024-09-14T22:37:52.504 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.505 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:52.505 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.505 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82538 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1097728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:52.505 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:37:52.505 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.045785848999998, 2024-09-14T22:37:52.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059971705, 2024-09-14T22:37:52.505 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:52.505 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:52.505 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:52.505 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:52.506 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:52.506 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:52.506 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82538 2024-09-14T22:37:52.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.506 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:52.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.506 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:52.506 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:37:52.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.506 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:37:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419429+0000", 2024-09-14T22:37:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.497e-06 2024-09-14T22:37:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419436+0000", 2024-09-14T22:37:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2099999999999996e-06 2024-09-14T22:37:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419453+0000", 2024-09-14T22:37:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6840000000000001e-05 2024-09-14T22:37:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419465+0000", 2024-09-14T22:37:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2283000000000001e-05 2024-09-14T22:37:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479182+0000", 2024-09-14T22:37:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059717339000000001 2024-09-14T22:37:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479297+0000", 2024-09-14T22:37:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000114259 2024-09-14T22:37:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479399+0000", 2024-09-14T22:37:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102277 2024-09-14T22:37:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82539 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:37:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.045732999000002, 2024-09-14T22:37:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060166935999999997, 2024-09-14T22:37:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82539 2024-09-14T22:37:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:37:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:37:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419482+0000", 2024-09-14T22:37:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4130000000000001e-06 2024-09-14T22:37:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419488+0000", 2024-09-14T22:37:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8460000000000002e-06 2024-09-14T22:37:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419505+0000", 2024-09-14T22:37:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6477000000000002e-05 2024-09-14T22:37:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419518+0000", 2024-09-14T22:37:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2797e-05 2024-09-14T22:37:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479438+0000", 2024-09-14T22:37:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059919897 2024-09-14T22:37:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479547+0000", 2024-09-14T22:37:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109807 2024-09-14T22:37:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479647+0000", 2024-09-14T22:37:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9698999999999995e-05 2024-09-14T22:37:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82542 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:37:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.045680556000001, 2024-09-14T22:37:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060367430999999999, 2024-09-14T22:37:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82542 2024-09-14T22:37:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:37:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:37:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419534+0000", 2024-09-14T22:37:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T22:37:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419542+0000", 2024-09-14T22:37:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8199999999999997e-06 2024-09-14T22:37:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419559+0000", 2024-09-14T22:37:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6852999999999999e-05 2024-09-14T22:37:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419571+0000", 2024-09-14T22:37:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2364000000000001e-05 2024-09-14T22:37:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479686+0000", 2024-09-14T22:37:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060115522999999997 2024-09-14T22:37:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479795+0000", 2024-09-14T22:37:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010831399999999999 2024-09-14T22:37:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479900+0000", 2024-09-14T22:37:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105284 2024-09-14T22:37:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82544 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:37:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.045626820000003, 2024-09-14T22:37:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060590515999999997, 2024-09-14T22:37:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:52.523 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:52.523 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82544 2024-09-14T22:37:52.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.523 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:52.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:52.523 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:37:52.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:52.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:37:52.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:52.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419588+0000", 2024-09-14T22:37:52.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.35e-06 2024-09-14T22:37:52.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:52.525 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419595+0000", 2024-09-14T22:37:52.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4030000000000001e-06 2024-09-14T22:37:52.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:52.525 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419612+0000", 2024-09-14T22:37:52.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6670000000000001e-05 2024-09-14T22:37:52.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.526 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:52.526 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419624+0000", 2024-09-14T22:37:52.526 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-05 2024-09-14T22:37:52.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.526 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:52.526 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479957+0000", 2024-09-14T22:37:52.526 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060333287999999999 2024-09-14T22:37:52.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:52.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480074+0000", 2024-09-14T22:37:52.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116443 2024-09-14T22:37:52.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:52.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480177+0000", 2024-09-14T22:37:52.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000103102 2024-09-14T22:37:52.527 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.528 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:52.528 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.528 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82546 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:52.528 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:37:52.528 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.045573900999997, 2024-09-14T22:37:52.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060795428999999998, 2024-09-14T22:37:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82546 2024-09-14T22:37:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:37:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:37:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419640+0000", 2024-09-14T22:37:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.254e-06 2024-09-14T22:37:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419647+0000", 2024-09-14T22:37:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9e-06 2024-09-14T22:37:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419664+0000", 2024-09-14T22:37:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6345999999999999e-05 2024-09-14T22:37:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419676+0000", 2024-09-14T22:37:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.224e-05 2024-09-14T22:37:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480218+0000", 2024-09-14T22:37:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060542133999999997 2024-09-14T22:37:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480336+0000", 2024-09-14T22:37:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117472 2024-09-14T22:37:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480435+0000", 2024-09-14T22:37:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9083e-05 2024-09-14T22:37:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82548 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:37:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.045521891, 2024-09-14T22:37:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061011636000000001, 2024-09-14T22:37:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82548 2024-09-14T22:37:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:37:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:37:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419693+0000", 2024-09-14T22:37:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.387e-06 2024-09-14T22:37:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419699+0000", 2024-09-14T22:37:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8730000000000001e-06 2024-09-14T22:37:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419716+0000", 2024-09-14T22:37:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6416e-05 2024-09-14T22:37:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419728+0000", 2024-09-14T22:37:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2597e-05 2024-09-14T22:37:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480476+0000", 2024-09-14T22:37:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060747989000000002 2024-09-14T22:37:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480583+0000", 2024-09-14T22:37:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010638500000000001 2024-09-14T22:37:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480703+0000", 2024-09-14T22:37:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011998900000000001 2024-09-14T22:37:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:52.539 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.539 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82550 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1982464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:52.539 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:37:52.539 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.045469138000001, 2024-09-14T22:37:52.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061295483999999997, 2024-09-14T22:37:52.539 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:52.539 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:52.539 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:52.539 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:52.539 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:52.540 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:52.540 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82550 2024-09-14T22:37:52.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.540 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:52.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:52.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:37:52.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:52.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:37:52.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:52.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419745+0000", 2024-09-14T22:37:52.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.406e-06 2024-09-14T22:37:52.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:52.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419753+0000", 2024-09-14T22:37:52.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8140000000000001e-06 2024-09-14T22:37:52.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:52.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419770+0000", 2024-09-14T22:37:52.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6719999999999999e-05 2024-09-14T22:37:52.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:52.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419782+0000", 2024-09-14T22:37:52.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2252999999999999e-05 2024-09-14T22:37:52.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:52.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480745+0000", 2024-09-14T22:37:52.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060962385000000001 2024-09-14T22:37:52.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:52.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480900+0000", 2024-09-14T22:37:52.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015513899999999999 2024-09-14T22:37:52.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:52.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481039+0000", 2024-09-14T22:37:52.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013976700000000001 2024-09-14T22:37:52.543 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.544 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:52.544 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.544 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82588 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1064960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:52.544 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:37:52.544 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.045415247999998, 2024-09-14T22:37:52.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061863732999999997, 2024-09-14T22:37:52.544 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:52.544 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:52.544 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:52.544 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:52.545 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:52.545 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:52.545 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82588 2024-09-14T22:37:52.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.545 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:52.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:52.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:37:52.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:52.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:37:52.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:52.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419799+0000", 2024-09-14T22:37:52.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4160000000000001e-06 2024-09-14T22:37:52.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:52.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419807+0000", 2024-09-14T22:37:52.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2169999999999994e-06 2024-09-14T22:37:52.547 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.547 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:52.547 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419824+0000", 2024-09-14T22:37:52.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6776e-05 2024-09-14T22:37:52.547 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.547 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:52.547 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419837+0000", 2024-09-14T22:37:52.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2680000000000001e-05 2024-09-14T22:37:52.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.548 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.548 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:52.548 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481395+0000", 2024-09-14T22:37:52.548 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061558416999999997 2024-09-14T22:37:52.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.548 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.548 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:52.548 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481532+0000", 2024-09-14T22:37:52.548 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000136861 2024-09-14T22:37:52.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.549 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:52.549 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481662+0000", 2024-09-14T22:37:52.549 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012936600000000001 2024-09-14T22:37:52.549 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.549 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:52.549 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.549 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82589 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:52.549 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:37:52.549 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.045360653000003, 2024-09-14T22:37:52.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062064592000000002, 2024-09-14T22:37:52.550 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:52.550 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:52.550 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:52.550 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:52.550 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:52.550 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:52.550 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82589 2024-09-14T22:37:52.550 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.550 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:52.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.550 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:52.551 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:37:52.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.551 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.551 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:52.551 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:37:52.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.551 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.551 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:52.551 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419854+0000", 2024-09-14T22:37:52.552 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.367e-06 2024-09-14T22:37:52.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.552 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:52.552 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419862+0000", 2024-09-14T22:37:52.552 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8930000000000005e-06 2024-09-14T22:37:52.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.552 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:52.552 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419896+0000", 2024-09-14T22:37:52.552 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4134000000000002e-05 2024-09-14T22:37:52.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.553 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.553 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:52.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419909+0000", 2024-09-14T22:37:52.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3332999999999999e-05 2024-09-14T22:37:52.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.553 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.553 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:52.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481702+0000", 2024-09-14T22:37:52.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061792947000000001 2024-09-14T22:37:52.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.554 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:52.554 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481808+0000", 2024-09-14T22:37:52.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105655 2024-09-14T22:37:52.554 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.554 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:52.554 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481917+0000", 2024-09-14T22:37:52.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109263 2024-09-14T22:37:52.554 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.554 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:52.554 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.555 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.555 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82590 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1359872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:52.555 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:37:52.555 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.045287045999999, 2024-09-14T22:37:52.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062362895000000002, 2024-09-14T22:37:52.555 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:52.555 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:52.555 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:52.555 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:52.555 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:52.556 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:52.556 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82590 2024-09-14T22:37:52.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.556 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:52.556 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.556 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:52.556 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:37:52.556 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.556 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.556 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:52.556 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:37:52.557 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.557 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.557 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:52.557 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419928+0000", 2024-09-14T22:37:52.557 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4470000000000001e-06 2024-09-14T22:37:52.557 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.557 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:52.557 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419942+0000", 2024-09-14T22:37:52.557 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4229e-05 2024-09-14T22:37:52.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.558 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:52.558 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419972+0000", 2024-09-14T22:37:52.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.065e-05 2024-09-14T22:37:52.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.558 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:52.558 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419985+0000", 2024-09-14T22:37:52.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2903e-05 2024-09-14T22:37:52.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.559 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:52.559 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481960+0000", 2024-09-14T22:37:52.559 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061974435000000001 2024-09-14T22:37:52.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.559 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.559 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:52.559 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482106+0000", 2024-09-14T22:37:52.559 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014643800000000001 2024-09-14T22:37:52.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.559 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.559 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:52.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482289+0000", 2024-09-14T22:37:52.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000182793 2024-09-14T22:37:52.560 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.560 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:52.560 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.560 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.560 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.560 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82592 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:52.561 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:37:52.561 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.045210824000002, 2024-09-14T22:37:52.561 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062631507000000003, 2024-09-14T22:37:52.561 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:52.561 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:52.561 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:52.561 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:52.561 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:52.561 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:52.561 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82592 2024-09-14T22:37:52.561 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.561 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:52.562 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.562 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:52.562 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:37:52.562 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.562 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.562 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.562 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:52.562 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:37:52.562 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.562 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.562 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.563 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:52.563 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420017+0000", 2024-09-14T22:37:52.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4744e-05 2024-09-14T22:37:52.563 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.563 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:52.563 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420036+0000", 2024-09-14T22:37:52.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8680000000000001e-05 2024-09-14T22:37:52.563 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.563 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:52.563 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420054+0000", 2024-09-14T22:37:52.564 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7799999999999999e-05 2024-09-14T22:37:52.564 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.564 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.564 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:52.564 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420067+0000", 2024-09-14T22:37:52.564 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3013000000000001e-05 2024-09-14T22:37:52.564 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.564 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.564 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:52.564 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482358+0000", 2024-09-14T22:37:52.564 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062291445000000001 2024-09-14T22:37:52.565 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.565 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.565 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:52.565 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482469+0000", 2024-09-14T22:37:52.565 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110608 2024-09-14T22:37:52.565 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.565 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.565 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:52.565 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482634+0000", 2024-09-14T22:37:52.565 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000165217 2024-09-14T22:37:52.565 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.566 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:52.566 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.566 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.566 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.566 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82594 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1654784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:52.566 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:37:52.566 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.045112848000002, 2024-09-14T22:37:52.566 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062893849000000002, 2024-09-14T22:37:52.566 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:52.566 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:52.567 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:52.567 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:52.567 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:52.567 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:52.567 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82594 2024-09-14T22:37:52.567 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.567 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:52.567 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.567 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:52.567 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:37:52.568 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.568 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.568 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.568 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:52.568 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:37:52.568 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.568 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.568 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.568 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:52.568 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420102+0000", 2024-09-14T22:37:52.568 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5e-06 2024-09-14T22:37:52.568 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.569 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.569 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:52.569 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420115+0000", 2024-09-14T22:37:52.569 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.324e-05 2024-09-14T22:37:52.569 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.569 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.569 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:52.569 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420133+0000", 2024-09-14T22:37:52.569 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8213e-05 2024-09-14T22:37:52.569 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.569 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.570 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:52.570 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420147+0000", 2024-09-14T22:37:52.570 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3284e-05 2024-09-14T22:37:52.570 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.570 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.570 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:52.570 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482688+0000", 2024-09-14T22:37:52.570 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062541425999999997 2024-09-14T22:37:52.570 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.570 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.570 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:52.570 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482794+0000", 2024-09-14T22:37:52.571 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010575500000000001 2024-09-14T22:37:52.571 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.571 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.571 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:52.571 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482994+0000", 2024-09-14T22:37:52.571 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020043099999999999 2024-09-14T22:37:52.571 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.571 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:52.571 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.571 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.571 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.571 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82596 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:52.572 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:37:52.572 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.045049548000001, 2024-09-14T22:37:52.572 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063187238000000007, 2024-09-14T22:37:52.572 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:52.572 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:52.572 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:52.572 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:52.572 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:52.572 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:52.572 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82596 2024-09-14T22:37:52.572 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.572 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:52.573 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.573 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:52.573 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:37:52.573 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.573 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.573 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.573 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:52.573 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:37:52.573 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.573 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.573 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.573 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:52.574 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420165+0000", 2024-09-14T22:37:52.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4100000000000001e-06 2024-09-14T22:37:52.574 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.574 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.574 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:52.574 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420173+0000", 2024-09-14T22:37:52.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8129999999999996e-06 2024-09-14T22:37:52.574 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.574 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.574 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:52.574 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420190+0000", 2024-09-14T22:37:52.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7076999999999999e-05 2024-09-14T22:37:52.574 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.575 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.575 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:52.575 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420203+0000", 2024-09-14T22:37:52.575 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2683e-05 2024-09-14T22:37:52.575 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.575 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.575 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:52.575 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483060+0000", 2024-09-14T22:37:52.575 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062857114000000006 2024-09-14T22:37:52.575 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.575 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.575 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:52.576 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483167+0000", 2024-09-14T22:37:52.576 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010713999999999999 2024-09-14T22:37:52.576 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.576 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.576 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:52.576 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483351+0000", 2024-09-14T22:37:52.576 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018400099999999999 2024-09-14T22:37:52.576 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.576 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:52.576 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.576 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.576 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.577 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82597 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1949696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:52.577 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:37:52.577 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.044992722000003, 2024-09-14T22:37:52.577 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063466513000000002, 2024-09-14T22:37:52.577 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:52.577 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:52.577 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:52.577 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:52.577 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:52.577 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:52.577 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82597 2024-09-14T22:37:52.577 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.578 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:52.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.578 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:52.578 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:37:52.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.578 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.578 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:52.578 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:37:52.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.578 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.579 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:52.579 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420221+0000", 2024-09-14T22:37:52.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.043e-06 2024-09-14T22:37:52.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.579 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:52.579 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420229+0000", 2024-09-14T22:37:52.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0469999999999994e-06 2024-09-14T22:37:52.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.579 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:52.579 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420246+0000", 2024-09-14T22:37:52.580 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6339999999999999e-05 2024-09-14T22:37:52.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.580 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:52.580 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420258+0000", 2024-09-14T22:37:52.580 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2296000000000001e-05 2024-09-14T22:37:52.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.580 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:52.580 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483402+0000", 2024-09-14T22:37:52.580 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063144318000000005 2024-09-14T22:37:52.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483530+0000", 2024-09-14T22:37:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012713199999999999 2024-09-14T22:37:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483687+0000", 2024-09-14T22:37:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015733699999999999 2024-09-14T22:37:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.582 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.582 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:52.582 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:52.584 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:37:52.585 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-14T22:37:52.585 INFO:teuthology.orchestra.run.smithi001.stdout:5096 punch from 0x1bad59b to 0x1baf124, (0x1b89 bytes) 2024-09-14T22:37:52.585 INFO:teuthology.orchestra.run.smithi001.stdout:5097 trunc from 0x428a51a to 0x4b7319f 2024-09-14T22:37:52.585 INFO:teuthology.orchestra.run.smithi001.stdout:5098 writesame 0x628211a thru 0x6284cea data_size 0x1(0x2bd1 bytes) 2024-09-14T22:37:52.585 INFO:teuthology.orchestra.run.smithi001.stdout:5100 punch from 0x141cec0 to 0x142c3bb, (0xf4fb bytes) 2024-09-14T22:37:52.585 INFO:teuthology.orchestra.run.smithi001.stdout:5101 writesame 0xbd36f2b thru 0xbd422b1 data_size 0x1(0xb387 bytes) 2024-09-14T22:37:52.585 INFO:teuthology.orchestra.run.smithi001.stdout:5102 write 0x82ea7e5 thru 0x82ed4ed (0x2d09 bytes) 2024-09-14T22:37:52.585 INFO:teuthology.orchestra.run.smithi001.stdout:5103 read 0x8132358 thru 0x813e958 (0xc601 bytes) 2024-09-14T22:37:52.585 INFO:teuthology.orchestra.run.smithi001.stdout:5105 writesame 0x29c63d thru 0x2a8319 data_size 0x1(0xbcdd bytes) 2024-09-14T22:37:52.586 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:52.578+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:52.676 INFO:teuthology.orchestra.run.smithi001.stdout:5107 read 0x2a11f89 thru 0x2a15675 (0x36ed bytes) 2024-09-14T22:37:52.677 INFO:teuthology.orchestra.run.smithi001.stdout:5108 read 0xb5af9d4 thru 0xb5b77b6 (0x7de3 bytes) 2024-09-14T22:37:52.677 INFO:teuthology.orchestra.run.smithi001.stdout:5110 punch from 0x61213c2 to 0x612719f, (0x5ddd bytes) 2024-09-14T22:37:52.677 INFO:teuthology.orchestra.run.smithi001.stdout:5111 writesame 0x3fcca8a thru 0x3fd8dce data_size 0x13(0xc345 bytes) 2024-09-14T22:37:52.684 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:52.678+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:52.739 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T22:37:52.765 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:52.765 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:52.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.765 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:145399 3.11 3:8c573c19:::rbd_object_map.113c827b523f:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:37:52.765 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:52.760368+0000", 2024-09-14T22:37:52.765 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0032393209999999999, 2024-09-14T22:37:52.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003302714, 2024-09-14T22:37:52.766 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:52.766 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:52.766 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "waiting for sub ops", 2024-09-14T22:37:52.766 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:52.766 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:52.766 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:52.766 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 145399 2024-09-14T22:37:52.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.766 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:52.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:52.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.760368+0000", 2024-09-14T22:37:52.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:52.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.760368+0000", 2024-09-14T22:37:52.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:52.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:52.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.760371+0000", 2024-09-14T22:37:52.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.587e-06 2024-09-14T22:37:52.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:52.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.760385+0000", 2024-09-14T22:37:52.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3883000000000001e-05 2024-09-14T22:37:52.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:52.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.760419+0000", 2024-09-14T22:37:52.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3832999999999999e-05 2024-09-14T22:37:52.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:52.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.760443+0000", 2024-09-14T22:37:52.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4589000000000001e-05 2024-09-14T22:37:52.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:52.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.760544+0000", 2024-09-14T22:37:52.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000100857 2024-09-14T22:37:52.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:52.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.760658+0000", 2024-09-14T22:37:52.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000113837 2024-09-14T22:37:52.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:37:52.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.761066+0000", 2024-09-14T22:37:52.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000407704 2024-09-14T22:37:52.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:52.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:52.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:52.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:52.762411+0000", 2024-09-14T22:37:52.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0013458680000000001 2024-09-14T22:37:52.771 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.771 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:52.771 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.771 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:52.771 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T22:37:52.771 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 1 2024-09-14T22:37:52.771 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:52.772 INFO:teuthology.orchestra.run.smithi001.stdout:5112 punch from 0x3e0f475 to 0x3e190aa, (0x9c35 bytes) 2024-09-14T22:37:52.772 INFO:teuthology.orchestra.run.smithi001.stdout:5114 read 0x75c56b2 thru 0x75cfc99 (0xa5e8 bytes) 2024-09-14T22:37:52.780 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:37:52.785 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:52.782+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:52.787 INFO:teuthology.orchestra.run.smithi001.stdout:5115 trunc from 0xbd422b2 to 0x4956cad 2024-09-14T22:37:52.803 INFO:teuthology.orchestra.run.smithi001.stdout:5116 read 0x478ff97 thru 0x47903cf (0x439 bytes) 2024-09-14T22:37:52.820 INFO:teuthology.orchestra.run.smithi001.stdout:5118 trunc from 0x4956cad to 0xbd9536d 2024-09-14T22:37:52.830 INFO:teuthology.orchestra.run.smithi001.stdout:5119 read 0x9ecf93c thru 0x9ed9f16 (0xa5db bytes) 2024-09-14T22:37:52.830 INFO:teuthology.orchestra.run.smithi001.stdout:5120 write 0x8a08f6c thru 0x8a18190 (0xf225 bytes) 2024-09-14T22:37:52.838 INFO:teuthology.orchestra.run.smithi001.stdout:5122 punch from 0xcfcbae to 0xd0412c, (0x757e bytes) 2024-09-14T22:37:52.838 INFO:teuthology.orchestra.run.smithi001.stdout:5123 writesame 0xa3d1519 thru 0xa3daac6 data_size 0x77(0x95ae bytes) 2024-09-14T22:37:52.847 INFO:teuthology.orchestra.run.smithi001.stdout:5124 punch from 0x651d8b0 to 0x65280b3, (0xa803 bytes) 2024-09-14T22:37:52.851 INFO:teuthology.orchestra.run.smithi001.stdout:5125 trunc from 0xbd9536d to 0xd8a6415 2024-09-14T22:37:52.861 INFO:teuthology.orchestra.run.smithi001.stdout:5127 punch from 0x111a7cc to 0x1127746, (0xcf7a bytes) 2024-09-14T22:37:52.862 INFO:teuthology.orchestra.run.smithi001.stdout:5128 punch from 0x75ef1f6 to 0x75f2f12, (0x3d1c bytes) 2024-09-14T22:37:52.862 INFO:teuthology.orchestra.run.smithi001.stdout:5129 writesame 0x5e267c3 thru 0x5e2cd06 data_size 0x2(0x6544 bytes) 2024-09-14T22:37:52.873 INFO:teuthology.orchestra.run.smithi001.stdout:5131 read 0x829d13b thru 0x82a94e0 (0xc3a6 bytes) 2024-09-14T22:37:52.873 INFO:teuthology.orchestra.run.smithi001.stdout:5132 write 0xa785fa2 thru 0xa793b75 (0xdbd4 bytes) 2024-09-14T22:37:52.886 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:52.882+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:52.886 INFO:teuthology.orchestra.run.smithi001.stdout:5133 trunc from 0xd8a6415 to 0x6a460de 2024-09-14T22:37:52.904 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:52.915 INFO:teuthology.orchestra.run.smithi001.stdout:5135 trunc from 0x6a460de to 0xe649ea0 2024-09-14T22:37:52.923 INFO:teuthology.orchestra.run.smithi001.stdout:5138 write 0x95d896c thru 0x95e66f8 (0xdd8d bytes) 2024-09-14T22:37:52.926 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-14T22:37:52.926 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:37:52.926 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-14T22:37:52.927 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 3.0 2024-09-14T22:37:52.945 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:52.946 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:52.946 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:37:52.946 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:37:52.946 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:52.947 INFO:teuthology.orchestra.run.smithi001.stdout:5139 trunc from 0xe649ea0 to 0x3bdc43a 2024-09-14T22:37:52.963 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:37:52.967 INFO:teuthology.orchestra.run.smithi001.stdout:5140 read 0x2835b54 thru 0x2842920 (0xcdcd bytes) 2024-09-14T22:37:52.972 INFO:teuthology.orchestra.run.smithi001.stdout:5142 trunc from 0x3bdc43a to 0x64adc30 2024-09-14T22:37:52.979 INFO:teuthology.orchestra.run.smithi001.stdout:5143 read 0x314afb8 thru 0x314d20d (0x2256 bytes) 2024-09-14T22:37:52.992 INFO:teuthology.orchestra.run.smithi001.stdout:5144 write 0xaa5a940 thru 0xaa5c779 (0x1e3a bytes) 2024-09-14T22:37:52.997 INFO:teuthology.orchestra.run.smithi001.stdout:5145 writesame 0x8da9c4d thru 0x8daaec0 data_size 0x2(0x1274 bytes) 2024-09-14T22:37:53.002 INFO:teuthology.orchestra.run.smithi001.stdout:5146 read 0x786b8ec thru 0x78731e2 (0x78f7 bytes) 2024-09-14T22:37:53.002 INFO:teuthology.orchestra.run.smithi001.stdout:5148 read 0x5a09ccf thru 0x5a13bde (0x9f10 bytes) 2024-09-14T22:37:53.002 INFO:teuthology.orchestra.run.smithi001.stdout:5150 writesame 0x8ccad3c thru 0x8ccce0d data_size 0x1(0x20d2 bytes) 2024-09-14T22:37:53.007 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:53.002+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:53.008 INFO:teuthology.orchestra.run.smithi001.stdout:5151 read 0x6a1dc97 thru 0x6a2524b (0x75b5 bytes) 2024-09-14T22:37:53.008 INFO:teuthology.orchestra.run.smithi001.stdout:5152 read 0x1e81b3 thru 0x1f4c1e (0xca6c bytes) 2024-09-14T22:37:53.016 INFO:teuthology.orchestra.run.smithi001.stdout:5155 write 0xbab9670 thru 0xbabc31d (0x2cae bytes) 2024-09-14T22:37:53.021 INFO:teuthology.orchestra.run.smithi001.stdout:5157 read 0x694e202 thru 0x6956858 (0x8657 bytes) 2024-09-14T22:37:53.022 INFO:teuthology.orchestra.run.smithi001.stdout:5158 read 0xb8bd66e thru 0xb8ca5f3 (0xcf86 bytes) 2024-09-14T22:37:53.022 INFO:teuthology.orchestra.run.smithi001.stdout:5159 read 0x8ecfeb5 thru 0x8ede404 (0xe550 bytes) 2024-09-14T22:37:53.023 INFO:teuthology.orchestra.run.smithi001.stdout:5163 writesame 0x3d3d297 thru 0x3d4cbdc data_size 0x1(0xf946 bytes) 2024-09-14T22:37:53.057 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:53.058 INFO:teuthology.orchestra.run.smithi001.stdout:5167 read 0x6d35f52 thru 0x6d45695 (0xf744 bytes) 2024-09-14T22:37:53.068 INFO:teuthology.orchestra.run.smithi001.stdout:5168 write 0xde75882 thru 0xde7ae12 (0x5591 bytes) 2024-09-14T22:37:53.077 INFO:teuthology.orchestra.run.smithi001.stdout:5169 writesame 0x1a117ec thru 0x1a203a2 data_size 0x1(0xebb7 bytes) 2024-09-14T22:37:53.082 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:37:53.103 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:53.103 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:53.103 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:53.103 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:53.103 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.104 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:37:53.104 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:37:53.104 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.093593633, 2024-09-14T22:37:53.104 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.398029152, 2024-09-14T22:37:53.104 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:53.104 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:53.104 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:53.104 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:53.104 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:37:53.104 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:37:53.104 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 1 2024-09-14T22:37:53.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.104 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:53.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:53.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:37:53.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:53.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:37:53.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:53.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006183+0000", 2024-09-14T22:37:53.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9059999999999998e-06 2024-09-14T22:37:53.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:53.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006198+0000", 2024-09-14T22:37:53.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.523e-05 2024-09-14T22:37:53.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:53.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006219+0000", 2024-09-14T22:37:53.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1342999999999999e-05 2024-09-14T22:37:53.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:53.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006240+0000", 2024-09-14T22:37:53.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1186999999999999e-05 2024-09-14T22:37:53.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.107 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:53.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006363+0000", 2024-09-14T22:37:53.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012258200000000001 2024-09-14T22:37:53.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.107 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:37:53.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006392+0000", 2024-09-14T22:37:53.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9607e-05 2024-09-14T22:37:53.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:53.108 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401084+0000", 2024-09-14T22:37:53.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.39469142699999998 2024-09-14T22:37:53.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:53.108 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401394+0000", 2024-09-14T22:37:53.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030987799999999998 2024-09-14T22:37:53.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:37:53.108 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401860+0000", 2024-09-14T22:37:53.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00046654299999999998 2024-09-14T22:37:53.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.109 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:53.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.403386+0000", 2024-09-14T22:37:53.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001525322 2024-09-14T22:37:53.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.109 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:53.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404042+0000", 2024-09-14T22:37:53.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065676099999999996 2024-09-14T22:37:53.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.109 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:53.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404136+0000", 2024-09-14T22:37:53.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4032999999999999e-05 2024-09-14T22:37:53.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:53.110 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404209+0000", 2024-09-14T22:37:53.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2333e-05 2024-09-14T22:37:53.110 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.110 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:53.110 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.110 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104182 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14352384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:53.110 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:37:53.110 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 34.610739647000003, 2024-09-14T22:37:53.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071642496999999999, 2024-09-14T22:37:53.110 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:53.111 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:53.111 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:53.111 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:53.111 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:53.111 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:53.111 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104182 2024-09-14T22:37:53.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.111 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:53.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:53.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:37:53.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.112 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:53.112 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:37:53.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.112 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:53.112 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489035+0000", 2024-09-14T22:37:53.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3430000000000001e-06 2024-09-14T22:37:53.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.112 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:53.112 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489041+0000", 2024-09-14T22:37:53.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8139999999999997e-06 2024-09-14T22:37:53.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:53.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489059+0000", 2024-09-14T22:37:53.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8193e-05 2024-09-14T22:37:53.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:53.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489073+0000", 2024-09-14T22:37:53.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:37:53.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:53.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560434+0000", 2024-09-14T22:37:53.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071361463999999999 2024-09-14T22:37:53.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:53.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560545+0000", 2024-09-14T22:37:53.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110263 2024-09-14T22:37:53.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:53.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560676+0000", 2024-09-14T22:37:53.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013138000000000001 2024-09-14T22:37:53.114 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.114 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:53.114 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.114 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104183 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14483456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:53.115 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:37:53.115 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 34.610616327999999, 2024-09-14T22:37:53.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071781943000000001, 2024-09-14T22:37:53.115 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:53.115 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:53.115 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:53.115 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:53.115 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:53.115 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:53.115 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104183 2024-09-14T22:37:53.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.116 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:53.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:53.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:37:53.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:53.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:37:53.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.117 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:53.117 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489159+0000", 2024-09-14T22:37:53.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5629999999999999e-06 2024-09-14T22:37:53.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.117 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:53.117 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489165+0000", 2024-09-14T22:37:53.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7070000000000003e-06 2024-09-14T22:37:53.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.117 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:53.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489186+0000", 2024-09-14T22:37:53.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.088e-05 2024-09-14T22:37:53.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:53.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489201+0000", 2024-09-14T22:37:53.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5203e-05 2024-09-14T22:37:53.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:53.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560713+0000", 2024-09-14T22:37:53.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071511447000000006 2024-09-14T22:37:53.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:53.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560819+0000", 2024-09-14T22:37:53.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106579 2024-09-14T22:37:53.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:53.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560939+0000", 2024-09-14T22:37:53.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119564 2024-09-14T22:37:53.119 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.119 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:53.119 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.119 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104184 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:53.120 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:37:53.120 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 34.610551885, 2024-09-14T22:37:53.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071970107000000005, 2024-09-14T22:37:53.120 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:53.120 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:53.120 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:53.120 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:53.120 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:53.120 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:53.120 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104184 2024-09-14T22:37:53.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.120 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:53.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:53.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:37:53.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.121 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:53.121 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:37:53.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.121 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:53.121 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489223+0000", 2024-09-14T22:37:53.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4470000000000001e-06 2024-09-14T22:37:53.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:53.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489229+0000", 2024-09-14T22:37:53.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8200000000000002e-06 2024-09-14T22:37:53.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:53.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489247+0000", 2024-09-14T22:37:53.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8150000000000001e-05 2024-09-14T22:37:53.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:53.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489260+0000", 2024-09-14T22:37:53.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3570000000000001e-05 2024-09-14T22:37:53.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:53.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560979+0000", 2024-09-14T22:37:53.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071718831999999996 2024-09-14T22:37:53.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:53.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561075+0000", 2024-09-14T22:37:53.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5848999999999997e-05 2024-09-14T22:37:53.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:53.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561191+0000", 2024-09-14T22:37:53.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116439 2024-09-14T22:37:53.123 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.123 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:53.124 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.124 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104185 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14745600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:53.124 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:37:53.124 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 34.610492301999997, 2024-09-14T22:37:53.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072173750999999994, 2024-09-14T22:37:53.124 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:53.124 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:53.124 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:53.124 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:53.124 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:53.124 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:53.124 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104185 2024-09-14T22:37:53.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.125 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:53.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:53.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:37:53.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:53.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:37:53.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.126 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:53.126 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489282+0000", 2024-09-14T22:37:53.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2500000000000001e-06 2024-09-14T22:37:53.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.126 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:53.126 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489287+0000", 2024-09-14T22:37:53.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1359999999999996e-06 2024-09-14T22:37:53.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.126 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:53.126 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489304+0000", 2024-09-14T22:37:53.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7033000000000001e-05 2024-09-14T22:37:53.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:53.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489318+0000", 2024-09-14T22:37:53.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3934000000000001e-05 2024-09-14T22:37:53.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:53.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561232+0000", 2024-09-14T22:37:53.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071913759999999993 2024-09-14T22:37:53.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:53.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561340+0000", 2024-09-14T22:37:53.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107921 2024-09-14T22:37:53.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:53.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561455+0000", 2024-09-14T22:37:53.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011471700000000001 2024-09-14T22:37:53.128 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.128 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:53.128 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.128 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104186 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14876672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:53.128 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:37:53.129 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 34.610436393000001, 2024-09-14T22:37:53.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072393374999999996, 2024-09-14T22:37:53.129 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:53.129 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:53.129 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:53.129 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:53.129 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:53.129 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:53.129 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104186 2024-09-14T22:37:53.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.129 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:53.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:53.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:37:53.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:53.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:37:53.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:53.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489338+0000", 2024-09-14T22:37:53.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.48e-06 2024-09-14T22:37:53.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:53.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489366+0000", 2024-09-14T22:37:53.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7307000000000001e-05 2024-09-14T22:37:53.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:53.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489385+0000", 2024-09-14T22:37:53.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9283000000000002e-05 2024-09-14T22:37:53.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:53.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489399+0000", 2024-09-14T22:37:53.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.452e-05 2024-09-14T22:37:53.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:53.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561494+0000", 2024-09-14T22:37:53.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072094885999999997 2024-09-14T22:37:53.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:53.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561604+0000", 2024-09-14T22:37:53.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010925900000000001 2024-09-14T22:37:53.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:53.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561730+0000", 2024-09-14T22:37:53.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012663999999999999 2024-09-14T22:37:53.133 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.133 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:53.133 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.133 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104187 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15007744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:53.134 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:37:53.134 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 34.610353023999998, 2024-09-14T22:37:53.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072599143000000005, 2024-09-14T22:37:53.134 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:53.134 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:53.134 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:53.134 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:53.134 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:53.134 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:53.134 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104187 2024-09-14T22:37:53.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.134 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:53.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:53.135 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:37:53.135 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.135 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.135 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:53.135 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:37:53.135 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.135 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.135 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:53.135 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489422+0000", 2024-09-14T22:37:53.135 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5340000000000001e-06 2024-09-14T22:37:53.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:53.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489428+0000", 2024-09-14T22:37:53.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1430000000000003e-06 2024-09-14T22:37:53.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:53.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489445+0000", 2024-09-14T22:37:53.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7042999999999999e-05 2024-09-14T22:37:53.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:53.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489458+0000", 2024-09-14T22:37:53.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3474e-05 2024-09-14T22:37:53.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:53.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561771+0000", 2024-09-14T22:37:53.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072312600000000005 2024-09-14T22:37:53.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:53.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561885+0000", 2024-09-14T22:37:53.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011411799999999999 2024-09-14T22:37:53.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:53.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562019+0000", 2024-09-14T22:37:53.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013423100000000001 2024-09-14T22:37:53.138 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.138 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:53.138 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.138 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104188 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15138816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:53.138 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:37:53.138 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 34.6102931, 2024-09-14T22:37:53.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072824624000000004, 2024-09-14T22:37:53.138 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:53.139 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:53.139 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:53.139 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:53.139 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:53.139 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:53.139 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104188 2024-09-14T22:37:53.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.139 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:53.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.139 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:53.139 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:37:53.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.140 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:53.140 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:37:53.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.140 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:53.140 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489481+0000", 2024-09-14T22:37:53.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.22e-06 2024-09-14T22:37:53.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.140 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:53.140 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489487+0000", 2024-09-14T22:37:53.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6930000000000004e-06 2024-09-14T22:37:53.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:53.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489508+0000", 2024-09-14T22:37:53.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1135999999999999e-05 2024-09-14T22:37:53.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:53.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489522+0000", 2024-09-14T22:37:53.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3817e-05 2024-09-14T22:37:53.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:53.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562066+0000", 2024-09-14T22:37:53.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072543813999999998 2024-09-14T22:37:53.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:53.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562177+0000", 2024-09-14T22:37:53.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000111264 2024-09-14T22:37:53.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:53.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562305+0000", 2024-09-14T22:37:53.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012768 2024-09-14T22:37:53.142 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.142 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:53.142 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.143 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104189 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15269888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:53.143 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:37:53.143 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 34.610232650999997, 2024-09-14T22:37:53.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073061300999999995, 2024-09-14T22:37:53.143 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:53.143 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:53.143 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:53.143 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:53.143 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:53.143 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:53.143 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104189 2024-09-14T22:37:53.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.143 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:53.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:53.144 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:37:53.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:53.144 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:37:53.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:53.144 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489542+0000", 2024-09-14T22:37:53.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1659999999999999e-06 2024-09-14T22:37:53.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:53.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489561+0000", 2024-09-14T22:37:53.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.908e-05 2024-09-14T22:37:53.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:53.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489579+0000", 2024-09-14T22:37:53.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8536999999999999e-05 2024-09-14T22:37:53.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:53.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489593+0000", 2024-09-14T22:37:53.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3977e-05 2024-09-14T22:37:53.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:53.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562347+0000", 2024-09-14T22:37:53.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072754092000000006 2024-09-14T22:37:53.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:53.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562458+0000", 2024-09-14T22:37:53.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110722 2024-09-14T22:37:53.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:53.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562602+0000", 2024-09-14T22:37:53.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014372699999999999 2024-09-14T22:37:53.147 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.147 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:53.147 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.147 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104190 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:53.147 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:37:53.147 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 34.610160354999998, 2024-09-14T22:37:53.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.07326133, 2024-09-14T22:37:53.147 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:53.147 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:53.147 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:53.147 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:53.148 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:53.148 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:53.148 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104190 2024-09-14T22:37:53.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.148 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:53.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.148 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:53.148 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:37:53.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.148 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:53.148 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:37:53.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.149 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:53.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489614+0000", 2024-09-14T22:37:53.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1999999999999999e-06 2024-09-14T22:37:53.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.149 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:53.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489620+0000", 2024-09-14T22:37:53.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.524e-06 2024-09-14T22:37:53.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.149 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:53.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489637+0000", 2024-09-14T22:37:53.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7825999999999998e-05 2024-09-14T22:37:53.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:53.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489671+0000", 2024-09-14T22:37:53.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3726000000000002e-05 2024-09-14T22:37:53.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:53.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562644+0000", 2024-09-14T22:37:53.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072973089000000005 2024-09-14T22:37:53.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:53.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562762+0000", 2024-09-14T22:37:53.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011743600000000001 2024-09-14T22:37:53.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:53.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562874+0000", 2024-09-14T22:37:53.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011252900000000001 2024-09-14T22:37:53.151 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.151 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:53.151 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.152 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104191 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15532032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:53.152 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:37:53.152 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 34.610051931999998, 2024-09-14T22:37:53.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073311166999999997, 2024-09-14T22:37:53.152 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:53.152 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:53.152 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:53.152 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:53.152 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:53.152 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:53.152 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104191 2024-09-14T22:37:53.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.152 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:53.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:53.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:37:53.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:53.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:37:53.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:53.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489723+0000", 2024-09-14T22:37:53.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6330000000000001e-06 2024-09-14T22:37:53.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:53.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489730+0000", 2024-09-14T22:37:53.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2300000000000002e-06 2024-09-14T22:37:53.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:53.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489751+0000", 2024-09-14T22:37:53.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0849999999999999e-05 2024-09-14T22:37:53.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:53.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489766+0000", 2024-09-14T22:37:53.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.524e-05 2024-09-14T22:37:53.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:53.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562896+0000", 2024-09-14T22:37:53.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073129662999999998 2024-09-14T22:37:53.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:53.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562960+0000", 2024-09-14T22:37:53.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3689999999999995e-05 2024-09-14T22:37:53.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:53.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563033+0000", 2024-09-14T22:37:53.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2861000000000006e-05 2024-09-14T22:37:53.156 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.156 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:53.156 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.156 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104192 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:53.156 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:37:53.156 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 34.609956126999997, 2024-09-14T22:37:53.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073463632000000001, 2024-09-14T22:37:53.156 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:53.156 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:53.157 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:53.157 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:53.157 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:53.157 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:53.157 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104192 2024-09-14T22:37:53.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.157 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:53.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:53.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:37:53.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:53.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:37:53.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:53.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489819+0000", 2024-09-14T22:37:53.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.424e-06 2024-09-14T22:37:53.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:53.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489825+0000", 2024-09-14T22:37:53.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.02e-06 2024-09-14T22:37:53.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:53.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489844+0000", 2024-09-14T22:37:53.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9760000000000001e-05 2024-09-14T22:37:53.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:53.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489860+0000", 2024-09-14T22:37:53.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5563e-05 2024-09-14T22:37:53.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:53.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563070+0000", 2024-09-14T22:37:53.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073209724000000004 2024-09-14T22:37:53.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:53.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563175+0000", 2024-09-14T22:37:53.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105183 2024-09-14T22:37:53.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:53.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563281+0000", 2024-09-14T22:37:53.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105958 2024-09-14T22:37:53.160 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.160 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:53.160 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.161 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104193 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15794176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:53.161 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:37:53.161 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 34.609893960000001, 2024-09-14T22:37:53.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073600040000000005, 2024-09-14T22:37:53.161 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:53.161 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:53.161 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:53.161 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:53.161 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:53.161 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:53.161 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104193 2024-09-14T22:37:53.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.161 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:53.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:53.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:37:53.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:53.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:37:53.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:53.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489881+0000", 2024-09-14T22:37:53.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5460000000000001e-06 2024-09-14T22:37:53.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:53.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489886+0000", 2024-09-14T22:37:53.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1739999999999999e-06 2024-09-14T22:37:53.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:53.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489912+0000", 2024-09-14T22:37:53.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6461999999999998e-05 2024-09-14T22:37:53.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:53.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489927+0000", 2024-09-14T22:37:53.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:37:53.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:53.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563310+0000", 2024-09-14T22:37:53.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073382977000000002 2024-09-14T22:37:53.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:53.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563385+0000", 2024-09-14T22:37:53.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5218999999999999e-05 2024-09-14T22:37:53.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:53.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563479+0000", 2024-09-14T22:37:53.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4622000000000003e-05 2024-09-14T22:37:53.165 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.165 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:53.165 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.165 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104194 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15925248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:53.165 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:37:53.165 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 34.609827367999998, 2024-09-14T22:37:53.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073745434999999998, 2024-09-14T22:37:53.165 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:53.165 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:53.165 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:53.166 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:53.166 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:53.166 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:53.166 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104194 2024-09-14T22:37:53.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.166 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:53.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:53.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:37:53.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:53.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:37:53.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:53.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489947+0000", 2024-09-14T22:37:53.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.327e-06 2024-09-14T22:37:53.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:53.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489953+0000", 2024-09-14T22:37:53.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9329999999999998e-06 2024-09-14T22:37:53.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:53.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489971+0000", 2024-09-14T22:37:53.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7994000000000001e-05 2024-09-14T22:37:53.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:53.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489985+0000", 2024-09-14T22:37:53.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4226000000000001e-05 2024-09-14T22:37:53.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:53.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563510+0000", 2024-09-14T22:37:53.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073524667000000002 2024-09-14T22:37:53.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:53.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563597+0000", 2024-09-14T22:37:53.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6550000000000003e-05 2024-09-14T22:37:53.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:53.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563691+0000", 2024-09-14T22:37:53.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4738e-05 2024-09-14T22:37:53.170 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.170 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:53.170 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.170 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104195 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:53.170 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:37:53.170 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 34.609769171000003, 2024-09-14T22:37:53.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073876598000000002, 2024-09-14T22:37:53.170 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:53.170 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:53.170 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:53.170 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:53.171 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:53.171 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:53.171 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104195 2024-09-14T22:37:53.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.171 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:53.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:53.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:37:53.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:53.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:37:53.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:53.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490006+0000", 2024-09-14T22:37:53.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4699999999999999e-06 2024-09-14T22:37:53.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:53.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490011+0000", 2024-09-14T22:37:53.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.366e-06 2024-09-14T22:37:53.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:53.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490046+0000", 2024-09-14T22:37:53.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5496e-05 2024-09-14T22:37:53.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:53.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490061+0000", 2024-09-14T22:37:53.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5054e-05 2024-09-14T22:37:53.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:53.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563719+0000", 2024-09-14T22:37:53.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073657412000000005 2024-09-14T22:37:53.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:53.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563792+0000", 2024-09-14T22:37:53.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3319999999999996e-05 2024-09-14T22:37:53.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:53.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563881+0000", 2024-09-14T22:37:53.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8480000000000007e-05 2024-09-14T22:37:53.174 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.174 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:53.174 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.175 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104196 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16187392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:53.175 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:37:53.175 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 34.609692021999997, 2024-09-14T22:37:53.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073997496999999995, 2024-09-14T22:37:53.175 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:53.175 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:53.175 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:53.175 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:53.175 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:53.175 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:53.176 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104196 2024-09-14T22:37:53.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.176 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:53.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:53.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:37:53.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:53.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:37:53.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:53.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490083+0000", 2024-09-14T22:37:53.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.347e-06 2024-09-14T22:37:53.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:53.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490089+0000", 2024-09-14T22:37:53.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9630000000000003e-06 2024-09-14T22:37:53.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:53.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490107+0000", 2024-09-14T22:37:53.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8859999999999999e-05 2024-09-14T22:37:53.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:53.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490122+0000", 2024-09-14T22:37:53.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.432e-05 2024-09-14T22:37:53.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:53.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563910+0000", 2024-09-14T22:37:53.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073788753999999998 2024-09-14T22:37:53.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:53.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563988+0000", 2024-09-14T22:37:53.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7862999999999996e-05 2024-09-14T22:37:53.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:53.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564079+0000", 2024-09-14T22:37:53.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0389999999999993e-05 2024-09-14T22:37:53.179 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.180 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:53.180 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.180 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104197 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16318464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:53.180 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:37:53.180 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 34.609600393000001, 2024-09-14T22:37:53.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074103547000000006, 2024-09-14T22:37:53.180 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:53.180 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:53.180 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:53.180 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:53.180 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:53.181 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:53.181 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104197 2024-09-14T22:37:53.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.181 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:53.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:53.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:37:53.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:53.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:37:53.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:53.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490174+0000", 2024-09-14T22:37:53.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.37e-06 2024-09-14T22:37:53.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:53.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490197+0000", 2024-09-14T22:37:53.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3017e-05 2024-09-14T22:37:53.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:53.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490219+0000", 2024-09-14T22:37:53.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.181e-05 2024-09-14T22:37:53.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:53.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490240+0000", 2024-09-14T22:37:53.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0786000000000001e-05 2024-09-14T22:37:53.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:53.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564109+0000", 2024-09-14T22:37:53.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073869031000000002 2024-09-14T22:37:53.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:53.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564186+0000", 2024-09-14T22:37:53.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6823999999999995e-05 2024-09-14T22:37:53.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:53.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564276+0000", 2024-09-14T22:37:53.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0709000000000002e-05 2024-09-14T22:37:53.184 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.184 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:53.184 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.185 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104198 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16449536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:53.185 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:37:53.185 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 34.609508157, 2024-09-14T22:37:53.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074167693000000007, 2024-09-14T22:37:53.185 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:53.185 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:53.185 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:53.185 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:53.185 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:53.185 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:53.185 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104198 2024-09-14T22:37:53.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.185 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:53.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:53.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:37:53.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:53.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:37:53.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:53.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490266+0000", 2024-09-14T22:37:53.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3599999999999999e-06 2024-09-14T22:37:53.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:53.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490272+0000", 2024-09-14T22:37:53.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9599999999999997e-06 2024-09-14T22:37:53.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:53.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490291+0000", 2024-09-14T22:37:53.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8485999999999999e-05 2024-09-14T22:37:53.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:53.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490305+0000", 2024-09-14T22:37:53.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4327e-05 2024-09-14T22:37:53.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:53.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564306+0000", 2024-09-14T22:37:53.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074000745000000007 2024-09-14T22:37:53.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:53.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564367+0000", 2024-09-14T22:37:53.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1501999999999995e-05 2024-09-14T22:37:53.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:53.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564433+0000", 2024-09-14T22:37:53.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5313000000000003e-05 2024-09-14T22:37:53.189 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.189 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:53.189 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.189 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104199 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16580608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:53.189 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:37:53.189 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 34.609426603999999, 2024-09-14T22:37:53.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074231631000000006, 2024-09-14T22:37:53.190 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:53.190 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:53.190 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:53.190 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:53.190 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:53.190 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:53.190 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104199 2024-09-14T22:37:53.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.190 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:53.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:53.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:37:53.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:53.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:37:53.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:53.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490349+0000", 2024-09-14T22:37:53.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9429999999999999e-06 2024-09-14T22:37:53.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:53.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490355+0000", 2024-09-14T22:37:53.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1670000000000004e-06 2024-09-14T22:37:53.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:53.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490374+0000", 2024-09-14T22:37:53.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9613e-05 2024-09-14T22:37:53.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:53.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490405+0000", 2024-09-14T22:37:53.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0936000000000003e-05 2024-09-14T22:37:53.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:53.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564451+0000", 2024-09-14T22:37:53.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074045266999999998 2024-09-14T22:37:53.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:53.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564494+0000", 2024-09-14T22:37:53.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3133000000000002e-05 2024-09-14T22:37:53.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:53.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564578+0000", 2024-09-14T22:37:53.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4572000000000003e-05 2024-09-14T22:37:53.193 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.194 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:53.194 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.194 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104200 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:53.194 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:37:53.194 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 34.607631376000001, 2024-09-14T22:37:53.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072582435000000001, 2024-09-14T22:37:53.194 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:53.194 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:53.194 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:53.194 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:53.194 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:53.194 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:53.195 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104200 2024-09-14T22:37:53.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.195 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:53.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:53.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:37:53.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:53.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:37:53.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:53.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:53.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492144+0000", 2024-09-14T22:37:53.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3070000000000001e-06 2024-09-14T22:37:53.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:53.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492155+0000", 2024-09-14T22:37:53.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0573e-05 2024-09-14T22:37:53.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:53.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492185+0000", 2024-09-14T22:37:53.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.008e-05 2024-09-14T22:37:53.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:53.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492205+0000", 2024-09-14T22:37:53.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9882999999999999e-05 2024-09-14T22:37:53.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:53.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564609+0000", 2024-09-14T22:37:53.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072404181999999997 2024-09-14T22:37:53.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:53.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564667+0000", 2024-09-14T22:37:53.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8102e-05 2024-09-14T22:37:53.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:53.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:53.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:53.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564724+0000", 2024-09-14T22:37:53.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7308000000000001e-05 2024-09-14T22:37:53.198 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.198 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:53.198 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.199 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:53.199 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:53.199 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:53.200 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:53.126+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:53.201 INFO:teuthology.orchestra.run.smithi001.stdout:5170 read 0xe65607 thru 0xe6946b (0x3e65 bytes) 2024-09-14T22:37:53.201 INFO:teuthology.orchestra.run.smithi001.stdout:5173 write 0x921028d thru 0x921d940 (0xd6b4 bytes) 2024-09-14T22:37:53.201 INFO:teuthology.orchestra.run.smithi001.stdout:5174 write 0xcb27119 thru 0xcb31ee6 (0xadce bytes) 2024-09-14T22:37:53.201 INFO:teuthology.orchestra.run.smithi001.stdout:5175 write 0x203f69c thru 0x20403be (0xd23 bytes) 2024-09-14T22:37:53.201 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:37:53.202 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:53.194+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:53.267 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:37:53.267 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:37:53.272 INFO:teuthology.orchestra.run.smithi001.stdout:5176 writesame 0x3380b18 thru 0x3381ea2 data_size 0x1(0x138b bytes) 2024-09-14T22:37:53.288 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:37:53.288 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:37:53.301 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:53.294+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:53.308 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:37:53.308 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:37:53.309 INFO:teuthology.orchestra.run.smithi001.stdout:5177 write 0x1c5c29b thru 0x1c67b13 (0xb879 bytes) 2024-09-14T22:37:53.311 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:53.312 INFO:teuthology.orchestra.run.smithi001.stdout:5179 read 0xbed3a4f thru 0xbee1b7b (0xe12d bytes) 2024-09-14T22:37:53.312 INFO:teuthology.orchestra.run.smithi001.stdout:5180 read 0x75a5041 thru 0x75aaba0 (0x5b60 bytes) 2024-09-14T22:37:53.312 INFO:teuthology.orchestra.run.smithi001.stdout:5182 punch from 0x50f2fb6 to 0x50f5cd4, (0x2d1e bytes) 2024-09-14T22:37:53.312 INFO:teuthology.orchestra.run.smithi001.stdout:5184 write 0xbe078f5 thru 0xbe14b2c (0xd238 bytes) 2024-09-14T22:37:53.315 INFO:teuthology.orchestra.run.smithi001.stdout:5185 read 0x1982def thru 0x198d3f0 (0xa602 bytes) 2024-09-14T22:37:53.316 INFO:teuthology.orchestra.run.smithi001.stdout:5188 read 0x6a7eaa3 thru 0x6a88f58 (0xa4b6 bytes) 2024-09-14T22:37:53.317 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:53.317 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:53.317 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:53.317 INFO:teuthology.orchestra.run.smithi001.stdout:5193 read 0x7f767e6 thru 0x7f79349 (0x2b64 bytes) 2024-09-14T22:37:53.317 INFO:teuthology.orchestra.run.smithi001.stdout:5196 read 0x2895926 thru 0x289fe47 (0xa522 bytes) 2024-09-14T22:37:53.317 INFO:teuthology.orchestra.run.smithi001.stdout:5197 write 0x6ea1fa7 thru 0x6ea527f (0x32d9 bytes) 2024-09-14T22:37:53.325 INFO:teuthology.orchestra.run.smithi001.stdout:5198 writesame 0xcefbb03 thru 0xcefbd2b data_size 0x7(0x229 bytes) 2024-09-14T22:37:53.326 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:37:53.327 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:37:53.328 INFO:teuthology.orchestra.run.smithi001.stdout:5200 write 0x927ca89 thru 0x9287b2e (0xb0a6 bytes) 2024-09-14T22:37:53.332 INFO:teuthology.orchestra.run.smithi001.stdout:5201 write 0xe498ac thru 0xe58ac7 (0xf21c bytes) 2024-09-14T22:37:53.358 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:37:53.358 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:37:53.381 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:53.381 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:53.390 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:37:53.390 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:37:53.396 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:37:53.413 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:37:53.414 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:37:53.414 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:53.410+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:53.419 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:53.419 INFO:teuthology.orchestra.run.smithi001.stdout:5203 read 0xac7285c thru 0xac777c1 (0x4f66 bytes) 2024-09-14T22:37:53.431 INFO:teuthology.orchestra.run.smithi001.stdout:5204 trunc from 0xde7ae13 to 0x5e6aa65 2024-09-14T22:37:53.432 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:37:53.433 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:37:53.451 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:53.451 INFO:teuthology.orchestra.run.smithi001.stdout:5206 punch from 0x20a864f to 0x20b7ed1, (0xf882 bytes) 2024-09-14T22:37:53.451 INFO:teuthology.orchestra.run.smithi001.stdout:5207 read 0x181c55 thru 0x18f1ea (0xd596 bytes) 2024-09-14T22:37:53.459 INFO:teuthology.orchestra.run.smithi001.stdout:5209 trunc from 0x5e6aa65 to 0x157889e 2024-09-14T22:37:53.502 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:53.498+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:53.534 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:53.534 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:53.549 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:37:53.603 INFO:teuthology.orchestra.run.smithi001.stdout:5210 write 0xdc858dc thru 0xdc896c5 (0x3dea bytes) 2024-09-14T22:37:53.604 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:53.598+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:53.611 INFO:teuthology.orchestra.run.smithi001.stdout:5211 punch from 0x803a27e to 0x803f1f3, (0x4f75 bytes) 2024-09-14T22:37:53.611 INFO:teuthology.orchestra.run.smithi001.stdout:5213 read 0xa852a23 thru 0xa85f3ba (0xc998 bytes) 2024-09-14T22:37:53.611 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:53.611 INFO:teuthology.orchestra.run.smithi001.stdout:5218 read 0x723b43 thru 0x72c27a (0x8738 bytes) 2024-09-14T22:37:53.617 INFO:teuthology.orchestra.run.smithi001.stdout:5220 trunc from 0xdc896c6 to 0xab61449 2024-09-14T22:37:53.634 INFO:teuthology.orchestra.run.smithi001.stdout:5221 write 0x3145e1e thru 0x3146d28 (0xf0b bytes) 2024-09-14T22:37:53.641 INFO:teuthology.orchestra.run.smithi001.stdout:5222 read 0x21462bb thru 0x2153171 (0xceb7 bytes) 2024-09-14T22:37:53.641 INFO:teuthology.orchestra.run.smithi001.stdout:5225 write 0x510d876 thru 0x5114868 (0x6ff3 bytes) 2024-09-14T22:37:53.649 INFO:teuthology.orchestra.run.smithi001.stdout:5226 writesame 0x4e2a76d thru 0x4e2e3d6 data_size 0xb(0x3c6a bytes) 2024-09-14T22:37:53.671 INFO:teuthology.orchestra.run.smithi001.stdout:5227 write 0xcb8c2c5 thru 0xcb93cf7 (0x7a33 bytes) 2024-09-14T22:37:53.678 INFO:teuthology.orchestra.run.smithi001.stdout:5228 write 0x34d9eeb thru 0x34e8da2 (0xeeb8 bytes) 2024-09-14T22:37:53.685 INFO:teuthology.orchestra.run.smithi001.stdout:5229 read 0x9415e74 thru 0x94234da (0xd667 bytes) 2024-09-14T22:37:53.685 INFO:teuthology.orchestra.run.smithi001.stdout:5231 write 0x4adee80 thru 0x4aeeb4d (0xfcce bytes) 2024-09-14T22:37:53.690 INFO:teuthology.orchestra.run.smithi001.stdout:5233 read 0xedf8a0 thru 0xee1ac7 (0x2228 bytes) 2024-09-14T22:37:53.691 INFO:teuthology.orchestra.run.smithi001.stdout:5235 punch from 0xc5fb8fd to 0xc60b88d, (0xff90 bytes) 2024-09-14T22:37:53.691 INFO:teuthology.orchestra.run.smithi001.stdout:5236 write 0x870bfd thru 0x8721c9 (0x15cd bytes) 2024-09-14T22:37:53.695 INFO:teuthology.orchestra.run.smithi001.stdout:5239 read 0x688c0ee thru 0x6891d2c (0x5c3f bytes) 2024-09-14T22:37:53.695 INFO:teuthology.orchestra.run.smithi001.stdout:5240 read 0xabca588 thru 0xabd117f (0x6bf8 bytes) 2024-09-14T22:37:53.695 INFO:teuthology.orchestra.run.smithi001.stdout:5241 writesame 0xc968e69 thru 0xc96b05a data_size 0x16(0x21f2 bytes) 2024-09-14T22:37:53.698 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:53.699 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:53.699 INFO:teuthology.orchestra.run.smithi001.stdout:5244 punch from 0x127985a to 0x127a587, (0xd2d bytes) 2024-09-14T22:37:53.704 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:53.698+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:53.714 INFO:teuthology.orchestra.run.smithi001.stdout:5245 trunc from 0xcb93cf8 to 0x1749d1a 2024-09-14T22:37:53.728 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:53.729 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:53.743 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:37:53.745 INFO:teuthology.orchestra.run.smithi001.stdout:5247 read 0x2d620b thru 0x2db948 (0x573e bytes) 2024-09-14T22:37:53.759 INFO:teuthology.orchestra.run.smithi001.stdout:5248 writesame 0x4380cdf thru 0x43887b0 data_size 0x2(0x7ad2 bytes) 2024-09-14T22:37:53.759 INFO:teuthology.orchestra.run.smithi001.stdout:5249 punch from 0x3a1b9b5 to 0x3a1eeff, (0x354a bytes) 2024-09-14T22:37:53.763 INFO:teuthology.orchestra.run.smithi001.stdout:5250 trunc from 0x43887b1 to 0x58735b 2024-09-14T22:37:53.799 INFO:teuthology.orchestra.run.smithi001.stdout:5251 trunc from 0x58735b to 0x9013d20 2024-09-14T22:37:53.805 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:53.802+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:53.809 INFO:teuthology.orchestra.run.smithi001.stdout:5254 read 0x3ac0bfa thru 0x3acec8c (0xe093 bytes) 2024-09-14T22:37:53.824 INFO:teuthology.orchestra.run.smithi001.stdout:5256 writesame 0xea9bd06 thru 0xeaa2917 data_size 0x3a(0x6c12 bytes) 2024-09-14T22:37:53.825 INFO:teuthology.orchestra.run.smithi001.stdout:5257 punch from 0x23d56d5 to 0x23ded4a, (0x9675 bytes) 2024-09-14T22:37:53.825 INFO:teuthology.orchestra.run.smithi001.stdout:5258 punch from 0x3a5ad4f to 0x3a5f41a, (0x46cb bytes) 2024-09-14T22:37:53.828 INFO:teuthology.orchestra.run.smithi001.stdout:5259 trunc from 0xeaa2918 to 0xb88d642 2024-09-14T22:37:53.839 INFO:teuthology.orchestra.run.smithi001.stdout:5261 read 0x4d9a5b9 thru 0x4d9aefc (0x944 bytes) 2024-09-14T22:37:53.839 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:53.839 INFO:teuthology.orchestra.run.smithi001.stdout:5264 read 0x2e87909 thru 0x2e88470 (0xb68 bytes) 2024-09-14T22:37:53.849 INFO:teuthology.orchestra.run.smithi001.stdout:5265 trunc from 0xb88d642 to 0x4d8801b 2024-09-14T22:37:53.861 INFO:teuthology.orchestra.run.smithi001.stdout:5266 write 0x788299 thru 0x78e780 (0x64e8 bytes) 2024-09-14T22:37:53.865 INFO:teuthology.orchestra.run.smithi001.stdout:5267 read 0x3a3ccd2 thru 0x3a44571 (0x78a0 bytes) 2024-09-14T22:37:53.872 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:53.872 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:53.879 INFO:teuthology.orchestra.run.smithi001.stdout:5268 writesame 0x6ed6f3f thru 0x6ed870b data_size 0x3(0x17cd bytes) 2024-09-14T22:37:53.879 INFO:teuthology.orchestra.run.smithi001.stdout:5269 write 0x5a9992b thru 0x5aa3c04 (0xa2da bytes) 2024-09-14T22:37:53.889 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:37:53.889 INFO:teuthology.orchestra.run.smithi001.stdout:5270 read 0x40f1000 thru 0x40f84d0 (0x74d1 bytes) 2024-09-14T22:37:53.913 INFO:teuthology.orchestra.run.smithi001.stdout:5271 write 0xe6076f3 thru 0xe613cc7 (0xc5d5 bytes) 2024-09-14T22:37:53.913 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:53.910+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:53.916 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2024-09-14T22:37:53.926 INFO:teuthology.orchestra.run.smithi001.stdout:moved 3 / 82 (3.65854%) 2024-09-14T22:37:53.926 INFO:teuthology.orchestra.run.smithi001.stdout:avg 10.25 2024-09-14T22:37:53.927 INFO:teuthology.orchestra.run.smithi001.stdout:stddev 2.86138 -> 2.94746 (expected baseline 2.99479) 2024-09-14T22:37:53.927 INFO:teuthology.orchestra.run.smithi001.stdout:min osd.2 with 6 -> 6 pgs (0.585366 -> 0.585366 * mean) 2024-09-14T22:37:53.927 INFO:teuthology.orchestra.run.smithi001.stdout:max osd.7 with 14 -> 15 pgs (1.36585 -> 1.46341 * mean) 2024-09-14T22:37:53.927 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:37:53.927 INFO:teuthology.orchestra.run.smithi001.stdout:oload 110 2024-09-14T22:37:53.927 INFO:teuthology.orchestra.run.smithi001.stdout:max_change 3 2024-09-14T22:37:53.927 INFO:teuthology.orchestra.run.smithi001.stdout:max_change_osds 4 2024-09-14T22:37:53.927 INFO:teuthology.orchestra.run.smithi001.stdout:average_utilization 0.0005 2024-09-14T22:37:53.927 INFO:teuthology.orchestra.run.smithi001.stdout:overload_utilization 0.0005 2024-09-14T22:37:53.927 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3 weight 1.0000 -> 0.7516 2024-09-14T22:37:53.927 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4 weight 1.0000 -> 0.7595 2024-09-14T22:37:53.927 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1 weight 1.0000 -> 0.8053 2024-09-14T22:37:53.946 INFO:teuthology.orchestra.run.smithi001.stdout:5272 trunc from 0xe613cc8 to 0xcb2219 2024-09-14T22:37:53.966 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-14T22:37:53.966 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 3.0 2024-09-14T22:37:53.967 INFO:teuthology.orchestra.run.smithi001.stdout:5273 read 0xb5b41b thru 0xb62b58 (0x773e bytes) 2024-09-14T22:37:53.967 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:53.967 INFO:teuthology.orchestra.run.smithi001.stdout:5275 read 0x2b8a11 thru 0x2bd9a2 (0x4f92 bytes) 2024-09-14T22:37:53.978 INFO:teuthology.orchestra.run.smithi001.stdout:5277 write 0x23a04dc thru 0x23a2aee (0x2613 bytes) 2024-09-14T22:37:53.985 INFO:teuthology.orchestra.run.smithi001.stdout:5280 read 0x239d4f4 thru 0x239fcb8 (0x27c5 bytes) 2024-09-14T22:37:53.992 INFO:teuthology.orchestra.run.smithi001.stdout:5281 write 0x2459a61 thru 0x245d370 (0x3910 bytes) 2024-09-14T22:37:54.000 INFO:teuthology.orchestra.run.smithi001.stdout:5282 write 0x1815e52 thru 0x1820bb8 (0xad67 bytes) 2024-09-14T22:37:54.006 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:54.002+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:54.027 INFO:teuthology.orchestra.run.smithi001.stdout:5285 write 0x7874bdf thru 0x7878aa1 (0x3ec3 bytes) 2024-09-14T22:37:54.036 INFO:teuthology.orchestra.run.smithi001.stdout:5286 punch from 0x4982401 to 0x49830d3, (0xcd2 bytes) 2024-09-14T22:37:54.036 INFO:teuthology.orchestra.run.smithi001.stdout:5287 read 0x73b4e95 thru 0x73c2c61 (0xddcd bytes) 2024-09-14T22:37:54.037 INFO:teuthology.orchestra.run.smithi001.stdout:5288 punch from 0x30db9c3 to 0x30e1a5d, (0x609a bytes) 2024-09-14T22:37:54.037 INFO:teuthology.orchestra.run.smithi001.stdout:5289 punch from 0x6367bb to 0x6436ee, (0xcf33 bytes) 2024-09-14T22:37:54.037 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:54.037 INFO:teuthology.orchestra.run.smithi001.stdout:5291 read 0x3e4f38 thru 0x3e8f12 (0x3fdb bytes) 2024-09-14T22:37:54.037 INFO:teuthology.orchestra.run.smithi001.stdout:5293 write 0x1b9a5f0 thru 0x1b9ca9f (0x24b0 bytes) 2024-09-14T22:37:54.049 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:54.049 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:54.056 INFO:teuthology.orchestra.run.smithi001.stdout:5296 write 0xc61fbed thru 0xc624d89 (0x519d bytes) 2024-09-14T22:37:54.065 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:37:54.066 INFO:teuthology.orchestra.run.smithi001.stdout:5297 read 0x621760f thru 0x6220366 (0x8d58 bytes) 2024-09-14T22:37:54.068 INFO:teuthology.orchestra.run.smithi001.stdout:5300 flatten 2024-09-14T22:37:54.093 INFO:teuthology.orchestra.run.smithi001.stdout:5301 trunc from 0xc624d8a to 0x689b18 2024-09-14T22:37:54.107 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:54.102+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:54.130 INFO:teuthology.orchestra.run.smithi001.stdout:5304 writesame 0x1f696e2 thru 0x1f6a273 data_size 0x1(0xb92 bytes) 2024-09-14T22:37:54.155 INFO:teuthology.orchestra.run.smithi001.stdout:5305 write 0xd0d8409 thru 0xd0dc600 (0x41f8 bytes) 2024-09-14T22:37:54.165 INFO:teuthology.orchestra.run.smithi001.stdout:5307 read 0x8ddc6bd thru 0x8ddd6a1 (0xfe5 bytes) 2024-09-14T22:37:54.165 INFO:teuthology.orchestra.run.smithi001.stdout:5309 punch from 0xc467249 to 0xc47251b, (0xb2d2 bytes) 2024-09-14T22:37:54.178 INFO:teuthology.orchestra.run.smithi001.stdout:5310 trunc from 0xd0dc601 to 0x4b58c66 2024-09-14T22:37:54.196 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:54.197 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:54.199 INFO:teuthology.orchestra.run.smithi001.stdout:5312 read 0x26da619 thru 0x26dd8ef (0x32d7 bytes) 2024-09-14T22:37:54.208 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:54.202+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:54.213 INFO:teuthology.orchestra.run.smithi001.stdout:5313 writesame 0x694b888 thru 0x69563f1 data_size 0x25(0xab6a bytes) 2024-09-14T22:37:54.213 INFO:teuthology.orchestra.run.smithi001.stdout:5316 read 0x1b733ed thru 0x1b7dddc (0xa9f0 bytes) 2024-09-14T22:37:54.218 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:37:54.229 INFO:teuthology.orchestra.run.smithi001.stdout:5317 write 0xb0c5947 thru 0xb0cba13 (0x60cd bytes) 2024-09-14T22:37:54.238 INFO:teuthology.orchestra.run.smithi001.stdout:5318 write 0x8c9323f thru 0x8c9dfe8 (0xadaa bytes) 2024-09-14T22:37:54.248 INFO:teuthology.orchestra.run.smithi001.stdout:5320 write 0x6edab2b thru 0x6eeaa0c (0xfee2 bytes) 2024-09-14T22:37:54.258 INFO:teuthology.orchestra.run.smithi001.stdout:5323 writesame 0x45192ef thru 0x4524555 data_size 0x1(0xb267 bytes) 2024-09-14T22:37:54.296 INFO:teuthology.orchestra.run.smithi001.stdout:5324 read 0x3573dc1 thru 0x358235f (0xe59f bytes) 2024-09-14T22:37:54.297 INFO:teuthology.orchestra.run.smithi001.stdout:5325 punch from 0x7569e88 to 0x7572b8b, (0x8d03 bytes) 2024-09-14T22:37:54.297 INFO:teuthology.orchestra.run.smithi001.stdout:5326 read 0x3f9d356 thru 0x3fa07a8 (0x3453 bytes) 2024-09-14T22:37:54.297 INFO:teuthology.orchestra.run.smithi001.stdout:5327 punch from 0x55eab25 to 0x55f0ccf, (0x61aa bytes) 2024-09-14T22:37:54.297 INFO:teuthology.orchestra.run.smithi001.stdout:5328 read 0x7316555 thru 0x7325c5c (0xf708 bytes) 2024-09-14T22:37:54.297 INFO:teuthology.orchestra.run.smithi001.stdout:5329 write 0x4ce0c75 thru 0x4ced833 (0xcbbf bytes) 2024-09-14T22:37:54.305 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:54.308 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:54.302+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:54.318 INFO:teuthology.orchestra.run.smithi001.stdout:5331 trunc from 0xb0cba14 to 0x315844a 2024-09-14T22:37:54.343 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:54.347 INFO:teuthology.orchestra.run.smithi001.stdout:5333 trunc from 0x315844a to 0x55910ec 2024-09-14T22:37:54.358 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:54.358 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:54.369 INFO:teuthology.orchestra.run.smithi001.stdout:5334 write 0x63c7978 thru 0x63d18f8 (0x9f81 bytes) 2024-09-14T22:37:54.374 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:37:54.406 INFO:teuthology.orchestra.run.smithi001.stdout:5335 write 0xe9f5eb4 thru 0xe9fdca8 (0x7df5 bytes) 2024-09-14T22:37:54.409 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:54.402+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:54.416 INFO:teuthology.orchestra.run.smithi001.stdout:5336 writesame 0xb2becd7 thru 0xb2c44ba data_size 0x19(0x57e4 bytes) 2024-09-14T22:37:54.425 INFO:teuthology.orchestra.run.smithi001.stdout:5337 writesame 0x6ae802a thru 0x6aee702 data_size 0x1(0x66d9 bytes) 2024-09-14T22:37:54.449 INFO:teuthology.orchestra.run.smithi001.stdout:5338 writesame 0x126f3f1 thru 0x127e7ed data_size 0x1(0xf3fd bytes) 2024-09-14T22:37:54.498 INFO:teuthology.orchestra.run.smithi001.stdout:5339 writesame 0xd1b96a3 thru 0xd1bc721 data_size 0xd(0x307f bytes) 2024-09-14T22:37:54.511 INFO:teuthology.orchestra.run.smithi001.stdout:5342 writesame 0xe1257b3 thru 0xe12bb7d data_size 0x1(0x63cb bytes) 2024-09-14T22:37:54.511 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:54.506+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:54.532 INFO:teuthology.orchestra.run.smithi001.stdout:5343 read 0x73c52ad thru 0x73d01b5 (0xaf09 bytes) 2024-09-14T22:37:54.532 INFO:teuthology.orchestra.run.smithi001.stdout:5344 punch from 0x68dbc96 to 0x68ddb91, (0x1efb bytes) 2024-09-14T22:37:54.532 INFO:teuthology.orchestra.run.smithi001.stdout:5345 read 0xe2bd7e6 thru 0xe2c1c21 (0x443c bytes) 2024-09-14T22:37:54.533 INFO:teuthology.orchestra.run.smithi001.stdout:5347 punch from 0xb93a9c1 to 0xb941f5e, (0x759d bytes) 2024-09-14T22:37:54.533 INFO:teuthology.orchestra.run.smithi001.stdout:5349 punch from 0x3bf0cba to 0x3bf9a09, (0x8d4f bytes) 2024-09-14T22:37:54.533 INFO:teuthology.orchestra.run.smithi001.stdout:5351 read 0xe40cd16 thru 0xe419fd9 (0xd2c4 bytes) 2024-09-14T22:37:54.539 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:54.540 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:54.550 INFO:teuthology.orchestra.run.smithi001.stdout:5352 trunc from 0xe9fdca9 to 0x78b4c97 2024-09-14T22:37:54.561 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:37:54.574 INFO:teuthology.orchestra.run.smithi001.stdout:5353 write 0x4c3c8dc thru 0x4c3f82a (0x2f4f bytes) 2024-09-14T22:37:54.581 INFO:teuthology.orchestra.run.smithi001.stdout:5354 read 0x76e677f thru 0x76eba0e (0x5290 bytes) 2024-09-14T22:37:54.601 INFO:teuthology.orchestra.run.smithi001.stdout:5355 write 0xc7cd188 thru 0xc7d9ed2 (0xcd4b bytes) 2024-09-14T22:37:54.612 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:54.606+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:54.612 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:54.613 INFO:teuthology.orchestra.run.smithi001.stdout:5357 writesame 0x86d6312 thru 0x86d6eaf data_size 0x1(0xb9e bytes) 2024-09-14T22:37:54.621 INFO:teuthology.orchestra.run.smithi001.stdout:5359 write 0x4ba2132 thru 0x4ba54e8 (0x33b7 bytes) 2024-09-14T22:37:54.629 INFO:teuthology.orchestra.run.smithi001.stdout:5360 write 0x78541b2 thru 0x786118c (0xcfdb bytes) 2024-09-14T22:37:54.638 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:54.638 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:54.653 INFO:teuthology.orchestra.run.smithi001.stdout:5366 trunc from 0xc7d9ed3 to 0x5eed4f7 2024-09-14T22:37:54.704 INFO:teuthology.orchestra.run.smithi001.stdout:5367 write 0x964bdbb thru 0x9652c22 (0x6e68 bytes) 2024-09-14T22:37:54.708 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:54.709 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:54.713 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:54.706+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:54.714 INFO:teuthology.orchestra.run.smithi001.stdout:5368 read 0x880fa02 thru 0x8819bd6 (0xa1d5 bytes) 2024-09-14T22:37:54.715 INFO:teuthology.orchestra.run.smithi001.stdout:5369 write 0x285d4f5 thru 0x286cb45 (0xf651 bytes) 2024-09-14T22:37:54.724 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:37:54.814 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:54.810+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:54.894 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:54.894 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:54.909 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:37:54.917 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:54.914+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:54.929 INFO:teuthology.orchestra.run.smithi001.stdout:moved 6 / 82 (7.31707%) 2024-09-14T22:37:54.929 INFO:teuthology.orchestra.run.smithi001.stdout:avg 10.25 2024-09-14T22:37:54.929 INFO:teuthology.orchestra.run.smithi001.stdout:stddev 2.94746 -> 3.49106 (expected baseline 2.99479) 2024-09-14T22:37:54.929 INFO:teuthology.orchestra.run.smithi001.stdout:min osd.2 with 6 -> 8 pgs (0.585366 -> 0.780488 * mean) 2024-09-14T22:37:54.929 INFO:teuthology.orchestra.run.smithi001.stdout:max osd.7 with 15 -> 16 pgs (1.46341 -> 1.56098 * mean) 2024-09-14T22:37:54.929 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:37:54.929 INFO:teuthology.orchestra.run.smithi001.stdout:oload 110 2024-09-14T22:37:54.929 INFO:teuthology.orchestra.run.smithi001.stdout:max_change 3 2024-09-14T22:37:54.929 INFO:teuthology.orchestra.run.smithi001.stdout:max_change_osds 4 2024-09-14T22:37:54.929 INFO:teuthology.orchestra.run.smithi001.stdout:average_utilization 0.0005 2024-09-14T22:37:54.929 INFO:teuthology.orchestra.run.smithi001.stdout:overload_utilization 0.0005 2024-09-14T22:37:54.930 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3 weight 0.7516 -> 0.5658 2024-09-14T22:37:54.930 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4 weight 0.7595 -> 0.5824 2024-09-14T22:37:54.930 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1 weight 0.8053 -> 0.6500 2024-09-14T22:37:54.949 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-14T22:37:54.950 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-09-14T22:37:55.041 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:55.038+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:55.057 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:55.057 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:55.073 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:37:55.118 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:55.110+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:55.218 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:55.210+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:55.237 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:55.237 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:55.251 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:37:55.319 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:55.314+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:55.385 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:55.385 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:55.399 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:37:55.443 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:55.438+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:55.455 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:37:55.522 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:55.514+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:55.562 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:55.562 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:55.578 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:37:55.612 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:37:55.613 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:37:55.629 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:37:55.629 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:37:55.649 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:37:55.649 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:37:55.665 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:37:55.665 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:37:55.667 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:55.662+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:55.684 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:37:55.684 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:37:55.705 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:37:55.705 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:37:55.718 INFO:teuthology.orchestra.run.smithi001.stdout:moved 5 / 82 (6.09756%) 2024-09-14T22:37:55.718 INFO:teuthology.orchestra.run.smithi001.stdout:avg 10.25 2024-09-14T22:37:55.718 INFO:teuthology.orchestra.run.smithi001.stdout:stddev 3.49106 -> 3.7333 (expected baseline 2.99479) 2024-09-14T22:37:55.718 INFO:teuthology.orchestra.run.smithi001.stdout:min osd.3 with 6 -> 4 pgs (0.585366 -> 0.390244 * mean) 2024-09-14T22:37:55.718 INFO:teuthology.orchestra.run.smithi001.stdout:max osd.7 with 16 -> 16 pgs (1.56098 -> 1.56098 * mean) 2024-09-14T22:37:55.718 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:37:55.718 INFO:teuthology.orchestra.run.smithi001.stdout:oload 110 2024-09-14T22:37:55.718 INFO:teuthology.orchestra.run.smithi001.stdout:max_change 0.05 2024-09-14T22:37:55.718 INFO:teuthology.orchestra.run.smithi001.stdout:max_change_osds 4 2024-09-14T22:37:55.718 INFO:teuthology.orchestra.run.smithi001.stdout:average_utilization 0.0005 2024-09-14T22:37:55.719 INFO:teuthology.orchestra.run.smithi001.stdout:overload_utilization 0.0005 2024-09-14T22:37:55.719 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3 weight 0.5658 -> 0.5158 2024-09-14T22:37:55.719 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4 weight 0.5824 -> 0.5324 2024-09-14T22:37:55.719 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1 weight 0.6500 -> 0.6000 2024-09-14T22:37:55.723 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:55.718+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:55.737 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:37:55.737 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:37:55.747 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-14T22:37:55.747 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 1.0 2024-09-14T22:37:55.747 INFO:teuthology.orchestra.run.smithi001.stdout:5370 read 0x20ade46 thru 0x20b89ab (0xab66 bytes) 2024-09-14T22:37:55.748 INFO:teuthology.orchestra.run.smithi001.stdout:5371 read 0x2ed5fdc thru 0x2ed8dec (0x2e11 bytes) 2024-09-14T22:37:55.748 INFO:teuthology.orchestra.run.smithi001.stdout:5372 read 0x76842b7 thru 0x76902bc (0xc006 bytes) 2024-09-14T22:37:55.758 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:55.758 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:55.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:55.758 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:55.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.758 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6395 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8386560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:37:55.759 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:37:55.759 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 91.821085855000007, 2024-09-14T22:37:55.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054534379000000001, 2024-09-14T22:37:55.759 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:55.759 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:55.759 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:55.759 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:55.759 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:55.759 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:55.759 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6395 2024-09-14T22:37:55.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.759 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:55.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:55.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:37:55.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:55.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:37:55.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:55.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933833+0000", 2024-09-14T22:37:55.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3859999999999999e-06 2024-09-14T22:37:55.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:55.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933838+0000", 2024-09-14T22:37:55.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8709999999999996e-06 2024-09-14T22:37:55.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:55.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933854+0000", 2024-09-14T22:37:55.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6189e-05 2024-09-14T22:37:55.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:55.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933871+0000", 2024-09-14T22:37:55.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.626e-05 2024-09-14T22:37:55.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:55.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988198+0000", 2024-09-14T22:37:55.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054327581999999999 2024-09-14T22:37:55.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:55.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988274+0000", 2024-09-14T22:37:55.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5759000000000004e-05 2024-09-14T22:37:55.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:55.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988366+0000", 2024-09-14T22:37:55.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2331999999999996e-05 2024-09-14T22:37:55.762 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.763 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:55.763 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.763 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104944 3.3 3:c3780f3b:::rbd_data.113ca561f7a9.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:55.763 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:37:55.763 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.721483982000002, 2024-09-14T22:37:55.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073041087000000005, 2024-09-14T22:37:55.763 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:55.763 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:55.763 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:55.763 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:55.763 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:55.763 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:55.763 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104944 2024-09-14T22:37:55.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.764 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:55.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.764 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:55.764 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:37:55.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.764 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:55.764 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:37:55.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.764 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:55.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033436+0000", 2024-09-14T22:37:55.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2060000000000001e-06 2024-09-14T22:37:55.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:55.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033448+0000", 2024-09-14T22:37:55.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.169e-05 2024-09-14T22:37:55.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:55.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033474+0000", 2024-09-14T22:37:55.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6400000000000001e-05 2024-09-14T22:37:55.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:55.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033489+0000", 2024-09-14T22:37:55.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5067e-05 2024-09-14T22:37:55.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:55.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033658+0000", 2024-09-14T22:37:55.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000168565 2024-09-14T22:37:55.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:55.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033793+0000", 2024-09-14T22:37:55.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013536900000000001 2024-09-14T22:37:55.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:37:55.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.047559+0000", 2024-09-14T22:37:55.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013766294 2024-09-14T22:37:55.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:55.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106157+0000", 2024-09-14T22:37:55.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058597110000000001 2024-09-14T22:37:55.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:55.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106290+0000", 2024-09-14T22:37:55.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000133244 2024-09-14T22:37:55.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:55.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106396+0000", 2024-09-14T22:37:55.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106246 2024-09-14T22:37:55.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:55.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106475+0000", 2024-09-14T22:37:55.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8895999999999998e-05 2024-09-14T22:37:55.768 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.768 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:55.768 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.768 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:111858 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 10403840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:55.768 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:37:55.769 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.931030861, 2024-09-14T22:37:55.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054398798999999998, 2024-09-14T22:37:55.769 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:55.769 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:55.769 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:55.769 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:55.769 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:55.769 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:55.769 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 111858 2024-09-14T22:37:55.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.769 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:55.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:55.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:37:55.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:55.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:37:55.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:55.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823888+0000", 2024-09-14T22:37:55.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0200000000000001e-07 2024-09-14T22:37:55.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:55.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823927+0000", 2024-09-14T22:37:55.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8940999999999999e-05 2024-09-14T22:37:55.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:55.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823940+0000", 2024-09-14T22:37:55.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3682e-05 2024-09-14T22:37:55.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:55.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823950+0000", 2024-09-14T22:37:55.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7110000000000007e-06 2024-09-14T22:37:55.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:55.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878202+0000", 2024-09-14T22:37:55.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054251969999999997 2024-09-14T22:37:55.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:55.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878249+0000", 2024-09-14T22:37:55.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6854999999999998e-05 2024-09-14T22:37:55.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:55.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878286+0000", 2024-09-14T22:37:55.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6838000000000002e-05 2024-09-14T22:37:55.772 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.773 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:55.773 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.773 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:111859 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 10452992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:55.773 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:37:55.773 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.930941742000002, 2024-09-14T22:37:55.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054435292000000003, 2024-09-14T22:37:55.773 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:55.773 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:55.773 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:55.773 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:55.774 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:55.774 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:55.774 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 111859 2024-09-14T22:37:55.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.774 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:55.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:55.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:37:55.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:55.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:37:55.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:55.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823977+0000", 2024-09-14T22:37:55.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3500000000000005e-07 2024-09-14T22:37:55.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:55.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823981+0000", 2024-09-14T22:37:55.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2050000000000004e-06 2024-09-14T22:37:55.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.776 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:55.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824007+0000", 2024-09-14T22:37:55.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.62e-05 2024-09-14T22:37:55.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.776 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:55.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824018+0000", 2024-09-14T22:37:55.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0389e-05 2024-09-14T22:37:55.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.776 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:55.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878312+0000", 2024-09-14T22:37:55.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054294754000000001 2024-09-14T22:37:55.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:55.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878350+0000", 2024-09-14T22:37:55.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7982000000000003e-05 2024-09-14T22:37:55.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:55.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878411+0000", 2024-09-14T22:37:55.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0927000000000001e-05 2024-09-14T22:37:55.777 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.777 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:55.777 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.778 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112030 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6316032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:55.778 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:37:55.778 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.9308874, 2024-09-14T22:37:55.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054597826000000002, 2024-09-14T22:37:55.778 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:55.778 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:55.778 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:55.778 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:55.778 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:55.778 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:55.778 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112030 2024-09-14T22:37:55.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.779 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:55.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:55.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:37:55.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:55.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:37:55.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:55.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824031+0000", 2024-09-14T22:37:55.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3600000000000003e-07 2024-09-14T22:37:55.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:55.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824036+0000", 2024-09-14T22:37:55.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7350000000000004e-06 2024-09-14T22:37:55.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:55.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824049+0000", 2024-09-14T22:37:55.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3329000000000001e-05 2024-09-14T22:37:55.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:55.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824057+0000", 2024-09-14T22:37:55.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2460000000000003e-06 2024-09-14T22:37:55.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:55.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878464+0000", 2024-09-14T22:37:55.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054406207999999998 2024-09-14T22:37:55.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:55.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878554+0000", 2024-09-14T22:37:55.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0199999999999997e-05 2024-09-14T22:37:55.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:55.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878628+0000", 2024-09-14T22:37:55.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4371999999999999e-05 2024-09-14T22:37:55.782 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.782 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:55.782 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.783 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112031 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6365184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:55.783 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:37:55.783 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.930849082000002, 2024-09-14T22:37:55.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054741868999999999, 2024-09-14T22:37:55.783 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:55.783 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:55.783 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:55.783 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:55.783 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:55.783 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:55.783 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112031 2024-09-14T22:37:55.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.783 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:55.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:55.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:37:55.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:55.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:37:55.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:55.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824070+0000", 2024-09-14T22:37:55.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5099999999999998e-07 2024-09-14T22:37:55.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:55.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824073+0000", 2024-09-14T22:37:55.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6940000000000001e-06 2024-09-14T22:37:55.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:55.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824084+0000", 2024-09-14T22:37:55.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0380000000000001e-05 2024-09-14T22:37:55.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:55.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824091+0000", 2024-09-14T22:37:55.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5730000000000002e-06 2024-09-14T22:37:55.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:55.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878670+0000", 2024-09-14T22:37:55.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054579176 2024-09-14T22:37:55.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:55.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878742+0000", 2024-09-14T22:37:55.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T22:37:55.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:55.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878811+0000", 2024-09-14T22:37:55.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8170999999999995e-05 2024-09-14T22:37:55.787 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.787 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:55.787 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.787 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112032 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6414336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:55.787 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:37:55.787 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.930815215999999, 2024-09-14T22:37:55.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054931916999999997, 2024-09-14T22:37:55.787 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:55.787 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:55.787 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:55.787 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:55.787 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:55.788 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:55.788 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112032 2024-09-14T22:37:55.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.788 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:55.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:55.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:37:55.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:55.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:37:55.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:55.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:37:55.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:37:55.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:55.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824107+0000", 2024-09-14T22:37:55.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5640000000000001e-06 2024-09-14T22:37:55.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:55.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824117+0000", 2024-09-14T22:37:55.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0366e-05 2024-09-14T22:37:55.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:55.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824158+0000", 2024-09-14T22:37:55.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0670000000000002e-05 2024-09-14T22:37:55.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:55.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878853+0000", 2024-09-14T22:37:55.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054694853000000002 2024-09-14T22:37:55.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:55.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878934+0000", 2024-09-14T22:37:55.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1754000000000004e-05 2024-09-14T22:37:55.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:55.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879034+0000", 2024-09-14T22:37:55.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000100115 2024-09-14T22:37:55.791 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.791 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:55.791 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.791 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112033 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6463488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:55.791 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:37:55.791 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.930746557999999, 2024-09-14T22:37:55.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054991076999999999, 2024-09-14T22:37:55.792 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:55.792 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:55.792 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:55.792 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:55.792 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:55.792 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:55.792 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112033 2024-09-14T22:37:55.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.792 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:55.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:55.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:37:55.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:55.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:37:55.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:55.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824172+0000", 2024-09-14T22:37:55.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9999999999999996e-07 2024-09-14T22:37:55.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:55.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824177+0000", 2024-09-14T22:37:55.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5310000000000003e-06 2024-09-14T22:37:55.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:55.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824188+0000", 2024-09-14T22:37:55.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1235e-05 2024-09-14T22:37:55.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:55.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824197+0000", 2024-09-14T22:37:55.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7069999999999998e-06 2024-09-14T22:37:55.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:55.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879075+0000", 2024-09-14T22:37:55.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054878749999999997 2024-09-14T22:37:55.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:55.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879118+0000", 2024-09-14T22:37:55.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2749e-05 2024-09-14T22:37:55.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:55.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879162+0000", 2024-09-14T22:37:55.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4205000000000001e-05 2024-09-14T22:37:55.796 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.796 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:55.796 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.796 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112034 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6512640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:55.796 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:37:55.796 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.930709607000001, 2024-09-14T22:37:55.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055181246000000003, 2024-09-14T22:37:55.796 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:55.796 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:55.796 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:55.797 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:55.797 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:55.797 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:55.797 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112034 2024-09-14T22:37:55.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.797 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:55.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:55.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:37:55.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:55.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:37:55.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:55.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824209+0000", 2024-09-14T22:37:55.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5499999999999998e-07 2024-09-14T22:37:55.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:55.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824212+0000", 2024-09-14T22:37:55.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6119999999999999e-06 2024-09-14T22:37:55.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:55.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824224+0000", 2024-09-14T22:37:55.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1127999999999999e-05 2024-09-14T22:37:55.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:55.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824232+0000", 2024-09-14T22:37:55.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4700000000000002e-06 2024-09-14T22:37:55.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:55.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879209+0000", 2024-09-14T22:37:55.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054977201000000003 2024-09-14T22:37:55.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:55.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879308+0000", 2024-09-14T22:37:55.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8419999999999996e-05 2024-09-14T22:37:55.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:55.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879389+0000", 2024-09-14T22:37:55.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1760000000000003e-05 2024-09-14T22:37:55.800 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.800 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:55.800 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.801 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112035 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6561792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:55.801 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:37:55.801 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.930615879000001, 2024-09-14T22:37:55.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055300522999999997, 2024-09-14T22:37:55.801 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:55.801 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:55.801 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:55.801 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:55.801 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:55.801 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:55.802 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112035 2024-09-14T22:37:55.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.802 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:55.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:55.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:37:55.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:55.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:37:55.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:55.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824303+0000", 2024-09-14T22:37:55.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5099999999999998e-07 2024-09-14T22:37:55.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:55.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824308+0000", 2024-09-14T22:37:55.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6689999999999997e-06 2024-09-14T22:37:55.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:55.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824744+0000", 2024-09-14T22:37:55.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00043601600000000001 2024-09-14T22:37:55.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:55.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824756+0000", 2024-09-14T22:37:55.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2476999999999999e-05 2024-09-14T22:37:55.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:55.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879434+0000", 2024-09-14T22:37:55.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054677630999999997 2024-09-14T22:37:55.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:55.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879518+0000", 2024-09-14T22:37:55.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4220000000000003e-05 2024-09-14T22:37:55.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:55.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879602+0000", 2024-09-14T22:37:55.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4559000000000001e-05 2024-09-14T22:37:55.805 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.805 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:55.805 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.805 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112036 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6610944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:55.805 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:37:55.805 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.930124141, 2024-09-14T22:37:55.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055036320999999999, 2024-09-14T22:37:55.806 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:55.806 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:55.806 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:55.806 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:55.806 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:55.806 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:55.806 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112036 2024-09-14T22:37:55.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.806 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:55.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:55.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:37:55.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:55.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:37:55.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:55.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824795+0000", 2024-09-14T22:37:55.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-06 2024-09-14T22:37:55.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:55.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824802+0000", 2024-09-14T22:37:55.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6680000000000004e-06 2024-09-14T22:37:55.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:55.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824814+0000", 2024-09-14T22:37:55.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2102e-05 2024-09-14T22:37:55.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:55.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824823+0000", 2024-09-14T22:37:55.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5550000000000008e-06 2024-09-14T22:37:55.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:55.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879648+0000", 2024-09-14T22:37:55.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054825167000000001 2024-09-14T22:37:55.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:55.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879747+0000", 2024-09-14T22:37:55.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8968999999999994e-05 2024-09-14T22:37:55.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:55.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879830+0000", 2024-09-14T22:37:55.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2633999999999995e-05 2024-09-14T22:37:55.810 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.810 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:55.810 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.810 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112037 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6660096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:55.810 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:37:55.810 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.930081432000001, 2024-09-14T22:37:55.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055122926000000003, 2024-09-14T22:37:55.810 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:55.810 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:55.810 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:55.811 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:55.811 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:55.811 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:55.811 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112037 2024-09-14T22:37:55.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.811 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:55.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:55.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:37:55.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:55.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:37:55.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:55.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824837+0000", 2024-09-14T22:37:55.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5499999999999998e-07 2024-09-14T22:37:55.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:55.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824840+0000", 2024-09-14T22:37:55.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0429999999999999e-06 2024-09-14T22:37:55.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:55.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824851+0000", 2024-09-14T22:37:55.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1046999999999999e-05 2024-09-14T22:37:55.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:55.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824859+0000", 2024-09-14T22:37:55.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1200000000000002e-06 2024-09-14T22:37:55.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:55.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879867+0000", 2024-09-14T22:37:55.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055007697000000001 2024-09-14T22:37:55.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:55.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879910+0000", 2024-09-14T22:37:55.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3402000000000002e-05 2024-09-14T22:37:55.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:55.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879959+0000", 2024-09-14T22:37:55.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9061999999999999e-05 2024-09-14T22:37:55.814 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.814 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:55.814 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.814 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112038 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6709248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:55.815 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:37:55.815 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.930046736000001, 2024-09-14T22:37:55.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055312828000000001, 2024-09-14T22:37:55.815 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:55.815 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:55.815 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:55.815 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:55.815 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:55.815 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:55.815 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112038 2024-09-14T22:37:55.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.815 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:55.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:55.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:37:55.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:55.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:37:55.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:55.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824872+0000", 2024-09-14T22:37:55.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T22:37:55.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:55.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824875+0000", 2024-09-14T22:37:55.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7519999999999998e-06 2024-09-14T22:37:55.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:55.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824917+0000", 2024-09-14T22:37:55.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1047999999999997e-05 2024-09-14T22:37:55.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:55.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824926+0000", 2024-09-14T22:37:55.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3749999999999992e-06 2024-09-14T22:37:55.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:55.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880006+0000", 2024-09-14T22:37:55.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055080525999999998 2024-09-14T22:37:55.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:55.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880102+0000", 2024-09-14T22:37:55.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5995999999999994e-05 2024-09-14T22:37:55.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:55.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880184+0000", 2024-09-14T22:37:55.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1478999999999997e-05 2024-09-14T22:37:55.818 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.819 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:55.819 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.819 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112039 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6758400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:55.819 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:37:55.819 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.929978066, 2024-09-14T22:37:55.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055449709999999999, 2024-09-14T22:37:55.819 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:55.819 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:55.819 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:55.819 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:55.819 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:55.819 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:55.820 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112039 2024-09-14T22:37:55.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.820 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:55.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:55.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:37:55.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:55.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:37:55.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:55.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824941+0000", 2024-09-14T22:37:55.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3699999999999999e-07 2024-09-14T22:37:55.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:55.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824946+0000", 2024-09-14T22:37:55.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0209999999999999e-06 2024-09-14T22:37:55.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:55.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824972+0000", 2024-09-14T22:37:55.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6750999999999998e-05 2024-09-14T22:37:55.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:55.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824982+0000", 2024-09-14T22:37:55.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5019999999999998e-06 2024-09-14T22:37:55.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:55.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880227+0000", 2024-09-14T22:37:55.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055245257999999998 2024-09-14T22:37:55.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:55.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880305+0000", 2024-09-14T22:37:55.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7424999999999998e-05 2024-09-14T22:37:55.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:55.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880389+0000", 2024-09-14T22:37:55.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4815999999999996e-05 2024-09-14T22:37:55.823 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.823 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:55.823 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.823 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112040 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6807552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:55.823 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:37:55.823 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.929921350000001, 2024-09-14T22:37:55.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055534615000000002, 2024-09-14T22:37:55.824 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:55.824 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:55.824 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:55.824 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:55.824 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:55.824 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:55.824 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112040 2024-09-14T22:37:55.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.824 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:55.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.824 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:55.824 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:37:55.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.825 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:55.825 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:37:55.825 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.825 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:55.825 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824997+0000", 2024-09-14T22:37:55.825 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7199999999999998e-07 2024-09-14T22:37:55.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.825 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:55.825 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825042+0000", 2024-09-14T22:37:55.825 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5065999999999998e-05 2024-09-14T22:37:55.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.826 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:55.826 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825060+0000", 2024-09-14T22:37:55.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7520000000000002e-05 2024-09-14T22:37:55.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.826 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:55.826 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825070+0000", 2024-09-14T22:37:55.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8230000000000006e-06 2024-09-14T22:37:55.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.826 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:55.826 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880427+0000", 2024-09-14T22:37:55.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055357244 2024-09-14T22:37:55.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:55.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880483+0000", 2024-09-14T22:37:55.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5807000000000003e-05 2024-09-14T22:37:55.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:55.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880531+0000", 2024-09-14T22:37:55.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8383000000000001e-05 2024-09-14T22:37:55.827 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.827 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:55.827 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.828 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112041 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6856704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:55.828 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:37:55.828 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.929832176000001, 2024-09-14T22:37:55.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055573887000000002, 2024-09-14T22:37:55.828 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:55.828 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:55.828 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:55.828 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:55.828 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:55.828 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:55.828 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112041 2024-09-14T22:37:55.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.828 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:55.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:55.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:37:55.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:55.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:37:55.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:55.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825087+0000", 2024-09-14T22:37:55.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2399999999999996e-07 2024-09-14T22:37:55.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:55.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825091+0000", 2024-09-14T22:37:55.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9729999999999999e-06 2024-09-14T22:37:55.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:55.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825101+0000", 2024-09-14T22:37:55.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.093e-05 2024-09-14T22:37:55.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:55.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825110+0000", 2024-09-14T22:37:55.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1450000000000008e-06 2024-09-14T22:37:55.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:55.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880559+0000", 2024-09-14T22:37:55.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055449684999999999 2024-09-14T22:37:55.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:55.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880601+0000", 2024-09-14T22:37:55.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1248000000000002e-05 2024-09-14T22:37:55.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:55.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880659+0000", 2024-09-14T22:37:55.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8981999999999999e-05 2024-09-14T22:37:55.832 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.832 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:55.832 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.832 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112042 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6905856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:55.832 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:37:55.832 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.929797442000002, 2024-09-14T22:37:55.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055781415000000001, 2024-09-14T22:37:55.832 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:55.832 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:55.832 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:55.832 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:55.832 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:55.833 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:55.833 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112042 2024-09-14T22:37:55.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.833 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:55.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:55.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:37:55.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:55.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:37:55.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.834 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:55.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825478+0000", 2024-09-14T22:37:55.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035775 2024-09-14T22:37:55.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.834 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:55.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825494+0000", 2024-09-14T22:37:55.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5994999999999998e-05 2024-09-14T22:37:55.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.834 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:55.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825520+0000", 2024-09-14T22:37:55.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5738999999999999e-05 2024-09-14T22:37:55.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.835 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:55.835 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825547+0000", 2024-09-14T22:37:55.835 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7667000000000001e-05 2024-09-14T22:37:55.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.835 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:55.835 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880725+0000", 2024-09-14T22:37:55.835 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055177962999999997 2024-09-14T22:37:55.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.835 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:55.835 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880816+0000", 2024-09-14T22:37:55.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0315000000000005e-05 2024-09-14T22:37:55.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:55.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880902+0000", 2024-09-14T22:37:55.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5986e-05 2024-09-14T22:37:55.836 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.836 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:55.836 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.836 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112043 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:55.836 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:37:55.836 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.929349437999999, 2024-09-14T22:37:55.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055537912000000002, 2024-09-14T22:37:55.837 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:55.837 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:55.837 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:55.837 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:55.837 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:55.837 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:55.837 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112043 2024-09-14T22:37:55.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.837 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:55.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.837 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:55.837 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:37:55.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:55.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:37:55.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:55.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825911+0000", 2024-09-14T22:37:55.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00034306899999999999 2024-09-14T22:37:55.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:55.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825918+0000", 2024-09-14T22:37:55.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.002e-06 2024-09-14T22:37:55.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:55.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825931+0000", 2024-09-14T22:37:55.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2401e-05 2024-09-14T22:37:55.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:55.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825942+0000", 2024-09-14T22:37:55.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0903e-05 2024-09-14T22:37:55.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:55.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880951+0000", 2024-09-14T22:37:55.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055009379999999997 2024-09-14T22:37:55.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:55.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881027+0000", 2024-09-14T22:37:55.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5944000000000003e-05 2024-09-14T22:37:55.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:55.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881106+0000", 2024-09-14T22:37:55.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9213000000000002e-05 2024-09-14T22:37:55.840 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.840 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:55.840 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.841 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112044 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 7004160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:55.841 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:37:55.841 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.928955769000002, 2024-09-14T22:37:55.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055265416999999997, 2024-09-14T22:37:55.841 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:55.841 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:55.841 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:55.841 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:55.841 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:55.841 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:55.841 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112044 2024-09-14T22:37:55.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.842 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:55.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:55.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:37:55.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:55.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:37:55.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:55.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826239+0000", 2024-09-14T22:37:55.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027676000000000003 2024-09-14T22:37:55.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:55.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826247+0000", 2024-09-14T22:37:55.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7440000000000007e-06 2024-09-14T22:37:55.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:55.844 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826275+0000", 2024-09-14T22:37:55.844 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8053000000000001e-05 2024-09-14T22:37:55.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.844 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:55.844 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826286+0000", 2024-09-14T22:37:55.844 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1780000000000001e-05 2024-09-14T22:37:55.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.844 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:55.844 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881137+0000", 2024-09-14T22:37:55.844 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054850500000000003 2024-09-14T22:37:55.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:55.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881181+0000", 2024-09-14T22:37:55.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4153000000000002e-05 2024-09-14T22:37:55.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:55.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881227+0000", 2024-09-14T22:37:55.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6427000000000001e-05 2024-09-14T22:37:55.845 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.845 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:55.845 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.846 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112045 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 7053312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:37:55.846 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:37:55.846 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.928593536000001, 2024-09-14T22:37:55.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055045050999999998, 2024-09-14T22:37:55.846 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:55.846 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:55.846 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:55.846 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:55.846 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:55.846 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:55.846 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112045 2024-09-14T22:37:55.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.847 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:55.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:55.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:37:55.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:55.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:37:55.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:55.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:55.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830250+0000", 2024-09-14T22:37:55.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0039258670000000004 2024-09-14T22:37:55.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:55.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830260+0000", 2024-09-14T22:37:55.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0088000000000001e-05 2024-09-14T22:37:55.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:55.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830281+0000", 2024-09-14T22:37:55.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0720000000000002e-05 2024-09-14T22:37:55.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:55.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830292+0000", 2024-09-14T22:37:55.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1455e-05 2024-09-14T22:37:55.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:55.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881255+0000", 2024-09-14T22:37:55.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.050962247000000002 2024-09-14T22:37:55.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:55.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881296+0000", 2024-09-14T22:37:55.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1557999999999998e-05 2024-09-14T22:37:55.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:55.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:55.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:55.850 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881369+0000", 2024-09-14T22:37:55.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3115999999999996e-05 2024-09-14T22:37:55.850 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.850 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:55.850 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.850 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:55.850 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:55.850 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:55.851 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:37:55.853 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:37:55.853 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:55.846+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:55.854 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:37:55.952 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T22:37:55.953 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:55.950+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:56.039 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:56.039 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:56.039 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:37:56.040 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:56.053 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:37:56.054 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:56.050+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:56.155 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:56.150+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:56.173 INFO:teuthology.orchestra.run.smithi001.stdout:moved 0 / 82 (0%) 2024-09-14T22:37:56.174 INFO:teuthology.orchestra.run.smithi001.stdout:avg 10.25 2024-09-14T22:37:56.174 INFO:teuthology.orchestra.run.smithi001.stdout:stddev 3.7333 -> 3.7333 (expected baseline 2.99479) 2024-09-14T22:37:56.174 INFO:teuthology.orchestra.run.smithi001.stdout:min osd.3 with 4 -> 4 pgs (0.390244 -> 0.390244 * mean) 2024-09-14T22:37:56.174 INFO:teuthology.orchestra.run.smithi001.stdout:max osd.7 with 16 -> 16 pgs (1.56098 -> 1.56098 * mean) 2024-09-14T22:37:56.174 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:37:56.174 INFO:teuthology.orchestra.run.smithi001.stdout:oload 1000 2024-09-14T22:37:56.174 INFO:teuthology.orchestra.run.smithi001.stdout:max_change 1 2024-09-14T22:37:56.174 INFO:teuthology.orchestra.run.smithi001.stdout:max_change_osds 4 2024-09-14T22:37:56.174 INFO:teuthology.orchestra.run.smithi001.stdout:average_utilization 0.0005 2024-09-14T22:37:56.174 INFO:teuthology.orchestra.run.smithi001.stdout:overload_utilization 0.0048 2024-09-14T22:37:56.174 INFO:teuthology.orchestra.run.smithi001.stderr:no change 2024-09-14T22:37:56.175 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:56.175 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:56.176 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:37:56.176 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:37:56.176 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:56.189 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:37:56.194 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-14T22:37:56.194 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 0.05 2024-09-14T22:37:56.279 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:56.274+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:56.357 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:56.350+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:56.364 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:56.364 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:56.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:56.364 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:56.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.364 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97197 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 796672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:56.364 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:37:56.364 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.289102274999998, 2024-09-14T22:37:56.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12434473, 2024-09-14T22:37:56.364 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:56.365 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:56.365 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:56.365 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:56.365 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:56.365 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:56.365 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97197 2024-09-14T22:37:56.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.365 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:56.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:56.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:37:56.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:56.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:37:56.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:56.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070524+0000", 2024-09-14T22:37:56.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0299999999999998e-07 2024-09-14T22:37:56.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:56.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070527+0000", 2024-09-14T22:37:56.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9569999999999998e-06 2024-09-14T22:37:56.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:56.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070535+0000", 2024-09-14T22:37:56.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7630000000000006e-06 2024-09-14T22:37:56.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:56.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070543+0000", 2024-09-14T22:37:56.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4869999999999996e-06 2024-09-14T22:37:56.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:56.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194525+0000", 2024-09-14T22:37:56.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123982096 2024-09-14T22:37:56.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:56.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194699+0000", 2024-09-14T22:37:56.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000174105 2024-09-14T22:37:56.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:56.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194868+0000", 2024-09-14T22:37:56.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016861900000000001 2024-09-14T22:37:56.368 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.368 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:56.369 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.369 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97198 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 805888~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:56.369 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:37:56.369 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.289073899000002, 2024-09-14T22:37:56.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.1248088, 2024-09-14T22:37:56.369 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:56.369 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:56.369 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:56.369 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:56.369 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:56.369 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:56.370 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97198 2024-09-14T22:37:56.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.370 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:56.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:56.370 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:37:56.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:56.370 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:37:56.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:56.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070552+0000", 2024-09-14T22:37:56.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T22:37:56.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:56.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070555+0000", 2024-09-14T22:37:56.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.853e-06 2024-09-14T22:37:56.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:56.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070565+0000", 2024-09-14T22:37:56.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.048e-05 2024-09-14T22:37:56.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:56.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070573+0000", 2024-09-14T22:37:56.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7200000000000006e-06 2024-09-14T22:37:56.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:56.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194954+0000", 2024-09-14T22:37:56.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12438089400000001 2024-09-14T22:37:56.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:56.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195145+0000", 2024-09-14T22:37:56.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019091299999999999 2024-09-14T22:37:56.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:56.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195360+0000", 2024-09-14T22:37:56.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021527300000000001 2024-09-14T22:37:56.373 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.373 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:56.373 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.373 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97199 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 815104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:56.373 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:37:56.373 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.289043302000003, 2024-09-14T22:37:56.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125251901, 2024-09-14T22:37:56.374 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:56.374 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:56.374 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:56.374 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:56.374 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:56.374 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:56.374 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97199 2024-09-14T22:37:56.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.374 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:56.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.374 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:56.374 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:37:56.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:56.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:37:56.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:56.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070583+0000", 2024-09-14T22:37:56.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6300000000000005e-07 2024-09-14T22:37:56.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:56.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070586+0000", 2024-09-14T22:37:56.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8270000000000002e-06 2024-09-14T22:37:56.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:56.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070597+0000", 2024-09-14T22:37:56.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1960000000000001e-05 2024-09-14T22:37:56.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:56.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070605+0000", 2024-09-14T22:37:56.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6459999999999993e-06 2024-09-14T22:37:56.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:56.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195432+0000", 2024-09-14T22:37:56.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124826926 2024-09-14T22:37:56.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:56.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195626+0000", 2024-09-14T22:37:56.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019357099999999999 2024-09-14T22:37:56.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:56.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195834+0000", 2024-09-14T22:37:56.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020830800000000001 2024-09-14T22:37:56.377 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.377 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:56.377 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.378 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97200 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 824320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:56.378 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:37:56.378 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.289011256000002, 2024-09-14T22:37:56.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125693477, 2024-09-14T22:37:56.378 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:56.378 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:56.378 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:56.378 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:56.378 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:56.378 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:56.378 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97200 2024-09-14T22:37:56.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.379 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:56.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.379 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:56.379 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:37:56.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.379 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:56.379 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:37:56.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.379 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:56.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070615+0000", 2024-09-14T22:37:56.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T22:37:56.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:56.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070618+0000", 2024-09-14T22:37:56.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.943e-06 2024-09-14T22:37:56.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:56.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070627+0000", 2024-09-14T22:37:56.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1099999999999992e-06 2024-09-14T22:37:56.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:56.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070634+0000", 2024-09-14T22:37:56.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:37:56.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:56.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195908+0000", 2024-09-14T22:37:56.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125273893 2024-09-14T22:37:56.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:56.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196097+0000", 2024-09-14T22:37:56.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018953899999999999 2024-09-14T22:37:56.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:56.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196308+0000", 2024-09-14T22:37:56.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021027799999999999 2024-09-14T22:37:56.382 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.382 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:56.382 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.383 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97201 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 833536~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:56.383 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:37:56.383 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.288981069000002, 2024-09-14T22:37:56.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126133826, 2024-09-14T22:37:56.383 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:56.383 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:56.383 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:56.383 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:56.383 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:56.383 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:56.383 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97201 2024-09-14T22:37:56.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.384 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:56.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:56.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:37:56.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:56.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:37:56.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:56.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074109+0000", 2024-09-14T22:37:56.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003464834 2024-09-14T22:37:56.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:56.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074134+0000", 2024-09-14T22:37:56.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5170000000000001e-05 2024-09-14T22:37:56.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:56.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074166+0000", 2024-09-14T22:37:56.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1347000000000003e-05 2024-09-14T22:37:56.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:56.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074179+0000", 2024-09-14T22:37:56.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.364e-05 2024-09-14T22:37:56.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:56.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196377+0000", 2024-09-14T22:37:56.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12219793900000001 2024-09-14T22:37:56.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:56.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196568+0000", 2024-09-14T22:37:56.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019097999999999999 2024-09-14T22:37:56.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:56.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196778+0000", 2024-09-14T22:37:56.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020991599999999999 2024-09-14T22:37:56.387 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.387 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:56.387 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.387 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97205 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 870400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:56.387 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:37:56.387 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.285311505999999, 2024-09-14T22:37:56.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12436148299999999, 2024-09-14T22:37:56.387 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:56.387 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:56.387 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:56.388 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:56.388 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:56.388 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:56.388 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97205 2024-09-14T22:37:56.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.388 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:56.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:56.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:37:56.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:56.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:37:56.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:56.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074315+0000", 2024-09-14T22:37:56.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9299999999999997e-07 2024-09-14T22:37:56.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:56.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074318+0000", 2024-09-14T22:37:56.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.19e-06 2024-09-14T22:37:56.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:56.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074327+0000", 2024-09-14T22:37:56.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8529999999999998e-06 2024-09-14T22:37:56.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:56.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074334+0000", 2024-09-14T22:37:56.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3869999999999997e-06 2024-09-14T22:37:56.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:56.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198365+0000", 2024-09-14T22:37:56.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124031222 2024-09-14T22:37:56.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:56.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198513+0000", 2024-09-14T22:37:56.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014772299999999999 2024-09-14T22:37:56.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:56.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198675+0000", 2024-09-14T22:37:56.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016241500000000001 2024-09-14T22:37:56.391 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.391 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:56.391 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.392 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97206 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 879616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:56.392 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:37:56.392 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.285280049999997, 2024-09-14T22:37:56.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124706284, 2024-09-14T22:37:56.392 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:56.392 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:56.392 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:56.392 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:56.392 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:56.392 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:56.392 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97206 2024-09-14T22:37:56.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.392 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:56.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:56.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:37:56.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:56.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:37:56.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:56.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074346+0000", 2024-09-14T22:37:56.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8999999999999996e-07 2024-09-14T22:37:56.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:56.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074349+0000", 2024-09-14T22:37:56.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.96e-06 2024-09-14T22:37:56.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:56.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074358+0000", 2024-09-14T22:37:56.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9169999999999995e-06 2024-09-14T22:37:56.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:56.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074365+0000", 2024-09-14T22:37:56.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3699999999999997e-06 2024-09-14T22:37:56.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:56.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198761+0000", 2024-09-14T22:37:56.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124395743 2024-09-14T22:37:56.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:56.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198899+0000", 2024-09-14T22:37:56.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013763499999999999 2024-09-14T22:37:56.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:56.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199052+0000", 2024-09-14T22:37:56.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015296900000000001 2024-09-14T22:37:56.395 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.395 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:56.395 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.396 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97207 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 888832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:56.396 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:37:56.396 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.285250837, 2024-09-14T22:37:56.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124988433, 2024-09-14T22:37:56.396 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:56.396 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:56.396 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:56.396 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:56.396 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:56.396 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:56.396 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97207 2024-09-14T22:37:56.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.396 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:56.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:56.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:37:56.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:56.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:37:56.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:56.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074375+0000", 2024-09-14T22:37:56.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:37:56.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:56.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074378+0000", 2024-09-14T22:37:56.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1829999999999998e-06 2024-09-14T22:37:56.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:56.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074389+0000", 2024-09-14T22:37:56.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0737000000000001e-05 2024-09-14T22:37:56.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:56.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074396+0000", 2024-09-14T22:37:56.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2559999999999996e-06 2024-09-14T22:37:56.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:56.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199139+0000", 2024-09-14T22:37:56.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12474253 2024-09-14T22:37:56.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:56.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199240+0000", 2024-09-14T22:37:56.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101221 2024-09-14T22:37:56.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:56.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199363+0000", 2024-09-14T22:37:56.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000122822 2024-09-14T22:37:56.399 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.399 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:56.399 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.400 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97208 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 898048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:56.400 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:37:56.400 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.285219533000003, 2024-09-14T22:37:56.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12524854899999999, 2024-09-14T22:37:56.400 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:56.400 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:56.400 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:56.400 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:56.400 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:56.400 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:56.400 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97208 2024-09-14T22:37:56.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.401 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:56.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:56.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:37:56.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:56.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:37:56.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:56.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:37:56.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:37:56.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:56.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074410+0000", 2024-09-14T22:37:56.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.18e-06 2024-09-14T22:37:56.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:56.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074435+0000", 2024-09-14T22:37:56.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5273e-05 2024-09-14T22:37:56.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:56.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074443+0000", 2024-09-14T22:37:56.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3000000000000002e-06 2024-09-14T22:37:56.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:56.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199433+0000", 2024-09-14T22:37:56.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124989502 2024-09-14T22:37:56.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:56.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199535+0000", 2024-09-14T22:37:56.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102557 2024-09-14T22:37:56.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:56.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199654+0000", 2024-09-14T22:37:56.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119061 2024-09-14T22:37:56.403 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.403 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:56.403 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.404 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97209 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 907264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:56.404 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:37:56.404 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.285171728000002, 2024-09-14T22:37:56.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125487354, 2024-09-14T22:37:56.404 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:56.404 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:56.404 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:56.404 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:56.404 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:56.404 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:56.404 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97209 2024-09-14T22:37:56.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.405 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:56.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:56.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:37:56.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:56.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:37:56.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:56.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:37:56.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9400000000000005e-07 2024-09-14T22:37:56.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:56.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074458+0000", 2024-09-14T22:37:56.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3299999999999999e-06 2024-09-14T22:37:56.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:56.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074467+0000", 2024-09-14T22:37:56.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.346e-06 2024-09-14T22:37:56.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:56.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074474+0000", 2024-09-14T22:37:56.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.374e-06 2024-09-14T22:37:56.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:56.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199720+0000", 2024-09-14T22:37:56.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12524561200000001 2024-09-14T22:37:56.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:56.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199820+0000", 2024-09-14T22:37:56.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010049599999999999 2024-09-14T22:37:56.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:56.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199941+0000", 2024-09-14T22:37:56.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012050200000000001 2024-09-14T22:37:56.407 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.408 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:56.408 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.408 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97210 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:56.408 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:37:56.408 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.285141734, 2024-09-14T22:37:56.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125744942, 2024-09-14T22:37:56.408 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:56.408 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:56.408 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:56.408 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:56.408 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:56.408 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:56.408 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97210 2024-09-14T22:37:56.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.409 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:56.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:56.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:37:56.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:56.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:37:56.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:56.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:37:56.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6599999999999996e-07 2024-09-14T22:37:56.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:56.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074487+0000", 2024-09-14T22:37:56.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8140000000000001e-06 2024-09-14T22:37:56.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:56.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074496+0000", 2024-09-14T22:37:56.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9330000000000007e-06 2024-09-14T22:37:56.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:56.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074503+0000", 2024-09-14T22:37:56.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4100000000000002e-06 2024-09-14T22:37:56.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:56.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200009+0000", 2024-09-14T22:37:56.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12550539899999999 2024-09-14T22:37:56.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:56.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200108+0000", 2024-09-14T22:37:56.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9182e-05 2024-09-14T22:37:56.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:56.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200229+0000", 2024-09-14T22:37:56.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120538 2024-09-14T22:37:56.412 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.412 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:56.412 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.412 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97211 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 925696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:56.412 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:37:56.412 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.285110774000003, 2024-09-14T22:37:56.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12599848299999999, 2024-09-14T22:37:56.413 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:56.413 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:56.413 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:56.413 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:56.413 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:56.413 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:56.413 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97211 2024-09-14T22:37:56.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.413 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:56.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:56.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:37:56.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:56.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:37:56.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:56.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074516+0000", 2024-09-14T22:37:56.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.42e-06 2024-09-14T22:37:56.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:56.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074521+0000", 2024-09-14T22:37:56.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7960000000000003e-06 2024-09-14T22:37:56.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:56.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074538+0000", 2024-09-14T22:37:56.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6900000000000001e-05 2024-09-14T22:37:56.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:56.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074555+0000", 2024-09-14T22:37:56.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.696e-05 2024-09-14T22:37:56.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:56.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200293+0000", 2024-09-14T22:37:56.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12573864100000001 2024-09-14T22:37:56.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:56.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200393+0000", 2024-09-14T22:37:56.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9264000000000004e-05 2024-09-14T22:37:56.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:56.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200513+0000", 2024-09-14T22:37:56.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012050200000000001 2024-09-14T22:37:56.416 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.416 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:56.416 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.417 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97212 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 934912~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:56.417 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:37:56.417 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.285036152000004, 2024-09-14T22:37:56.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12621127300000001, 2024-09-14T22:37:56.417 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:56.417 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:56.417 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:56.417 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:56.417 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:56.417 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:56.417 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97212 2024-09-14T22:37:56.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.418 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:56.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:56.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:37:56.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:56.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:37:56.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:56.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074590+0000", 2024-09-14T22:37:56.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1969999999999999e-06 2024-09-14T22:37:56.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:56.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074596+0000", 2024-09-14T22:37:56.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7629999999999997e-06 2024-09-14T22:37:56.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:56.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074614+0000", 2024-09-14T22:37:56.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7643e-05 2024-09-14T22:37:56.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:56.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074635+0000", 2024-09-14T22:37:56.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.075e-05 2024-09-14T22:37:56.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:56.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200580+0000", 2024-09-14T22:37:56.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125945482 2024-09-14T22:37:56.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:56.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200683+0000", 2024-09-14T22:37:56.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010267099999999999 2024-09-14T22:37:56.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:56.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200800+0000", 2024-09-14T22:37:56.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117767 2024-09-14T22:37:56.421 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.421 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:56.421 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.421 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97213 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 944128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:56.421 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:37:56.421 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.284970801999997, 2024-09-14T22:37:56.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126430709, 2024-09-14T22:37:56.422 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:56.422 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:56.422 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:56.422 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:56.422 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:56.422 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:56.422 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97213 2024-09-14T22:37:56.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.422 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:56.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:56.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:37:56.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:56.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:37:56.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:56.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078210+0000", 2024-09-14T22:37:56.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0035556839999999999 2024-09-14T22:37:56.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:56.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078231+0000", 2024-09-14T22:37:56.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0716e-05 2024-09-14T22:37:56.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:56.424 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078260+0000", 2024-09-14T22:37:56.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9473000000000001e-05 2024-09-14T22:37:56.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:56.424 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078279+0000", 2024-09-14T22:37:56.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8600000000000001e-05 2024-09-14T22:37:56.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:56.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200864+0000", 2024-09-14T22:37:56.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.122585295 2024-09-14T22:37:56.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:56.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200965+0000", 2024-09-14T22:37:56.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101004 2024-09-14T22:37:56.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:56.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.201085+0000", 2024-09-14T22:37:56.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119937 2024-09-14T22:37:56.426 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.426 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:56.426 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.426 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97219 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 999424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:56.426 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:37:56.426 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.280988524999998, 2024-09-14T22:37:56.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124138558, 2024-09-14T22:37:56.426 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:56.426 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:56.426 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:56.427 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:56.427 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:56.427 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:56.427 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97219 2024-09-14T22:37:56.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.427 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:56.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:56.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:37:56.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:56.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:37:56.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.428 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.428 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:56.428 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078638+0000", 2024-09-14T22:37:56.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.173e-06 2024-09-14T22:37:56.428 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.428 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:56.428 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078644+0000", 2024-09-14T22:37:56.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2570000000000004e-06 2024-09-14T22:37:56.428 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:56.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078672+0000", 2024-09-14T22:37:56.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7322999999999999e-05 2024-09-14T22:37:56.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:56.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078687+0000", 2024-09-14T22:37:56.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5639999999999999e-05 2024-09-14T22:37:56.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:56.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202567+0000", 2024-09-14T22:37:56.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123880134 2024-09-14T22:37:56.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:56.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202670+0000", 2024-09-14T22:37:56.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010273899999999999 2024-09-14T22:37:56.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:56.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202775+0000", 2024-09-14T22:37:56.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010529199999999999 2024-09-14T22:37:56.430 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.430 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:56.430 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.430 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97220 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1008640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:56.431 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:37:56.431 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.280919292, 2024-09-14T22:37:56.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12435937, 2024-09-14T22:37:56.431 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:56.431 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:56.431 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:56.431 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:56.431 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:56.431 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:56.431 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97220 2024-09-14T22:37:56.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.431 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:56.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.432 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:56.432 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:37:56.432 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.432 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.432 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.432 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:56.432 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:37:56.432 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.432 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.432 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.432 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:56.432 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078707+0000", 2024-09-14T22:37:56.432 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.266e-06 2024-09-14T22:37:56.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.433 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.433 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:56.433 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078713+0000", 2024-09-14T22:37:56.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7270000000000004e-06 2024-09-14T22:37:56.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.433 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.433 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:56.433 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078737+0000", 2024-09-14T22:37:56.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3640000000000001e-05 2024-09-14T22:37:56.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.433 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.433 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:56.433 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078751+0000", 2024-09-14T22:37:56.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.45e-05 2024-09-14T22:37:56.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:56.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202841+0000", 2024-09-14T22:37:56.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124089938 2024-09-14T22:37:56.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:56.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202944+0000", 2024-09-14T22:37:56.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010334 2024-09-14T22:37:56.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:56.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203065+0000", 2024-09-14T22:37:56.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012095900000000001 2024-09-14T22:37:56.434 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.435 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:56.435 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.435 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97221 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1017856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:56.435 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:37:56.435 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.280853528999998, 2024-09-14T22:37:56.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124574005, 2024-09-14T22:37:56.435 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:56.435 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:56.435 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:56.435 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:56.435 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:56.435 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:56.436 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97221 2024-09-14T22:37:56.436 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.436 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:56.436 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.436 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:56.436 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:37:56.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.436 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.436 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.436 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:56.436 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:37:56.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.436 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.436 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.436 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:56.437 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078773+0000", 2024-09-14T22:37:56.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.13e-06 2024-09-14T22:37:56.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.437 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.437 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:56.437 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078779+0000", 2024-09-14T22:37:56.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0260000000000004e-06 2024-09-14T22:37:56.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.437 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.437 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:56.437 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078797+0000", 2024-09-14T22:37:56.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.806e-05 2024-09-14T22:37:56.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.437 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:56.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078810+0000", 2024-09-14T22:37:56.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.308e-05 2024-09-14T22:37:56.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:56.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203131+0000", 2024-09-14T22:37:56.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12432077499999999 2024-09-14T22:37:56.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:56.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203227+0000", 2024-09-14T22:37:56.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6575000000000003e-05 2024-09-14T22:37:56.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:56.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203346+0000", 2024-09-14T22:37:56.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118359 2024-09-14T22:37:56.439 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.439 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:56.439 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.439 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97222 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:56.439 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:37:56.439 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.280797159999999, 2024-09-14T22:37:56.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12480110799999999, 2024-09-14T22:37:56.439 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:56.440 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:56.440 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:56.440 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:56.440 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:56.440 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:56.440 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97222 2024-09-14T22:37:56.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.440 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:56.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:56.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:37:56.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:56.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:37:56.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:56.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078829+0000", 2024-09-14T22:37:56.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.057e-06 2024-09-14T22:37:56.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:56.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078836+0000", 2024-09-14T22:37:56.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6159999999999998e-06 2024-09-14T22:37:56.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:56.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078856+0000", 2024-09-14T22:37:56.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0100000000000001e-05 2024-09-14T22:37:56.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:56.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078869+0000", 2024-09-14T22:37:56.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.272e-05 2024-09-14T22:37:56.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:56.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203412+0000", 2024-09-14T22:37:56.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12454283300000001 2024-09-14T22:37:56.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.443 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:56.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203511+0000", 2024-09-14T22:37:56.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9893000000000001e-05 2024-09-14T22:37:56.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.443 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:56.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203629+0000", 2024-09-14T22:37:56.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117889 2024-09-14T22:37:56.443 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.443 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:56.443 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.444 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97223 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1036288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:56.444 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:37:56.444 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.280425172999998, 2024-09-14T22:37:56.444 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124706279, 2024-09-14T22:37:56.444 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:56.444 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:56.444 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:56.444 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:56.444 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:56.444 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:56.444 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97223 2024-09-14T22:37:56.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.445 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:56.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:56.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:37:56.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:56.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:37:56.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:56.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079202+0000", 2024-09-14T22:37:56.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8330000000000001e-06 2024-09-14T22:37:56.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.446 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.446 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:56.446 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079212+0000", 2024-09-14T22:37:56.446 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0207000000000001e-05 2024-09-14T22:37:56.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.446 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.446 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:56.446 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079238+0000", 2024-09-14T22:37:56.446 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5626000000000002e-05 2024-09-14T22:37:56.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.446 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.446 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:56.446 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079256+0000", 2024-09-14T22:37:56.446 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8386999999999999e-05 2024-09-14T22:37:56.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.447 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:56.447 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203691+0000", 2024-09-14T22:37:56.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124434538 2024-09-14T22:37:56.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.447 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:56.447 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203786+0000", 2024-09-14T22:37:56.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5344000000000001e-05 2024-09-14T22:37:56.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.447 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:56.447 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203906+0000", 2024-09-14T22:37:56.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120344 2024-09-14T22:37:56.448 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.448 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:56.448 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.448 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97224 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1045504~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:37:56.448 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:37:56.448 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 46.280341352999997, 2024-09-14T22:37:56.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12513321199999999, 2024-09-14T22:37:56.448 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:56.448 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:56.448 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:56.448 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:56.448 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:56.449 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:56.449 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97224 2024-09-14T22:37:56.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.449 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:56.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.449 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:56.449 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:37:56.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.449 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:56.449 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:37:56.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.450 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:56.450 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082581+0000", 2024-09-14T22:37:56.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0032971530000000001 2024-09-14T22:37:56.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.450 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.450 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:56.450 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082592+0000", 2024-09-14T22:37:56.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1345999999999999e-05 2024-09-14T22:37:56.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.450 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.450 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:56.450 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082618+0000", 2024-09-14T22:37:56.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5873000000000001e-05 2024-09-14T22:37:56.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.450 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:56.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082638+0000", 2024-09-14T22:37:56.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9565999999999999e-05 2024-09-14T22:37:56.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:56.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204004+0000", 2024-09-14T22:37:56.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.121366038 2024-09-14T22:37:56.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:56.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204188+0000", 2024-09-14T22:37:56.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018427 2024-09-14T22:37:56.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:56.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204417+0000", 2024-09-14T22:37:56.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022896599999999999 2024-09-14T22:37:56.452 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.452 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:56.452 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.452 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.452 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:56.452 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:56.453 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:37:56.458 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:56.454+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:56.542 INFO:tasks.ceph.mgr.x.smithi176.stderr:2024-09-14T22:37:56.534+0000 7f10a3ce0640 -1 mgr.server reply reply (3) No such process no change 2024-09-14T22:37:56.543 INFO:teuthology.orchestra.run.smithi001.stdout:moved 1 / 82 (1.21951%) 2024-09-14T22:37:56.543 INFO:teuthology.orchestra.run.smithi001.stdout:avg 10.25 2024-09-14T22:37:56.543 INFO:teuthology.orchestra.run.smithi001.stdout:stddev 3.7333 -> 3.79967 (expected baseline 2.99479) 2024-09-14T22:37:56.543 INFO:teuthology.orchestra.run.smithi001.stdout:min osd.3 with 4 -> 4 pgs (0.390244 -> 0.390244 * mean) 2024-09-14T22:37:56.543 INFO:teuthology.orchestra.run.smithi001.stdout:max osd.7 with 16 -> 16 pgs (1.56098 -> 1.56098 * mean) 2024-09-14T22:37:56.544 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:37:56.544 INFO:teuthology.orchestra.run.smithi001.stdout:oload 110 2024-09-14T22:37:56.544 INFO:teuthology.orchestra.run.smithi001.stdout:max_change 0.05 2024-09-14T22:37:56.544 INFO:teuthology.orchestra.run.smithi001.stdout:max_change_osds 4 2024-09-14T22:37:56.544 INFO:teuthology.orchestra.run.smithi001.stdout:average_utilization 0.0005 2024-09-14T22:37:56.544 INFO:teuthology.orchestra.run.smithi001.stdout:overload_utilization 0.0005 2024-09-14T22:37:56.544 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3 weight 0.5158 -> 0.4658 2024-09-14T22:37:56.544 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4 weight 0.5324 -> 0.4824 2024-09-14T22:37:56.544 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1 weight 0.6000 -> 0.5500 2024-09-14T22:37:56.545 INFO:teuthology.orchestra.run.smithi001.stderr:no change 2024-09-14T22:37:56.559 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:56.554+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:56.625 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:56.625 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:56.625 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:37:56.625 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:56.640 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:37:56.660 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:56.658+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:56.726 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2024-09-14T22:37:56.801 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:56.798+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:56.820 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:56.820 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:56.820 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:37:56.820 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:37:56.820 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:56.837 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:37:56.883 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:56.878+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:56.962 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:56.958+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:56.969 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:56.969 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:56.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:56.969 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:56.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.969 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:72252 3.2 3:442c6d72:::rbd_data.113cb4c83968.0000000000000015:head [zero 4456448~393216] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-14T22:37:56.969 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:37:56.969 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.264071759999993, 2024-09-14T22:37:56.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0176591500000001, 2024-09-14T22:37:56.970 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:56.970 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:56.970 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T22:37:56.970 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:56.970 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:56.970 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:56.970 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 72252 2024-09-14T22:37:56.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.970 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:56.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:56.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:37:56.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:56.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:37:56.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:56.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700101+0000", 2024-09-14T22:37:56.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4570000000000001e-06 2024-09-14T22:37:56.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:56.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700114+0000", 2024-09-14T22:37:56.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2799999999999999e-05 2024-09-14T22:37:56.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:56.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700146+0000", 2024-09-14T22:37:56.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2116000000000002e-05 2024-09-14T22:37:56.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:56.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700170+0000", 2024-09-14T22:37:56.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3770000000000001e-05 2024-09-14T22:37:56.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:56.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700285+0000", 2024-09-14T22:37:56.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000115228 2024-09-14T22:37:56.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op must wait for map", 2024-09-14T22:37:56.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700297+0000", 2024-09-14T22:37:56.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1681999999999999e-05 2024-09-14T22:37:56.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:56.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.703312+0000", 2024-09-14T22:37:56.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003015046 2024-09-14T22:37:56.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:37:56.974 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.703327+0000", 2024-09-14T22:37:56.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5481999999999998e-05 2024-09-14T22:37:56.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:56.974 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:51.717566+0000", 2024-09-14T22:37:56.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0142383509999999 2024-09-14T22:37:56.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:56.974 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:51.717757+0000", 2024-09-14T22:37:56.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000191218 2024-09-14T22:37:56.974 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.975 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:56.975 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.975 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:74156 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T22:37:56.975 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:37:56.975 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 61.44960305, 2024-09-14T22:37:56.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062073521, 2024-09-14T22:37:56.975 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:56.975 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:56.975 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:56.975 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:56.975 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:56.976 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:56.976 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74156 2024-09-14T22:37:56.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.976 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:56.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.976 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:56.976 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:37:56.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.976 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:56.976 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:37:56.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:56.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514569+0000", 2024-09-14T22:37:56.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0229999999999999e-06 2024-09-14T22:37:56.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:56.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514596+0000", 2024-09-14T22:37:56.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6903e-05 2024-09-14T22:37:56.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:56.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514623+0000", 2024-09-14T22:37:56.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7206e-05 2024-09-14T22:37:56.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:56.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514637+0000", 2024-09-14T22:37:56.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:37:56.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:56.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514781+0000", 2024-09-14T22:37:56.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014394899999999999 2024-09-14T22:37:56.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:56.979 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.515005+0000", 2024-09-14T22:37:56.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022458499999999999 2024-09-14T22:37:56.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.979 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:37:56.979 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.527785+0000", 2024-09-14T22:37:56.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012780112 2024-09-14T22:37:56.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.979 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:56.979 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576209+0000", 2024-09-14T22:37:56.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.048423992999999999 2024-09-14T22:37:56.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.980 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:56.980 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576346+0000", 2024-09-14T22:37:56.980 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000136909 2024-09-14T22:37:56.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.980 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:56.980 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576517+0000", 2024-09-14T22:37:56.980 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000170848 2024-09-14T22:37:56.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.980 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:56.980 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576640+0000", 2024-09-14T22:37:56.980 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000122953 2024-09-14T22:37:56.980 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.981 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:56.981 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.981 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82513 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 557056~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:56.981 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:37:56.981 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 57.544971230999998, 2024-09-14T22:37:56.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059345486000000003, 2024-09-14T22:37:56.981 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:56.981 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:56.981 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:56.981 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:56.981 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:56.981 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:56.982 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82513 2024-09-14T22:37:56.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.982 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:56.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:56.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:37:56.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:56.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:37:56.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:56.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419200+0000", 2024-09-14T22:37:56.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.184e-06 2024-09-14T22:37:56.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:56.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419207+0000", 2024-09-14T22:37:56.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2690000000000002e-06 2024-09-14T22:37:56.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:56.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419224+0000", 2024-09-14T22:37:56.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6623e-05 2024-09-14T22:37:56.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.984 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:56.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419236+0000", 2024-09-14T22:37:56.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2367e-05 2024-09-14T22:37:56.984 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.985 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:56.985 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478294+0000", 2024-09-14T22:37:56.985 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059057927000000003 2024-09-14T22:37:56.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.985 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.985 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:56.985 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478402+0000", 2024-09-14T22:37:56.985 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107921 2024-09-14T22:37:56.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.985 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.985 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:56.985 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478544+0000", 2024-09-14T22:37:56.985 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014219499999999999 2024-09-14T22:37:56.985 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.986 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:56.986 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.986 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.986 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82514 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 704512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:56.986 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:37:56.986 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 57.544912128, 2024-09-14T22:37:56.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059580793, 2024-09-14T22:37:56.986 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:56.986 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:56.987 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:56.987 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:56.987 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:56.987 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:56.987 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82514 2024-09-14T22:37:56.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.987 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:56.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:56.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:37:56.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.988 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:56.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:37:56.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.988 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:56.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419259+0000", 2024-09-14T22:37:56.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.353e-06 2024-09-14T22:37:56.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.989 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:56.989 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419276+0000", 2024-09-14T22:37:56.989 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7567e-05 2024-09-14T22:37:56.989 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.989 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.989 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:56.989 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419294+0000", 2024-09-14T22:37:56.989 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.713e-05 2024-09-14T22:37:56.989 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.989 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.989 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:56.989 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419306+0000", 2024-09-14T22:37:56.989 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.279e-05 2024-09-14T22:37:56.989 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.990 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:56.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478589+0000", 2024-09-14T22:37:56.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059282978 2024-09-14T22:37:56.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.990 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:56.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478694+0000", 2024-09-14T22:37:56.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000104561 2024-09-14T22:37:56.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.990 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:56.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478838+0000", 2024-09-14T22:37:56.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014441399999999999 2024-09-14T22:37:56.991 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.991 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:56.991 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.991 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.991 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82515 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:56.991 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:37:56.991 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 57.544846978000002, 2024-09-14T22:37:56.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059815123999999997, 2024-09-14T22:37:56.991 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:56.991 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:56.991 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:56.991 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:56.992 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:56.992 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:56.992 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82515 2024-09-14T22:37:56.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.992 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:56.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:56.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:37:56.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:56.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:37:56.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.993 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.993 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:56.993 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419324+0000", 2024-09-14T22:37:56.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2130000000000001e-06 2024-09-14T22:37:56.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.993 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.993 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:56.993 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419331+0000", 2024-09-14T22:37:56.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4730000000000003e-06 2024-09-14T22:37:56.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:56.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419348+0000", 2024-09-14T22:37:56.994 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6269999999999998e-05 2024-09-14T22:37:56.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:56.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419360+0000", 2024-09-14T22:37:56.994 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2243e-05 2024-09-14T22:37:56.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:56.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478884+0000", 2024-09-14T22:37:56.994 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059524369000000001 2024-09-14T22:37:56.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.995 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:56.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478990+0000", 2024-09-14T22:37:56.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010569 2024-09-14T22:37:56.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.995 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:56.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479138+0000", 2024-09-14T22:37:56.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000147866 2024-09-14T22:37:56.995 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.995 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:56.995 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:56.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.996 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82516 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 999424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:56.996 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:37:56.996 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 57.544794473000003, 2024-09-14T22:37:56.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061979595999999998, 2024-09-14T22:37:56.996 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:56.996 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:56.996 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:56.996 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:56.996 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:56.996 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:56.996 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82516 2024-09-14T22:37:56.996 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.996 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:56.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.997 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:56.997 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:37:56.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.997 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:56.997 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:37:56.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:56.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:56.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419376+0000", 2024-09-14T22:37:56.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.237e-06 2024-09-14T22:37:56.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:56.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419383+0000", 2024-09-14T22:37:56.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8170000000000001e-06 2024-09-14T22:37:56.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:56.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419400+0000", 2024-09-14T22:37:56.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6636000000000001e-05 2024-09-14T22:37:56.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:56.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419412+0000", 2024-09-14T22:37:56.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2357e-05 2024-09-14T22:37:56.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:56.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:56.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.000 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481080+0000", 2024-09-14T22:37:57.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061668032999999997 2024-09-14T22:37:57.000 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.000 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.000 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481193+0000", 2024-09-14T22:37:57.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112335 2024-09-14T22:37:57.000 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.000 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.000 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481355+0000", 2024-09-14T22:37:57.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016218100000000001 2024-09-14T22:37:57.000 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.001 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.001 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.001 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82538 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1097728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:57.001 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:37:57.001 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 57.544742403000001, 2024-09-14T22:37:57.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059971705, 2024-09-14T22:37:57.001 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.001 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.001 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:57.001 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.001 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:57.002 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:57.002 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82538 2024-09-14T22:37:57.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.002 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.002 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.002 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.002 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:37:57.002 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.002 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.002 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.002 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:37:57.002 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419429+0000", 2024-09-14T22:37:57.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.497e-06 2024-09-14T22:37:57.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419436+0000", 2024-09-14T22:37:57.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2099999999999996e-06 2024-09-14T22:37:57.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419453+0000", 2024-09-14T22:37:57.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6840000000000001e-05 2024-09-14T22:37:57.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.004 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419465+0000", 2024-09-14T22:37:57.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2283000000000001e-05 2024-09-14T22:37:57.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.004 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479182+0000", 2024-09-14T22:37:57.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059717339000000001 2024-09-14T22:37:57.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.004 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479297+0000", 2024-09-14T22:37:57.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000114259 2024-09-14T22:37:57.005 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.005 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.005 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.005 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479399+0000", 2024-09-14T22:37:57.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102277 2024-09-14T22:37:57.005 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.005 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.005 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.005 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.005 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.005 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82539 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:57.005 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:37:57.006 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 57.544689552999998, 2024-09-14T22:37:57.006 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060166935999999997, 2024-09-14T22:37:57.006 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.006 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.006 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:57.006 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.006 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:57.006 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:57.006 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82539 2024-09-14T22:37:57.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.006 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.006 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.006 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.006 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:37:57.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.007 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.007 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:37:57.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.007 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.007 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419482+0000", 2024-09-14T22:37:57.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4130000000000001e-06 2024-09-14T22:37:57.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419488+0000", 2024-09-14T22:37:57.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8460000000000002e-06 2024-09-14T22:37:57.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419505+0000", 2024-09-14T22:37:57.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6477000000000002e-05 2024-09-14T22:37:57.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419518+0000", 2024-09-14T22:37:57.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2797e-05 2024-09-14T22:37:57.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.009 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479438+0000", 2024-09-14T22:37:57.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059919897 2024-09-14T22:37:57.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.009 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479547+0000", 2024-09-14T22:37:57.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109807 2024-09-14T22:37:57.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.010 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479647+0000", 2024-09-14T22:37:57.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9698999999999995e-05 2024-09-14T22:37:57.010 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.010 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.010 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.010 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82542 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:57.010 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:37:57.010 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 57.544637109999996, 2024-09-14T22:37:57.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060367430999999999, 2024-09-14T22:37:57.010 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.010 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.010 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:57.011 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.011 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:57.011 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:57.011 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82542 2024-09-14T22:37:57.011 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.011 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.011 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:37:57.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:37:57.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419534+0000", 2024-09-14T22:37:57.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T22:37:57.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419542+0000", 2024-09-14T22:37:57.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8199999999999997e-06 2024-09-14T22:37:57.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419559+0000", 2024-09-14T22:37:57.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6852999999999999e-05 2024-09-14T22:37:57.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.014 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419571+0000", 2024-09-14T22:37:57.014 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2364000000000001e-05 2024-09-14T22:37:57.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.014 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479686+0000", 2024-09-14T22:37:57.014 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060115522999999997 2024-09-14T22:37:57.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479795+0000", 2024-09-14T22:37:57.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010831399999999999 2024-09-14T22:37:57.015 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.015 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.015 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479900+0000", 2024-09-14T22:37:57.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105284 2024-09-14T22:37:57.015 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.015 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.015 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.016 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.016 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.016 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82544 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:57.016 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:37:57.016 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 57.544583373999998, 2024-09-14T22:37:57.016 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060590515999999997, 2024-09-14T22:37:57.016 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.016 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.016 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:57.016 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.016 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:57.016 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:57.016 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82544 2024-09-14T22:37:57.017 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.017 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:37:57.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.017 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:37:57.018 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.018 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.018 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.018 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419588+0000", 2024-09-14T22:37:57.018 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.35e-06 2024-09-14T22:37:57.018 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.018 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.018 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419595+0000", 2024-09-14T22:37:57.018 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4030000000000001e-06 2024-09-14T22:37:57.018 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.019 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419612+0000", 2024-09-14T22:37:57.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6670000000000001e-05 2024-09-14T22:37:57.019 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.019 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.019 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419624+0000", 2024-09-14T22:37:57.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-05 2024-09-14T22:37:57.019 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.019 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.019 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479957+0000", 2024-09-14T22:37:57.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060333287999999999 2024-09-14T22:37:57.020 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.020 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.020 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.020 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480074+0000", 2024-09-14T22:37:57.020 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116443 2024-09-14T22:37:57.020 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.020 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.020 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.020 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480177+0000", 2024-09-14T22:37:57.020 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000103102 2024-09-14T22:37:57.020 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.021 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.021 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.021 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.021 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.021 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82546 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:57.021 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:37:57.021 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 57.544530455, 2024-09-14T22:37:57.021 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060795428999999998, 2024-09-14T22:37:57.021 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.021 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.021 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:57.021 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.022 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:57.022 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:57.022 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82546 2024-09-14T22:37:57.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.022 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.022 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.022 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:37:57.022 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.022 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.022 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:37:57.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419640+0000", 2024-09-14T22:37:57.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.254e-06 2024-09-14T22:37:57.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419647+0000", 2024-09-14T22:37:57.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9e-06 2024-09-14T22:37:57.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419664+0000", 2024-09-14T22:37:57.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6345999999999999e-05 2024-09-14T22:37:57.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.024 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.024 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419676+0000", 2024-09-14T22:37:57.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.224e-05 2024-09-14T22:37:57.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.024 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.024 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480218+0000", 2024-09-14T22:37:57.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060542133999999997 2024-09-14T22:37:57.025 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.025 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.025 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.025 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480336+0000", 2024-09-14T22:37:57.025 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117472 2024-09-14T22:37:57.025 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.025 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.025 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.025 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480435+0000", 2024-09-14T22:37:57.025 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9083e-05 2024-09-14T22:37:57.025 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.026 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.026 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.026 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.026 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.026 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82548 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:57.026 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:37:57.026 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 57.544478445000003, 2024-09-14T22:37:57.026 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061011636000000001, 2024-09-14T22:37:57.026 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.026 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.027 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:57.027 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.027 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:57.027 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:57.027 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82548 2024-09-14T22:37:57.027 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.027 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.028 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.028 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.028 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:37:57.028 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.028 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.028 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.028 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.029 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:37:57.029 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.029 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.029 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.029 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.029 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419693+0000", 2024-09-14T22:37:57.029 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.387e-06 2024-09-14T22:37:57.029 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.029 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.029 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.029 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419699+0000", 2024-09-14T22:37:57.029 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8730000000000001e-06 2024-09-14T22:37:57.030 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.030 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.030 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.030 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419716+0000", 2024-09-14T22:37:57.030 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6416e-05 2024-09-14T22:37:57.030 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.030 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.030 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.030 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419728+0000", 2024-09-14T22:37:57.030 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2597e-05 2024-09-14T22:37:57.030 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.030 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.030 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.030 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480476+0000", 2024-09-14T22:37:57.030 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060747989000000002 2024-09-14T22:37:57.031 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.031 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.031 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.031 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480583+0000", 2024-09-14T22:37:57.031 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010638500000000001 2024-09-14T22:37:57.031 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.031 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.031 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.031 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480703+0000", 2024-09-14T22:37:57.031 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011998900000000001 2024-09-14T22:37:57.031 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.031 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.031 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.031 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.031 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.032 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82550 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1982464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:57.032 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:37:57.032 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 57.544425691999997, 2024-09-14T22:37:57.032 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061295483999999997, 2024-09-14T22:37:57.032 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.032 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.032 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:57.032 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.032 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:57.032 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:57.032 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82550 2024-09-14T22:37:57.032 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.032 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.033 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.033 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.033 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:37:57.033 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.033 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.033 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.033 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.033 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:37:57.033 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.033 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.033 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.034 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.034 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419745+0000", 2024-09-14T22:37:57.034 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.406e-06 2024-09-14T22:37:57.034 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.034 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.034 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.034 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419753+0000", 2024-09-14T22:37:57.034 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8140000000000001e-06 2024-09-14T22:37:57.034 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.034 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.034 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.034 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419770+0000", 2024-09-14T22:37:57.034 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6719999999999999e-05 2024-09-14T22:37:57.034 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.035 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.035 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.035 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419782+0000", 2024-09-14T22:37:57.035 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2252999999999999e-05 2024-09-14T22:37:57.035 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.035 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.035 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.035 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480745+0000", 2024-09-14T22:37:57.035 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060962385000000001 2024-09-14T22:37:57.035 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.035 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.035 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.035 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480900+0000", 2024-09-14T22:37:57.035 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015513899999999999 2024-09-14T22:37:57.035 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.036 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.036 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.036 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481039+0000", 2024-09-14T22:37:57.036 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013976700000000001 2024-09-14T22:37:57.036 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.036 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.036 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.036 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.036 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.036 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82588 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1064960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:57.036 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:37:57.037 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 57.544371802000001, 2024-09-14T22:37:57.037 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061863732999999997, 2024-09-14T22:37:57.037 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.037 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.037 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:57.037 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.037 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:57.037 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:57.037 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82588 2024-09-14T22:37:57.037 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.037 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.037 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.037 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.037 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:37:57.038 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.038 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.038 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.038 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.038 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:37:57.038 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.038 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.038 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.038 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.038 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419799+0000", 2024-09-14T22:37:57.038 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4160000000000001e-06 2024-09-14T22:37:57.038 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.038 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.038 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.039 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419807+0000", 2024-09-14T22:37:57.039 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2169999999999994e-06 2024-09-14T22:37:57.039 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.039 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.039 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.039 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419824+0000", 2024-09-14T22:37:57.039 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6776e-05 2024-09-14T22:37:57.039 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.039 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.039 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.039 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419837+0000", 2024-09-14T22:37:57.039 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2680000000000001e-05 2024-09-14T22:37:57.039 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.039 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.040 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.040 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481395+0000", 2024-09-14T22:37:57.040 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061558416999999997 2024-09-14T22:37:57.040 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.040 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.040 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.040 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481532+0000", 2024-09-14T22:37:57.040 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000136861 2024-09-14T22:37:57.040 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.040 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.040 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.040 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481662+0000", 2024-09-14T22:37:57.040 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012936600000000001 2024-09-14T22:37:57.040 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.041 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.041 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.041 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.041 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.041 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82589 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:57.041 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:37:57.041 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 57.544317206999999, 2024-09-14T22:37:57.041 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062064592000000002, 2024-09-14T22:37:57.041 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.041 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.041 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:57.042 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.042 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:57.042 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:57.042 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82589 2024-09-14T22:37:57.042 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.042 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.042 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.042 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.042 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:37:57.042 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.042 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.042 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.043 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.043 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:37:57.043 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.043 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.043 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.043 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.043 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419854+0000", 2024-09-14T22:37:57.043 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.367e-06 2024-09-14T22:37:57.043 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.043 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.043 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.044 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419862+0000", 2024-09-14T22:37:57.044 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8930000000000005e-06 2024-09-14T22:37:57.044 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.044 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.044 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.044 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419896+0000", 2024-09-14T22:37:57.044 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4134000000000002e-05 2024-09-14T22:37:57.044 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.044 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.044 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.044 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419909+0000", 2024-09-14T22:37:57.044 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3332999999999999e-05 2024-09-14T22:37:57.045 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.045 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.045 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.045 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481702+0000", 2024-09-14T22:37:57.045 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061792947000000001 2024-09-14T22:37:57.045 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.045 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.045 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.045 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481808+0000", 2024-09-14T22:37:57.045 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105655 2024-09-14T22:37:57.046 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.046 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.046 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.046 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481917+0000", 2024-09-14T22:37:57.046 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109263 2024-09-14T22:37:57.046 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.046 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.046 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.046 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.046 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.046 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82590 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1359872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:57.046 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:37:57.046 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 57.544243600000001, 2024-09-14T22:37:57.046 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062362895000000002, 2024-09-14T22:37:57.047 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.047 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.047 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:57.047 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.047 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:57.047 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:57.047 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82590 2024-09-14T22:37:57.047 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.047 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.047 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.047 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.047 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:37:57.048 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.048 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.048 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.048 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.048 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:37:57.048 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.048 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.048 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.048 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.048 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419928+0000", 2024-09-14T22:37:57.048 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4470000000000001e-06 2024-09-14T22:37:57.049 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.049 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.049 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.049 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419942+0000", 2024-09-14T22:37:57.049 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4229e-05 2024-09-14T22:37:57.049 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.049 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.049 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.049 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419972+0000", 2024-09-14T22:37:57.049 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.065e-05 2024-09-14T22:37:57.050 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.050 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.050 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.050 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419985+0000", 2024-09-14T22:37:57.050 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2903e-05 2024-09-14T22:37:57.050 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.050 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.050 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.050 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481960+0000", 2024-09-14T22:37:57.050 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061974435000000001 2024-09-14T22:37:57.050 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.050 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482106+0000", 2024-09-14T22:37:57.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014643800000000001 2024-09-14T22:37:57.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482289+0000", 2024-09-14T22:37:57.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000182793 2024-09-14T22:37:57.051 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.051 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.051 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.052 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82592 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:57.052 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:37:57.052 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 57.544167377999997, 2024-09-14T22:37:57.052 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062631507000000003, 2024-09-14T22:37:57.052 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.052 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.052 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:57.052 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.052 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:57.052 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:57.052 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82592 2024-09-14T22:37:57.052 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.053 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:37:57.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:37:57.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.054 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.054 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420017+0000", 2024-09-14T22:37:57.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4744e-05 2024-09-14T22:37:57.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.054 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.054 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420036+0000", 2024-09-14T22:37:57.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8680000000000001e-05 2024-09-14T22:37:57.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.054 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420054+0000", 2024-09-14T22:37:57.055 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7799999999999999e-05 2024-09-14T22:37:57.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420067+0000", 2024-09-14T22:37:57.055 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3013000000000001e-05 2024-09-14T22:37:57.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482358+0000", 2024-09-14T22:37:57.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062291445000000001 2024-09-14T22:37:57.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.056 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482469+0000", 2024-09-14T22:37:57.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110608 2024-09-14T22:37:57.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.056 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482634+0000", 2024-09-14T22:37:57.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000165217 2024-09-14T22:37:57.056 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.056 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.057 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.057 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82594 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1654784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:57.057 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:37:57.057 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 57.544069401999998, 2024-09-14T22:37:57.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062893849000000002, 2024-09-14T22:37:57.058 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.058 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.058 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:57.058 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.058 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:57.058 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:57.058 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82594 2024-09-14T22:37:57.058 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.058 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.058 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.058 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:37:57.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.059 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.059 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:37:57.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.059 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.059 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420102+0000", 2024-09-14T22:37:57.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5e-06 2024-09-14T22:37:57.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420115+0000", 2024-09-14T22:37:57.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.324e-05 2024-09-14T22:37:57.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420133+0000", 2024-09-14T22:37:57.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8213e-05 2024-09-14T22:37:57.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420147+0000", 2024-09-14T22:37:57.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3284e-05 2024-09-14T22:37:57.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482688+0000", 2024-09-14T22:37:57.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062541425999999997 2024-09-14T22:37:57.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482794+0000", 2024-09-14T22:37:57.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010575500000000001 2024-09-14T22:37:57.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482994+0000", 2024-09-14T22:37:57.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020043099999999999 2024-09-14T22:37:57.062 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.062 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.062 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.062 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82596 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:57.062 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:37:57.063 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 57.544006101999997, 2024-09-14T22:37:57.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063187238000000007, 2024-09-14T22:37:57.063 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.063 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.063 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:57.063 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.063 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:57.063 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:57.063 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82596 2024-09-14T22:37:57.063 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.063 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.063 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.063 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.063 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:37:57.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.064 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.064 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:37:57.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.064 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.064 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420165+0000", 2024-09-14T22:37:57.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4100000000000001e-06 2024-09-14T22:37:57.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420173+0000", 2024-09-14T22:37:57.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8129999999999996e-06 2024-09-14T22:37:57.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420190+0000", 2024-09-14T22:37:57.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7076999999999999e-05 2024-09-14T22:37:57.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420203+0000", 2024-09-14T22:37:57.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2683e-05 2024-09-14T22:37:57.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483060+0000", 2024-09-14T22:37:57.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062857114000000006 2024-09-14T22:37:57.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483167+0000", 2024-09-14T22:37:57.067 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010713999999999999 2024-09-14T22:37:57.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.067 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.067 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483351+0000", 2024-09-14T22:37:57.067 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018400099999999999 2024-09-14T22:37:57.067 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.067 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.067 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.068 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82597 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1949696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:37:57.068 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:37:57.068 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 57.543949275999999, 2024-09-14T22:37:57.068 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063466513000000002, 2024-09-14T22:37:57.068 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.068 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.068 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:57.068 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.068 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:57.068 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:57.068 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82597 2024-09-14T22:37:57.068 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.068 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.069 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:37:57.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.069 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:37:57.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420221+0000", 2024-09-14T22:37:57.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.043e-06 2024-09-14T22:37:57.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420229+0000", 2024-09-14T22:37:57.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0469999999999994e-06 2024-09-14T22:37:57.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.071 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.071 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420246+0000", 2024-09-14T22:37:57.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6339999999999999e-05 2024-09-14T22:37:57.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.071 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.072 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420258+0000", 2024-09-14T22:37:57.072 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2296000000000001e-05 2024-09-14T22:37:57.072 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.072 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.072 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.072 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483402+0000", 2024-09-14T22:37:57.072 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063144318000000005 2024-09-14T22:37:57.072 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.072 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.072 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.072 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483530+0000", 2024-09-14T22:37:57.072 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012713199999999999 2024-09-14T22:37:57.072 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.073 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.073 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483687+0000", 2024-09-14T22:37:57.073 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015733699999999999 2024-09-14T22:37:57.073 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.073 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.073 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.073 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.073 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.073 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:57.075 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:37:57.075 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:57.066+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:57.175 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:57.170+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:57.259 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:57.259 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:57.260 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:37:57.260 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:57.275 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:37:57.276 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:57.270+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:57.377 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:57.370+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:57.392 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:57.392 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:37:57.392 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:37:57.392 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:37:57.393 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:57.407 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:37:57.478 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:57.474+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:57.578 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:37:57.579 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:37:57.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:37:57.579 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:37:57.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.579 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:37:57.579 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:37:57.579 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 117.56792489, 2024-09-14T22:37:57.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.398029152, 2024-09-14T22:37:57.579 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.579 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.579 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:37:57.579 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.579 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:37:57.580 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:37:57.580 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 1 2024-09-14T22:37:57.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.580 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.580 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.580 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:37:57.580 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.580 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.580 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:37:57.580 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.581 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006183+0000", 2024-09-14T22:37:57.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9059999999999998e-06 2024-09-14T22:37:57.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.581 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006198+0000", 2024-09-14T22:37:57.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.523e-05 2024-09-14T22:37:57.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.581 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006219+0000", 2024-09-14T22:37:57.582 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1342999999999999e-05 2024-09-14T22:37:57.582 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.582 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.582 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006240+0000", 2024-09-14T22:37:57.582 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1186999999999999e-05 2024-09-14T22:37:57.582 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.582 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.582 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006363+0000", 2024-09-14T22:37:57.582 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012258200000000001 2024-09-14T22:37:57.582 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.582 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:37:57.583 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006392+0000", 2024-09-14T22:37:57.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9607e-05 2024-09-14T22:37:57.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.583 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.583 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401084+0000", 2024-09-14T22:37:57.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.39469142699999998 2024-09-14T22:37:57.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.583 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.583 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401394+0000", 2024-09-14T22:37:57.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030987799999999998 2024-09-14T22:37:57.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:37:57.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401860+0000", 2024-09-14T22:37:57.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00046654299999999998 2024-09-14T22:37:57.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:37:57.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.403386+0000", 2024-09-14T22:37:57.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001525322 2024-09-14T22:37:57.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:37:57.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404042+0000", 2024-09-14T22:37:57.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065676099999999996 2024-09-14T22:37:57.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:37:57.585 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404136+0000", 2024-09-14T22:37:57.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4032999999999999e-05 2024-09-14T22:37:57.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.585 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404209+0000", 2024-09-14T22:37:57.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2333e-05 2024-09-14T22:37:57.585 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.586 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.586 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.586 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104182 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14352384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:57.586 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:37:57.587 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.085070903999998, 2024-09-14T22:37:57.587 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071642496999999999, 2024-09-14T22:37:57.587 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.587 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.587 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:57.587 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.587 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:57.587 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:57.587 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104182 2024-09-14T22:37:57.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.587 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.587 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.587 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:37:57.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:37:57.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489035+0000", 2024-09-14T22:37:57.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3430000000000001e-06 2024-09-14T22:37:57.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489041+0000", 2024-09-14T22:37:57.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8139999999999997e-06 2024-09-14T22:37:57.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489059+0000", 2024-09-14T22:37:57.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8193e-05 2024-09-14T22:37:57.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489073+0000", 2024-09-14T22:37:57.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:37:57.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.590 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560434+0000", 2024-09-14T22:37:57.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071361463999999999 2024-09-14T22:37:57.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.590 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560545+0000", 2024-09-14T22:37:57.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110263 2024-09-14T22:37:57.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.591 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560676+0000", 2024-09-14T22:37:57.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013138000000000001 2024-09-14T22:37:57.591 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.591 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.591 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.591 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104183 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14483456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:57.591 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:37:57.591 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.084947585000002, 2024-09-14T22:37:57.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071781943000000001, 2024-09-14T22:37:57.591 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.591 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.591 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:57.592 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.592 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:57.592 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:57.592 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104183 2024-09-14T22:37:57.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.592 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.592 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:37:57.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.593 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:37:57.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.593 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489159+0000", 2024-09-14T22:37:57.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5629999999999999e-06 2024-09-14T22:37:57.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489165+0000", 2024-09-14T22:37:57.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7070000000000003e-06 2024-09-14T22:37:57.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489186+0000", 2024-09-14T22:37:57.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.088e-05 2024-09-14T22:37:57.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489201+0000", 2024-09-14T22:37:57.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5203e-05 2024-09-14T22:37:57.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.595 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.595 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560713+0000", 2024-09-14T22:37:57.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071511447000000006 2024-09-14T22:37:57.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.595 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.595 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560819+0000", 2024-09-14T22:37:57.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106579 2024-09-14T22:37:57.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.595 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.595 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560939+0000", 2024-09-14T22:37:57.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119564 2024-09-14T22:37:57.596 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.596 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.596 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.596 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.596 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.596 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104184 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:57.596 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:37:57.596 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.084883142000002, 2024-09-14T22:37:57.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071970107000000005, 2024-09-14T22:37:57.596 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.596 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.596 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:57.597 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.597 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:57.597 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:57.597 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104184 2024-09-14T22:37:57.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.597 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.597 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.597 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:37:57.597 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.597 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:37:57.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489223+0000", 2024-09-14T22:37:57.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4470000000000001e-06 2024-09-14T22:37:57.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489229+0000", 2024-09-14T22:37:57.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8200000000000002e-06 2024-09-14T22:37:57.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489247+0000", 2024-09-14T22:37:57.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8150000000000001e-05 2024-09-14T22:37:57.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489260+0000", 2024-09-14T22:37:57.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3570000000000001e-05 2024-09-14T22:37:57.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560979+0000", 2024-09-14T22:37:57.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071718831999999996 2024-09-14T22:37:57.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.600 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561075+0000", 2024-09-14T22:37:57.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5848999999999997e-05 2024-09-14T22:37:57.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.600 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561191+0000", 2024-09-14T22:37:57.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116439 2024-09-14T22:37:57.600 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.600 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.600 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.601 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104185 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14745600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:57.601 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:37:57.601 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.084823559, 2024-09-14T22:37:57.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072173750999999994, 2024-09-14T22:37:57.601 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.601 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.601 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:57.601 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.601 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:57.601 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:57.601 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104185 2024-09-14T22:37:57.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.601 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.602 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:37:57.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.602 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:37:57.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.602 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489282+0000", 2024-09-14T22:37:57.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2500000000000001e-06 2024-09-14T22:37:57.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489287+0000", 2024-09-14T22:37:57.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1359999999999996e-06 2024-09-14T22:37:57.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489304+0000", 2024-09-14T22:37:57.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7033000000000001e-05 2024-09-14T22:37:57.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489318+0000", 2024-09-14T22:37:57.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3934000000000001e-05 2024-09-14T22:37:57.604 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.604 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.604 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561232+0000", 2024-09-14T22:37:57.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071913759999999993 2024-09-14T22:37:57.604 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.605 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.605 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561340+0000", 2024-09-14T22:37:57.605 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107921 2024-09-14T22:37:57.605 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.605 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.605 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.605 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561455+0000", 2024-09-14T22:37:57.605 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011471700000000001 2024-09-14T22:37:57.605 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.605 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.605 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.605 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.606 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104186 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14876672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:57.606 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:37:57.606 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.084767650000003, 2024-09-14T22:37:57.606 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072393374999999996, 2024-09-14T22:37:57.606 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.606 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.606 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:57.606 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.606 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:57.606 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:57.606 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104186 2024-09-14T22:37:57.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.607 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:37:57.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:37:57.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.608 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489338+0000", 2024-09-14T22:37:57.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.48e-06 2024-09-14T22:37:57.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.608 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489366+0000", 2024-09-14T22:37:57.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7307000000000001e-05 2024-09-14T22:37:57.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.608 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489385+0000", 2024-09-14T22:37:57.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9283000000000002e-05 2024-09-14T22:37:57.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.609 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.609 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.609 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489399+0000", 2024-09-14T22:37:57.609 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.452e-05 2024-09-14T22:37:57.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.609 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.609 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.609 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561494+0000", 2024-09-14T22:37:57.609 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072094885999999997 2024-09-14T22:37:57.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.609 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.609 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.609 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561604+0000", 2024-09-14T22:37:57.610 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010925900000000001 2024-09-14T22:37:57.610 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.610 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.610 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.610 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561730+0000", 2024-09-14T22:37:57.610 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012663999999999999 2024-09-14T22:37:57.610 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.610 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.610 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.610 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.610 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.610 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104187 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15007744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:57.610 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:37:57.610 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.084684281000001, 2024-09-14T22:37:57.611 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072599143000000005, 2024-09-14T22:37:57.611 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.611 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.611 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:57.611 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.611 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:57.611 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:57.611 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104187 2024-09-14T22:37:57.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.611 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.612 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:37:57.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.612 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:37:57.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.612 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489422+0000", 2024-09-14T22:37:57.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5340000000000001e-06 2024-09-14T22:37:57.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.613 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489428+0000", 2024-09-14T22:37:57.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1430000000000003e-06 2024-09-14T22:37:57.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.613 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489445+0000", 2024-09-14T22:37:57.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7042999999999999e-05 2024-09-14T22:37:57.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.613 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489458+0000", 2024-09-14T22:37:57.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3474e-05 2024-09-14T22:37:57.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.614 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561771+0000", 2024-09-14T22:37:57.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072312600000000005 2024-09-14T22:37:57.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.614 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561885+0000", 2024-09-14T22:37:57.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011411799999999999 2024-09-14T22:37:57.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.614 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.615 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562019+0000", 2024-09-14T22:37:57.615 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013423100000000001 2024-09-14T22:37:57.615 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.615 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.615 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.615 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.615 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104188 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15138816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:57.615 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:37:57.615 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.084624357000003, 2024-09-14T22:37:57.615 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072824624000000004, 2024-09-14T22:37:57.615 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.615 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.615 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:57.616 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.616 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:57.616 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:57.616 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104188 2024-09-14T22:37:57.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.616 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.616 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.616 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.616 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:37:57.616 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.616 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.616 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.616 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:37:57.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.617 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.617 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489481+0000", 2024-09-14T22:37:57.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.22e-06 2024-09-14T22:37:57.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.617 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.617 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489487+0000", 2024-09-14T22:37:57.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6930000000000004e-06 2024-09-14T22:37:57.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.618 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.618 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489508+0000", 2024-09-14T22:37:57.618 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1135999999999999e-05 2024-09-14T22:37:57.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.618 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.618 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489522+0000", 2024-09-14T22:37:57.618 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3817e-05 2024-09-14T22:37:57.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.618 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.618 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562066+0000", 2024-09-14T22:37:57.618 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072543813999999998 2024-09-14T22:37:57.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562177+0000", 2024-09-14T22:37:57.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000111264 2024-09-14T22:37:57.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562305+0000", 2024-09-14T22:37:57.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012768 2024-09-14T22:37:57.619 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.619 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.619 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.619 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104189 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15269888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:57.620 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:37:57.620 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.084563908, 2024-09-14T22:37:57.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073061300999999995, 2024-09-14T22:37:57.620 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.620 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.620 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:57.620 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.620 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:57.620 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:57.620 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104189 2024-09-14T22:37:57.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.620 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.621 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.621 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:37:57.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.621 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.621 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:37:57.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.621 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.621 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489542+0000", 2024-09-14T22:37:57.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1659999999999999e-06 2024-09-14T22:37:57.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489561+0000", 2024-09-14T22:37:57.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.908e-05 2024-09-14T22:37:57.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489579+0000", 2024-09-14T22:37:57.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8536999999999999e-05 2024-09-14T22:37:57.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489593+0000", 2024-09-14T22:37:57.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3977e-05 2024-09-14T22:37:57.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.623 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.623 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562347+0000", 2024-09-14T22:37:57.623 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072754092000000006 2024-09-14T22:37:57.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.623 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.623 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562458+0000", 2024-09-14T22:37:57.623 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110722 2024-09-14T22:37:57.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.623 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.623 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562602+0000", 2024-09-14T22:37:57.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014372699999999999 2024-09-14T22:37:57.624 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.624 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.624 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.624 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.624 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104190 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:57.624 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:37:57.624 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.084491612000001, 2024-09-14T22:37:57.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.07326133, 2024-09-14T22:37:57.624 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.624 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.624 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:57.624 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.625 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:57.625 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:57.625 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104190 2024-09-14T22:37:57.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.625 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.625 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.625 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.625 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:37:57.625 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.625 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.625 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.625 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:37:57.625 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.626 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489614+0000", 2024-09-14T22:37:57.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1999999999999999e-06 2024-09-14T22:37:57.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.626 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489620+0000", 2024-09-14T22:37:57.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.524e-06 2024-09-14T22:37:57.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.627 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489637+0000", 2024-09-14T22:37:57.627 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7825999999999998e-05 2024-09-14T22:37:57.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.627 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.627 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489671+0000", 2024-09-14T22:37:57.627 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3726000000000002e-05 2024-09-14T22:37:57.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.627 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.627 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562644+0000", 2024-09-14T22:37:57.627 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072973089000000005 2024-09-14T22:37:57.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.628 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562762+0000", 2024-09-14T22:37:57.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011743600000000001 2024-09-14T22:37:57.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.628 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562874+0000", 2024-09-14T22:37:57.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011252900000000001 2024-09-14T22:37:57.628 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.628 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.628 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.628 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104191 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15532032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:57.629 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:37:57.629 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.084383189, 2024-09-14T22:37:57.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073311166999999997, 2024-09-14T22:37:57.629 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.629 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.629 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:57.629 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.629 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:57.629 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:57.629 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104191 2024-09-14T22:37:57.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.629 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.629 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.629 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.630 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:37:57.630 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.630 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.630 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.630 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:37:57.630 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.630 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.630 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.630 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489723+0000", 2024-09-14T22:37:57.630 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6330000000000001e-06 2024-09-14T22:37:57.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.630 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489730+0000", 2024-09-14T22:37:57.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2300000000000002e-06 2024-09-14T22:37:57.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489751+0000", 2024-09-14T22:37:57.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0849999999999999e-05 2024-09-14T22:37:57.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489766+0000", 2024-09-14T22:37:57.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.524e-05 2024-09-14T22:37:57.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.632 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.632 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562896+0000", 2024-09-14T22:37:57.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073129662999999998 2024-09-14T22:37:57.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.632 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.632 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562960+0000", 2024-09-14T22:37:57.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3689999999999995e-05 2024-09-14T22:37:57.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.632 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.632 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563033+0000", 2024-09-14T22:37:57.633 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2861000000000006e-05 2024-09-14T22:37:57.633 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.633 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.633 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.633 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104192 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:57.633 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:37:57.633 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.084287384, 2024-09-14T22:37:57.633 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073463632000000001, 2024-09-14T22:37:57.633 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.633 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.633 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:57.633 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.634 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:57.634 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:57.634 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104192 2024-09-14T22:37:57.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.634 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.634 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.634 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.634 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:37:57.634 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.634 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.634 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.634 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:37:57.634 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.635 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.635 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.635 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489819+0000", 2024-09-14T22:37:57.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.424e-06 2024-09-14T22:37:57.635 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.635 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.635 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.635 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489825+0000", 2024-09-14T22:37:57.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.02e-06 2024-09-14T22:37:57.635 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.635 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.635 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.635 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489844+0000", 2024-09-14T22:37:57.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9760000000000001e-05 2024-09-14T22:37:57.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.636 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.636 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.636 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489860+0000", 2024-09-14T22:37:57.636 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5563e-05 2024-09-14T22:37:57.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.636 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.636 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.636 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563070+0000", 2024-09-14T22:37:57.636 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073209724000000004 2024-09-14T22:37:57.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.636 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.636 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.636 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563175+0000", 2024-09-14T22:37:57.637 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105183 2024-09-14T22:37:57.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.637 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.637 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.637 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563281+0000", 2024-09-14T22:37:57.637 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105958 2024-09-14T22:37:57.637 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.637 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.637 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.637 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.637 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104193 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15794176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:57.637 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:37:57.638 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.084225216999997, 2024-09-14T22:37:57.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073600040000000005, 2024-09-14T22:37:57.638 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.638 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.638 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:57.638 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.638 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:57.638 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:57.638 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104193 2024-09-14T22:37:57.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.638 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:37:57.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.639 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.639 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.639 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:37:57.639 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.639 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.639 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.639 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489881+0000", 2024-09-14T22:37:57.639 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5460000000000001e-06 2024-09-14T22:37:57.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.639 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.639 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.639 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489886+0000", 2024-09-14T22:37:57.640 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1739999999999999e-06 2024-09-14T22:37:57.640 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.640 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.640 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.640 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489912+0000", 2024-09-14T22:37:57.640 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6461999999999998e-05 2024-09-14T22:37:57.640 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.640 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.640 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.640 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489927+0000", 2024-09-14T22:37:57.640 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:37:57.640 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.640 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.640 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.641 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563310+0000", 2024-09-14T22:37:57.641 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073382977000000002 2024-09-14T22:37:57.641 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.641 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.641 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.641 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563385+0000", 2024-09-14T22:37:57.641 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5218999999999999e-05 2024-09-14T22:37:57.641 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.641 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.641 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.641 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563479+0000", 2024-09-14T22:37:57.641 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4622000000000003e-05 2024-09-14T22:37:57.641 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.641 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.642 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.642 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.642 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104194 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15925248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:57.642 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:37:57.642 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.084158625000001, 2024-09-14T22:37:57.642 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073745434999999998, 2024-09-14T22:37:57.642 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.642 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.642 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:57.642 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.642 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:57.642 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:57.642 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104194 2024-09-14T22:37:57.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.643 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.643 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.643 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:37:57.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.643 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.643 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:37:57.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.644 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.644 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.644 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489947+0000", 2024-09-14T22:37:57.644 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.327e-06 2024-09-14T22:37:57.644 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.644 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.644 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.644 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489953+0000", 2024-09-14T22:37:57.644 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9329999999999998e-06 2024-09-14T22:37:57.644 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.644 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.644 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.644 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489971+0000", 2024-09-14T22:37:57.645 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7994000000000001e-05 2024-09-14T22:37:57.645 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.645 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.645 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.645 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489985+0000", 2024-09-14T22:37:57.645 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4226000000000001e-05 2024-09-14T22:37:57.645 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.645 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.645 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.645 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563510+0000", 2024-09-14T22:37:57.645 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073524667000000002 2024-09-14T22:37:57.645 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.645 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.645 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.646 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563597+0000", 2024-09-14T22:37:57.646 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6550000000000003e-05 2024-09-14T22:37:57.646 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.646 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.646 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.646 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563691+0000", 2024-09-14T22:37:57.646 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4738e-05 2024-09-14T22:37:57.646 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.646 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.646 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.646 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.646 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.646 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104195 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:57.647 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:37:57.647 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.084100427999999, 2024-09-14T22:37:57.647 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073876598000000002, 2024-09-14T22:37:57.647 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.647 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.647 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:57.647 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.647 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:57.647 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:57.647 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104195 2024-09-14T22:37:57.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.647 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.648 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.648 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:37:57.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.648 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.648 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.648 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.648 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:37:57.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.648 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.648 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.648 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.648 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490006+0000", 2024-09-14T22:37:57.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4699999999999999e-06 2024-09-14T22:37:57.649 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.649 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.649 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.649 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490011+0000", 2024-09-14T22:37:57.649 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.366e-06 2024-09-14T22:37:57.649 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.649 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.649 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.649 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490046+0000", 2024-09-14T22:37:57.649 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5496e-05 2024-09-14T22:37:57.649 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.649 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.649 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.649 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490061+0000", 2024-09-14T22:37:57.650 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5054e-05 2024-09-14T22:37:57.650 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.650 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.650 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.650 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563719+0000", 2024-09-14T22:37:57.650 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073657412000000005 2024-09-14T22:37:57.650 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.650 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.650 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.650 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563792+0000", 2024-09-14T22:37:57.650 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3319999999999996e-05 2024-09-14T22:37:57.650 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.650 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.651 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.651 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563881+0000", 2024-09-14T22:37:57.651 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8480000000000007e-05 2024-09-14T22:37:57.651 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.651 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.651 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.651 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.651 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.651 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104196 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16187392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:57.651 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:37:57.651 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.084023279, 2024-09-14T22:37:57.651 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073997496999999995, 2024-09-14T22:37:57.651 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.651 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.652 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:57.652 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.652 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:57.652 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:57.652 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104196 2024-09-14T22:37:57.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.652 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.652 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.652 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:37:57.652 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.652 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.653 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.653 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:37:57.653 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.653 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.653 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.653 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.653 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490083+0000", 2024-09-14T22:37:57.653 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.347e-06 2024-09-14T22:37:57.653 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.653 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.653 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.653 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490089+0000", 2024-09-14T22:37:57.653 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9630000000000003e-06 2024-09-14T22:37:57.654 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.654 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.654 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.654 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490107+0000", 2024-09-14T22:37:57.654 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8859999999999999e-05 2024-09-14T22:37:57.654 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.654 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.654 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.654 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490122+0000", 2024-09-14T22:37:57.654 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.432e-05 2024-09-14T22:37:57.654 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.654 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.654 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.655 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563910+0000", 2024-09-14T22:37:57.655 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073788753999999998 2024-09-14T22:37:57.655 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.655 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.655 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.655 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563988+0000", 2024-09-14T22:37:57.655 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7862999999999996e-05 2024-09-14T22:37:57.655 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.655 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.655 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.655 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564079+0000", 2024-09-14T22:37:57.655 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0389999999999993e-05 2024-09-14T22:37:57.655 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.655 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.656 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.656 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.656 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.656 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104197 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16318464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:57.656 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:37:57.656 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.083931649999997, 2024-09-14T22:37:57.656 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074103547000000006, 2024-09-14T22:37:57.656 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.656 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.656 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:57.656 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.656 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:57.656 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:57.657 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104197 2024-09-14T22:37:57.657 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.657 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.657 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.657 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.657 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:37:57.657 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.657 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.657 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.657 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.657 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:37:57.657 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.657 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.657 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.658 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.658 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490174+0000", 2024-09-14T22:37:57.658 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.37e-06 2024-09-14T22:37:57.658 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.658 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.658 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.658 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490197+0000", 2024-09-14T22:37:57.658 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3017e-05 2024-09-14T22:37:57.658 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.658 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.658 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.658 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490219+0000", 2024-09-14T22:37:57.658 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.181e-05 2024-09-14T22:37:57.658 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.659 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.659 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.659 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490240+0000", 2024-09-14T22:37:57.659 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0786000000000001e-05 2024-09-14T22:37:57.659 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.659 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.659 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.659 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564109+0000", 2024-09-14T22:37:57.659 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073869031000000002 2024-09-14T22:37:57.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.660 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.660 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564186+0000", 2024-09-14T22:37:57.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6823999999999995e-05 2024-09-14T22:37:57.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.660 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.660 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564276+0000", 2024-09-14T22:37:57.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0709000000000002e-05 2024-09-14T22:37:57.660 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.660 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.660 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.661 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104198 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16449536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:57.661 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:37:57.661 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.083839414000003, 2024-09-14T22:37:57.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074167693000000007, 2024-09-14T22:37:57.661 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.661 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.661 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:57.661 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.661 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:57.661 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:57.661 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104198 2024-09-14T22:37:57.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.662 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:37:57.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:37:57.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490266+0000", 2024-09-14T22:37:57.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3599999999999999e-06 2024-09-14T22:37:57.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.663 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490272+0000", 2024-09-14T22:37:57.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9599999999999997e-06 2024-09-14T22:37:57.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.663 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490291+0000", 2024-09-14T22:37:57.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8485999999999999e-05 2024-09-14T22:37:57.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.664 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.664 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490305+0000", 2024-09-14T22:37:57.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4327e-05 2024-09-14T22:37:57.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.664 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.664 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564306+0000", 2024-09-14T22:37:57.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074000745000000007 2024-09-14T22:37:57.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.664 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.664 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564367+0000", 2024-09-14T22:37:57.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1501999999999995e-05 2024-09-14T22:37:57.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564433+0000", 2024-09-14T22:37:57.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5313000000000003e-05 2024-09-14T22:37:57.665 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.665 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.665 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.665 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104199 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16580608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:57.665 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:37:57.665 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.083757861000002, 2024-09-14T22:37:57.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074231631000000006, 2024-09-14T22:37:57.666 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.666 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.666 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:57.666 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.666 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:57.666 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:57.666 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104199 2024-09-14T22:37:57.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.666 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:37:57.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:37:57.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490349+0000", 2024-09-14T22:37:57.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9429999999999999e-06 2024-09-14T22:37:57.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490355+0000", 2024-09-14T22:37:57.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1670000000000004e-06 2024-09-14T22:37:57.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490374+0000", 2024-09-14T22:37:57.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9613e-05 2024-09-14T22:37:57.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490405+0000", 2024-09-14T22:37:57.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0936000000000003e-05 2024-09-14T22:37:57.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564451+0000", 2024-09-14T22:37:57.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074045266999999998 2024-09-14T22:37:57.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564494+0000", 2024-09-14T22:37:57.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3133000000000002e-05 2024-09-14T22:37:57.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564578+0000", 2024-09-14T22:37:57.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4572000000000003e-05 2024-09-14T22:37:57.669 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.669 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.669 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.670 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104200 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:37:57.670 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:37:57.670 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.081962633000003, 2024-09-14T22:37:57.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072582435000000001, 2024-09-14T22:37:57.670 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:37:57.670 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:37:57.670 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:37:57.670 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:37:57.670 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:37:57.670 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:37:57.670 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104200 2024-09-14T22:37:57.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.671 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:37:57.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:37:57.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:37:57.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:37:57.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:37:57.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:37:57.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:37:57.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492144+0000", 2024-09-14T22:37:57.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3070000000000001e-06 2024-09-14T22:37:57.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:37:57.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492155+0000", 2024-09-14T22:37:57.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0573e-05 2024-09-14T22:37:57.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:37:57.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492185+0000", 2024-09-14T22:37:57.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.008e-05 2024-09-14T22:37:57.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:37:57.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492205+0000", 2024-09-14T22:37:57.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9882999999999999e-05 2024-09-14T22:37:57.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.673 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:37:57.673 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564609+0000", 2024-09-14T22:37:57.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072404181999999997 2024-09-14T22:37:57.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.673 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:37:57.673 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564667+0000", 2024-09-14T22:37:57.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8102e-05 2024-09-14T22:37:57.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:37:57.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:37:57.673 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:37:57.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564724+0000", 2024-09-14T22:37:57.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7308000000000001e-05 2024-09-14T22:37:57.674 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.674 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.674 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.674 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:37:57.674 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:37:57.674 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:37:57.675 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:57.578+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:57.675 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:37:57.729 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:57.722+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:57.750 INFO:teuthology.orchestra.run.smithi001.stdout:5373 write 0xcc180d8 thru 0xcc265f8 (0xe521 bytes) 2024-09-14T22:37:57.766 INFO:teuthology.orchestra.run.smithi001.stdout:5374 read 0xc0d9cf4 thru 0xc0e71a7 (0xd4b4 bytes) 2024-09-14T22:37:57.767 INFO:teuthology.orchestra.run.smithi001.stdout:5375 write 0x4afbfa0 thru 0x4b00f75 (0x4fd6 bytes) 2024-09-14T22:37:57.771 INFO:teuthology.orchestra.run.smithi001.stdout:5377 read 0xdb4ece thru 0xdb773f (0x2872 bytes) 2024-09-14T22:37:57.771 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:57.772 INFO:teuthology.orchestra.run.smithi001.stdout:5380 read 0x1f23978 thru 0x1f30edd (0xd566 bytes) 2024-09-14T22:37:57.772 INFO:teuthology.orchestra.run.smithi001.stdout:5381 write 0x5367895 thru 0x5375a56 (0xe1c2 bytes) 2024-09-14T22:37:57.781 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:57.774+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:57.786 INFO:teuthology.orchestra.run.smithi001.stdout:5382 trunc from 0xcc265f9 to 0xbefaa92 2024-09-14T22:37:57.808 INFO:teuthology.orchestra.run.smithi001.stdout:5383 read 0x63471ab thru 0x634c37b (0x51d1 bytes) 2024-09-14T22:37:57.808 INFO:teuthology.orchestra.run.smithi001.stdout:5387 punch from 0xaf87fbb to 0xaf945c1, (0xc606 bytes) 2024-09-14T22:37:57.812 INFO:teuthology.orchestra.run.smithi001.stdout:5389 trunc from 0xbefaa92 to 0x942ef1f 2024-09-14T22:37:57.838 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:57.838 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:37:57.838 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:37:57.839 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:57.854 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:37:57.855 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:37:57.901 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:57.898+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:57.982 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:57.978+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:58.041 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:37:58.041 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:37:58.041 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:37:58.041 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:37:58.041 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:37:58.058 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:37:58.061 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:37:58.062 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:37:58.082 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:37:58.082 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:37:58.083 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:58.078+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:58.116 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:37:58.117 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:37:58.146 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:37:58.146 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:37:58.168 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:37:58.169 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:37:58.184 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:58.178+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:58.192 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:37:58.192 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:37:58.206 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:58.206 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:58.214 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:37:58.214 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:37:58.221 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:37:58.236 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:37:58.236 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:37:58.301 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:58.298+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:58.379 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:58.379 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:58.385 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:58.378+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:58.397 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:37:58.486 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:37:58.482+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:37:58.587 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:58.587 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:58.587 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:58.582+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:58.601 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:37:58.688 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:58.682+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:58.758 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T22:37:58.784 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:58.785 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:58.789 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:58.782+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:58.800 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:37:58.890 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:37:58.882+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:37:58.966 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:58.966 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:58.980 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:37:58.991 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:58.986+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:59.092 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:59.086+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:59.105 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:59.105 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:59.119 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:37:59.193 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:37:59.186+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:37:59.245 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:59.245 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:59.259 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:37:59.301 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:59.298+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:59.395 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:59.390+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:37:59.421 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:59.421 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:59.435 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:37:59.521 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:37:59.518+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:37:59.597 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:37:59.590+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:37:59.606 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:59.606 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:59.620 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:37:59.698 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:59.694+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:59.728 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2024-09-14T22:37:59.745 INFO:teuthology.orchestra.run.smithi001.stdout:5390 writesame 0x8788830 thru 0x878c7b1 data_size 0xb(0x3f82 bytes) 2024-09-14T22:37:59.759 INFO:teuthology.orchestra.run.smithi001.stdout:5391 punch from 0x50b3fd4 to 0x50c288b, (0xe8b7 bytes) 2024-09-14T22:37:59.759 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:37:59.759 INFO:teuthology.orchestra.run.smithi001.stdout:5393 punch from 0x1bdfce1 to 0x1be4321, (0x4640 bytes) 2024-09-14T22:37:59.766 INFO:teuthology.orchestra.run.smithi001.stdout:5394 trunc from 0x942ef1f to 0x4a9fcb3 2024-09-14T22:37:59.792 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:59.793 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:59.799 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:37:59.794+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:37:59.808 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:37:59.900 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:37:59.894+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:37:59.937 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:37:59.937 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:37:59.952 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:38:00.001 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:37:59.998+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:00.102 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:00.098+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:00.119 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:00.119 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:00.133 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:38:00.202 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:00.198+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:00.255 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:00.255 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:00.256 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:38:00.269 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:38:00.305 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:00.302+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:00.394 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:00.394 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:00.404 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:00.398+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:00.408 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:38:00.410 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:38:00.410 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:38:00.426 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:38:00.427 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:38:00.444 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:38:00.444 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:38:00.460 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:38:00.461 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:38:00.478 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:38:00.478 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:38:00.496 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:38:00.496 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:38:00.506 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:00.502+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:00.516 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:38:00.516 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:38:00.529 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:00.529 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:00.529 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:38:00.529 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:38:00.530 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:00.537 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:38:00.537 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:38:00.543 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:38:00.607 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:00.598+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:00.664 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:00.664 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:38:00.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:38:00.665 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:38:00.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.665 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97197 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 796672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:00.665 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:38:00.665 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 50.589242061999997, 2024-09-14T22:38:00.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12434473, 2024-09-14T22:38:00.665 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:00.665 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:00.665 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:00.665 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:00.665 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:00.666 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:00.666 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97197 2024-09-14T22:38:00.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.666 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:00.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:00.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:38:00.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:00.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:38:00.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:00.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070524+0000", 2024-09-14T22:38:00.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0299999999999998e-07 2024-09-14T22:38:00.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:00.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070527+0000", 2024-09-14T22:38:00.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9569999999999998e-06 2024-09-14T22:38:00.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:00.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070535+0000", 2024-09-14T22:38:00.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7630000000000006e-06 2024-09-14T22:38:00.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:00.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070543+0000", 2024-09-14T22:38:00.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4869999999999996e-06 2024-09-14T22:38:00.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:00.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194525+0000", 2024-09-14T22:38:00.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123982096 2024-09-14T22:38:00.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:00.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194699+0000", 2024-09-14T22:38:00.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000174105 2024-09-14T22:38:00.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:00.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194868+0000", 2024-09-14T22:38:00.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016861900000000001 2024-09-14T22:38:00.669 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.669 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:00.669 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.670 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97198 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 805888~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:00.670 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:38:00.670 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 50.589213686000001, 2024-09-14T22:38:00.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.1248088, 2024-09-14T22:38:00.670 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:00.670 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:00.670 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:00.670 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:00.670 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:00.670 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:00.670 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97198 2024-09-14T22:38:00.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.670 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:00.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:00.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:38:00.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:00.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:38:00.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:00.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070552+0000", 2024-09-14T22:38:00.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T22:38:00.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:00.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070555+0000", 2024-09-14T22:38:00.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.853e-06 2024-09-14T22:38:00.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:00.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070565+0000", 2024-09-14T22:38:00.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.048e-05 2024-09-14T22:38:00.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.673 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:00.673 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070573+0000", 2024-09-14T22:38:00.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7200000000000006e-06 2024-09-14T22:38:00.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.673 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:00.673 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194954+0000", 2024-09-14T22:38:00.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12438089400000001 2024-09-14T22:38:00.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.673 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:00.673 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195145+0000", 2024-09-14T22:38:00.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019091299999999999 2024-09-14T22:38:00.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:00.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195360+0000", 2024-09-14T22:38:00.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021527300000000001 2024-09-14T22:38:00.674 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.674 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:00.674 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.674 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97199 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 815104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:00.674 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:38:00.674 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 50.589183089000002, 2024-09-14T22:38:00.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125251901, 2024-09-14T22:38:00.675 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:00.675 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:00.675 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:00.675 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:00.675 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:00.675 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:00.675 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97199 2024-09-14T22:38:00.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.675 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:00.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:00.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:38:00.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:00.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:38:00.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:00.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070583+0000", 2024-09-14T22:38:00.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6300000000000005e-07 2024-09-14T22:38:00.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:00.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070586+0000", 2024-09-14T22:38:00.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8270000000000002e-06 2024-09-14T22:38:00.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:00.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070597+0000", 2024-09-14T22:38:00.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1960000000000001e-05 2024-09-14T22:38:00.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:00.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070605+0000", 2024-09-14T22:38:00.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6459999999999993e-06 2024-09-14T22:38:00.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:00.678 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195432+0000", 2024-09-14T22:38:00.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124826926 2024-09-14T22:38:00.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:00.678 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195626+0000", 2024-09-14T22:38:00.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019357099999999999 2024-09-14T22:38:00.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:00.678 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195834+0000", 2024-09-14T22:38:00.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020830800000000001 2024-09-14T22:38:00.679 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.679 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:00.679 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.679 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97200 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 824320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:00.679 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:38:00.679 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 50.589151043000001, 2024-09-14T22:38:00.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125693477, 2024-09-14T22:38:00.679 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:00.679 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:00.679 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:00.679 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:00.680 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:00.680 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:00.680 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97200 2024-09-14T22:38:00.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.680 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:00.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:00.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:38:00.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:00.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:38:00.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:00.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070615+0000", 2024-09-14T22:38:00.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T22:38:00.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:00.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070618+0000", 2024-09-14T22:38:00.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.943e-06 2024-09-14T22:38:00.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:00.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070627+0000", 2024-09-14T22:38:00.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1099999999999992e-06 2024-09-14T22:38:00.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:00.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070634+0000", 2024-09-14T22:38:00.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:38:00.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:00.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195908+0000", 2024-09-14T22:38:00.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125273893 2024-09-14T22:38:00.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:00.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196097+0000", 2024-09-14T22:38:00.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018953899999999999 2024-09-14T22:38:00.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:00.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196308+0000", 2024-09-14T22:38:00.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021027799999999999 2024-09-14T22:38:00.683 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.683 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:00.683 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.684 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97201 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 833536~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:00.684 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:38:00.684 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 50.589120856000001, 2024-09-14T22:38:00.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126133826, 2024-09-14T22:38:00.684 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:00.684 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:00.684 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:00.684 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:00.684 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:00.684 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:00.684 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97201 2024-09-14T22:38:00.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.685 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:00.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:00.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:38:00.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:00.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:38:00.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:00.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074109+0000", 2024-09-14T22:38:00.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003464834 2024-09-14T22:38:00.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:00.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074134+0000", 2024-09-14T22:38:00.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5170000000000001e-05 2024-09-14T22:38:00.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:00.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074166+0000", 2024-09-14T22:38:00.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1347000000000003e-05 2024-09-14T22:38:00.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:00.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074179+0000", 2024-09-14T22:38:00.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.364e-05 2024-09-14T22:38:00.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:00.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196377+0000", 2024-09-14T22:38:00.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12219793900000001 2024-09-14T22:38:00.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:00.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196568+0000", 2024-09-14T22:38:00.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019097999999999999 2024-09-14T22:38:00.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:00.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196778+0000", 2024-09-14T22:38:00.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020991599999999999 2024-09-14T22:38:00.688 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.688 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:00.688 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.689 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97205 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 870400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:00.689 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:38:00.689 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 50.585451292999998, 2024-09-14T22:38:00.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12436148299999999, 2024-09-14T22:38:00.689 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:00.689 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:00.689 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:00.689 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:00.689 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:00.689 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:00.689 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97205 2024-09-14T22:38:00.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.689 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:00.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:00.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:38:00.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:00.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:38:00.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:00.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074315+0000", 2024-09-14T22:38:00.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9299999999999997e-07 2024-09-14T22:38:00.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:00.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074318+0000", 2024-09-14T22:38:00.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.19e-06 2024-09-14T22:38:00.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:00.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074327+0000", 2024-09-14T22:38:00.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8529999999999998e-06 2024-09-14T22:38:00.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.692 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:00.692 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074334+0000", 2024-09-14T22:38:00.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3869999999999997e-06 2024-09-14T22:38:00.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.692 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:00.692 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198365+0000", 2024-09-14T22:38:00.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124031222 2024-09-14T22:38:00.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.692 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:00.692 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198513+0000", 2024-09-14T22:38:00.693 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014772299999999999 2024-09-14T22:38:00.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.693 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:00.693 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198675+0000", 2024-09-14T22:38:00.693 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016241500000000001 2024-09-14T22:38:00.693 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.693 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:00.693 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.693 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97206 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 879616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:00.693 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:38:00.693 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 50.585419837000003, 2024-09-14T22:38:00.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124706284, 2024-09-14T22:38:00.694 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:00.694 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:00.694 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:00.694 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:00.694 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:00.694 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:00.694 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97206 2024-09-14T22:38:00.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.694 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:00.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.694 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:00.695 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:38:00.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.695 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.695 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:00.695 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:38:00.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.695 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.695 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:00.695 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074346+0000", 2024-09-14T22:38:00.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8999999999999996e-07 2024-09-14T22:38:00.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.696 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:00.696 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074349+0000", 2024-09-14T22:38:00.696 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.96e-06 2024-09-14T22:38:00.696 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.696 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:00.696 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074358+0000", 2024-09-14T22:38:00.696 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9169999999999995e-06 2024-09-14T22:38:00.696 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.696 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:00.696 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074365+0000", 2024-09-14T22:38:00.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3699999999999997e-06 2024-09-14T22:38:00.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:00.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198761+0000", 2024-09-14T22:38:00.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124395743 2024-09-14T22:38:00.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:00.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198899+0000", 2024-09-14T22:38:00.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013763499999999999 2024-09-14T22:38:00.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:00.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199052+0000", 2024-09-14T22:38:00.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015296900000000001 2024-09-14T22:38:00.698 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.698 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:00.698 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.698 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97207 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 888832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:00.698 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:38:00.698 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 50.585390623999999, 2024-09-14T22:38:00.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124988433, 2024-09-14T22:38:00.699 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:00.699 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:00.699 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:00.699 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:00.699 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:00.699 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:00.699 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97207 2024-09-14T22:38:00.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.699 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:00.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:00.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:38:00.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.700 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:00.700 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:38:00.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.700 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:00.700 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074375+0000", 2024-09-14T22:38:00.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:38:00.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.700 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:00.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074378+0000", 2024-09-14T22:38:00.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1829999999999998e-06 2024-09-14T22:38:00.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:00.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074389+0000", 2024-09-14T22:38:00.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0737000000000001e-05 2024-09-14T22:38:00.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:00.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074396+0000", 2024-09-14T22:38:00.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2559999999999996e-06 2024-09-14T22:38:00.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:00.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199139+0000", 2024-09-14T22:38:00.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12474253 2024-09-14T22:38:00.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:00.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199240+0000", 2024-09-14T22:38:00.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101221 2024-09-14T22:38:00.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:00.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199363+0000", 2024-09-14T22:38:00.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000122822 2024-09-14T22:38:00.702 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.703 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:00.703 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.703 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97208 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 898048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:00.703 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:38:00.703 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 50.585359320000002, 2024-09-14T22:38:00.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12524854899999999, 2024-09-14T22:38:00.703 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:00.703 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:00.703 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:00.703 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:00.703 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:00.704 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:00.704 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97208 2024-09-14T22:38:00.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.704 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:00.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:00.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:38:00.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:00.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:38:00.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:00.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:38:00.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:38:00.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:00.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074410+0000", 2024-09-14T22:38:00.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.18e-06 2024-09-14T22:38:00.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:00.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074435+0000", 2024-09-14T22:38:00.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5273e-05 2024-09-14T22:38:00.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:00.706 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074443+0000", 2024-09-14T22:38:00.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3000000000000002e-06 2024-09-14T22:38:00.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:00.706 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199433+0000", 2024-09-14T22:38:00.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124989502 2024-09-14T22:38:00.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:00.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199535+0000", 2024-09-14T22:38:00.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102557 2024-09-14T22:38:00.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:00.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199654+0000", 2024-09-14T22:38:00.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119061 2024-09-14T22:38:00.707 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.707 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:00.707 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.707 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97209 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 907264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:00.707 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:38:00.708 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 50.585311515000001, 2024-09-14T22:38:00.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125487354, 2024-09-14T22:38:00.708 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:00.708 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:00.708 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:00.708 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:00.708 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:00.708 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:00.708 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97209 2024-09-14T22:38:00.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.708 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:00.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:00.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:38:00.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:00.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:38:00.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:00.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:38:00.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9400000000000005e-07 2024-09-14T22:38:00.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:00.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074458+0000", 2024-09-14T22:38:00.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3299999999999999e-06 2024-09-14T22:38:00.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:00.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074467+0000", 2024-09-14T22:38:00.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.346e-06 2024-09-14T22:38:00.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:00.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074474+0000", 2024-09-14T22:38:00.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.374e-06 2024-09-14T22:38:00.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:00.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199720+0000", 2024-09-14T22:38:00.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12524561200000001 2024-09-14T22:38:00.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:00.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199820+0000", 2024-09-14T22:38:00.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010049599999999999 2024-09-14T22:38:00.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:00.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199941+0000", 2024-09-14T22:38:00.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012050200000000001 2024-09-14T22:38:00.712 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.712 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:00.712 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.712 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97210 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:00.712 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:38:00.712 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 50.585281520999999, 2024-09-14T22:38:00.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125744942, 2024-09-14T22:38:00.712 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:00.712 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:00.713 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:00.713 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:00.713 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:00.713 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:00.713 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97210 2024-09-14T22:38:00.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.713 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:00.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:00.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:38:00.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:00.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:38:00.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:00.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:38:00.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6599999999999996e-07 2024-09-14T22:38:00.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:00.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074487+0000", 2024-09-14T22:38:00.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8140000000000001e-06 2024-09-14T22:38:00.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:00.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074496+0000", 2024-09-14T22:38:00.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9330000000000007e-06 2024-09-14T22:38:00.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:00.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074503+0000", 2024-09-14T22:38:00.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4100000000000002e-06 2024-09-14T22:38:00.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:00.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200009+0000", 2024-09-14T22:38:00.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12550539899999999 2024-09-14T22:38:00.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.716 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:00.716 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200108+0000", 2024-09-14T22:38:00.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9182e-05 2024-09-14T22:38:00.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.716 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:00.716 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200229+0000", 2024-09-14T22:38:00.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120538 2024-09-14T22:38:00.716 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.716 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:00.716 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.717 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.717 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.717 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97211 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 925696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:00.717 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:38:00.717 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 50.585250561000002, 2024-09-14T22:38:00.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12599848299999999, 2024-09-14T22:38:00.717 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:00.717 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:00.717 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:00.717 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:00.717 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:00.717 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:00.717 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97211 2024-09-14T22:38:00.717 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.718 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:00.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:00.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:38:00.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:00.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:38:00.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:00.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074516+0000", 2024-09-14T22:38:00.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.42e-06 2024-09-14T22:38:00.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:00.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074521+0000", 2024-09-14T22:38:00.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7960000000000003e-06 2024-09-14T22:38:00.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:00.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074538+0000", 2024-09-14T22:38:00.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6900000000000001e-05 2024-09-14T22:38:00.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:00.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074555+0000", 2024-09-14T22:38:00.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.696e-05 2024-09-14T22:38:00.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:00.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200293+0000", 2024-09-14T22:38:00.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12573864100000001 2024-09-14T22:38:00.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:00.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200393+0000", 2024-09-14T22:38:00.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9264000000000004e-05 2024-09-14T22:38:00.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.721 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:00.721 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200513+0000", 2024-09-14T22:38:00.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012050200000000001 2024-09-14T22:38:00.721 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.721 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:00.721 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.721 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97212 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 934912~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:00.721 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:38:00.721 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 50.585175939000003, 2024-09-14T22:38:00.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12621127300000001, 2024-09-14T22:38:00.721 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:00.722 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:00.722 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:00.722 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:00.722 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:00.722 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:00.722 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97212 2024-09-14T22:38:00.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.722 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:00.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.722 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:00.722 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:38:00.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:00.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:38:00.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:00.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074590+0000", 2024-09-14T22:38:00.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1969999999999999e-06 2024-09-14T22:38:00.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:00.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074596+0000", 2024-09-14T22:38:00.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7629999999999997e-06 2024-09-14T22:38:00.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:00.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074614+0000", 2024-09-14T22:38:00.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7643e-05 2024-09-14T22:38:00.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:00.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074635+0000", 2024-09-14T22:38:00.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.075e-05 2024-09-14T22:38:00.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:00.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200580+0000", 2024-09-14T22:38:00.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125945482 2024-09-14T22:38:00.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:00.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200683+0000", 2024-09-14T22:38:00.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010267099999999999 2024-09-14T22:38:00.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.726 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:00.726 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200800+0000", 2024-09-14T22:38:00.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117767 2024-09-14T22:38:00.726 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.726 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:00.726 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.726 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97213 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 944128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:00.726 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:38:00.726 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 50.585110589000003, 2024-09-14T22:38:00.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126430709, 2024-09-14T22:38:00.726 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:00.726 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:00.727 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:00.727 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:00.727 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:00.727 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:00.727 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97213 2024-09-14T22:38:00.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.727 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:00.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:00.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:38:00.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.728 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:00.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:38:00.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.728 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:00.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078210+0000", 2024-09-14T22:38:00.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0035556839999999999 2024-09-14T22:38:00.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.728 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:00.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078231+0000", 2024-09-14T22:38:00.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0716e-05 2024-09-14T22:38:00.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:00.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078260+0000", 2024-09-14T22:38:00.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9473000000000001e-05 2024-09-14T22:38:00.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:00.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078279+0000", 2024-09-14T22:38:00.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8600000000000001e-05 2024-09-14T22:38:00.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:00.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200864+0000", 2024-09-14T22:38:00.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.122585295 2024-09-14T22:38:00.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:00.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200965+0000", 2024-09-14T22:38:00.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101004 2024-09-14T22:38:00.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:00.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.201085+0000", 2024-09-14T22:38:00.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119937 2024-09-14T22:38:00.730 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.730 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:00.730 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.731 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97219 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 999424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:00.731 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:38:00.731 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 50.581128311999997, 2024-09-14T22:38:00.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124138558, 2024-09-14T22:38:00.731 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:00.731 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:00.731 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:00.731 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:00.731 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:00.731 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:00.731 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97219 2024-09-14T22:38:00.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.732 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:00.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:00.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:38:00.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:00.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:38:00.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:00.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078638+0000", 2024-09-14T22:38:00.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.173e-06 2024-09-14T22:38:00.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:00.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078644+0000", 2024-09-14T22:38:00.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2570000000000004e-06 2024-09-14T22:38:00.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:00.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078672+0000", 2024-09-14T22:38:00.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7322999999999999e-05 2024-09-14T22:38:00.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:00.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078687+0000", 2024-09-14T22:38:00.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5639999999999999e-05 2024-09-14T22:38:00.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:00.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202567+0000", 2024-09-14T22:38:00.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123880134 2024-09-14T22:38:00.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:00.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202670+0000", 2024-09-14T22:38:00.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010273899999999999 2024-09-14T22:38:00.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:00.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202775+0000", 2024-09-14T22:38:00.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010529199999999999 2024-09-14T22:38:00.735 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.735 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:00.735 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.735 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97220 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1008640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:00.735 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:38:00.735 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 50.581059078999999, 2024-09-14T22:38:00.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12435937, 2024-09-14T22:38:00.736 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:00.736 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:00.736 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:00.736 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:00.736 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:00.736 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:00.736 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97220 2024-09-14T22:38:00.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.736 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:00.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:00.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:38:00.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:00.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:38:00.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:00.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078707+0000", 2024-09-14T22:38:00.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.266e-06 2024-09-14T22:38:00.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:00.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078713+0000", 2024-09-14T22:38:00.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7270000000000004e-06 2024-09-14T22:38:00.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:00.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078737+0000", 2024-09-14T22:38:00.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3640000000000001e-05 2024-09-14T22:38:00.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:00.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078751+0000", 2024-09-14T22:38:00.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.45e-05 2024-09-14T22:38:00.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:00.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202841+0000", 2024-09-14T22:38:00.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124089938 2024-09-14T22:38:00.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:00.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202944+0000", 2024-09-14T22:38:00.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010334 2024-09-14T22:38:00.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:00.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203065+0000", 2024-09-14T22:38:00.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012095900000000001 2024-09-14T22:38:00.740 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.740 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:00.740 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.740 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97221 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1017856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:00.740 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:38:00.740 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 50.580993315999997, 2024-09-14T22:38:00.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124574005, 2024-09-14T22:38:00.740 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:00.740 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:00.741 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:00.741 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:00.741 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:00.741 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:00.741 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97221 2024-09-14T22:38:00.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.741 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:00.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:00.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:38:00.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:00.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:38:00.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:00.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078773+0000", 2024-09-14T22:38:00.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.13e-06 2024-09-14T22:38:00.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:00.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078779+0000", 2024-09-14T22:38:00.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0260000000000004e-06 2024-09-14T22:38:00.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.743 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:00.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078797+0000", 2024-09-14T22:38:00.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.806e-05 2024-09-14T22:38:00.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.743 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:00.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078810+0000", 2024-09-14T22:38:00.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.308e-05 2024-09-14T22:38:00.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.744 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:00.744 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203131+0000", 2024-09-14T22:38:00.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12432077499999999 2024-09-14T22:38:00.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.744 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:00.744 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203227+0000", 2024-09-14T22:38:00.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6575000000000003e-05 2024-09-14T22:38:00.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.744 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:00.744 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203346+0000", 2024-09-14T22:38:00.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118359 2024-09-14T22:38:00.744 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.745 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:00.745 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.745 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97222 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:00.745 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:38:00.745 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 50.580936946999998, 2024-09-14T22:38:00.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12480110799999999, 2024-09-14T22:38:00.745 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:00.745 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:00.745 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:00.745 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:00.745 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:00.746 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:00.746 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97222 2024-09-14T22:38:00.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.746 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:00.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:00.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:38:00.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:00.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:38:00.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:00.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078829+0000", 2024-09-14T22:38:00.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.057e-06 2024-09-14T22:38:00.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:00.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078836+0000", 2024-09-14T22:38:00.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6159999999999998e-06 2024-09-14T22:38:00.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:00.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078856+0000", 2024-09-14T22:38:00.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0100000000000001e-05 2024-09-14T22:38:00.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:00.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078869+0000", 2024-09-14T22:38:00.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.272e-05 2024-09-14T22:38:00.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:00.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203412+0000", 2024-09-14T22:38:00.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12454283300000001 2024-09-14T22:38:00.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:00.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203511+0000", 2024-09-14T22:38:00.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9893000000000001e-05 2024-09-14T22:38:00.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:00.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203629+0000", 2024-09-14T22:38:00.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117889 2024-09-14T22:38:00.749 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.749 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:00.749 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.750 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97223 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1036288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:00.750 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:38:00.750 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 50.580564959999997, 2024-09-14T22:38:00.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124706279, 2024-09-14T22:38:00.750 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:00.750 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:00.750 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:00.750 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:00.750 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:00.750 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:00.750 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97223 2024-09-14T22:38:00.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.751 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:00.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:00.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:38:00.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:00.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:38:00.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:00.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079202+0000", 2024-09-14T22:38:00.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8330000000000001e-06 2024-09-14T22:38:00.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:00.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079212+0000", 2024-09-14T22:38:00.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0207000000000001e-05 2024-09-14T22:38:00.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:00.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079238+0000", 2024-09-14T22:38:00.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5626000000000002e-05 2024-09-14T22:38:00.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.753 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:00.753 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079256+0000", 2024-09-14T22:38:00.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8386999999999999e-05 2024-09-14T22:38:00.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.753 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:00.753 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203691+0000", 2024-09-14T22:38:00.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124434538 2024-09-14T22:38:00.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.753 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:00.753 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203786+0000", 2024-09-14T22:38:00.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5344000000000001e-05 2024-09-14T22:38:00.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:00.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203906+0000", 2024-09-14T22:38:00.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120344 2024-09-14T22:38:00.754 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.754 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:00.754 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.754 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97224 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1045504~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:00.754 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:38:00.754 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 50.580481140000003, 2024-09-14T22:38:00.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12513321199999999, 2024-09-14T22:38:00.754 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:00.755 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:00.755 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:00.755 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:00.755 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:00.755 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:00.755 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97224 2024-09-14T22:38:00.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.755 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:00.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:00.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:38:00.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:00.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:38:00.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:00.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:00.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082581+0000", 2024-09-14T22:38:00.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0032971530000000001 2024-09-14T22:38:00.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:00.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082592+0000", 2024-09-14T22:38:00.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1345999999999999e-05 2024-09-14T22:38:00.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:00.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082618+0000", 2024-09-14T22:38:00.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5873000000000001e-05 2024-09-14T22:38:00.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:00.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082638+0000", 2024-09-14T22:38:00.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9565999999999999e-05 2024-09-14T22:38:00.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.758 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:00.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204004+0000", 2024-09-14T22:38:00.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.121366038 2024-09-14T22:38:00.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.758 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:00.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204188+0000", 2024-09-14T22:38:00.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018427 2024-09-14T22:38:00.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:00.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:00.758 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:00.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204417+0000", 2024-09-14T22:38:00.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022896599999999999 2024-09-14T22:38:00.758 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.759 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:00.759 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.759 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:00.759 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:00.759 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:00.760 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:38:00.761 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:00.754+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:00.769 INFO:teuthology.orchestra.run.smithi001.stdout:5396 punch from 0xb138d6 to 0xb1cf5d, (0x9687 bytes) 2024-09-14T22:38:00.794 INFO:teuthology.orchestra.run.smithi001.stdout:5397 write 0x895716b thru 0x8965c2c (0xeac2 bytes) 2024-09-14T22:38:00.804 INFO:teuthology.orchestra.run.smithi001.stdout:5399 read 0x242e72d thru 0x243cc7a (0xe54e bytes) 2024-09-14T22:38:00.833 INFO:teuthology.orchestra.run.smithi001.stdout:5400 writesame 0xe1d5ce5 thru 0xe1de3dd data_size 0x1(0x86f9 bytes) 2024-09-14T22:38:00.833 INFO:teuthology.orchestra.run.smithi001.stdout:5401 write 0x22a478a thru 0x22ab63a (0x6eb1 bytes) 2024-09-14T22:38:00.856 INFO:teuthology.orchestra.run.smithi001.stdout:5402 write 0xe9cf901 thru 0xe9db2e8 (0xb9e8 bytes) 2024-09-14T22:38:00.861 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:00.854+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:00.864 INFO:teuthology.orchestra.run.smithi001.stdout:5403 write 0xb1cf45 thru 0xb24bbb (0x7c77 bytes) 2024-09-14T22:38:00.873 INFO:teuthology.orchestra.run.smithi001.stdout:5405 write 0x28f3c1a thru 0x28f93e4 (0x57cb bytes) 2024-09-14T22:38:00.876 INFO:teuthology.orchestra.run.smithi001.stdout:5406 read 0x116f61b thru 0x1173e8a (0x4870 bytes) 2024-09-14T22:38:00.877 INFO:teuthology.orchestra.run.smithi001.stdout:5407 punch from 0xd571ad0 to 0xd57c3c9, (0xa8f9 bytes) 2024-09-14T22:38:00.878 INFO:teuthology.orchestra.run.smithi001.stdout:5408 read 0xd8ce056 thru 0xd8d61cb (0x8176 bytes) 2024-09-14T22:38:00.878 INFO:teuthology.orchestra.run.smithi001.stdout:5409 writesame 0x76b642d thru 0x76c5a0c data_size 0x70(0xf5e0 bytes) 2024-09-14T22:38:00.886 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:00.887 INFO:teuthology.orchestra.run.smithi001.stdout:5411 write 0xc59cbee thru 0xc5aa772 (0xdb85 bytes) 2024-09-14T22:38:00.913 INFO:teuthology.orchestra.run.smithi001.stdout:5412 trunc from 0xe9db2e9 to 0x82dceb6 2024-09-14T22:38:00.937 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:00.937 INFO:teuthology.orchestra.run.smithi001.stdout:5414 read 0x697b6a8 thru 0x697f150 (0x3aa9 bytes) 2024-09-14T22:38:00.942 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:00.942 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:00.942 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:38:00.943 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:00.953 INFO:teuthology.orchestra.run.smithi001.stdout:5415 trunc from 0x82dceb6 to 0x25d1b63 2024-09-14T22:38:00.957 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:38:00.962 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:00.958+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:01.017 INFO:teuthology.orchestra.run.smithi001.stdout:5417 write 0xaabba7b thru 0xaac7ac7 (0xc04d bytes) 2024-09-14T22:38:01.037 INFO:teuthology.orchestra.run.smithi001.stdout:5418 write 0xb29b802 thru 0xb2a7c9c (0xc49b bytes) 2024-09-14T22:38:01.064 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:01.058+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:01.066 INFO:teuthology.orchestra.run.smithi001.stdout:5419 write 0xe5f46c8 thru 0xe600bdf (0xc518 bytes) 2024-09-14T22:38:01.074 INFO:teuthology.orchestra.run.smithi001.stdout:5420 read 0x60be4f1 thru 0x60ca42d (0xbf3d bytes) 2024-09-14T22:38:01.082 INFO:teuthology.orchestra.run.smithi001.stdout:5422 trunc from 0xe600be0 to 0xbb75556 2024-09-14T22:38:01.103 INFO:teuthology.orchestra.run.smithi001.stdout:5423 read 0x7573229 thru 0x7575e8c (0x2c64 bytes) 2024-09-14T22:38:01.103 INFO:teuthology.orchestra.run.smithi001.stdout:5424 writesame 0x9d6fb4f thru 0x9d74e76 data_size 0xc(0x5328 bytes) 2024-09-14T22:38:01.112 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:01.112 INFO:teuthology.orchestra.run.smithi001.stdout:5426 write 0x587bba3 thru 0x5881159 (0x55b7 bytes) 2024-09-14T22:38:01.123 INFO:teuthology.orchestra.run.smithi001.stdout:5427 punch from 0x1b03882 to 0x1b087f2, (0x4f70 bytes) 2024-09-14T22:38:01.123 INFO:teuthology.orchestra.run.smithi001.stdout:5428 read 0xaad56c4 thru 0xaad7578 (0x1eb5 bytes) 2024-09-14T22:38:01.123 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:01.124 INFO:teuthology.orchestra.run.smithi001.stdout:5430 read 0xa1a1f1 thru 0xa1be81 (0x1c91 bytes) 2024-09-14T22:38:01.124 INFO:teuthology.orchestra.run.smithi001.stdout:5431 write 0x2721f64 thru 0x2729bd7 (0x7c74 bytes) 2024-09-14T22:38:01.128 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:01.135 INFO:teuthology.orchestra.run.smithi001.stdout:5433 trunc from 0xbb75556 to 0x98054ba 2024-09-14T22:38:01.137 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:01.137 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:01.137 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:38:01.137 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:38:01.137 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:01.154 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:38:01.165 INFO:teuthology.orchestra.run.smithi001.stdout:5434 trunc from 0x98054ba to 0xac2c31e 2024-09-14T22:38:01.177 INFO:teuthology.orchestra.run.smithi001.stdout:5435 read 0xa092c55 thru 0xa0941a6 (0x1552 bytes) 2024-09-14T22:38:01.188 INFO:teuthology.orchestra.run.smithi001.stdout:5437 trunc from 0xac2c31e to 0xed18628 2024-09-14T22:38:01.199 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:01.194+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:01.211 INFO:teuthology.orchestra.run.smithi001.stdout:5438 trunc from 0xed18628 to 0xb1a0619 2024-09-14T22:38:01.224 INFO:teuthology.orchestra.run.smithi001.stdout:5439 writesame 0x8fc4a0f thru 0x8fc556b data_size 0x1(0xb5d bytes) 2024-09-14T22:38:01.231 INFO:teuthology.orchestra.run.smithi001.stdout:5440 writesame 0x14002bf thru 0x140df26 data_size 0xc(0xdc68 bytes) 2024-09-14T22:38:01.237 INFO:teuthology.orchestra.run.smithi001.stdout:5441 writesame 0x44a8af2 thru 0x44a8f50 data_size 0x1(0x45f bytes) 2024-09-14T22:38:01.244 INFO:teuthology.orchestra.run.smithi001.stdout:5443 read 0x5e3c76d thru 0x5e4254f (0x5de3 bytes) 2024-09-14T22:38:01.245 INFO:teuthology.orchestra.run.smithi001.stdout:5445 write 0x31f4036 thru 0x31f7a4f (0x3a1a bytes) 2024-09-14T22:38:01.251 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:01.261 INFO:teuthology.orchestra.run.smithi001.stdout:5447 trunc from 0xb1a0619 to 0xe64bb0f 2024-09-14T22:38:01.265 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:01.262+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:01.275 INFO:teuthology.orchestra.run.smithi001.stdout:5448 write 0x91474ba thru 0x91515a1 (0xa0e8 bytes) 2024-09-14T22:38:01.287 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:01.287 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:38:01.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:38:01.288 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:38:01.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.288 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:72252 3.2 3:442c6d72:::rbd_data.113cb4c83968.0000000000000015:head [zero 4456448~393216] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-14T22:38:01.288 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:38:01.288 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 70.582962754999997, 2024-09-14T22:38:01.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0176591500000001, 2024-09-14T22:38:01.288 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.288 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.288 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T22:38:01.288 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.288 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.288 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.288 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 72252 2024-09-14T22:38:01.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.289 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:38:01.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:38:01.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700101+0000", 2024-09-14T22:38:01.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4570000000000001e-06 2024-09-14T22:38:01.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700114+0000", 2024-09-14T22:38:01.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2799999999999999e-05 2024-09-14T22:38:01.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700146+0000", 2024-09-14T22:38:01.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2116000000000002e-05 2024-09-14T22:38:01.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700170+0000", 2024-09-14T22:38:01.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3770000000000001e-05 2024-09-14T22:38:01.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700285+0000", 2024-09-14T22:38:01.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000115228 2024-09-14T22:38:01.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op must wait for map", 2024-09-14T22:38:01.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700297+0000", 2024-09-14T22:38:01.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1681999999999999e-05 2024-09-14T22:38:01.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.703312+0000", 2024-09-14T22:38:01.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003015046 2024-09-14T22:38:01.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:38:01.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.703327+0000", 2024-09-14T22:38:01.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5481999999999998e-05 2024-09-14T22:38:01.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:51.717566+0000", 2024-09-14T22:38:01.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0142383509999999 2024-09-14T22:38:01.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:51.717757+0000", 2024-09-14T22:38:01.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000191218 2024-09-14T22:38:01.293 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.293 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.293 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.293 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:74156 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T22:38:01.293 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:38:01.293 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 65.768494044999997, 2024-09-14T22:38:01.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062073521, 2024-09-14T22:38:01.294 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.294 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.294 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:01.294 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.294 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.294 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.294 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74156 2024-09-14T22:38:01.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.294 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:38:01.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:38:01.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514569+0000", 2024-09-14T22:38:01.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0229999999999999e-06 2024-09-14T22:38:01.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514596+0000", 2024-09-14T22:38:01.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6903e-05 2024-09-14T22:38:01.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.296 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514623+0000", 2024-09-14T22:38:01.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7206e-05 2024-09-14T22:38:01.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.296 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514637+0000", 2024-09-14T22:38:01.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:38:01.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514781+0000", 2024-09-14T22:38:01.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014394899999999999 2024-09-14T22:38:01.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.515005+0000", 2024-09-14T22:38:01.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022458499999999999 2024-09-14T22:38:01.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:38:01.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.527785+0000", 2024-09-14T22:38:01.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012780112 2024-09-14T22:38:01.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:38:01.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576209+0000", 2024-09-14T22:38:01.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.048423992999999999 2024-09-14T22:38:01.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:01.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576346+0000", 2024-09-14T22:38:01.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000136909 2024-09-14T22:38:01.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:38:01.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576517+0000", 2024-09-14T22:38:01.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000170848 2024-09-14T22:38:01.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.299 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576640+0000", 2024-09-14T22:38:01.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000122953 2024-09-14T22:38:01.299 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.299 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.299 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.299 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82513 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 557056~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:01.299 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:38:01.299 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 61.863862226000002, 2024-09-14T22:38:01.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059345486000000003, 2024-09-14T22:38:01.300 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.300 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.300 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.300 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.300 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.300 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.300 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82513 2024-09-14T22:38:01.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.300 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.300 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:38:01.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:38:01.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419200+0000", 2024-09-14T22:38:01.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.184e-06 2024-09-14T22:38:01.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419207+0000", 2024-09-14T22:38:01.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2690000000000002e-06 2024-09-14T22:38:01.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419224+0000", 2024-09-14T22:38:01.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6623e-05 2024-09-14T22:38:01.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419236+0000", 2024-09-14T22:38:01.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2367e-05 2024-09-14T22:38:01.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.303 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478294+0000", 2024-09-14T22:38:01.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059057927000000003 2024-09-14T22:38:01.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.303 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.303 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478402+0000", 2024-09-14T22:38:01.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107921 2024-09-14T22:38:01.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.303 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.303 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478544+0000", 2024-09-14T22:38:01.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014219499999999999 2024-09-14T22:38:01.303 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.303 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.304 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.304 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82514 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 704512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:01.304 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:38:01.304 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 61.863803122999997, 2024-09-14T22:38:01.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059580793, 2024-09-14T22:38:01.304 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.304 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.304 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.304 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.304 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.304 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.305 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82514 2024-09-14T22:38:01.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.305 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:38:01.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:38:01.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419259+0000", 2024-09-14T22:38:01.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.353e-06 2024-09-14T22:38:01.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419276+0000", 2024-09-14T22:38:01.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7567e-05 2024-09-14T22:38:01.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419294+0000", 2024-09-14T22:38:01.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.713e-05 2024-09-14T22:38:01.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419306+0000", 2024-09-14T22:38:01.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.279e-05 2024-09-14T22:38:01.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478589+0000", 2024-09-14T22:38:01.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059282978 2024-09-14T22:38:01.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478694+0000", 2024-09-14T22:38:01.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000104561 2024-09-14T22:38:01.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.308 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478838+0000", 2024-09-14T22:38:01.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014441399999999999 2024-09-14T22:38:01.308 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.308 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.308 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.308 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82515 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:01.308 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:38:01.308 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 61.863737972999999, 2024-09-14T22:38:01.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059815123999999997, 2024-09-14T22:38:01.308 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.308 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.309 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.309 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.309 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.309 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.309 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82515 2024-09-14T22:38:01.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.309 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:38:01.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:38:01.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419324+0000", 2024-09-14T22:38:01.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2130000000000001e-06 2024-09-14T22:38:01.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419331+0000", 2024-09-14T22:38:01.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4730000000000003e-06 2024-09-14T22:38:01.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419348+0000", 2024-09-14T22:38:01.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6269999999999998e-05 2024-09-14T22:38:01.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419360+0000", 2024-09-14T22:38:01.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2243e-05 2024-09-14T22:38:01.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478884+0000", 2024-09-14T22:38:01.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059524369000000001 2024-09-14T22:38:01.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.312 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478990+0000", 2024-09-14T22:38:01.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010569 2024-09-14T22:38:01.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.312 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479138+0000", 2024-09-14T22:38:01.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000147866 2024-09-14T22:38:01.312 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.312 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.312 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.313 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82516 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 999424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:01.313 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:38:01.313 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 61.863685468, 2024-09-14T22:38:01.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061979595999999998, 2024-09-14T22:38:01.313 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.313 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.313 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.313 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.313 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.313 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.313 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82516 2024-09-14T22:38:01.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.313 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.314 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:38:01.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.314 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:38:01.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.314 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419376+0000", 2024-09-14T22:38:01.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.237e-06 2024-09-14T22:38:01.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419383+0000", 2024-09-14T22:38:01.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8170000000000001e-06 2024-09-14T22:38:01.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419400+0000", 2024-09-14T22:38:01.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6636000000000001e-05 2024-09-14T22:38:01.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419412+0000", 2024-09-14T22:38:01.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2357e-05 2024-09-14T22:38:01.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481080+0000", 2024-09-14T22:38:01.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061668032999999997 2024-09-14T22:38:01.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481193+0000", 2024-09-14T22:38:01.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112335 2024-09-14T22:38:01.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481355+0000", 2024-09-14T22:38:01.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016218100000000001 2024-09-14T22:38:01.317 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.317 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.317 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.317 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82538 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1097728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:01.317 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:38:01.317 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 61.863633397999997, 2024-09-14T22:38:01.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059971705, 2024-09-14T22:38:01.317 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.317 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.317 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.317 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.317 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.318 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.318 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82538 2024-09-14T22:38:01.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.318 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:38:01.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:38:01.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419429+0000", 2024-09-14T22:38:01.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.497e-06 2024-09-14T22:38:01.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419436+0000", 2024-09-14T22:38:01.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2099999999999996e-06 2024-09-14T22:38:01.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419453+0000", 2024-09-14T22:38:01.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6840000000000001e-05 2024-09-14T22:38:01.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419465+0000", 2024-09-14T22:38:01.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2283000000000001e-05 2024-09-14T22:38:01.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479182+0000", 2024-09-14T22:38:01.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059717339000000001 2024-09-14T22:38:01.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479297+0000", 2024-09-14T22:38:01.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000114259 2024-09-14T22:38:01.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479399+0000", 2024-09-14T22:38:01.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102277 2024-09-14T22:38:01.321 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.321 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.321 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.321 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82539 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:01.321 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:38:01.321 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 61.863580548000002, 2024-09-14T22:38:01.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060166935999999997, 2024-09-14T22:38:01.322 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.322 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.322 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.322 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.322 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.322 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.322 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82539 2024-09-14T22:38:01.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.322 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:38:01.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:38:01.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419482+0000", 2024-09-14T22:38:01.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4130000000000001e-06 2024-09-14T22:38:01.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419488+0000", 2024-09-14T22:38:01.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8460000000000002e-06 2024-09-14T22:38:01.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419505+0000", 2024-09-14T22:38:01.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6477000000000002e-05 2024-09-14T22:38:01.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419518+0000", 2024-09-14T22:38:01.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2797e-05 2024-09-14T22:38:01.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479438+0000", 2024-09-14T22:38:01.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059919897 2024-09-14T22:38:01.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479547+0000", 2024-09-14T22:38:01.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109807 2024-09-14T22:38:01.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479647+0000", 2024-09-14T22:38:01.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9698999999999995e-05 2024-09-14T22:38:01.325 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.325 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.326 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.326 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82542 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:01.326 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:38:01.326 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 61.863528105, 2024-09-14T22:38:01.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060367430999999999, 2024-09-14T22:38:01.326 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.326 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.326 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.326 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.326 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.326 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.326 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82542 2024-09-14T22:38:01.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.327 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:38:01.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:38:01.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.328 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419534+0000", 2024-09-14T22:38:01.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T22:38:01.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.328 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419542+0000", 2024-09-14T22:38:01.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8199999999999997e-06 2024-09-14T22:38:01.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419559+0000", 2024-09-14T22:38:01.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6852999999999999e-05 2024-09-14T22:38:01.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419571+0000", 2024-09-14T22:38:01.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2364000000000001e-05 2024-09-14T22:38:01.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479686+0000", 2024-09-14T22:38:01.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060115522999999997 2024-09-14T22:38:01.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479795+0000", 2024-09-14T22:38:01.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010831399999999999 2024-09-14T22:38:01.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479900+0000", 2024-09-14T22:38:01.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105284 2024-09-14T22:38:01.330 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.330 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.330 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.331 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82544 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:01.331 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:38:01.331 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 61.863474369000002, 2024-09-14T22:38:01.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060590515999999997, 2024-09-14T22:38:01.331 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.331 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.331 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.331 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.331 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.331 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.331 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82544 2024-09-14T22:38:01.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.331 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:38:01.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:38:01.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419588+0000", 2024-09-14T22:38:01.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.35e-06 2024-09-14T22:38:01.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419595+0000", 2024-09-14T22:38:01.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4030000000000001e-06 2024-09-14T22:38:01.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419612+0000", 2024-09-14T22:38:01.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6670000000000001e-05 2024-09-14T22:38:01.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419624+0000", 2024-09-14T22:38:01.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-05 2024-09-14T22:38:01.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479957+0000", 2024-09-14T22:38:01.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060333287999999999 2024-09-14T22:38:01.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480074+0000", 2024-09-14T22:38:01.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116443 2024-09-14T22:38:01.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480177+0000", 2024-09-14T22:38:01.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000103102 2024-09-14T22:38:01.335 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.335 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.335 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.335 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82546 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:01.335 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:38:01.335 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 61.863421449999997, 2024-09-14T22:38:01.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060795428999999998, 2024-09-14T22:38:01.335 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.335 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.335 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.336 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.336 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.336 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.336 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82546 2024-09-14T22:38:01.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.336 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:38:01.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:38:01.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419640+0000", 2024-09-14T22:38:01.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.254e-06 2024-09-14T22:38:01.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419647+0000", 2024-09-14T22:38:01.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9e-06 2024-09-14T22:38:01.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419664+0000", 2024-09-14T22:38:01.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6345999999999999e-05 2024-09-14T22:38:01.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419676+0000", 2024-09-14T22:38:01.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.224e-05 2024-09-14T22:38:01.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480218+0000", 2024-09-14T22:38:01.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060542133999999997 2024-09-14T22:38:01.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480336+0000", 2024-09-14T22:38:01.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117472 2024-09-14T22:38:01.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480435+0000", 2024-09-14T22:38:01.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9083e-05 2024-09-14T22:38:01.339 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.339 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.339 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.339 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82548 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:01.339 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:38:01.340 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 61.86336944, 2024-09-14T22:38:01.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061011636000000001, 2024-09-14T22:38:01.340 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.340 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.340 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.340 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.340 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.340 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.340 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82548 2024-09-14T22:38:01.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.340 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:38:01.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:38:01.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419693+0000", 2024-09-14T22:38:01.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.387e-06 2024-09-14T22:38:01.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419699+0000", 2024-09-14T22:38:01.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8730000000000001e-06 2024-09-14T22:38:01.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419716+0000", 2024-09-14T22:38:01.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6416e-05 2024-09-14T22:38:01.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419728+0000", 2024-09-14T22:38:01.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2597e-05 2024-09-14T22:38:01.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480476+0000", 2024-09-14T22:38:01.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060747989000000002 2024-09-14T22:38:01.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480583+0000", 2024-09-14T22:38:01.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010638500000000001 2024-09-14T22:38:01.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480703+0000", 2024-09-14T22:38:01.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011998900000000001 2024-09-14T22:38:01.343 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.343 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.343 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.344 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82550 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1982464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:01.344 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:38:01.344 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 61.863316687000001, 2024-09-14T22:38:01.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061295483999999997, 2024-09-14T22:38:01.344 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.344 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.344 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.344 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.344 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.344 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.344 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82550 2024-09-14T22:38:01.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.345 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:38:01.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:38:01.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419745+0000", 2024-09-14T22:38:01.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.406e-06 2024-09-14T22:38:01.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419753+0000", 2024-09-14T22:38:01.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8140000000000001e-06 2024-09-14T22:38:01.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419770+0000", 2024-09-14T22:38:01.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6719999999999999e-05 2024-09-14T22:38:01.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419782+0000", 2024-09-14T22:38:01.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2252999999999999e-05 2024-09-14T22:38:01.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.347 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480745+0000", 2024-09-14T22:38:01.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060962385000000001 2024-09-14T22:38:01.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.347 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480900+0000", 2024-09-14T22:38:01.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015513899999999999 2024-09-14T22:38:01.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481039+0000", 2024-09-14T22:38:01.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013976700000000001 2024-09-14T22:38:01.348 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.348 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.348 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.348 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82588 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1064960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:01.348 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:38:01.348 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 61.863262796999997, 2024-09-14T22:38:01.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061863732999999997, 2024-09-14T22:38:01.348 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.348 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.349 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.349 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.349 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.349 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.349 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82588 2024-09-14T22:38:01.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.349 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:38:01.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:38:01.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419799+0000", 2024-09-14T22:38:01.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4160000000000001e-06 2024-09-14T22:38:01.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419807+0000", 2024-09-14T22:38:01.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2169999999999994e-06 2024-09-14T22:38:01.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419824+0000", 2024-09-14T22:38:01.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6776e-05 2024-09-14T22:38:01.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419837+0000", 2024-09-14T22:38:01.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2680000000000001e-05 2024-09-14T22:38:01.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481395+0000", 2024-09-14T22:38:01.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061558416999999997 2024-09-14T22:38:01.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481532+0000", 2024-09-14T22:38:01.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000136861 2024-09-14T22:38:01.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481662+0000", 2024-09-14T22:38:01.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012936600000000001 2024-09-14T22:38:01.352 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.352 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.352 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.352 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82589 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:01.352 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:38:01.353 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 61.863208202000003, 2024-09-14T22:38:01.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062064592000000002, 2024-09-14T22:38:01.353 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.353 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.353 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.353 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.353 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.353 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.353 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82589 2024-09-14T22:38:01.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.353 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.353 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:38:01.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.354 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:38:01.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.354 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419854+0000", 2024-09-14T22:38:01.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.367e-06 2024-09-14T22:38:01.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.354 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419862+0000", 2024-09-14T22:38:01.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8930000000000005e-06 2024-09-14T22:38:01.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419896+0000", 2024-09-14T22:38:01.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4134000000000002e-05 2024-09-14T22:38:01.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419909+0000", 2024-09-14T22:38:01.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3332999999999999e-05 2024-09-14T22:38:01.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481702+0000", 2024-09-14T22:38:01.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061792947000000001 2024-09-14T22:38:01.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.356 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.356 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481808+0000", 2024-09-14T22:38:01.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105655 2024-09-14T22:38:01.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.356 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.356 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481917+0000", 2024-09-14T22:38:01.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109263 2024-09-14T22:38:01.356 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.356 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.356 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.357 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82590 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1359872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:01.357 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:38:01.357 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 61.863134594999998, 2024-09-14T22:38:01.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062362895000000002, 2024-09-14T22:38:01.357 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.357 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.357 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.357 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.357 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.357 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.357 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82590 2024-09-14T22:38:01.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.358 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:38:01.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:38:01.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419928+0000", 2024-09-14T22:38:01.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4470000000000001e-06 2024-09-14T22:38:01.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419942+0000", 2024-09-14T22:38:01.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4229e-05 2024-09-14T22:38:01.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419972+0000", 2024-09-14T22:38:01.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.065e-05 2024-09-14T22:38:01.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419985+0000", 2024-09-14T22:38:01.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2903e-05 2024-09-14T22:38:01.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481960+0000", 2024-09-14T22:38:01.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061974435000000001 2024-09-14T22:38:01.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482106+0000", 2024-09-14T22:38:01.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014643800000000001 2024-09-14T22:38:01.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.361 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482289+0000", 2024-09-14T22:38:01.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000182793 2024-09-14T22:38:01.361 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.361 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.361 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.361 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82592 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:01.361 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:38:01.362 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 61.863058373000001, 2024-09-14T22:38:01.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062631507000000003, 2024-09-14T22:38:01.362 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.362 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.362 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.362 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.362 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.362 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.362 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82592 2024-09-14T22:38:01.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.362 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:38:01.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:38:01.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420017+0000", 2024-09-14T22:38:01.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4744e-05 2024-09-14T22:38:01.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420036+0000", 2024-09-14T22:38:01.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8680000000000001e-05 2024-09-14T22:38:01.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.364 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420054+0000", 2024-09-14T22:38:01.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7799999999999999e-05 2024-09-14T22:38:01.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.364 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420067+0000", 2024-09-14T22:38:01.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3013000000000001e-05 2024-09-14T22:38:01.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482358+0000", 2024-09-14T22:38:01.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062291445000000001 2024-09-14T22:38:01.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482469+0000", 2024-09-14T22:38:01.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110608 2024-09-14T22:38:01.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482634+0000", 2024-09-14T22:38:01.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000165217 2024-09-14T22:38:01.365 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.366 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.366 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.366 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82594 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1654784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:01.366 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:38:01.366 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 61.862960397000002, 2024-09-14T22:38:01.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062893849000000002, 2024-09-14T22:38:01.366 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.366 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.366 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.366 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.366 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.366 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.367 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82594 2024-09-14T22:38:01.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.367 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:38:01.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:38:01.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420102+0000", 2024-09-14T22:38:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5e-06 2024-09-14T22:38:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420115+0000", 2024-09-14T22:38:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.324e-05 2024-09-14T22:38:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420133+0000", 2024-09-14T22:38:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8213e-05 2024-09-14T22:38:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420147+0000", 2024-09-14T22:38:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3284e-05 2024-09-14T22:38:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482688+0000", 2024-09-14T22:38:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062541425999999997 2024-09-14T22:38:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482794+0000", 2024-09-14T22:38:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010575500000000001 2024-09-14T22:38:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482994+0000", 2024-09-14T22:38:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020043099999999999 2024-09-14T22:38:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82596 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:38:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 61.862897097000001, 2024-09-14T22:38:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063187238000000007, 2024-09-14T22:38:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82596 2024-09-14T22:38:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:38:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:38:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420165+0000", 2024-09-14T22:38:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4100000000000001e-06 2024-09-14T22:38:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420173+0000", 2024-09-14T22:38:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8129999999999996e-06 2024-09-14T22:38:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420190+0000", 2024-09-14T22:38:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7076999999999999e-05 2024-09-14T22:38:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420203+0000", 2024-09-14T22:38:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2683e-05 2024-09-14T22:38:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483060+0000", 2024-09-14T22:38:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062857114000000006 2024-09-14T22:38:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483167+0000", 2024-09-14T22:38:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010713999999999999 2024-09-14T22:38:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483351+0000", 2024-09-14T22:38:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018400099999999999 2024-09-14T22:38:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82597 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1949696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:38:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 61.862840271000003, 2024-09-14T22:38:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063466513000000002, 2024-09-14T22:38:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82597 2024-09-14T22:38:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:38:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:38:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420221+0000", 2024-09-14T22:38:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.043e-06 2024-09-14T22:38:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420229+0000", 2024-09-14T22:38:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0469999999999994e-06 2024-09-14T22:38:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420246+0000", 2024-09-14T22:38:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6339999999999999e-05 2024-09-14T22:38:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420258+0000", 2024-09-14T22:38:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2296000000000001e-05 2024-09-14T22:38:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483402+0000", 2024-09-14T22:38:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063144318000000005 2024-09-14T22:38:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483530+0000", 2024-09-14T22:38:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012713199999999999 2024-09-14T22:38:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483687+0000", 2024-09-14T22:38:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015733699999999999 2024-09-14T22:38:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.379 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:01.381 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:38:01.382 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:01.374+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:01.382 INFO:teuthology.orchestra.run.smithi001.stdout:5450 trunc from 0xe64bb0f to 0xc75f0b3 2024-09-14T22:38:01.382 INFO:teuthology.orchestra.run.smithi001.stdout:5451 read 0x797975b thru 0x79829b1 (0x9257 bytes) 2024-09-14T22:38:01.382 INFO:teuthology.orchestra.run.smithi001.stdout:5452 punch from 0xa0905bd to 0xa09ac5b, (0xa69e bytes) 2024-09-14T22:38:01.382 INFO:teuthology.orchestra.run.smithi001.stdout:5453 write 0x6e01a3 thru 0x6e1fc3 (0x1e21 bytes) 2024-09-14T22:38:01.382 INFO:teuthology.orchestra.run.smithi001.stdout:5455 read 0xa206486 thru 0xa20f7db (0x9356 bytes) 2024-09-14T22:38:01.382 INFO:teuthology.orchestra.run.smithi001.stdout:5457 write 0xb4f60cd thru 0xb4fc095 (0x5fc9 bytes) 2024-09-14T22:38:01.382 INFO:teuthology.orchestra.run.smithi001.stdout:5459 trunc from 0xc75f0b3 to 0xa70f50 2024-09-14T22:38:01.397 INFO:teuthology.orchestra.run.smithi001.stdout:5461 write 0x52adb6e thru 0x52b1fc6 (0x4459 bytes) 2024-09-14T22:38:01.404 INFO:teuthology.orchestra.run.smithi001.stdout:5466 read 0xc60a89 thru 0xc65fff (0x5577 bytes) 2024-09-14T22:38:01.409 INFO:teuthology.orchestra.run.smithi001.stdout:5467 trunc from 0x52b1fc7 to 0x691bccf 2024-09-14T22:38:01.422 INFO:teuthology.orchestra.run.smithi001.stdout:5468 read 0x3ebe33c thru 0x3ec3f46 (0x5c0b bytes) 2024-09-14T22:38:01.422 INFO:teuthology.orchestra.run.smithi001.stdout:5469 trunc from 0x691bccf to 0x6910697 2024-09-14T22:38:01.438 INFO:teuthology.orchestra.run.smithi001.stdout:5470 read 0x1569204 thru 0x156d52d (0x432a bytes) 2024-09-14T22:38:01.438 INFO:teuthology.orchestra.run.smithi001.stdout:5472 read 0x64a16a2 thru 0x64a8017 (0x6976 bytes) 2024-09-14T22:38:01.438 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:01.439 INFO:teuthology.orchestra.run.smithi001.stdout:5474 clone 36 order 20 su 65536 sc 4 2024-09-14T22:38:01.482 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:01.474+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:01.551 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:01.551 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:01.551 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:38:01.551 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:01.562 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-14T22:38:01.563 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-14T22:38:01.566 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:38:01.611 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:01.606+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:01.684 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:01.678+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:01.735 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:01.735 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:01.735 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:38:01.735 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:38:01.735 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:01.749 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:38:01.753 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T22:38:01.795 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:01.790+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:01.873 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:01.873 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:38:01.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:38:01.874 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:38:01.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.874 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:38:01.874 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:38:01.874 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 121.863841889, 2024-09-14T22:38:01.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.398029152, 2024-09-14T22:38:01.874 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.874 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.874 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:01.874 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.874 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:38:01.875 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:38:01.875 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 1 2024-09-14T22:38:01.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.875 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.875 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.875 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:38:01.875 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:38:01.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006183+0000", 2024-09-14T22:38:01.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9059999999999998e-06 2024-09-14T22:38:01.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006198+0000", 2024-09-14T22:38:01.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.523e-05 2024-09-14T22:38:01.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006219+0000", 2024-09-14T22:38:01.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1342999999999999e-05 2024-09-14T22:38:01.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006240+0000", 2024-09-14T22:38:01.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1186999999999999e-05 2024-09-14T22:38:01.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006363+0000", 2024-09-14T22:38:01.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012258200000000001 2024-09-14T22:38:01.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:38:01.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006392+0000", 2024-09-14T22:38:01.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9607e-05 2024-09-14T22:38:01.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.879 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.879 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401084+0000", 2024-09-14T22:38:01.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.39469142699999998 2024-09-14T22:38:01.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.879 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.879 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401394+0000", 2024-09-14T22:38:01.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030987799999999998 2024-09-14T22:38:01.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:38:01.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401860+0000", 2024-09-14T22:38:01.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00046654299999999998 2024-09-14T22:38:01.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:38:01.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.403386+0000", 2024-09-14T22:38:01.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001525322 2024-09-14T22:38:01.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:01.881 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404042+0000", 2024-09-14T22:38:01.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065676099999999996 2024-09-14T22:38:01.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:38:01.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404136+0000", 2024-09-14T22:38:01.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4032999999999999e-05 2024-09-14T22:38:01.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404209+0000", 2024-09-14T22:38:01.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2333e-05 2024-09-14T22:38:01.882 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.882 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.882 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.883 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104182 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14352384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:01.883 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:38:01.883 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.380987902999998, 2024-09-14T22:38:01.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071642496999999999, 2024-09-14T22:38:01.883 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.883 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.883 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.883 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.883 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.884 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.884 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104182 2024-09-14T22:38:01.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.884 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.884 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.884 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:38:01.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.884 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:38:01.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489035+0000", 2024-09-14T22:38:01.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3430000000000001e-06 2024-09-14T22:38:01.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489041+0000", 2024-09-14T22:38:01.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8139999999999997e-06 2024-09-14T22:38:01.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489059+0000", 2024-09-14T22:38:01.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8193e-05 2024-09-14T22:38:01.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489073+0000", 2024-09-14T22:38:01.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:38:01.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.887 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.887 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560434+0000", 2024-09-14T22:38:01.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071361463999999999 2024-09-14T22:38:01.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.887 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.887 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560545+0000", 2024-09-14T22:38:01.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110263 2024-09-14T22:38:01.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.887 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.888 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560676+0000", 2024-09-14T22:38:01.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013138000000000001 2024-09-14T22:38:01.888 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.888 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.888 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.888 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.888 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.888 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104183 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14483456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:01.888 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:38:01.888 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.380864584000001, 2024-09-14T22:38:01.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071781943000000001, 2024-09-14T22:38:01.889 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.889 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.889 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.889 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.889 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.889 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.889 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104183 2024-09-14T22:38:01.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.889 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:38:01.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:38:01.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489159+0000", 2024-09-14T22:38:01.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5629999999999999e-06 2024-09-14T22:38:01.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489165+0000", 2024-09-14T22:38:01.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7070000000000003e-06 2024-09-14T22:38:01.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.892 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.892 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489186+0000", 2024-09-14T22:38:01.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.088e-05 2024-09-14T22:38:01.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.892 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.892 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489201+0000", 2024-09-14T22:38:01.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5203e-05 2024-09-14T22:38:01.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560713+0000", 2024-09-14T22:38:01.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071511447000000006 2024-09-14T22:38:01.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560819+0000", 2024-09-14T22:38:01.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106579 2024-09-14T22:38:01.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.894 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.894 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560939+0000", 2024-09-14T22:38:01.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119564 2024-09-14T22:38:01.894 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.894 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.894 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.894 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104184 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:01.895 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:38:01.895 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.380800141000002, 2024-09-14T22:38:01.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071970107000000005, 2024-09-14T22:38:01.895 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.895 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.895 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.895 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.895 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.895 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.895 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104184 2024-09-14T22:38:01.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.896 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.896 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.896 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:38:01.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.896 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.896 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:38:01.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489223+0000", 2024-09-14T22:38:01.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4470000000000001e-06 2024-09-14T22:38:01.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489229+0000", 2024-09-14T22:38:01.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8200000000000002e-06 2024-09-14T22:38:01.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489247+0000", 2024-09-14T22:38:01.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8150000000000001e-05 2024-09-14T22:38:01.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489260+0000", 2024-09-14T22:38:01.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3570000000000001e-05 2024-09-14T22:38:01.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.899 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.899 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560979+0000", 2024-09-14T22:38:01.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071718831999999996 2024-09-14T22:38:01.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.899 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.899 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561075+0000", 2024-09-14T22:38:01.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5848999999999997e-05 2024-09-14T22:38:01.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.899 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.900 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561191+0000", 2024-09-14T22:38:01.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116439 2024-09-14T22:38:01.900 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.900 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.900 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.900 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.900 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104185 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14745600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:01.900 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:38:01.900 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.380740557999999, 2024-09-14T22:38:01.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072173750999999994, 2024-09-14T22:38:01.901 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.901 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.901 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.901 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.901 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.901 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.901 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104185 2024-09-14T22:38:01.901 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.901 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.901 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:38:01.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:38:01.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489282+0000", 2024-09-14T22:38:01.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2500000000000001e-06 2024-09-14T22:38:01.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489287+0000", 2024-09-14T22:38:01.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1359999999999996e-06 2024-09-14T22:38:01.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.904 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489304+0000", 2024-09-14T22:38:01.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7033000000000001e-05 2024-09-14T22:38:01.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.904 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.904 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489318+0000", 2024-09-14T22:38:01.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3934000000000001e-05 2024-09-14T22:38:01.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.904 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.904 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561232+0000", 2024-09-14T22:38:01.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071913759999999993 2024-09-14T22:38:01.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.905 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.905 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561340+0000", 2024-09-14T22:38:01.905 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107921 2024-09-14T22:38:01.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.905 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.905 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561455+0000", 2024-09-14T22:38:01.905 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011471700000000001 2024-09-14T22:38:01.905 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.906 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.906 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.906 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104186 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14876672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:01.906 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:38:01.906 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.380684649000003, 2024-09-14T22:38:01.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072393374999999996, 2024-09-14T22:38:01.906 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.906 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.906 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.906 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.907 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.907 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.907 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104186 2024-09-14T22:38:01.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.907 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:38:01.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:38:01.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489338+0000", 2024-09-14T22:38:01.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.48e-06 2024-09-14T22:38:01.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489366+0000", 2024-09-14T22:38:01.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7307000000000001e-05 2024-09-14T22:38:01.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.909 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.909 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489385+0000", 2024-09-14T22:38:01.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9283000000000002e-05 2024-09-14T22:38:01.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.909 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.909 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489399+0000", 2024-09-14T22:38:01.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.452e-05 2024-09-14T22:38:01.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.910 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561494+0000", 2024-09-14T22:38:01.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072094885999999997 2024-09-14T22:38:01.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.910 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561604+0000", 2024-09-14T22:38:01.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010925900000000001 2024-09-14T22:38:01.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561730+0000", 2024-09-14T22:38:01.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012663999999999999 2024-09-14T22:38:01.911 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.911 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.911 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.911 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104187 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15007744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:01.912 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:38:01.912 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.38060128, 2024-09-14T22:38:01.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072599143000000005, 2024-09-14T22:38:01.912 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.912 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.912 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.912 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.912 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.912 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.912 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104187 2024-09-14T22:38:01.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.913 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:38:01.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:38:01.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.914 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489422+0000", 2024-09-14T22:38:01.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5340000000000001e-06 2024-09-14T22:38:01.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.914 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489428+0000", 2024-09-14T22:38:01.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1430000000000003e-06 2024-09-14T22:38:01.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489445+0000", 2024-09-14T22:38:01.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7042999999999999e-05 2024-09-14T22:38:01.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489458+0000", 2024-09-14T22:38:01.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3474e-05 2024-09-14T22:38:01.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561771+0000", 2024-09-14T22:38:01.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072312600000000005 2024-09-14T22:38:01.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561885+0000", 2024-09-14T22:38:01.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011411799999999999 2024-09-14T22:38:01.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562019+0000", 2024-09-14T22:38:01.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013423100000000001 2024-09-14T22:38:01.917 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.917 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.917 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.917 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104188 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15138816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:01.917 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:38:01.917 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.380541356000002, 2024-09-14T22:38:01.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072824624000000004, 2024-09-14T22:38:01.917 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.918 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.918 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.918 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.918 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.918 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.918 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104188 2024-09-14T22:38:01.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.918 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:38:01.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:38:01.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489481+0000", 2024-09-14T22:38:01.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.22e-06 2024-09-14T22:38:01.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489487+0000", 2024-09-14T22:38:01.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6930000000000004e-06 2024-09-14T22:38:01.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489508+0000", 2024-09-14T22:38:01.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1135999999999999e-05 2024-09-14T22:38:01.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489522+0000", 2024-09-14T22:38:01.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3817e-05 2024-09-14T22:38:01.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562066+0000", 2024-09-14T22:38:01.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072543813999999998 2024-09-14T22:38:01.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562177+0000", 2024-09-14T22:38:01.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000111264 2024-09-14T22:38:01.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562305+0000", 2024-09-14T22:38:01.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012768 2024-09-14T22:38:01.922 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.922 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.922 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.923 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104189 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15269888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:01.923 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:38:01.923 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.380480906999999, 2024-09-14T22:38:01.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073061300999999995, 2024-09-14T22:38:01.923 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.923 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.923 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.923 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.923 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.923 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.923 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104189 2024-09-14T22:38:01.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.924 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:38:01.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:38:01.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489542+0000", 2024-09-14T22:38:01.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1659999999999999e-06 2024-09-14T22:38:01.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489561+0000", 2024-09-14T22:38:01.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.908e-05 2024-09-14T22:38:01.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489579+0000", 2024-09-14T22:38:01.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8536999999999999e-05 2024-09-14T22:38:01.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489593+0000", 2024-09-14T22:38:01.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3977e-05 2024-09-14T22:38:01.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562347+0000", 2024-09-14T22:38:01.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072754092000000006 2024-09-14T22:38:01.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.927 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562458+0000", 2024-09-14T22:38:01.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110722 2024-09-14T22:38:01.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.927 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562602+0000", 2024-09-14T22:38:01.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014372699999999999 2024-09-14T22:38:01.928 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.928 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.928 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.928 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104190 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:01.928 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:38:01.928 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.380408611, 2024-09-14T22:38:01.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.07326133, 2024-09-14T22:38:01.928 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.929 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.929 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.929 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.929 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.929 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.929 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104190 2024-09-14T22:38:01.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.929 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:38:01.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:38:01.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489614+0000", 2024-09-14T22:38:01.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1999999999999999e-06 2024-09-14T22:38:01.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489620+0000", 2024-09-14T22:38:01.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.524e-06 2024-09-14T22:38:01.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489637+0000", 2024-09-14T22:38:01.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7825999999999998e-05 2024-09-14T22:38:01.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.932 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.932 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.932 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489671+0000", 2024-09-14T22:38:01.932 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3726000000000002e-05 2024-09-14T22:38:01.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.932 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.932 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.932 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562644+0000", 2024-09-14T22:38:01.932 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072973089000000005 2024-09-14T22:38:01.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562762+0000", 2024-09-14T22:38:01.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011743600000000001 2024-09-14T22:38:01.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562874+0000", 2024-09-14T22:38:01.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011252900000000001 2024-09-14T22:38:01.933 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.933 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.933 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.934 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104191 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15532032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:01.934 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:38:01.934 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.380300188, 2024-09-14T22:38:01.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073311166999999997, 2024-09-14T22:38:01.934 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.934 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.934 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.934 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.934 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.935 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.935 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104191 2024-09-14T22:38:01.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.935 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:38:01.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.936 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:38:01.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.936 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489723+0000", 2024-09-14T22:38:01.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6330000000000001e-06 2024-09-14T22:38:01.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.937 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.937 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.937 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489730+0000", 2024-09-14T22:38:01.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2300000000000002e-06 2024-09-14T22:38:01.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.937 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.937 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.937 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489751+0000", 2024-09-14T22:38:01.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0849999999999999e-05 2024-09-14T22:38:01.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.938 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.938 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489766+0000", 2024-09-14T22:38:01.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.524e-05 2024-09-14T22:38:01.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.938 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.938 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562896+0000", 2024-09-14T22:38:01.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073129662999999998 2024-09-14T22:38:01.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562960+0000", 2024-09-14T22:38:01.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3689999999999995e-05 2024-09-14T22:38:01.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563033+0000", 2024-09-14T22:38:01.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2861000000000006e-05 2024-09-14T22:38:01.939 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.939 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.939 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.940 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104192 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:01.940 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:38:01.940 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.380204382999999, 2024-09-14T22:38:01.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073463632000000001, 2024-09-14T22:38:01.940 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.940 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.940 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.940 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.940 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.940 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.941 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104192 2024-09-14T22:38:01.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.941 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.941 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.941 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:38:01.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.941 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.941 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:38:01.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.942 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.942 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.942 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.942 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489819+0000", 2024-09-14T22:38:01.942 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.424e-06 2024-09-14T22:38:01.942 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.942 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.942 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.942 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489825+0000", 2024-09-14T22:38:01.942 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.02e-06 2024-09-14T22:38:01.942 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489844+0000", 2024-09-14T22:38:01.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9760000000000001e-05 2024-09-14T22:38:01.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489860+0000", 2024-09-14T22:38:01.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5563e-05 2024-09-14T22:38:01.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563070+0000", 2024-09-14T22:38:01.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073209724000000004 2024-09-14T22:38:01.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563175+0000", 2024-09-14T22:38:01.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105183 2024-09-14T22:38:01.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563281+0000", 2024-09-14T22:38:01.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105958 2024-09-14T22:38:01.945 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.945 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.945 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.945 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104193 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15794176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:01.945 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:38:01.945 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.380142216000003, 2024-09-14T22:38:01.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073600040000000005, 2024-09-14T22:38:01.945 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.945 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.946 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.946 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.946 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.946 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.946 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104193 2024-09-14T22:38:01.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.946 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.946 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.946 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:38:01.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:38:01.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489881+0000", 2024-09-14T22:38:01.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5460000000000001e-06 2024-09-14T22:38:01.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489886+0000", 2024-09-14T22:38:01.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1739999999999999e-06 2024-09-14T22:38:01.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489912+0000", 2024-09-14T22:38:01.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6461999999999998e-05 2024-09-14T22:38:01.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489927+0000", 2024-09-14T22:38:01.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:38:01.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563310+0000", 2024-09-14T22:38:01.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073382977000000002 2024-09-14T22:38:01.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563385+0000", 2024-09-14T22:38:01.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5218999999999999e-05 2024-09-14T22:38:01.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.950 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.950 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563479+0000", 2024-09-14T22:38:01.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4622000000000003e-05 2024-09-14T22:38:01.950 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.950 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.950 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.950 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104194 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15925248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:01.951 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:38:01.951 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.380075624, 2024-09-14T22:38:01.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073745434999999998, 2024-09-14T22:38:01.951 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.951 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.951 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.951 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.951 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.951 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.951 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104194 2024-09-14T22:38:01.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.952 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:38:01.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:38:01.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.953 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489947+0000", 2024-09-14T22:38:01.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.327e-06 2024-09-14T22:38:01.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.953 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489953+0000", 2024-09-14T22:38:01.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9329999999999998e-06 2024-09-14T22:38:01.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489971+0000", 2024-09-14T22:38:01.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7994000000000001e-05 2024-09-14T22:38:01.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489985+0000", 2024-09-14T22:38:01.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4226000000000001e-05 2024-09-14T22:38:01.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563510+0000", 2024-09-14T22:38:01.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073524667000000002 2024-09-14T22:38:01.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563597+0000", 2024-09-14T22:38:01.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6550000000000003e-05 2024-09-14T22:38:01.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563691+0000", 2024-09-14T22:38:01.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4738e-05 2024-09-14T22:38:01.956 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.956 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.956 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.956 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104195 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:01.956 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:38:01.956 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.380017426999999, 2024-09-14T22:38:01.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073876598000000002, 2024-09-14T22:38:01.956 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.956 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.957 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.957 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.957 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.957 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.957 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104195 2024-09-14T22:38:01.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.957 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.957 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.957 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:38:01.957 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.958 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.958 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.958 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:38:01.958 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.958 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.958 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.958 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490006+0000", 2024-09-14T22:38:01.958 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4699999999999999e-06 2024-09-14T22:38:01.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.958 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.958 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.959 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490011+0000", 2024-09-14T22:38:01.959 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.366e-06 2024-09-14T22:38:01.959 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.959 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.959 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.959 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490046+0000", 2024-09-14T22:38:01.959 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5496e-05 2024-09-14T22:38:01.959 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.959 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.959 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.959 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490061+0000", 2024-09-14T22:38:01.959 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5054e-05 2024-09-14T22:38:01.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563719+0000", 2024-09-14T22:38:01.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073657412000000005 2024-09-14T22:38:01.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563792+0000", 2024-09-14T22:38:01.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3319999999999996e-05 2024-09-14T22:38:01.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.961 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563881+0000", 2024-09-14T22:38:01.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8480000000000007e-05 2024-09-14T22:38:01.961 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.961 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.961 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.961 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104196 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16187392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:01.961 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:38:01.962 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.379940277999999, 2024-09-14T22:38:01.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073997496999999995, 2024-09-14T22:38:01.962 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.962 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.962 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.962 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.962 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.962 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.962 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104196 2024-09-14T22:38:01.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.962 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.963 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.963 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:38:01.963 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.963 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.963 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.963 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.963 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:38:01.963 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.963 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.964 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490083+0000", 2024-09-14T22:38:01.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.347e-06 2024-09-14T22:38:01.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.964 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490089+0000", 2024-09-14T22:38:01.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9630000000000003e-06 2024-09-14T22:38:01.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490107+0000", 2024-09-14T22:38:01.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8859999999999999e-05 2024-09-14T22:38:01.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490122+0000", 2024-09-14T22:38:01.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.432e-05 2024-09-14T22:38:01.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563910+0000", 2024-09-14T22:38:01.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073788753999999998 2024-09-14T22:38:01.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563988+0000", 2024-09-14T22:38:01.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7862999999999996e-05 2024-09-14T22:38:01.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564079+0000", 2024-09-14T22:38:01.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0389999999999993e-05 2024-09-14T22:38:01.967 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.967 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.967 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.967 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104197 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16318464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:01.967 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:38:01.967 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.379848649000003, 2024-09-14T22:38:01.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074103547000000006, 2024-09-14T22:38:01.967 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.967 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.968 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.968 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.968 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.968 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.968 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104197 2024-09-14T22:38:01.968 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.968 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.968 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.968 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.968 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:38:01.968 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:38:01.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490174+0000", 2024-09-14T22:38:01.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.37e-06 2024-09-14T22:38:01.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490197+0000", 2024-09-14T22:38:01.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3017e-05 2024-09-14T22:38:01.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490219+0000", 2024-09-14T22:38:01.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.181e-05 2024-09-14T22:38:01.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490240+0000", 2024-09-14T22:38:01.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0786000000000001e-05 2024-09-14T22:38:01.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564109+0000", 2024-09-14T22:38:01.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073869031000000002 2024-09-14T22:38:01.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564186+0000", 2024-09-14T22:38:01.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6823999999999995e-05 2024-09-14T22:38:01.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564276+0000", 2024-09-14T22:38:01.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0709000000000002e-05 2024-09-14T22:38:01.972 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.972 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.972 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.973 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104198 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16449536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:01.973 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:38:01.973 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.379756413000003, 2024-09-14T22:38:01.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074167693000000007, 2024-09-14T22:38:01.973 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.973 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.973 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.973 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.973 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.973 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.974 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104198 2024-09-14T22:38:01.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.974 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.974 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:38:01.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.974 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:38:01.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490266+0000", 2024-09-14T22:38:01.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3599999999999999e-06 2024-09-14T22:38:01.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490272+0000", 2024-09-14T22:38:01.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9599999999999997e-06 2024-09-14T22:38:01.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.976 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.976 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490291+0000", 2024-09-14T22:38:01.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8485999999999999e-05 2024-09-14T22:38:01.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.976 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.976 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490305+0000", 2024-09-14T22:38:01.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4327e-05 2024-09-14T22:38:01.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.976 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.976 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564306+0000", 2024-09-14T22:38:01.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074000745000000007 2024-09-14T22:38:01.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564367+0000", 2024-09-14T22:38:01.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1501999999999995e-05 2024-09-14T22:38:01.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564433+0000", 2024-09-14T22:38:01.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5313000000000003e-05 2024-09-14T22:38:01.978 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.978 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.978 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.978 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104199 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16580608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:01.978 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:38:01.978 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.379674860000002, 2024-09-14T22:38:01.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074231631000000006, 2024-09-14T22:38:01.978 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.978 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.979 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.979 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.979 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.979 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.979 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104199 2024-09-14T22:38:01.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.979 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.979 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.979 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:38:01.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.980 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.980 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:38:01.980 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.980 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.980 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490349+0000", 2024-09-14T22:38:01.980 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9429999999999999e-06 2024-09-14T22:38:01.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.981 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.981 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490355+0000", 2024-09-14T22:38:01.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1670000000000004e-06 2024-09-14T22:38:01.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.981 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.981 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490374+0000", 2024-09-14T22:38:01.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9613e-05 2024-09-14T22:38:01.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490405+0000", 2024-09-14T22:38:01.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0936000000000003e-05 2024-09-14T22:38:01.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564451+0000", 2024-09-14T22:38:01.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074045266999999998 2024-09-14T22:38:01.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564494+0000", 2024-09-14T22:38:01.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3133000000000002e-05 2024-09-14T22:38:01.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564578+0000", 2024-09-14T22:38:01.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4572000000000003e-05 2024-09-14T22:38:01.983 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.983 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.983 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.984 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104200 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:01.984 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:38:01.984 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.377879632000003, 2024-09-14T22:38:01.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072582435000000001, 2024-09-14T22:38:01.984 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:01.984 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:01.984 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:01.984 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:01.984 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:01.984 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:01.985 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104200 2024-09-14T22:38:01.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.985 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:01.985 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.985 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:01.985 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:38:01.985 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.985 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.985 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:01.985 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:38:01.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:01.986 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.986 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:01.986 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492144+0000", 2024-09-14T22:38:01.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3070000000000001e-06 2024-09-14T22:38:01.986 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.986 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:01.986 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492155+0000", 2024-09-14T22:38:01.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0573e-05 2024-09-14T22:38:01.986 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:01.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492185+0000", 2024-09-14T22:38:01.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.008e-05 2024-09-14T22:38:01.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:01.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492205+0000", 2024-09-14T22:38:01.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9882999999999999e-05 2024-09-14T22:38:01.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:01.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564609+0000", 2024-09-14T22:38:01.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072404181999999997 2024-09-14T22:38:01.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.988 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:01.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564667+0000", 2024-09-14T22:38:01.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8102e-05 2024-09-14T22:38:01.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:01.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:01.988 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:01.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564724+0000", 2024-09-14T22:38:01.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7308000000000001e-05 2024-09-14T22:38:01.988 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.988 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.989 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.989 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:01.989 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:01.989 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:01.991 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.0 to 1 (10000) 2024-09-14T22:38:01.991 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:38:01.992 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-14T22:38:01.992 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:01.986+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:02.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-14T22:38:02.082+0000 7f8b7dced640 0 --2- 172.21.15.1:0/3850537859 >> [v2:172.21.15.1:3301/0,v1:172.21.15.1:6790/0] conn(0x7f8b78072fe0 0x7f8b780afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-14T22:38:02.091 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:02.086+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:02.171 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:02.171 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:02.171 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:38:02.171 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:02.185 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-14T22:38:02.190 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:38:02.233 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:02.230+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:02.291 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:02.286+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:02.304 INFO:teuthology.orchestra.run.smithi001.stdout:5475 punch from 0x394f1f1 to 0x395a5f4, (0xb403 bytes) 2024-09-14T22:38:02.313 INFO:teuthology.orchestra.run.smithi001.stdout:5476 write 0x515c0ce thru 0x5163101 (0x7034 bytes) 2024-09-14T22:38:02.357 INFO:teuthology.orchestra.run.smithi001.stdout:5477 write 0xbfbae81 thru 0xbfc3fe4 (0x9164 bytes) 2024-09-14T22:38:02.358 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:02.358 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:02.358 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:38:02.358 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:38:02.358 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:02.367 INFO:teuthology.orchestra.run.smithi001.stdout:5478 punch from 0x8e99b48 to 0x8ea74ba, (0xd972 bytes) 2024-09-14T22:38:02.374 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:38:02.381 INFO:teuthology.orchestra.run.smithi001.stdout:5480 write 0xdfde5d4 thru 0xdfedb6b (0xf598 bytes) 2024-09-14T22:38:02.390 INFO:teuthology.orchestra.run.smithi001.stdout:5481 write 0x82f6076 thru 0x82fdcab (0x7c36 bytes) 2024-09-14T22:38:02.392 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:02.386+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:02.398 INFO:teuthology.orchestra.run.smithi001.stdout:5483 punch from 0xa630bc4 to 0xa630fdd, (0x419 bytes) 2024-09-14T22:38:02.398 INFO:teuthology.orchestra.run.smithi001.stdout:5484 punch from 0x4c5aab5 to 0x4c5dda9, (0x32f4 bytes) 2024-09-14T22:38:02.398 INFO:teuthology.orchestra.run.smithi001.stdout:5485 punch from 0x5688c5e to 0x568fe40, (0x71e2 bytes) 2024-09-14T22:38:02.398 INFO:teuthology.orchestra.run.smithi001.stdout:5486 read 0x4e65838 thru 0x4e6e5dc (0x8da5 bytes) 2024-09-14T22:38:02.398 INFO:teuthology.orchestra.run.smithi001.stdout:5488 writesame 0x531bbda thru 0x532a47a data_size 0x3(0xe8a1 bytes) 2024-09-14T22:38:02.408 INFO:teuthology.orchestra.run.smithi001.stdout:5489 write 0x4d3134a thru 0x4d406a9 (0xf360 bytes) 2024-09-14T22:38:02.419 INFO:teuthology.orchestra.run.smithi001.stdout:5490 write 0x6680a9b thru 0x6682ee8 (0x244e bytes) 2024-09-14T22:38:02.426 INFO:teuthology.orchestra.run.smithi001.stdout:5491 write 0x7fafc0b thru 0x7fb5a56 (0x5e4c bytes) 2024-09-14T22:38:02.436 INFO:teuthology.orchestra.run.smithi001.stdout:5492 read 0x38f7dc2 thru 0x3905fa7 (0xe1e6 bytes) 2024-09-14T22:38:02.454 INFO:teuthology.orchestra.run.smithi001.stdout:5493 trunc from 0xdfedb6c to 0x3dd8f5f 2024-09-14T22:38:02.493 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:02.490+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:02.494 INFO:teuthology.orchestra.run.smithi001.stdout:5495 read 0xfe9ebd thru 0xff8370 (0xe4b4 bytes) 2024-09-14T22:38:02.517 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:02.518 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:38:02.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:38:02.518 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:38:02.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.518 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144987 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3424256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:02.518 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549297+0000", 2024-09-14T22:38:02.518 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.966857353, 2024-09-14T22:38:02.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.079931556000000001, 2024-09-14T22:38:02.518 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:02.518 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:02.518 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:02.518 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:02.519 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:02.519 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:02.519 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144987 2024-09-14T22:38:02.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.519 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:02.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:02.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549297+0000", 2024-09-14T22:38:02.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:02.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549297+0000", 2024-09-14T22:38:02.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:02.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549298+0000", 2024-09-14T22:38:02.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2699999999999999e-07 2024-09-14T22:38:02.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:02.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549301+0000", 2024-09-14T22:38:02.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2959999999999999e-06 2024-09-14T22:38:02.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:02.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549314+0000", 2024-09-14T22:38:02.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2588999999999999e-05 2024-09-14T22:38:02.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:02.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549323+0000", 2024-09-14T22:38:02.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.927e-06 2024-09-14T22:38:02.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:02.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629038+0000", 2024-09-14T22:38:02.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.079714220000000002 2024-09-14T22:38:02.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.522 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:02.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629116+0000", 2024-09-14T22:38:02.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7966000000000005e-05 2024-09-14T22:38:02.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.522 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:02.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629228+0000", 2024-09-14T22:38:02.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112831 2024-09-14T22:38:02.522 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.522 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:02.523 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.523 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144988 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3325952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:02.523 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549335+0000", 2024-09-14T22:38:02.523 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9668191280000009, 2024-09-14T22:38:02.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080179116999999994, 2024-09-14T22:38:02.523 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:02.523 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:02.523 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:02.523 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:02.523 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:02.523 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:02.524 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144988 2024-09-14T22:38:02.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.524 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:02.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:02.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549335+0000", 2024-09-14T22:38:02.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:02.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549335+0000", 2024-09-14T22:38:02.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:02.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549336+0000", 2024-09-14T22:38:02.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7999999999999995e-07 2024-09-14T22:38:02.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:02.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549339+0000", 2024-09-14T22:38:02.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.836e-06 2024-09-14T22:38:02.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:02.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549352+0000", 2024-09-14T22:38:02.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3069e-05 2024-09-14T22:38:02.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:02.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549359+0000", 2024-09-14T22:38:02.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.357e-06 2024-09-14T22:38:02.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:02.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629288+0000", 2024-09-14T22:38:02.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.079928492000000004 2024-09-14T22:38:02.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.527 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:02.527 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629412+0000", 2024-09-14T22:38:02.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012413699999999999 2024-09-14T22:38:02.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.527 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:02.527 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629514+0000", 2024-09-14T22:38:02.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000102546 2024-09-14T22:38:02.527 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.527 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:02.527 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.528 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144989 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3522560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:02.528 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:38:02.528 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9667847199999997, 2024-09-14T22:38:02.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080393122999999997, 2024-09-14T22:38:02.528 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:02.528 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:02.528 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:02.528 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:02.528 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:02.528 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:02.528 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144989 2024-09-14T22:38:02.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.528 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:02.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:02.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:38:02.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:02.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:38:02.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:02.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:38:02.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3600000000000003e-07 2024-09-14T22:38:02.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:02.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549373+0000", 2024-09-14T22:38:02.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8169999999999999e-06 2024-09-14T22:38:02.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:02.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549385+0000", 2024-09-14T22:38:02.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1803e-05 2024-09-14T22:38:02.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:02.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549394+0000", 2024-09-14T22:38:02.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4290000000000007e-06 2024-09-14T22:38:02.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:02.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629576+0000", 2024-09-14T22:38:02.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080181551000000004 2024-09-14T22:38:02.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:02.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629654+0000", 2024-09-14T22:38:02.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7633999999999996e-05 2024-09-14T22:38:02.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.532 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:02.532 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629763+0000", 2024-09-14T22:38:02.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109153 2024-09-14T22:38:02.532 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.532 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:02.532 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.532 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144990 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:02.532 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:38:02.532 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9667416959999997, 2024-09-14T22:38:02.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080614496999999993, 2024-09-14T22:38:02.533 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:02.533 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:02.533 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:02.533 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:02.533 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:02.533 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:02.533 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144990 2024-09-14T22:38:02.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.533 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:02.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.533 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:02.533 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:38:02.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:02.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:38:02.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:02.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:38:02.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0999999999999997e-07 2024-09-14T22:38:02.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:02.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549419+0000", 2024-09-14T22:38:02.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5339999999999999e-06 2024-09-14T22:38:02.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:02.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549439+0000", 2024-09-14T22:38:02.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0035e-05 2024-09-14T22:38:02.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:02.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549449+0000", 2024-09-14T22:38:02.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0429e-05 2024-09-14T22:38:02.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:02.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629815+0000", 2024-09-14T22:38:02.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080365467999999995 2024-09-14T22:38:02.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:02.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629922+0000", 2024-09-14T22:38:02.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106914 2024-09-14T22:38:02.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:02.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630027+0000", 2024-09-14T22:38:02.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105207 2024-09-14T22:38:02.537 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.537 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:02.537 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.537 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144991 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 2998272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:02.537 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549463+0000", 2024-09-14T22:38:02.537 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9666913919999995, 2024-09-14T22:38:02.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080847637, 2024-09-14T22:38:02.537 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:02.537 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:02.537 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:02.538 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:02.538 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:02.538 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:02.538 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144991 2024-09-14T22:38:02.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.538 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:02.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:02.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549463+0000", 2024-09-14T22:38:02.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:02.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549463+0000", 2024-09-14T22:38:02.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:02.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549464+0000", 2024-09-14T22:38:02.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9999999999999997e-07 2024-09-14T22:38:02.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:02.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549467+0000", 2024-09-14T22:38:02.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0800000000000002e-06 2024-09-14T22:38:02.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:02.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549478+0000", 2024-09-14T22:38:02.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1813e-05 2024-09-14T22:38:02.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:02.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549488+0000", 2024-09-14T22:38:02.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7030000000000001e-06 2024-09-14T22:38:02.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:02.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630090+0000", 2024-09-14T22:38:02.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080601962999999999 2024-09-14T22:38:02.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.541 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:02.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630187+0000", 2024-09-14T22:38:02.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6552000000000007e-05 2024-09-14T22:38:02.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.541 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:02.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630311+0000", 2024-09-14T22:38:02.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000123826 2024-09-14T22:38:02.541 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.541 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:02.541 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.542 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144992 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3620864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:02.542 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:38:02.542 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9666555839999997, 2024-09-14T22:38:02.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081097875999999999, 2024-09-14T22:38:02.542 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:02.542 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:02.542 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:02.542 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:02.542 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:02.542 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:02.542 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144992 2024-09-14T22:38:02.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.543 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:02.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:02.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:38:02.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:02.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:38:02.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:02.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:38:02.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5400000000000001e-07 2024-09-14T22:38:02.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:02.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549502+0000", 2024-09-14T22:38:02.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7889999999999999e-06 2024-09-14T22:38:02.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:02.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549513+0000", 2024-09-14T22:38:02.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1204e-05 2024-09-14T22:38:02.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.545 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:02.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549523+0000", 2024-09-14T22:38:02.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5990000000000008e-06 2024-09-14T22:38:02.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.545 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:02.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630362+0000", 2024-09-14T22:38:02.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080839272000000004 2024-09-14T22:38:02.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.546 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:02.546 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630469+0000", 2024-09-14T22:38:02.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107285 2024-09-14T22:38:02.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.546 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:02.546 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630597+0000", 2024-09-14T22:38:02.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012707300000000001 2024-09-14T22:38:02.546 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.546 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:02.546 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.547 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144993 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:02.547 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:38:02.547 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9666225080000004, 2024-09-14T22:38:02.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081338411999999999, 2024-09-14T22:38:02.547 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:02.547 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:02.547 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:02.547 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:02.547 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:02.547 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:02.547 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144993 2024-09-14T22:38:02.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.548 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:02.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:02.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:38:02.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:02.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:38:02.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:02.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:38:02.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5300000000000004e-07 2024-09-14T22:38:02.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:02.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549544+0000", 2024-09-14T22:38:02.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1912e-05 2024-09-14T22:38:02.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:02.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549557+0000", 2024-09-14T22:38:02.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2413e-05 2024-09-14T22:38:02.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.550 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:02.550 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549567+0000", 2024-09-14T22:38:02.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8660000000000001e-06 2024-09-14T22:38:02.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.550 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:02.550 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630651+0000", 2024-09-14T22:38:02.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081084229999999993 2024-09-14T22:38:02.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.550 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:02.550 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630729+0000", 2024-09-14T22:38:02.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8265999999999999e-05 2024-09-14T22:38:02.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.551 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:02.551 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630870+0000", 2024-09-14T22:38:02.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014107200000000001 2024-09-14T22:38:02.551 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.551 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:02.551 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.551 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144994 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:02.551 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:38:02.551 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9665755409999992, 2024-09-14T22:38:02.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081547958000000004, 2024-09-14T22:38:02.552 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:02.552 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:02.552 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:02.552 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:02.552 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:02.552 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:02.552 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144994 2024-09-14T22:38:02.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.552 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:02.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:02.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:38:02.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:02.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:38:02.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:02.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:38:02.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6599999999999996e-07 2024-09-14T22:38:02.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:02.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549582+0000", 2024-09-14T22:38:02.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.004e-06 2024-09-14T22:38:02.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:02.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549594+0000", 2024-09-14T22:38:02.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1459e-05 2024-09-14T22:38:02.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:02.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549603+0000", 2024-09-14T22:38:02.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6129999999999993e-06 2024-09-14T22:38:02.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.555 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:02.555 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630932+0000", 2024-09-14T22:38:02.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081328558999999995 2024-09-14T22:38:02.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.555 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:02.555 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631027+0000", 2024-09-14T22:38:02.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4742999999999997e-05 2024-09-14T22:38:02.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.555 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:02.555 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631127+0000", 2024-09-14T22:38:02.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9914000000000006e-05 2024-09-14T22:38:02.556 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.556 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:02.556 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.556 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144995 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3391488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:02.556 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549615+0000", 2024-09-14T22:38:02.556 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9665394030000005, 2024-09-14T22:38:02.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081734251999999993, 2024-09-14T22:38:02.556 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:02.556 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:02.556 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:02.556 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:02.557 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:02.557 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:02.557 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144995 2024-09-14T22:38:02.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.557 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:02.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:02.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549615+0000", 2024-09-14T22:38:02.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:02.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549615+0000", 2024-09-14T22:38:02.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:02.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549616+0000", 2024-09-14T22:38:02.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9299999999999997e-07 2024-09-14T22:38:02.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:02.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549618+0000", 2024-09-14T22:38:02.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7269999999999999e-06 2024-09-14T22:38:02.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:02.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549630+0000", 2024-09-14T22:38:02.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1313e-05 2024-09-14T22:38:02.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:02.559 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549639+0000", 2024-09-14T22:38:02.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4159999999999993e-06 2024-09-14T22:38:02.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:02.559 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631183+0000", 2024-09-14T22:38:02.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081543633000000004 2024-09-14T22:38:02.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:02.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631252+0000", 2024-09-14T22:38:02.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9567999999999995e-05 2024-09-14T22:38:02.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.560 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:02.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631349+0000", 2024-09-14T22:38:02.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6801999999999999e-05 2024-09-14T22:38:02.560 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.560 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:02.560 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.560 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144996 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3194880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:02.561 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:38:02.561 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9665037250000008, 2024-09-14T22:38:02.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081947213000000005, 2024-09-14T22:38:02.561 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:02.561 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:02.561 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:02.561 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:02.561 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:02.561 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:02.561 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144996 2024-09-14T22:38:02.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.561 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:02.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:02.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:38:02.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:02.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:38:02.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:02.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:38:02.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3e-07 2024-09-14T22:38:02.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:02.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549654+0000", 2024-09-14T22:38:02.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6900000000000001e-06 2024-09-14T22:38:02.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:02.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549665+0000", 2024-09-14T22:38:02.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1392999999999999e-05 2024-09-14T22:38:02.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:02.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549675+0000", 2024-09-14T22:38:02.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5459999999999997e-06 2024-09-14T22:38:02.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.564 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:02.564 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631401+0000", 2024-09-14T22:38:02.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081726036000000002 2024-09-14T22:38:02.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.564 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:02.564 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631492+0000", 2024-09-14T22:38:02.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0886999999999999e-05 2024-09-14T22:38:02.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.564 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:02.565 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631598+0000", 2024-09-14T22:38:02.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106031 2024-09-14T22:38:02.565 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.565 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:02.565 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.565 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144997 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:02.565 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549692+0000", 2024-09-14T22:38:02.566 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9664620609999997, 2024-09-14T22:38:02.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082149925999999998, 2024-09-14T22:38:02.566 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:02.566 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:02.566 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:02.566 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:02.566 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:02.566 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:02.566 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144997 2024-09-14T22:38:02.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.566 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:02.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:02.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549692+0000", 2024-09-14T22:38:02.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:02.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549692+0000", 2024-09-14T22:38:02.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:02.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549693+0000", 2024-09-14T22:38:02.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.540000000000001e-07 2024-09-14T22:38:02.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:02.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549698+0000", 2024-09-14T22:38:02.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1460000000000004e-06 2024-09-14T22:38:02.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:02.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549715+0000", 2024-09-14T22:38:02.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6983e-05 2024-09-14T22:38:02.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:02.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549728+0000", 2024-09-14T22:38:02.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2326e-05 2024-09-14T22:38:02.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:02.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631649+0000", 2024-09-14T22:38:02.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081921830000000001 2024-09-14T22:38:02.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:02.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631726+0000", 2024-09-14T22:38:02.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6759000000000001e-05 2024-09-14T22:38:02.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:02.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631842+0000", 2024-09-14T22:38:02.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011592800000000001 2024-09-14T22:38:02.570 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.570 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:02.570 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.570 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144998 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 2965504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:02.570 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549741+0000", 2024-09-14T22:38:02.570 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9664127689999997, 2024-09-14T22:38:02.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082394832000000001, 2024-09-14T22:38:02.570 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:02.570 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:02.570 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:02.570 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:02.571 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:02.571 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:02.571 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144998 2024-09-14T22:38:02.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.571 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:02.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:02.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549741+0000", 2024-09-14T22:38:02.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:02.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549741+0000", 2024-09-14T22:38:02.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:02.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549742+0000", 2024-09-14T22:38:02.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.06e-07 2024-09-14T22:38:02.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:02.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549757+0000", 2024-09-14T22:38:02.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4839e-05 2024-09-14T22:38:02.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:02.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549770+0000", 2024-09-14T22:38:02.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2879e-05 2024-09-14T22:38:02.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:02.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549780+0000", 2024-09-14T22:38:02.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0304e-05 2024-09-14T22:38:02.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:02.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631904+0000", 2024-09-14T22:38:02.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082124168999999997 2024-09-14T22:38:02.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:02.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631995+0000", 2024-09-14T22:38:02.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0993000000000001e-05 2024-09-14T22:38:02.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:02.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632136+0000", 2024-09-14T22:38:02.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014094199999999999 2024-09-14T22:38:02.574 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.574 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:02.574 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.574 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144999 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3588096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:02.574 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:38:02.575 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9663625590000002, 2024-09-14T22:38:02.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082593742999999997, 2024-09-14T22:38:02.575 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:02.575 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:02.575 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:02.575 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:02.575 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:02.575 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:02.575 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144999 2024-09-14T22:38:02.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.575 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:02.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:02.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:38:02.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:02.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:38:02.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:02.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:38:02.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1999999999999999e-07 2024-09-14T22:38:02.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:02.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549795+0000", 2024-09-14T22:38:02.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9569999999999998e-06 2024-09-14T22:38:02.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:02.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549807+0000", 2024-09-14T22:38:02.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1691999999999999e-05 2024-09-14T22:38:02.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:02.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549817+0000", 2024-09-14T22:38:02.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5100000000000004e-06 2024-09-14T22:38:02.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:02.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632190+0000", 2024-09-14T22:38:02.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082373374999999999 2024-09-14T22:38:02.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:02.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632264+0000", 2024-09-14T22:38:02.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3972000000000003e-05 2024-09-14T22:38:02.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:02.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632385+0000", 2024-09-14T22:38:02.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121517 2024-09-14T22:38:02.579 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.579 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:02.579 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.579 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145000 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3063808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:02.579 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549828+0000", 2024-09-14T22:38:02.579 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9663262810000006, 2024-09-14T22:38:02.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082860235000000004, 2024-09-14T22:38:02.579 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:02.579 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:02.579 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:02.580 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:02.580 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:02.580 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:02.580 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145000 2024-09-14T22:38:02.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.580 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:02.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:02.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549828+0000", 2024-09-14T22:38:02.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:02.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549828+0000", 2024-09-14T22:38:02.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:02.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549829+0000", 2024-09-14T22:38:02.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3300000000000002e-07 2024-09-14T22:38:02.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:02.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549831+0000", 2024-09-14T22:38:02.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6740000000000001e-06 2024-09-14T22:38:02.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:02.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549842+0000", 2024-09-14T22:38:02.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0316000000000001e-05 2024-09-14T22:38:02.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:02.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549851+0000", 2024-09-14T22:38:02.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7229999999999999e-06 2024-09-14T22:38:02.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:02.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632448+0000", 2024-09-14T22:38:02.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082596648999999994 2024-09-14T22:38:02.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:02.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632568+0000", 2024-09-14T22:38:02.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119592 2024-09-14T22:38:02.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:02.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632688+0000", 2024-09-14T22:38:02.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120648 2024-09-14T22:38:02.584 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.584 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:02.584 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.584 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145001 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3162112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:02.584 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549863+0000", 2024-09-14T22:38:02.584 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9662910529999991, 2024-09-14T22:38:02.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083105534999999994, 2024-09-14T22:38:02.584 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:02.584 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:02.585 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:02.585 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:02.585 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:02.585 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:02.585 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145001 2024-09-14T22:38:02.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.585 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:02.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:02.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549863+0000", 2024-09-14T22:38:02.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:02.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549863+0000", 2024-09-14T22:38:02.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:02.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549864+0000", 2024-09-14T22:38:02.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4300000000000003e-07 2024-09-14T22:38:02.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:02.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549867+0000", 2024-09-14T22:38:02.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.757e-06 2024-09-14T22:38:02.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:02.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549878+0000", 2024-09-14T22:38:02.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1345999999999999e-05 2024-09-14T22:38:02.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:02.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549888+0000", 2024-09-14T22:38:02.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6460000000000005e-06 2024-09-14T22:38:02.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:02.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632754+0000", 2024-09-14T22:38:02.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082866258999999998 2024-09-14T22:38:02.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:02.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632841+0000", 2024-09-14T22:38:02.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6971000000000005e-05 2024-09-14T22:38:02.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:02.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632969+0000", 2024-09-14T22:38:02.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012791300000000001 2024-09-14T22:38:02.588 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.589 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:02.589 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.589 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145002 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3260416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:02.589 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549933+0000", 2024-09-14T22:38:02.589 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9662215500000002, 2024-09-14T22:38:02.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083291027000000004, 2024-09-14T22:38:02.589 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:02.589 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:02.589 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:02.589 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:02.589 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:02.590 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:02.590 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145002 2024-09-14T22:38:02.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.590 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:02.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:02.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549933+0000", 2024-09-14T22:38:02.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:02.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549933+0000", 2024-09-14T22:38:02.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:02.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549934+0000", 2024-09-14T22:38:02.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8700000000000004e-07 2024-09-14T22:38:02.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:02.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549938+0000", 2024-09-14T22:38:02.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5029999999999999e-06 2024-09-14T22:38:02.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:02.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549952+0000", 2024-09-14T22:38:02.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4382999999999999e-05 2024-09-14T22:38:02.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:02.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549963+0000", 2024-09-14T22:38:02.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0665999999999999e-05 2024-09-14T22:38:02.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:02.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633025+0000", 2024-09-14T22:38:02.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083061727000000002 2024-09-14T22:38:02.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:02.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633117+0000", 2024-09-14T22:38:02.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1812999999999997e-05 2024-09-14T22:38:02.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:02.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633224+0000", 2024-09-14T22:38:02.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107048 2024-09-14T22:38:02.593 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.593 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:02.593 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.594 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145003 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3457024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:02.594 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549985+0000", 2024-09-14T22:38:02.594 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9661691319999992, 2024-09-14T22:38:02.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083465871999999997, 2024-09-14T22:38:02.594 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:02.594 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:02.594 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:02.594 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:02.594 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:02.594 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:02.594 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145003 2024-09-14T22:38:02.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.594 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:02.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:02.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549985+0000", 2024-09-14T22:38:02.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:02.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549985+0000", 2024-09-14T22:38:02.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:02.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549986+0000", 2024-09-14T22:38:02.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7299999999999995e-07 2024-09-14T22:38:02.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:02.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549989+0000", 2024-09-14T22:38:02.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8169999999999999e-06 2024-09-14T22:38:02.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:02.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549998+0000", 2024-09-14T22:38:02.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4290000000000007e-06 2024-09-14T22:38:02.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:02.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550009+0000", 2024-09-14T22:38:02.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0633e-05 2024-09-14T22:38:02.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:02.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633275+0000", 2024-09-14T22:38:02.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083266397000000006 2024-09-14T22:38:02.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:02.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633353+0000", 2024-09-14T22:38:02.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8129999999999999e-05 2024-09-14T22:38:02.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:02.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633451+0000", 2024-09-14T22:38:02.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7793000000000004e-05 2024-09-14T22:38:02.598 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.598 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:02.598 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.598 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145004 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3358720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:02.599 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.550029+0000", 2024-09-14T22:38:02.599 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9661255610000001, 2024-09-14T22:38:02.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083662089999999995, 2024-09-14T22:38:02.599 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:02.599 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:02.599 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:02.599 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:02.599 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:02.599 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:02.599 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145004 2024-09-14T22:38:02.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.599 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:02.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:02.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550029+0000", 2024-09-14T22:38:02.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:02.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550029+0000", 2024-09-14T22:38:02.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:02.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550030+0000", 2024-09-14T22:38:02.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.33e-07 2024-09-14T22:38:02.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:02.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550035+0000", 2024-09-14T22:38:02.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3430000000000003e-06 2024-09-14T22:38:02.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:02.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550053+0000", 2024-09-14T22:38:02.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8485e-05 2024-09-14T22:38:02.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:02.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550064+0000", 2024-09-14T22:38:02.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0264e-05 2024-09-14T22:38:02.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:02.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633502+0000", 2024-09-14T22:38:02.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083438498999999999 2024-09-14T22:38:02.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:02.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633592+0000", 2024-09-14T22:38:02.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.022e-05 2024-09-14T22:38:02.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:02.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633691+0000", 2024-09-14T22:38:02.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8345999999999997e-05 2024-09-14T22:38:02.603 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.603 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:02.603 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.603 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145005 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3555328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:02.603 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.550077+0000", 2024-09-14T22:38:02.603 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9660771009999998, 2024-09-14T22:38:02.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083854340999999999, 2024-09-14T22:38:02.604 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:02.604 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:02.604 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:02.604 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:02.604 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:02.604 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:02.604 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145005 2024-09-14T22:38:02.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.604 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:02.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:02.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550077+0000", 2024-09-14T22:38:02.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:02.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550077+0000", 2024-09-14T22:38:02.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:02.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550078+0000", 2024-09-14T22:38:02.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8700000000000005e-07 2024-09-14T22:38:02.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:02.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550101+0000", 2024-09-14T22:38:02.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3632e-05 2024-09-14T22:38:02.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:02.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550115+0000", 2024-09-14T22:38:02.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3893000000000001e-05 2024-09-14T22:38:02.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:02.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550126+0000", 2024-09-14T22:38:02.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0426e-05 2024-09-14T22:38:02.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:02.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633742+0000", 2024-09-14T22:38:02.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083616633999999995 2024-09-14T22:38:02.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:02.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633816+0000", 2024-09-14T22:38:02.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3690999999999996e-05 2024-09-14T22:38:02.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:02.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633932+0000", 2024-09-14T22:38:02.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011537800000000001 2024-09-14T22:38:02.607 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.608 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:02.608 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.608 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145006 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3653632~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:02.608 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.550236+0000", 2024-09-14T22:38:02.608 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9659185949999998, 2024-09-14T22:38:02.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083924027999999998, 2024-09-14T22:38:02.608 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:02.609 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:02.609 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:02.609 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:02.609 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:02.609 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:02.609 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145006 2024-09-14T22:38:02.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.609 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:02.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:02.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550236+0000", 2024-09-14T22:38:02.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:02.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550236+0000", 2024-09-14T22:38:02.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:02.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:02.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550237+0000", 2024-09-14T22:38:02.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2929999999999999e-06 2024-09-14T22:38:02.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:02.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550243+0000", 2024-09-14T22:38:02.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0390000000000001e-06 2024-09-14T22:38:02.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:02.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550256+0000", 2024-09-14T22:38:02.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3393e-05 2024-09-14T22:38:02.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:02.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550271+0000", 2024-09-14T22:38:02.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4166e-05 2024-09-14T22:38:02.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:02.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633974+0000", 2024-09-14T22:38:02.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083703825999999995 2024-09-14T22:38:02.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:02.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.634047+0000", 2024-09-14T22:38:02.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2935999999999994e-05 2024-09-14T22:38:02.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:02.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:02.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:02.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.634160+0000", 2024-09-14T22:38:02.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011237499999999999 2024-09-14T22:38:02.613 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.613 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:02.613 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.613 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:02.613 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:02.613 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:02.613 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:38:02.614 INFO:teuthology.orchestra.run.smithi001.stdout:5496 write 0xb9b975c thru 0xb9c388b (0xa130 bytes) 2024-09-14T22:38:02.616 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:38:02.616 INFO:teuthology.orchestra.run.smithi001.stdout:5497 write 0x3b679bd thru 0x3b749ac (0xcff0 bytes) 2024-09-14T22:38:02.617 INFO:teuthology.orchestra.run.smithi001.stdout:5498 write 0xcb391c thru 0xcbbe90 (0x8575 bytes) 2024-09-14T22:38:02.617 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:02.617 INFO:teuthology.orchestra.run.smithi001.stdout:5500 trunc from 0xb9c388c to 0x9c015d6 2024-09-14T22:38:02.617 INFO:teuthology.orchestra.run.smithi001.stdout:5501 write 0xd325cb8 thru 0xd32eb7d (0x8ec6 bytes) 2024-09-14T22:38:02.617 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:02.610+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:02.618 INFO:teuthology.orchestra.run.smithi001.stdout:5502 write 0x986addd thru 0x987a65b (0xf87f bytes) 2024-09-14T22:38:02.629 INFO:teuthology.orchestra.run.smithi001.stdout:5503 punch from 0x9188d06 to 0x9190207, (0x7501 bytes) 2024-09-14T22:38:02.629 INFO:teuthology.orchestra.run.smithi001.stdout:5504 read 0x34a8b9d thru 0x34af2e0 (0x6744 bytes) 2024-09-14T22:38:02.629 INFO:teuthology.orchestra.run.smithi001.stdout:5505 punch from 0x62b19a2 to 0x62b940a, (0x7a68 bytes) 2024-09-14T22:38:02.629 INFO:teuthology.orchestra.run.smithi001.stdout:5506 punch from 0xc99f1d to 0xc9e5a0, (0x4683 bytes) 2024-09-14T22:38:02.629 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:02.629 INFO:teuthology.orchestra.run.smithi001.stdout:5509 read 0x742acd thru 0x747d9d (0x52d1 bytes) 2024-09-14T22:38:02.630 INFO:teuthology.orchestra.run.smithi001.stdout:5510 punch from 0xad0798f to 0xad0cdd2, (0x5443 bytes) 2024-09-14T22:38:02.630 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:02.630 INFO:teuthology.orchestra.run.smithi001.stdout:5513 write 0x7cf9a61 thru 0x7cfe66d (0x4c0d bytes) 2024-09-14T22:38:02.637 INFO:teuthology.orchestra.run.smithi001.stdout:5514 writesame 0x3f30933 thru 0x3f3c1ad data_size 0x1(0xb87b bytes) 2024-09-14T22:38:02.694 INFO:teuthology.orchestra.run.smithi001.stdout:5515 punch from 0x9e01bc9 to 0x9e0f846, (0xdc7d bytes) 2024-09-14T22:38:02.700 INFO:teuthology.orchestra.run.smithi001.stdout:5516 trunc from 0xd32eb7e to 0xa63d511 2024-09-14T22:38:02.716 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:02.710+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:02.723 INFO:teuthology.orchestra.run.smithi001.stdout:5517 read 0x5f304c9 thru 0x5f32d70 (0x28a8 bytes) 2024-09-14T22:38:02.724 INFO:teuthology.orchestra.run.smithi001.stdout:5519 trunc from 0xa63d511 to 0x9f24ee0 2024-09-14T22:38:02.739 INFO:teuthology.orchestra.run.smithi001.stdout:5523 trunc from 0x9f24ee0 to 0xba1bd85 2024-09-14T22:38:02.750 INFO:teuthology.orchestra.run.smithi001.stdout:5524 write 0x224beb thru 0x2341e5 (0xf5fb bytes) 2024-09-14T22:38:02.762 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:02.762 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:02.762 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:38:02.762 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:02.780 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:38:02.780 INFO:teuthology.orchestra.run.smithi001.stdout:5527 punch from 0x294010d to 0x2943f3e, (0x3e31 bytes) 2024-09-14T22:38:02.780 INFO:teuthology.orchestra.run.smithi001.stdout:5528 read 0xb82e51e thru 0xb83a72e (0xc211 bytes) 2024-09-14T22:38:02.780 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:02.786 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:38:02.787 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:38:02.791 INFO:teuthology.orchestra.run.smithi001.stdout:5532 writesame 0xd9bf3b2 thru 0xd9c5395 data_size 0x44(0x5fe4 bytes) 2024-09-14T22:38:02.791 INFO:teuthology.orchestra.run.smithi001.stdout:5533 read 0xa75c9da thru 0xa762e26 (0x644d bytes) 2024-09-14T22:38:02.791 INFO:teuthology.orchestra.run.smithi001.stdout:5534 read 0x171be94 thru 0x17295cf (0xd73c bytes) 2024-09-14T22:38:02.791 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:02.792 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:02.811 INFO:teuthology.orchestra.run.smithi001.stdout:5538 trunc from 0xd9c5396 to 0x1482861 2024-09-14T22:38:02.817 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:38:02.817 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:38:02.817 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:02.810+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:02.835 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:38:02.835 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:38:02.847 INFO:teuthology.orchestra.run.smithi001.stdout:5539 write 0xc25f56 thru 0xc354b0 (0xf55b bytes) 2024-09-14T22:38:02.854 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:38:02.855 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:38:02.874 INFO:teuthology.orchestra.run.smithi001.stdout:5540 write 0x45dda85 thru 0x45e8d14 (0xb290 bytes) 2024-09-14T22:38:02.880 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:38:02.880 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:38:02.884 INFO:teuthology.orchestra.run.smithi001.stdout:5541 punch from 0x40adff0 to 0x40b908a, (0xb09a bytes) 2024-09-14T22:38:02.884 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:02.884 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:02.884 INFO:teuthology.orchestra.run.smithi001.stdout:5546 punch from 0x380ee6 to 0x38d397, (0xc4b1 bytes) 2024-09-14T22:38:02.884 INFO:teuthology.orchestra.run.smithi001.stdout:5548 punch from 0x2df7806 to 0x2e02857, (0xb051 bytes) 2024-09-14T22:38:02.901 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:38:02.901 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:38:02.907 INFO:teuthology.orchestra.run.smithi001.stdout:5549 write 0xd26831e thru 0xd2689fd (0x6e0 bytes) 2024-09-14T22:38:02.913 INFO:teuthology.orchestra.run.smithi001.stdout:5550 read 0x228ea16 thru 0x2297ace (0x90b9 bytes) 2024-09-14T22:38:02.914 INFO:teuthology.orchestra.run.smithi001.stdout:5552 read 0x10deed9 thru 0x10e526d (0x6395 bytes) 2024-09-14T22:38:02.914 INFO:teuthology.orchestra.run.smithi001.stdout:5553 trunc from 0xd2689fe to 0xcaf60f7 2024-09-14T22:38:02.917 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:02.910+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:02.925 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:38:02.925 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:38:02.926 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:02.926 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:02.947 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:38:02.948 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:38:02.948 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:38:02.950 INFO:teuthology.orchestra.run.smithi001.stdout:5555 trunc from 0xcaf60f7 to 0x16cdfda 2024-09-14T22:38:02.962 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.1 to 1 (10000) 2024-09-14T22:38:02.962 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2024-09-14T22:38:02.993 INFO:teuthology.orchestra.run.smithi001.stdout:5556 punch from 0x14d86ba to 0x14d8ee8, (0x82e bytes) 2024-09-14T22:38:02.998 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-14T22:38:03.001 INFO:teuthology.orchestra.run.smithi001.stdout:5557 trunc from 0x16cdfda to 0x6e9a867 2024-09-14T22:38:03.015 INFO:teuthology.orchestra.run.smithi001.stdout:5559 trunc from 0x6e9a867 to 0x9ec5f63 2024-09-14T22:38:03.018 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:03.014+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:03.024 INFO:teuthology.orchestra.run.smithi001.stdout:5560 read 0x473e870 thru 0x473f695 (0xe26 bytes) 2024-09-14T22:38:03.029 INFO:teuthology.orchestra.run.smithi001.stdout:5561 trunc from 0x9ec5f63 to 0xdd0cd69 2024-09-14T22:38:03.039 INFO:teuthology.orchestra.run.smithi001.stdout:5562 punch from 0xacd29e1 to 0xacdbc69, (0x9288 bytes) 2024-09-14T22:38:03.057 INFO:teuthology.orchestra.run.smithi001.stdout:5564 trunc from 0xdd0cd69 to 0xd674e6 2024-09-14T22:38:03.094 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:03.094 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:03.096 INFO:teuthology.orchestra.run.smithi001.stdout:5565 punch from 0xa7092a to 0xa7fc35, (0xf30b bytes) 2024-09-14T22:38:03.117 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:38:03.118 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:03.110+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:03.119 INFO:teuthology.orchestra.run.smithi001.stdout:5567 write 0x8e26375 thru 0x8e2f3cf (0x905b bytes) 2024-09-14T22:38:03.127 INFO:teuthology.orchestra.run.smithi001.stdout:5569 read 0x1dd09eb thru 0x1dde9ab (0xdfc1 bytes) 2024-09-14T22:38:03.127 INFO:teuthology.orchestra.run.smithi001.stdout:5570 punch from 0xc7d3bd to 0xc8acac, (0xd8ef bytes) 2024-09-14T22:38:03.127 INFO:teuthology.orchestra.run.smithi001.stdout:5571 read 0x5527643 thru 0x5527afb (0x4b9 bytes) 2024-09-14T22:38:03.127 INFO:teuthology.orchestra.run.smithi001.stdout:5573 read 0x61abff9 thru 0x61ad6ac (0x16b4 bytes) 2024-09-14T22:38:03.127 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:03.127 INFO:teuthology.orchestra.run.smithi001.stdout:5575 read 0x795a9d0 thru 0x795f4fe (0x4b2f bytes) 2024-09-14T22:38:03.127 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:03.127 INFO:teuthology.orchestra.run.smithi001.stdout:5580 write 0x4e618d5 thru 0x4e6f365 (0xda91 bytes) 2024-09-14T22:38:03.134 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:03.134 INFO:teuthology.orchestra.run.smithi001.stdout:5582 punch from 0x7cf3767 to 0x7cf466b, (0xf04 bytes) 2024-09-14T22:38:03.134 INFO:teuthology.orchestra.run.smithi001.stdout:5583 write 0x7a5b538 thru 0x7a699b5 (0xe47e bytes) 2024-09-14T22:38:03.145 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:03.150 INFO:teuthology.orchestra.run.smithi001.stdout:5586 trunc from 0x8e2f3d0 to 0xcc5ca1a 2024-09-14T22:38:03.159 INFO:teuthology.orchestra.run.smithi001.stdout:5587 read 0x2f910d7 thru 0x2f976ca (0x65f4 bytes) 2024-09-14T22:38:03.159 INFO:teuthology.orchestra.run.smithi001.stdout:5588 flatten 2024-09-14T22:38:03.219 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:03.214+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:03.226 INFO:teuthology.orchestra.run.smithi001.stdout:5592 read 0x2b7acc3 thru 0x2b827be (0x7afc bytes) 2024-09-14T22:38:03.226 INFO:teuthology.orchestra.run.smithi001.stdout:5593 read 0xb9253f5 thru 0xb933551 (0xe15d bytes) 2024-09-14T22:38:03.226 INFO:teuthology.orchestra.run.smithi001.stdout:5594 read 0x1058f07 thru 0x1058fa2 (0x9c bytes) 2024-09-14T22:38:03.226 INFO:teuthology.orchestra.run.smithi001.stdout:5597 write 0x67e90cc thru 0x67ec0cf (0x3004 bytes) 2024-09-14T22:38:03.231 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:03.231 INFO:teuthology.orchestra.run.smithi001.stdout:5599 write 0xbf5b61b thru 0xbf5d874 (0x225a bytes) 2024-09-14T22:38:03.238 INFO:teuthology.orchestra.run.smithi001.stdout:5600 read 0x4d4f8a4 thru 0x4d52478 (0x2bd5 bytes) 2024-09-14T22:38:03.238 INFO:teuthology.orchestra.run.smithi001.stdout:5601 punch from 0x87f2f40 to 0x87fb05f, (0x811f bytes) 2024-09-14T22:38:03.238 INFO:teuthology.orchestra.run.smithi001.stdout:5602 writesame 0x45a6cda thru 0x45af7aa data_size 0x1(0x8ad1 bytes) 2024-09-14T22:38:03.251 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:03.251 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:03.266 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:38:03.277 INFO:teuthology.orchestra.run.smithi001.stdout:5603 write 0x2cfe126 thru 0x2d06ecb (0x8da6 bytes) 2024-09-14T22:38:03.297 INFO:teuthology.orchestra.run.smithi001.stdout:5604 trunc from 0xcc5ca1a to 0x58f28e9 2024-09-14T22:38:03.318 INFO:teuthology.orchestra.run.smithi001.stdout:5605 punch from 0x4749669 to 0x475256a, (0x8f01 bytes) 2024-09-14T22:38:03.319 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:03.314+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:03.330 INFO:teuthology.orchestra.run.smithi001.stdout:5608 write 0x778943d thru 0x7795a5c (0xc620 bytes) 2024-09-14T22:38:03.340 INFO:teuthology.orchestra.run.smithi001.stdout:5609 write 0x3dae633 thru 0x3db2132 (0x3b00 bytes) 2024-09-14T22:38:03.349 INFO:teuthology.orchestra.run.smithi001.stdout:5610 writesame 0x28e9bec thru 0x28eab3f data_size 0x12(0xf54 bytes) 2024-09-14T22:38:03.379 INFO:teuthology.orchestra.run.smithi001.stdout:5612 write 0xc5a54a1 thru 0xc5b416a (0xecca bytes) 2024-09-14T22:38:03.389 INFO:teuthology.orchestra.run.smithi001.stdout:5613 write 0x40193fc thru 0x401cc77 (0x387c bytes) 2024-09-14T22:38:03.408 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:03.408 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:03.418 INFO:teuthology.orchestra.run.smithi001.stdout:5614 trunc from 0xc5b416b to 0x1110fa3 2024-09-14T22:38:03.419 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:03.414+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:03.430 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:38:03.445 INFO:teuthology.orchestra.run.smithi001.stdout:5616 punch from 0x105760e to 0x105d0f1, (0x5ae3 bytes) 2024-09-14T22:38:03.477 INFO:teuthology.orchestra.run.smithi001.stdout:5618 write 0xed27089 thru 0xed2a889 (0x3801 bytes) 2024-09-14T22:38:03.485 INFO:teuthology.orchestra.run.smithi001.stdout:5619 writesame 0x239df8f thru 0x23a8e7b data_size 0x45(0xaeed bytes) 2024-09-14T22:38:03.495 INFO:teuthology.orchestra.run.smithi001.stdout:5620 punch from 0x19b767a to 0x19c0fb9, (0x993f bytes) 2024-09-14T22:38:03.495 INFO:teuthology.orchestra.run.smithi001.stdout:5623 punch from 0xd3ea612 to 0xd3f836f, (0xdd5d bytes) 2024-09-14T22:38:03.495 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:03.495 INFO:teuthology.orchestra.run.smithi001.stdout:5625 write 0x677a092 thru 0x677c1db (0x214a bytes) 2024-09-14T22:38:03.502 INFO:teuthology.orchestra.run.smithi001.stdout:5626 write 0xc258253 thru 0xc26364c (0xb3fa bytes) 2024-09-14T22:38:03.514 INFO:teuthology.orchestra.run.smithi001.stdout:5627 punch from 0x183a9b6 to 0x1847d75, (0xd3bf bytes) 2024-09-14T22:38:03.514 INFO:teuthology.orchestra.run.smithi001.stdout:5628 write 0x9c07b79 thru 0x9c129c8 (0xae50 bytes) 2024-09-14T22:38:03.521 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:03.514+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:03.525 INFO:teuthology.orchestra.run.smithi001.stdout:5630 write 0x597b5ee thru 0x5985b03 (0xa516 bytes) 2024-09-14T22:38:03.560 INFO:teuthology.orchestra.run.smithi001.stdout:5631 trunc from 0xed2a88a to 0x45fb75d 2024-09-14T22:38:03.577 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:03.577 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:03.588 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:03.592 INFO:teuthology.orchestra.run.smithi001.stdout:5633 trunc from 0x45fb75d to 0x2627ffb 2024-09-14T22:38:03.594 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:38:03.605 INFO:teuthology.orchestra.run.smithi001.stdout:5634 punch from 0xd2fcf3 to 0xd3b976, (0xbc83 bytes) 2024-09-14T22:38:03.605 INFO:teuthology.orchestra.run.smithi001.stdout:5635 read 0xb9aef3 thru 0xba91a6 (0xe2b4 bytes) 2024-09-14T22:38:03.605 INFO:teuthology.orchestra.run.smithi001.stdout:5636 read 0x24556ae thru 0x2456e5a (0x17ad bytes) 2024-09-14T22:38:03.605 INFO:teuthology.orchestra.run.smithi001.stdout:5637 read 0x15ca2d6 thru 0x15d07b6 (0x64e1 bytes) 2024-09-14T22:38:03.622 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:03.614+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:03.640 INFO:teuthology.orchestra.run.smithi001.stdout:5638 writesame 0xce91972 thru 0xce9ff20 data_size 0x1(0xe5af bytes) 2024-09-14T22:38:03.641 INFO:teuthology.orchestra.run.smithi001.stdout:5639 read 0xbcdb91f thru 0xbce689e (0xaf80 bytes) 2024-09-14T22:38:03.641 INFO:teuthology.orchestra.run.smithi001.stdout:5640 write 0x65d10c thru 0x663be9 (0x6ade bytes) 2024-09-14T22:38:03.648 INFO:teuthology.orchestra.run.smithi001.stdout:5643 write 0x371142d thru 0x371f446 (0xe01a bytes) 2024-09-14T22:38:03.656 INFO:teuthology.orchestra.run.smithi001.stdout:5644 write 0x54a1aa7 thru 0x54afec0 (0xe41a bytes) 2024-09-14T22:38:03.662 INFO:teuthology.orchestra.run.smithi001.stdout:5645 writesame 0x8e10c8e thru 0x8e1c24f data_size 0x63(0xb5c2 bytes) 2024-09-14T22:38:03.670 INFO:teuthology.orchestra.run.smithi001.stdout:5646 write 0xb731d1d thru 0xb737ac4 (0x5da8 bytes) 2024-09-14T22:38:03.677 INFO:teuthology.orchestra.run.smithi001.stdout:5648 read 0x9e9aa19 thru 0x9ea61c1 (0xb7a9 bytes) 2024-09-14T22:38:03.677 INFO:teuthology.orchestra.run.smithi001.stdout:5649 read 0x94aa55b thru 0x94b96c4 (0xf16a bytes) 2024-09-14T22:38:03.677 INFO:teuthology.orchestra.run.smithi001.stdout:5650 write 0x3714aad thru 0x3717615 (0x2b69 bytes) 2024-09-14T22:38:03.685 INFO:teuthology.orchestra.run.smithi001.stdout:5651 writesame 0x1b61d5f thru 0x1b6227b data_size 0xb(0x51d bytes) 2024-09-14T22:38:03.695 INFO:teuthology.orchestra.run.smithi001.stdout:5652 write 0x16c31d4 thru 0x16cad22 (0x7b4f bytes) 2024-09-14T22:38:03.723 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:03.718+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:03.729 INFO:teuthology.orchestra.run.smithi001.stdout:5653 trunc from 0xce9ff21 to 0x13a7c98 2024-09-14T22:38:03.735 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:03.735 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:03.752 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:38:03.755 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:03.773 INFO:teuthology.orchestra.run.smithi001.stdout:5656 write 0x4412acc thru 0x441b052 (0x8587 bytes) 2024-09-14T22:38:03.803 INFO:teuthology.orchestra.run.smithi001.stdout:5657 write 0xbf7a268 thru 0xbf8844d (0xe1e6 bytes) 2024-09-14T22:38:03.812 INFO:teuthology.orchestra.run.smithi001.stdout:5658 writesame 0x697e4bc thru 0x698c4c3 data_size 0x86(0xe008 bytes) 2024-09-14T22:38:03.822 INFO:teuthology.orchestra.run.smithi001.stdout:5661 writesame 0x1ea5cf6 thru 0x1ea9f8e data_size 0x1(0x4299 bytes) 2024-09-14T22:38:03.824 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:03.818+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:03.847 INFO:teuthology.orchestra.run.smithi001.stdout:5663 read 0x8289b29 thru 0x8294e81 (0xb359 bytes) 2024-09-14T22:38:03.847 INFO:teuthology.orchestra.run.smithi001.stdout:5664 write 0xb21b8ff thru 0xb21e9b0 (0x30b2 bytes) 2024-09-14T22:38:03.874 INFO:teuthology.orchestra.run.smithi001.stdout:5665 trunc from 0xbf8844e to 0x33f709f 2024-09-14T22:38:03.904 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:03.904 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:03.925 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:38:03.926 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:03.918+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:03.928 INFO:teuthology.orchestra.run.smithi001.stdout:5666 write 0xb4c8cf0 thru 0xb4d3451 (0xa762 bytes) 2024-09-14T22:38:03.948 INFO:teuthology.orchestra.run.smithi001.stdout:5668 trunc from 0xb4d3452 to 0x4b1c6c1 2024-09-14T22:38:03.971 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.2 to 1 (10000) 2024-09-14T22:38:03.975 INFO:teuthology.orchestra.run.smithi001.stdout:5669 read 0xa7fc2 thru 0xb456a (0xc5a9 bytes) 2024-09-14T22:38:04.000 INFO:teuthology.orchestra.run.smithi001.stdout:5670 write 0x95ac36e thru 0x95b4fb5 (0x8c48 bytes) 2024-09-14T22:38:04.004 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-14T22:38:04.011 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:04.025 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:04.018+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:04.026 INFO:teuthology.orchestra.run.smithi001.stdout:5673 writesame 0xc3007ab thru 0xc30fbe9 data_size 0x1(0xf43f bytes) 2024-09-14T22:38:04.026 INFO:teuthology.orchestra.run.smithi001.stdout:5675 write 0x5c57375 thru 0x5c61ce4 (0xa970 bytes) 2024-09-14T22:38:04.037 INFO:teuthology.orchestra.run.smithi001.stdout:5676 read 0x7fa2379 thru 0x7fa44c9 (0x2151 bytes) 2024-09-14T22:38:04.037 INFO:teuthology.orchestra.run.smithi001.stdout:5677 read 0xb59decd thru 0xb5a7416 (0x954a bytes) 2024-09-14T22:38:04.038 INFO:teuthology.orchestra.run.smithi001.stdout:5678 trunc from 0xc30fbea to 0xc59d13e 2024-09-14T22:38:04.047 INFO:teuthology.orchestra.run.smithi001.stdout:5679 read 0x17afee9 thru 0x17b2d03 (0x2e1b bytes) 2024-09-14T22:38:04.047 INFO:teuthology.orchestra.run.smithi001.stdout:5680 punch from 0xb72d134 to 0xb73be7f, (0xed4b bytes) 2024-09-14T22:38:04.047 INFO:teuthology.orchestra.run.smithi001.stdout:5681 read 0x926d3bc thru 0x92700d1 (0x2d16 bytes) 2024-09-14T22:38:04.047 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:04.047 INFO:teuthology.orchestra.run.smithi001.stdout:5683 write 0x34958aa thru 0x34a2b61 (0xd2b8 bytes) 2024-09-14T22:38:04.057 INFO:teuthology.orchestra.run.smithi001.stdout:5685 read 0x9d30ca7 thru 0x9d32192 (0x14ec bytes) 2024-09-14T22:38:04.057 INFO:teuthology.orchestra.run.smithi001.stdout:5686 write 0x9e2e9de thru 0x9e36fe9 (0x860c bytes) 2024-09-14T22:38:04.066 INFO:teuthology.orchestra.run.smithi001.stdout:5689 write 0x3ac9048 thru 0x3ac9593 (0x54c bytes) 2024-09-14T22:38:04.070 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:04.070 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:04.073 INFO:teuthology.orchestra.run.smithi001.stdout:5690 read 0x82e7f82 thru 0x82e878f (0x80e bytes) 2024-09-14T22:38:04.073 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:04.073 INFO:teuthology.orchestra.run.smithi001.stdout:5693 read 0x59ba422 thru 0x59c953e (0xf11d bytes) 2024-09-14T22:38:04.073 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:04.073 INFO:teuthology.orchestra.run.smithi001.stdout:5695 writesame 0x1a7361b thru 0x1a7d7b8 data_size 0x89(0xa19e bytes) 2024-09-14T22:38:04.080 INFO:teuthology.orchestra.run.smithi001.stdout:5697 write 0x120dfa6 thru 0x1218bfb (0xac56 bytes) 2024-09-14T22:38:04.087 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:38:04.089 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:04.089 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:04.089 INFO:teuthology.orchestra.run.smithi001.stdout:5702 read 0x43963f3 thru 0x43a0d20 (0xa92e bytes) 2024-09-14T22:38:04.089 INFO:teuthology.orchestra.run.smithi001.stdout:5703 write 0x243cdb thru 0x251bcb (0xdef1 bytes) 2024-09-14T22:38:04.092 INFO:teuthology.orchestra.run.smithi001.stdout:5705 read 0xc2e695d thru 0xc2ee684 (0x7d28 bytes) 2024-09-14T22:38:04.092 INFO:teuthology.orchestra.run.smithi001.stdout:5706 read 0x22ccd52 thru 0x22ce96b (0x1c1a bytes) 2024-09-14T22:38:04.092 INFO:teuthology.orchestra.run.smithi001.stdout:5707 writesame 0x3a12003 thru 0x3a1d633 data_size 0x1(0xb631 bytes) 2024-09-14T22:38:04.120 INFO:teuthology.orchestra.run.smithi001.stdout:5709 read 0x7e3345e thru 0x7e38bd7 (0x577a bytes) 2024-09-14T22:38:04.120 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:04.121 INFO:teuthology.orchestra.run.smithi001.stdout:5711 read 0x3608dc3 thru 0x360c022 (0x3260 bytes) 2024-09-14T22:38:04.121 INFO:teuthology.orchestra.run.smithi001.stdout:5712 write 0x354e51f thru 0x355e3b2 (0xfe94 bytes) 2024-09-14T22:38:04.126 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:04.122+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:04.129 INFO:teuthology.orchestra.run.smithi001.stdout:5716 read 0x817d71 thru 0x81bfa6 (0x4236 bytes) 2024-09-14T22:38:04.129 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:04.129 INFO:teuthology.orchestra.run.smithi001.stdout:5718 punch from 0xaf095a to 0xb00901, (0xffa7 bytes) 2024-09-14T22:38:04.129 INFO:teuthology.orchestra.run.smithi001.stdout:5719 read 0xa616e44 thru 0xa6175ea (0x7a7 bytes) 2024-09-14T22:38:04.129 INFO:teuthology.orchestra.run.smithi001.stdout:5720 read 0x3a0e9cc thru 0x3a15b6a (0x719f bytes) 2024-09-14T22:38:04.130 INFO:teuthology.orchestra.run.smithi001.stdout:5722 punch from 0x946a1a8 to 0x9476c5e, (0xcab6 bytes) 2024-09-14T22:38:04.130 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:04.130 INFO:teuthology.orchestra.run.smithi001.stdout:5725 writesame 0x11dbe thru 0x21407 data_size 0x41(0xf64a bytes) 2024-09-14T22:38:04.132 INFO:teuthology.orchestra.run.smithi001.stdout:5726 read 0x26316d0 thru 0x263cc4c (0xb57d bytes) 2024-09-14T22:38:04.140 INFO:teuthology.orchestra.run.smithi001.stdout:5727 writesame 0xcc78bd8 thru 0xcc7f17d data_size 0x3(0x65a6 bytes) 2024-09-14T22:38:04.140 INFO:teuthology.orchestra.run.smithi001.stdout:5728 read 0xc6416bd thru 0xc644eac (0x37f0 bytes) 2024-09-14T22:38:04.140 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:04.140 INFO:teuthology.orchestra.run.smithi001.stdout:5730 write 0x68ee412 thru 0x68f023e (0x1e2d bytes) 2024-09-14T22:38:04.146 INFO:teuthology.orchestra.run.smithi001.stdout:5731 writesame 0x3de6d7e thru 0x3df38b2 data_size 0x1(0xcb35 bytes) 2024-09-14T22:38:04.180 INFO:teuthology.orchestra.run.smithi001.stdout:5732 read 0x566fa3e thru 0x567f44e (0xfa11 bytes) 2024-09-14T22:38:04.180 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:04.180 INFO:teuthology.orchestra.run.smithi001.stdout:5734 write 0x3337477 thru 0x3344e87 (0xda11 bytes) 2024-09-14T22:38:04.187 INFO:teuthology.orchestra.run.smithi001.stdout:5735 trunc from 0xcc7f17e to 0xc669abb 2024-09-14T22:38:04.195 INFO:teuthology.orchestra.run.smithi001.stdout:5736 write 0x8bc276d thru 0x8bc9649 (0x6edd bytes) 2024-09-14T22:38:04.217 INFO:teuthology.orchestra.run.smithi001.stdout:5738 trunc from 0xc669abb to 0x57d08f1 2024-09-14T22:38:04.226 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:04.222+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:04.234 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:04.234 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:04.249 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:38:04.255 INFO:teuthology.orchestra.run.smithi001.stdout:5739 trunc from 0x57d08f1 to 0x96558b7 2024-09-14T22:38:04.267 INFO:teuthology.orchestra.run.smithi001.stdout:5740 punch from 0x90a57df to 0x90b2f4c, (0xd76d bytes) 2024-09-14T22:38:04.267 INFO:teuthology.orchestra.run.smithi001.stdout:5741 write 0x51d8c80 thru 0x51de880 (0x5c01 bytes) 2024-09-14T22:38:04.273 INFO:teuthology.orchestra.run.smithi001.stdout:5742 read 0x1befb82 thru 0x1bf2db4 (0x3233 bytes) 2024-09-14T22:38:04.273 INFO:teuthology.orchestra.run.smithi001.stdout:5743 clone 37 order 21 su 65536 sc 4 2024-09-14T22:38:04.328 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:04.322+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:04.371 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:04.371 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:04.384 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:38:04.431 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:04.426+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:04.529 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:04.522+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:04.548 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:04.548 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:04.563 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:38:04.655 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:04.650+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:04.730 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:04.730 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:04.731 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:04.726+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:04.744 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:38:04.832 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:04.830+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:04.912 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:04.912 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:04.926 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:38:04.971 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:04.966+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:04.972 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:38:04.976 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.3 to 1 (10000) 2024-09-14T22:38:04.990 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T22:38:05.008 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-14T22:38:05.050 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:05.046+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:05.051 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:05.051 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:05.064 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:38:05.122 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:38:05.123 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:38:05.134 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:05.130+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:05.151 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:38:05.151 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:38:05.169 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:38:05.169 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:38:05.191 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:38:05.191 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:38:05.211 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:38:05.212 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:38:05.230 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:05.230 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:05.232 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:38:05.232 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:38:05.234 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:05.230+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:05.244 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:38:05.252 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:38:05.252 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:38:05.270 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:38:05.270 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:38:05.336 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:05.330+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:05.367 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:05.367 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:05.367 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:38:05.367 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:38:05.367 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:05.381 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:38:05.437 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:05.434+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:05.538 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:05.530+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:05.549 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:05.549 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:38:05.549 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:38:05.550 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:38:05.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.550 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:72252 3.2 3:442c6d72:::rbd_data.113cb4c83968.0000000000000015:head [zero 4456448~393216] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-14T22:38:05.550 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:38:05.550 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 74.844894929000006, 2024-09-14T22:38:05.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0176591500000001, 2024-09-14T22:38:05.550 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:05.550 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:05.550 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T22:38:05.550 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:05.550 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:05.550 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:05.551 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 72252 2024-09-14T22:38:05.551 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.551 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:05.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.551 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:05.551 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:38:05.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.551 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.551 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:05.551 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:38:05.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.551 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.552 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:05.552 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700101+0000", 2024-09-14T22:38:05.552 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4570000000000001e-06 2024-09-14T22:38:05.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.552 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:05.552 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700114+0000", 2024-09-14T22:38:05.552 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2799999999999999e-05 2024-09-14T22:38:05.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.552 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:05.552 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700146+0000", 2024-09-14T22:38:05.552 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2116000000000002e-05 2024-09-14T22:38:05.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.553 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.553 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:05.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700170+0000", 2024-09-14T22:38:05.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3770000000000001e-05 2024-09-14T22:38:05.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.553 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.553 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:05.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700285+0000", 2024-09-14T22:38:05.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000115228 2024-09-14T22:38:05.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.553 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.553 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op must wait for map", 2024-09-14T22:38:05.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700297+0000", 2024-09-14T22:38:05.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1681999999999999e-05 2024-09-14T22:38:05.554 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.554 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:05.554 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.703312+0000", 2024-09-14T22:38:05.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003015046 2024-09-14T22:38:05.554 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.554 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:38:05.554 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.703327+0000", 2024-09-14T22:38:05.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5481999999999998e-05 2024-09-14T22:38:05.554 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.554 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:05.555 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:51.717566+0000", 2024-09-14T22:38:05.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0142383509999999 2024-09-14T22:38:05.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.555 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.555 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:05.555 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:51.717757+0000", 2024-09-14T22:38:05.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000191218 2024-09-14T22:38:05.555 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.555 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:05.555 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.555 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.555 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:74156 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T22:38:05.556 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:38:05.556 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 70.030426219000006, 2024-09-14T22:38:05.556 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062073521, 2024-09-14T22:38:05.556 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:05.556 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:05.556 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:05.556 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:05.556 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:05.556 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:05.556 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74156 2024-09-14T22:38:05.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.556 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:05.556 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.557 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:05.557 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:38:05.557 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.557 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.557 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:05.557 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:38:05.557 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.557 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.557 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:05.557 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514569+0000", 2024-09-14T22:38:05.557 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0229999999999999e-06 2024-09-14T22:38:05.557 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.558 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:05.558 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514596+0000", 2024-09-14T22:38:05.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6903e-05 2024-09-14T22:38:05.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.558 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:05.558 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514623+0000", 2024-09-14T22:38:05.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7206e-05 2024-09-14T22:38:05.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.558 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:05.558 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514637+0000", 2024-09-14T22:38:05.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:38:05.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.559 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.559 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:05.559 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514781+0000", 2024-09-14T22:38:05.559 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014394899999999999 2024-09-14T22:38:05.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.559 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.559 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:05.559 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.515005+0000", 2024-09-14T22:38:05.559 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022458499999999999 2024-09-14T22:38:05.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.559 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.559 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:38:05.559 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.527785+0000", 2024-09-14T22:38:05.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012780112 2024-09-14T22:38:05.560 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.560 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.560 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:38:05.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576209+0000", 2024-09-14T22:38:05.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.048423992999999999 2024-09-14T22:38:05.560 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.560 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.560 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:05.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576346+0000", 2024-09-14T22:38:05.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000136909 2024-09-14T22:38:05.560 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.560 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.561 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:38:05.561 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576517+0000", 2024-09-14T22:38:05.561 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000170848 2024-09-14T22:38:05.561 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.561 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.561 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:05.561 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576640+0000", 2024-09-14T22:38:05.561 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000122953 2024-09-14T22:38:05.561 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.561 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:05.561 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.561 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.561 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.561 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82513 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 557056~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:05.561 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:38:05.562 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.125794400000004, 2024-09-14T22:38:05.562 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059345486000000003, 2024-09-14T22:38:05.562 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:05.562 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:05.562 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:05.562 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:05.562 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:05.562 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:05.562 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82513 2024-09-14T22:38:05.562 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.562 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:05.562 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.562 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:05.562 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:38:05.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.563 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.563 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:05.563 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:38:05.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.563 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.563 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:05.563 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419200+0000", 2024-09-14T22:38:05.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.184e-06 2024-09-14T22:38:05.563 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.563 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:05.564 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419207+0000", 2024-09-14T22:38:05.564 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2690000000000002e-06 2024-09-14T22:38:05.564 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.564 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.564 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:05.564 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419224+0000", 2024-09-14T22:38:05.564 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6623e-05 2024-09-14T22:38:05.564 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.564 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.564 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:05.564 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419236+0000", 2024-09-14T22:38:05.564 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2367e-05 2024-09-14T22:38:05.565 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.565 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.565 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:05.565 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478294+0000", 2024-09-14T22:38:05.565 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059057927000000003 2024-09-14T22:38:05.565 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.565 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.565 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:05.565 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478402+0000", 2024-09-14T22:38:05.565 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107921 2024-09-14T22:38:05.565 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.565 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.565 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:05.565 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478544+0000", 2024-09-14T22:38:05.566 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014219499999999999 2024-09-14T22:38:05.566 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.566 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:05.566 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.566 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.566 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.566 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82514 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 704512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:05.566 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:38:05.566 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.125735297000006, 2024-09-14T22:38:05.566 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059580793, 2024-09-14T22:38:05.566 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:05.566 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:05.566 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:05.566 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:05.567 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:05.567 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:05.567 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82514 2024-09-14T22:38:05.567 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.567 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:05.567 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.567 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:05.567 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:38:05.567 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.567 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.567 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.567 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:05.567 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:38:05.568 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.568 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.568 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.568 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:05.568 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419259+0000", 2024-09-14T22:38:05.568 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.353e-06 2024-09-14T22:38:05.568 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.568 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.568 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:05.568 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419276+0000", 2024-09-14T22:38:05.568 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7567e-05 2024-09-14T22:38:05.568 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.568 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.569 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:05.569 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419294+0000", 2024-09-14T22:38:05.569 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.713e-05 2024-09-14T22:38:05.569 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.569 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.569 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:05.569 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419306+0000", 2024-09-14T22:38:05.569 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.279e-05 2024-09-14T22:38:05.569 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.569 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.569 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:05.569 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478589+0000", 2024-09-14T22:38:05.570 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059282978 2024-09-14T22:38:05.570 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.570 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.570 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:05.570 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478694+0000", 2024-09-14T22:38:05.570 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000104561 2024-09-14T22:38:05.570 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.570 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.570 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:05.570 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478838+0000", 2024-09-14T22:38:05.570 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014441399999999999 2024-09-14T22:38:05.570 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.570 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:05.571 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.571 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.571 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.571 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82515 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:05.571 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:38:05.571 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.125670146999994, 2024-09-14T22:38:05.571 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059815123999999997, 2024-09-14T22:38:05.571 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:05.571 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:05.571 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:05.571 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:05.571 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:05.571 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:05.571 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82515 2024-09-14T22:38:05.572 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.572 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:05.572 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.572 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:05.572 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:38:05.572 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.572 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.572 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.572 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:05.572 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:38:05.572 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.572 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.572 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.573 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:05.573 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419324+0000", 2024-09-14T22:38:05.573 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2130000000000001e-06 2024-09-14T22:38:05.573 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.573 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.573 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:05.573 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419331+0000", 2024-09-14T22:38:05.573 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4730000000000003e-06 2024-09-14T22:38:05.573 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.573 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.573 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:05.573 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419348+0000", 2024-09-14T22:38:05.573 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6269999999999998e-05 2024-09-14T22:38:05.573 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.574 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.574 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:05.574 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419360+0000", 2024-09-14T22:38:05.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2243e-05 2024-09-14T22:38:05.574 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.574 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.574 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:05.574 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478884+0000", 2024-09-14T22:38:05.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059524369000000001 2024-09-14T22:38:05.574 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.574 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.574 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:05.574 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478990+0000", 2024-09-14T22:38:05.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010569 2024-09-14T22:38:05.575 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.575 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.575 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:05.575 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479138+0000", 2024-09-14T22:38:05.575 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000147866 2024-09-14T22:38:05.575 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.575 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:05.575 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.575 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.575 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.575 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82516 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 999424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:05.575 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:38:05.575 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.125617641999995, 2024-09-14T22:38:05.575 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061979595999999998, 2024-09-14T22:38:05.576 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:05.576 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:05.576 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:05.576 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:05.576 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:05.576 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:05.576 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82516 2024-09-14T22:38:05.576 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.576 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:05.576 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.576 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:05.576 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:38:05.576 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.577 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.577 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.577 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:05.577 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:38:05.577 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.577 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.577 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.577 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:05.577 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419376+0000", 2024-09-14T22:38:05.577 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.237e-06 2024-09-14T22:38:05.577 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.577 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.577 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:05.578 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419383+0000", 2024-09-14T22:38:05.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8170000000000001e-06 2024-09-14T22:38:05.578 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.578 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:05.578 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419400+0000", 2024-09-14T22:38:05.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6636000000000001e-05 2024-09-14T22:38:05.578 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.578 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:05.578 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419412+0000", 2024-09-14T22:38:05.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2357e-05 2024-09-14T22:38:05.578 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.579 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:05.579 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481080+0000", 2024-09-14T22:38:05.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061668032999999997 2024-09-14T22:38:05.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.579 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:05.579 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481193+0000", 2024-09-14T22:38:05.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112335 2024-09-14T22:38:05.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.579 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:05.580 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481355+0000", 2024-09-14T22:38:05.580 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016218100000000001 2024-09-14T22:38:05.580 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.580 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:05.580 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.580 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82538 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1097728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:05.580 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:38:05.580 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.125565571999999, 2024-09-14T22:38:05.580 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059971705, 2024-09-14T22:38:05.580 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:05.580 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:05.581 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:05.581 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:05.581 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:05.581 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:05.581 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82538 2024-09-14T22:38:05.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.581 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:05.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.581 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:05.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:38:05.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.582 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:05.582 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:38:05.582 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.582 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.582 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:05.582 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419429+0000", 2024-09-14T22:38:05.582 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.497e-06 2024-09-14T22:38:05.582 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.582 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:05.582 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419436+0000", 2024-09-14T22:38:05.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2099999999999996e-06 2024-09-14T22:38:05.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.583 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:05.583 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419453+0000", 2024-09-14T22:38:05.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6840000000000001e-05 2024-09-14T22:38:05.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.583 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:05.583 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419465+0000", 2024-09-14T22:38:05.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2283000000000001e-05 2024-09-14T22:38:05.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:05.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479182+0000", 2024-09-14T22:38:05.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059717339000000001 2024-09-14T22:38:05.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:05.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479297+0000", 2024-09-14T22:38:05.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000114259 2024-09-14T22:38:05.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:05.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479399+0000", 2024-09-14T22:38:05.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102277 2024-09-14T22:38:05.585 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.585 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:05.585 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.585 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82539 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:05.585 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:38:05.585 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.125512721999996, 2024-09-14T22:38:05.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060166935999999997, 2024-09-14T22:38:05.585 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:05.585 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:05.585 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:05.585 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:05.586 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:05.586 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:05.586 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82539 2024-09-14T22:38:05.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.586 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:05.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.586 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:05.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:38:05.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.586 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:05.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:38:05.587 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.587 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:05.587 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419482+0000", 2024-09-14T22:38:05.587 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4130000000000001e-06 2024-09-14T22:38:05.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.587 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:05.587 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419488+0000", 2024-09-14T22:38:05.587 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8460000000000002e-06 2024-09-14T22:38:05.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:05.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419505+0000", 2024-09-14T22:38:05.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6477000000000002e-05 2024-09-14T22:38:05.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:05.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419518+0000", 2024-09-14T22:38:05.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2797e-05 2024-09-14T22:38:05.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:05.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479438+0000", 2024-09-14T22:38:05.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059919897 2024-09-14T22:38:05.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:05.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479547+0000", 2024-09-14T22:38:05.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109807 2024-09-14T22:38:05.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:05.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479647+0000", 2024-09-14T22:38:05.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9698999999999995e-05 2024-09-14T22:38:05.589 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.589 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:05.589 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.590 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82542 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:05.590 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:38:05.590 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.125460278999995, 2024-09-14T22:38:05.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060367430999999999, 2024-09-14T22:38:05.590 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:05.590 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:05.590 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:05.590 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:05.590 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:05.590 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:05.590 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82542 2024-09-14T22:38:05.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.591 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:05.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.591 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:05.591 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:38:05.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.591 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:05.591 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:38:05.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.592 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:05.592 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419534+0000", 2024-09-14T22:38:05.592 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T22:38:05.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.592 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:05.592 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419542+0000", 2024-09-14T22:38:05.592 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8199999999999997e-06 2024-09-14T22:38:05.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.592 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:05.592 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419559+0000", 2024-09-14T22:38:05.592 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6852999999999999e-05 2024-09-14T22:38:05.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.593 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:05.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419571+0000", 2024-09-14T22:38:05.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2364000000000001e-05 2024-09-14T22:38:05.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.593 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:05.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479686+0000", 2024-09-14T22:38:05.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060115522999999997 2024-09-14T22:38:05.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.593 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:05.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479795+0000", 2024-09-14T22:38:05.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010831399999999999 2024-09-14T22:38:05.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:05.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479900+0000", 2024-09-14T22:38:05.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105284 2024-09-14T22:38:05.594 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.594 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:05.594 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.595 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82544 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:05.595 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:38:05.595 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.125406542999997, 2024-09-14T22:38:05.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060590515999999997, 2024-09-14T22:38:05.595 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:05.595 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:05.595 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:05.595 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:05.595 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:05.595 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:05.595 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82544 2024-09-14T22:38:05.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.595 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:05.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.596 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:05.596 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:38:05.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.596 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.596 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.596 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:05.596 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:38:05.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.596 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.596 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.596 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:05.596 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419588+0000", 2024-09-14T22:38:05.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.35e-06 2024-09-14T22:38:05.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.597 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:05.597 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419595+0000", 2024-09-14T22:38:05.597 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4030000000000001e-06 2024-09-14T22:38:05.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.597 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:05.597 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419612+0000", 2024-09-14T22:38:05.597 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6670000000000001e-05 2024-09-14T22:38:05.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.597 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:05.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419624+0000", 2024-09-14T22:38:05.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-05 2024-09-14T22:38:05.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:05.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479957+0000", 2024-09-14T22:38:05.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060333287999999999 2024-09-14T22:38:05.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:05.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480074+0000", 2024-09-14T22:38:05.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116443 2024-09-14T22:38:05.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:05.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480177+0000", 2024-09-14T22:38:05.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000103102 2024-09-14T22:38:05.599 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.599 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:05.599 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.599 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82546 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:05.599 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:38:05.599 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.125353623999999, 2024-09-14T22:38:05.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060795428999999998, 2024-09-14T22:38:05.599 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:05.599 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:05.600 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:05.600 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:05.600 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:05.600 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:05.600 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82546 2024-09-14T22:38:05.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.600 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:05.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.600 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:05.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:38:05.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:05.601 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:38:05.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:05.601 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419640+0000", 2024-09-14T22:38:05.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.254e-06 2024-09-14T22:38:05.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:05.601 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419647+0000", 2024-09-14T22:38:05.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9e-06 2024-09-14T22:38:05.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.602 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:05.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419664+0000", 2024-09-14T22:38:05.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6345999999999999e-05 2024-09-14T22:38:05.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.602 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:05.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419676+0000", 2024-09-14T22:38:05.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.224e-05 2024-09-14T22:38:05.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:05.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480218+0000", 2024-09-14T22:38:05.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060542133999999997 2024-09-14T22:38:05.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:05.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480336+0000", 2024-09-14T22:38:05.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117472 2024-09-14T22:38:05.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:05.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480435+0000", 2024-09-14T22:38:05.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9083e-05 2024-09-14T22:38:05.603 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.604 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:05.604 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.604 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.604 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82548 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:05.604 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:38:05.604 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.125301613999994, 2024-09-14T22:38:05.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061011636000000001, 2024-09-14T22:38:05.604 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:05.604 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:05.604 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:05.604 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:05.604 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:05.605 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:05.605 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82548 2024-09-14T22:38:05.605 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.605 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:05.605 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.605 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:05.605 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:38:05.605 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.605 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.605 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.605 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:05.605 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:38:05.605 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.606 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:05.606 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419693+0000", 2024-09-14T22:38:05.606 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.387e-06 2024-09-14T22:38:05.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.606 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:05.606 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419699+0000", 2024-09-14T22:38:05.606 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8730000000000001e-06 2024-09-14T22:38:05.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.606 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:05.606 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419716+0000", 2024-09-14T22:38:05.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6416e-05 2024-09-14T22:38:05.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:05.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419728+0000", 2024-09-14T22:38:05.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2597e-05 2024-09-14T22:38:05.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:05.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480476+0000", 2024-09-14T22:38:05.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060747989000000002 2024-09-14T22:38:05.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:05.608 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480583+0000", 2024-09-14T22:38:05.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010638500000000001 2024-09-14T22:38:05.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:05.608 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480703+0000", 2024-09-14T22:38:05.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011998900000000001 2024-09-14T22:38:05.608 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.608 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:05.608 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.609 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82550 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1982464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:05.609 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:38:05.609 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.125248861000003, 2024-09-14T22:38:05.609 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061295483999999997, 2024-09-14T22:38:05.609 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:05.609 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:05.609 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:05.609 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:05.609 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:05.609 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:05.609 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82550 2024-09-14T22:38:05.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.610 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:05.610 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.610 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:05.610 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:38:05.610 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.610 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.610 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.610 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:05.610 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:38:05.610 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.610 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.610 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.611 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:05.611 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419745+0000", 2024-09-14T22:38:05.611 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.406e-06 2024-09-14T22:38:05.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.611 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:05.611 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419753+0000", 2024-09-14T22:38:05.611 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8140000000000001e-06 2024-09-14T22:38:05.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.611 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:05.611 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419770+0000", 2024-09-14T22:38:05.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6719999999999999e-05 2024-09-14T22:38:05.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:05.612 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419782+0000", 2024-09-14T22:38:05.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2252999999999999e-05 2024-09-14T22:38:05.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:05.612 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480745+0000", 2024-09-14T22:38:05.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060962385000000001 2024-09-14T22:38:05.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.613 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:05.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480900+0000", 2024-09-14T22:38:05.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015513899999999999 2024-09-14T22:38:05.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.613 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:05.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481039+0000", 2024-09-14T22:38:05.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013976700000000001 2024-09-14T22:38:05.613 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.613 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:05.613 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.614 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.614 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82588 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1064960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:05.614 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:38:05.614 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.125194970999999, 2024-09-14T22:38:05.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061863732999999997, 2024-09-14T22:38:05.614 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:05.614 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:05.614 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:05.614 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:05.614 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:05.614 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:05.614 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82588 2024-09-14T22:38:05.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.615 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:05.615 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.615 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:05.615 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:38:05.615 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.615 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.615 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:05.615 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:38:05.615 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.615 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.616 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:05.616 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419799+0000", 2024-09-14T22:38:05.616 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4160000000000001e-06 2024-09-14T22:38:05.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.616 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.616 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:05.616 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419807+0000", 2024-09-14T22:38:05.616 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2169999999999994e-06 2024-09-14T22:38:05.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.616 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.616 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:05.616 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419824+0000", 2024-09-14T22:38:05.616 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6776e-05 2024-09-14T22:38:05.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.617 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:05.617 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419837+0000", 2024-09-14T22:38:05.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2680000000000001e-05 2024-09-14T22:38:05.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.617 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:05.617 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481395+0000", 2024-09-14T22:38:05.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061558416999999997 2024-09-14T22:38:05.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.617 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:05.617 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481532+0000", 2024-09-14T22:38:05.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000136861 2024-09-14T22:38:05.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.618 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:05.618 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481662+0000", 2024-09-14T22:38:05.618 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012936600000000001 2024-09-14T22:38:05.618 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.618 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:05.618 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.618 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82589 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:05.618 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:38:05.618 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.125140376000004, 2024-09-14T22:38:05.618 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062064592000000002, 2024-09-14T22:38:05.619 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:05.619 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:05.619 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:05.619 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:05.619 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:05.619 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:05.619 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82589 2024-09-14T22:38:05.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.619 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:05.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:05.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:38:05.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:05.620 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:38:05.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:05.620 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419854+0000", 2024-09-14T22:38:05.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.367e-06 2024-09-14T22:38:05.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:05.620 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419862+0000", 2024-09-14T22:38:05.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8930000000000005e-06 2024-09-14T22:38:05.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.621 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:05.621 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419896+0000", 2024-09-14T22:38:05.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4134000000000002e-05 2024-09-14T22:38:05.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.621 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:05.621 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419909+0000", 2024-09-14T22:38:05.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3332999999999999e-05 2024-09-14T22:38:05.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.621 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:05.621 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481702+0000", 2024-09-14T22:38:05.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061792947000000001 2024-09-14T22:38:05.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:05.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481808+0000", 2024-09-14T22:38:05.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105655 2024-09-14T22:38:05.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:05.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481917+0000", 2024-09-14T22:38:05.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109263 2024-09-14T22:38:05.622 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.622 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:05.622 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.623 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82590 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1359872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:05.623 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:38:05.623 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.125066769, 2024-09-14T22:38:05.623 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062362895000000002, 2024-09-14T22:38:05.623 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:05.623 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:05.623 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:05.623 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:05.623 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:05.623 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:05.623 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82590 2024-09-14T22:38:05.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.624 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:05.624 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.624 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:05.624 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:38:05.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.624 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.624 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:05.624 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:38:05.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.624 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.624 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:05.624 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419928+0000", 2024-09-14T22:38:05.625 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4470000000000001e-06 2024-09-14T22:38:05.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.625 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.625 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:05.625 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419942+0000", 2024-09-14T22:38:05.625 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4229e-05 2024-09-14T22:38:05.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.625 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.625 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:05.625 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419972+0000", 2024-09-14T22:38:05.625 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.065e-05 2024-09-14T22:38:05.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.625 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:05.626 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419985+0000", 2024-09-14T22:38:05.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2903e-05 2024-09-14T22:38:05.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:05.626 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481960+0000", 2024-09-14T22:38:05.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061974435000000001 2024-09-14T22:38:05.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:05.626 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482106+0000", 2024-09-14T22:38:05.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014643800000000001 2024-09-14T22:38:05.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.627 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:05.627 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482289+0000", 2024-09-14T22:38:05.627 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000182793 2024-09-14T22:38:05.627 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.627 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:05.627 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.627 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82592 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:05.627 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:38:05.627 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.124990546999996, 2024-09-14T22:38:05.627 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062631507000000003, 2024-09-14T22:38:05.627 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:05.627 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:05.628 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:05.628 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:05.628 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:05.628 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:05.628 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82592 2024-09-14T22:38:05.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.628 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:05.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:05.628 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:38:05.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:05.629 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:38:05.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.629 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.629 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:05.629 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420017+0000", 2024-09-14T22:38:05.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4744e-05 2024-09-14T22:38:05.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.629 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.629 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:05.629 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420036+0000", 2024-09-14T22:38:05.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8680000000000001e-05 2024-09-14T22:38:05.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.630 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.630 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:05.630 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420054+0000", 2024-09-14T22:38:05.630 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7799999999999999e-05 2024-09-14T22:38:05.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.630 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.630 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:05.630 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420067+0000", 2024-09-14T22:38:05.630 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3013000000000001e-05 2024-09-14T22:38:05.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.630 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.630 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:05.630 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482358+0000", 2024-09-14T22:38:05.630 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062291445000000001 2024-09-14T22:38:05.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:05.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482469+0000", 2024-09-14T22:38:05.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110608 2024-09-14T22:38:05.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:05.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482634+0000", 2024-09-14T22:38:05.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000165217 2024-09-14T22:38:05.631 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.631 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:05.631 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.632 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82594 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1654784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:05.632 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:38:05.632 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.124892571000004, 2024-09-14T22:38:05.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062893849000000002, 2024-09-14T22:38:05.632 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:05.632 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:05.632 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:05.632 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:05.632 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:05.632 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:05.632 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82594 2024-09-14T22:38:05.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.632 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:05.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.633 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:05.633 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:38:05.633 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.633 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:05.633 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:38:05.633 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.633 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:05.633 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420102+0000", 2024-09-14T22:38:05.634 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5e-06 2024-09-14T22:38:05.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.634 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.634 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:05.634 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420115+0000", 2024-09-14T22:38:05.634 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.324e-05 2024-09-14T22:38:05.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.634 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.634 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:05.634 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420133+0000", 2024-09-14T22:38:05.634 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8213e-05 2024-09-14T22:38:05.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.634 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.634 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:05.634 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420147+0000", 2024-09-14T22:38:05.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3284e-05 2024-09-14T22:38:05.635 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.635 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.635 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:05.635 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482688+0000", 2024-09-14T22:38:05.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062541425999999997 2024-09-14T22:38:05.635 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.635 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.635 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:05.635 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482794+0000", 2024-09-14T22:38:05.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010575500000000001 2024-09-14T22:38:05.635 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.635 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.635 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:05.636 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482994+0000", 2024-09-14T22:38:05.636 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020043099999999999 2024-09-14T22:38:05.636 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.636 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:05.636 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.636 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.636 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82596 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:05.636 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:38:05.636 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.124829270999996, 2024-09-14T22:38:05.636 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063187238000000007, 2024-09-14T22:38:05.636 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:05.636 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:05.636 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:05.637 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:05.637 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:05.637 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:05.637 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82596 2024-09-14T22:38:05.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.637 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:05.637 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.637 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:05.637 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:38:05.637 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.637 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.637 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:05.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:38:05.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:05.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420165+0000", 2024-09-14T22:38:05.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4100000000000001e-06 2024-09-14T22:38:05.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:05.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420173+0000", 2024-09-14T22:38:05.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8129999999999996e-06 2024-09-14T22:38:05.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:05.639 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420190+0000", 2024-09-14T22:38:05.639 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7076999999999999e-05 2024-09-14T22:38:05.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.639 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.639 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:05.639 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420203+0000", 2024-09-14T22:38:05.639 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2683e-05 2024-09-14T22:38:05.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.639 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.639 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:05.639 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483060+0000", 2024-09-14T22:38:05.639 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062857114000000006 2024-09-14T22:38:05.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.640 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.640 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:05.640 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483167+0000", 2024-09-14T22:38:05.640 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010713999999999999 2024-09-14T22:38:05.640 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.640 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.640 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:05.640 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483351+0000", 2024-09-14T22:38:05.640 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018400099999999999 2024-09-14T22:38:05.640 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.640 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:05.640 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.640 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.640 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.640 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82597 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1949696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:05.641 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:38:05.641 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.124772445000005, 2024-09-14T22:38:05.641 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063466513000000002, 2024-09-14T22:38:05.641 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:05.641 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:05.641 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:05.641 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:05.641 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:05.641 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:05.641 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82597 2024-09-14T22:38:05.641 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.641 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:05.641 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.641 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:05.641 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:38:05.642 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.642 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.642 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:05.642 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:38:05.642 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:05.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.642 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.642 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:05.642 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420221+0000", 2024-09-14T22:38:05.642 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.043e-06 2024-09-14T22:38:05.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.642 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.643 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:05.643 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420229+0000", 2024-09-14T22:38:05.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0469999999999994e-06 2024-09-14T22:38:05.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.643 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:05.643 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420246+0000", 2024-09-14T22:38:05.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6339999999999999e-05 2024-09-14T22:38:05.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.643 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:05.643 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420258+0000", 2024-09-14T22:38:05.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2296000000000001e-05 2024-09-14T22:38:05.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.644 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.644 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:05.644 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483402+0000", 2024-09-14T22:38:05.644 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063144318000000005 2024-09-14T22:38:05.644 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.644 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.644 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:05.644 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483530+0000", 2024-09-14T22:38:05.644 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012713199999999999 2024-09-14T22:38:05.644 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:05.644 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:05.644 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:05.644 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483687+0000", 2024-09-14T22:38:05.644 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015733699999999999 2024-09-14T22:38:05.644 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.645 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:05.645 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.645 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:05.645 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:05.645 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:05.646 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:38:05.647 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:05.642+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:05.715 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.4 to 1 (10000) 2024-09-14T22:38:05.715 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2024-09-14T22:38:05.747 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:05.742+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:05.753 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-14T22:38:05.820 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:05.820 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:05.820 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:38:05.820 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:05.834 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:38:05.875 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:05.870+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:05.950 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:05.942+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:05.995 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:05.995 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:05.995 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:38:05.996 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:38:05.996 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:06.010 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:38:06.051 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:06.046+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:06.152 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:06.146+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:06.172 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:06.172 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:38:06.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:38:06.173 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:38:06.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.173 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:38:06.173 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:38:06.173 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 126.163365282, 2024-09-14T22:38:06.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.398029152, 2024-09-14T22:38:06.173 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:06.173 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:06.173 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:06.173 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:06.173 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:38:06.173 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:38:06.174 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 1 2024-09-14T22:38:06.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.174 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:06.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:06.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:38:06.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:06.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:38:06.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:06.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006183+0000", 2024-09-14T22:38:06.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9059999999999998e-06 2024-09-14T22:38:06.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:06.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006198+0000", 2024-09-14T22:38:06.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.523e-05 2024-09-14T22:38:06.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:06.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006219+0000", 2024-09-14T22:38:06.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1342999999999999e-05 2024-09-14T22:38:06.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006240+0000", 2024-09-14T22:38:06.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1186999999999999e-05 2024-09-14T22:38:06.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:06.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006363+0000", 2024-09-14T22:38:06.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012258200000000001 2024-09-14T22:38:06.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:38:06.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006392+0000", 2024-09-14T22:38:06.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9607e-05 2024-09-14T22:38:06.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:06.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401084+0000", 2024-09-14T22:38:06.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.39469142699999998 2024-09-14T22:38:06.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:06.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401394+0000", 2024-09-14T22:38:06.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030987799999999998 2024-09-14T22:38:06.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:38:06.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401860+0000", 2024-09-14T22:38:06.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00046654299999999998 2024-09-14T22:38:06.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:38:06.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.403386+0000", 2024-09-14T22:38:06.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001525322 2024-09-14T22:38:06.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:06.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404042+0000", 2024-09-14T22:38:06.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065676099999999996 2024-09-14T22:38:06.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:38:06.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404136+0000", 2024-09-14T22:38:06.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4032999999999999e-05 2024-09-14T22:38:06.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:06.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404209+0000", 2024-09-14T22:38:06.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2333e-05 2024-09-14T22:38:06.179 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.180 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:06.180 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.180 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104182 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14352384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:06.180 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:38:06.180 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.680511295999999, 2024-09-14T22:38:06.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071642496999999999, 2024-09-14T22:38:06.180 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:06.180 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:06.180 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:06.180 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:06.180 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:06.181 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.181 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104182 2024-09-14T22:38:06.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.181 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:06.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:06.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:38:06.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:06.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:38:06.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:06.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489035+0000", 2024-09-14T22:38:06.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3430000000000001e-06 2024-09-14T22:38:06.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:06.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489041+0000", 2024-09-14T22:38:06.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8139999999999997e-06 2024-09-14T22:38:06.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:06.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489059+0000", 2024-09-14T22:38:06.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8193e-05 2024-09-14T22:38:06.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489073+0000", 2024-09-14T22:38:06.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:38:06.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:06.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560434+0000", 2024-09-14T22:38:06.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071361463999999999 2024-09-14T22:38:06.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:06.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560545+0000", 2024-09-14T22:38:06.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110263 2024-09-14T22:38:06.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:06.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560676+0000", 2024-09-14T22:38:06.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013138000000000001 2024-09-14T22:38:06.184 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.184 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:06.184 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.184 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104183 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14483456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:06.185 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:38:06.185 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.680387977000002, 2024-09-14T22:38:06.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071781943000000001, 2024-09-14T22:38:06.185 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:06.185 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:06.185 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:06.185 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:06.185 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:06.185 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.185 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104183 2024-09-14T22:38:06.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.185 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:06.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:06.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:38:06.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:06.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:38:06.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:06.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489159+0000", 2024-09-14T22:38:06.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5629999999999999e-06 2024-09-14T22:38:06.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:06.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489165+0000", 2024-09-14T22:38:06.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7070000000000003e-06 2024-09-14T22:38:06.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:06.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489186+0000", 2024-09-14T22:38:06.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.088e-05 2024-09-14T22:38:06.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489201+0000", 2024-09-14T22:38:06.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5203e-05 2024-09-14T22:38:06.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:06.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560713+0000", 2024-09-14T22:38:06.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071511447000000006 2024-09-14T22:38:06.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:06.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560819+0000", 2024-09-14T22:38:06.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106579 2024-09-14T22:38:06.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:06.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560939+0000", 2024-09-14T22:38:06.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119564 2024-09-14T22:38:06.189 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.189 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:06.189 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.189 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104184 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:06.189 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:38:06.189 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.680323534000003, 2024-09-14T22:38:06.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071970107000000005, 2024-09-14T22:38:06.189 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:06.189 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:06.189 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:06.190 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:06.190 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:06.190 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.190 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104184 2024-09-14T22:38:06.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.190 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:06.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:06.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:38:06.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:06.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:38:06.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:06.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489223+0000", 2024-09-14T22:38:06.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4470000000000001e-06 2024-09-14T22:38:06.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:06.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489229+0000", 2024-09-14T22:38:06.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8200000000000002e-06 2024-09-14T22:38:06.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:06.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489247+0000", 2024-09-14T22:38:06.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8150000000000001e-05 2024-09-14T22:38:06.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489260+0000", 2024-09-14T22:38:06.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3570000000000001e-05 2024-09-14T22:38:06.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:06.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560979+0000", 2024-09-14T22:38:06.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071718831999999996 2024-09-14T22:38:06.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:06.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561075+0000", 2024-09-14T22:38:06.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5848999999999997e-05 2024-09-14T22:38:06.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:06.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561191+0000", 2024-09-14T22:38:06.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116439 2024-09-14T22:38:06.193 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.193 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:06.194 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.194 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104185 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14745600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:06.194 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:38:06.194 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.680263951000001, 2024-09-14T22:38:06.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072173750999999994, 2024-09-14T22:38:06.194 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:06.194 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:06.194 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:06.194 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:06.194 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:06.194 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.194 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104185 2024-09-14T22:38:06.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.195 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:06.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:06.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:38:06.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:06.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:38:06.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:06.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489282+0000", 2024-09-14T22:38:06.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2500000000000001e-06 2024-09-14T22:38:06.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:06.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489287+0000", 2024-09-14T22:38:06.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1359999999999996e-06 2024-09-14T22:38:06.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:06.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489304+0000", 2024-09-14T22:38:06.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7033000000000001e-05 2024-09-14T22:38:06.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489318+0000", 2024-09-14T22:38:06.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3934000000000001e-05 2024-09-14T22:38:06.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:06.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561232+0000", 2024-09-14T22:38:06.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071913759999999993 2024-09-14T22:38:06.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:06.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561340+0000", 2024-09-14T22:38:06.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107921 2024-09-14T22:38:06.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:06.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561455+0000", 2024-09-14T22:38:06.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011471700000000001 2024-09-14T22:38:06.198 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.198 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:06.198 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.198 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104186 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14876672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:06.198 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:38:06.198 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.680208041999997, 2024-09-14T22:38:06.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072393374999999996, 2024-09-14T22:38:06.198 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:06.199 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:06.199 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:06.199 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:06.199 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:06.199 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.199 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104186 2024-09-14T22:38:06.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.199 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:06.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:06.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:38:06.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:06.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:38:06.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:06.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489338+0000", 2024-09-14T22:38:06.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.48e-06 2024-09-14T22:38:06.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:06.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489366+0000", 2024-09-14T22:38:06.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7307000000000001e-05 2024-09-14T22:38:06.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:06.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489385+0000", 2024-09-14T22:38:06.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9283000000000002e-05 2024-09-14T22:38:06.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489399+0000", 2024-09-14T22:38:06.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.452e-05 2024-09-14T22:38:06.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:06.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561494+0000", 2024-09-14T22:38:06.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072094885999999997 2024-09-14T22:38:06.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:06.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561604+0000", 2024-09-14T22:38:06.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010925900000000001 2024-09-14T22:38:06.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:06.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561730+0000", 2024-09-14T22:38:06.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012663999999999999 2024-09-14T22:38:06.202 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.202 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:06.202 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.203 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104187 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15007744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:06.203 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:38:06.203 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.680124673000002, 2024-09-14T22:38:06.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072599143000000005, 2024-09-14T22:38:06.203 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:06.203 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:06.203 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:06.203 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:06.203 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:06.203 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.203 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104187 2024-09-14T22:38:06.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.204 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:06.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:06.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:38:06.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:06.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:38:06.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:06.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489422+0000", 2024-09-14T22:38:06.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5340000000000001e-06 2024-09-14T22:38:06.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:06.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489428+0000", 2024-09-14T22:38:06.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1430000000000003e-06 2024-09-14T22:38:06.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:06.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489445+0000", 2024-09-14T22:38:06.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7042999999999999e-05 2024-09-14T22:38:06.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489458+0000", 2024-09-14T22:38:06.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3474e-05 2024-09-14T22:38:06.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:06.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561771+0000", 2024-09-14T22:38:06.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072312600000000005 2024-09-14T22:38:06.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:06.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561885+0000", 2024-09-14T22:38:06.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011411799999999999 2024-09-14T22:38:06.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:06.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562019+0000", 2024-09-14T22:38:06.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013423100000000001 2024-09-14T22:38:06.207 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.207 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:06.207 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.207 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104188 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15138816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:06.207 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:38:06.207 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.680064749000003, 2024-09-14T22:38:06.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072824624000000004, 2024-09-14T22:38:06.208 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:06.208 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:06.208 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:06.208 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:06.208 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:06.208 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.208 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104188 2024-09-14T22:38:06.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.208 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:06.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:06.208 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:38:06.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:06.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:38:06.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:06.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489481+0000", 2024-09-14T22:38:06.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.22e-06 2024-09-14T22:38:06.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:06.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489487+0000", 2024-09-14T22:38:06.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6930000000000004e-06 2024-09-14T22:38:06.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:06.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489508+0000", 2024-09-14T22:38:06.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1135999999999999e-05 2024-09-14T22:38:06.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489522+0000", 2024-09-14T22:38:06.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3817e-05 2024-09-14T22:38:06.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:06.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562066+0000", 2024-09-14T22:38:06.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072543813999999998 2024-09-14T22:38:06.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:06.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562177+0000", 2024-09-14T22:38:06.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000111264 2024-09-14T22:38:06.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:06.212 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562305+0000", 2024-09-14T22:38:06.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012768 2024-09-14T22:38:06.212 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.212 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:06.212 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.212 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104189 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15269888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:06.212 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:38:06.212 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.6800043, 2024-09-14T22:38:06.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073061300999999995, 2024-09-14T22:38:06.212 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:06.212 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:06.213 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:06.213 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:06.213 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:06.213 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.213 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104189 2024-09-14T22:38:06.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.213 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:06.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:06.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:38:06.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:06.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:38:06.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:06.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489542+0000", 2024-09-14T22:38:06.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1659999999999999e-06 2024-09-14T22:38:06.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:06.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489561+0000", 2024-09-14T22:38:06.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.908e-05 2024-09-14T22:38:06.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:06.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489579+0000", 2024-09-14T22:38:06.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8536999999999999e-05 2024-09-14T22:38:06.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489593+0000", 2024-09-14T22:38:06.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3977e-05 2024-09-14T22:38:06.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:06.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562347+0000", 2024-09-14T22:38:06.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072754092000000006 2024-09-14T22:38:06.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:06.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562458+0000", 2024-09-14T22:38:06.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110722 2024-09-14T22:38:06.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:06.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562602+0000", 2024-09-14T22:38:06.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014372699999999999 2024-09-14T22:38:06.216 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.216 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:06.216 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.217 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104190 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:06.217 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:38:06.217 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.679932004000001, 2024-09-14T22:38:06.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.07326133, 2024-09-14T22:38:06.217 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:06.217 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:06.217 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:06.217 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:06.217 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:06.217 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.217 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104190 2024-09-14T22:38:06.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.218 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:06.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:06.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:38:06.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:06.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:38:06.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:06.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489614+0000", 2024-09-14T22:38:06.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1999999999999999e-06 2024-09-14T22:38:06.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:06.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489620+0000", 2024-09-14T22:38:06.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.524e-06 2024-09-14T22:38:06.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:06.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489637+0000", 2024-09-14T22:38:06.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7825999999999998e-05 2024-09-14T22:38:06.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489671+0000", 2024-09-14T22:38:06.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3726000000000002e-05 2024-09-14T22:38:06.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:06.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562644+0000", 2024-09-14T22:38:06.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072973089000000005 2024-09-14T22:38:06.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:06.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562762+0000", 2024-09-14T22:38:06.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011743600000000001 2024-09-14T22:38:06.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:06.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562874+0000", 2024-09-14T22:38:06.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011252900000000001 2024-09-14T22:38:06.221 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.221 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:06.221 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.221 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104191 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15532032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:06.222 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:38:06.222 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.679823581000001, 2024-09-14T22:38:06.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073311166999999997, 2024-09-14T22:38:06.222 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:06.222 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:06.222 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:06.222 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:06.222 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:06.222 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.222 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104191 2024-09-14T22:38:06.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.222 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:06.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:06.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:38:06.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:06.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:38:06.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:06.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489723+0000", 2024-09-14T22:38:06.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6330000000000001e-06 2024-09-14T22:38:06.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:06.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489730+0000", 2024-09-14T22:38:06.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2300000000000002e-06 2024-09-14T22:38:06.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:06.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489751+0000", 2024-09-14T22:38:06.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0849999999999999e-05 2024-09-14T22:38:06.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489766+0000", 2024-09-14T22:38:06.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.524e-05 2024-09-14T22:38:06.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:06.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562896+0000", 2024-09-14T22:38:06.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073129662999999998 2024-09-14T22:38:06.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:06.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562960+0000", 2024-09-14T22:38:06.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3689999999999995e-05 2024-09-14T22:38:06.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:06.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563033+0000", 2024-09-14T22:38:06.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2861000000000006e-05 2024-09-14T22:38:06.226 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.226 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:06.226 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.226 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104192 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:06.226 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:38:06.226 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.679727776, 2024-09-14T22:38:06.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073463632000000001, 2024-09-14T22:38:06.227 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:06.227 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:06.227 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:06.227 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:06.227 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:06.227 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.227 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104192 2024-09-14T22:38:06.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.227 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:06.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:06.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:38:06.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:06.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:38:06.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:06.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489819+0000", 2024-09-14T22:38:06.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.424e-06 2024-09-14T22:38:06.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:06.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489825+0000", 2024-09-14T22:38:06.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.02e-06 2024-09-14T22:38:06.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:06.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489844+0000", 2024-09-14T22:38:06.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9760000000000001e-05 2024-09-14T22:38:06.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489860+0000", 2024-09-14T22:38:06.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5563e-05 2024-09-14T22:38:06.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:06.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563070+0000", 2024-09-14T22:38:06.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073209724000000004 2024-09-14T22:38:06.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:06.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563175+0000", 2024-09-14T22:38:06.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105183 2024-09-14T22:38:06.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:06.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563281+0000", 2024-09-14T22:38:06.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105958 2024-09-14T22:38:06.231 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.231 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:06.231 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.232 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104193 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15794176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:06.232 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:38:06.232 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.679665608999997, 2024-09-14T22:38:06.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073600040000000005, 2024-09-14T22:38:06.232 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:06.232 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:06.232 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:06.232 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:06.232 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:06.232 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.232 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104193 2024-09-14T22:38:06.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.233 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:06.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:06.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:38:06.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:06.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:38:06.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:06.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489881+0000", 2024-09-14T22:38:06.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5460000000000001e-06 2024-09-14T22:38:06.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:06.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489886+0000", 2024-09-14T22:38:06.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1739999999999999e-06 2024-09-14T22:38:06.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:06.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489912+0000", 2024-09-14T22:38:06.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6461999999999998e-05 2024-09-14T22:38:06.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489927+0000", 2024-09-14T22:38:06.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:38:06.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:06.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563310+0000", 2024-09-14T22:38:06.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073382977000000002 2024-09-14T22:38:06.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:06.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563385+0000", 2024-09-14T22:38:06.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5218999999999999e-05 2024-09-14T22:38:06.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:06.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563479+0000", 2024-09-14T22:38:06.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4622000000000003e-05 2024-09-14T22:38:06.236 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.236 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:06.236 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.237 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104194 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15925248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:06.237 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:38:06.237 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.679599017000001, 2024-09-14T22:38:06.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073745434999999998, 2024-09-14T22:38:06.237 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:06.237 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:06.237 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:06.237 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:06.237 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:06.237 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.237 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104194 2024-09-14T22:38:06.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.237 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:06.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:06.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:38:06.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:06.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:38:06.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:06.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489947+0000", 2024-09-14T22:38:06.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.327e-06 2024-09-14T22:38:06.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:06.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489953+0000", 2024-09-14T22:38:06.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9329999999999998e-06 2024-09-14T22:38:06.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:06.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489971+0000", 2024-09-14T22:38:06.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7994000000000001e-05 2024-09-14T22:38:06.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489985+0000", 2024-09-14T22:38:06.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4226000000000001e-05 2024-09-14T22:38:06.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:06.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563510+0000", 2024-09-14T22:38:06.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073524667000000002 2024-09-14T22:38:06.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:06.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563597+0000", 2024-09-14T22:38:06.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6550000000000003e-05 2024-09-14T22:38:06.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:06.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563691+0000", 2024-09-14T22:38:06.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4738e-05 2024-09-14T22:38:06.241 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.241 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:06.241 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.241 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104195 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:06.241 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:38:06.241 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.67954082, 2024-09-14T22:38:06.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073876598000000002, 2024-09-14T22:38:06.242 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:06.242 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:06.242 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:06.242 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:06.242 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:06.242 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.242 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104195 2024-09-14T22:38:06.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.242 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:06.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:06.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:38:06.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:06.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:38:06.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:06.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490006+0000", 2024-09-14T22:38:06.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4699999999999999e-06 2024-09-14T22:38:06.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:06.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490011+0000", 2024-09-14T22:38:06.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.366e-06 2024-09-14T22:38:06.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:06.244 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490046+0000", 2024-09-14T22:38:06.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5496e-05 2024-09-14T22:38:06.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.244 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490061+0000", 2024-09-14T22:38:06.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5054e-05 2024-09-14T22:38:06.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:06.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563719+0000", 2024-09-14T22:38:06.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073657412000000005 2024-09-14T22:38:06.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:06.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563792+0000", 2024-09-14T22:38:06.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3319999999999996e-05 2024-09-14T22:38:06.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:06.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563881+0000", 2024-09-14T22:38:06.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8480000000000007e-05 2024-09-14T22:38:06.246 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.246 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:06.246 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.246 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104196 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16187392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:06.246 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:38:06.246 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.679463671000001, 2024-09-14T22:38:06.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073997496999999995, 2024-09-14T22:38:06.246 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:06.246 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:06.246 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:06.246 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:06.247 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:06.247 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.247 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104196 2024-09-14T22:38:06.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.247 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:06.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:06.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:38:06.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:06.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:38:06.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.248 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:06.248 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490083+0000", 2024-09-14T22:38:06.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.347e-06 2024-09-14T22:38:06.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.248 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:06.248 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490089+0000", 2024-09-14T22:38:06.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9630000000000003e-06 2024-09-14T22:38:06.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.248 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:06.249 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490107+0000", 2024-09-14T22:38:06.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8859999999999999e-05 2024-09-14T22:38:06.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.249 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.249 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490122+0000", 2024-09-14T22:38:06.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.432e-05 2024-09-14T22:38:06.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.249 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:06.249 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563910+0000", 2024-09-14T22:38:06.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073788753999999998 2024-09-14T22:38:06.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:06.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563988+0000", 2024-09-14T22:38:06.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7862999999999996e-05 2024-09-14T22:38:06.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:06.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564079+0000", 2024-09-14T22:38:06.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0389999999999993e-05 2024-09-14T22:38:06.250 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.250 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:06.250 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.251 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104197 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16318464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:06.251 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:38:06.251 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.679372041999997, 2024-09-14T22:38:06.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074103547000000006, 2024-09-14T22:38:06.251 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:06.251 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:06.251 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:06.251 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:06.251 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:06.251 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.251 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104197 2024-09-14T22:38:06.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.251 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:06.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:06.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:38:06.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:06.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:38:06.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:06.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490174+0000", 2024-09-14T22:38:06.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.37e-06 2024-09-14T22:38:06.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:06.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490197+0000", 2024-09-14T22:38:06.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3017e-05 2024-09-14T22:38:06.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:06.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490219+0000", 2024-09-14T22:38:06.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.181e-05 2024-09-14T22:38:06.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490240+0000", 2024-09-14T22:38:06.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0786000000000001e-05 2024-09-14T22:38:06.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.254 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:06.254 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564109+0000", 2024-09-14T22:38:06.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073869031000000002 2024-09-14T22:38:06.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.254 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:06.254 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564186+0000", 2024-09-14T22:38:06.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6823999999999995e-05 2024-09-14T22:38:06.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.254 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:06.254 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564276+0000", 2024-09-14T22:38:06.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0709000000000002e-05 2024-09-14T22:38:06.255 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.255 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:06.255 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.255 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104198 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16449536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:06.255 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:38:06.255 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.679279805999997, 2024-09-14T22:38:06.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074167693000000007, 2024-09-14T22:38:06.255 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:06.255 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:06.255 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:06.256 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:06.256 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:06.256 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.256 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104198 2024-09-14T22:38:06.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.256 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:06.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:06.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:38:06.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:06.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:38:06.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:06.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490266+0000", 2024-09-14T22:38:06.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3599999999999999e-06 2024-09-14T22:38:06.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:06.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490272+0000", 2024-09-14T22:38:06.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9599999999999997e-06 2024-09-14T22:38:06.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:06.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490291+0000", 2024-09-14T22:38:06.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8485999999999999e-05 2024-09-14T22:38:06.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490305+0000", 2024-09-14T22:38:06.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4327e-05 2024-09-14T22:38:06.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:06.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564306+0000", 2024-09-14T22:38:06.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074000745000000007 2024-09-14T22:38:06.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:06.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564367+0000", 2024-09-14T22:38:06.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1501999999999995e-05 2024-09-14T22:38:06.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:06.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564433+0000", 2024-09-14T22:38:06.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5313000000000003e-05 2024-09-14T22:38:06.259 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.259 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:06.259 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.260 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104199 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16580608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:06.260 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:38:06.260 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.679198253000003, 2024-09-14T22:38:06.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074231631000000006, 2024-09-14T22:38:06.260 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:06.260 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:06.260 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:06.260 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:06.260 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:06.260 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.260 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104199 2024-09-14T22:38:06.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.261 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:06.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:06.261 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:38:06.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:06.261 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:38:06.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:06.261 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490349+0000", 2024-09-14T22:38:06.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9429999999999999e-06 2024-09-14T22:38:06.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:06.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490355+0000", 2024-09-14T22:38:06.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1670000000000004e-06 2024-09-14T22:38:06.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:06.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490374+0000", 2024-09-14T22:38:06.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9613e-05 2024-09-14T22:38:06.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.263 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490405+0000", 2024-09-14T22:38:06.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0936000000000003e-05 2024-09-14T22:38:06.263 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.263 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.263 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:06.263 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564451+0000", 2024-09-14T22:38:06.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074045266999999998 2024-09-14T22:38:06.263 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.263 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.263 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:06.263 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564494+0000", 2024-09-14T22:38:06.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3133000000000002e-05 2024-09-14T22:38:06.263 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:06.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564578+0000", 2024-09-14T22:38:06.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4572000000000003e-05 2024-09-14T22:38:06.264 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.264 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:06.264 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.264 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104200 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:06.264 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:38:06.264 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.677403024999997, 2024-09-14T22:38:06.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072582435000000001, 2024-09-14T22:38:06.265 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:06.265 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:06.265 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:06.265 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:06.265 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:06.265 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.265 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104200 2024-09-14T22:38:06.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.265 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:06.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:06.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:38:06.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:06.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:38:06.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:06.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:06.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492144+0000", 2024-09-14T22:38:06.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3070000000000001e-06 2024-09-14T22:38:06.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:06.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492155+0000", 2024-09-14T22:38:06.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0573e-05 2024-09-14T22:38:06.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:06.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492185+0000", 2024-09-14T22:38:06.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.008e-05 2024-09-14T22:38:06.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.268 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.268 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492205+0000", 2024-09-14T22:38:06.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9882999999999999e-05 2024-09-14T22:38:06.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.268 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:06.268 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564609+0000", 2024-09-14T22:38:06.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072404181999999997 2024-09-14T22:38:06.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.268 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:06.268 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564667+0000", 2024-09-14T22:38:06.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8102e-05 2024-09-14T22:38:06.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:06.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:06.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:06.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564724+0000", 2024-09-14T22:38:06.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7308000000000001e-05 2024-09-14T22:38:06.269 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.269 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:06.269 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.269 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:06.269 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:06.269 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:06.271 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:38:06.271 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:06.266+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:06.372 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:06.366+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:06.399 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:06.399 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:06.399 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:38:06.399 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:06.413 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:38:06.473 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:06.466+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:06.538 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:06.538 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:06.538 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:38:06.538 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:38:06.538 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:06.551 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:38:06.574 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:06.566+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:06.665 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:06.665 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:38:06.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:38:06.665 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:38:06.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.665 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144987 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3424256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:06.665 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549297+0000", 2024-09-14T22:38:06.666 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.113243739, 2024-09-14T22:38:06.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.079931556000000001, 2024-09-14T22:38:06.666 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:06.666 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:06.666 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:06.666 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:06.666 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:06.666 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.666 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144987 2024-09-14T22:38:06.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.666 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:06.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:06.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549297+0000", 2024-09-14T22:38:06.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:06.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549297+0000", 2024-09-14T22:38:06.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:06.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549298+0000", 2024-09-14T22:38:06.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2699999999999999e-07 2024-09-14T22:38:06.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:06.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549301+0000", 2024-09-14T22:38:06.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2959999999999999e-06 2024-09-14T22:38:06.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:06.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549314+0000", 2024-09-14T22:38:06.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2588999999999999e-05 2024-09-14T22:38:06.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549323+0000", 2024-09-14T22:38:06.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.927e-06 2024-09-14T22:38:06.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:06.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629038+0000", 2024-09-14T22:38:06.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.079714220000000002 2024-09-14T22:38:06.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:06.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629116+0000", 2024-09-14T22:38:06.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7966000000000005e-05 2024-09-14T22:38:06.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:06.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629228+0000", 2024-09-14T22:38:06.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112831 2024-09-14T22:38:06.670 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.670 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:06.670 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.670 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144988 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3325952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:06.670 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549335+0000", 2024-09-14T22:38:06.670 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.113205514000001, 2024-09-14T22:38:06.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080179116999999994, 2024-09-14T22:38:06.670 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:06.670 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:06.671 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:06.671 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:06.671 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:06.671 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.671 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144988 2024-09-14T22:38:06.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.671 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:06.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:06.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549335+0000", 2024-09-14T22:38:06.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:06.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549335+0000", 2024-09-14T22:38:06.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:06.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549336+0000", 2024-09-14T22:38:06.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7999999999999995e-07 2024-09-14T22:38:06.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:06.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549339+0000", 2024-09-14T22:38:06.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.836e-06 2024-09-14T22:38:06.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:06.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549352+0000", 2024-09-14T22:38:06.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3069e-05 2024-09-14T22:38:06.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549359+0000", 2024-09-14T22:38:06.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.357e-06 2024-09-14T22:38:06.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:06.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629288+0000", 2024-09-14T22:38:06.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.079928492000000004 2024-09-14T22:38:06.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:06.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629412+0000", 2024-09-14T22:38:06.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012413699999999999 2024-09-14T22:38:06.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:06.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629514+0000", 2024-09-14T22:38:06.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000102546 2024-09-14T22:38:06.675 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.675 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:06.675 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.676 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144989 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3522560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:06.676 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:38:06.676 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.113171105999999, 2024-09-14T22:38:06.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080393122999999997, 2024-09-14T22:38:06.676 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:06.676 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:06.676 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:06.676 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:06.676 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:06.676 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.676 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144989 2024-09-14T22:38:06.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.677 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:06.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:06.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:38:06.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:06.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:38:06.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:06.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:38:06.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3600000000000003e-07 2024-09-14T22:38:06.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:06.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549373+0000", 2024-09-14T22:38:06.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8169999999999999e-06 2024-09-14T22:38:06.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:06.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549385+0000", 2024-09-14T22:38:06.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1803e-05 2024-09-14T22:38:06.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549394+0000", 2024-09-14T22:38:06.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4290000000000007e-06 2024-09-14T22:38:06.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:06.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629576+0000", 2024-09-14T22:38:06.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080181551000000004 2024-09-14T22:38:06.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:06.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629654+0000", 2024-09-14T22:38:06.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7633999999999996e-05 2024-09-14T22:38:06.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:06.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629763+0000", 2024-09-14T22:38:06.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109153 2024-09-14T22:38:06.680 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.680 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:06.680 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.681 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144990 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:06.681 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:38:06.681 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.113128081999999, 2024-09-14T22:38:06.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080614496999999993, 2024-09-14T22:38:06.681 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:06.681 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:06.681 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:06.681 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:06.681 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:06.681 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.681 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144990 2024-09-14T22:38:06.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.682 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:06.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:06.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:38:06.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:06.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:38:06.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:06.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:38:06.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0999999999999997e-07 2024-09-14T22:38:06.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:06.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549419+0000", 2024-09-14T22:38:06.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5339999999999999e-06 2024-09-14T22:38:06.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:06.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549439+0000", 2024-09-14T22:38:06.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0035e-05 2024-09-14T22:38:06.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549449+0000", 2024-09-14T22:38:06.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0429e-05 2024-09-14T22:38:06.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:06.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629815+0000", 2024-09-14T22:38:06.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080365467999999995 2024-09-14T22:38:06.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:06.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629922+0000", 2024-09-14T22:38:06.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106914 2024-09-14T22:38:06.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:06.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630027+0000", 2024-09-14T22:38:06.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105207 2024-09-14T22:38:06.685 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.685 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:06.685 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.685 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144991 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 2998272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:06.685 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549463+0000", 2024-09-14T22:38:06.685 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.113077777999999, 2024-09-14T22:38:06.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080847637, 2024-09-14T22:38:06.686 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:06.686 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:06.686 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:06.686 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:06.686 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:06.686 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.686 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144991 2024-09-14T22:38:06.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.686 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:06.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:06.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549463+0000", 2024-09-14T22:38:06.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:06.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549463+0000", 2024-09-14T22:38:06.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:06.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549464+0000", 2024-09-14T22:38:06.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9999999999999997e-07 2024-09-14T22:38:06.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:06.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549467+0000", 2024-09-14T22:38:06.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0800000000000002e-06 2024-09-14T22:38:06.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.688 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:06.688 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549478+0000", 2024-09-14T22:38:06.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1813e-05 2024-09-14T22:38:06.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.688 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.688 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549488+0000", 2024-09-14T22:38:06.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7030000000000001e-06 2024-09-14T22:38:06.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:06.689 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630090+0000", 2024-09-14T22:38:06.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080601962999999999 2024-09-14T22:38:06.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:06.689 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630187+0000", 2024-09-14T22:38:06.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6552000000000007e-05 2024-09-14T22:38:06.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:06.689 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630311+0000", 2024-09-14T22:38:06.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000123826 2024-09-14T22:38:06.690 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.690 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:06.690 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.690 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144992 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3620864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:06.690 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:38:06.690 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.113041969999999, 2024-09-14T22:38:06.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081097875999999999, 2024-09-14T22:38:06.690 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:06.690 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:06.690 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:06.690 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:06.690 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:06.691 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.691 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144992 2024-09-14T22:38:06.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.691 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:06.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:06.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:38:06.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:06.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:38:06.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.692 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:06.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:38:06.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5400000000000001e-07 2024-09-14T22:38:06.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.692 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:06.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549502+0000", 2024-09-14T22:38:06.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7889999999999999e-06 2024-09-14T22:38:06.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.692 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:06.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549513+0000", 2024-09-14T22:38:06.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1204e-05 2024-09-14T22:38:06.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549523+0000", 2024-09-14T22:38:06.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5990000000000008e-06 2024-09-14T22:38:06.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:06.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630362+0000", 2024-09-14T22:38:06.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080839272000000004 2024-09-14T22:38:06.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:06.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630469+0000", 2024-09-14T22:38:06.694 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107285 2024-09-14T22:38:06.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.694 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:06.694 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630597+0000", 2024-09-14T22:38:06.694 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012707300000000001 2024-09-14T22:38:06.694 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.694 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:06.694 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.694 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144993 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:06.694 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:38:06.694 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.113008894, 2024-09-14T22:38:06.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081338411999999999, 2024-09-14T22:38:06.695 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:06.695 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:06.695 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:06.695 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:06.695 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:06.695 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.695 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144993 2024-09-14T22:38:06.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.695 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:06.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:06.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:38:06.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:06.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:38:06.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:06.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:38:06.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5300000000000004e-07 2024-09-14T22:38:06.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:06.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549544+0000", 2024-09-14T22:38:06.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1912e-05 2024-09-14T22:38:06.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:06.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549557+0000", 2024-09-14T22:38:06.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2413e-05 2024-09-14T22:38:06.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549567+0000", 2024-09-14T22:38:06.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8660000000000001e-06 2024-09-14T22:38:06.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:06.698 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630651+0000", 2024-09-14T22:38:06.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081084229999999993 2024-09-14T22:38:06.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.698 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:06.698 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630729+0000", 2024-09-14T22:38:06.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8265999999999999e-05 2024-09-14T22:38:06.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.698 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:06.698 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630870+0000", 2024-09-14T22:38:06.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014107200000000001 2024-09-14T22:38:06.698 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.698 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:06.698 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.699 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144994 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:06.699 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:38:06.699 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.112961927000001, 2024-09-14T22:38:06.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081547958000000004, 2024-09-14T22:38:06.699 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:06.699 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:06.699 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:06.699 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:06.699 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:06.699 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.699 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144994 2024-09-14T22:38:06.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.700 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:06.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:06.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:38:06.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:06.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:38:06.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:06.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:38:06.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6599999999999996e-07 2024-09-14T22:38:06.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:06.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549582+0000", 2024-09-14T22:38:06.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.004e-06 2024-09-14T22:38:06.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:06.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549594+0000", 2024-09-14T22:38:06.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1459e-05 2024-09-14T22:38:06.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549603+0000", 2024-09-14T22:38:06.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6129999999999993e-06 2024-09-14T22:38:06.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:06.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630932+0000", 2024-09-14T22:38:06.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081328558999999995 2024-09-14T22:38:06.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:06.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631027+0000", 2024-09-14T22:38:06.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4742999999999997e-05 2024-09-14T22:38:06.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:06.703 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631127+0000", 2024-09-14T22:38:06.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9914000000000006e-05 2024-09-14T22:38:06.703 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.703 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:06.703 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.703 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144995 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3391488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:06.703 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549615+0000", 2024-09-14T22:38:06.704 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.112925789, 2024-09-14T22:38:06.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081734251999999993, 2024-09-14T22:38:06.704 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:06.704 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:06.704 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:06.704 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:06.704 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:06.704 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.704 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144995 2024-09-14T22:38:06.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.704 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:06.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:06.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549615+0000", 2024-09-14T22:38:06.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:06.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549615+0000", 2024-09-14T22:38:06.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:06.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549616+0000", 2024-09-14T22:38:06.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9299999999999997e-07 2024-09-14T22:38:06.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:06.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549618+0000", 2024-09-14T22:38:06.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7269999999999999e-06 2024-09-14T22:38:06.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:06.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549630+0000", 2024-09-14T22:38:06.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1313e-05 2024-09-14T22:38:06.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549639+0000", 2024-09-14T22:38:06.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4159999999999993e-06 2024-09-14T22:38:06.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:06.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631183+0000", 2024-09-14T22:38:06.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081543633000000004 2024-09-14T22:38:06.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:06.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631252+0000", 2024-09-14T22:38:06.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9567999999999995e-05 2024-09-14T22:38:06.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:06.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631349+0000", 2024-09-14T22:38:06.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6801999999999999e-05 2024-09-14T22:38:06.707 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.708 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:06.708 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.708 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144996 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3194880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:06.708 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:38:06.708 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.112890111, 2024-09-14T22:38:06.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081947213000000005, 2024-09-14T22:38:06.708 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:06.708 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:06.708 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:06.708 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:06.708 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:06.708 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.709 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144996 2024-09-14T22:38:06.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.709 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:06.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:06.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:38:06.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:06.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:38:06.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:06.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:38:06.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3e-07 2024-09-14T22:38:06.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:06.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549654+0000", 2024-09-14T22:38:06.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6900000000000001e-06 2024-09-14T22:38:06.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:06.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549665+0000", 2024-09-14T22:38:06.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1392999999999999e-05 2024-09-14T22:38:06.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549675+0000", 2024-09-14T22:38:06.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5459999999999997e-06 2024-09-14T22:38:06.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:06.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631401+0000", 2024-09-14T22:38:06.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081726036000000002 2024-09-14T22:38:06.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:06.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631492+0000", 2024-09-14T22:38:06.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0886999999999999e-05 2024-09-14T22:38:06.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:06.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631598+0000", 2024-09-14T22:38:06.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106031 2024-09-14T22:38:06.712 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.712 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:06.712 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.712 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144997 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:06.712 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549692+0000", 2024-09-14T22:38:06.712 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.112848446999999, 2024-09-14T22:38:06.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082149925999999998, 2024-09-14T22:38:06.713 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:06.713 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:06.713 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:06.713 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:06.713 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:06.713 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.713 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144997 2024-09-14T22:38:06.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.713 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:06.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:06.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549692+0000", 2024-09-14T22:38:06.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:06.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549692+0000", 2024-09-14T22:38:06.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:06.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549693+0000", 2024-09-14T22:38:06.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.540000000000001e-07 2024-09-14T22:38:06.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:06.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549698+0000", 2024-09-14T22:38:06.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1460000000000004e-06 2024-09-14T22:38:06.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:06.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549715+0000", 2024-09-14T22:38:06.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6983e-05 2024-09-14T22:38:06.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549728+0000", 2024-09-14T22:38:06.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2326e-05 2024-09-14T22:38:06.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:06.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631649+0000", 2024-09-14T22:38:06.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081921830000000001 2024-09-14T22:38:06.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:06.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631726+0000", 2024-09-14T22:38:06.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6759000000000001e-05 2024-09-14T22:38:06.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:06.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631842+0000", 2024-09-14T22:38:06.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011592800000000001 2024-09-14T22:38:06.716 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.716 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:06.716 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.717 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144998 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 2965504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:06.717 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549741+0000", 2024-09-14T22:38:06.717 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.112799154999999, 2024-09-14T22:38:06.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082394832000000001, 2024-09-14T22:38:06.717 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:06.717 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:06.717 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:06.717 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:06.717 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:06.717 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.717 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144998 2024-09-14T22:38:06.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.717 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:06.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:06.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549741+0000", 2024-09-14T22:38:06.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:06.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549741+0000", 2024-09-14T22:38:06.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:06.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549742+0000", 2024-09-14T22:38:06.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.06e-07 2024-09-14T22:38:06.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:06.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549757+0000", 2024-09-14T22:38:06.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4839e-05 2024-09-14T22:38:06.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:06.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549770+0000", 2024-09-14T22:38:06.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2879e-05 2024-09-14T22:38:06.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549780+0000", 2024-09-14T22:38:06.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0304e-05 2024-09-14T22:38:06.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:06.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631904+0000", 2024-09-14T22:38:06.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082124168999999997 2024-09-14T22:38:06.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:06.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631995+0000", 2024-09-14T22:38:06.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0993000000000001e-05 2024-09-14T22:38:06.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:06.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632136+0000", 2024-09-14T22:38:06.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014094199999999999 2024-09-14T22:38:06.721 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.721 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:06.721 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.721 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144999 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3588096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:06.722 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:38:06.722 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.112748945, 2024-09-14T22:38:06.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082593742999999997, 2024-09-14T22:38:06.722 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:06.722 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:06.722 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:06.722 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:06.722 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:06.722 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.722 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144999 2024-09-14T22:38:06.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.722 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:06.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:06.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:38:06.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:06.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:38:06.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:06.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:38:06.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1999999999999999e-07 2024-09-14T22:38:06.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:06.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549795+0000", 2024-09-14T22:38:06.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9569999999999998e-06 2024-09-14T22:38:06.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:06.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549807+0000", 2024-09-14T22:38:06.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1691999999999999e-05 2024-09-14T22:38:06.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549817+0000", 2024-09-14T22:38:06.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5100000000000004e-06 2024-09-14T22:38:06.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:06.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632190+0000", 2024-09-14T22:38:06.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082373374999999999 2024-09-14T22:38:06.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:06.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632264+0000", 2024-09-14T22:38:06.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3972000000000003e-05 2024-09-14T22:38:06.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:06.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632385+0000", 2024-09-14T22:38:06.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121517 2024-09-14T22:38:06.726 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.726 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:06.726 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.726 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145000 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3063808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:06.726 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549828+0000", 2024-09-14T22:38:06.726 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.112712667, 2024-09-14T22:38:06.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082860235000000004, 2024-09-14T22:38:06.726 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:06.727 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:06.727 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:06.727 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:06.727 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:06.727 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.727 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145000 2024-09-14T22:38:06.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.727 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:06.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:06.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549828+0000", 2024-09-14T22:38:06.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:06.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549828+0000", 2024-09-14T22:38:06.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:06.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549829+0000", 2024-09-14T22:38:06.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3300000000000002e-07 2024-09-14T22:38:06.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:06.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549831+0000", 2024-09-14T22:38:06.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6740000000000001e-06 2024-09-14T22:38:06.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:06.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549842+0000", 2024-09-14T22:38:06.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0316000000000001e-05 2024-09-14T22:38:06.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549851+0000", 2024-09-14T22:38:06.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7229999999999999e-06 2024-09-14T22:38:06.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:06.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632448+0000", 2024-09-14T22:38:06.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082596648999999994 2024-09-14T22:38:06.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:06.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632568+0000", 2024-09-14T22:38:06.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119592 2024-09-14T22:38:06.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:06.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632688+0000", 2024-09-14T22:38:06.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120648 2024-09-14T22:38:06.730 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.730 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:06.731 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.731 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145001 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3162112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:06.731 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549863+0000", 2024-09-14T22:38:06.731 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.112677439, 2024-09-14T22:38:06.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083105534999999994, 2024-09-14T22:38:06.731 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:06.731 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:06.731 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:06.731 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:06.731 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:06.731 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.732 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145001 2024-09-14T22:38:06.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.732 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:06.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:06.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549863+0000", 2024-09-14T22:38:06.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:06.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549863+0000", 2024-09-14T22:38:06.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:06.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549864+0000", 2024-09-14T22:38:06.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4300000000000003e-07 2024-09-14T22:38:06.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:06.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549867+0000", 2024-09-14T22:38:06.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.757e-06 2024-09-14T22:38:06.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:06.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549878+0000", 2024-09-14T22:38:06.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1345999999999999e-05 2024-09-14T22:38:06.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549888+0000", 2024-09-14T22:38:06.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6460000000000005e-06 2024-09-14T22:38:06.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:06.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632754+0000", 2024-09-14T22:38:06.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082866258999999998 2024-09-14T22:38:06.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:06.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632841+0000", 2024-09-14T22:38:06.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6971000000000005e-05 2024-09-14T22:38:06.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:06.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632969+0000", 2024-09-14T22:38:06.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012791300000000001 2024-09-14T22:38:06.735 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.735 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:06.735 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.735 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145002 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3260416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:06.735 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549933+0000", 2024-09-14T22:38:06.735 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.112607936, 2024-09-14T22:38:06.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083291027000000004, 2024-09-14T22:38:06.736 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:06.736 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:06.736 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:06.736 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:06.736 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:06.736 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.736 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145002 2024-09-14T22:38:06.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.736 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:06.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.736 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:06.736 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549933+0000", 2024-09-14T22:38:06.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:06.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549933+0000", 2024-09-14T22:38:06.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:06.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549934+0000", 2024-09-14T22:38:06.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8700000000000004e-07 2024-09-14T22:38:06.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:06.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549938+0000", 2024-09-14T22:38:06.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5029999999999999e-06 2024-09-14T22:38:06.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:06.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549952+0000", 2024-09-14T22:38:06.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4382999999999999e-05 2024-09-14T22:38:06.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549963+0000", 2024-09-14T22:38:06.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0665999999999999e-05 2024-09-14T22:38:06.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:06.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633025+0000", 2024-09-14T22:38:06.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083061727000000002 2024-09-14T22:38:06.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:06.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633117+0000", 2024-09-14T22:38:06.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1812999999999997e-05 2024-09-14T22:38:06.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:06.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633224+0000", 2024-09-14T22:38:06.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107048 2024-09-14T22:38:06.739 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.739 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:06.740 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.740 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145003 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3457024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:06.740 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549985+0000", 2024-09-14T22:38:06.740 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.112555518000001, 2024-09-14T22:38:06.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083465871999999997, 2024-09-14T22:38:06.740 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:06.740 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:06.740 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:06.740 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:06.740 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:06.741 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.741 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145003 2024-09-14T22:38:06.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.741 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:06.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:06.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549985+0000", 2024-09-14T22:38:06.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:06.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549985+0000", 2024-09-14T22:38:06.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:06.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549986+0000", 2024-09-14T22:38:06.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7299999999999995e-07 2024-09-14T22:38:06.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:06.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549989+0000", 2024-09-14T22:38:06.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8169999999999999e-06 2024-09-14T22:38:06.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:06.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549998+0000", 2024-09-14T22:38:06.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4290000000000007e-06 2024-09-14T22:38:06.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550009+0000", 2024-09-14T22:38:06.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0633e-05 2024-09-14T22:38:06.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:06.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633275+0000", 2024-09-14T22:38:06.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083266397000000006 2024-09-14T22:38:06.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:06.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633353+0000", 2024-09-14T22:38:06.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8129999999999999e-05 2024-09-14T22:38:06.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:06.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633451+0000", 2024-09-14T22:38:06.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7793000000000004e-05 2024-09-14T22:38:06.744 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.744 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:06.745 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.745 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145004 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3358720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:06.745 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.550029+0000", 2024-09-14T22:38:06.745 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.112511947, 2024-09-14T22:38:06.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083662089999999995, 2024-09-14T22:38:06.745 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:06.745 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:06.745 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:06.745 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:06.745 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:06.746 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.746 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145004 2024-09-14T22:38:06.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.746 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:06.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:06.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550029+0000", 2024-09-14T22:38:06.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:06.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550029+0000", 2024-09-14T22:38:06.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:06.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550030+0000", 2024-09-14T22:38:06.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.33e-07 2024-09-14T22:38:06.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:06.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550035+0000", 2024-09-14T22:38:06.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3430000000000003e-06 2024-09-14T22:38:06.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:06.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550053+0000", 2024-09-14T22:38:06.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8485e-05 2024-09-14T22:38:06.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550064+0000", 2024-09-14T22:38:06.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0264e-05 2024-09-14T22:38:06.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:06.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633502+0000", 2024-09-14T22:38:06.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083438498999999999 2024-09-14T22:38:06.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:06.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633592+0000", 2024-09-14T22:38:06.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.022e-05 2024-09-14T22:38:06.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:06.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633691+0000", 2024-09-14T22:38:06.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8345999999999997e-05 2024-09-14T22:38:06.749 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.750 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:06.750 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.750 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145005 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3555328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:06.750 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.550077+0000", 2024-09-14T22:38:06.750 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.112463486999999, 2024-09-14T22:38:06.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083854340999999999, 2024-09-14T22:38:06.750 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:06.750 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:06.750 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:06.750 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:06.750 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:06.750 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.751 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145005 2024-09-14T22:38:06.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.751 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:06.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:06.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550077+0000", 2024-09-14T22:38:06.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:06.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550077+0000", 2024-09-14T22:38:06.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:06.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550078+0000", 2024-09-14T22:38:06.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8700000000000005e-07 2024-09-14T22:38:06.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:06.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550101+0000", 2024-09-14T22:38:06.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3632e-05 2024-09-14T22:38:06.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:06.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550115+0000", 2024-09-14T22:38:06.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3893000000000001e-05 2024-09-14T22:38:06.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550126+0000", 2024-09-14T22:38:06.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0426e-05 2024-09-14T22:38:06.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:06.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633742+0000", 2024-09-14T22:38:06.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083616633999999995 2024-09-14T22:38:06.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:06.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633816+0000", 2024-09-14T22:38:06.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3690999999999996e-05 2024-09-14T22:38:06.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:06.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633932+0000", 2024-09-14T22:38:06.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011537800000000001 2024-09-14T22:38:06.754 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.754 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:06.754 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.754 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145006 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3653632~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:06.754 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.550236+0000", 2024-09-14T22:38:06.754 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.112304980999999, 2024-09-14T22:38:06.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083924027999999998, 2024-09-14T22:38:06.755 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:06.755 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:06.755 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:06.755 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:06.755 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:06.755 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:06.755 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145006 2024-09-14T22:38:06.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.755 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:06.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:06.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550236+0000", 2024-09-14T22:38:06.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:06.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550236+0000", 2024-09-14T22:38:06.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:06.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550237+0000", 2024-09-14T22:38:06.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2929999999999999e-06 2024-09-14T22:38:06.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:06.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550243+0000", 2024-09-14T22:38:06.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0390000000000001e-06 2024-09-14T22:38:06.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:06.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550256+0000", 2024-09-14T22:38:06.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3393e-05 2024-09-14T22:38:06.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550271+0000", 2024-09-14T22:38:06.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4166e-05 2024-09-14T22:38:06.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:06.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633974+0000", 2024-09-14T22:38:06.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083703825999999995 2024-09-14T22:38:06.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:06.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.634047+0000", 2024-09-14T22:38:06.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2935999999999994e-05 2024-09-14T22:38:06.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:06.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.634160+0000", 2024-09-14T22:38:06.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011237499999999999 2024-09-14T22:38:06.758 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.758 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:06.758 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.758 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.758 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:06.759 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:06.760 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:38:06.761 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.5 to 1 (10000) 2024-09-14T22:38:06.761 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-14T22:38:06.762 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:06.754+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:06.860 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:06.854+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:06.939 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:06.939 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:38:06.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.939 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDPGRecoveryDeleteReply(3.1 e114,113 3:857cc082:::rbd_data.113c7218b071.0000000000000183:head,62'165)", 2024-09-14T22:38:06.939 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:38:06.938907+0000", 2024-09-14T22:38:06.940 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.00033284800000000001, 2024-09-14T22:38:06.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00035555100000000001, 2024-09-14T22:38:06.940 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:06.940 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:06.940 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-14T22:38:06.940 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:06.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:06.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.938907+0000", 2024-09-14T22:38:06.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:06.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.938907+0000", 2024-09-14T22:38:06.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:06.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:06.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.938908+0000", 2024-09-14T22:38:06.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.032e-06 2024-09-14T22:38:06.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:06.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.938917+0000", 2024-09-14T22:38:06.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6840000000000002e-06 2024-09-14T22:38:06.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:06.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.938930+0000", 2024-09-14T22:38:06.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3200000000000001e-05 2024-09-14T22:38:06.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:06.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.938937+0000", 2024-09-14T22:38:06.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9650000000000002e-06 2024-09-14T22:38:06.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:06.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:06.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:06.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.939008+0000", 2024-09-14T22:38:06.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1172000000000002e-05 2024-09-14T22:38:06.942 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.943 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:06.943 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.943 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:06.943 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:38:06.943 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 1 2024-09-14T22:38:06.943 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:06.956 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:38:06.962 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:06.954+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:07.062 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:07.058+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:07.117 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:07.117 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:07.117 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:38:07.117 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:38:07.117 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:07.131 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:38:07.164 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:07.158+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:07.265 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:07.258+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:07.287 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:07.287 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:38:07.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:38:07.288 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:38:07.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.288 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:122960 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 2347008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:07.288 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:41.998325+0000", 2024-09-14T22:38:07.288 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 25.285975996000001, 2024-09-14T22:38:07.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056570400999999999, 2024-09-14T22:38:07.288 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:07.288 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:07.288 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:07.288 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:07.288 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:07.288 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:07.289 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 122960 2024-09-14T22:38:07.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.289 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:07.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:07.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998325+0000", 2024-09-14T22:38:07.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:07.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998325+0000", 2024-09-14T22:38:07.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:07.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998326+0000", 2024-09-14T22:38:07.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9299999999999998e-07 2024-09-14T22:38:07.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:07.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998329+0000", 2024-09-14T22:38:07.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1370000000000002e-06 2024-09-14T22:38:07.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:07.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998340+0000", 2024-09-14T22:38:07.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1039999999999999e-05 2024-09-14T22:38:07.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:07.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998348+0000", 2024-09-14T22:38:07.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1920000000000005e-06 2024-09-14T22:38:07.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:07.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054700+0000", 2024-09-14T22:38:07.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056351714999999997 2024-09-14T22:38:07.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.292 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:07.292 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054771+0000", 2024-09-14T22:38:07.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1009999999999999e-05 2024-09-14T22:38:07.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.292 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:07.292 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054896+0000", 2024-09-14T22:38:07.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000124714 2024-09-14T22:38:07.292 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.292 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:07.292 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.293 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.293 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:122961 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 2351104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:07.293 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:41.998357+0000", 2024-09-14T22:38:07.293 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 25.285944041, 2024-09-14T22:38:07.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056974912000000003, 2024-09-14T22:38:07.293 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:07.293 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:07.293 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:07.293 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:07.293 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:07.293 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:07.293 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 122961 2024-09-14T22:38:07.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.294 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:07.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:07.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998357+0000", 2024-09-14T22:38:07.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:07.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998357+0000", 2024-09-14T22:38:07.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:07.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998358+0000", 2024-09-14T22:38:07.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9699999999999996e-07 2024-09-14T22:38:07.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:07.295 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998361+0000", 2024-09-14T22:38:07.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1370000000000002e-06 2024-09-14T22:38:07.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:07.295 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998371+0000", 2024-09-14T22:38:07.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5789999999999993e-06 2024-09-14T22:38:07.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:07.295 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998379+0000", 2024-09-14T22:38:07.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0129999999999993e-06 2024-09-14T22:38:07.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.296 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:07.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054998+0000", 2024-09-14T22:38:07.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056618961000000002 2024-09-14T22:38:07.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.296 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:07.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.055126+0000", 2024-09-14T22:38:07.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012795100000000001 2024-09-14T22:38:07.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:07.297 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.055332+0000", 2024-09-14T22:38:07.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020667400000000001 2024-09-14T22:38:07.297 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.297 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:07.297 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.297 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123259 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 741376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:07.297 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:38:07.297 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 25.272218523999999, 2024-09-14T22:38:07.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056672581999999999, 2024-09-14T22:38:07.297 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:07.298 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:07.298 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:07.298 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:07.298 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:07.298 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:07.298 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123259 2024-09-14T22:38:07.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.298 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:07.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:07.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:38:07.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:07.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:38:07.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:07.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:38:07.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1600000000000001e-07 2024-09-14T22:38:07.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:07.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012086+0000", 2024-09-14T22:38:07.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8499999999999998e-06 2024-09-14T22:38:07.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:07.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012097+0000", 2024-09-14T22:38:07.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0943e-05 2024-09-14T22:38:07.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:07.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012105+0000", 2024-09-14T22:38:07.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0830000000000004e-06 2024-09-14T22:38:07.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:07.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068584+0000", 2024-09-14T22:38:07.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056478387999999997 2024-09-14T22:38:07.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:07.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068644+0000", 2024-09-14T22:38:07.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9984e-05 2024-09-14T22:38:07.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:07.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068755+0000", 2024-09-14T22:38:07.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111718 2024-09-14T22:38:07.302 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.302 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:07.302 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.302 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123260 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 778240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:07.302 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012118+0000", 2024-09-14T22:38:07.302 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 25.272183299000002, 2024-09-14T22:38:07.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056825887999999998, 2024-09-14T22:38:07.302 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:07.302 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:07.302 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:07.302 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:07.302 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:07.303 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:07.303 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123260 2024-09-14T22:38:07.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.303 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:07.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:07.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012118+0000", 2024-09-14T22:38:07.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:07.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012118+0000", 2024-09-14T22:38:07.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:07.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012119+0000", 2024-09-14T22:38:07.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.13e-07 2024-09-14T22:38:07.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:07.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012130+0000", 2024-09-14T22:38:07.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1552999999999999e-05 2024-09-14T22:38:07.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:07.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012141+0000", 2024-09-14T22:38:07.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.03e-05 2024-09-14T22:38:07.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.306 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:07.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012149+0000", 2024-09-14T22:38:07.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1419999999999993e-06 2024-09-14T22:38:07.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:07.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068800+0000", 2024-09-14T22:38:07.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056651574000000003 2024-09-14T22:38:07.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:07.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068858+0000", 2024-09-14T22:38:07.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8047000000000002e-05 2024-09-14T22:38:07.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:07.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068944+0000", 2024-09-14T22:38:07.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5659000000000001e-05 2024-09-14T22:38:07.309 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.309 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:07.309 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.310 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123261 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 761856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:07.310 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012161+0000", 2024-09-14T22:38:07.310 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 25.272139897999999, 2024-09-14T22:38:07.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057113753000000003, 2024-09-14T22:38:07.310 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:07.311 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:07.311 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:07.311 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:07.311 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:07.311 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:07.311 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123261 2024-09-14T22:38:07.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.312 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:07.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.312 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:07.312 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012161+0000", 2024-09-14T22:38:07.312 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:07.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012161+0000", 2024-09-14T22:38:07.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:07.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012162+0000", 2024-09-14T22:38:07.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4399999999999999e-07 2024-09-14T22:38:07.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:07.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012177+0000", 2024-09-14T22:38:07.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5099000000000001e-05 2024-09-14T22:38:07.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.315 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:07.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012191+0000", 2024-09-14T22:38:07.316 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3631999999999999e-05 2024-09-14T22:38:07.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.316 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:07.316 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012199+0000", 2024-09-14T22:38:07.316 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5769999999999994e-06 2024-09-14T22:38:07.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:07.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069004+0000", 2024-09-14T22:38:07.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056804504999999998 2024-09-14T22:38:07.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:07.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069137+0000", 2024-09-14T22:38:07.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013302600000000001 2024-09-14T22:38:07.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:07.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069275+0000", 2024-09-14T22:38:07.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013816999999999999 2024-09-14T22:38:07.319 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.319 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:07.319 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.320 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123262 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 724992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:07.320 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012224+0000", 2024-09-14T22:38:07.320 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 25.272077451000001, 2024-09-14T22:38:07.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057417959999999997, 2024-09-14T22:38:07.320 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:07.320 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:07.321 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:07.321 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:07.321 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:07.321 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:07.321 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123262 2024-09-14T22:38:07.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.321 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:07.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:07.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012224+0000", 2024-09-14T22:38:07.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:07.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012224+0000", 2024-09-14T22:38:07.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:07.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012225+0000", 2024-09-14T22:38:07.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4000000000000001e-07 2024-09-14T22:38:07.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.325 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:07.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012228+0000", 2024-09-14T22:38:07.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5729999999999999e-06 2024-09-14T22:38:07.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:07.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012241+0000", 2024-09-14T22:38:07.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3210000000000001e-05 2024-09-14T22:38:07.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:07.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012255+0000", 2024-09-14T22:38:07.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3852000000000001e-05 2024-09-14T22:38:07.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:07.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069341+0000", 2024-09-14T22:38:07.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057086139000000001 2024-09-14T22:38:07.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:07.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069477+0000", 2024-09-14T22:38:07.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013595 2024-09-14T22:38:07.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:07.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069642+0000", 2024-09-14T22:38:07.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016459600000000001 2024-09-14T22:38:07.329 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.329 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:07.330 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.330 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123263 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 782336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:07.330 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012265+0000", 2024-09-14T22:38:07.330 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 25.272035909, 2024-09-14T22:38:07.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057665095, 2024-09-14T22:38:07.331 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:07.331 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:07.331 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:07.331 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:07.331 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:07.331 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:07.332 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123263 2024-09-14T22:38:07.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.332 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:07.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:07.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012265+0000", 2024-09-14T22:38:07.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:07.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012265+0000", 2024-09-14T22:38:07.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:07.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012266+0000", 2024-09-14T22:38:07.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8999999999999996e-07 2024-09-14T22:38:07.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:07.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012269+0000", 2024-09-14T22:38:07.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.796e-06 2024-09-14T22:38:07.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:07.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012279+0000", 2024-09-14T22:38:07.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7389999999999994e-06 2024-09-14T22:38:07.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:07.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012286+0000", 2024-09-14T22:38:07.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8199999999999997e-06 2024-09-14T22:38:07.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:07.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069705+0000", 2024-09-14T22:38:07.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057418590999999998 2024-09-14T22:38:07.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:07.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069799+0000", 2024-09-14T22:38:07.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4419999999999994e-05 2024-09-14T22:38:07.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:07.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069931+0000", 2024-09-14T22:38:07.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013113900000000001 2024-09-14T22:38:07.339 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.340 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:07.340 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.340 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123264 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 745472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:07.340 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012295+0000", 2024-09-14T22:38:07.341 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 25.272005908000001, 2024-09-14T22:38:07.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057959498999999998, 2024-09-14T22:38:07.341 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:07.341 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:07.341 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:07.341 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:07.342 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:07.342 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:07.342 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123264 2024-09-14T22:38:07.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.342 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:07.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:07.343 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012295+0000", 2024-09-14T22:38:07.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.343 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:07.343 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012295+0000", 2024-09-14T22:38:07.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:07.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012296+0000", 2024-09-14T22:38:07.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9699999999999996e-07 2024-09-14T22:38:07.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:07.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012299+0000", 2024-09-14T22:38:07.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.79e-06 2024-09-14T22:38:07.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:07.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012320+0000", 2024-09-14T22:38:07.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1242000000000001e-05 2024-09-14T22:38:07.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:07.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012338+0000", 2024-09-14T22:38:07.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7768999999999999e-05 2024-09-14T22:38:07.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:07.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069995+0000", 2024-09-14T22:38:07.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057657 2024-09-14T22:38:07.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:07.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070132+0000", 2024-09-14T22:38:07.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013727800000000001 2024-09-14T22:38:07.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:07.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070255+0000", 2024-09-14T22:38:07.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012282300000000001 2024-09-14T22:38:07.350 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.350 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:07.350 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.351 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123265 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1056768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:07.351 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012348+0000", 2024-09-14T22:38:07.351 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 25.271952953, 2024-09-14T22:38:07.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058145087999999998, 2024-09-14T22:38:07.351 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:07.352 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:07.352 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:07.352 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:07.352 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:07.352 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:07.352 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123265 2024-09-14T22:38:07.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.353 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:07.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:07.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012348+0000", 2024-09-14T22:38:07.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:07.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012348+0000", 2024-09-14T22:38:07.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:07.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012349+0000", 2024-09-14T22:38:07.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6599999999999996e-07 2024-09-14T22:38:07.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:07.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012352+0000", 2024-09-14T22:38:07.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3139999999999999e-06 2024-09-14T22:38:07.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:07.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012365+0000", 2024-09-14T22:38:07.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2625e-05 2024-09-14T22:38:07.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:07.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012373+0000", 2024-09-14T22:38:07.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2470000000000008e-06 2024-09-14T22:38:07.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:07.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070286+0000", 2024-09-14T22:38:07.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057912547000000002 2024-09-14T22:38:07.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:07.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070366+0000", 2024-09-14T22:38:07.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0468000000000003e-05 2024-09-14T22:38:07.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:07.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070493+0000", 2024-09-14T22:38:07.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012722100000000001 2024-09-14T22:38:07.357 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.357 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:07.357 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.358 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123266 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 765952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:07.358 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:38:07.358 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 25.271918705000001, 2024-09-14T22:38:07.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058386949, 2024-09-14T22:38:07.358 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:07.358 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:07.358 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:07.358 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:07.358 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:07.359 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:07.359 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123266 2024-09-14T22:38:07.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.359 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:07.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:07.359 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:38:07.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:07.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:38:07.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:07.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:38:07.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9299999999999998e-07 2024-09-14T22:38:07.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:07.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012386+0000", 2024-09-14T22:38:07.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8729999999999999e-06 2024-09-14T22:38:07.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:07.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012406+0000", 2024-09-14T22:38:07.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0103e-05 2024-09-14T22:38:07.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:07.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012419+0000", 2024-09-14T22:38:07.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.311e-05 2024-09-14T22:38:07.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:07.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070550+0000", 2024-09-14T22:38:07.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058130902999999998 2024-09-14T22:38:07.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:07.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070650+0000", 2024-09-14T22:38:07.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010005000000000001 2024-09-14T22:38:07.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:07.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070770+0000", 2024-09-14T22:38:07.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119317 2024-09-14T22:38:07.364 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.364 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:07.365 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.365 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123267 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1077248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:07.365 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012429+0000", 2024-09-14T22:38:07.365 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 25.271872791, 2024-09-14T22:38:07.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058620343999999998, 2024-09-14T22:38:07.366 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:07.366 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:07.366 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:07.366 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:07.366 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:07.367 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:07.367 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123267 2024-09-14T22:38:07.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.367 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:07.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:07.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012429+0000", 2024-09-14T22:38:07.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:07.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012429+0000", 2024-09-14T22:38:07.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:07.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013488+0000", 2024-09-14T22:38:07.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001059061 2024-09-14T22:38:07.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:07.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013499+0000", 2024-09-14T22:38:07.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1348999999999999e-05 2024-09-14T22:38:07.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.371 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:07.371 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013534+0000", 2024-09-14T22:38:07.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5206000000000001e-05 2024-09-14T22:38:07.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:07.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013558+0000", 2024-09-14T22:38:07.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4324999999999998e-05 2024-09-14T22:38:07.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:07.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070834+0000", 2024-09-14T22:38:07.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057275110999999997 2024-09-14T22:38:07.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:07.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070937+0000", 2024-09-14T22:38:07.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010306099999999999 2024-09-14T22:38:07.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:07.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071049+0000", 2024-09-14T22:38:07.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112231 2024-09-14T22:38:07.373 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.374 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:07.374 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.374 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123268 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1097728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:07.374 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013593+0000", 2024-09-14T22:38:07.374 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 25.270708068000001, 2024-09-14T22:38:07.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057751793000000003, 2024-09-14T22:38:07.374 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:07.374 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:07.374 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:07.374 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:07.374 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:07.374 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:07.375 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123268 2024-09-14T22:38:07.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.375 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:07.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:07.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013593+0000", 2024-09-14T22:38:07.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:07.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013593+0000", 2024-09-14T22:38:07.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:07.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013595+0000", 2024-09-14T22:38:07.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.59e-06 2024-09-14T22:38:07.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:07.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013602+0000", 2024-09-14T22:38:07.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3560000000000004e-06 2024-09-14T22:38:07.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:07.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013625+0000", 2024-09-14T22:38:07.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2419e-05 2024-09-14T22:38:07.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:07.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013650+0000", 2024-09-14T22:38:07.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5151999999999999e-05 2024-09-14T22:38:07.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:07.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071096+0000", 2024-09-14T22:38:07.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057445877999999999 2024-09-14T22:38:07.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:07.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071212+0000", 2024-09-14T22:38:07.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116215 2024-09-14T22:38:07.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:07.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071345+0000", 2024-09-14T22:38:07.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013318299999999999 2024-09-14T22:38:07.378 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.378 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:07.378 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.378 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123269 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1060864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:07.378 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013676+0000", 2024-09-14T22:38:07.378 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 25.270625309, 2024-09-14T22:38:07.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058027982999999998, 2024-09-14T22:38:07.379 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:07.379 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:07.379 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:07.379 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:07.379 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:07.379 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:07.379 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123269 2024-09-14T22:38:07.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.379 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:07.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.379 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:07.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013676+0000", 2024-09-14T22:38:07.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:07.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013676+0000", 2024-09-14T22:38:07.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:07.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013677+0000", 2024-09-14T22:38:07.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.387e-06 2024-09-14T22:38:07.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:07.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013684+0000", 2024-09-14T22:38:07.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0530000000000001e-06 2024-09-14T22:38:07.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:07.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013706+0000", 2024-09-14T22:38:07.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1121999999999999e-05 2024-09-14T22:38:07.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:07.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013722+0000", 2024-09-14T22:38:07.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6249999999999999e-05 2024-09-14T22:38:07.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:07.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071402+0000", 2024-09-14T22:38:07.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057679945000000003 2024-09-14T22:38:07.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:07.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071547+0000", 2024-09-14T22:38:07.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000145632 2024-09-14T22:38:07.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:07.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071704+0000", 2024-09-14T22:38:07.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000156594 2024-09-14T22:38:07.382 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.382 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:07.383 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.383 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123270 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1064960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:07.383 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013744+0000", 2024-09-14T22:38:07.383 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 25.270557358000001, 2024-09-14T22:38:07.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058355872000000003, 2024-09-14T22:38:07.383 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:07.383 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:07.383 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:07.383 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:07.383 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:07.383 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:07.383 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123270 2024-09-14T22:38:07.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.384 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:07.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:07.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013744+0000", 2024-09-14T22:38:07.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:07.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013744+0000", 2024-09-14T22:38:07.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:07.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013745+0000", 2024-09-14T22:38:07.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2589999999999999e-06 2024-09-14T22:38:07.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.385 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:07.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013751+0000", 2024-09-14T22:38:07.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1929999999999998e-06 2024-09-14T22:38:07.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.385 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:07.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013778+0000", 2024-09-14T22:38:07.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6472e-05 2024-09-14T22:38:07.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:07.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013795+0000", 2024-09-14T22:38:07.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6626e-05 2024-09-14T22:38:07.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:07.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071775+0000", 2024-09-14T22:38:07.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057980232 2024-09-14T22:38:07.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:07.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071911+0000", 2024-09-14T22:38:07.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000136129 2024-09-14T22:38:07.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:07.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072100+0000", 2024-09-14T22:38:07.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000188961 2024-09-14T22:38:07.387 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.387 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:07.387 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.387 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123271 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1081344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:07.387 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013818+0000", 2024-09-14T22:38:07.387 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 25.270483162000001, 2024-09-14T22:38:07.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058549565999999997, 2024-09-14T22:38:07.387 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:07.388 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:07.388 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:07.388 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:07.388 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:07.388 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:07.388 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123271 2024-09-14T22:38:07.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.388 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:07.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:07.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013818+0000", 2024-09-14T22:38:07.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:07.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013818+0000", 2024-09-14T22:38:07.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:07.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013958+0000", 2024-09-14T22:38:07.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013934 2024-09-14T22:38:07.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:07.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013968+0000", 2024-09-14T22:38:07.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0329000000000001e-05 2024-09-14T22:38:07.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:07.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013995+0000", 2024-09-14T22:38:07.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7613000000000001e-05 2024-09-14T22:38:07.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:07.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014014+0000", 2024-09-14T22:38:07.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8338999999999998e-05 2024-09-14T22:38:07.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:07.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072155+0000", 2024-09-14T22:38:07.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058140989999999997 2024-09-14T22:38:07.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:07.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072247+0000", 2024-09-14T22:38:07.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1774999999999995e-05 2024-09-14T22:38:07.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:07.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072368+0000", 2024-09-14T22:38:07.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012118 2024-09-14T22:38:07.391 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.391 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:07.391 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.392 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123272 3.1 3.4b235a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:07.392 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.014046+0000", 2024-09-14T22:38:07.392 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 25.270255486, 2024-09-14T22:38:07.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058364077, 2024-09-14T22:38:07.392 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:07.392 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:07.392 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-14T22:38:07.392 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:07.392 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:07.392 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:07.392 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123272 2024-09-14T22:38:07.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.393 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:07.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:07.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014046+0000", 2024-09-14T22:38:07.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:07.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014046+0000", 2024-09-14T22:38:07.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:07.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014047+0000", 2024-09-14T22:38:07.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.533e-06 2024-09-14T22:38:07.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:07.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014085+0000", 2024-09-14T22:38:07.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7734999999999997e-05 2024-09-14T22:38:07.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:07.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014111+0000", 2024-09-14T22:38:07.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6055999999999999e-05 2024-09-14T22:38:07.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.397 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:07.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014136+0000", 2024-09-14T22:38:07.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4592000000000001e-05 2024-09-14T22:38:07.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.397 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:07.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072402+0000", 2024-09-14T22:38:07.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058266251999999998 2024-09-14T22:38:07.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.397 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:07.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072410+0000", 2024-09-14T22:38:07.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.909e-06 2024-09-14T22:38:07.398 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.398 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:07.398 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.398 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123273 3.1 3.4b235a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:07.398 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.014161+0000", 2024-09-14T22:38:07.398 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 25.270140458, 2024-09-14T22:38:07.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058284415999999999, 2024-09-14T22:38:07.398 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:07.398 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:07.398 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-14T22:38:07.398 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:07.398 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:07.399 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:07.399 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123273 2024-09-14T22:38:07.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.399 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:07.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:07.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014161+0000", 2024-09-14T22:38:07.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:07.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014161+0000", 2024-09-14T22:38:07.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:07.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.017996+0000", 2024-09-14T22:38:07.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0038350670000000002 2024-09-14T22:38:07.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:07.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.018009+0000", 2024-09-14T22:38:07.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3305999999999999e-05 2024-09-14T22:38:07.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:07.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.018044+0000", 2024-09-14T22:38:07.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4792000000000002e-05 2024-09-14T22:38:07.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:07.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.018067+0000", 2024-09-14T22:38:07.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2595e-05 2024-09-14T22:38:07.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:07.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072439+0000", 2024-09-14T22:38:07.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054372192 2024-09-14T22:38:07.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:07.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072445+0000", 2024-09-14T22:38:07.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4640000000000003e-06 2024-09-14T22:38:07.402 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.402 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:07.402 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.402 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:148469 3.18 3:1f6f7244:::rbd_header.113c827b523f:head [watch reconnect cookie 140250011614800 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-14T22:38:07.402 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:55.746449+0000", 2024-09-14T22:38:07.402 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 11.537851846000001, 2024-09-14T22:38:07.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.996484173, 2024-09-14T22:38:07.402 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:07.402 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:07.403 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:07.403 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:07.403 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:07.403 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:07.403 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 148469 2024-09-14T22:38:07.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.403 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:07.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:07.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746449+0000", 2024-09-14T22:38:07.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:07.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746449+0000", 2024-09-14T22:38:07.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:07.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746452+0000", 2024-09-14T22:38:07.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0019999999999998e-06 2024-09-14T22:38:07.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:07.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746461+0000", 2024-09-14T22:38:07.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0919999999999998e-06 2024-09-14T22:38:07.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:07.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746475+0000", 2024-09-14T22:38:07.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4853e-05 2024-09-14T22:38:07.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:07.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746484+0000", 2024-09-14T22:38:07.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3780000000000001e-06 2024-09-14T22:38:07.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:07.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746559+0000", 2024-09-14T22:38:07.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5259000000000006e-05 2024-09-14T22:38:07.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:07.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746572+0000", 2024-09-14T22:38:07.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.271e-05 2024-09-14T22:38:07.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:07.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727807+0000", 2024-09-14T22:38:07.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98123555900000003 2024-09-14T22:38:07.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:07.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727826+0000", 2024-09-14T22:38:07.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8828e-05 2024-09-14T22:38:07.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:07.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.739033+0000", 2024-09-14T22:38:07.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.011207062 2024-09-14T22:38:07.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:07.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.739479+0000", 2024-09-14T22:38:07.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044563200000000001 2024-09-14T22:38:07.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:07.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.742934+0000", 2024-09-14T22:38:07.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0034547979999999998 2024-09-14T22:38:07.407 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.408 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:07.408 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.408 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:148473 3.18 3:1f6f7244:::rbd_header.113c827b523f:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-14T22:38:07.408 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:55.755969+0000", 2024-09-14T22:38:07.408 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 11.52833208, 2024-09-14T22:38:07.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9872532889999999, 2024-09-14T22:38:07.408 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:07.408 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:07.408 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:07.408 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:07.408 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:07.408 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:07.409 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 148473 2024-09-14T22:38:07.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.409 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:07.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:07.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755969+0000", 2024-09-14T22:38:07.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:07.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755969+0000", 2024-09-14T22:38:07.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:07.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755971+0000", 2024-09-14T22:38:07.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.674e-06 2024-09-14T22:38:07.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:07.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755979+0000", 2024-09-14T22:38:07.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8599999999999993e-06 2024-09-14T22:38:07.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:07.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755992+0000", 2024-09-14T22:38:07.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.361e-05 2024-09-14T22:38:07.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:07.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.756001+0000", 2024-09-14T22:38:07.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.422e-06 2024-09-14T22:38:07.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:07.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.756069+0000", 2024-09-14T22:38:07.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8209999999999999e-05 2024-09-14T22:38:07.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:07.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.756077+0000", 2024-09-14T22:38:07.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.994e-06 2024-09-14T22:38:07.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:07.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727872+0000", 2024-09-14T22:38:07.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.97179484999999999 2024-09-14T22:38:07.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:07.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727885+0000", 2024-09-14T22:38:07.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3101e-05 2024-09-14T22:38:07.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:07.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.740559+0000", 2024-09-14T22:38:07.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0126736300000001 2024-09-14T22:38:07.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:07.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.740777+0000", 2024-09-14T22:38:07.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021822000000000001 2024-09-14T22:38:07.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 2", 2024-09-14T22:38:07.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.741114+0000", 2024-09-14T22:38:07.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000336793 2024-09-14T22:38:07.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:07.414 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.742668+0000", 2024-09-14T22:38:07.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001554061 2024-09-14T22:38:07.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:38:07.414 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.742947+0000", 2024-09-14T22:38:07.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027937200000000001 2024-09-14T22:38:07.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:38:07.414 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743030+0000", 2024-09-14T22:38:07.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3385000000000002e-05 2024-09-14T22:38:07.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:07.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743223+0000", 2024-09-14T22:38:07.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00019210700000000001 2024-09-14T22:38:07.415 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.415 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:07.415 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.415 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:148474 3.18 3:1f6f7244:::rbd_header.113c827b523f:head [watch reconnect cookie 140250011614800 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104)", 2024-09-14T22:38:07.415 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:56.728273+0000", 2024-09-14T22:38:07.415 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.556028832999999, 2024-09-14T22:38:07.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0180497770000001, 2024-09-14T22:38:07.416 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:07.416 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:07.416 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:07.416 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:07.416 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:07.416 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:07.416 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 148474 2024-09-14T22:38:07.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.416 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:07.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:07.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728273+0000", 2024-09-14T22:38:07.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:07.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728273+0000", 2024-09-14T22:38:07.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:07.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728274+0000", 2024-09-14T22:38:07.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7570000000000001e-06 2024-09-14T22:38:07.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:07.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728281+0000", 2024-09-14T22:38:07.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5089999999999999e-06 2024-09-14T22:38:07.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:07.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728298+0000", 2024-09-14T22:38:07.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6727000000000001e-05 2024-09-14T22:38:07.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:07.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728309+0000", 2024-09-14T22:38:07.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1021e-05 2024-09-14T22:38:07.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:07.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.730078+0000", 2024-09-14T22:38:07.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017694010000000001 2024-09-14T22:38:07.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:07.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.730096+0000", 2024-09-14T22:38:07.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8039000000000001e-05 2024-09-14T22:38:07.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:07.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.741672+0000", 2024-09-14T22:38:07.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.011575817 2024-09-14T22:38:07.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2024-09-14T22:38:07.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.741827+0000", 2024-09-14T22:38:07.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000155563 2024-09-14T22:38:07.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:07.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743299+0000", 2024-09-14T22:38:07.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0014712729999999999 2024-09-14T22:38:07.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:07.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743415+0000", 2024-09-14T22:38:07.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011594700000000001 2024-09-14T22:38:07.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:07.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.746322+0000", 2024-09-14T22:38:07.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0029077230000000001 2024-09-14T22:38:07.421 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.421 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:07.421 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.421 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.421 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:07.421 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:07.421 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:38:07.423 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:38:07.424 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:07.418+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:07.524 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:07.518+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:07.596 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:07.597 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:38:07.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.597 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:150772 3.1a 3.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T22:38:07.597 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:38:06.775360+0000", 2024-09-14T22:38:07.597 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.82147419399999999, 2024-09-14T22:38:07.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.82150291499999994, 2024-09-14T22:38:07.597 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:07.597 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:07.597 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "waiting for peered", 2024-09-14T22:38:07.597 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:07.597 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:07.597 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:07.598 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 150772 2024-09-14T22:38:07.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.598 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:07.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:07.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.775360+0000", 2024-09-14T22:38:07.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:07.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.775360+0000", 2024-09-14T22:38:07.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:07.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.775361+0000", 2024-09-14T22:38:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8800000000000002e-07 2024-09-14T22:38:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.775365+0000", 2024-09-14T22:38:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9659999999999998e-06 2024-09-14T22:38:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.775375+0000", 2024-09-14T22:38:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0635999999999999e-05 2024-09-14T22:38:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:07.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.775384+0000", 2024-09-14T22:38:07.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1699999999999997e-06 2024-09-14T22:38:07.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:07.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.775428+0000", 2024-09-14T22:38:07.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4263e-05 2024-09-14T22:38:07.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:07.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:07.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:07.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.775433+0000", 2024-09-14T22:38:07.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6520000000000003e-06 2024-09-14T22:38:07.601 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.601 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:07.601 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.601 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:07.601 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:38:07.601 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 1 2024-09-14T22:38:07.601 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:07.613 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:38:07.634 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:38:07.634 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:38:07.634 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:07.630+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:07.652 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:38:07.652 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:38:07.669 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:38:07.669 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:38:07.686 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:38:07.686 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:38:07.709 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:38:07.710 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:38:07.726 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:07.718+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:07.732 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.6 to 1 (10000) 2024-09-14T22:38:07.750 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T22:38:07.753 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:38:07.753 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:38:07.755 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:07.755 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:07.763 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-14T22:38:07.776 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:38:07.777 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:38:07.777 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:38:07.802 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:38:07.803 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:38:07.832 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:07.826+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:07.919 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:07.919 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:07.927 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:07.922+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:07.940 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:38:08.028 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:08.022+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:08.076 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:08.076 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:08.093 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:38:08.137 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:08.134+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:08.211 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:08.212 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:08.226 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:38:08.230 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:08.222+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:08.330 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:08.322+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:08.400 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:08.400 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:08.414 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:38:08.459 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:08.454+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:08.532 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:08.526+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:08.552 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:08.552 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:08.567 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:38:08.659 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:08.654+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:08.728 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2024-09-14T22:38:08.733 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:08.726+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:08.736 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.7 to 1 (10000) 2024-09-14T22:38:08.751 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:08.751 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:08.767 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:38:08.768 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-14T22:38:08.769 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:38:08.857 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:08.854+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:08.919 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-14T22:38:08.935 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:08.930+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:08.947 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:08.947 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:08.962 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:38:09.036 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:09.030+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:09.133 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:38:09.133 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-14T22:38:09.137 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:09.134+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:09.141 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:09.141 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:09.152 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":175,"stamp":"2024-09-14T22:38:08.893716+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1367241930,"num_objects":1841,"num_object_clones":465,"num_object_copies":3682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":131,"num_objects_misplaced":216,"num_objects_unfound":0,"num_objects_dirty":1841,"num_whiteouts":734,"num_read":187652,"num_read_kb":762324,"num_write":21832,"num_write_kb":373555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2650,"num_bytes_recovered":141833077,"num_keys_recovered":285,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16623,"ondisk_log_size":16623,"up":82,"acting":78,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":440744,"kb_used_data":66220,"kb_used_omap":17,"kb_used_meta":374318,"kb_avail":754533976,"statfs":{"total":773094113280,"available":772642791424,"internally_reserved":0,"allocated":67809280,"data_stored":533331632,"data_compressed":15173301,"data_compressed_allocated":45735936,"data_compressed_original":521584667,"omap_allocated":18180,"internal_metadata":383301884},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12059945,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":131,"num_objects_misplaced":216,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":1851,"num_read_kb":11554,"num_write":416,"num_write_kb":6255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":715,"num_bytes_recovered":38851718,"num_keys_recovered":103,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.919495"},"pg_stats":[{"pgid":"3.1f","version":"110'365","reported_seq":4741,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.730587+0000","last_change":"2024-09-14T22:37:25.739233+0000","last_active":"2024-09-14T22:38:07.730587+0000","last_peered":"2024-09-14T22:38:07.730587+0000","last_clean":"2024-09-14T22:38:07.730587+0000","last_became_active":"2024-09-14T22:37:25.718906+0000","last_became_peered":"2024-09-14T22:37:25.718906+0000","last_unstale":"2024-09-14T22:38:07.730587+0000","last_undegraded":"2024-09-14T22:38:07.730587+0000","last_fullsized":"2024-09-14T22:38:07.730587+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:08:36.649997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017190879999999999,"stat_sum":{"num_bytes":55797417,"num_objects":60,"num_object_clones":16,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":26,"num_read":4758,"num_read_kb":16316,"num_write":371,"num_write_kb":7083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":3277632,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"110'1037","reported_seq":5800,"reported_epoch":115,"state":"peering","last_fresh":"2024-09-14T22:38:07.549207+0000","last_change":"2024-09-14T22:38:07.905499+0000","last_active":"2024-09-14T22:38:07.905499+0000","last_peered":"2024-09-14T22:38:07.549207+0000","last_clean":"2024-09-14T22:38:03.339287+0000","last_became_active":"2024-09-14T22:38:06.762464+0000","last_became_peered":"2024-09-14T22:38:06.762464+0000","last_unstale":"2024-09-14T22:38:07.549207+0000","last_undegraded":"2024-09-14T22:38:07.549207+0000","last_fullsized":"2024-09-14T22:38:07.549207+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1037,"log_dups_size":0,"ondisk_log_size":1037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001723619,"stat_sum":{"num_bytes":73734311,"num_objects":54,"num_object_clones":15,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":22,"num_read":10207,"num_read_kb":24686,"num_write":1581,"num_write_kb":8898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":7412288,"num_keys_recovered":90,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"110'581","reported_seq":9965,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T22:38:05.720237+0000","last_change":"2024-09-14T22:37:51.741030+0000","last_active":"2024-09-14T22:38:05.720237+0000","last_peered":"2024-09-14T22:38:05.720237+0000","last_clean":"2024-09-14T22:38:05.720237+0000","last_became_active":"2024-09-14T22:37:51.730971+0000","last_became_peered":"2024-09-14T22:37:51.730971+0000","last_unstale":"2024-09-14T22:38:05.720237+0000","last_undegraded":"2024-09-14T22:38:05.720237+0000","last_fullsized":"2024-09-14T22:38:05.720237+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:50:14.998252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053914200000000005,"stat_sum":{"num_bytes":76488346,"num_objects":68,"num_object_clones":22,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":31,"num_read":10659,"num_read_kb":63051,"num_write":795,"num_write_kb":13593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":8030815,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"112'552","reported_seq":5710,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.733410+0000","last_change":"2024-09-14T22:36:17.702413+0000","last_active":"2024-09-14T22:38:07.733410+0000","last_peered":"2024-09-14T22:38:07.733410+0000","last_clean":"2024-09-14T22:38:07.733410+0000","last_became_active":"2024-09-14T22:36:15.719470+0000","last_became_peered":"2024-09-14T22:36:15.719470+0000","last_unstale":"2024-09-14T22:38:07.733410+0000","last_undegraded":"2024-09-14T22:38:07.733410+0000","last_fullsized":"2024-09-14T22:38:07.733410+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:37:33.259684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055079499999999997,"stat_sum":{"num_bytes":35950567,"num_objects":59,"num_object_clones":15,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":26,"num_read":5999,"num_read_kb":26064,"num_write":728,"num_write_kb":8503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"110'469","reported_seq":4871,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.729968+0000","last_change":"2024-09-14T22:37:25.736579+0000","last_active":"2024-09-14T22:38:07.729968+0000","last_peered":"2024-09-14T22:38:07.729968+0000","last_clean":"2024-09-14T22:38:07.729968+0000","last_became_active":"2024-09-14T22:37:25.718519+0000","last_became_peered":"2024-09-14T22:37:25.718519+0000","last_unstale":"2024-09-14T22:38:07.729968+0000","last_undegraded":"2024-09-14T22:38:07.729968+0000","last_fullsized":"2024-09-14T22:38:07.729968+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:39:37.098899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092291499999999996,"stat_sum":{"num_bytes":30938989,"num_objects":50,"num_object_clones":14,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":21,"num_read":5129,"num_read_kb":32004,"num_write":569,"num_write_kb":13290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":7858104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"108'403","reported_seq":3305,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-14T22:38:01.422014+0000","last_change":"2024-09-14T22:37:59.734243+0000","last_active":"2024-09-14T22:38:01.422014+0000","last_peered":"2024-09-14T22:38:01.422014+0000","last_clean":"2024-09-14T22:38:01.422014+0000","last_became_active":"2024-09-14T22:37:59.732128+0000","last_became_peered":"2024-09-14T22:37:59.732128+0000","last_unstale":"2024-09-14T22:38:01.422014+0000","last_undegraded":"2024-09-14T22:38:01.422014+0000","last_fullsized":"2024-09-14T22:38:01.422014+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:15:08.486754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012502780000000001,"stat_sum":{"num_bytes":11058244,"num_objects":68,"num_object_clones":19,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":34,"num_read":3839,"num_read_kb":12469,"num_write":407,"num_write_kb":6714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":5018269,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"110'363","reported_seq":5104,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-14T22:38:06.723038+0000","last_change":"2024-09-14T22:36:17.705423+0000","last_active":"2024-09-14T22:38:06.723038+0000","last_peered":"2024-09-14T22:38:06.723038+0000","last_clean":"2024-09-14T22:38:06.723038+0000","last_became_active":"2024-09-14T22:36:15.700990+0000","last_became_peered":"2024-09-14T22:36:15.700990+0000","last_unstale":"2024-09-14T22:38:06.723038+0000","last_undegraded":"2024-09-14T22:38:06.723038+0000","last_fullsized":"2024-09-14T22:38:06.723038+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:55:28.951912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010520620000000001,"stat_sum":{"num_bytes":22178140,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":23,"num_read":5058,"num_read_kb":15226,"num_write":369,"num_write_kb":5409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"109'401","reported_seq":4427,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.736451+0000","last_change":"2024-09-14T22:38:07.735828+0000","last_active":"2024-09-14T22:38:07.736451+0000","last_peered":"2024-09-14T22:38:07.736451+0000","last_clean":"2024-09-14T22:38:07.736451+0000","last_became_active":"2024-09-14T22:38:07.734308+0000","last_became_peered":"2024-09-14T22:38:07.734308+0000","last_unstale":"2024-09-14T22:38:07.736451+0000","last_undegraded":"2024-09-14T22:38:07.736451+0000","last_fullsized":"2024-09-14T22:38:07.736451+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:46:56.903331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002063484,"stat_sum":{"num_bytes":36374590,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":14,"num_read":4669,"num_read_kb":35713,"num_write":566,"num_write_kb":27648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":20067293,"num_keys_recovered":21,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"111'310","reported_seq":3791,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.730687+0000","last_change":"2024-09-14T22:36:17.721195+0000","last_active":"2024-09-14T22:38:07.730687+0000","last_peered":"2024-09-14T22:38:07.730687+0000","last_clean":"2024-09-14T22:38:07.730687+0000","last_became_active":"2024-09-14T22:36:15.715869+0000","last_became_peered":"2024-09-14T22:36:15.715869+0000","last_unstale":"2024-09-14T22:38:07.730687+0000","last_undegraded":"2024-09-14T22:38:07.730687+0000","last_fullsized":"2024-09-14T22:38:07.730687+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:29:33.587279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001160672,"stat_sum":{"num_bytes":19652458,"num_objects":53,"num_object_clones":10,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":24,"num_read":3697,"num_read_kb":15655,"num_write":372,"num_write_kb":20137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"111'471","reported_seq":6163,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.732153+0000","last_change":"2024-09-14T22:36:17.703048+0000","last_active":"2024-09-14T22:38:07.732153+0000","last_peered":"2024-09-14T22:38:07.732153+0000","last_clean":"2024-09-14T22:38:07.732153+0000","last_became_active":"2024-09-14T22:36:15.725078+0000","last_became_peered":"2024-09-14T22:36:15.725078+0000","last_unstale":"2024-09-14T22:38:07.732153+0000","last_undegraded":"2024-09-14T22:38:07.732153+0000","last_fullsized":"2024-09-14T22:38:07.732153+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:50:22.638298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010500290000000001,"stat_sum":{"num_bytes":72644541,"num_objects":65,"num_object_clones":18,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":23,"num_read":6586,"num_read_kb":51137,"num_write":652,"num_write_kb":18892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"111'284","reported_seq":3336,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.737355+0000","last_change":"2024-09-14T22:36:18.009371+0000","last_active":"2024-09-14T22:38:07.737355+0000","last_peered":"2024-09-14T22:38:07.737355+0000","last_clean":"2024-09-14T22:38:07.737355+0000","last_became_active":"2024-09-14T22:36:15.717341+0000","last_became_peered":"2024-09-14T22:36:15.717341+0000","last_unstale":"2024-09-14T22:38:07.737355+0000","last_undegraded":"2024-09-14T22:38:07.737355+0000","last_fullsized":"2024-09-14T22:38:07.737355+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:53:54.367159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054571999999999997,"stat_sum":{"num_bytes":12714098,"num_objects":40,"num_object_clones":11,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":17,"num_read":3096,"num_read_kb":5369,"num_write":293,"num_write_kb":6253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"111'1003","reported_seq":8239,"reported_epoch":115,"state":"active+recovery_wait+remapped","last_fresh":"2024-09-14T22:38:07.793613+0000","last_change":"2024-09-14T22:38:07.793613+0000","last_active":"2024-09-14T22:38:07.793613+0000","last_peered":"2024-09-14T22:38:07.793613+0000","last_clean":"2024-09-14T22:38:04.200895+0000","last_became_active":"2024-09-14T22:38:07.790899+0000","last_became_peered":"2024-09-14T22:38:07.790899+0000","last_unstale":"2024-09-14T22:38:07.793613+0000","last_undegraded":"2024-09-14T22:38:07.793613+0000","last_fullsized":"2024-09-14T22:38:07.793613+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1003,"log_dups_size":0,"ondisk_log_size":1003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010804009999999999,"stat_sum":{"num_bytes":44119068,"num_objects":71,"num_object_clones":19,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":134,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":27,"num_read":10080,"num_read_kb":27226,"num_write":1483,"num_write_kb":11837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":8405044,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[{"shards":"6,7","objects":71}],"blocked_by":[],"up_primary":4,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":205,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.733471+0000","last_change":"2024-09-14T22:36:08.042860+0000","last_active":"2024-09-14T22:38:07.733471+0000","last_peered":"2024-09-14T22:38:07.733471+0000","last_clean":"2024-09-14T22:38:07.733471+0000","last_became_active":"2024-09-14T22:36:06.040961+0000","last_became_peered":"2024-09-14T22:36:06.040961+0000","last_unstale":"2024-09-14T22:38:07.733471+0000","last_undegraded":"2024-09-14T22:38:07.733471+0000","last_fullsized":"2024-09-14T22:38:07.733471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:03:48.547834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010780950000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"111'459","reported_seq":2533,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.729874+0000","last_change":"2024-09-14T22:36:17.720897+0000","last_active":"2024-09-14T22:38:07.729874+0000","last_peered":"2024-09-14T22:38:07.729874+0000","last_clean":"2024-09-14T22:38:07.729874+0000","last_became_active":"2024-09-14T22:36:15.700982+0000","last_became_peered":"2024-09-14T22:36:15.700982+0000","last_unstale":"2024-09-14T22:38:07.729874+0000","last_undegraded":"2024-09-14T22:38:07.729874+0000","last_fullsized":"2024-09-14T22:38:07.729874+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:39:25.124678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010495459999999999,"stat_sum":{"num_bytes":28962458,"num_objects":66,"num_object_clones":17,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":24,"num_read":3553,"num_read_kb":12439,"num_write":490,"num_write_kb":10188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":165,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.734671+0000","last_change":"2024-09-14T22:38:03.977889+0000","last_active":"2024-09-14T22:38:07.734671+0000","last_peered":"2024-09-14T22:38:07.734671+0000","last_clean":"2024-09-14T22:38:07.734671+0000","last_became_active":"2024-09-14T22:38:03.976899+0000","last_became_peered":"2024-09-14T22:38:03.976899+0000","last_unstale":"2024-09-14T22:38:07.734671+0000","last_undegraded":"2024-09-14T22:38:07.734671+0000","last_fullsized":"2024-09-14T22:38:07.734671+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:00:41.381700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"111'406","reported_seq":5172,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.738188+0000","last_change":"2024-09-14T22:36:21.722517+0000","last_active":"2024-09-14T22:38:07.738188+0000","last_peered":"2024-09-14T22:38:07.738188+0000","last_clean":"2024-09-14T22:38:07.738188+0000","last_became_active":"2024-09-14T22:36:21.722085+0000","last_became_peered":"2024-09-14T22:36:21.722085+0000","last_unstale":"2024-09-14T22:38:07.738188+0000","last_undegraded":"2024-09-14T22:38:07.738188+0000","last_fullsized":"2024-09-14T22:38:07.738188+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:05:24.340148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020774109999999999,"stat_sum":{"num_bytes":52168094,"num_objects":52,"num_object_clones":11,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":21,"num_read":5682,"num_read_kb":28617,"num_write":472,"num_write_kb":8463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":205,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.737893+0000","last_change":"2024-09-14T22:36:08.054606+0000","last_active":"2024-09-14T22:38:07.737893+0000","last_peered":"2024-09-14T22:38:07.737893+0000","last_clean":"2024-09-14T22:38:07.737893+0000","last_became_active":"2024-09-14T22:36:06.266908+0000","last_became_peered":"2024-09-14T22:36:06.266908+0000","last_unstale":"2024-09-14T22:38:07.737893+0000","last_undegraded":"2024-09-14T22:38:07.737893+0000","last_fullsized":"2024-09-14T22:38:07.737893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:36.738643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098249300000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"112'1318","reported_seq":4868,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T22:38:05.719085+0000","last_change":"2024-09-14T22:37:17.676125+0000","last_active":"2024-09-14T22:38:05.719085+0000","last_peered":"2024-09-14T22:38:05.719085+0000","last_clean":"2024-09-14T22:38:05.719085+0000","last_became_active":"2024-09-14T22:37:17.674322+0000","last_became_peered":"2024-09-14T22:37:17.674322+0000","last_unstale":"2024-09-14T22:38:05.719085+0000","last_undegraded":"2024-09-14T22:38:05.719085+0000","last_fullsized":"2024-09-14T22:38:05.719085+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1318,"log_dups_size":0,"ondisk_log_size":1318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:29:16.921603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012564830000000001,"stat_sum":{"num_bytes":27908088,"num_objects":62,"num_object_clones":15,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":26,"num_read":9453,"num_read_kb":23521,"num_write":2354,"num_write_kb":17866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":4264139,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":155,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.730090+0000","last_change":"2024-09-14T22:36:41.930961+0000","last_active":"2024-09-14T22:38:07.730090+0000","last_peered":"2024-09-14T22:38:07.730090+0000","last_clean":"2024-09-14T22:38:07.730090+0000","last_became_active":"2024-09-14T22:36:41.930619+0000","last_became_peered":"2024-09-14T22:36:41.930619+0000","last_unstale":"2024-09-14T22:38:07.730090+0000","last_undegraded":"2024-09-14T22:38:07.730090+0000","last_fullsized":"2024-09-14T22:38:07.730090+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:04:40.582999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"111'439","reported_seq":3922,"reported_epoch":115,"state":"active+undersized+remapped","last_fresh":"2024-09-14T22:38:07.837387+0000","last_change":"2024-09-14T22:38:07.837834+0000","last_active":"2024-09-14T22:38:07.837387+0000","last_peered":"2024-09-14T22:38:07.837387+0000","last_clean":"2024-09-14T22:38:04.091024+0000","last_became_active":"2024-09-14T22:38:06.757565+0000","last_became_peered":"2024-09-14T22:38:06.757565+0000","last_unstale":"2024-09-14T22:38:07.837387+0000","last_undegraded":"2024-09-14T22:38:07.837387+0000","last_fullsized":"2024-09-14T22:38:06.724796+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071220099999999998,"stat_sum":{"num_bytes":28456380,"num_objects":53,"num_object_clones":14,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":21,"num_read":4388,"num_read_kb":9216,"num_write":480,"num_write_kb":5683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":5797306,"num_keys_recovered":72,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5],"avail_no_missing":["5","2","3"],"object_location_counts":[{"shards":"5","objects":53}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"112'320","reported_seq":4760,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.728956+0000","last_change":"2024-09-14T22:36:17.711068+0000","last_active":"2024-09-14T22:38:07.728956+0000","last_peered":"2024-09-14T22:38:07.728956+0000","last_clean":"2024-09-14T22:38:07.728956+0000","last_became_active":"2024-09-14T22:36:15.706147+0000","last_became_peered":"2024-09-14T22:36:15.706147+0000","last_unstale":"2024-09-14T22:38:07.728956+0000","last_undegraded":"2024-09-14T22:38:07.728956+0000","last_fullsized":"2024-09-14T22:38:07.728956+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:37:40.830454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082625500000000002,"stat_sum":{"num_bytes":52612246,"num_objects":53,"num_object_clones":16,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":22,"num_read":4326,"num_read_kb":25402,"num_write":389,"num_write_kb":13872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":207,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.732075+0000","last_change":"2024-09-14T22:36:08.055938+0000","last_active":"2024-09-14T22:38:07.732075+0000","last_peered":"2024-09-14T22:38:07.732075+0000","last_clean":"2024-09-14T22:38:07.732075+0000","last_became_active":"2024-09-14T22:36:06.039600+0000","last_became_peered":"2024-09-14T22:36:06.039600+0000","last_unstale":"2024-09-14T22:38:07.732075+0000","last_undegraded":"2024-09-14T22:38:07.732075+0000","last_fullsized":"2024-09-14T22:38:07.732075+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:49:09.245563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021617870000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"111'818","reported_seq":8653,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T22:38:05.718388+0000","last_change":"2024-09-14T22:37:16.669227+0000","last_active":"2024-09-14T22:38:05.718388+0000","last_peered":"2024-09-14T22:38:05.718388+0000","last_clean":"2024-09-14T22:38:05.718388+0000","last_became_active":"2024-09-14T22:37:16.662230+0000","last_became_peered":"2024-09-14T22:37:16.662230+0000","last_unstale":"2024-09-14T22:38:05.718388+0000","last_undegraded":"2024-09-14T22:38:05.718388+0000","last_fullsized":"2024-09-14T22:38:05.718388+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":818,"log_dups_size":0,"ondisk_log_size":818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:38:34.241178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021500769999999998,"stat_sum":{"num_bytes":79368880,"num_objects":62,"num_object_clones":13,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":18,"num_read":9980,"num_read_kb":26598,"num_write":1149,"num_write_kb":14782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":4851034,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":137,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T22:38:05.724182+0000","last_change":"2024-09-14T22:38:05.724182+0000","last_active":"2024-09-14T22:38:05.724182+0000","last_peered":"2024-09-14T22:38:05.724182+0000","last_clean":"2024-09-14T22:38:05.724182+0000","last_became_active":"2024-09-14T22:38:05.723636+0000","last_became_peered":"2024-09-14T22:38:05.723636+0000","last_unstale":"2024-09-14T22:38:05.724182+0000","last_undegraded":"2024-09-14T22:38:05.724182+0000","last_fullsized":"2024-09-14T22:38:05.724182+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:14:09.687576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"110'338","reported_seq":5001,"reported_epoch":114,"state":"peering","last_fresh":"2024-09-14T22:38:07.150202+0000","last_change":"2024-09-14T22:38:07.730383+0000","last_active":"2024-09-14T22:38:07.730383+0000","last_peered":"2024-09-14T22:38:07.150202+0000","last_clean":"2024-09-14T22:38:03.971944+0000","last_became_active":"2024-09-14T22:38:06.752878+0000","last_became_peered":"2024-09-14T22:38:06.752878+0000","last_unstale":"2024-09-14T22:38:07.150202+0000","last_undegraded":"2024-09-14T22:38:07.150202+0000","last_fullsized":"2024-09-14T22:38:06.726031+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085549400000000005,"stat_sum":{"num_bytes":60476484,"num_objects":48,"num_object_clones":13,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":17,"num_read":4236,"num_read_kb":21069,"num_write":352,"num_write_kb":6997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":3679238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,4,5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":221,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.735420+0000","last_change":"2024-09-14T22:38:03.976161+0000","last_active":"2024-09-14T22:38:07.735420+0000","last_peered":"2024-09-14T22:38:07.735420+0000","last_clean":"2024-09-14T22:38:07.735420+0000","last_became_active":"2024-09-14T22:38:03.975240+0000","last_became_peered":"2024-09-14T22:38:03.975240+0000","last_unstale":"2024-09-14T22:38:07.735420+0000","last_undegraded":"2024-09-14T22:38:07.735420+0000","last_fullsized":"2024-09-14T22:38:07.735420+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:34:10.534483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010817520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"109'997","reported_seq":3989,"reported_epoch":115,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-14T22:38:07.901819+0000","last_change":"2024-09-14T22:38:07.841030+0000","last_active":"2024-09-14T22:38:07.901819+0000","last_peered":"2024-09-14T22:38:07.901819+0000","last_clean":"2024-09-14T22:38:03.978022+0000","last_became_active":"2024-09-14T22:38:06.768721+0000","last_became_peered":"2024-09-14T22:38:06.768721+0000","last_unstale":"2024-09-14T22:38:07.901819+0000","last_undegraded":"2024-09-14T22:38:07.901819+0000","last_fullsized":"2024-09-14T22:38:06.724937+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":997,"log_dups_size":0,"ondisk_log_size":997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56349086,"num_objects":66,"num_object_clones":14,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":58,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":24,"num_read":7202,"num_read_kb":15917,"num_write":1503,"num_write_kb":11829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":181921,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5],"avail_no_missing":["5","6"],"object_location_counts":[{"shards":"5","objects":8},{"shards":"5,6","objects":58}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":213,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.731808+0000","last_change":"2024-09-14T22:37:13.625072+0000","last_active":"2024-09-14T22:38:07.731808+0000","last_peered":"2024-09-14T22:38:07.731808+0000","last_clean":"2024-09-14T22:38:07.731808+0000","last_became_active":"2024-09-14T22:37:13.624669+0000","last_became_peered":"2024-09-14T22:37:13.624669+0000","last_unstale":"2024-09-14T22:38:07.731808+0000","last_undegraded":"2024-09-14T22:38:07.731808+0000","last_fullsized":"2024-09-14T22:38:07.731808+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:54:20.545098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011401880000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":357,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.735553+0000","last_change":"2024-09-14T22:36:00.402862+0000","last_active":"2024-09-14T22:38:07.735553+0000","last_peered":"2024-09-14T22:38:07.735553+0000","last_clean":"2024-09-14T22:38:07.735553+0000","last_became_active":"2024-09-14T22:36:00.400529+0000","last_became_peered":"2024-09-14T22:36:00.400529+0000","last_unstale":"2024-09-14T22:38:07.735553+0000","last_undegraded":"2024-09-14T22:38:07.735553+0000","last_fullsized":"2024-09-14T22:38:07.735553+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_clean_scrub_stamp":"2024-09-14T22:35:58.989025+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:07:12.951728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"114'541","reported_seq":6940,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-09-14T22:38:06.950103+0000","last_change":"2024-09-14T22:38:07.905458+0000","last_active":"2024-09-14T22:38:07.905458+0000","last_peered":"2024-09-14T22:38:06.950103+0000","last_clean":"2024-09-14T22:38:03.386180+0000","last_became_active":"2024-09-14T22:38:06.741191+0000","last_became_peered":"2024-09-14T22:38:06.741191+0000","last_unstale":"2024-09-14T22:38:06.950103+0000","last_undegraded":"2024-09-14T22:38:06.950103+0000","last_fullsized":"2024-09-14T22:38:06.722325+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001381511,"stat_sum":{"num_bytes":39365030,"num_objects":59,"num_object_clones":15,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":21,"num_read":7580,"num_read_kb":21053,"num_write":866,"num_write_kb":14446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":18690892,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"100'325","reported_seq":6915,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-14T22:38:00.759646+0000","last_change":"2024-09-14T22:37:58.738191+0000","last_active":"2024-09-14T22:38:00.759646+0000","last_peered":"2024-09-14T22:38:00.759646+0000","last_clean":"2024-09-14T22:38:00.759646+0000","last_became_active":"2024-09-14T22:37:58.736525+0000","last_became_peered":"2024-09-14T22:37:58.736525+0000","last_unstale":"2024-09-14T22:38:00.759646+0000","last_undegraded":"2024-09-14T22:38:00.759646+0000","last_fullsized":"2024-09-14T22:38:00.759646+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:00:41.790690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060272699999999995,"stat_sum":{"num_bytes":62873372,"num_objects":79,"num_object_clones":20,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":32,"num_read":6769,"num_read_kb":18458,"num_write":347,"num_write_kb":6853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":10366320,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"114'580","reported_seq":3648,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.733687+0000","last_change":"2024-09-14T22:36:17.776247+0000","last_active":"2024-09-14T22:38:07.733687+0000","last_peered":"2024-09-14T22:38:07.733687+0000","last_clean":"2024-09-14T22:38:07.733687+0000","last_became_active":"2024-09-14T22:36:15.718794+0000","last_became_peered":"2024-09-14T22:36:15.718794+0000","last_unstale":"2024-09-14T22:38:07.733687+0000","last_undegraded":"2024-09-14T22:38:07.733687+0000","last_fullsized":"2024-09-14T22:38:07.733687+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:52:03.394497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088662699999999997,"stat_sum":{"num_bytes":29108648,"num_objects":58,"num_object_clones":12,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":25,"num_read":5503,"num_read_kb":24737,"num_write":545,"num_write_kb":14408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"110'377","reported_seq":2220,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.726784+0000","last_change":"2024-09-14T22:36:17.733002+0000","last_active":"2024-09-14T22:38:07.726784+0000","last_peered":"2024-09-14T22:38:07.726784+0000","last_clean":"2024-09-14T22:38:07.726784+0000","last_became_active":"2024-09-14T22:36:15.702885+0000","last_became_peered":"2024-09-14T22:36:15.702885+0000","last_unstale":"2024-09-14T22:38:07.726784+0000","last_undegraded":"2024-09-14T22:38:07.726784+0000","last_fullsized":"2024-09-14T22:38:07.726784+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:28:04.078434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015180580000000001,"stat_sum":{"num_bytes":36984245,"num_objects":57,"num_object_clones":17,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":25,"num_read":2773,"num_read_kb":11024,"num_write":344,"num_write_kb":6734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"111'347","reported_seq":6456,"reported_epoch":114,"state":"peering","last_fresh":"2024-09-14T22:38:07.250183+0000","last_change":"2024-09-14T22:38:07.733053+0000","last_active":"2024-09-14T22:38:07.733053+0000","last_peered":"2024-09-14T22:38:07.250183+0000","last_clean":"2024-09-14T22:38:04.272478+0000","last_became_active":"2024-09-14T22:38:06.757572+0000","last_became_peered":"2024-09-14T22:38:06.757572+0000","last_unstale":"2024-09-14T22:38:07.250183+0000","last_undegraded":"2024-09-14T22:38:07.250183+0000","last_fullsized":"2024-09-14T22:38:06.725506+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082782099999999996,"stat_sum":{"num_bytes":50022508,"num_objects":57,"num_object_clones":17,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":24,"num_read":5737,"num_read_kb":27105,"num_write":432,"num_write_kb":17759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":23218688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"112'486","reported_seq":3686,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.737616+0000","last_change":"2024-09-14T22:36:18.009776+0000","last_active":"2024-09-14T22:38:07.737616+0000","last_peered":"2024-09-14T22:38:07.737616+0000","last_clean":"2024-09-14T22:38:07.737616+0000","last_became_active":"2024-09-14T22:36:15.723205+0000","last_became_peered":"2024-09-14T22:36:15.723205+0000","last_unstale":"2024-09-14T22:38:07.737616+0000","last_undegraded":"2024-09-14T22:38:07.737616+0000","last_fullsized":"2024-09-14T22:38:07.737616+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:21:43.223903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087783999999999996,"stat_sum":{"num_bytes":26214933,"num_objects":56,"num_object_clones":15,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":18,"num_read":4039,"num_read_kb":15967,"num_write":626,"num_write_kb":5807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"110'397","reported_seq":2154,"reported_epoch":115,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-14T22:38:07.912406+0000","last_change":"2024-09-14T22:38:07.773929+0000","last_active":"2024-09-14T22:38:07.912406+0000","last_peered":"2024-09-14T22:38:07.912406+0000","last_clean":"2024-09-14T22:38:04.976355+0000","last_became_active":"2024-09-14T22:38:07.762760+0000","last_became_peered":"2024-09-14T22:38:07.762760+0000","last_unstale":"2024-09-14T22:38:07.912406+0000","last_undegraded":"2024-09-14T22:38:07.912406+0000","last_fullsized":"2024-09-14T22:38:07.733682+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000776448,"stat_sum":{"num_bytes":28271111,"num_objects":46,"num_object_clones":10,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":24,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":19,"num_read":2287,"num_read_kb":8126,"num_write":482,"num_write_kb":3663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":3477302,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7],"avail_no_missing":["7","1"],"object_location_counts":[{"shards":"1,7","objects":24},{"shards":"7","objects":22}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"110'349","reported_seq":7862,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.738451+0000","last_change":"2024-09-14T22:36:18.010182+0000","last_active":"2024-09-14T22:38:07.738451+0000","last_peered":"2024-09-14T22:38:07.738451+0000","last_clean":"2024-09-14T22:38:07.738451+0000","last_became_active":"2024-09-14T22:36:15.719693+0000","last_became_peered":"2024-09-14T22:36:15.719693+0000","last_unstale":"2024-09-14T22:38:07.738451+0000","last_undegraded":"2024-09-14T22:38:07.738451+0000","last_fullsized":"2024-09-14T22:38:07.738451+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:07:18.549829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001045011,"stat_sum":{"num_bytes":25859668,"num_objects":54,"num_object_clones":12,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":16,"num_read":8144,"num_read_kb":23241,"num_write":432,"num_write_kb":9514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"111'373","reported_seq":3573,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.731097+0000","last_change":"2024-09-14T22:37:26.718624+0000","last_active":"2024-09-14T22:38:07.731097+0000","last_peered":"2024-09-14T22:38:07.731097+0000","last_clean":"2024-09-14T22:38:07.731097+0000","last_became_active":"2024-09-14T22:37:26.717088+0000","last_became_peered":"2024-09-14T22:37:26.717088+0000","last_unstale":"2024-09-14T22:38:07.731097+0000","last_undegraded":"2024-09-14T22:38:07.731097+0000","last_fullsized":"2024-09-14T22:38:07.731097+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:44:33.957303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094227299999999998,"stat_sum":{"num_bytes":40421858,"num_objects":59,"num_object_clones":17,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":24,"num_read":3286,"num_read_kb":29016,"num_write":472,"num_write_kb":14692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"111'566","reported_seq":2141,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.737145+0000","last_change":"2024-09-14T22:36:18.009347+0000","last_active":"2024-09-14T22:38:07.737145+0000","last_peered":"2024-09-14T22:38:07.737145+0000","last_clean":"2024-09-14T22:38:07.737145+0000","last_became_active":"2024-09-14T22:36:15.715569+0000","last_became_peered":"2024-09-14T22:36:15.715569+0000","last_unstale":"2024-09-14T22:38:07.737145+0000","last_undegraded":"2024-09-14T22:38:07.737145+0000","last_fullsized":"2024-09-14T22:38:07.737145+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:49:10.243383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054921600000000005,"stat_sum":{"num_bytes":31469256,"num_objects":46,"num_object_clones":9,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":21,"num_read":3273,"num_read_kb":11450,"num_write":793,"num_write_kb":10903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"112'396","reported_seq":7801,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.730054+0000","last_change":"2024-09-14T22:36:17.720448+0000","last_active":"2024-09-14T22:38:07.730054+0000","last_peered":"2024-09-14T22:38:07.730054+0000","last_clean":"2024-09-14T22:38:07.730054+0000","last_became_active":"2024-09-14T22:36:15.716696+0000","last_became_peered":"2024-09-14T22:36:15.716696+0000","last_unstale":"2024-09-14T22:38:07.730054+0000","last_undegraded":"2024-09-14T22:38:07.730054+0000","last_fullsized":"2024-09-14T22:38:07.730054+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:29:26.675216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086547600000000001,"stat_sum":{"num_bytes":62973071,"num_objects":59,"num_object_clones":11,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":22,"num_read":8846,"num_read_kb":49493,"num_write":448,"num_write_kb":13013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"111'432","reported_seq":6701,"reported_epoch":115,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-14T22:38:07.784296+0000","last_change":"2024-09-14T22:38:07.784773+0000","last_active":"2024-09-14T22:38:07.784296+0000","last_peered":"2024-09-14T22:38:07.784296+0000","last_clean":"2024-09-14T22:38:04.976079+0000","last_became_active":"2024-09-14T22:38:07.782564+0000","last_became_peered":"2024-09-14T22:38:07.782564+0000","last_unstale":"2024-09-14T22:38:07.784296+0000","last_undegraded":"2024-09-14T22:38:07.744011+0000","last_fullsized":"2024-09-14T22:38:07.740588+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010286189999999999,"stat_sum":{"num_bytes":55135358,"num_objects":67,"num_object_clones":16,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":26,"num_read":6702,"num_read_kb":34859,"num_write":509,"num_write_kb":15408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":7236792,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7],"avail_no_missing":["7","2"],"object_location_counts":[{"shards":"2,7","objects":67}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1366651543,"num_objects":1837,"num_object_clones":465,"num_object_copies":3674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":131,"num_objects_misplaced":216,"num_objects_unfound":0,"num_objects_dirty":1837,"num_whiteouts":734,"num_read":187536,"num_read_kb":762224,"num_write":21671,"num_write_kb":371137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2650,"num_bytes_recovered":141833077,"num_keys_recovered":285,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57712640,"data_stored":532595149,"data_compressed":15508251,"data_compressed_allocated":46325760,"data_compressed_original":525836315,"omap_allocated":4238,"internal_metadata":0},"log_size":16503,"ondisk_log_size":16503,"up":64,"acting":60,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542173,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64008,"kb_used_data":10612,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94307832,"statfs":{"total":96636764160,"available":96571219968,"internally_reserved":0,"allocated":10866688,"data_stored":80695705,"data_compressed":2679345,"data_compressed_allocated":7299072,"data_compressed_original":78695629,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542174,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46212,"kb_used_data":6696,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94325628,"statfs":{"total":96636764160,"available":96589443072,"internally_reserved":0,"allocated":6856704,"data_stored":59226182,"data_compressed":1430526,"data_compressed_allocated":4620288,"data_compressed_original":58106961,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542174,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57724,"kb_used_data":8476,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94314116,"statfs":{"total":96636764160,"available":96577654784,"internally_reserved":0,"allocated":8679424,"data_stored":62262855,"data_compressed":1952669,"data_compressed_allocated":5775360,"data_compressed_original":60700882,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542173,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59848,"kb_used_data":6444,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94311992,"statfs":{"total":96636764160,"available":96575479808,"internally_reserved":0,"allocated":6598656,"data_stored":53579741,"data_compressed":1525905,"data_compressed_allocated":4202496,"data_compressed_original":52353202,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542174,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64580,"kb_used_data":7024,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94307260,"statfs":{"total":96636764160,"available":96570634240,"internally_reserved":0,"allocated":7192576,"data_stored":55404111,"data_compressed":1661968,"data_compressed_allocated":4685824,"data_compressed_original":54081269,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":66,"seq":283467841548,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46372,"kb_used_data":9552,"kb_used_omap":6,"kb_used_meta":36793,"kb_avail":94325468,"statfs":{"total":96636764160,"available":96589279232,"internally_reserved":0,"allocated":9781248,"data_stored":80505484,"data_compressed":2031111,"data_compressed_allocated":7008256,"data_compressed_original":79013785,"omap_allocated":7025,"internal_metadata":37676175},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61316,"kb_used_data":7920,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94310524,"statfs":{"total":96636764160,"available":96573976576,"internally_reserved":0,"allocated":8110080,"data_stored":60353117,"data_compressed":1818437,"data_compressed_allocated":5406720,"data_compressed_original":58920539,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542173,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40684,"kb_used_data":9496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331156,"statfs":{"total":96636764160,"available":96595103744,"internally_reserved":0,"allocated":9723904,"data_stored":81304437,"data_compressed":2073340,"data_compressed_allocated":6737920,"data_compressed_original":79712400,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8310784,"data_stored":80079672,"data_compressed":2067531,"data_compressed_allocated":6701056,"data_compressed_original":79138960,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6762496,"data_stored":59725024,"data_compressed":1818437,"data_compressed_allocated":5406720,"data_compressed_original":58920539,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8437760,"data_stored":79877410,"data_compressed":2031111,"data_compressed_allocated":7008256,"data_compressed_original":79013785,"omap_allocated":4238,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6512640,"data_stored":59227200,"data_compressed":1858117,"data_compressed_allocated":5214208,"data_compressed_original":58455797,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5439488,"data_stored":54007001,"data_compressed":1676324,"data_compressed_allocated":4337664,"data_compressed_original":53377202,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7307264,"data_stored":61622278,"data_compressed":1952669,"data_compressed_allocated":5775360,"data_compressed_original":60700882,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5500928,"data_stored":58591785,"data_compressed":1430526,"data_compressed_allocated":4620288,"data_compressed_original":58106961,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":9441280,"data_stored":79464779,"data_compressed":2673536,"data_compressed_allocated":7262208,"data_compressed_original":78122189,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:38:09.154 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:38:09.157 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:38:09.237 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:09.230+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:09.292 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:09.292 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:09.308 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:38:09.355 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:09.350+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:09.442 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:09.438+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:09.491 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:09.491 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:09.507 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:38:09.513 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:38:09.514 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-14T22:38:09.533 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":175,"stamp":"2024-09-14T22:38:08.893716+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1367241930,"num_objects":1841,"num_object_clones":465,"num_object_copies":3682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":131,"num_objects_misplaced":216,"num_objects_unfound":0,"num_objects_dirty":1841,"num_whiteouts":734,"num_read":187652,"num_read_kb":762324,"num_write":21832,"num_write_kb":373555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2650,"num_bytes_recovered":141833077,"num_keys_recovered":285,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16623,"ondisk_log_size":16623,"up":82,"acting":78,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":440744,"kb_used_data":66220,"kb_used_omap":17,"kb_used_meta":374318,"kb_avail":754533976,"statfs":{"total":773094113280,"available":772642791424,"internally_reserved":0,"allocated":67809280,"data_stored":533331632,"data_compressed":15173301,"data_compressed_allocated":45735936,"data_compressed_original":521584667,"omap_allocated":18180,"internal_metadata":383301884},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12059945,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":131,"num_objects_misplaced":216,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":1851,"num_read_kb":11554,"num_write":416,"num_write_kb":6255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":715,"num_bytes_recovered":38851718,"num_keys_recovered":103,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.919495"},"pg_stats":[{"pgid":"3.1f","version":"110'365","reported_seq":4741,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.730587+0000","last_change":"2024-09-14T22:37:25.739233+0000","last_active":"2024-09-14T22:38:07.730587+0000","last_peered":"2024-09-14T22:38:07.730587+0000","last_clean":"2024-09-14T22:38:07.730587+0000","last_became_active":"2024-09-14T22:37:25.718906+0000","last_became_peered":"2024-09-14T22:37:25.718906+0000","last_unstale":"2024-09-14T22:38:07.730587+0000","last_undegraded":"2024-09-14T22:38:07.730587+0000","last_fullsized":"2024-09-14T22:38:07.730587+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:08:36.649997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017190879999999999,"stat_sum":{"num_bytes":55797417,"num_objects":60,"num_object_clones":16,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":26,"num_read":4758,"num_read_kb":16316,"num_write":371,"num_write_kb":7083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":3277632,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"110'1037","reported_seq":5800,"reported_epoch":115,"state":"peering","last_fresh":"2024-09-14T22:38:07.549207+0000","last_change":"2024-09-14T22:38:07.905499+0000","last_active":"2024-09-14T22:38:07.905499+0000","last_peered":"2024-09-14T22:38:07.549207+0000","last_clean":"2024-09-14T22:38:03.339287+0000","last_became_active":"2024-09-14T22:38:06.762464+0000","last_became_peered":"2024-09-14T22:38:06.762464+0000","last_unstale":"2024-09-14T22:38:07.549207+0000","last_undegraded":"2024-09-14T22:38:07.549207+0000","last_fullsized":"2024-09-14T22:38:07.549207+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1037,"log_dups_size":0,"ondisk_log_size":1037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001723619,"stat_sum":{"num_bytes":73734311,"num_objects":54,"num_object_clones":15,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":22,"num_read":10207,"num_read_kb":24686,"num_write":1581,"num_write_kb":8898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":7412288,"num_keys_recovered":90,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"110'581","reported_seq":9965,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T22:38:05.720237+0000","last_change":"2024-09-14T22:37:51.741030+0000","last_active":"2024-09-14T22:38:05.720237+0000","last_peered":"2024-09-14T22:38:05.720237+0000","last_clean":"2024-09-14T22:38:05.720237+0000","last_became_active":"2024-09-14T22:37:51.730971+0000","last_became_peered":"2024-09-14T22:37:51.730971+0000","last_unstale":"2024-09-14T22:38:05.720237+0000","last_undegraded":"2024-09-14T22:38:05.720237+0000","last_fullsized":"2024-09-14T22:38:05.720237+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:50:14.998252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053914200000000005,"stat_sum":{"num_bytes":76488346,"num_objects":68,"num_object_clones":22,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":31,"num_read":10659,"num_read_kb":63051,"num_write":795,"num_write_kb":13593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":8030815,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"112'552","reported_seq":5710,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.733410+0000","last_change":"2024-09-14T22:36:17.702413+0000","last_active":"2024-09-14T22:38:07.733410+0000","last_peered":"2024-09-14T22:38:07.733410+0000","last_clean":"2024-09-14T22:38:07.733410+0000","last_became_active":"2024-09-14T22:36:15.719470+0000","last_became_peered":"2024-09-14T22:36:15.719470+0000","last_unstale":"2024-09-14T22:38:07.733410+0000","last_undegraded":"2024-09-14T22:38:07.733410+0000","last_fullsized":"2024-09-14T22:38:07.733410+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:37:33.259684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055079499999999997,"stat_sum":{"num_bytes":35950567,"num_objects":59,"num_object_clones":15,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":26,"num_read":5999,"num_read_kb":26064,"num_write":728,"num_write_kb":8503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"110'469","reported_seq":4871,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.729968+0000","last_change":"2024-09-14T22:37:25.736579+0000","last_active":"2024-09-14T22:38:07.729968+0000","last_peered":"2024-09-14T22:38:07.729968+0000","last_clean":"2024-09-14T22:38:07.729968+0000","last_became_active":"2024-09-14T22:37:25.718519+0000","last_became_peered":"2024-09-14T22:37:25.718519+0000","last_unstale":"2024-09-14T22:38:07.729968+0000","last_undegraded":"2024-09-14T22:38:07.729968+0000","last_fullsized":"2024-09-14T22:38:07.729968+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:39:37.098899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092291499999999996,"stat_sum":{"num_bytes":30938989,"num_objects":50,"num_object_clones":14,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":21,"num_read":5129,"num_read_kb":32004,"num_write":569,"num_write_kb":13290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":7858104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"108'403","reported_seq":3305,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-14T22:38:01.422014+0000","last_change":"2024-09-14T22:37:59.734243+0000","last_active":"2024-09-14T22:38:01.422014+0000","last_peered":"2024-09-14T22:38:01.422014+0000","last_clean":"2024-09-14T22:38:01.422014+0000","last_became_active":"2024-09-14T22:37:59.732128+0000","last_became_peered":"2024-09-14T22:37:59.732128+0000","last_unstale":"2024-09-14T22:38:01.422014+0000","last_undegraded":"2024-09-14T22:38:01.422014+0000","last_fullsized":"2024-09-14T22:38:01.422014+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:15:08.486754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012502780000000001,"stat_sum":{"num_bytes":11058244,"num_objects":68,"num_object_clones":19,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":34,"num_read":3839,"num_read_kb":12469,"num_write":407,"num_write_kb":6714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":5018269,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"110'363","reported_seq":5104,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-14T22:38:06.723038+0000","last_change":"2024-09-14T22:36:17.705423+0000","last_active":"2024-09-14T22:38:06.723038+0000","last_peered":"2024-09-14T22:38:06.723038+0000","last_clean":"2024-09-14T22:38:06.723038+0000","last_became_active":"2024-09-14T22:36:15.700990+0000","last_became_peered":"2024-09-14T22:36:15.700990+0000","last_unstale":"2024-09-14T22:38:06.723038+0000","last_undegraded":"2024-09-14T22:38:06.723038+0000","last_fullsized":"2024-09-14T22:38:06.723038+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:55:28.951912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010520620000000001,"stat_sum":{"num_bytes":22178140,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":23,"num_read":5058,"num_read_kb":15226,"num_write":369,"num_write_kb":5409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"109'401","reported_seq":4427,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.736451+0000","last_change":"2024-09-14T22:38:07.735828+0000","last_active":"2024-09-14T22:38:07.736451+0000","last_peered":"2024-09-14T22:38:07.736451+0000","last_clean":"2024-09-14T22:38:07.736451+0000","last_became_active":"2024-09-14T22:38:07.734308+0000","last_became_peered":"2024-09-14T22:38:07.734308+0000","last_unstale":"2024-09-14T22:38:07.736451+0000","last_undegraded":"2024-09-14T22:38:07.736451+0000","last_fullsized":"2024-09-14T22:38:07.736451+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:46:56.903331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002063484,"stat_sum":{"num_bytes":36374590,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":14,"num_read":4669,"num_read_kb":35713,"num_write":566,"num_write_kb":27648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":20067293,"num_keys_recovered":21,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"111'310","reported_seq":3791,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.730687+0000","last_change":"2024-09-14T22:36:17.721195+0000","last_active":"2024-09-14T22:38:07.730687+0000","last_peered":"2024-09-14T22:38:07.730687+0000","last_clean":"2024-09-14T22:38:07.730687+0000","last_became_active":"2024-09-14T22:36:15.715869+0000","last_became_peered":"2024-09-14T22:36:15.715869+0000","last_unstale":"2024-09-14T22:38:07.730687+0000","last_undegraded":"2024-09-14T22:38:07.730687+0000","last_fullsized":"2024-09-14T22:38:07.730687+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:29:33.587279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001160672,"stat_sum":{"num_bytes":19652458,"num_objects":53,"num_object_clones":10,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":24,"num_read":3697,"num_read_kb":15655,"num_write":372,"num_write_kb":20137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"111'471","reported_seq":6163,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.732153+0000","last_change":"2024-09-14T22:36:17.703048+0000","last_active":"2024-09-14T22:38:07.732153+0000","last_peered":"2024-09-14T22:38:07.732153+0000","last_clean":"2024-09-14T22:38:07.732153+0000","last_became_active":"2024-09-14T22:36:15.725078+0000","last_became_peered":"2024-09-14T22:36:15.725078+0000","last_unstale":"2024-09-14T22:38:07.732153+0000","last_undegraded":"2024-09-14T22:38:07.732153+0000","last_fullsized":"2024-09-14T22:38:07.732153+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:50:22.638298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010500290000000001,"stat_sum":{"num_bytes":72644541,"num_objects":65,"num_object_clones":18,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":23,"num_read":6586,"num_read_kb":51137,"num_write":652,"num_write_kb":18892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"111'284","reported_seq":3336,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.737355+0000","last_change":"2024-09-14T22:36:18.009371+0000","last_active":"2024-09-14T22:38:07.737355+0000","last_peered":"2024-09-14T22:38:07.737355+0000","last_clean":"2024-09-14T22:38:07.737355+0000","last_became_active":"2024-09-14T22:36:15.717341+0000","last_became_peered":"2024-09-14T22:36:15.717341+0000","last_unstale":"2024-09-14T22:38:07.737355+0000","last_undegraded":"2024-09-14T22:38:07.737355+0000","last_fullsized":"2024-09-14T22:38:07.737355+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:53:54.367159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054571999999999997,"stat_sum":{"num_bytes":12714098,"num_objects":40,"num_object_clones":11,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":17,"num_read":3096,"num_read_kb":5369,"num_write":293,"num_write_kb":6253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"111'1003","reported_seq":8239,"reported_epoch":115,"state":"active+recovery_wait+remapped","last_fresh":"2024-09-14T22:38:07.793613+0000","last_change":"2024-09-14T22:38:07.793613+0000","last_active":"2024-09-14T22:38:07.793613+0000","last_peered":"2024-09-14T22:38:07.793613+0000","last_clean":"2024-09-14T22:38:04.200895+0000","last_became_active":"2024-09-14T22:38:07.790899+0000","last_became_peered":"2024-09-14T22:38:07.790899+0000","last_unstale":"2024-09-14T22:38:07.793613+0000","last_undegraded":"2024-09-14T22:38:07.793613+0000","last_fullsized":"2024-09-14T22:38:07.793613+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1003,"log_dups_size":0,"ondisk_log_size":1003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010804009999999999,"stat_sum":{"num_bytes":44119068,"num_objects":71,"num_object_clones":19,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":134,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":27,"num_read":10080,"num_read_kb":27226,"num_write":1483,"num_write_kb":11837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":8405044,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[{"shards":"6,7","objects":71}],"blocked_by":[],"up_primary":4,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":205,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.733471+0000","last_change":"2024-09-14T22:36:08.042860+0000","last_active":"2024-09-14T22:38:07.733471+0000","last_peered":"2024-09-14T22:38:07.733471+0000","last_clean":"2024-09-14T22:38:07.733471+0000","last_became_active":"2024-09-14T22:36:06.040961+0000","last_became_peered":"2024-09-14T22:36:06.040961+0000","last_unstale":"2024-09-14T22:38:07.733471+0000","last_undegraded":"2024-09-14T22:38:07.733471+0000","last_fullsized":"2024-09-14T22:38:07.733471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:03:48.547834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010780950000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"111'459","reported_seq":2533,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.729874+0000","last_change":"2024-09-14T22:36:17.720897+0000","last_active":"2024-09-14T22:38:07.729874+0000","last_peered":"2024-09-14T22:38:07.729874+0000","last_clean":"2024-09-14T22:38:07.729874+0000","last_became_active":"2024-09-14T22:36:15.700982+0000","last_became_peered":"2024-09-14T22:36:15.700982+0000","last_unstale":"2024-09-14T22:38:07.729874+0000","last_undegraded":"2024-09-14T22:38:07.729874+0000","last_fullsized":"2024-09-14T22:38:07.729874+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:39:25.124678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010495459999999999,"stat_sum":{"num_bytes":28962458,"num_objects":66,"num_object_clones":17,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":24,"num_read":3553,"num_read_kb":12439,"num_write":490,"num_write_kb":10188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":165,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.734671+0000","last_change":"2024-09-14T22:38:03.977889+0000","last_active":"2024-09-14T22:38:07.734671+0000","last_peered":"2024-09-14T22:38:07.734671+0000","last_clean":"2024-09-14T22:38:07.734671+0000","last_became_active":"2024-09-14T22:38:03.976899+0000","last_became_peered":"2024-09-14T22:38:03.976899+0000","last_unstale":"2024-09-14T22:38:07.734671+0000","last_undegraded":"2024-09-14T22:38:07.734671+0000","last_fullsized":"2024-09-14T22:38:07.734671+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:00:41.381700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"111'406","reported_seq":5172,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.738188+0000","last_change":"2024-09-14T22:36:21.722517+0000","last_active":"2024-09-14T22:38:07.738188+0000","last_peered":"2024-09-14T22:38:07.738188+0000","last_clean":"2024-09-14T22:38:07.738188+0000","last_became_active":"2024-09-14T22:36:21.722085+0000","last_became_peered":"2024-09-14T22:36:21.722085+0000","last_unstale":"2024-09-14T22:38:07.738188+0000","last_undegraded":"2024-09-14T22:38:07.738188+0000","last_fullsized":"2024-09-14T22:38:07.738188+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:05:24.340148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020774109999999999,"stat_sum":{"num_bytes":52168094,"num_objects":52,"num_object_clones":11,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":21,"num_read":5682,"num_read_kb":28617,"num_write":472,"num_write_kb":8463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":205,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.737893+0000","last_change":"2024-09-14T22:36:08.054606+0000","last_active":"2024-09-14T22:38:07.737893+0000","last_peered":"2024-09-14T22:38:07.737893+0000","last_clean":"2024-09-14T22:38:07.737893+0000","last_became_active":"2024-09-14T22:36:06.266908+0000","last_became_peered":"2024-09-14T22:36:06.266908+0000","last_unstale":"2024-09-14T22:38:07.737893+0000","last_undegraded":"2024-09-14T22:38:07.737893+0000","last_fullsized":"2024-09-14T22:38:07.737893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:36.738643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098249300000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"112'1318","reported_seq":4868,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T22:38:05.719085+0000","last_change":"2024-09-14T22:37:17.676125+0000","last_active":"2024-09-14T22:38:05.719085+0000","last_peered":"2024-09-14T22:38:05.719085+0000","last_clean":"2024-09-14T22:38:05.719085+0000","last_became_active":"2024-09-14T22:37:17.674322+0000","last_became_peered":"2024-09-14T22:37:17.674322+0000","last_unstale":"2024-09-14T22:38:05.719085+0000","last_undegraded":"2024-09-14T22:38:05.719085+0000","last_fullsized":"2024-09-14T22:38:05.719085+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1318,"log_dups_size":0,"ondisk_log_size":1318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:29:16.921603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012564830000000001,"stat_sum":{"num_bytes":27908088,"num_objects":62,"num_object_clones":15,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":26,"num_read":9453,"num_read_kb":23521,"num_write":2354,"num_write_kb":17866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":4264139,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":155,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.730090+0000","last_change":"2024-09-14T22:36:41.930961+0000","last_active":"2024-09-14T22:38:07.730090+0000","last_peered":"2024-09-14T22:38:07.730090+0000","last_clean":"2024-09-14T22:38:07.730090+0000","last_became_active":"2024-09-14T22:36:41.930619+0000","last_became_peered":"2024-09-14T22:36:41.930619+0000","last_unstale":"2024-09-14T22:38:07.730090+0000","last_undegraded":"2024-09-14T22:38:07.730090+0000","last_fullsized":"2024-09-14T22:38:07.730090+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:04:40.582999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"111'439","reported_seq":3922,"reported_epoch":115,"state":"active+undersized+remapped","last_fresh":"2024-09-14T22:38:07.837387+0000","last_change":"2024-09-14T22:38:07.837834+0000","last_active":"2024-09-14T22:38:07.837387+0000","last_peered":"2024-09-14T22:38:07.837387+0000","last_clean":"2024-09-14T22:38:04.091024+0000","last_became_active":"2024-09-14T22:38:06.757565+0000","last_became_peered":"2024-09-14T22:38:06.757565+0000","last_unstale":"2024-09-14T22:38:07.837387+0000","last_undegraded":"2024-09-14T22:38:07.837387+0000","last_fullsized":"2024-09-14T22:38:06.724796+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071220099999999998,"stat_sum":{"num_bytes":28456380,"num_objects":53,"num_object_clones":14,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":21,"num_read":4388,"num_read_kb":9216,"num_write":480,"num_write_kb":5683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":5797306,"num_keys_recovered":72,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5],"avail_no_missing":["5","2","3"],"object_location_counts":[{"shards":"5","objects":53}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"112'320","reported_seq":4760,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.728956+0000","last_change":"2024-09-14T22:36:17.711068+0000","last_active":"2024-09-14T22:38:07.728956+0000","last_peered":"2024-09-14T22:38:07.728956+0000","last_clean":"2024-09-14T22:38:07.728956+0000","last_became_active":"2024-09-14T22:36:15.706147+0000","last_became_peered":"2024-09-14T22:36:15.706147+0000","last_unstale":"2024-09-14T22:38:07.728956+0000","last_undegraded":"2024-09-14T22:38:07.728956+0000","last_fullsized":"2024-09-14T22:38:07.728956+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:37:40.830454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082625500000000002,"stat_sum":{"num_bytes":52612246,"num_objects":53,"num_object_clones":16,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":22,"num_read":4326,"num_read_kb":25402,"num_write":389,"num_write_kb":13872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":207,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.732075+0000","last_change":"2024-09-14T22:36:08.055938+0000","last_active":"2024-09-14T22:38:07.732075+0000","last_peered":"2024-09-14T22:38:07.732075+0000","last_clean":"2024-09-14T22:38:07.732075+0000","last_became_active":"2024-09-14T22:36:06.039600+0000","last_became_peered":"2024-09-14T22:36:06.039600+0000","last_unstale":"2024-09-14T22:38:07.732075+0000","last_undegraded":"2024-09-14T22:38:07.732075+0000","last_fullsized":"2024-09-14T22:38:07.732075+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:49:09.245563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021617870000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"111'818","reported_seq":8653,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T22:38:05.718388+0000","last_change":"2024-09-14T22:37:16.669227+0000","last_active":"2024-09-14T22:38:05.718388+0000","last_peered":"2024-09-14T22:38:05.718388+0000","last_clean":"2024-09-14T22:38:05.718388+0000","last_became_active":"2024-09-14T22:37:16.662230+0000","last_became_peered":"2024-09-14T22:37:16.662230+0000","last_unstale":"2024-09-14T22:38:05.718388+0000","last_undegraded":"2024-09-14T22:38:05.718388+0000","last_fullsized":"2024-09-14T22:38:05.718388+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":818,"log_dups_size":0,"ondisk_log_size":818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:38:34.241178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021500769999999998,"stat_sum":{"num_bytes":79368880,"num_objects":62,"num_object_clones":13,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":18,"num_read":9980,"num_read_kb":26598,"num_write":1149,"num_write_kb":14782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":4851034,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":137,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T22:38:05.724182+0000","last_change":"2024-09-14T22:38:05.724182+0000","last_active":"2024-09-14T22:38:05.724182+0000","last_peered":"2024-09-14T22:38:05.724182+0000","last_clean":"2024-09-14T22:38:05.724182+0000","last_became_active":"2024-09-14T22:38:05.723636+0000","last_became_peered":"2024-09-14T22:38:05.723636+0000","last_unstale":"2024-09-14T22:38:05.724182+0000","last_undegraded":"2024-09-14T22:38:05.724182+0000","last_fullsized":"2024-09-14T22:38:05.724182+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:14:09.687576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"110'338","reported_seq":5001,"reported_epoch":114,"state":"peering","last_fresh":"2024-09-14T22:38:07.150202+0000","last_change":"2024-09-14T22:38:07.730383+0000","last_active":"2024-09-14T22:38:07.730383+0000","last_peered":"2024-09-14T22:38:07.150202+0000","last_clean":"2024-09-14T22:38:03.971944+0000","last_became_active":"2024-09-14T22:38:06.752878+0000","last_became_peered":"2024-09-14T22:38:06.752878+0000","last_unstale":"2024-09-14T22:38:07.150202+0000","last_undegraded":"2024-09-14T22:38:07.150202+0000","last_fullsized":"2024-09-14T22:38:06.726031+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085549400000000005,"stat_sum":{"num_bytes":60476484,"num_objects":48,"num_object_clones":13,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":17,"num_read":4236,"num_read_kb":21069,"num_write":352,"num_write_kb":6997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":3679238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,4,5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":221,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.735420+0000","last_change":"2024-09-14T22:38:03.976161+0000","last_active":"2024-09-14T22:38:07.735420+0000","last_peered":"2024-09-14T22:38:07.735420+0000","last_clean":"2024-09-14T22:38:07.735420+0000","last_became_active":"2024-09-14T22:38:03.975240+0000","last_became_peered":"2024-09-14T22:38:03.975240+0000","last_unstale":"2024-09-14T22:38:07.735420+0000","last_undegraded":"2024-09-14T22:38:07.735420+0000","last_fullsized":"2024-09-14T22:38:07.735420+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:34:10.534483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010817520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"109'997","reported_seq":3989,"reported_epoch":115,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-14T22:38:07.901819+0000","last_change":"2024-09-14T22:38:07.841030+0000","last_active":"2024-09-14T22:38:07.901819+0000","last_peered":"2024-09-14T22:38:07.901819+0000","last_clean":"2024-09-14T22:38:03.978022+0000","last_became_active":"2024-09-14T22:38:06.768721+0000","last_became_peered":"2024-09-14T22:38:06.768721+0000","last_unstale":"2024-09-14T22:38:07.901819+0000","last_undegraded":"2024-09-14T22:38:07.901819+0000","last_fullsized":"2024-09-14T22:38:06.724937+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":997,"log_dups_size":0,"ondisk_log_size":997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56349086,"num_objects":66,"num_object_clones":14,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":58,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":24,"num_read":7202,"num_read_kb":15917,"num_write":1503,"num_write_kb":11829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":181921,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5],"avail_no_missing":["5","6"],"object_location_counts":[{"shards":"5","objects":8},{"shards":"5,6","objects":58}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":213,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.731808+0000","last_change":"2024-09-14T22:37:13.625072+0000","last_active":"2024-09-14T22:38:07.731808+0000","last_peered":"2024-09-14T22:38:07.731808+0000","last_clean":"2024-09-14T22:38:07.731808+0000","last_became_active":"2024-09-14T22:37:13.624669+0000","last_became_peered":"2024-09-14T22:37:13.624669+0000","last_unstale":"2024-09-14T22:38:07.731808+0000","last_undegraded":"2024-09-14T22:38:07.731808+0000","last_fullsized":"2024-09-14T22:38:07.731808+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:54:20.545098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011401880000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":357,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.735553+0000","last_change":"2024-09-14T22:36:00.402862+0000","last_active":"2024-09-14T22:38:07.735553+0000","last_peered":"2024-09-14T22:38:07.735553+0000","last_clean":"2024-09-14T22:38:07.735553+0000","last_became_active":"2024-09-14T22:36:00.400529+0000","last_became_peered":"2024-09-14T22:36:00.400529+0000","last_unstale":"2024-09-14T22:38:07.735553+0000","last_undegraded":"2024-09-14T22:38:07.735553+0000","last_fullsized":"2024-09-14T22:38:07.735553+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_clean_scrub_stamp":"2024-09-14T22:35:58.989025+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:07:12.951728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"114'541","reported_seq":6940,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-09-14T22:38:06.950103+0000","last_change":"2024-09-14T22:38:07.905458+0000","last_active":"2024-09-14T22:38:07.905458+0000","last_peered":"2024-09-14T22:38:06.950103+0000","last_clean":"2024-09-14T22:38:03.386180+0000","last_became_active":"2024-09-14T22:38:06.741191+0000","last_became_peered":"2024-09-14T22:38:06.741191+0000","last_unstale":"2024-09-14T22:38:06.950103+0000","last_undegraded":"2024-09-14T22:38:06.950103+0000","last_fullsized":"2024-09-14T22:38:06.722325+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001381511,"stat_sum":{"num_bytes":39365030,"num_objects":59,"num_object_clones":15,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":21,"num_read":7580,"num_read_kb":21053,"num_write":866,"num_write_kb":14446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":18690892,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"100'325","reported_seq":6915,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-14T22:38:00.759646+0000","last_change":"2024-09-14T22:37:58.738191+0000","last_active":"2024-09-14T22:38:00.759646+0000","last_peered":"2024-09-14T22:38:00.759646+0000","last_clean":"2024-09-14T22:38:00.759646+0000","last_became_active":"2024-09-14T22:37:58.736525+0000","last_became_peered":"2024-09-14T22:37:58.736525+0000","last_unstale":"2024-09-14T22:38:00.759646+0000","last_undegraded":"2024-09-14T22:38:00.759646+0000","last_fullsized":"2024-09-14T22:38:00.759646+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:00:41.790690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060272699999999995,"stat_sum":{"num_bytes":62873372,"num_objects":79,"num_object_clones":20,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":32,"num_read":6769,"num_read_kb":18458,"num_write":347,"num_write_kb":6853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":10366320,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"114'580","reported_seq":3648,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.733687+0000","last_change":"2024-09-14T22:36:17.776247+0000","last_active":"2024-09-14T22:38:07.733687+0000","last_peered":"2024-09-14T22:38:07.733687+0000","last_clean":"2024-09-14T22:38:07.733687+0000","last_became_active":"2024-09-14T22:36:15.718794+0000","last_became_peered":"2024-09-14T22:36:15.718794+0000","last_unstale":"2024-09-14T22:38:07.733687+0000","last_undegraded":"2024-09-14T22:38:07.733687+0000","last_fullsized":"2024-09-14T22:38:07.733687+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:52:03.394497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088662699999999997,"stat_sum":{"num_bytes":29108648,"num_objects":58,"num_object_clones":12,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":25,"num_read":5503,"num_read_kb":24737,"num_write":545,"num_write_kb":14408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"110'377","reported_seq":2220,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.726784+0000","last_change":"2024-09-14T22:36:17.733002+0000","last_active":"2024-09-14T22:38:07.726784+0000","last_peered":"2024-09-14T22:38:07.726784+0000","last_clean":"2024-09-14T22:38:07.726784+0000","last_became_active":"2024-09-14T22:36:15.702885+0000","last_became_peered":"2024-09-14T22:36:15.702885+0000","last_unstale":"2024-09-14T22:38:07.726784+0000","last_undegraded":"2024-09-14T22:38:07.726784+0000","last_fullsized":"2024-09-14T22:38:07.726784+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:28:04.078434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015180580000000001,"stat_sum":{"num_bytes":36984245,"num_objects":57,"num_object_clones":17,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":25,"num_read":2773,"num_read_kb":11024,"num_write":344,"num_write_kb":6734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"111'347","reported_seq":6456,"reported_epoch":114,"state":"peering","last_fresh":"2024-09-14T22:38:07.250183+0000","last_change":"2024-09-14T22:38:07.733053+0000","last_active":"2024-09-14T22:38:07.733053+0000","last_peered":"2024-09-14T22:38:07.250183+0000","last_clean":"2024-09-14T22:38:04.272478+0000","last_became_active":"2024-09-14T22:38:06.757572+0000","last_became_peered":"2024-09-14T22:38:06.757572+0000","last_unstale":"2024-09-14T22:38:07.250183+0000","last_undegraded":"2024-09-14T22:38:07.250183+0000","last_fullsized":"2024-09-14T22:38:06.725506+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082782099999999996,"stat_sum":{"num_bytes":50022508,"num_objects":57,"num_object_clones":17,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":24,"num_read":5737,"num_read_kb":27105,"num_write":432,"num_write_kb":17759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":23218688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"112'486","reported_seq":3686,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.737616+0000","last_change":"2024-09-14T22:36:18.009776+0000","last_active":"2024-09-14T22:38:07.737616+0000","last_peered":"2024-09-14T22:38:07.737616+0000","last_clean":"2024-09-14T22:38:07.737616+0000","last_became_active":"2024-09-14T22:36:15.723205+0000","last_became_peered":"2024-09-14T22:36:15.723205+0000","last_unstale":"2024-09-14T22:38:07.737616+0000","last_undegraded":"2024-09-14T22:38:07.737616+0000","last_fullsized":"2024-09-14T22:38:07.737616+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:21:43.223903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087783999999999996,"stat_sum":{"num_bytes":26214933,"num_objects":56,"num_object_clones":15,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":18,"num_read":4039,"num_read_kb":15967,"num_write":626,"num_write_kb":5807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"110'397","reported_seq":2154,"reported_epoch":115,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-14T22:38:07.912406+0000","last_change":"2024-09-14T22:38:07.773929+0000","last_active":"2024-09-14T22:38:07.912406+0000","last_peered":"2024-09-14T22:38:07.912406+0000","last_clean":"2024-09-14T22:38:04.976355+0000","last_became_active":"2024-09-14T22:38:07.762760+0000","last_became_peered":"2024-09-14T22:38:07.762760+0000","last_unstale":"2024-09-14T22:38:07.912406+0000","last_undegraded":"2024-09-14T22:38:07.912406+0000","last_fullsized":"2024-09-14T22:38:07.733682+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000776448,"stat_sum":{"num_bytes":28271111,"num_objects":46,"num_object_clones":10,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":24,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":19,"num_read":2287,"num_read_kb":8126,"num_write":482,"num_write_kb":3663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":3477302,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7],"avail_no_missing":["7","1"],"object_location_counts":[{"shards":"1,7","objects":24},{"shards":"7","objects":22}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"110'349","reported_seq":7862,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.738451+0000","last_change":"2024-09-14T22:36:18.010182+0000","last_active":"2024-09-14T22:38:07.738451+0000","last_peered":"2024-09-14T22:38:07.738451+0000","last_clean":"2024-09-14T22:38:07.738451+0000","last_became_active":"2024-09-14T22:36:15.719693+0000","last_became_peered":"2024-09-14T22:36:15.719693+0000","last_unstale":"2024-09-14T22:38:07.738451+0000","last_undegraded":"2024-09-14T22:38:07.738451+0000","last_fullsized":"2024-09-14T22:38:07.738451+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:07:18.549829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001045011,"stat_sum":{"num_bytes":25859668,"num_objects":54,"num_object_clones":12,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":16,"num_read":8144,"num_read_kb":23241,"num_write":432,"num_write_kb":9514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"111'373","reported_seq":3573,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.731097+0000","last_change":"2024-09-14T22:37:26.718624+0000","last_active":"2024-09-14T22:38:07.731097+0000","last_peered":"2024-09-14T22:38:07.731097+0000","last_clean":"2024-09-14T22:38:07.731097+0000","last_became_active":"2024-09-14T22:37:26.717088+0000","last_became_peered":"2024-09-14T22:37:26.717088+0000","last_unstale":"2024-09-14T22:38:07.731097+0000","last_undegraded":"2024-09-14T22:38:07.731097+0000","last_fullsized":"2024-09-14T22:38:07.731097+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:44:33.957303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094227299999999998,"stat_sum":{"num_bytes":40421858,"num_objects":59,"num_object_clones":17,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":24,"num_read":3286,"num_read_kb":29016,"num_write":472,"num_write_kb":14692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"111'566","reported_seq":2141,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.737145+0000","last_change":"2024-09-14T22:36:18.009347+0000","last_active":"2024-09-14T22:38:07.737145+0000","last_peered":"2024-09-14T22:38:07.737145+0000","last_clean":"2024-09-14T22:38:07.737145+0000","last_became_active":"2024-09-14T22:36:15.715569+0000","last_became_peered":"2024-09-14T22:36:15.715569+0000","last_unstale":"2024-09-14T22:38:07.737145+0000","last_undegraded":"2024-09-14T22:38:07.737145+0000","last_fullsized":"2024-09-14T22:38:07.737145+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:49:10.243383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054921600000000005,"stat_sum":{"num_bytes":31469256,"num_objects":46,"num_object_clones":9,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":21,"num_read":3273,"num_read_kb":11450,"num_write":793,"num_write_kb":10903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"112'396","reported_seq":7801,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.730054+0000","last_change":"2024-09-14T22:36:17.720448+0000","last_active":"2024-09-14T22:38:07.730054+0000","last_peered":"2024-09-14T22:38:07.730054+0000","last_clean":"2024-09-14T22:38:07.730054+0000","last_became_active":"2024-09-14T22:36:15.716696+0000","last_became_peered":"2024-09-14T22:36:15.716696+0000","last_unstale":"2024-09-14T22:38:07.730054+0000","last_undegraded":"2024-09-14T22:38:07.730054+0000","last_fullsized":"2024-09-14T22:38:07.730054+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:29:26.675216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086547600000000001,"stat_sum":{"num_bytes":62973071,"num_objects":59,"num_object_clones":11,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":22,"num_read":8846,"num_read_kb":49493,"num_write":448,"num_write_kb":13013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"111'432","reported_seq":6701,"reported_epoch":115,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-14T22:38:07.784296+0000","last_change":"2024-09-14T22:38:07.784773+0000","last_active":"2024-09-14T22:38:07.784296+0000","last_peered":"2024-09-14T22:38:07.784296+0000","last_clean":"2024-09-14T22:38:04.976079+0000","last_became_active":"2024-09-14T22:38:07.782564+0000","last_became_peered":"2024-09-14T22:38:07.782564+0000","last_unstale":"2024-09-14T22:38:07.784296+0000","last_undegraded":"2024-09-14T22:38:07.744011+0000","last_fullsized":"2024-09-14T22:38:07.740588+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010286189999999999,"stat_sum":{"num_bytes":55135358,"num_objects":67,"num_object_clones":16,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":26,"num_read":6702,"num_read_kb":34859,"num_write":509,"num_write_kb":15408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":7236792,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7],"avail_no_missing":["7","2"],"object_location_counts":[{"shards":"2,7","objects":67}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1366651543,"num_objects":1837,"num_object_clones":465,"num_object_copies":3674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":131,"num_objects_misplaced":216,"num_objects_unfound":0,"num_objects_dirty":1837,"num_whiteouts":734,"num_read":187536,"num_read_kb":762224,"num_write":21671,"num_write_kb":371137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2650,"num_bytes_recovered":141833077,"num_keys_recovered":285,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57712640,"data_stored":532595149,"data_compressed":15508251,"data_compressed_allocated":46325760,"data_compressed_original":525836315,"omap_allocated":4238,"internal_metadata":0},"log_size":16503,"ondisk_log_size":16503,"up":64,"acting":60,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542173,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64008,"kb_used_data":10612,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94307832,"statfs":{"total":96636764160,"available":96571219968,"internally_reserved":0,"allocated":10866688,"data_stored":80695705,"data_compressed":2679345,"data_compressed_allocated":7299072,"data_compressed_original":78695629,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542174,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46212,"kb_used_data":6696,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94325628,"statfs":{"total":96636764160,"available":96589443072,"internally_reserved":0,"allocated":6856704,"data_stored":59226182,"data_compressed":1430526,"data_compressed_allocated":4620288,"data_compressed_original":58106961,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542174,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57724,"kb_used_data":8476,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94314116,"statfs":{"total":96636764160,"available":96577654784,"internally_reserved":0,"allocated":8679424,"data_stored":62262855,"data_compressed":1952669,"data_compressed_allocated":5775360,"data_compressed_original":60700882,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542173,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59848,"kb_used_data":6444,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94311992,"statfs":{"total":96636764160,"available":96575479808,"internally_reserved":0,"allocated":6598656,"data_stored":53579741,"data_compressed":1525905,"data_compressed_allocated":4202496,"data_compressed_original":52353202,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542174,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64580,"kb_used_data":7024,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94307260,"statfs":{"total":96636764160,"available":96570634240,"internally_reserved":0,"allocated":7192576,"data_stored":55404111,"data_compressed":1661968,"data_compressed_allocated":4685824,"data_compressed_original":54081269,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":66,"seq":283467841548,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46372,"kb_used_data":9552,"kb_used_omap":6,"kb_used_meta":36793,"kb_avail":94325468,"statfs":{"total":96636764160,"available":96589279232,"internally_reserved":0,"allocated":9781248,"data_stored":80505484,"data_compressed":2031111,"data_compressed_allocated":7008256,"data_compressed_original":79013785,"omap_allocated":7025,"internal_metadata":37676175},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61316,"kb_used_data":7920,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94310524,"statfs":{"total":96636764160,"available":96573976576,"internally_reserved":0,"allocated":8110080,"data_stored":60353117,"data_compressed":1818437,"data_compressed_allocated":5406720,"data_compressed_original":58920539,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542173,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40684,"kb_used_data":9496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331156,"statfs":{"total":96636764160,"available":96595103744,"internally_reserved":0,"allocated":9723904,"data_stored":81304437,"data_compressed":2073340,"data_compressed_allocated":6737920,"data_compressed_original":79712400,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8310784,"data_stored":80079672,"data_compressed":2067531,"data_compressed_allocated":6701056,"data_compressed_original":79138960,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6762496,"data_stored":59725024,"data_compressed":1818437,"data_compressed_allocated":5406720,"data_compressed_original":58920539,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8437760,"data_stored":79877410,"data_compressed":2031111,"data_compressed_allocated":7008256,"data_compressed_original":79013785,"omap_allocated":4238,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6512640,"data_stored":59227200,"data_compressed":1858117,"data_compressed_allocated":5214208,"data_compressed_original":58455797,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5439488,"data_stored":54007001,"data_compressed":1676324,"data_compressed_allocated":4337664,"data_compressed_original":53377202,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7307264,"data_stored":61622278,"data_compressed":1952669,"data_compressed_allocated":5775360,"data_compressed_original":60700882,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5500928,"data_stored":58591785,"data_compressed":1430526,"data_compressed_allocated":4620288,"data_compressed_original":58106961,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":9441280,"data_stored":79464779,"data_compressed":2673536,"data_compressed_allocated":7262208,"data_compressed_original":78122189,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:38:09.535 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-14T22:38:09.577 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:09.574+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:09.644 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:09.638+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:09.649 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:09.649 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:09.666 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:38:09.789 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:09.742+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:09.797 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:09.797 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:09.811 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:38:09.833 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:38:09.855 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:09.850+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:09.896 INFO:teuthology.orchestra.run.smithi001.stdout:5744 flatten 2024-09-14T22:38:09.940 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:09.940 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:09.947 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:09.942+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:09.959 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:38:09.987 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:38:09.987 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:38:09.988 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:38:09.988 INFO:teuthology.orchestra.run.smithi001.stdout:{"fsid":"f07f407a-b17d-42c4-bf73-92dd80114a4f","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"nodeep-scrub flag(s) set","count":12},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 131/3682 objects degraded (3.558%), 1 pg degraded","count":1},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":144,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":117,"num_osds":8,"num_up_osds":8,"osd_up_since":1726353432,"num_in_osds":8,"osd_in_since":1726353348,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":32},{"state_name":"peering","count":3},{"state_name":"active+recovering+undersized+remapped","count":2},{"state_name":"active+undersized+remapped","count":1},{"state_name":"creating+peering","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+remapped","count":1}],"num_pgs":41,"num_pools":3,"num_objects":1841,"data_bytes":1367241930,"bytes_used":451321856,"bytes_avail":772642791424,"bytes_total":773094113280,"inactive_pgs_ratio":0.097560971975326538,"degraded_objects":131,"degraded_total":3682,"degraded_ratio":0.035578489951113526,"misplaced_objects":216,"misplaced_total":3682,"misplaced_ratio":0.058663769690385663,"recovering_objects_per_sec":182,"recovering_bytes_per_sec":9912427,"recovering_keys_per_sec":26,"num_objects_recovered":715,"num_bytes_recovered":38851718,"num_keys_recovered":103,"read_bytes_sec":3018576,"write_bytes_sec":1634169,"read_op_per_sec":472,"write_op_per_sec":106},"fsmap":{"epoch":1,"btime":"2024-09-14T22:35:45:929971+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-14T22:36:58.875005+0000","services":{}},"progress_events":{}} 2024-09-14T22:38:10.005 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:38:10.005 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:38:10.008 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-14T22:38:10.008 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:38:10.024 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:38:10.024 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:38:10.041 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:38:10.041 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:38:10.049 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:10.042+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:10.060 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:38:10.060 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:38:10.079 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:38:10.079 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:38:10.099 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:38:10.099 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:38:10.120 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:38:10.120 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:38:10.122 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:10.122 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:10.122 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:38:10.122 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:10.137 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:38:10.189 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:10.186+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:10.250 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:10.242+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:10.263 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:10.263 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:10.263 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:38:10.263 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:38:10.263 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:10.277 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:38:10.313 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:38:10.314 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-14T22:38:10.332 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":176,"stamp":"2024-09-14T22:38:09.731156+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1367241930,"num_objects":1841,"num_object_clones":465,"num_object_copies":3682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":131,"num_objects_misplaced":216,"num_objects_unfound":0,"num_objects_dirty":1841,"num_whiteouts":734,"num_read":187652,"num_read_kb":762324,"num_write":21832,"num_write_kb":373555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2650,"num_bytes_recovered":141833077,"num_keys_recovered":285,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16623,"ondisk_log_size":16623,"up":82,"acting":78,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":440744,"kb_used_data":66220,"kb_used_omap":17,"kb_used_meta":374318,"kb_avail":754533976,"statfs":{"total":773094113280,"available":772642791424,"internally_reserved":0,"allocated":67809280,"data_stored":533331632,"data_compressed":15173301,"data_compressed_allocated":45735936,"data_compressed_original":521584667,"omap_allocated":18180,"internal_metadata":383301884},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12059945,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":131,"num_objects_misplaced":216,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":1851,"num_read_kb":11554,"num_write":416,"num_write_kb":6255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":715,"num_bytes_recovered":38851718,"num_keys_recovered":103,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.016603"},"pg_stats":[{"pgid":"3.1f","version":"110'365","reported_seq":4741,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.730587+0000","last_change":"2024-09-14T22:37:25.739233+0000","last_active":"2024-09-14T22:38:07.730587+0000","last_peered":"2024-09-14T22:38:07.730587+0000","last_clean":"2024-09-14T22:38:07.730587+0000","last_became_active":"2024-09-14T22:37:25.718906+0000","last_became_peered":"2024-09-14T22:37:25.718906+0000","last_unstale":"2024-09-14T22:38:07.730587+0000","last_undegraded":"2024-09-14T22:38:07.730587+0000","last_fullsized":"2024-09-14T22:38:07.730587+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:08:36.649997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017190879999999999,"stat_sum":{"num_bytes":55797417,"num_objects":60,"num_object_clones":16,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":26,"num_read":4758,"num_read_kb":16316,"num_write":371,"num_write_kb":7083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":3277632,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"110'1037","reported_seq":5800,"reported_epoch":115,"state":"peering","last_fresh":"2024-09-14T22:38:07.549207+0000","last_change":"2024-09-14T22:38:07.905499+0000","last_active":"2024-09-14T22:38:07.905499+0000","last_peered":"2024-09-14T22:38:07.549207+0000","last_clean":"2024-09-14T22:38:03.339287+0000","last_became_active":"2024-09-14T22:38:06.762464+0000","last_became_peered":"2024-09-14T22:38:06.762464+0000","last_unstale":"2024-09-14T22:38:07.549207+0000","last_undegraded":"2024-09-14T22:38:07.549207+0000","last_fullsized":"2024-09-14T22:38:07.549207+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1037,"log_dups_size":0,"ondisk_log_size":1037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001723619,"stat_sum":{"num_bytes":73734311,"num_objects":54,"num_object_clones":15,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":22,"num_read":10207,"num_read_kb":24686,"num_write":1581,"num_write_kb":8898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":7412288,"num_keys_recovered":90,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"110'581","reported_seq":9965,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T22:38:05.720237+0000","last_change":"2024-09-14T22:37:51.741030+0000","last_active":"2024-09-14T22:38:05.720237+0000","last_peered":"2024-09-14T22:38:05.720237+0000","last_clean":"2024-09-14T22:38:05.720237+0000","last_became_active":"2024-09-14T22:37:51.730971+0000","last_became_peered":"2024-09-14T22:37:51.730971+0000","last_unstale":"2024-09-14T22:38:05.720237+0000","last_undegraded":"2024-09-14T22:38:05.720237+0000","last_fullsized":"2024-09-14T22:38:05.720237+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:50:14.998252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053914200000000005,"stat_sum":{"num_bytes":76488346,"num_objects":68,"num_object_clones":22,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":31,"num_read":10659,"num_read_kb":63051,"num_write":795,"num_write_kb":13593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":8030815,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"112'552","reported_seq":5710,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.733410+0000","last_change":"2024-09-14T22:36:17.702413+0000","last_active":"2024-09-14T22:38:07.733410+0000","last_peered":"2024-09-14T22:38:07.733410+0000","last_clean":"2024-09-14T22:38:07.733410+0000","last_became_active":"2024-09-14T22:36:15.719470+0000","last_became_peered":"2024-09-14T22:36:15.719470+0000","last_unstale":"2024-09-14T22:38:07.733410+0000","last_undegraded":"2024-09-14T22:38:07.733410+0000","last_fullsized":"2024-09-14T22:38:07.733410+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:37:33.259684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055079499999999997,"stat_sum":{"num_bytes":35950567,"num_objects":59,"num_object_clones":15,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":26,"num_read":5999,"num_read_kb":26064,"num_write":728,"num_write_kb":8503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"110'469","reported_seq":4871,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.729968+0000","last_change":"2024-09-14T22:37:25.736579+0000","last_active":"2024-09-14T22:38:07.729968+0000","last_peered":"2024-09-14T22:38:07.729968+0000","last_clean":"2024-09-14T22:38:07.729968+0000","last_became_active":"2024-09-14T22:37:25.718519+0000","last_became_peered":"2024-09-14T22:37:25.718519+0000","last_unstale":"2024-09-14T22:38:07.729968+0000","last_undegraded":"2024-09-14T22:38:07.729968+0000","last_fullsized":"2024-09-14T22:38:07.729968+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:39:37.098899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092291499999999996,"stat_sum":{"num_bytes":30938989,"num_objects":50,"num_object_clones":14,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":21,"num_read":5129,"num_read_kb":32004,"num_write":569,"num_write_kb":13290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":7858104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"108'403","reported_seq":3305,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-14T22:38:01.422014+0000","last_change":"2024-09-14T22:37:59.734243+0000","last_active":"2024-09-14T22:38:01.422014+0000","last_peered":"2024-09-14T22:38:01.422014+0000","last_clean":"2024-09-14T22:38:01.422014+0000","last_became_active":"2024-09-14T22:37:59.732128+0000","last_became_peered":"2024-09-14T22:37:59.732128+0000","last_unstale":"2024-09-14T22:38:01.422014+0000","last_undegraded":"2024-09-14T22:38:01.422014+0000","last_fullsized":"2024-09-14T22:38:01.422014+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:15:08.486754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012502780000000001,"stat_sum":{"num_bytes":11058244,"num_objects":68,"num_object_clones":19,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":34,"num_read":3839,"num_read_kb":12469,"num_write":407,"num_write_kb":6714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":5018269,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"110'363","reported_seq":5104,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-14T22:38:06.723038+0000","last_change":"2024-09-14T22:36:17.705423+0000","last_active":"2024-09-14T22:38:06.723038+0000","last_peered":"2024-09-14T22:38:06.723038+0000","last_clean":"2024-09-14T22:38:06.723038+0000","last_became_active":"2024-09-14T22:36:15.700990+0000","last_became_peered":"2024-09-14T22:36:15.700990+0000","last_unstale":"2024-09-14T22:38:06.723038+0000","last_undegraded":"2024-09-14T22:38:06.723038+0000","last_fullsized":"2024-09-14T22:38:06.723038+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:55:28.951912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010520620000000001,"stat_sum":{"num_bytes":22178140,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":23,"num_read":5058,"num_read_kb":15226,"num_write":369,"num_write_kb":5409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"109'401","reported_seq":4427,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.736451+0000","last_change":"2024-09-14T22:38:07.735828+0000","last_active":"2024-09-14T22:38:07.736451+0000","last_peered":"2024-09-14T22:38:07.736451+0000","last_clean":"2024-09-14T22:38:07.736451+0000","last_became_active":"2024-09-14T22:38:07.734308+0000","last_became_peered":"2024-09-14T22:38:07.734308+0000","last_unstale":"2024-09-14T22:38:07.736451+0000","last_undegraded":"2024-09-14T22:38:07.736451+0000","last_fullsized":"2024-09-14T22:38:07.736451+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:46:56.903331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002063484,"stat_sum":{"num_bytes":36374590,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":14,"num_read":4669,"num_read_kb":35713,"num_write":566,"num_write_kb":27648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":20067293,"num_keys_recovered":21,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"111'310","reported_seq":3791,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.730687+0000","last_change":"2024-09-14T22:36:17.721195+0000","last_active":"2024-09-14T22:38:07.730687+0000","last_peered":"2024-09-14T22:38:07.730687+0000","last_clean":"2024-09-14T22:38:07.730687+0000","last_became_active":"2024-09-14T22:36:15.715869+0000","last_became_peered":"2024-09-14T22:36:15.715869+0000","last_unstale":"2024-09-14T22:38:07.730687+0000","last_undegraded":"2024-09-14T22:38:07.730687+0000","last_fullsized":"2024-09-14T22:38:07.730687+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:29:33.587279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001160672,"stat_sum":{"num_bytes":19652458,"num_objects":53,"num_object_clones":10,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":24,"num_read":3697,"num_read_kb":15655,"num_write":372,"num_write_kb":20137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"111'471","reported_seq":6163,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.732153+0000","last_change":"2024-09-14T22:36:17.703048+0000","last_active":"2024-09-14T22:38:07.732153+0000","last_peered":"2024-09-14T22:38:07.732153+0000","last_clean":"2024-09-14T22:38:07.732153+0000","last_became_active":"2024-09-14T22:36:15.725078+0000","last_became_peered":"2024-09-14T22:36:15.725078+0000","last_unstale":"2024-09-14T22:38:07.732153+0000","last_undegraded":"2024-09-14T22:38:07.732153+0000","last_fullsized":"2024-09-14T22:38:07.732153+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:50:22.638298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010500290000000001,"stat_sum":{"num_bytes":72644541,"num_objects":65,"num_object_clones":18,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":23,"num_read":6586,"num_read_kb":51137,"num_write":652,"num_write_kb":18892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"111'284","reported_seq":3336,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.737355+0000","last_change":"2024-09-14T22:36:18.009371+0000","last_active":"2024-09-14T22:38:07.737355+0000","last_peered":"2024-09-14T22:38:07.737355+0000","last_clean":"2024-09-14T22:38:07.737355+0000","last_became_active":"2024-09-14T22:36:15.717341+0000","last_became_peered":"2024-09-14T22:36:15.717341+0000","last_unstale":"2024-09-14T22:38:07.737355+0000","last_undegraded":"2024-09-14T22:38:07.737355+0000","last_fullsized":"2024-09-14T22:38:07.737355+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:53:54.367159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054571999999999997,"stat_sum":{"num_bytes":12714098,"num_objects":40,"num_object_clones":11,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":17,"num_read":3096,"num_read_kb":5369,"num_write":293,"num_write_kb":6253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"111'1003","reported_seq":8239,"reported_epoch":115,"state":"active+recovery_wait+remapped","last_fresh":"2024-09-14T22:38:07.793613+0000","last_change":"2024-09-14T22:38:07.793613+0000","last_active":"2024-09-14T22:38:07.793613+0000","last_peered":"2024-09-14T22:38:07.793613+0000","last_clean":"2024-09-14T22:38:04.200895+0000","last_became_active":"2024-09-14T22:38:07.790899+0000","last_became_peered":"2024-09-14T22:38:07.790899+0000","last_unstale":"2024-09-14T22:38:07.793613+0000","last_undegraded":"2024-09-14T22:38:07.793613+0000","last_fullsized":"2024-09-14T22:38:07.793613+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1003,"log_dups_size":0,"ondisk_log_size":1003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010804009999999999,"stat_sum":{"num_bytes":44119068,"num_objects":71,"num_object_clones":19,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":134,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":27,"num_read":10080,"num_read_kb":27226,"num_write":1483,"num_write_kb":11837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":8405044,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[{"shards":"6,7","objects":71}],"blocked_by":[],"up_primary":4,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":205,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.733471+0000","last_change":"2024-09-14T22:36:08.042860+0000","last_active":"2024-09-14T22:38:07.733471+0000","last_peered":"2024-09-14T22:38:07.733471+0000","last_clean":"2024-09-14T22:38:07.733471+0000","last_became_active":"2024-09-14T22:36:06.040961+0000","last_became_peered":"2024-09-14T22:36:06.040961+0000","last_unstale":"2024-09-14T22:38:07.733471+0000","last_undegraded":"2024-09-14T22:38:07.733471+0000","last_fullsized":"2024-09-14T22:38:07.733471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:03:48.547834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010780950000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"111'459","reported_seq":2533,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.729874+0000","last_change":"2024-09-14T22:36:17.720897+0000","last_active":"2024-09-14T22:38:07.729874+0000","last_peered":"2024-09-14T22:38:07.729874+0000","last_clean":"2024-09-14T22:38:07.729874+0000","last_became_active":"2024-09-14T22:36:15.700982+0000","last_became_peered":"2024-09-14T22:36:15.700982+0000","last_unstale":"2024-09-14T22:38:07.729874+0000","last_undegraded":"2024-09-14T22:38:07.729874+0000","last_fullsized":"2024-09-14T22:38:07.729874+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:39:25.124678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010495459999999999,"stat_sum":{"num_bytes":28962458,"num_objects":66,"num_object_clones":17,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":24,"num_read":3553,"num_read_kb":12439,"num_write":490,"num_write_kb":10188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":165,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.734671+0000","last_change":"2024-09-14T22:38:03.977889+0000","last_active":"2024-09-14T22:38:07.734671+0000","last_peered":"2024-09-14T22:38:07.734671+0000","last_clean":"2024-09-14T22:38:07.734671+0000","last_became_active":"2024-09-14T22:38:03.976899+0000","last_became_peered":"2024-09-14T22:38:03.976899+0000","last_unstale":"2024-09-14T22:38:07.734671+0000","last_undegraded":"2024-09-14T22:38:07.734671+0000","last_fullsized":"2024-09-14T22:38:07.734671+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:00:41.381700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"111'406","reported_seq":5172,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.738188+0000","last_change":"2024-09-14T22:36:21.722517+0000","last_active":"2024-09-14T22:38:07.738188+0000","last_peered":"2024-09-14T22:38:07.738188+0000","last_clean":"2024-09-14T22:38:07.738188+0000","last_became_active":"2024-09-14T22:36:21.722085+0000","last_became_peered":"2024-09-14T22:36:21.722085+0000","last_unstale":"2024-09-14T22:38:07.738188+0000","last_undegraded":"2024-09-14T22:38:07.738188+0000","last_fullsized":"2024-09-14T22:38:07.738188+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:05:24.340148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020774109999999999,"stat_sum":{"num_bytes":52168094,"num_objects":52,"num_object_clones":11,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":21,"num_read":5682,"num_read_kb":28617,"num_write":472,"num_write_kb":8463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":205,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.737893+0000","last_change":"2024-09-14T22:36:08.054606+0000","last_active":"2024-09-14T22:38:07.737893+0000","last_peered":"2024-09-14T22:38:07.737893+0000","last_clean":"2024-09-14T22:38:07.737893+0000","last_became_active":"2024-09-14T22:36:06.266908+0000","last_became_peered":"2024-09-14T22:36:06.266908+0000","last_unstale":"2024-09-14T22:38:07.737893+0000","last_undegraded":"2024-09-14T22:38:07.737893+0000","last_fullsized":"2024-09-14T22:38:07.737893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:36.738643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098249300000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"112'1318","reported_seq":4868,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T22:38:05.719085+0000","last_change":"2024-09-14T22:37:17.676125+0000","last_active":"2024-09-14T22:38:05.719085+0000","last_peered":"2024-09-14T22:38:05.719085+0000","last_clean":"2024-09-14T22:38:05.719085+0000","last_became_active":"2024-09-14T22:37:17.674322+0000","last_became_peered":"2024-09-14T22:37:17.674322+0000","last_unstale":"2024-09-14T22:38:05.719085+0000","last_undegraded":"2024-09-14T22:38:05.719085+0000","last_fullsized":"2024-09-14T22:38:05.719085+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1318,"log_dups_size":0,"ondisk_log_size":1318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:29:16.921603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012564830000000001,"stat_sum":{"num_bytes":27908088,"num_objects":62,"num_object_clones":15,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":26,"num_read":9453,"num_read_kb":23521,"num_write":2354,"num_write_kb":17866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":4264139,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":155,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.730090+0000","last_change":"2024-09-14T22:36:41.930961+0000","last_active":"2024-09-14T22:38:07.730090+0000","last_peered":"2024-09-14T22:38:07.730090+0000","last_clean":"2024-09-14T22:38:07.730090+0000","last_became_active":"2024-09-14T22:36:41.930619+0000","last_became_peered":"2024-09-14T22:36:41.930619+0000","last_unstale":"2024-09-14T22:38:07.730090+0000","last_undegraded":"2024-09-14T22:38:07.730090+0000","last_fullsized":"2024-09-14T22:38:07.730090+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:04:40.582999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"111'439","reported_seq":3922,"reported_epoch":115,"state":"active+undersized+remapped","last_fresh":"2024-09-14T22:38:07.837387+0000","last_change":"2024-09-14T22:38:07.837834+0000","last_active":"2024-09-14T22:38:07.837387+0000","last_peered":"2024-09-14T22:38:07.837387+0000","last_clean":"2024-09-14T22:38:04.091024+0000","last_became_active":"2024-09-14T22:38:06.757565+0000","last_became_peered":"2024-09-14T22:38:06.757565+0000","last_unstale":"2024-09-14T22:38:07.837387+0000","last_undegraded":"2024-09-14T22:38:07.837387+0000","last_fullsized":"2024-09-14T22:38:06.724796+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071220099999999998,"stat_sum":{"num_bytes":28456380,"num_objects":53,"num_object_clones":14,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":21,"num_read":4388,"num_read_kb":9216,"num_write":480,"num_write_kb":5683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":5797306,"num_keys_recovered":72,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5],"avail_no_missing":["5","2","3"],"object_location_counts":[{"shards":"5","objects":53}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"112'320","reported_seq":4760,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.728956+0000","last_change":"2024-09-14T22:36:17.711068+0000","last_active":"2024-09-14T22:38:07.728956+0000","last_peered":"2024-09-14T22:38:07.728956+0000","last_clean":"2024-09-14T22:38:07.728956+0000","last_became_active":"2024-09-14T22:36:15.706147+0000","last_became_peered":"2024-09-14T22:36:15.706147+0000","last_unstale":"2024-09-14T22:38:07.728956+0000","last_undegraded":"2024-09-14T22:38:07.728956+0000","last_fullsized":"2024-09-14T22:38:07.728956+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:37:40.830454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082625500000000002,"stat_sum":{"num_bytes":52612246,"num_objects":53,"num_object_clones":16,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":22,"num_read":4326,"num_read_kb":25402,"num_write":389,"num_write_kb":13872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":207,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.732075+0000","last_change":"2024-09-14T22:36:08.055938+0000","last_active":"2024-09-14T22:38:07.732075+0000","last_peered":"2024-09-14T22:38:07.732075+0000","last_clean":"2024-09-14T22:38:07.732075+0000","last_became_active":"2024-09-14T22:36:06.039600+0000","last_became_peered":"2024-09-14T22:36:06.039600+0000","last_unstale":"2024-09-14T22:38:07.732075+0000","last_undegraded":"2024-09-14T22:38:07.732075+0000","last_fullsized":"2024-09-14T22:38:07.732075+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:49:09.245563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021617870000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"111'818","reported_seq":8653,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T22:38:05.718388+0000","last_change":"2024-09-14T22:37:16.669227+0000","last_active":"2024-09-14T22:38:05.718388+0000","last_peered":"2024-09-14T22:38:05.718388+0000","last_clean":"2024-09-14T22:38:05.718388+0000","last_became_active":"2024-09-14T22:37:16.662230+0000","last_became_peered":"2024-09-14T22:37:16.662230+0000","last_unstale":"2024-09-14T22:38:05.718388+0000","last_undegraded":"2024-09-14T22:38:05.718388+0000","last_fullsized":"2024-09-14T22:38:05.718388+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":818,"log_dups_size":0,"ondisk_log_size":818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:38:34.241178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021500769999999998,"stat_sum":{"num_bytes":79368880,"num_objects":62,"num_object_clones":13,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":18,"num_read":9980,"num_read_kb":26598,"num_write":1149,"num_write_kb":14782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":4851034,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":137,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T22:38:05.724182+0000","last_change":"2024-09-14T22:38:05.724182+0000","last_active":"2024-09-14T22:38:05.724182+0000","last_peered":"2024-09-14T22:38:05.724182+0000","last_clean":"2024-09-14T22:38:05.724182+0000","last_became_active":"2024-09-14T22:38:05.723636+0000","last_became_peered":"2024-09-14T22:38:05.723636+0000","last_unstale":"2024-09-14T22:38:05.724182+0000","last_undegraded":"2024-09-14T22:38:05.724182+0000","last_fullsized":"2024-09-14T22:38:05.724182+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:14:09.687576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"110'338","reported_seq":5001,"reported_epoch":114,"state":"peering","last_fresh":"2024-09-14T22:38:07.150202+0000","last_change":"2024-09-14T22:38:07.730383+0000","last_active":"2024-09-14T22:38:07.730383+0000","last_peered":"2024-09-14T22:38:07.150202+0000","last_clean":"2024-09-14T22:38:03.971944+0000","last_became_active":"2024-09-14T22:38:06.752878+0000","last_became_peered":"2024-09-14T22:38:06.752878+0000","last_unstale":"2024-09-14T22:38:07.150202+0000","last_undegraded":"2024-09-14T22:38:07.150202+0000","last_fullsized":"2024-09-14T22:38:06.726031+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085549400000000005,"stat_sum":{"num_bytes":60476484,"num_objects":48,"num_object_clones":13,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":17,"num_read":4236,"num_read_kb":21069,"num_write":352,"num_write_kb":6997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":3679238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,4,5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":221,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.735420+0000","last_change":"2024-09-14T22:38:03.976161+0000","last_active":"2024-09-14T22:38:07.735420+0000","last_peered":"2024-09-14T22:38:07.735420+0000","last_clean":"2024-09-14T22:38:07.735420+0000","last_became_active":"2024-09-14T22:38:03.975240+0000","last_became_peered":"2024-09-14T22:38:03.975240+0000","last_unstale":"2024-09-14T22:38:07.735420+0000","last_undegraded":"2024-09-14T22:38:07.735420+0000","last_fullsized":"2024-09-14T22:38:07.735420+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:34:10.534483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010817520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"109'997","reported_seq":3989,"reported_epoch":115,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-14T22:38:07.901819+0000","last_change":"2024-09-14T22:38:07.841030+0000","last_active":"2024-09-14T22:38:07.901819+0000","last_peered":"2024-09-14T22:38:07.901819+0000","last_clean":"2024-09-14T22:38:03.978022+0000","last_became_active":"2024-09-14T22:38:06.768721+0000","last_became_peered":"2024-09-14T22:38:06.768721+0000","last_unstale":"2024-09-14T22:38:07.901819+0000","last_undegraded":"2024-09-14T22:38:07.901819+0000","last_fullsized":"2024-09-14T22:38:06.724937+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":997,"log_dups_size":0,"ondisk_log_size":997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56349086,"num_objects":66,"num_object_clones":14,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":58,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":24,"num_read":7202,"num_read_kb":15917,"num_write":1503,"num_write_kb":11829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":181921,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5],"avail_no_missing":["5","6"],"object_location_counts":[{"shards":"5","objects":8},{"shards":"5,6","objects":58}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":213,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.731808+0000","last_change":"2024-09-14T22:37:13.625072+0000","last_active":"2024-09-14T22:38:07.731808+0000","last_peered":"2024-09-14T22:38:07.731808+0000","last_clean":"2024-09-14T22:38:07.731808+0000","last_became_active":"2024-09-14T22:37:13.624669+0000","last_became_peered":"2024-09-14T22:37:13.624669+0000","last_unstale":"2024-09-14T22:38:07.731808+0000","last_undegraded":"2024-09-14T22:38:07.731808+0000","last_fullsized":"2024-09-14T22:38:07.731808+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:54:20.545098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011401880000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":357,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.735553+0000","last_change":"2024-09-14T22:36:00.402862+0000","last_active":"2024-09-14T22:38:07.735553+0000","last_peered":"2024-09-14T22:38:07.735553+0000","last_clean":"2024-09-14T22:38:07.735553+0000","last_became_active":"2024-09-14T22:36:00.400529+0000","last_became_peered":"2024-09-14T22:36:00.400529+0000","last_unstale":"2024-09-14T22:38:07.735553+0000","last_undegraded":"2024-09-14T22:38:07.735553+0000","last_fullsized":"2024-09-14T22:38:07.735553+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_clean_scrub_stamp":"2024-09-14T22:35:58.989025+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:07:12.951728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"114'541","reported_seq":6940,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-09-14T22:38:06.950103+0000","last_change":"2024-09-14T22:38:07.905458+0000","last_active":"2024-09-14T22:38:07.905458+0000","last_peered":"2024-09-14T22:38:06.950103+0000","last_clean":"2024-09-14T22:38:03.386180+0000","last_became_active":"2024-09-14T22:38:06.741191+0000","last_became_peered":"2024-09-14T22:38:06.741191+0000","last_unstale":"2024-09-14T22:38:06.950103+0000","last_undegraded":"2024-09-14T22:38:06.950103+0000","last_fullsized":"2024-09-14T22:38:06.722325+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001381511,"stat_sum":{"num_bytes":39365030,"num_objects":59,"num_object_clones":15,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":21,"num_read":7580,"num_read_kb":21053,"num_write":866,"num_write_kb":14446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":18690892,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"100'325","reported_seq":6915,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-14T22:38:00.759646+0000","last_change":"2024-09-14T22:37:58.738191+0000","last_active":"2024-09-14T22:38:00.759646+0000","last_peered":"2024-09-14T22:38:00.759646+0000","last_clean":"2024-09-14T22:38:00.759646+0000","last_became_active":"2024-09-14T22:37:58.736525+0000","last_became_peered":"2024-09-14T22:37:58.736525+0000","last_unstale":"2024-09-14T22:38:00.759646+0000","last_undegraded":"2024-09-14T22:38:00.759646+0000","last_fullsized":"2024-09-14T22:38:00.759646+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:00:41.790690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060272699999999995,"stat_sum":{"num_bytes":62873372,"num_objects":79,"num_object_clones":20,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":32,"num_read":6769,"num_read_kb":18458,"num_write":347,"num_write_kb":6853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":10366320,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"114'580","reported_seq":3648,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.733687+0000","last_change":"2024-09-14T22:36:17.776247+0000","last_active":"2024-09-14T22:38:07.733687+0000","last_peered":"2024-09-14T22:38:07.733687+0000","last_clean":"2024-09-14T22:38:07.733687+0000","last_became_active":"2024-09-14T22:36:15.718794+0000","last_became_peered":"2024-09-14T22:36:15.718794+0000","last_unstale":"2024-09-14T22:38:07.733687+0000","last_undegraded":"2024-09-14T22:38:07.733687+0000","last_fullsized":"2024-09-14T22:38:07.733687+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:52:03.394497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088662699999999997,"stat_sum":{"num_bytes":29108648,"num_objects":58,"num_object_clones":12,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":25,"num_read":5503,"num_read_kb":24737,"num_write":545,"num_write_kb":14408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"110'377","reported_seq":2220,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.726784+0000","last_change":"2024-09-14T22:36:17.733002+0000","last_active":"2024-09-14T22:38:07.726784+0000","last_peered":"2024-09-14T22:38:07.726784+0000","last_clean":"2024-09-14T22:38:07.726784+0000","last_became_active":"2024-09-14T22:36:15.702885+0000","last_became_peered":"2024-09-14T22:36:15.702885+0000","last_unstale":"2024-09-14T22:38:07.726784+0000","last_undegraded":"2024-09-14T22:38:07.726784+0000","last_fullsized":"2024-09-14T22:38:07.726784+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:28:04.078434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015180580000000001,"stat_sum":{"num_bytes":36984245,"num_objects":57,"num_object_clones":17,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":25,"num_read":2773,"num_read_kb":11024,"num_write":344,"num_write_kb":6734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"111'347","reported_seq":6456,"reported_epoch":114,"state":"peering","last_fresh":"2024-09-14T22:38:07.250183+0000","last_change":"2024-09-14T22:38:07.733053+0000","last_active":"2024-09-14T22:38:07.733053+0000","last_peered":"2024-09-14T22:38:07.250183+0000","last_clean":"2024-09-14T22:38:04.272478+0000","last_became_active":"2024-09-14T22:38:06.757572+0000","last_became_peered":"2024-09-14T22:38:06.757572+0000","last_unstale":"2024-09-14T22:38:07.250183+0000","last_undegraded":"2024-09-14T22:38:07.250183+0000","last_fullsized":"2024-09-14T22:38:06.725506+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082782099999999996,"stat_sum":{"num_bytes":50022508,"num_objects":57,"num_object_clones":17,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":24,"num_read":5737,"num_read_kb":27105,"num_write":432,"num_write_kb":17759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":23218688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"112'486","reported_seq":3686,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.737616+0000","last_change":"2024-09-14T22:36:18.009776+0000","last_active":"2024-09-14T22:38:07.737616+0000","last_peered":"2024-09-14T22:38:07.737616+0000","last_clean":"2024-09-14T22:38:07.737616+0000","last_became_active":"2024-09-14T22:36:15.723205+0000","last_became_peered":"2024-09-14T22:36:15.723205+0000","last_unstale":"2024-09-14T22:38:07.737616+0000","last_undegraded":"2024-09-14T22:38:07.737616+0000","last_fullsized":"2024-09-14T22:38:07.737616+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:21:43.223903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087783999999999996,"stat_sum":{"num_bytes":26214933,"num_objects":56,"num_object_clones":15,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":18,"num_read":4039,"num_read_kb":15967,"num_write":626,"num_write_kb":5807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"110'397","reported_seq":2154,"reported_epoch":115,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-14T22:38:07.912406+0000","last_change":"2024-09-14T22:38:07.773929+0000","last_active":"2024-09-14T22:38:07.912406+0000","last_peered":"2024-09-14T22:38:07.912406+0000","last_clean":"2024-09-14T22:38:04.976355+0000","last_became_active":"2024-09-14T22:38:07.762760+0000","last_became_peered":"2024-09-14T22:38:07.762760+0000","last_unstale":"2024-09-14T22:38:07.912406+0000","last_undegraded":"2024-09-14T22:38:07.912406+0000","last_fullsized":"2024-09-14T22:38:07.733682+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000776448,"stat_sum":{"num_bytes":28271111,"num_objects":46,"num_object_clones":10,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":24,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":19,"num_read":2287,"num_read_kb":8126,"num_write":482,"num_write_kb":3663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":3477302,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7],"avail_no_missing":["7","1"],"object_location_counts":[{"shards":"1,7","objects":24},{"shards":"7","objects":22}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"110'349","reported_seq":7862,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.738451+0000","last_change":"2024-09-14T22:36:18.010182+0000","last_active":"2024-09-14T22:38:07.738451+0000","last_peered":"2024-09-14T22:38:07.738451+0000","last_clean":"2024-09-14T22:38:07.738451+0000","last_became_active":"2024-09-14T22:36:15.719693+0000","last_became_peered":"2024-09-14T22:36:15.719693+0000","last_unstale":"2024-09-14T22:38:07.738451+0000","last_undegraded":"2024-09-14T22:38:07.738451+0000","last_fullsized":"2024-09-14T22:38:07.738451+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:07:18.549829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001045011,"stat_sum":{"num_bytes":25859668,"num_objects":54,"num_object_clones":12,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":16,"num_read":8144,"num_read_kb":23241,"num_write":432,"num_write_kb":9514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"111'373","reported_seq":3573,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.731097+0000","last_change":"2024-09-14T22:37:26.718624+0000","last_active":"2024-09-14T22:38:07.731097+0000","last_peered":"2024-09-14T22:38:07.731097+0000","last_clean":"2024-09-14T22:38:07.731097+0000","last_became_active":"2024-09-14T22:37:26.717088+0000","last_became_peered":"2024-09-14T22:37:26.717088+0000","last_unstale":"2024-09-14T22:38:07.731097+0000","last_undegraded":"2024-09-14T22:38:07.731097+0000","last_fullsized":"2024-09-14T22:38:07.731097+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:44:33.957303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094227299999999998,"stat_sum":{"num_bytes":40421858,"num_objects":59,"num_object_clones":17,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":24,"num_read":3286,"num_read_kb":29016,"num_write":472,"num_write_kb":14692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"111'566","reported_seq":2141,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.737145+0000","last_change":"2024-09-14T22:36:18.009347+0000","last_active":"2024-09-14T22:38:07.737145+0000","last_peered":"2024-09-14T22:38:07.737145+0000","last_clean":"2024-09-14T22:38:07.737145+0000","last_became_active":"2024-09-14T22:36:15.715569+0000","last_became_peered":"2024-09-14T22:36:15.715569+0000","last_unstale":"2024-09-14T22:38:07.737145+0000","last_undegraded":"2024-09-14T22:38:07.737145+0000","last_fullsized":"2024-09-14T22:38:07.737145+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:49:10.243383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054921600000000005,"stat_sum":{"num_bytes":31469256,"num_objects":46,"num_object_clones":9,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":21,"num_read":3273,"num_read_kb":11450,"num_write":793,"num_write_kb":10903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"112'396","reported_seq":7801,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-14T22:38:07.730054+0000","last_change":"2024-09-14T22:36:17.720448+0000","last_active":"2024-09-14T22:38:07.730054+0000","last_peered":"2024-09-14T22:38:07.730054+0000","last_clean":"2024-09-14T22:38:07.730054+0000","last_became_active":"2024-09-14T22:36:15.716696+0000","last_became_peered":"2024-09-14T22:36:15.716696+0000","last_unstale":"2024-09-14T22:38:07.730054+0000","last_undegraded":"2024-09-14T22:38:07.730054+0000","last_fullsized":"2024-09-14T22:38:07.730054+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:29:26.675216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086547600000000001,"stat_sum":{"num_bytes":62973071,"num_objects":59,"num_object_clones":11,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":22,"num_read":8846,"num_read_kb":49493,"num_write":448,"num_write_kb":13013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"111'432","reported_seq":6701,"reported_epoch":115,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-14T22:38:07.784296+0000","last_change":"2024-09-14T22:38:07.784773+0000","last_active":"2024-09-14T22:38:07.784296+0000","last_peered":"2024-09-14T22:38:07.784296+0000","last_clean":"2024-09-14T22:38:04.976079+0000","last_became_active":"2024-09-14T22:38:07.782564+0000","last_became_peered":"2024-09-14T22:38:07.782564+0000","last_unstale":"2024-09-14T22:38:07.784296+0000","last_undegraded":"2024-09-14T22:38:07.744011+0000","last_fullsized":"2024-09-14T22:38:07.740588+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010286189999999999,"stat_sum":{"num_bytes":55135358,"num_objects":67,"num_object_clones":16,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":26,"num_read":6702,"num_read_kb":34859,"num_write":509,"num_write_kb":15408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":7236792,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7],"avail_no_missing":["7","2"],"object_location_counts":[{"shards":"2,7","objects":67}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1366651543,"num_objects":1837,"num_object_clones":465,"num_object_copies":3674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":131,"num_objects_misplaced":216,"num_objects_unfound":0,"num_objects_dirty":1837,"num_whiteouts":734,"num_read":187536,"num_read_kb":762224,"num_write":21671,"num_write_kb":371137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2650,"num_bytes_recovered":141833077,"num_keys_recovered":285,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57712640,"data_stored":532595149,"data_compressed":15508251,"data_compressed_allocated":46325760,"data_compressed_original":525836315,"omap_allocated":4238,"internal_metadata":0},"log_size":16503,"ondisk_log_size":16503,"up":64,"acting":60,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542173,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64008,"kb_used_data":10612,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94307832,"statfs":{"total":96636764160,"available":96571219968,"internally_reserved":0,"allocated":10866688,"data_stored":80695705,"data_compressed":2679345,"data_compressed_allocated":7299072,"data_compressed_original":78695629,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542174,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46212,"kb_used_data":6696,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94325628,"statfs":{"total":96636764160,"available":96589443072,"internally_reserved":0,"allocated":6856704,"data_stored":59226182,"data_compressed":1430526,"data_compressed_allocated":4620288,"data_compressed_original":58106961,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542174,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57724,"kb_used_data":8476,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94314116,"statfs":{"total":96636764160,"available":96577654784,"internally_reserved":0,"allocated":8679424,"data_stored":62262855,"data_compressed":1952669,"data_compressed_allocated":5775360,"data_compressed_original":60700882,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542173,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59848,"kb_used_data":6444,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94311992,"statfs":{"total":96636764160,"available":96575479808,"internally_reserved":0,"allocated":6598656,"data_stored":53579741,"data_compressed":1525905,"data_compressed_allocated":4202496,"data_compressed_original":52353202,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542174,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64580,"kb_used_data":7024,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94307260,"statfs":{"total":96636764160,"available":96570634240,"internally_reserved":0,"allocated":7192576,"data_stored":55404111,"data_compressed":1661968,"data_compressed_allocated":4685824,"data_compressed_original":54081269,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":66,"seq":283467841548,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46372,"kb_used_data":9552,"kb_used_omap":6,"kb_used_meta":36793,"kb_avail":94325468,"statfs":{"total":96636764160,"available":96589279232,"internally_reserved":0,"allocated":9781248,"data_stored":80505484,"data_compressed":2031111,"data_compressed_allocated":7008256,"data_compressed_original":79013785,"omap_allocated":7025,"internal_metadata":37676175},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61316,"kb_used_data":7920,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94310524,"statfs":{"total":96636764160,"available":96573976576,"internally_reserved":0,"allocated":8110080,"data_stored":60353117,"data_compressed":1818437,"data_compressed_allocated":5406720,"data_compressed_original":58920539,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542173,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40684,"kb_used_data":9496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331156,"statfs":{"total":96636764160,"available":96595103744,"internally_reserved":0,"allocated":9723904,"data_stored":81304437,"data_compressed":2073340,"data_compressed_allocated":6737920,"data_compressed_original":79712400,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8310784,"data_stored":80079672,"data_compressed":2067531,"data_compressed_allocated":6701056,"data_compressed_original":79138960,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6762496,"data_stored":59725024,"data_compressed":1818437,"data_compressed_allocated":5406720,"data_compressed_original":58920539,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8437760,"data_stored":79877410,"data_compressed":2031111,"data_compressed_allocated":7008256,"data_compressed_original":79013785,"omap_allocated":4238,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6512640,"data_stored":59227200,"data_compressed":1858117,"data_compressed_allocated":5214208,"data_compressed_original":58455797,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5439488,"data_stored":54007001,"data_compressed":1676324,"data_compressed_allocated":4337664,"data_compressed_original":53377202,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7307264,"data_stored":61622278,"data_compressed":1952669,"data_compressed_allocated":5775360,"data_compressed_original":60700882,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5500928,"data_stored":58591785,"data_compressed":1430526,"data_compressed_allocated":4620288,"data_compressed_original":58106961,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":9441280,"data_stored":79464779,"data_compressed":2673536,"data_compressed_allocated":7262208,"data_compressed_original":78122189,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:38:10.363 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:10.358+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:10.447 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:10.447 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:38:10.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:38:10.447 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:38:10.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.447 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:38:10.447 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:38:10.447 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 130.436230107, 2024-09-14T22:38:10.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.398029152, 2024-09-14T22:38:10.447 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:10.447 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:10.448 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:10.448 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:10.448 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:38:10.448 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:38:10.448 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 1 2024-09-14T22:38:10.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.448 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:10.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.448 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:10.448 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:38:10.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.448 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:10.449 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:38:10.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.449 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:10.449 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006183+0000", 2024-09-14T22:38:10.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9059999999999998e-06 2024-09-14T22:38:10.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.449 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:10.449 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006198+0000", 2024-09-14T22:38:10.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.523e-05 2024-09-14T22:38:10.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.450 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:10.450 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006219+0000", 2024-09-14T22:38:10.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1342999999999999e-05 2024-09-14T22:38:10.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.450 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.450 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:10.450 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006240+0000", 2024-09-14T22:38:10.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1186999999999999e-05 2024-09-14T22:38:10.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.450 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.450 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:10.450 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006363+0000", 2024-09-14T22:38:10.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012258200000000001 2024-09-14T22:38:10.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:38:10.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006392+0000", 2024-09-14T22:38:10.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9607e-05 2024-09-14T22:38:10.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:10.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401084+0000", 2024-09-14T22:38:10.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.39469142699999998 2024-09-14T22:38:10.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:10.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401394+0000", 2024-09-14T22:38:10.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030987799999999998 2024-09-14T22:38:10.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:38:10.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401860+0000", 2024-09-14T22:38:10.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00046654299999999998 2024-09-14T22:38:10.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:38:10.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.403386+0000", 2024-09-14T22:38:10.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001525322 2024-09-14T22:38:10.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:10.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404042+0000", 2024-09-14T22:38:10.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065676099999999996 2024-09-14T22:38:10.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:38:10.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404136+0000", 2024-09-14T22:38:10.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4032999999999999e-05 2024-09-14T22:38:10.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:10.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404209+0000", 2024-09-14T22:38:10.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2333e-05 2024-09-14T22:38:10.454 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.454 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:10.454 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.454 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.454 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.454 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104182 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14352384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:10.454 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:38:10.454 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.953376120999998, 2024-09-14T22:38:10.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071642496999999999, 2024-09-14T22:38:10.454 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:10.454 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:10.454 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:10.454 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:10.455 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:10.455 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:10.455 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104182 2024-09-14T22:38:10.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.455 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:10.455 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.455 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:10.455 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:38:10.455 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.455 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.455 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:10.455 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:38:10.455 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:10.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489035+0000", 2024-09-14T22:38:10.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3430000000000001e-06 2024-09-14T22:38:10.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:10.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489041+0000", 2024-09-14T22:38:10.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8139999999999997e-06 2024-09-14T22:38:10.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:10.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489059+0000", 2024-09-14T22:38:10.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8193e-05 2024-09-14T22:38:10.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:10.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489073+0000", 2024-09-14T22:38:10.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:38:10.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:10.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560434+0000", 2024-09-14T22:38:10.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071361463999999999 2024-09-14T22:38:10.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:10.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560545+0000", 2024-09-14T22:38:10.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110263 2024-09-14T22:38:10.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:10.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560676+0000", 2024-09-14T22:38:10.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013138000000000001 2024-09-14T22:38:10.458 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.458 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:10.458 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.458 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104183 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14483456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:10.459 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:38:10.459 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.953252802000002, 2024-09-14T22:38:10.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071781943000000001, 2024-09-14T22:38:10.459 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:10.459 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:10.459 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:10.459 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:10.459 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:10.459 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:10.459 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104183 2024-09-14T22:38:10.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.459 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:10.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.459 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:10.460 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:38:10.460 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.460 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.460 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:10.460 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:38:10.460 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.460 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.460 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:10.460 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489159+0000", 2024-09-14T22:38:10.460 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5629999999999999e-06 2024-09-14T22:38:10.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.460 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:10.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489165+0000", 2024-09-14T22:38:10.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7070000000000003e-06 2024-09-14T22:38:10.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:10.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489186+0000", 2024-09-14T22:38:10.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.088e-05 2024-09-14T22:38:10.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:10.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489201+0000", 2024-09-14T22:38:10.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5203e-05 2024-09-14T22:38:10.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:10.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560713+0000", 2024-09-14T22:38:10.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071511447000000006 2024-09-14T22:38:10.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.465 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:10.465 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560819+0000", 2024-09-14T22:38:10.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106579 2024-09-14T22:38:10.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.465 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:10.465 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560939+0000", 2024-09-14T22:38:10.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119564 2024-09-14T22:38:10.465 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.465 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:10.465 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.465 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104184 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:10.466 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:38:10.466 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.953188359000002, 2024-09-14T22:38:10.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071970107000000005, 2024-09-14T22:38:10.466 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:10.466 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:10.466 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:10.466 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:10.466 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:10.466 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:10.466 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104184 2024-09-14T22:38:10.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.466 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:10.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:10.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:38:10.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:10.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:38:10.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:10.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489223+0000", 2024-09-14T22:38:10.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4470000000000001e-06 2024-09-14T22:38:10.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:10.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489229+0000", 2024-09-14T22:38:10.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8200000000000002e-06 2024-09-14T22:38:10.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:10.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489247+0000", 2024-09-14T22:38:10.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8150000000000001e-05 2024-09-14T22:38:10.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:10.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489260+0000", 2024-09-14T22:38:10.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3570000000000001e-05 2024-09-14T22:38:10.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:10.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560979+0000", 2024-09-14T22:38:10.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071718831999999996 2024-09-14T22:38:10.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:10.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561075+0000", 2024-09-14T22:38:10.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5848999999999997e-05 2024-09-14T22:38:10.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:10.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561191+0000", 2024-09-14T22:38:10.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116439 2024-09-14T22:38:10.470 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.470 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:10.470 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.470 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.470 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.470 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104185 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14745600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:10.470 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:38:10.470 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.953128776, 2024-09-14T22:38:10.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072173750999999994, 2024-09-14T22:38:10.470 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:10.470 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:10.470 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:10.471 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:10.471 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:10.471 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:10.471 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104185 2024-09-14T22:38:10.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.471 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:10.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:10.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:38:10.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:10.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:38:10.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:10.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489282+0000", 2024-09-14T22:38:10.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2500000000000001e-06 2024-09-14T22:38:10.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:10.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489287+0000", 2024-09-14T22:38:10.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1359999999999996e-06 2024-09-14T22:38:10.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:10.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489304+0000", 2024-09-14T22:38:10.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7033000000000001e-05 2024-09-14T22:38:10.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:10.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489318+0000", 2024-09-14T22:38:10.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3934000000000001e-05 2024-09-14T22:38:10.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:10.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561232+0000", 2024-09-14T22:38:10.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071913759999999993 2024-09-14T22:38:10.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:10.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561340+0000", 2024-09-14T22:38:10.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107921 2024-09-14T22:38:10.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:10.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561455+0000", 2024-09-14T22:38:10.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011471700000000001 2024-09-14T22:38:10.474 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.474 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:10.475 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.475 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104186 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14876672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:10.475 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:38:10.475 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.953072867000003, 2024-09-14T22:38:10.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072393374999999996, 2024-09-14T22:38:10.475 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:10.475 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:10.475 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:10.475 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:10.475 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:10.475 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:10.476 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104186 2024-09-14T22:38:10.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.476 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:10.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:10.476 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:38:10.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:10.476 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:38:10.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:10.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489338+0000", 2024-09-14T22:38:10.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.48e-06 2024-09-14T22:38:10.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:10.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489366+0000", 2024-09-14T22:38:10.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7307000000000001e-05 2024-09-14T22:38:10.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:10.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489385+0000", 2024-09-14T22:38:10.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9283000000000002e-05 2024-09-14T22:38:10.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:10.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489399+0000", 2024-09-14T22:38:10.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.452e-05 2024-09-14T22:38:10.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:10.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561494+0000", 2024-09-14T22:38:10.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072094885999999997 2024-09-14T22:38:10.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:10.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561604+0000", 2024-09-14T22:38:10.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010925900000000001 2024-09-14T22:38:10.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:10.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561730+0000", 2024-09-14T22:38:10.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012663999999999999 2024-09-14T22:38:10.479 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.479 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:10.479 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.480 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104187 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15007744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:10.480 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:38:10.480 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.952989498000001, 2024-09-14T22:38:10.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072599143000000005, 2024-09-14T22:38:10.480 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:10.480 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:10.480 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:10.480 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:10.480 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:10.481 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:10.481 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104187 2024-09-14T22:38:10.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.481 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:10.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:10.481 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:38:10.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:10.481 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:38:10.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:10.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489422+0000", 2024-09-14T22:38:10.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5340000000000001e-06 2024-09-14T22:38:10.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:10.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489428+0000", 2024-09-14T22:38:10.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1430000000000003e-06 2024-09-14T22:38:10.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:10.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489445+0000", 2024-09-14T22:38:10.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7042999999999999e-05 2024-09-14T22:38:10.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:10.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489458+0000", 2024-09-14T22:38:10.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3474e-05 2024-09-14T22:38:10.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:10.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561771+0000", 2024-09-14T22:38:10.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072312600000000005 2024-09-14T22:38:10.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.484 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:10.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561885+0000", 2024-09-14T22:38:10.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011411799999999999 2024-09-14T22:38:10.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.484 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:10.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562019+0000", 2024-09-14T22:38:10.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013423100000000001 2024-09-14T22:38:10.484 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.484 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:10.485 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.485 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104188 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15138816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:10.485 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:38:10.485 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.952929574000002, 2024-09-14T22:38:10.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072824624000000004, 2024-09-14T22:38:10.485 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:10.485 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:10.485 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:10.485 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:10.485 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:10.485 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:10.486 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104188 2024-09-14T22:38:10.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.486 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:10.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.486 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:10.486 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:38:10.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.486 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:10.486 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:38:10.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:10.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489481+0000", 2024-09-14T22:38:10.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.22e-06 2024-09-14T22:38:10.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:10.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489487+0000", 2024-09-14T22:38:10.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6930000000000004e-06 2024-09-14T22:38:10.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:10.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489508+0000", 2024-09-14T22:38:10.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1135999999999999e-05 2024-09-14T22:38:10.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.488 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:10.488 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489522+0000", 2024-09-14T22:38:10.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3817e-05 2024-09-14T22:38:10.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.488 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:10.488 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562066+0000", 2024-09-14T22:38:10.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072543813999999998 2024-09-14T22:38:10.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.488 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:10.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562177+0000", 2024-09-14T22:38:10.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000111264 2024-09-14T22:38:10.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:10.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562305+0000", 2024-09-14T22:38:10.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012768 2024-09-14T22:38:10.489 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.489 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:10.489 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.489 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104189 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15269888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:10.489 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:38:10.490 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.952869124999999, 2024-09-14T22:38:10.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073061300999999995, 2024-09-14T22:38:10.490 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:10.490 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:10.490 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:10.490 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:10.490 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:10.490 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:10.490 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104189 2024-09-14T22:38:10.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.490 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:10.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:10.490 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:38:10.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:10.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:38:10.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:10.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489542+0000", 2024-09-14T22:38:10.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1659999999999999e-06 2024-09-14T22:38:10.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.492 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:10.492 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489561+0000", 2024-09-14T22:38:10.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.908e-05 2024-09-14T22:38:10.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.492 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:10.492 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489579+0000", 2024-09-14T22:38:10.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8536999999999999e-05 2024-09-14T22:38:10.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.492 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:10.492 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489593+0000", 2024-09-14T22:38:10.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3977e-05 2024-09-14T22:38:10.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.493 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:10.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562347+0000", 2024-09-14T22:38:10.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072754092000000006 2024-09-14T22:38:10.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.493 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:10.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562458+0000", 2024-09-14T22:38:10.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110722 2024-09-14T22:38:10.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.493 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:10.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562602+0000", 2024-09-14T22:38:10.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014372699999999999 2024-09-14T22:38:10.494 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.494 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:10.494 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.494 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104190 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:10.494 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:38:10.494 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.952796829, 2024-09-14T22:38:10.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.07326133, 2024-09-14T22:38:10.494 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:10.494 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:10.495 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:10.495 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:10.495 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:10.495 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:10.495 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104190 2024-09-14T22:38:10.495 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.495 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:10.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.495 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:10.495 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:38:10.495 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.495 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.496 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:10.496 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:38:10.496 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.496 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:10.496 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489614+0000", 2024-09-14T22:38:10.496 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1999999999999999e-06 2024-09-14T22:38:10.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.497 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:10.497 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489620+0000", 2024-09-14T22:38:10.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.524e-06 2024-09-14T22:38:10.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.497 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:10.497 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489637+0000", 2024-09-14T22:38:10.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7825999999999998e-05 2024-09-14T22:38:10.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.497 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:10.497 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489671+0000", 2024-09-14T22:38:10.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3726000000000002e-05 2024-09-14T22:38:10.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:10.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562644+0000", 2024-09-14T22:38:10.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072973089000000005 2024-09-14T22:38:10.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:10.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562762+0000", 2024-09-14T22:38:10.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011743600000000001 2024-09-14T22:38:10.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:10.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562874+0000", 2024-09-14T22:38:10.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011252900000000001 2024-09-14T22:38:10.499 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.499 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:10.499 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.499 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104191 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15532032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:10.499 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:38:10.500 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.952688406, 2024-09-14T22:38:10.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073311166999999997, 2024-09-14T22:38:10.500 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:10.500 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:10.500 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:10.500 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:10.500 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:10.500 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:10.500 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104191 2024-09-14T22:38:10.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.501 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:10.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.501 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:10.501 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:38:10.501 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.501 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.501 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:10.501 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:38:10.501 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.501 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.501 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:10.502 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489723+0000", 2024-09-14T22:38:10.502 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6330000000000001e-06 2024-09-14T22:38:10.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.502 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:10.502 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489730+0000", 2024-09-14T22:38:10.502 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2300000000000002e-06 2024-09-14T22:38:10.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.502 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:10.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489751+0000", 2024-09-14T22:38:10.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0849999999999999e-05 2024-09-14T22:38:10.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.503 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:10.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489766+0000", 2024-09-14T22:38:10.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.524e-05 2024-09-14T22:38:10.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.503 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:10.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562896+0000", 2024-09-14T22:38:10.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073129662999999998 2024-09-14T22:38:10.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:10.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562960+0000", 2024-09-14T22:38:10.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3689999999999995e-05 2024-09-14T22:38:10.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:10.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563033+0000", 2024-09-14T22:38:10.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2861000000000006e-05 2024-09-14T22:38:10.504 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.505 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:10.505 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.505 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104192 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:10.505 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:38:10.505 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.952592600999999, 2024-09-14T22:38:10.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073463632000000001, 2024-09-14T22:38:10.505 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:10.505 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:10.505 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:10.505 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:10.505 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:10.506 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:10.506 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104192 2024-09-14T22:38:10.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.506 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:10.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.506 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:10.506 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:38:10.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.506 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:10.506 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:38:10.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.507 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:10.507 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489819+0000", 2024-09-14T22:38:10.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.424e-06 2024-09-14T22:38:10.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.507 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:10.507 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489825+0000", 2024-09-14T22:38:10.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.02e-06 2024-09-14T22:38:10.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.507 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:10.507 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489844+0000", 2024-09-14T22:38:10.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9760000000000001e-05 2024-09-14T22:38:10.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:10.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489860+0000", 2024-09-14T22:38:10.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5563e-05 2024-09-14T22:38:10.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:10.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563070+0000", 2024-09-14T22:38:10.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073209724000000004 2024-09-14T22:38:10.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:10.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563175+0000", 2024-09-14T22:38:10.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105183 2024-09-14T22:38:10.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:10.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563281+0000", 2024-09-14T22:38:10.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105958 2024-09-14T22:38:10.509 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.509 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:10.510 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.510 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104193 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15794176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:10.510 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:38:10.510 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.952530434000003, 2024-09-14T22:38:10.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073600040000000005, 2024-09-14T22:38:10.510 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:10.510 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:10.510 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:10.510 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:10.510 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:10.510 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:10.510 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104193 2024-09-14T22:38:10.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.511 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:10.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.511 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:10.511 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:38:10.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.511 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:10.511 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:38:10.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.511 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:10.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489881+0000", 2024-09-14T22:38:10.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5460000000000001e-06 2024-09-14T22:38:10.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:10.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489886+0000", 2024-09-14T22:38:10.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1739999999999999e-06 2024-09-14T22:38:10.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:10.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489912+0000", 2024-09-14T22:38:10.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6461999999999998e-05 2024-09-14T22:38:10.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:10.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489927+0000", 2024-09-14T22:38:10.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:38:10.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:10.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563310+0000", 2024-09-14T22:38:10.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073382977000000002 2024-09-14T22:38:10.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:10.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563385+0000", 2024-09-14T22:38:10.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5218999999999999e-05 2024-09-14T22:38:10.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:10.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563479+0000", 2024-09-14T22:38:10.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4622000000000003e-05 2024-09-14T22:38:10.514 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.514 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:10.514 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.514 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104194 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15925248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:10.514 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:38:10.514 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.952463842, 2024-09-14T22:38:10.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073745434999999998, 2024-09-14T22:38:10.514 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:10.515 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:10.515 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:10.515 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:10.515 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:10.515 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:10.515 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104194 2024-09-14T22:38:10.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.515 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:10.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:10.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:38:10.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.516 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:10.516 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:38:10.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.516 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:10.516 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489947+0000", 2024-09-14T22:38:10.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.327e-06 2024-09-14T22:38:10.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.516 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:10.516 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489953+0000", 2024-09-14T22:38:10.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9329999999999998e-06 2024-09-14T22:38:10.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.517 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:10.517 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489971+0000", 2024-09-14T22:38:10.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7994000000000001e-05 2024-09-14T22:38:10.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.517 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:10.517 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489985+0000", 2024-09-14T22:38:10.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4226000000000001e-05 2024-09-14T22:38:10.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.517 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:10.517 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563510+0000", 2024-09-14T22:38:10.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073524667000000002 2024-09-14T22:38:10.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:10.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563597+0000", 2024-09-14T22:38:10.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6550000000000003e-05 2024-09-14T22:38:10.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:10.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563691+0000", 2024-09-14T22:38:10.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4738e-05 2024-09-14T22:38:10.518 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.518 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:10.518 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.519 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104195 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:10.519 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:38:10.519 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.952405644999999, 2024-09-14T22:38:10.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073876598000000002, 2024-09-14T22:38:10.519 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:10.519 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:10.519 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:10.519 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:10.519 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:10.519 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:10.519 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104195 2024-09-14T22:38:10.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.519 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:10.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:10.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:38:10.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:10.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:38:10.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:10.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490006+0000", 2024-09-14T22:38:10.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4699999999999999e-06 2024-09-14T22:38:10.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.521 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:10.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490011+0000", 2024-09-14T22:38:10.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.366e-06 2024-09-14T22:38:10.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.521 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:10.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490046+0000", 2024-09-14T22:38:10.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5496e-05 2024-09-14T22:38:10.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.521 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:10.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490061+0000", 2024-09-14T22:38:10.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5054e-05 2024-09-14T22:38:10.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:10.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563719+0000", 2024-09-14T22:38:10.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073657412000000005 2024-09-14T22:38:10.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:10.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563792+0000", 2024-09-14T22:38:10.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3319999999999996e-05 2024-09-14T22:38:10.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:10.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563881+0000", 2024-09-14T22:38:10.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8480000000000007e-05 2024-09-14T22:38:10.523 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.523 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:10.523 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.523 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104196 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16187392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:10.523 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:38:10.523 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.952328496, 2024-09-14T22:38:10.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073997496999999995, 2024-09-14T22:38:10.523 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:10.523 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:10.523 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:10.523 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:10.523 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:10.524 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:10.524 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104196 2024-09-14T22:38:10.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.524 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:10.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:10.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:38:10.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:10.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:38:10.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:10.525 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490083+0000", 2024-09-14T22:38:10.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.347e-06 2024-09-14T22:38:10.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:10.525 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490089+0000", 2024-09-14T22:38:10.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9630000000000003e-06 2024-09-14T22:38:10.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:10.526 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490107+0000", 2024-09-14T22:38:10.526 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8859999999999999e-05 2024-09-14T22:38:10.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.526 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:10.526 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490122+0000", 2024-09-14T22:38:10.526 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.432e-05 2024-09-14T22:38:10.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.526 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:10.526 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563910+0000", 2024-09-14T22:38:10.526 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073788753999999998 2024-09-14T22:38:10.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.526 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:10.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563988+0000", 2024-09-14T22:38:10.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7862999999999996e-05 2024-09-14T22:38:10.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:10.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564079+0000", 2024-09-14T22:38:10.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0389999999999993e-05 2024-09-14T22:38:10.527 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.527 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:10.528 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.528 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104197 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16318464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:10.528 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:38:10.528 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.952236867000003, 2024-09-14T22:38:10.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074103547000000006, 2024-09-14T22:38:10.528 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:10.528 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:10.528 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:10.528 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:10.528 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:10.528 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:10.529 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104197 2024-09-14T22:38:10.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.529 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:10.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:10.529 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:38:10.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:10.529 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:38:10.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:10.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490174+0000", 2024-09-14T22:38:10.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.37e-06 2024-09-14T22:38:10.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:10.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490197+0000", 2024-09-14T22:38:10.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3017e-05 2024-09-14T22:38:10.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:10.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490219+0000", 2024-09-14T22:38:10.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.181e-05 2024-09-14T22:38:10.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:10.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490240+0000", 2024-09-14T22:38:10.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0786000000000001e-05 2024-09-14T22:38:10.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:10.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564109+0000", 2024-09-14T22:38:10.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073869031000000002 2024-09-14T22:38:10.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:10.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564186+0000", 2024-09-14T22:38:10.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6823999999999995e-05 2024-09-14T22:38:10.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.532 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:10.532 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564276+0000", 2024-09-14T22:38:10.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0709000000000002e-05 2024-09-14T22:38:10.532 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.532 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:10.532 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.532 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104198 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16449536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:10.532 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:38:10.532 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.952144631000003, 2024-09-14T22:38:10.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074167693000000007, 2024-09-14T22:38:10.532 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:10.533 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:10.533 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:10.533 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:10.533 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:10.533 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:10.533 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104198 2024-09-14T22:38:10.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.533 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:10.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:10.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:38:10.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.534 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:10.534 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:38:10.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.534 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:10.534 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490266+0000", 2024-09-14T22:38:10.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3599999999999999e-06 2024-09-14T22:38:10.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.534 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:10.534 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490272+0000", 2024-09-14T22:38:10.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9599999999999997e-06 2024-09-14T22:38:10.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:10.535 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490291+0000", 2024-09-14T22:38:10.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8485999999999999e-05 2024-09-14T22:38:10.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:10.535 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490305+0000", 2024-09-14T22:38:10.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4327e-05 2024-09-14T22:38:10.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:10.535 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564306+0000", 2024-09-14T22:38:10.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074000745000000007 2024-09-14T22:38:10.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:10.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564367+0000", 2024-09-14T22:38:10.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1501999999999995e-05 2024-09-14T22:38:10.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:10.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564433+0000", 2024-09-14T22:38:10.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5313000000000003e-05 2024-09-14T22:38:10.536 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.536 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:10.536 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.537 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104199 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16580608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:10.537 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:38:10.537 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.952063078000002, 2024-09-14T22:38:10.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074231631000000006, 2024-09-14T22:38:10.537 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:10.537 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:10.537 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:10.537 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:10.537 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:10.537 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:10.537 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104199 2024-09-14T22:38:10.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.537 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:10.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.538 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:10.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:38:10.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.538 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:10.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:38:10.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.538 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:10.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490349+0000", 2024-09-14T22:38:10.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9429999999999999e-06 2024-09-14T22:38:10.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:10.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490355+0000", 2024-09-14T22:38:10.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1670000000000004e-06 2024-09-14T22:38:10.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:10.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490374+0000", 2024-09-14T22:38:10.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9613e-05 2024-09-14T22:38:10.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:10.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490405+0000", 2024-09-14T22:38:10.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0936000000000003e-05 2024-09-14T22:38:10.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:10.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564451+0000", 2024-09-14T22:38:10.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074045266999999998 2024-09-14T22:38:10.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:10.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564494+0000", 2024-09-14T22:38:10.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3133000000000002e-05 2024-09-14T22:38:10.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:10.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564578+0000", 2024-09-14T22:38:10.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4572000000000003e-05 2024-09-14T22:38:10.541 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.541 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:10.541 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.541 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104200 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:10.541 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:38:10.541 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.950267850000003, 2024-09-14T22:38:10.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072582435000000001, 2024-09-14T22:38:10.541 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:10.541 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:10.541 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:10.542 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:10.542 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:10.542 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:10.542 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104200 2024-09-14T22:38:10.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.542 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:10.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:10.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:38:10.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:10.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:38:10.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:10.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:10.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492144+0000", 2024-09-14T22:38:10.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3070000000000001e-06 2024-09-14T22:38:10.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:10.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492155+0000", 2024-09-14T22:38:10.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0573e-05 2024-09-14T22:38:10.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:10.544 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492185+0000", 2024-09-14T22:38:10.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.008e-05 2024-09-14T22:38:10.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.544 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:10.544 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492205+0000", 2024-09-14T22:38:10.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9882999999999999e-05 2024-09-14T22:38:10.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.544 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:10.544 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564609+0000", 2024-09-14T22:38:10.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072404181999999997 2024-09-14T22:38:10.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:10.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564667+0000", 2024-09-14T22:38:10.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8102e-05 2024-09-14T22:38:10.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:10.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:10.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:10.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564724+0000", 2024-09-14T22:38:10.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7308000000000001e-05 2024-09-14T22:38:10.545 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.545 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:10.545 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.545 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:10.545 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:10.545 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:10.547 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:38:10.547 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:10.542+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:10.648 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:10.642+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:10.681 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:10.681 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:10.682 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:38:10.682 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:10.697 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:38:10.770 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T22:38:10.786 INFO:teuthology.orchestra.run.smithi001.stdout:5745 punch from 0x10dcb67 to 0x10e842e, (0xb8c7 bytes) 2024-09-14T22:38:10.786 INFO:teuthology.orchestra.run.smithi001.stdout:5746 read 0x94b213a thru 0x94c1910 (0xf7d7 bytes) 2024-09-14T22:38:10.786 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:10.786 INFO:teuthology.orchestra.run.smithi001.stdout:5748 writesame 0x465b353 thru 0x46690a2 data_size 0x8(0xdd50 bytes) 2024-09-14T22:38:10.786 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:10.782+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:10.799 INFO:teuthology.orchestra.run.smithi001.stdout:5749 read 0x5c7697b thru 0x5c76b9b (0x221 bytes) 2024-09-14T22:38:10.818 INFO:teuthology.orchestra.run.smithi001.stdout:5752 write 0xd6ee6f0 thru 0xd6f152a (0x2e3b bytes) 2024-09-14T22:38:10.828 INFO:teuthology.orchestra.run.smithi001.stdout:5755 write 0x845ab23 thru 0x84692e2 (0xe7c0 bytes) 2024-09-14T22:38:10.838 INFO:teuthology.orchestra.run.smithi001.stdout:5756 write 0x2435e27 thru 0x243788c (0x1a66 bytes) 2024-09-14T22:38:10.844 INFO:teuthology.orchestra.run.smithi001.stdout:5757 write 0x39c9895 thru 0x39cdbe9 (0x4355 bytes) 2024-09-14T22:38:10.847 INFO:teuthology.orchestra.run.smithi001.stdout:5759 read 0x7990faa thru 0x799f35c (0xe3b3 bytes) 2024-09-14T22:38:10.847 INFO:teuthology.orchestra.run.smithi001.stdout:5761 read 0x7e4bae3 thru 0x7e5a516 (0xea34 bytes) 2024-09-14T22:38:10.847 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:10.847 INFO:teuthology.orchestra.run.smithi001.stdout:5764 trunc from 0xd6f152b to 0xd6be7cd 2024-09-14T22:38:10.850 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:10.846+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:10.859 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:10.866 INFO:teuthology.orchestra.run.smithi001.stdout:5766 trunc from 0xd6be7cd to 0x9681217 2024-09-14T22:38:10.878 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:10.878 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:10.878 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:38:10.878 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:38:10.878 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:10.897 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:38:10.908 INFO:teuthology.orchestra.run.smithi001.stdout:5767 writesame 0xec26361 thru 0xec2afba data_size 0x1d(0x4c5a bytes) 2024-09-14T22:38:10.912 INFO:teuthology.orchestra.run.smithi001.stdout:5768 trunc from 0xec2afbb to 0xc7bb819 2024-09-14T22:38:10.921 INFO:teuthology.orchestra.run.smithi001.stdout:5769 write 0x306cfde thru 0x3075964 (0x8987 bytes) 2024-09-14T22:38:10.924 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:10.925 INFO:teuthology.orchestra.run.smithi001.stdout:5772 read 0xa378a3 thru 0xa3845c (0xbba bytes) 2024-09-14T22:38:10.925 INFO:teuthology.orchestra.run.smithi001.stdout:5775 read 0x9e2ff9 thru 0x9eb7b1 (0x87b9 bytes) 2024-09-14T22:38:10.935 INFO:teuthology.orchestra.run.smithi001.stdout:5776 trunc from 0xc7bb819 to 0x672576f 2024-09-14T22:38:10.951 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:10.946+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:10.954 INFO:teuthology.orchestra.run.smithi001.stdout:5777 writesame 0x5984163 thru 0x5987d6f data_size 0x1(0x3c0d bytes) 2024-09-14T22:38:10.968 INFO:teuthology.orchestra.run.smithi001.stdout:5778 read 0x53a4a4f thru 0x53a5160 (0x712 bytes) 2024-09-14T22:38:10.968 INFO:teuthology.orchestra.run.smithi001.stdout:5780 punch from 0x1a3989d to 0x1a417ab, (0x7f0e bytes) 2024-09-14T22:38:10.968 INFO:teuthology.orchestra.run.smithi001.stdout:5781 read 0x45936d3 thru 0x459c686 (0x8fb4 bytes) 2024-09-14T22:38:10.976 INFO:teuthology.orchestra.run.smithi001.stdout:5782 trunc from 0x672576f to 0xc02244 2024-09-14T22:38:11.009 INFO:teuthology.orchestra.run.smithi001.stdout:5783 read 0x724a8f thru 0x734626 (0xfb98 bytes) 2024-09-14T22:38:11.009 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:11.022 INFO:teuthology.orchestra.run.smithi001.stdout:5786 trunc from 0xc02244 to 0x93dc279 2024-09-14T22:38:11.031 INFO:teuthology.orchestra.run.smithi001.stdout:5788 read 0x11dffb8 thru 0x11e519b (0x51e4 bytes) 2024-09-14T22:38:11.031 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:11.031 INFO:teuthology.orchestra.run.smithi001.stdout:5790 read 0x899a717 thru 0x899e9fd (0x42e7 bytes) 2024-09-14T22:38:11.031 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:11.031 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:11.031 INFO:teuthology.orchestra.run.smithi001.stdout:5793 read 0x5aff5b6 thru 0x5b08127 (0x8b72 bytes) 2024-09-14T22:38:11.031 INFO:teuthology.orchestra.run.smithi001.stdout:5794 read 0x7a70315 thru 0x7a7aac8 (0xa7b4 bytes) 2024-09-14T22:38:11.031 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:11.037 INFO:teuthology.orchestra.run.smithi001.stdout:5796 trunc from 0x93dc279 to 0xd78991c 2024-09-14T22:38:11.042 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:11.043 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:38:11.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:38:11.043 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:38:11.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.043 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144987 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3424256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:11.043 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549297+0000", 2024-09-14T22:38:11.043 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.492282491000001, 2024-09-14T22:38:11.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.079931556000000001, 2024-09-14T22:38:11.043 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.043 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.043 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.043 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.043 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.043 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.044 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144987 2024-09-14T22:38:11.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.044 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.044 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.044 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549297+0000", 2024-09-14T22:38:11.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.044 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.044 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549297+0000", 2024-09-14T22:38:11.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.045 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.045 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549298+0000", 2024-09-14T22:38:11.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2699999999999999e-07 2024-09-14T22:38:11.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.045 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.045 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549301+0000", 2024-09-14T22:38:11.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2959999999999999e-06 2024-09-14T22:38:11.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.045 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.045 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549314+0000", 2024-09-14T22:38:11.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2588999999999999e-05 2024-09-14T22:38:11.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549323+0000", 2024-09-14T22:38:11.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.927e-06 2024-09-14T22:38:11.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629038+0000", 2024-09-14T22:38:11.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.079714220000000002 2024-09-14T22:38:11.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629116+0000", 2024-09-14T22:38:11.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7966000000000005e-05 2024-09-14T22:38:11.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629228+0000", 2024-09-14T22:38:11.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112831 2024-09-14T22:38:11.047 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.047 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.047 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.047 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144988 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3325952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:11.048 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549335+0000", 2024-09-14T22:38:11.048 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.492244266, 2024-09-14T22:38:11.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080179116999999994, 2024-09-14T22:38:11.048 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.048 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.048 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.048 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.048 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.048 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.048 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144988 2024-09-14T22:38:11.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.048 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549335+0000", 2024-09-14T22:38:11.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549335+0000", 2024-09-14T22:38:11.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549336+0000", 2024-09-14T22:38:11.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7999999999999995e-07 2024-09-14T22:38:11.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.050 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.050 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549339+0000", 2024-09-14T22:38:11.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.836e-06 2024-09-14T22:38:11.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.050 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.050 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549352+0000", 2024-09-14T22:38:11.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3069e-05 2024-09-14T22:38:11.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.050 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.050 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549359+0000", 2024-09-14T22:38:11.051 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.357e-06 2024-09-14T22:38:11.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.051 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.051 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629288+0000", 2024-09-14T22:38:11.051 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.079928492000000004 2024-09-14T22:38:11.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.051 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.051 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629412+0000", 2024-09-14T22:38:11.051 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012413699999999999 2024-09-14T22:38:11.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.051 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.052 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629514+0000", 2024-09-14T22:38:11.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000102546 2024-09-14T22:38:11.052 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.052 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.052 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.052 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144989 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3522560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:11.052 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:38:11.052 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.492209857999999, 2024-09-14T22:38:11.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080393122999999997, 2024-09-14T22:38:11.052 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.052 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.052 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.053 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.053 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.053 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.053 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144989 2024-09-14T22:38:11.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.053 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:38:11.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:38:11.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:38:11.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3600000000000003e-07 2024-09-14T22:38:11.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549373+0000", 2024-09-14T22:38:11.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8169999999999999e-06 2024-09-14T22:38:11.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549385+0000", 2024-09-14T22:38:11.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1803e-05 2024-09-14T22:38:11.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549394+0000", 2024-09-14T22:38:11.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4290000000000007e-06 2024-09-14T22:38:11.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629576+0000", 2024-09-14T22:38:11.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080181551000000004 2024-09-14T22:38:11.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.056 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.056 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629654+0000", 2024-09-14T22:38:11.056 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7633999999999996e-05 2024-09-14T22:38:11.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.056 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.056 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629763+0000", 2024-09-14T22:38:11.056 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109153 2024-09-14T22:38:11.056 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.056 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.056 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.057 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144990 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:11.057 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:38:11.057 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.492166833999999, 2024-09-14T22:38:11.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080614496999999993, 2024-09-14T22:38:11.057 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.057 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.057 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.057 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.057 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.057 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.057 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144990 2024-09-14T22:38:11.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.058 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.058 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.058 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:38:11.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.058 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.058 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:38:11.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.059 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:38:11.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0999999999999997e-07 2024-09-14T22:38:11.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.059 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549419+0000", 2024-09-14T22:38:11.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5339999999999999e-06 2024-09-14T22:38:11.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.060 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549439+0000", 2024-09-14T22:38:11.060 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0035e-05 2024-09-14T22:38:11.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.060 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.060 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549449+0000", 2024-09-14T22:38:11.060 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0429e-05 2024-09-14T22:38:11.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.060 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.060 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629815+0000", 2024-09-14T22:38:11.060 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080365467999999995 2024-09-14T22:38:11.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.061 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.061 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.061 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629922+0000", 2024-09-14T22:38:11.061 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106914 2024-09-14T22:38:11.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.061 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.061 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.061 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630027+0000", 2024-09-14T22:38:11.061 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105207 2024-09-14T22:38:11.061 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.061 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.061 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.062 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144991 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 2998272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:11.062 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549463+0000", 2024-09-14T22:38:11.062 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.492116530000001, 2024-09-14T22:38:11.062 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080847637, 2024-09-14T22:38:11.062 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.062 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.062 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.062 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.062 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.062 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.062 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144991 2024-09-14T22:38:11.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.063 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.063 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.063 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549463+0000", 2024-09-14T22:38:11.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.063 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.063 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549463+0000", 2024-09-14T22:38:11.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.064 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.064 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549464+0000", 2024-09-14T22:38:11.064 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9999999999999997e-07 2024-09-14T22:38:11.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.064 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.064 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549467+0000", 2024-09-14T22:38:11.064 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0800000000000002e-06 2024-09-14T22:38:11.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.064 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.064 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549478+0000", 2024-09-14T22:38:11.065 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1813e-05 2024-09-14T22:38:11.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.065 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.065 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549488+0000", 2024-09-14T22:38:11.065 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7030000000000001e-06 2024-09-14T22:38:11.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.065 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.065 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630090+0000", 2024-09-14T22:38:11.065 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080601962999999999 2024-09-14T22:38:11.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.065 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.066 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630187+0000", 2024-09-14T22:38:11.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6552000000000007e-05 2024-09-14T22:38:11.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.066 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.066 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630311+0000", 2024-09-14T22:38:11.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000123826 2024-09-14T22:38:11.066 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.066 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.066 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.066 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144992 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3620864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:11.066 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:38:11.067 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.492080722000001, 2024-09-14T22:38:11.067 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081097875999999999, 2024-09-14T22:38:11.067 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.067 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.067 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.067 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.067 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.067 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.067 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144992 2024-09-14T22:38:11.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.067 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.067 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.067 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.068 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:38:11.068 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.068 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.068 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:38:11.068 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.068 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.068 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:38:11.068 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5400000000000001e-07 2024-09-14T22:38:11.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.069 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.069 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549502+0000", 2024-09-14T22:38:11.069 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7889999999999999e-06 2024-09-14T22:38:11.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.069 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.069 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549513+0000", 2024-09-14T22:38:11.069 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1204e-05 2024-09-14T22:38:11.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.069 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.069 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549523+0000", 2024-09-14T22:38:11.069 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5990000000000008e-06 2024-09-14T22:38:11.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.070 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630362+0000", 2024-09-14T22:38:11.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080839272000000004 2024-09-14T22:38:11.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.070 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630469+0000", 2024-09-14T22:38:11.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107285 2024-09-14T22:38:11.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.070 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630597+0000", 2024-09-14T22:38:11.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012707300000000001 2024-09-14T22:38:11.071 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.071 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.071 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.071 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144993 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:11.071 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:38:11.071 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.492047646, 2024-09-14T22:38:11.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081338411999999999, 2024-09-14T22:38:11.071 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.071 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.071 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.071 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.072 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.072 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.072 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144993 2024-09-14T22:38:11.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.072 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.072 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.072 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:38:11.072 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.072 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.072 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:38:11.072 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.073 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:38:11.073 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5300000000000004e-07 2024-09-14T22:38:11.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.073 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549544+0000", 2024-09-14T22:38:11.073 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1912e-05 2024-09-14T22:38:11.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549557+0000", 2024-09-14T22:38:11.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2413e-05 2024-09-14T22:38:11.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.074 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549567+0000", 2024-09-14T22:38:11.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8660000000000001e-06 2024-09-14T22:38:11.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.074 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630651+0000", 2024-09-14T22:38:11.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081084229999999993 2024-09-14T22:38:11.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.076 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630729+0000", 2024-09-14T22:38:11.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8265999999999999e-05 2024-09-14T22:38:11.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.076 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630870+0000", 2024-09-14T22:38:11.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014107200000000001 2024-09-14T22:38:11.077 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.077 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.077 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.077 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144994 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:11.077 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:38:11.077 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.492000679, 2024-09-14T22:38:11.077 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081547958000000004, 2024-09-14T22:38:11.077 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.077 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.077 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.077 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.078 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.078 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.078 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144994 2024-09-14T22:38:11.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.078 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.078 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.078 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:38:11.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.078 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.078 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:38:11.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.079 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.079 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:38:11.079 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6599999999999996e-07 2024-09-14T22:38:11.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.079 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.079 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549582+0000", 2024-09-14T22:38:11.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.004e-06 2024-09-14T22:38:11.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549594+0000", 2024-09-14T22:38:11.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1459e-05 2024-09-14T22:38:11.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549603+0000", 2024-09-14T22:38:11.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6129999999999993e-06 2024-09-14T22:38:11.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630932+0000", 2024-09-14T22:38:11.081 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081328558999999995 2024-09-14T22:38:11.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631027+0000", 2024-09-14T22:38:11.081 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4742999999999997e-05 2024-09-14T22:38:11.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631127+0000", 2024-09-14T22:38:11.081 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9914000000000006e-05 2024-09-14T22:38:11.082 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.082 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.082 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.082 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144995 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3391488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:11.082 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549615+0000", 2024-09-14T22:38:11.082 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.491964541000002, 2024-09-14T22:38:11.082 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081734251999999993, 2024-09-14T22:38:11.082 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.082 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.082 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.082 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.083 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.083 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.083 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144995 2024-09-14T22:38:11.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.083 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.083 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.083 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549615+0000", 2024-09-14T22:38:11.083 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.083 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.083 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549615+0000", 2024-09-14T22:38:11.083 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.084 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.084 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549616+0000", 2024-09-14T22:38:11.084 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9299999999999997e-07 2024-09-14T22:38:11.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.084 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.084 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549618+0000", 2024-09-14T22:38:11.084 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7269999999999999e-06 2024-09-14T22:38:11.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.084 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.085 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549630+0000", 2024-09-14T22:38:11.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1313e-05 2024-09-14T22:38:11.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.085 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.085 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549639+0000", 2024-09-14T22:38:11.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4159999999999993e-06 2024-09-14T22:38:11.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.085 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.085 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631183+0000", 2024-09-14T22:38:11.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081543633000000004 2024-09-14T22:38:11.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.085 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631252+0000", 2024-09-14T22:38:11.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9567999999999995e-05 2024-09-14T22:38:11.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.086 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631349+0000", 2024-09-14T22:38:11.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6801999999999999e-05 2024-09-14T22:38:11.086 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.086 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.086 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.086 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144996 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3194880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:11.086 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:38:11.087 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.491928862999998, 2024-09-14T22:38:11.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081947213000000005, 2024-09-14T22:38:11.087 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.087 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.087 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.087 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.087 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.087 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.087 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144996 2024-09-14T22:38:11.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.087 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.087 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:38:11.088 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.088 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.088 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:38:11.088 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.088 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.088 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:38:11.088 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3e-07 2024-09-14T22:38:11.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.089 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549654+0000", 2024-09-14T22:38:11.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6900000000000001e-06 2024-09-14T22:38:11.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.089 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549665+0000", 2024-09-14T22:38:11.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1392999999999999e-05 2024-09-14T22:38:11.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.089 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549675+0000", 2024-09-14T22:38:11.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5459999999999997e-06 2024-09-14T22:38:11.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631401+0000", 2024-09-14T22:38:11.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081726036000000002 2024-09-14T22:38:11.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631492+0000", 2024-09-14T22:38:11.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0886999999999999e-05 2024-09-14T22:38:11.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631598+0000", 2024-09-14T22:38:11.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106031 2024-09-14T22:38:11.091 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.091 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.091 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.091 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144997 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:11.091 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549692+0000", 2024-09-14T22:38:11.091 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.491887199000001, 2024-09-14T22:38:11.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082149925999999998, 2024-09-14T22:38:11.091 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.091 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.091 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.091 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.092 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.092 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.092 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144997 2024-09-14T22:38:11.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.092 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.092 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.092 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549692+0000", 2024-09-14T22:38:11.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.092 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.092 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549692+0000", 2024-09-14T22:38:11.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.093 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549693+0000", 2024-09-14T22:38:11.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.540000000000001e-07 2024-09-14T22:38:11.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.093 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549698+0000", 2024-09-14T22:38:11.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1460000000000004e-06 2024-09-14T22:38:11.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.093 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549715+0000", 2024-09-14T22:38:11.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6983e-05 2024-09-14T22:38:11.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549728+0000", 2024-09-14T22:38:11.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2326e-05 2024-09-14T22:38:11.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631649+0000", 2024-09-14T22:38:11.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081921830000000001 2024-09-14T22:38:11.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631726+0000", 2024-09-14T22:38:11.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6759000000000001e-05 2024-09-14T22:38:11.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631842+0000", 2024-09-14T22:38:11.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011592800000000001 2024-09-14T22:38:11.095 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.095 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.095 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.095 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144998 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 2965504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:11.095 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549741+0000", 2024-09-14T22:38:11.096 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.491837907000001, 2024-09-14T22:38:11.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082394832000000001, 2024-09-14T22:38:11.096 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.096 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.096 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.096 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.096 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.096 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.096 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144998 2024-09-14T22:38:11.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.097 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.097 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549741+0000", 2024-09-14T22:38:11.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.097 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549741+0000", 2024-09-14T22:38:11.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549742+0000", 2024-09-14T22:38:11.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.06e-07 2024-09-14T22:38:11.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.098 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549757+0000", 2024-09-14T22:38:11.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4839e-05 2024-09-14T22:38:11.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.098 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549770+0000", 2024-09-14T22:38:11.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2879e-05 2024-09-14T22:38:11.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.099 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549780+0000", 2024-09-14T22:38:11.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0304e-05 2024-09-14T22:38:11.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.099 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631904+0000", 2024-09-14T22:38:11.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082124168999999997 2024-09-14T22:38:11.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.099 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631995+0000", 2024-09-14T22:38:11.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0993000000000001e-05 2024-09-14T22:38:11.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.100 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632136+0000", 2024-09-14T22:38:11.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014094199999999999 2024-09-14T22:38:11.100 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.100 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.100 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.100 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144999 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3588096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:11.100 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:38:11.100 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.491787696999999, 2024-09-14T22:38:11.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082593742999999997, 2024-09-14T22:38:11.100 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.100 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.101 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.101 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.101 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.101 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.101 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144999 2024-09-14T22:38:11.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.101 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:38:11.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.102 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:38:11.102 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.102 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:38:11.102 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1999999999999999e-07 2024-09-14T22:38:11.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.102 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549795+0000", 2024-09-14T22:38:11.102 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9569999999999998e-06 2024-09-14T22:38:11.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549807+0000", 2024-09-14T22:38:11.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1691999999999999e-05 2024-09-14T22:38:11.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549817+0000", 2024-09-14T22:38:11.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5100000000000004e-06 2024-09-14T22:38:11.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632190+0000", 2024-09-14T22:38:11.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082373374999999999 2024-09-14T22:38:11.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632264+0000", 2024-09-14T22:38:11.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3972000000000003e-05 2024-09-14T22:38:11.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632385+0000", 2024-09-14T22:38:11.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121517 2024-09-14T22:38:11.104 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.104 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.104 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.105 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145000 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3063808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:11.105 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549828+0000", 2024-09-14T22:38:11.105 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.491751419, 2024-09-14T22:38:11.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082860235000000004, 2024-09-14T22:38:11.105 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.105 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.105 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.105 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.105 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.105 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.105 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145000 2024-09-14T22:38:11.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.106 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549828+0000", 2024-09-14T22:38:11.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549828+0000", 2024-09-14T22:38:11.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549829+0000", 2024-09-14T22:38:11.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3300000000000002e-07 2024-09-14T22:38:11.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549831+0000", 2024-09-14T22:38:11.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6740000000000001e-06 2024-09-14T22:38:11.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549842+0000", 2024-09-14T22:38:11.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0316000000000001e-05 2024-09-14T22:38:11.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549851+0000", 2024-09-14T22:38:11.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7229999999999999e-06 2024-09-14T22:38:11.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632448+0000", 2024-09-14T22:38:11.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082596648999999994 2024-09-14T22:38:11.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632568+0000", 2024-09-14T22:38:11.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119592 2024-09-14T22:38:11.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632688+0000", 2024-09-14T22:38:11.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120648 2024-09-14T22:38:11.109 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.109 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.109 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.109 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145001 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3162112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:11.109 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549863+0000", 2024-09-14T22:38:11.109 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.491716190999998, 2024-09-14T22:38:11.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083105534999999994, 2024-09-14T22:38:11.110 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.110 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.110 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.110 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.110 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.110 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.110 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145001 2024-09-14T22:38:11.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.110 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549863+0000", 2024-09-14T22:38:11.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.111 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.111 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549863+0000", 2024-09-14T22:38:11.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.111 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.111 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549864+0000", 2024-09-14T22:38:11.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4300000000000003e-07 2024-09-14T22:38:11.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549867+0000", 2024-09-14T22:38:11.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.757e-06 2024-09-14T22:38:11.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549878+0000", 2024-09-14T22:38:11.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1345999999999999e-05 2024-09-14T22:38:11.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549888+0000", 2024-09-14T22:38:11.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6460000000000005e-06 2024-09-14T22:38:11.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632754+0000", 2024-09-14T22:38:11.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082866258999999998 2024-09-14T22:38:11.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632841+0000", 2024-09-14T22:38:11.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6971000000000005e-05 2024-09-14T22:38:11.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632969+0000", 2024-09-14T22:38:11.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012791300000000001 2024-09-14T22:38:11.114 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.114 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.114 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.114 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145002 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3260416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:11.114 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549933+0000", 2024-09-14T22:38:11.114 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.491646687999999, 2024-09-14T22:38:11.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083291027000000004, 2024-09-14T22:38:11.114 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.114 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.115 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.115 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.115 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.115 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.115 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145002 2024-09-14T22:38:11.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.115 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549933+0000", 2024-09-14T22:38:11.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549933+0000", 2024-09-14T22:38:11.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549934+0000", 2024-09-14T22:38:11.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8700000000000004e-07 2024-09-14T22:38:11.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549938+0000", 2024-09-14T22:38:11.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5029999999999999e-06 2024-09-14T22:38:11.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549952+0000", 2024-09-14T22:38:11.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4382999999999999e-05 2024-09-14T22:38:11.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549963+0000", 2024-09-14T22:38:11.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0665999999999999e-05 2024-09-14T22:38:11.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633025+0000", 2024-09-14T22:38:11.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083061727000000002 2024-09-14T22:38:11.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633117+0000", 2024-09-14T22:38:11.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1812999999999997e-05 2024-09-14T22:38:11.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633224+0000", 2024-09-14T22:38:11.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107048 2024-09-14T22:38:11.118 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.118 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.118 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.119 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145003 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3457024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:11.119 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549985+0000", 2024-09-14T22:38:11.119 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.49159427, 2024-09-14T22:38:11.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083465871999999997, 2024-09-14T22:38:11.119 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.119 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.119 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.119 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.119 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.119 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.119 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145003 2024-09-14T22:38:11.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.120 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549985+0000", 2024-09-14T22:38:11.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549985+0000", 2024-09-14T22:38:11.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549986+0000", 2024-09-14T22:38:11.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7299999999999995e-07 2024-09-14T22:38:11.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549989+0000", 2024-09-14T22:38:11.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8169999999999999e-06 2024-09-14T22:38:11.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549998+0000", 2024-09-14T22:38:11.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4290000000000007e-06 2024-09-14T22:38:11.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.122 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550009+0000", 2024-09-14T22:38:11.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0633e-05 2024-09-14T22:38:11.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.122 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633275+0000", 2024-09-14T22:38:11.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083266397000000006 2024-09-14T22:38:11.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633353+0000", 2024-09-14T22:38:11.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8129999999999999e-05 2024-09-14T22:38:11.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633451+0000", 2024-09-14T22:38:11.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7793000000000004e-05 2024-09-14T22:38:11.123 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.123 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.123 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.123 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145004 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3358720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:11.124 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.550029+0000", 2024-09-14T22:38:11.124 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.491550699000001, 2024-09-14T22:38:11.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083662089999999995, 2024-09-14T22:38:11.124 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.124 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.124 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.124 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.124 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.124 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.124 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145004 2024-09-14T22:38:11.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.124 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.125 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550029+0000", 2024-09-14T22:38:11.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.125 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550029+0000", 2024-09-14T22:38:11.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.125 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550030+0000", 2024-09-14T22:38:11.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.33e-07 2024-09-14T22:38:11.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550035+0000", 2024-09-14T22:38:11.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3430000000000003e-06 2024-09-14T22:38:11.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550053+0000", 2024-09-14T22:38:11.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8485e-05 2024-09-14T22:38:11.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550064+0000", 2024-09-14T22:38:11.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0264e-05 2024-09-14T22:38:11.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633502+0000", 2024-09-14T22:38:11.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083438498999999999 2024-09-14T22:38:11.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633592+0000", 2024-09-14T22:38:11.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.022e-05 2024-09-14T22:38:11.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633691+0000", 2024-09-14T22:38:11.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8345999999999997e-05 2024-09-14T22:38:11.128 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.128 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.128 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.128 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145005 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3555328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:11.128 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.550077+0000", 2024-09-14T22:38:11.128 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.491502238999999, 2024-09-14T22:38:11.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083854340999999999, 2024-09-14T22:38:11.128 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.129 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.129 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.129 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.129 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.129 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.129 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145005 2024-09-14T22:38:11.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.129 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550077+0000", 2024-09-14T22:38:11.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550077+0000", 2024-09-14T22:38:11.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550078+0000", 2024-09-14T22:38:11.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8700000000000005e-07 2024-09-14T22:38:11.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550101+0000", 2024-09-14T22:38:11.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3632e-05 2024-09-14T22:38:11.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550115+0000", 2024-09-14T22:38:11.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3893000000000001e-05 2024-09-14T22:38:11.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550126+0000", 2024-09-14T22:38:11.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0426e-05 2024-09-14T22:38:11.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633742+0000", 2024-09-14T22:38:11.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083616633999999995 2024-09-14T22:38:11.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633816+0000", 2024-09-14T22:38:11.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3690999999999996e-05 2024-09-14T22:38:11.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633932+0000", 2024-09-14T22:38:11.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011537800000000001 2024-09-14T22:38:11.132 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.132 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.132 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.133 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145006 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3653632~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:11.133 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.550236+0000", 2024-09-14T22:38:11.133 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.491343733000001, 2024-09-14T22:38:11.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083924027999999998, 2024-09-14T22:38:11.133 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.133 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.133 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.133 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.133 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.133 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.133 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145006 2024-09-14T22:38:11.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.134 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.134 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550236+0000", 2024-09-14T22:38:11.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.134 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550236+0000", 2024-09-14T22:38:11.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.134 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550237+0000", 2024-09-14T22:38:11.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2929999999999999e-06 2024-09-14T22:38:11.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550243+0000", 2024-09-14T22:38:11.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0390000000000001e-06 2024-09-14T22:38:11.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550256+0000", 2024-09-14T22:38:11.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3393e-05 2024-09-14T22:38:11.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550271+0000", 2024-09-14T22:38:11.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4166e-05 2024-09-14T22:38:11.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633974+0000", 2024-09-14T22:38:11.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083703825999999995 2024-09-14T22:38:11.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.634047+0000", 2024-09-14T22:38:11.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2935999999999994e-05 2024-09-14T22:38:11.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.634160+0000", 2024-09-14T22:38:11.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011237499999999999 2024-09-14T22:38:11.137 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.137 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.137 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.137 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.137 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.137 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:11.138 INFO:teuthology.orchestra.run.smithi001.stdout:5797 read 0x80c050d thru 0x80c7d89 (0x787d bytes) 2024-09-14T22:38:11.138 INFO:teuthology.orchestra.run.smithi001.stdout:5799 read 0x792954a thru 0x7938509 (0xefc0 bytes) 2024-09-14T22:38:11.138 INFO:teuthology.orchestra.run.smithi001.stdout:5800 writesame 0x58d0c89 thru 0x58da150 data_size 0x24(0x94c8 bytes) 2024-09-14T22:38:11.138 INFO:teuthology.orchestra.run.smithi001.stdout:5801 clone 38 order 21 su 16384 sc 2 2024-09-14T22:38:11.139 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:38:11.140 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:11.130+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:11.239 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:11.234+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:11.313 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:11.313 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:11.314 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:38:11.314 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:11.328 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:38:11.341 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:11.334+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:11.442 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:11.438+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:11.498 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:11.498 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:11.498 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:38:11.498 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:38:11.498 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:11.513 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:38:11.557 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:11.554+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:11.644 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:11.644 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:38:11.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:38:11.644 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:38:11.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.644 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:122960 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 2347008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:11.644 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:41.998325+0000", 2024-09-14T22:38:11.644 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.642114548999999, 2024-09-14T22:38:11.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056570400999999999, 2024-09-14T22:38:11.644 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.645 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.645 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.645 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.645 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.645 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.645 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 122960 2024-09-14T22:38:11.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.645 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998325+0000", 2024-09-14T22:38:11.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998325+0000", 2024-09-14T22:38:11.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998326+0000", 2024-09-14T22:38:11.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9299999999999998e-07 2024-09-14T22:38:11.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998329+0000", 2024-09-14T22:38:11.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1370000000000002e-06 2024-09-14T22:38:11.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998340+0000", 2024-09-14T22:38:11.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1039999999999999e-05 2024-09-14T22:38:11.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998348+0000", 2024-09-14T22:38:11.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1920000000000005e-06 2024-09-14T22:38:11.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054700+0000", 2024-09-14T22:38:11.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056351714999999997 2024-09-14T22:38:11.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054771+0000", 2024-09-14T22:38:11.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1009999999999999e-05 2024-09-14T22:38:11.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054896+0000", 2024-09-14T22:38:11.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000124714 2024-09-14T22:38:11.648 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.648 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.648 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.648 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:122961 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 2351104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:11.649 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:41.998357+0000", 2024-09-14T22:38:11.649 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.642082594000001, 2024-09-14T22:38:11.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056974912000000003, 2024-09-14T22:38:11.649 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.649 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.649 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.649 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.649 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.649 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.649 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 122961 2024-09-14T22:38:11.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.649 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998357+0000", 2024-09-14T22:38:11.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998357+0000", 2024-09-14T22:38:11.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998358+0000", 2024-09-14T22:38:11.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9699999999999996e-07 2024-09-14T22:38:11.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998361+0000", 2024-09-14T22:38:11.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1370000000000002e-06 2024-09-14T22:38:11.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998371+0000", 2024-09-14T22:38:11.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5789999999999993e-06 2024-09-14T22:38:11.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998379+0000", 2024-09-14T22:38:11.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0129999999999993e-06 2024-09-14T22:38:11.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054998+0000", 2024-09-14T22:38:11.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056618961000000002 2024-09-14T22:38:11.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.055126+0000", 2024-09-14T22:38:11.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012795100000000001 2024-09-14T22:38:11.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.055332+0000", 2024-09-14T22:38:11.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020667400000000001 2024-09-14T22:38:11.653 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.653 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.653 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.653 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123259 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 741376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:11.653 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:38:11.653 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.628357077, 2024-09-14T22:38:11.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056672581999999999, 2024-09-14T22:38:11.653 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.653 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.654 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.654 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.654 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.654 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.654 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123259 2024-09-14T22:38:11.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.654 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:38:11.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:38:11.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:38:11.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1600000000000001e-07 2024-09-14T22:38:11.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012086+0000", 2024-09-14T22:38:11.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8499999999999998e-06 2024-09-14T22:38:11.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012097+0000", 2024-09-14T22:38:11.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0943e-05 2024-09-14T22:38:11.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012105+0000", 2024-09-14T22:38:11.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0830000000000004e-06 2024-09-14T22:38:11.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068584+0000", 2024-09-14T22:38:11.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056478387999999997 2024-09-14T22:38:11.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068644+0000", 2024-09-14T22:38:11.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9984e-05 2024-09-14T22:38:11.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068755+0000", 2024-09-14T22:38:11.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111718 2024-09-14T22:38:11.657 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.657 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.657 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.657 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123260 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 778240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:11.658 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012118+0000", 2024-09-14T22:38:11.658 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.628321851999999, 2024-09-14T22:38:11.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056825887999999998, 2024-09-14T22:38:11.658 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.658 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.658 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.658 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.658 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.658 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.658 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123260 2024-09-14T22:38:11.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.659 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012118+0000", 2024-09-14T22:38:11.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012118+0000", 2024-09-14T22:38:11.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012119+0000", 2024-09-14T22:38:11.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.13e-07 2024-09-14T22:38:11.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012130+0000", 2024-09-14T22:38:11.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1552999999999999e-05 2024-09-14T22:38:11.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012141+0000", 2024-09-14T22:38:11.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.03e-05 2024-09-14T22:38:11.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012149+0000", 2024-09-14T22:38:11.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1419999999999993e-06 2024-09-14T22:38:11.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068800+0000", 2024-09-14T22:38:11.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056651574000000003 2024-09-14T22:38:11.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068858+0000", 2024-09-14T22:38:11.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8047000000000002e-05 2024-09-14T22:38:11.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068944+0000", 2024-09-14T22:38:11.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5659000000000001e-05 2024-09-14T22:38:11.662 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.662 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.662 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.663 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123261 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 761856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:11.663 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012161+0000", 2024-09-14T22:38:11.663 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.628278451, 2024-09-14T22:38:11.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057113753000000003, 2024-09-14T22:38:11.663 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.663 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.663 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.663 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.663 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.663 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.663 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123261 2024-09-14T22:38:11.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.663 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012161+0000", 2024-09-14T22:38:11.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012161+0000", 2024-09-14T22:38:11.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012162+0000", 2024-09-14T22:38:11.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4399999999999999e-07 2024-09-14T22:38:11.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012177+0000", 2024-09-14T22:38:11.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5099000000000001e-05 2024-09-14T22:38:11.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012191+0000", 2024-09-14T22:38:11.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3631999999999999e-05 2024-09-14T22:38:11.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.666 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012199+0000", 2024-09-14T22:38:11.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5769999999999994e-06 2024-09-14T22:38:11.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.666 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069004+0000", 2024-09-14T22:38:11.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056804504999999998 2024-09-14T22:38:11.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.666 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069137+0000", 2024-09-14T22:38:11.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013302600000000001 2024-09-14T22:38:11.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069275+0000", 2024-09-14T22:38:11.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013816999999999999 2024-09-14T22:38:11.667 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.667 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.667 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.667 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123262 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 724992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:11.667 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012224+0000", 2024-09-14T22:38:11.667 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.628216003999999, 2024-09-14T22:38:11.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057417959999999997, 2024-09-14T22:38:11.667 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.668 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.668 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.668 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.668 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.668 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.668 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123262 2024-09-14T22:38:11.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.668 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012224+0000", 2024-09-14T22:38:11.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012224+0000", 2024-09-14T22:38:11.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012225+0000", 2024-09-14T22:38:11.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4000000000000001e-07 2024-09-14T22:38:11.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012228+0000", 2024-09-14T22:38:11.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5729999999999999e-06 2024-09-14T22:38:11.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012241+0000", 2024-09-14T22:38:11.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3210000000000001e-05 2024-09-14T22:38:11.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012255+0000", 2024-09-14T22:38:11.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3852000000000001e-05 2024-09-14T22:38:11.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069341+0000", 2024-09-14T22:38:11.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057086139000000001 2024-09-14T22:38:11.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069477+0000", 2024-09-14T22:38:11.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013595 2024-09-14T22:38:11.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069642+0000", 2024-09-14T22:38:11.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016459600000000001 2024-09-14T22:38:11.672 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.672 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.672 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.672 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123263 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 782336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:11.672 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012265+0000", 2024-09-14T22:38:11.672 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.628174462, 2024-09-14T22:38:11.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057665095, 2024-09-14T22:38:11.672 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.672 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.672 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.672 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.673 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.673 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.673 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123263 2024-09-14T22:38:11.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.673 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012265+0000", 2024-09-14T22:38:11.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012265+0000", 2024-09-14T22:38:11.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012266+0000", 2024-09-14T22:38:11.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8999999999999996e-07 2024-09-14T22:38:11.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012269+0000", 2024-09-14T22:38:11.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.796e-06 2024-09-14T22:38:11.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012279+0000", 2024-09-14T22:38:11.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7389999999999994e-06 2024-09-14T22:38:11.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012286+0000", 2024-09-14T22:38:11.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8199999999999997e-06 2024-09-14T22:38:11.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069705+0000", 2024-09-14T22:38:11.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057418590999999998 2024-09-14T22:38:11.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069799+0000", 2024-09-14T22:38:11.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4419999999999994e-05 2024-09-14T22:38:11.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069931+0000", 2024-09-14T22:38:11.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013113900000000001 2024-09-14T22:38:11.676 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.676 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.676 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.676 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123264 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 745472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:11.676 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012295+0000", 2024-09-14T22:38:11.677 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.628144461000002, 2024-09-14T22:38:11.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057959498999999998, 2024-09-14T22:38:11.677 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.677 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.677 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.677 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.677 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.677 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.677 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123264 2024-09-14T22:38:11.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.677 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012295+0000", 2024-09-14T22:38:11.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012295+0000", 2024-09-14T22:38:11.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012296+0000", 2024-09-14T22:38:11.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9699999999999996e-07 2024-09-14T22:38:11.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012299+0000", 2024-09-14T22:38:11.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.79e-06 2024-09-14T22:38:11.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012320+0000", 2024-09-14T22:38:11.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1242000000000001e-05 2024-09-14T22:38:11.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012338+0000", 2024-09-14T22:38:11.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7768999999999999e-05 2024-09-14T22:38:11.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069995+0000", 2024-09-14T22:38:11.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057657 2024-09-14T22:38:11.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070132+0000", 2024-09-14T22:38:11.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013727800000000001 2024-09-14T22:38:11.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070255+0000", 2024-09-14T22:38:11.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012282300000000001 2024-09-14T22:38:11.681 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.681 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.681 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.681 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123265 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1056768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:11.681 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012348+0000", 2024-09-14T22:38:11.681 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.628091506000001, 2024-09-14T22:38:11.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058145087999999998, 2024-09-14T22:38:11.681 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.681 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.681 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.681 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.681 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.681 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.682 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123265 2024-09-14T22:38:11.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.682 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012348+0000", 2024-09-14T22:38:11.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012348+0000", 2024-09-14T22:38:11.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012349+0000", 2024-09-14T22:38:11.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6599999999999996e-07 2024-09-14T22:38:11.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012352+0000", 2024-09-14T22:38:11.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3139999999999999e-06 2024-09-14T22:38:11.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012365+0000", 2024-09-14T22:38:11.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2625e-05 2024-09-14T22:38:11.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012373+0000", 2024-09-14T22:38:11.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2470000000000008e-06 2024-09-14T22:38:11.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070286+0000", 2024-09-14T22:38:11.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057912547000000002 2024-09-14T22:38:11.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070366+0000", 2024-09-14T22:38:11.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0468000000000003e-05 2024-09-14T22:38:11.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070493+0000", 2024-09-14T22:38:11.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012722100000000001 2024-09-14T22:38:11.685 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.685 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.685 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.685 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123266 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 765952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:11.685 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:38:11.685 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.628057257999998, 2024-09-14T22:38:11.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058386949, 2024-09-14T22:38:11.686 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.686 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.686 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.686 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.686 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.687 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.687 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123266 2024-09-14T22:38:11.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.688 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.688 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.688 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:38:11.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.688 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.688 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:38:11.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.689 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:38:11.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9299999999999998e-07 2024-09-14T22:38:11.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.689 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012386+0000", 2024-09-14T22:38:11.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8729999999999999e-06 2024-09-14T22:38:11.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.689 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012406+0000", 2024-09-14T22:38:11.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0103e-05 2024-09-14T22:38:11.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012419+0000", 2024-09-14T22:38:11.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.311e-05 2024-09-14T22:38:11.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070550+0000", 2024-09-14T22:38:11.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058130902999999998 2024-09-14T22:38:11.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070650+0000", 2024-09-14T22:38:11.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010005000000000001 2024-09-14T22:38:11.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070770+0000", 2024-09-14T22:38:11.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119317 2024-09-14T22:38:11.691 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.691 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.691 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.691 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123267 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1077248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:11.692 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012429+0000", 2024-09-14T22:38:11.692 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.628011344000001, 2024-09-14T22:38:11.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058620343999999998, 2024-09-14T22:38:11.692 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.692 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.692 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.692 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.692 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.692 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.692 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123267 2024-09-14T22:38:11.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.692 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.692 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012429+0000", 2024-09-14T22:38:11.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012429+0000", 2024-09-14T22:38:11.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013488+0000", 2024-09-14T22:38:11.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001059061 2024-09-14T22:38:11.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.694 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013499+0000", 2024-09-14T22:38:11.694 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1348999999999999e-05 2024-09-14T22:38:11.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.694 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.694 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013534+0000", 2024-09-14T22:38:11.694 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5206000000000001e-05 2024-09-14T22:38:11.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.694 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.694 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013558+0000", 2024-09-14T22:38:11.694 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4324999999999998e-05 2024-09-14T22:38:11.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070834+0000", 2024-09-14T22:38:11.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057275110999999997 2024-09-14T22:38:11.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070937+0000", 2024-09-14T22:38:11.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010306099999999999 2024-09-14T22:38:11.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071049+0000", 2024-09-14T22:38:11.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112231 2024-09-14T22:38:11.695 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.695 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.696 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.696 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123268 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1097728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:11.696 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013593+0000", 2024-09-14T22:38:11.696 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.626846620999999, 2024-09-14T22:38:11.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057751793000000003, 2024-09-14T22:38:11.696 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.696 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.696 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.696 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.696 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.696 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.696 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123268 2024-09-14T22:38:11.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.697 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013593+0000", 2024-09-14T22:38:11.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013593+0000", 2024-09-14T22:38:11.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013595+0000", 2024-09-14T22:38:11.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.59e-06 2024-09-14T22:38:11.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.698 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.698 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013602+0000", 2024-09-14T22:38:11.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3560000000000004e-06 2024-09-14T22:38:11.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.698 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.698 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013625+0000", 2024-09-14T22:38:11.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2419e-05 2024-09-14T22:38:11.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.698 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013650+0000", 2024-09-14T22:38:11.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5151999999999999e-05 2024-09-14T22:38:11.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071096+0000", 2024-09-14T22:38:11.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057445877999999999 2024-09-14T22:38:11.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071212+0000", 2024-09-14T22:38:11.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116215 2024-09-14T22:38:11.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071345+0000", 2024-09-14T22:38:11.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013318299999999999 2024-09-14T22:38:11.700 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.700 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.700 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.700 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123269 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1060864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:11.700 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013676+0000", 2024-09-14T22:38:11.700 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.626763862000001, 2024-09-14T22:38:11.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058027982999999998, 2024-09-14T22:38:11.700 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.700 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.700 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.701 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.701 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.701 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.701 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123269 2024-09-14T22:38:11.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.701 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013676+0000", 2024-09-14T22:38:11.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013676+0000", 2024-09-14T22:38:11.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013677+0000", 2024-09-14T22:38:11.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.387e-06 2024-09-14T22:38:11.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013684+0000", 2024-09-14T22:38:11.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0530000000000001e-06 2024-09-14T22:38:11.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013706+0000", 2024-09-14T22:38:11.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1121999999999999e-05 2024-09-14T22:38:11.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.703 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013722+0000", 2024-09-14T22:38:11.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6249999999999999e-05 2024-09-14T22:38:11.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.703 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071402+0000", 2024-09-14T22:38:11.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057679945000000003 2024-09-14T22:38:11.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.703 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071547+0000", 2024-09-14T22:38:11.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000145632 2024-09-14T22:38:11.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071704+0000", 2024-09-14T22:38:11.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000156594 2024-09-14T22:38:11.704 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.704 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.704 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.704 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123270 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1064960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:11.704 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013744+0000", 2024-09-14T22:38:11.705 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.626695910999999, 2024-09-14T22:38:11.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058355872000000003, 2024-09-14T22:38:11.705 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.705 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.705 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.705 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.705 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.705 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.705 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123270 2024-09-14T22:38:11.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.705 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013744+0000", 2024-09-14T22:38:11.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013744+0000", 2024-09-14T22:38:11.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013745+0000", 2024-09-14T22:38:11.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2589999999999999e-06 2024-09-14T22:38:11.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013751+0000", 2024-09-14T22:38:11.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1929999999999998e-06 2024-09-14T22:38:11.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013778+0000", 2024-09-14T22:38:11.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6472e-05 2024-09-14T22:38:11.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013795+0000", 2024-09-14T22:38:11.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6626e-05 2024-09-14T22:38:11.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071775+0000", 2024-09-14T22:38:11.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057980232 2024-09-14T22:38:11.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071911+0000", 2024-09-14T22:38:11.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000136129 2024-09-14T22:38:11.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072100+0000", 2024-09-14T22:38:11.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000188961 2024-09-14T22:38:11.708 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.708 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.709 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.709 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123271 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1081344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:11.709 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013818+0000", 2024-09-14T22:38:11.709 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.626621714999999, 2024-09-14T22:38:11.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058549565999999997, 2024-09-14T22:38:11.709 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.709 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.709 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.709 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.709 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.709 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.709 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123271 2024-09-14T22:38:11.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.710 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013818+0000", 2024-09-14T22:38:11.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013818+0000", 2024-09-14T22:38:11.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013958+0000", 2024-09-14T22:38:11.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013934 2024-09-14T22:38:11.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013968+0000", 2024-09-14T22:38:11.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0329000000000001e-05 2024-09-14T22:38:11.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013995+0000", 2024-09-14T22:38:11.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7613000000000001e-05 2024-09-14T22:38:11.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014014+0000", 2024-09-14T22:38:11.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8338999999999998e-05 2024-09-14T22:38:11.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072155+0000", 2024-09-14T22:38:11.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058140989999999997 2024-09-14T22:38:11.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072247+0000", 2024-09-14T22:38:11.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1774999999999995e-05 2024-09-14T22:38:11.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072368+0000", 2024-09-14T22:38:11.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012118 2024-09-14T22:38:11.713 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.713 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.713 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.713 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123272 3.1 3.4b235a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:11.713 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.014046+0000", 2024-09-14T22:38:11.713 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.626394039000001, 2024-09-14T22:38:11.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058364077, 2024-09-14T22:38:11.713 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.713 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.714 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-14T22:38:11.714 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.714 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.714 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.714 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123272 2024-09-14T22:38:11.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.714 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014046+0000", 2024-09-14T22:38:11.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014046+0000", 2024-09-14T22:38:11.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014047+0000", 2024-09-14T22:38:11.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.533e-06 2024-09-14T22:38:11.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014085+0000", 2024-09-14T22:38:11.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7734999999999997e-05 2024-09-14T22:38:11.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014111+0000", 2024-09-14T22:38:11.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6055999999999999e-05 2024-09-14T22:38:11.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014136+0000", 2024-09-14T22:38:11.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4592000000000001e-05 2024-09-14T22:38:11.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072402+0000", 2024-09-14T22:38:11.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058266251999999998 2024-09-14T22:38:11.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.717 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072410+0000", 2024-09-14T22:38:11.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.909e-06 2024-09-14T22:38:11.717 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.717 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.717 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.717 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123273 3.1 3.4b235a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:11.717 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.014161+0000", 2024-09-14T22:38:11.717 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.626279011000001, 2024-09-14T22:38:11.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058284415999999999, 2024-09-14T22:38:11.717 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.717 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.717 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-14T22:38:11.718 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.718 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.718 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.718 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123273 2024-09-14T22:38:11.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.718 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014161+0000", 2024-09-14T22:38:11.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014161+0000", 2024-09-14T22:38:11.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.017996+0000", 2024-09-14T22:38:11.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0038350670000000002 2024-09-14T22:38:11.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.018009+0000", 2024-09-14T22:38:11.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3305999999999999e-05 2024-09-14T22:38:11.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.018044+0000", 2024-09-14T22:38:11.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4792000000000002e-05 2024-09-14T22:38:11.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.018067+0000", 2024-09-14T22:38:11.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2595e-05 2024-09-14T22:38:11.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072439+0000", 2024-09-14T22:38:11.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054372192 2024-09-14T22:38:11.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072445+0000", 2024-09-14T22:38:11.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4640000000000003e-06 2024-09-14T22:38:11.721 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.721 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.721 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.721 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:148469 3.18 3:1f6f7244:::rbd_header.113c827b523f:head [watch reconnect cookie 140250011614800 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-14T22:38:11.721 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:55.746449+0000", 2024-09-14T22:38:11.721 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.893990399, 2024-09-14T22:38:11.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.996484173, 2024-09-14T22:38:11.721 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.722 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.722 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.722 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.722 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.722 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.722 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 148469 2024-09-14T22:38:11.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.722 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.722 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746449+0000", 2024-09-14T22:38:11.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746449+0000", 2024-09-14T22:38:11.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746452+0000", 2024-09-14T22:38:11.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0019999999999998e-06 2024-09-14T22:38:11.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746461+0000", 2024-09-14T22:38:11.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0919999999999998e-06 2024-09-14T22:38:11.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746475+0000", 2024-09-14T22:38:11.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4853e-05 2024-09-14T22:38:11.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746484+0000", 2024-09-14T22:38:11.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3780000000000001e-06 2024-09-14T22:38:11.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746559+0000", 2024-09-14T22:38:11.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5259000000000006e-05 2024-09-14T22:38:11.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:11.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746572+0000", 2024-09-14T22:38:11.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.271e-05 2024-09-14T22:38:11.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727807+0000", 2024-09-14T22:38:11.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98123555900000003 2024-09-14T22:38:11.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:11.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727826+0000", 2024-09-14T22:38:11.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8828e-05 2024-09-14T22:38:11.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.739033+0000", 2024-09-14T22:38:11.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.011207062 2024-09-14T22:38:11.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.739479+0000", 2024-09-14T22:38:11.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044563200000000001 2024-09-14T22:38:11.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.742934+0000", 2024-09-14T22:38:11.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0034547979999999998 2024-09-14T22:38:11.727 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.727 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.727 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.727 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:148473 3.18 3:1f6f7244:::rbd_header.113c827b523f:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-14T22:38:11.727 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:55.755969+0000", 2024-09-14T22:38:11.727 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.884470632999999, 2024-09-14T22:38:11.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9872532889999999, 2024-09-14T22:38:11.727 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.727 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.727 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:11.728 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.728 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.728 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.728 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 148473 2024-09-14T22:38:11.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.728 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755969+0000", 2024-09-14T22:38:11.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755969+0000", 2024-09-14T22:38:11.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755971+0000", 2024-09-14T22:38:11.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.674e-06 2024-09-14T22:38:11.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755979+0000", 2024-09-14T22:38:11.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8599999999999993e-06 2024-09-14T22:38:11.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755992+0000", 2024-09-14T22:38:11.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.361e-05 2024-09-14T22:38:11.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.756001+0000", 2024-09-14T22:38:11.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.422e-06 2024-09-14T22:38:11.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.756069+0000", 2024-09-14T22:38:11.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8209999999999999e-05 2024-09-14T22:38:11.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.731 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:11.731 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.756077+0000", 2024-09-14T22:38:11.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.994e-06 2024-09-14T22:38:11.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.731 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.731 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727872+0000", 2024-09-14T22:38:11.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.97179484999999999 2024-09-14T22:38:11.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:11.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727885+0000", 2024-09-14T22:38:11.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3101e-05 2024-09-14T22:38:11.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.740559+0000", 2024-09-14T22:38:11.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0126736300000001 2024-09-14T22:38:11.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.740777+0000", 2024-09-14T22:38:11.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021822000000000001 2024-09-14T22:38:11.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 2", 2024-09-14T22:38:11.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.741114+0000", 2024-09-14T22:38:11.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000336793 2024-09-14T22:38:11.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:11.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.742668+0000", 2024-09-14T22:38:11.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001554061 2024-09-14T22:38:11.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:38:11.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.742947+0000", 2024-09-14T22:38:11.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027937200000000001 2024-09-14T22:38:11.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:38:11.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743030+0000", 2024-09-14T22:38:11.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3385000000000002e-05 2024-09-14T22:38:11.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743223+0000", 2024-09-14T22:38:11.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00019210700000000001 2024-09-14T22:38:11.734 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.735 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.735 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.735 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:148474 3.18 3:1f6f7244:::rbd_header.113c827b523f:head [watch reconnect cookie 140250011614800 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104)", 2024-09-14T22:38:11.735 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:56.728273+0000", 2024-09-14T22:38:11.735 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.912167386, 2024-09-14T22:38:11.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0180497770000001, 2024-09-14T22:38:11.735 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:11.735 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:11.735 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:11.735 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:11.735 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:11.735 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:11.736 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 148474 2024-09-14T22:38:11.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.736 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:11.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.736 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:11.736 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728273+0000", 2024-09-14T22:38:11.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.736 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:11.736 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728273+0000", 2024-09-14T22:38:11.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:11.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:11.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728274+0000", 2024-09-14T22:38:11.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7570000000000001e-06 2024-09-14T22:38:11.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:11.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728281+0000", 2024-09-14T22:38:11.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5089999999999999e-06 2024-09-14T22:38:11.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:11.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728298+0000", 2024-09-14T22:38:11.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6727000000000001e-05 2024-09-14T22:38:11.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:11.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728309+0000", 2024-09-14T22:38:11.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1021e-05 2024-09-14T22:38:11.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.730078+0000", 2024-09-14T22:38:11.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017694010000000001 2024-09-14T22:38:11.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:11.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.730096+0000", 2024-09-14T22:38:11.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8039000000000001e-05 2024-09-14T22:38:11.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.741672+0000", 2024-09-14T22:38:11.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.011575817 2024-09-14T22:38:11.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2024-09-14T22:38:11.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.741827+0000", 2024-09-14T22:38:11.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000155563 2024-09-14T22:38:11.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:11.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743299+0000", 2024-09-14T22:38:11.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0014712729999999999 2024-09-14T22:38:11.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:11.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743415+0000", 2024-09-14T22:38:11.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011594700000000001 2024-09-14T22:38:11.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:11.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:11.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:11.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.746322+0000", 2024-09-14T22:38:11.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0029077230000000001 2024-09-14T22:38:11.740 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.740 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.740 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.741 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:11.741 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:11.741 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:11.741 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:11.638+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:11.742 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:38:11.743 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:11.738+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:11.845 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:11.838+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:11.916 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:11.916 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:11.917 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:38:11.917 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:11.931 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:38:11.946 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:11.942+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:11.988 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2024-09-14T22:38:12.057 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:12.054+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:12.101 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:12.102 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:12.102 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:38:12.102 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:38:12.102 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:12.118 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:38:12.142 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:38:12.166 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:12.162+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:12.247 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:12.242+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:12.287 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-14T22:38:12.295 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:12.295 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:38:12.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:38:12.295 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:38:12.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.296 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:31370 3.0 3:02f48f58:::rbd_data.113cda0183a0.0000000000000000:6 [read 813056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:38:12.296 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:38:12.296 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 98.931210855000003, 2024-09-14T22:38:12.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047519796000000003, 2024-09-14T22:38:12.296 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:12.296 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:12.296 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:12.296 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:12.296 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:12.296 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:12.296 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 31370 2024-09-14T22:38:12.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.296 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:12.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:12.297 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:38:12.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:12.297 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:38:12.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:12.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:38:12.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0999999999999999e-07 2024-09-14T22:38:12.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:12.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362401+0000", 2024-09-14T22:38:12.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2900000000000001e-06 2024-09-14T22:38:12.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:12.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362406+0000", 2024-09-14T22:38:12.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6960000000000002e-06 2024-09-14T22:38:12.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:12.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362412+0000", 2024-09-14T22:38:12.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.823e-06 2024-09-14T22:38:12.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:12.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.400270+0000", 2024-09-14T22:38:12.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037857737000000002 2024-09-14T22:38:12.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:12.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.400342+0000", 2024-09-14T22:38:12.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1797000000000004e-05 2024-09-14T22:38:12.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:12.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.409918+0000", 2024-09-14T22:38:12.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0095760429999999994 2024-09-14T22:38:12.302 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.302 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:12.302 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.303 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:113947 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 5603328~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:12.303 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.923453+0000", 2024-09-14T22:38:12.303 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.370155723000003, 2024-09-14T22:38:12.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047310420999999998, 2024-09-14T22:38:12.303 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:12.303 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:12.303 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:12.303 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:12.303 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:12.304 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:12.304 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 113947 2024-09-14T22:38:12.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.304 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:12.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:12.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.923453+0000", 2024-09-14T22:38:12.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:12.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.923453+0000", 2024-09-14T22:38:12.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:12.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926004+0000", 2024-09-14T22:38:12.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002551382 2024-09-14T22:38:12.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:12.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926009+0000", 2024-09-14T22:38:12.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5639999999999998e-06 2024-09-14T22:38:12.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:12.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926022+0000", 2024-09-14T22:38:12.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2734999999999999e-05 2024-09-14T22:38:12.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.306 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:12.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926029+0000", 2024-09-14T22:38:12.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2520000000000002e-06 2024-09-14T22:38:12.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.306 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:12.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.970650+0000", 2024-09-14T22:38:12.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044621277000000001 2024-09-14T22:38:12.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.306 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:12.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.970700+0000", 2024-09-14T22:38:12.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9713000000000003e-05 2024-09-14T22:38:12.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:12.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.970763+0000", 2024-09-14T22:38:12.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3497999999999994e-05 2024-09-14T22:38:12.307 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.307 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:12.307 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.307 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114136 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3448832~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:12.307 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932179+0000", 2024-09-14T22:38:12.308 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.361429356999999, 2024-09-14T22:38:12.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047236017999999998, 2024-09-14T22:38:12.308 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:12.308 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:12.308 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:12.308 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:12.308 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:12.308 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:12.308 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114136 2024-09-14T22:38:12.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.308 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:12.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:12.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932179+0000", 2024-09-14T22:38:12.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:12.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932179+0000", 2024-09-14T22:38:12.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:12.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932324+0000", 2024-09-14T22:38:12.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014451000000000001 2024-09-14T22:38:12.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:12.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932328+0000", 2024-09-14T22:38:12.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5889999999999996e-06 2024-09-14T22:38:12.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:12.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932338+0000", 2024-09-14T22:38:12.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3000000000000007e-06 2024-09-14T22:38:12.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:12.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932344+0000", 2024-09-14T22:38:12.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7150000000000001e-06 2024-09-14T22:38:12.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:12.311 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979233+0000", 2024-09-14T22:38:12.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046888711 2024-09-14T22:38:12.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.311 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:12.311 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979313+0000", 2024-09-14T22:38:12.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9487000000000007e-05 2024-09-14T22:38:12.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.311 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:12.311 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979415+0000", 2024-09-14T22:38:12.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000102706 2024-09-14T22:38:12.311 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.311 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:12.311 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.312 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114138 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3497984~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:12.312 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932356+0000", 2024-09-14T22:38:12.312 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.361252471, 2024-09-14T22:38:12.312 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047285958000000003, 2024-09-14T22:38:12.312 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:12.312 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:12.312 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:12.312 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:12.312 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:12.312 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:12.312 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114138 2024-09-14T22:38:12.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.313 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:12.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:12.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932356+0000", 2024-09-14T22:38:12.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:12.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932356+0000", 2024-09-14T22:38:12.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:12.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932498+0000", 2024-09-14T22:38:12.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000142182 2024-09-14T22:38:12.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:12.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932503+0000", 2024-09-14T22:38:12.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.301e-06 2024-09-14T22:38:12.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:12.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932514+0000", 2024-09-14T22:38:12.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1484000000000001e-05 2024-09-14T22:38:12.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:12.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932521+0000", 2024-09-14T22:38:12.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0740000000000004e-06 2024-09-14T22:38:12.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.315 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:12.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979451+0000", 2024-09-14T22:38:12.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046929379 2024-09-14T22:38:12.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.315 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:12.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979535+0000", 2024-09-14T22:38:12.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4423000000000001e-05 2024-09-14T22:38:12.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.316 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:12.316 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979642+0000", 2024-09-14T22:38:12.316 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010711499999999999 2024-09-14T22:38:12.316 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.316 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:12.316 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.316 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114139 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3547136~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:12.316 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932554+0000", 2024-09-14T22:38:12.316 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.361054703999997, 2024-09-14T22:38:12.316 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047292230999999997, 2024-09-14T22:38:12.316 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:12.316 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:12.316 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:12.317 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:12.317 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:12.317 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:12.317 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114139 2024-09-14T22:38:12.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.317 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:12.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:12.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932554+0000", 2024-09-14T22:38:12.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:12.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932554+0000", 2024-09-14T22:38:12.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:12.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932819+0000", 2024-09-14T22:38:12.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026538400000000002 2024-09-14T22:38:12.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:12.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932825+0000", 2024-09-14T22:38:12.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3469999999999998e-06 2024-09-14T22:38:12.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:12.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932836+0000", 2024-09-14T22:38:12.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1e-05 2024-09-14T22:38:12.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:12.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932843+0000", 2024-09-14T22:38:12.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1790000000000002e-06 2024-09-14T22:38:12.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:12.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979673+0000", 2024-09-14T22:38:12.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046830389 2024-09-14T22:38:12.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:12.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979750+0000", 2024-09-14T22:38:12.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6312999999999999e-05 2024-09-14T22:38:12.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.320 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:12.320 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979846+0000", 2024-09-14T22:38:12.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6619000000000004e-05 2024-09-14T22:38:12.320 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.320 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:12.320 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.320 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114141 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3596288~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:12.320 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932855+0000", 2024-09-14T22:38:12.320 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.360754106000002, 2024-09-14T22:38:12.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047201870999999999, 2024-09-14T22:38:12.321 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:12.321 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:12.321 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:12.321 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:12.321 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:12.321 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:12.321 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114141 2024-09-14T22:38:12.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.321 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:12.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.321 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:12.321 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932855+0000", 2024-09-14T22:38:12.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:12.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932855+0000", 2024-09-14T22:38:12.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:12.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934033+0000", 2024-09-14T22:38:12.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001178264 2024-09-14T22:38:12.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:12.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934038+0000", 2024-09-14T22:38:12.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4820000000000002e-06 2024-09-14T22:38:12.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:12.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934048+0000", 2024-09-14T22:38:12.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9890000000000004e-06 2024-09-14T22:38:12.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:12.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934055+0000", 2024-09-14T22:38:12.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8329999999999996e-06 2024-09-14T22:38:12.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:12.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979878+0000", 2024-09-14T22:38:12.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045822773999999997 2024-09-14T22:38:12.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:12.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979958+0000", 2024-09-14T22:38:12.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0477999999999998e-05 2024-09-14T22:38:12.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:12.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.980056+0000", 2024-09-14T22:38:12.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8051e-05 2024-09-14T22:38:12.324 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.324 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:12.324 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.325 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114159 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3940352~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:12.325 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.934179+0000", 2024-09-14T22:38:12.325 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.359430070000002, 2024-09-14T22:38:12.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047341439999999999, 2024-09-14T22:38:12.325 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:12.325 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:12.325 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:12.325 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:12.325 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:12.325 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:12.325 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114159 2024-09-14T22:38:12.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.326 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:12.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:12.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934179+0000", 2024-09-14T22:38:12.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:12.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934179+0000", 2024-09-14T22:38:12.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:12.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935021+0000", 2024-09-14T22:38:12.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00084276800000000003 2024-09-14T22:38:12.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:12.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935027+0000", 2024-09-14T22:38:12.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2959999999999998e-06 2024-09-14T22:38:12.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:12.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935038+0000", 2024-09-14T22:38:12.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1198e-05 2024-09-14T22:38:12.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:12.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935054+0000", 2024-09-14T22:38:12.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5764e-05 2024-09-14T22:38:12.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:12.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.981341+0000", 2024-09-14T22:38:12.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.04628782 2024-09-14T22:38:12.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:12.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.981415+0000", 2024-09-14T22:38:12.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3349999999999994e-05 2024-09-14T22:38:12.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:12.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.981520+0000", 2024-09-14T22:38:12.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105244 2024-09-14T22:38:12.329 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.329 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:12.329 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.329 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114180 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4136960~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:12.329 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:38:12.329 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.358430534, 2024-09-14T22:38:12.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047190621000000002, 2024-09-14T22:38:12.329 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:12.330 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:12.330 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:12.330 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:12.330 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:12.330 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:12.330 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114180 2024-09-14T22:38:12.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.330 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:12.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:12.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:38:12.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:12.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:38:12.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:12.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:38:12.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6800000000000002e-07 2024-09-14T22:38:12.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:12.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935180+0000", 2024-09-14T22:38:12.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.42e-06 2024-09-14T22:38:12.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:12.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935185+0000", 2024-09-14T22:38:12.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1359999999999996e-06 2024-09-14T22:38:12.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:12.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935188+0000", 2024-09-14T22:38:12.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.711e-06 2024-09-14T22:38:12.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:12.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982201+0000", 2024-09-14T22:38:12.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047013309000000003 2024-09-14T22:38:12.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:12.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982278+0000", 2024-09-14T22:38:12.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6705000000000005e-05 2024-09-14T22:38:12.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:12.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982369+0000", 2024-09-14T22:38:12.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1071999999999998e-05 2024-09-14T22:38:12.333 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.334 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:12.334 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.334 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114182 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4186112~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:12.334 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:38:12.334 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.358401911999998, 2024-09-14T22:38:12.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047368990999999999, 2024-09-14T22:38:12.334 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:12.334 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:12.334 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:12.334 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:12.334 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:12.335 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:12.335 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114182 2024-09-14T22:38:12.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.335 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:12.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:12.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:38:12.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:12.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:38:12.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:12.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:38:12.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7099999999999998e-07 2024-09-14T22:38:12.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:12.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935209+0000", 2024-09-14T22:38:12.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0260000000000002e-06 2024-09-14T22:38:12.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:12.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935213+0000", 2024-09-14T22:38:12.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3429999999999998e-06 2024-09-14T22:38:12.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:12.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935217+0000", 2024-09-14T22:38:12.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2480000000000001e-06 2024-09-14T22:38:12.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:12.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982406+0000", 2024-09-14T22:38:12.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047188920000000002 2024-09-14T22:38:12.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:12.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982483+0000", 2024-09-14T22:38:12.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7008999999999994e-05 2024-09-14T22:38:12.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:12.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982576+0000", 2024-09-14T22:38:12.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3173999999999999e-05 2024-09-14T22:38:12.338 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.338 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:12.338 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.338 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114185 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4202496~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:12.338 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.935221+0000", 2024-09-14T22:38:12.338 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.358387747000002, 2024-09-14T22:38:12.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047601128999999999, 2024-09-14T22:38:12.338 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:12.339 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:12.339 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:12.339 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:12.339 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:12.339 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:12.339 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114185 2024-09-14T22:38:12.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.339 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:12.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:12.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935221+0000", 2024-09-14T22:38:12.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:12.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935221+0000", 2024-09-14T22:38:12.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:12.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935970+0000", 2024-09-14T22:38:12.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074930899999999998 2024-09-14T22:38:12.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:12.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935976+0000", 2024-09-14T22:38:12.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5439999999999998e-06 2024-09-14T22:38:12.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:12.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935988+0000", 2024-09-14T22:38:12.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1779e-05 2024-09-14T22:38:12.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:12.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935996+0000", 2024-09-14T22:38:12.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.106e-06 2024-09-14T22:38:12.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:12.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982607+0000", 2024-09-14T22:38:12.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046611504999999998 2024-09-14T22:38:12.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:12.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982686+0000", 2024-09-14T22:38:12.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8805000000000002e-05 2024-09-14T22:38:12.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:12.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982822+0000", 2024-09-14T22:38:12.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000136081 2024-09-14T22:38:12.342 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.342 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:12.342 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.343 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114189 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4300800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:12.343 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.936048+0000", 2024-09-14T22:38:12.343 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.357560614999997, 2024-09-14T22:38:12.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047231539000000003, 2024-09-14T22:38:12.343 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:12.343 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:12.343 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:12.343 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:12.343 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:12.343 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:12.343 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114189 2024-09-14T22:38:12.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.343 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:12.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:12.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936048+0000", 2024-09-14T22:38:12.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:12.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936048+0000", 2024-09-14T22:38:12.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:12.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936049+0000", 2024-09-14T22:38:12.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4099999999999999e-07 2024-09-14T22:38:12.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:12.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936052+0000", 2024-09-14T22:38:12.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0029999999999999e-06 2024-09-14T22:38:12.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:12.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936058+0000", 2024-09-14T22:38:12.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7880000000000001e-06 2024-09-14T22:38:12.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:12.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936064+0000", 2024-09-14T22:38:12.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2279999999999998e-06 2024-09-14T22:38:12.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:12.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983120+0000", 2024-09-14T22:38:12.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047056384999999999 2024-09-14T22:38:12.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:12.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983194+0000", 2024-09-14T22:38:12.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3811000000000002e-05 2024-09-14T22:38:12.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:12.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983280+0000", 2024-09-14T22:38:12.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5883000000000004e-05 2024-09-14T22:38:12.347 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.347 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:12.347 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.347 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114191 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4349952~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:12.347 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:38:12.347 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.357536875000001, 2024-09-14T22:38:12.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047365365, 2024-09-14T22:38:12.347 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:12.347 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:12.348 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:12.348 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:12.348 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:12.348 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:12.348 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114191 2024-09-14T22:38:12.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.348 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:12.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:12.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:38:12.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:12.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:38:12.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:12.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:38:12.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0499999999999999e-07 2024-09-14T22:38:12.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:12.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936075+0000", 2024-09-14T22:38:12.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.734e-06 2024-09-14T22:38:12.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:12.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936082+0000", 2024-09-14T22:38:12.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8360000000000003e-06 2024-09-14T22:38:12.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:12.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936087+0000", 2024-09-14T22:38:12.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3129999999999998e-06 2024-09-14T22:38:12.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:12.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983319+0000", 2024-09-14T22:38:12.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047232140999999998 2024-09-14T22:38:12.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:12.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983365+0000", 2024-09-14T22:38:12.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5776999999999999e-05 2024-09-14T22:38:12.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:12.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983437+0000", 2024-09-14T22:38:12.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2158999999999998e-05 2024-09-14T22:38:12.351 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.351 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:12.351 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.351 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114193 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4399104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:12.351 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.936095+0000", 2024-09-14T22:38:12.352 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.357513939, 2024-09-14T22:38:12.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047505915000000003, 2024-09-14T22:38:12.352 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:12.352 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:12.352 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:12.352 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:12.352 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:12.352 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:12.352 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114193 2024-09-14T22:38:12.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.352 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:12.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:12.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936095+0000", 2024-09-14T22:38:12.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:12.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936095+0000", 2024-09-14T22:38:12.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:12.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938021+0000", 2024-09-14T22:38:12.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001926178 2024-09-14T22:38:12.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:12.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938027+0000", 2024-09-14T22:38:12.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3110000000000002e-06 2024-09-14T22:38:12.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:12.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938041+0000", 2024-09-14T22:38:12.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3344e-05 2024-09-14T22:38:12.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:12.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938049+0000", 2024-09-14T22:38:12.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2279999999999998e-06 2024-09-14T22:38:12.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:12.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983468+0000", 2024-09-14T22:38:12.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045419494999999997 2024-09-14T22:38:12.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:12.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983524+0000", 2024-09-14T22:38:12.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5952000000000002e-05 2024-09-14T22:38:12.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:12.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983601+0000", 2024-09-14T22:38:12.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6407000000000002e-05 2024-09-14T22:38:12.355 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.356 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:12.356 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.356 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114710 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4857856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:12.356 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.962571+0000", 2024-09-14T22:38:12.356 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.331037666999997, 2024-09-14T22:38:12.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047253314999999997, 2024-09-14T22:38:12.356 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:12.356 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:12.356 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:12.356 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:12.356 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:12.356 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:12.357 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114710 2024-09-14T22:38:12.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.357 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:12.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:12.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962571+0000", 2024-09-14T22:38:12.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:12.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962571+0000", 2024-09-14T22:38:12.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:12.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962572+0000", 2024-09-14T22:38:12.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1399999999999997e-07 2024-09-14T22:38:12.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:12.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962574+0000", 2024-09-14T22:38:12.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8679999999999999e-06 2024-09-14T22:38:12.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:12.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962583+0000", 2024-09-14T22:38:12.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8310000000000007e-06 2024-09-14T22:38:12.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:12.359 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962589+0000", 2024-09-14T22:38:12.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9529999999999996e-06 2024-09-14T22:38:12.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:12.359 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009692+0000", 2024-09-14T22:38:12.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047103249999999999 2024-09-14T22:38:12.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:12.359 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009767+0000", 2024-09-14T22:38:12.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4672000000000006e-05 2024-09-14T22:38:12.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:12.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009824+0000", 2024-09-14T22:38:12.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7226999999999999e-05 2024-09-14T22:38:12.360 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.360 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:12.360 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.360 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114712 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4907008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:12.360 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:38:12.360 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.331012186999999, 2024-09-14T22:38:12.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047421087000000001, 2024-09-14T22:38:12.360 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:12.361 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:12.361 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:12.361 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:12.361 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:12.361 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:12.361 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114712 2024-09-14T22:38:12.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.361 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:12.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:12.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:38:12.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:12.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:38:12.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:12.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:38:12.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.03e-07 2024-09-14T22:38:12.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:12.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962599+0000", 2024-09-14T22:38:12.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3199999999999998e-06 2024-09-14T22:38:12.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:12.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962606+0000", 2024-09-14T22:38:12.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5289999999999997e-06 2024-09-14T22:38:12.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:12.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962612+0000", 2024-09-14T22:38:12.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.947e-06 2024-09-14T22:38:12.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:12.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009866+0000", 2024-09-14T22:38:12.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047254681999999999 2024-09-14T22:38:12.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:12.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009924+0000", 2024-09-14T22:38:12.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7556000000000003e-05 2024-09-14T22:38:12.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:12.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010018+0000", 2024-09-14T22:38:12.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.365e-05 2024-09-14T22:38:12.364 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.364 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:12.364 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.364 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114714 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4956160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:12.365 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.962618+0000", 2024-09-14T22:38:12.365 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.330990669000002, 2024-09-14T22:38:12.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.04759302, 2024-09-14T22:38:12.365 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:12.365 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:12.365 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:12.365 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:12.365 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:12.365 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:12.365 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114714 2024-09-14T22:38:12.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.365 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:12.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:12.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962618+0000", 2024-09-14T22:38:12.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:12.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962618+0000", 2024-09-14T22:38:12.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:12.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966011+0000", 2024-09-14T22:38:12.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003393067 2024-09-14T22:38:12.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:12.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966016+0000", 2024-09-14T22:38:12.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3580000000000002e-06 2024-09-14T22:38:12.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:12.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966032+0000", 2024-09-14T22:38:12.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5356e-05 2024-09-14T22:38:12.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:12.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966040+0000", 2024-09-14T22:38:12.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1980000000000001e-06 2024-09-14T22:38:12.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:12.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010055+0000", 2024-09-14T22:38:12.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044014676000000003 2024-09-14T22:38:12.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:12.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010127+0000", 2024-09-14T22:38:12.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2206000000000006e-05 2024-09-14T22:38:12.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:12.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010211+0000", 2024-09-14T22:38:12.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4159000000000005e-05 2024-09-14T22:38:12.368 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.368 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:12.369 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.369 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114754 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 6283264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:12.369 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:38:12.369 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.326815756000002, 2024-09-14T22:38:12.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047228150000000003, 2024-09-14T22:38:12.369 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:12.369 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:12.369 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:12.369 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:12.369 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:12.369 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:12.369 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114754 2024-09-14T22:38:12.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.370 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:12.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:12.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:38:12.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:12.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:38:12.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:12.371 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:38:12.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7399999999999999e-07 2024-09-14T22:38:12.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.371 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:12.371 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966796+0000", 2024-09-14T22:38:12.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3630000000000001e-06 2024-09-14T22:38:12.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.371 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:12.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966802+0000", 2024-09-14T22:38:12.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4679999999999997e-06 2024-09-14T22:38:12.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:12.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966807+0000", 2024-09-14T22:38:12.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1109999999999999e-06 2024-09-14T22:38:12.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:12.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.013929+0000", 2024-09-14T22:38:12.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047121720999999998 2024-09-14T22:38:12.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:12.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.013968+0000", 2024-09-14T22:38:12.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8877000000000001e-05 2024-09-14T22:38:12.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:12.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014021+0000", 2024-09-14T22:38:12.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3236000000000003e-05 2024-09-14T22:38:12.373 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.373 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:12.373 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.373 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114789 3.19 3:9836bb25:::rbd_object_map.113c14c67864.000000000000001b:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:12.374 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.966813+0000", 2024-09-14T22:38:12.374 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.326795294, 2024-09-14T22:38:12.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05001311, 2024-09-14T22:38:12.374 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:12.374 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:12.374 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:12.374 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:12.374 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:12.374 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:12.374 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114789 2024-09-14T22:38:12.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.374 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:12.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:12.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966813+0000", 2024-09-14T22:38:12.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:12.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966813+0000", 2024-09-14T22:38:12.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:12.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966814+0000", 2024-09-14T22:38:12.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9200000000000002e-07 2024-09-14T22:38:12.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:12.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966817+0000", 2024-09-14T22:38:12.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8380000000000002e-06 2024-09-14T22:38:12.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:12.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966831+0000", 2024-09-14T22:38:12.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4209999999999999e-05 2024-09-14T22:38:12.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:12.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966845+0000", 2024-09-14T22:38:12.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3799e-05 2024-09-14T22:38:12.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:12.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014046+0000", 2024-09-14T22:38:12.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047201604000000001 2024-09-14T22:38:12.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:12.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014116+0000", 2024-09-14T22:38:12.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9387000000000005e-05 2024-09-14T22:38:12.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-14T22:38:12.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014469+0000", 2024-09-14T22:38:12.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00035355800000000002 2024-09-14T22:38:12.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:12.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016719+0000", 2024-09-14T22:38:12.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0022498459999999998 2024-09-14T22:38:12.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:38:12.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016735+0000", 2024-09-14T22:38:12.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5792999999999999e-05 2024-09-14T22:38:12.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:38:12.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016790+0000", 2024-09-14T22:38:12.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4715e-05 2024-09-14T22:38:12.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.379 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:12.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016827+0000", 2024-09-14T22:38:12.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6968000000000002e-05 2024-09-14T22:38:12.379 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.379 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:12.379 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.379 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114803 3.19 3:9836bb25:::rbd_object_map.113c14c67864.000000000000001b:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:12.379 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.966853+0000", 2024-09-14T22:38:12.379 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.326755863999999, 2024-09-14T22:38:12.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050656093999999999, 2024-09-14T22:38:12.379 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:12.380 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:12.380 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:12.380 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:12.380 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:12.380 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:12.380 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114803 2024-09-14T22:38:12.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.380 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:12.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:12.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966853+0000", 2024-09-14T22:38:12.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:12.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966853+0000", 2024-09-14T22:38:12.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:12.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969930+0000", 2024-09-14T22:38:12.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030769759999999999 2024-09-14T22:38:12.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:12.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969936+0000", 2024-09-14T22:38:12.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7150000000000003e-06 2024-09-14T22:38:12.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:12.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969949+0000", 2024-09-14T22:38:12.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3166000000000001e-05 2024-09-14T22:38:12.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:12.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969957+0000", 2024-09-14T22:38:12.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8269999999999998e-06 2024-09-14T22:38:12.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:12.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014996+0000", 2024-09-14T22:38:12.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045039202 2024-09-14T22:38:12.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:12.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.015085+0000", 2024-09-14T22:38:12.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8987999999999996e-05 2024-09-14T22:38:12.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-14T22:38:12.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.015318+0000", 2024-09-14T22:38:12.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023284999999999999 2024-09-14T22:38:12.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:12.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016898+0000", 2024-09-14T22:38:12.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0015807379999999999 2024-09-14T22:38:12.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:38:12.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.017455+0000", 2024-09-14T22:38:12.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00055664600000000005 2024-09-14T22:38:12.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:38:12.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.017483+0000", 2024-09-14T22:38:12.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8238e-05 2024-09-14T22:38:12.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:12.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.017509+0000", 2024-09-14T22:38:12.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5748000000000001e-05 2024-09-14T22:38:12.385 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.385 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:12.385 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.385 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:150759 3.8 3:15dc94e5:::rbd_header.113c88f88d4a:head [create,call rbd.create in=9b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e113)", 2024-09-14T22:38:12.385 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:38:05.726302+0000", 2024-09-14T22:38:12.385 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.5673071719999996, 2024-09-14T22:38:12.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0241160819999999, 2024-09-14T22:38:12.385 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:12.385 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:12.385 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:12.385 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:12.386 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:12.386 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:12.386 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 150759 2024-09-14T22:38:12.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.386 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:12.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:12.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.726302+0000", 2024-09-14T22:38:12.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:12.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.726302+0000", 2024-09-14T22:38:12.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:12.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:12.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.726304+0000", 2024-09-14T22:38:12.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.075e-06 2024-09-14T22:38:12.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:12.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.726312+0000", 2024-09-14T22:38:12.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2050000000000002e-06 2024-09-14T22:38:12.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:12.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.727969+0000", 2024-09-14T22:38:12.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001657499 2024-09-14T22:38:12.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:12.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.727985+0000", 2024-09-14T22:38:12.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6082999999999998e-05 2024-09-14T22:38:12.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:12.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.728193+0000", 2024-09-14T22:38:12.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000208082 2024-09-14T22:38:12.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:12.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.728205+0000", 2024-09-14T22:38:12.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1897e-05 2024-09-14T22:38:12.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:12.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.741926+0000", 2024-09-14T22:38:12.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.013720591 2024-09-14T22:38:12.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:12.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.742076+0000", 2024-09-14T22:38:12.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015024099999999999 2024-09-14T22:38:12.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-14T22:38:12.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.742415+0000", 2024-09-14T22:38:12.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00033887700000000002 2024-09-14T22:38:12.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:38:12.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.743436+0000", 2024-09-14T22:38:12.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010210639999999999 2024-09-14T22:38:12.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:12.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.750216+0000", 2024-09-14T22:38:12.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0067798950000000002 2024-09-14T22:38:12.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:38:12.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.750288+0000", 2024-09-14T22:38:12.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2192000000000003e-05 2024-09-14T22:38:12.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:12.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:12.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:12.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.750418+0000", 2024-09-14T22:38:12.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000129381 2024-09-14T22:38:12.391 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.391 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:12.391 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.391 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:12.391 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:12.391 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:12.393 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:38:12.393 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:38:12.393 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:38:12.394 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:38:12.394 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:38:12.394 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:38:12.394 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:38:12.395 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:38:12.395 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:38:12.395 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:38:12.395 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:38:12.396 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:12.386+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:12.413 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:38:12.414 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:38:12.435 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:38:12.435 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:38:12.456 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:38:12.457 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:38:12.493 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:12.486+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:12.544 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:12.636 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:12.637 INFO:teuthology.orchestra.run.smithi001.stdout:5802 write 0x92c57d8 thru 0x92c8012 (0x283b bytes) 2024-09-14T22:38:12.638 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:38:12.638 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:12.639 INFO:teuthology.orchestra.run.smithi001.stdout:5804 read 0x76178f3 thru 0x762049f (0x8bad bytes) 2024-09-14T22:38:12.639 INFO:teuthology.orchestra.run.smithi001.stdout:5805 write 0x2dbc6b4 thru 0x2dc83e1 (0xbd2e bytes) 2024-09-14T22:38:12.639 INFO:teuthology.orchestra.run.smithi001.stdout:5806 punch from 0x476e555 to 0x476eabb, (0x566 bytes) 2024-09-14T22:38:12.639 INFO:teuthology.orchestra.run.smithi001.stdout:5807 write 0x7c1d699 thru 0x7c2d21b (0xfb83 bytes) 2024-09-14T22:38:12.639 INFO:teuthology.orchestra.run.smithi001.stdout:5808 trunc from 0xd78991c to 0x7b4c5af 2024-09-14T22:38:12.639 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:12.634+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:12.647 INFO:teuthology.orchestra.run.smithi001.stdout:5809 trunc from 0x7b4c5af to 0xdf1ca5f 2024-09-14T22:38:12.663 INFO:teuthology.orchestra.run.smithi001.stdout:5811 punch from 0x2f29bf4 to 0x2f39952, (0xfd5e bytes) 2024-09-14T22:38:12.663 INFO:teuthology.orchestra.run.smithi001.stdout:5813 write 0xcdbf233 thru 0xcdc31b9 (0x3f87 bytes) 2024-09-14T22:38:12.672 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:12.672 INFO:teuthology.orchestra.run.smithi001.stdout:5815 read 0x8cc08ef thru 0x8cc4b2f (0x4241 bytes) 2024-09-14T22:38:12.672 INFO:teuthology.orchestra.run.smithi001.stdout:5816 read 0x8ebb453 thru 0x8ec428f (0x8e3d bytes) 2024-09-14T22:38:12.672 INFO:teuthology.orchestra.run.smithi001.stdout:5817 write 0xb4909e4 thru 0xb491a84 (0x10a1 bytes) 2024-09-14T22:38:12.678 INFO:teuthology.orchestra.run.smithi001.stdout:5819 read 0x573ab5a thru 0x574aae0 (0xff87 bytes) 2024-09-14T22:38:12.700 INFO:teuthology.orchestra.run.smithi001.stdout:5820 trunc from 0xdf1ca5f to 0x21e3f7c 2024-09-14T22:38:12.734 INFO:teuthology.orchestra.run.smithi001.stdout:5821 write 0x176128b thru 0x176f72a (0xe4a0 bytes) 2024-09-14T22:38:12.740 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:12.734+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:12.744 INFO:teuthology.orchestra.run.smithi001.stdout:5823 punch from 0x30d4e to 0x312e5, (0x597 bytes) 2024-09-14T22:38:12.760 INFO:teuthology.orchestra.run.smithi001.stdout:5825 writesame 0x2d2c6d5 thru 0x2d32214 data_size 0x28(0x5b40 bytes) 2024-09-14T22:38:12.773 INFO:teuthology.orchestra.run.smithi001.stdout:5826 writesame 0x2ee3cef thru 0x2eeafd4 data_size 0x16(0x72e6 bytes) 2024-09-14T22:38:12.785 INFO:teuthology.orchestra.run.smithi001.stdout:5827 trunc from 0x2eeafd5 to 0x853b7aa 2024-09-14T22:38:12.789 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:12.789 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:12.798 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:12.804 INFO:teuthology.orchestra.run.smithi001.stdout:5830 trunc from 0x853b7aa to 0x5e75e34 2024-09-14T22:38:12.805 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:38:12.819 INFO:teuthology.orchestra.run.smithi001.stdout:5831 write 0x5b28c2b thru 0x5b34943 (0xbd19 bytes) 2024-09-14T22:38:12.852 INFO:teuthology.orchestra.run.smithi001.stdout:5833 write 0xac413ed thru 0xac4b179 (0x9d8d bytes) 2024-09-14T22:38:12.852 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:12.846+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:12.859 INFO:teuthology.orchestra.run.smithi001.stdout:5834 clone 39 order 18 su 131072 sc 5 2024-09-14T22:38:12.941 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:12.934+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:12.952 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:12.952 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:12.967 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:38:13.041 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:13.038+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:13.107 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:13.107 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:13.122 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:38:13.136 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone38 from 0xac4b17a (overlap 0x21e3f7c) to 0xa2aca5 2024-09-14T22:38:13.167 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:13.162+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:13.243 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:13.238+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:13.243 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:13.243 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:13.257 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:38:13.335 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:38:13.351 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-14T22:38:13.351 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:13.346+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:13.422 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:13.422 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:13.436 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:38:13.483 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:13.478+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:13.546 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:13.542+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:13.561 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:13.561 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:13.575 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:38:13.647 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:13.642+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:13.656 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:38:13.656 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-14T22:38:13.676 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2024-09-14T22:38:13.002429+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1384900561,"num_objects":1859,"num_object_clones":465,"num_object_copies":3718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1859,"num_whiteouts":734,"num_read":188922,"num_read_kb":782371,"num_write":22126,"num_write_kb":381014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3299,"num_bytes_recovered":173602381,"num_keys_recovered":342,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16841,"ondisk_log_size":16841,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":442352,"kb_used_data":63684,"kb_used_omap":17,"kb_used_meta":378478,"kb_avail":754532368,"statfs":{"total":773094113280,"available":772641144832,"internally_reserved":0,"allocated":65212416,"data_stored":506714228,"data_compressed":14244193,"data_compressed_allocated":43364352,"data_compressed_original":495178731,"omap_allocated":18180,"internal_metadata":387561724},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17658631,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-131,"num_objects_misplaced":-216,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":1270,"num_read_kb":20047,"num_write":294,"num_write_kb":7459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":649,"num_bytes_recovered":31769304,"num_keys_recovered":57,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.108712"},"pg_stats":[{"pgid":"3.1f","version":"119'367","reported_seq":4752,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.590916+0000","last_change":"2024-09-14T22:37:25.739233+0000","last_active":"2024-09-14T22:38:12.590916+0000","last_peered":"2024-09-14T22:38:12.590916+0000","last_clean":"2024-09-14T22:38:12.590916+0000","last_became_active":"2024-09-14T22:37:25.718906+0000","last_became_peered":"2024-09-14T22:37:25.718906+0000","last_unstale":"2024-09-14T22:38:12.590916+0000","last_undegraded":"2024-09-14T22:38:12.590916+0000","last_fullsized":"2024-09-14T22:38:12.590916+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:47:47.368174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017190879999999999,"stat_sum":{"num_bytes":55797417,"num_objects":60,"num_object_clones":16,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":26,"num_read":4759,"num_read_kb":16316,"num_write":373,"num_write_kb":7115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":3277632,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"118'1040","reported_seq":5819,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:11.992258+0000","last_change":"2024-09-14T22:38:08.744177+0000","last_active":"2024-09-14T22:38:11.992258+0000","last_peered":"2024-09-14T22:38:11.992258+0000","last_clean":"2024-09-14T22:38:11.992258+0000","last_became_active":"2024-09-14T22:38:08.741149+0000","last_became_peered":"2024-09-14T22:38:08.741149+0000","last_unstale":"2024-09-14T22:38:11.992258+0000","last_undegraded":"2024-09-14T22:38:11.992258+0000","last_fullsized":"2024-09-14T22:38:11.992258+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1040,"log_dups_size":0,"ondisk_log_size":1040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:20:39.731232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001723619,"stat_sum":{"num_bytes":76601511,"num_objects":56,"num_object_clones":15,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":22,"num_read":10210,"num_read_kb":24720,"num_write":1588,"num_write_kb":10683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":7412288,"num_keys_recovered":90,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"118'583","reported_seq":9977,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-14T22:38:10.736644+0000","last_change":"2024-09-14T22:37:51.741030+0000","last_active":"2024-09-14T22:38:10.736644+0000","last_peered":"2024-09-14T22:38:10.736644+0000","last_clean":"2024-09-14T22:38:10.736644+0000","last_became_active":"2024-09-14T22:37:51.730971+0000","last_became_peered":"2024-09-14T22:37:51.730971+0000","last_unstale":"2024-09-14T22:38:10.736644+0000","last_undegraded":"2024-09-14T22:38:10.736644+0000","last_fullsized":"2024-09-14T22:38:10.736644+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:50:14.998252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053914200000000005,"stat_sum":{"num_bytes":76488346,"num_objects":68,"num_object_clones":22,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":31,"num_read":10660,"num_read_kb":63051,"num_write":799,"num_write_kb":13692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":8030815,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"119'557","reported_seq":5730,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.618057+0000","last_change":"2024-09-14T22:36:17.702413+0000","last_active":"2024-09-14T22:38:12.618057+0000","last_peered":"2024-09-14T22:38:12.618057+0000","last_clean":"2024-09-14T22:38:12.618057+0000","last_became_active":"2024-09-14T22:36:15.719470+0000","last_became_peered":"2024-09-14T22:36:15.719470+0000","last_unstale":"2024-09-14T22:38:12.618057+0000","last_undegraded":"2024-09-14T22:38:12.618057+0000","last_fullsized":"2024-09-14T22:38:12.618057+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":557,"log_dups_size":0,"ondisk_log_size":557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:58:05.414233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055079499999999997,"stat_sum":{"num_bytes":35950567,"num_objects":59,"num_object_clones":15,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":26,"num_read":6008,"num_read_kb":26114,"num_write":733,"num_write_kb":8524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"118'471","reported_seq":4882,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:11.989100+0000","last_change":"2024-09-14T22:37:25.736579+0000","last_active":"2024-09-14T22:38:11.989100+0000","last_peered":"2024-09-14T22:38:11.989100+0000","last_clean":"2024-09-14T22:38:11.989100+0000","last_became_active":"2024-09-14T22:37:25.718519+0000","last_became_peered":"2024-09-14T22:37:25.718519+0000","last_unstale":"2024-09-14T22:38:11.989100+0000","last_undegraded":"2024-09-14T22:38:11.989100+0000","last_fullsized":"2024-09-14T22:38:11.989100+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:09:56.374762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092291499999999996,"stat_sum":{"num_bytes":30938989,"num_objects":50,"num_object_clones":14,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":21,"num_read":5129,"num_read_kb":32004,"num_write":571,"num_write_kb":13306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":7858104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"118'406","reported_seq":3347,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.849338+0000","last_change":"2024-09-14T22:38:10.745128+0000","last_active":"2024-09-14T22:38:12.849338+0000","last_peered":"2024-09-14T22:38:12.849338+0000","last_clean":"2024-09-14T22:38:12.849338+0000","last_became_active":"2024-09-14T22:38:10.742092+0000","last_became_peered":"2024-09-14T22:38:10.742092+0000","last_unstale":"2024-09-14T22:38:12.849338+0000","last_undegraded":"2024-09-14T22:38:12.849338+0000","last_fullsized":"2024-09-14T22:38:12.849338+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:44:24.632361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012502780000000001,"stat_sum":{"num_bytes":11058244,"num_objects":68,"num_object_clones":19,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":34,"num_read":3864,"num_read_kb":12494,"num_write":410,"num_write_kb":6721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":10036538,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"110'363","reported_seq":5112,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-14T22:38:10.742406+0000","last_change":"2024-09-14T22:36:17.705423+0000","last_active":"2024-09-14T22:38:10.742406+0000","last_peered":"2024-09-14T22:38:10.742406+0000","last_clean":"2024-09-14T22:38:10.742406+0000","last_became_active":"2024-09-14T22:36:15.700990+0000","last_became_peered":"2024-09-14T22:36:15.700990+0000","last_unstale":"2024-09-14T22:38:10.742406+0000","last_undegraded":"2024-09-14T22:38:10.742406+0000","last_fullsized":"2024-09-14T22:38:10.742406+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:55:28.951912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010520620000000001,"stat_sum":{"num_bytes":22178140,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":23,"num_read":5058,"num_read_kb":15226,"num_write":369,"num_write_kb":5409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"118'405","reported_seq":4451,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:11.992236+0000","last_change":"2024-09-14T22:38:07.735828+0000","last_active":"2024-09-14T22:38:11.992236+0000","last_peered":"2024-09-14T22:38:11.992236+0000","last_clean":"2024-09-14T22:38:11.992236+0000","last_became_active":"2024-09-14T22:38:07.734308+0000","last_became_peered":"2024-09-14T22:38:07.734308+0000","last_unstale":"2024-09-14T22:38:11.992236+0000","last_undegraded":"2024-09-14T22:38:11.992236+0000","last_fullsized":"2024-09-14T22:38:11.992236+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:17:54.535972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002063484,"stat_sum":{"num_bytes":36374590,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":14,"num_read":4705,"num_read_kb":35743,"num_write":569,"num_write_kb":27694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":20067293,"num_keys_recovered":21,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"119'349","reported_seq":3848,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.855763+0000","last_change":"2024-09-14T22:36:17.721195+0000","last_active":"2024-09-14T22:38:12.855763+0000","last_peered":"2024-09-14T22:38:12.855763+0000","last_clean":"2024-09-14T22:38:12.855763+0000","last_became_active":"2024-09-14T22:36:15.715869+0000","last_became_peered":"2024-09-14T22:36:15.715869+0000","last_unstale":"2024-09-14T22:38:12.855763+0000","last_undegraded":"2024-09-14T22:38:12.855763+0000","last_fullsized":"2024-09-14T22:38:12.855763+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:08:39.994970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001160672,"stat_sum":{"num_bytes":23103658,"num_objects":56,"num_object_clones":10,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":24,"num_read":3873,"num_read_kb":17259,"num_write":437,"num_write_kb":21983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"119'474","reported_seq":6175,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.670227+0000","last_change":"2024-09-14T22:36:17.703048+0000","last_active":"2024-09-14T22:38:12.670227+0000","last_peered":"2024-09-14T22:38:12.670227+0000","last_clean":"2024-09-14T22:38:12.670227+0000","last_became_active":"2024-09-14T22:36:15.725078+0000","last_became_peered":"2024-09-14T22:36:15.725078+0000","last_unstale":"2024-09-14T22:38:12.670227+0000","last_undegraded":"2024-09-14T22:38:12.670227+0000","last_fullsized":"2024-09-14T22:38:12.670227+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:47:39.115115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010500290000000001,"stat_sum":{"num_bytes":72882446,"num_objects":66,"num_object_clones":18,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":23,"num_read":6586,"num_read_kb":51137,"num_write":655,"num_write_kb":18943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"118'287","reported_seq":3350,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.004653+0000","last_change":"2024-09-14T22:36:18.009371+0000","last_active":"2024-09-14T22:38:12.004653+0000","last_peered":"2024-09-14T22:38:12.004653+0000","last_clean":"2024-09-14T22:38:12.004653+0000","last_became_active":"2024-09-14T22:36:15.717341+0000","last_became_peered":"2024-09-14T22:36:15.717341+0000","last_unstale":"2024-09-14T22:38:12.004653+0000","last_undegraded":"2024-09-14T22:38:12.004653+0000","last_fullsized":"2024-09-14T22:38:12.004653+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:18:04.062775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054571999999999997,"stat_sum":{"num_bytes":14418034,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":17,"num_read":3100,"num_read_kb":5383,"num_write":300,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"119'1020","reported_seq":8336,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.859757+0000","last_change":"2024-09-14T22:38:10.748696+0000","last_active":"2024-09-14T22:38:12.859757+0000","last_peered":"2024-09-14T22:38:12.859757+0000","last_clean":"2024-09-14T22:38:12.859757+0000","last_became_active":"2024-09-14T22:38:10.741866+0000","last_became_peered":"2024-09-14T22:38:10.741866+0000","last_unstale":"2024-09-14T22:38:12.859757+0000","last_undegraded":"2024-09-14T22:38:12.859757+0000","last_fullsized":"2024-09-14T22:38:12.859757+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1020,"log_dups_size":0,"ondisk_log_size":1020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:38:06.421661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010804009999999999,"stat_sum":{"num_bytes":44119068,"num_objects":72,"num_object_clones":19,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":27,"num_read":10279,"num_read_kb":28738,"num_write":1500,"num_write_kb":11862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":17498216,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":213,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:11.991311+0000","last_change":"2024-09-14T22:36:08.042860+0000","last_active":"2024-09-14T22:38:11.991311+0000","last_peered":"2024-09-14T22:38:11.991311+0000","last_clean":"2024-09-14T22:38:11.991311+0000","last_became_active":"2024-09-14T22:36:06.040961+0000","last_became_peered":"2024-09-14T22:36:06.040961+0000","last_unstale":"2024-09-14T22:38:11.991311+0000","last_undegraded":"2024-09-14T22:38:11.991311+0000","last_fullsized":"2024-09-14T22:38:11.991311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:03:48.547834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010780950000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"119'462","reported_seq":2550,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.619229+0000","last_change":"2024-09-14T22:36:17.720897+0000","last_active":"2024-09-14T22:38:12.619229+0000","last_peered":"2024-09-14T22:38:12.619229+0000","last_clean":"2024-09-14T22:38:12.619229+0000","last_became_active":"2024-09-14T22:36:15.700982+0000","last_became_peered":"2024-09-14T22:36:15.700982+0000","last_unstale":"2024-09-14T22:38:12.619229+0000","last_undegraded":"2024-09-14T22:38:12.619229+0000","last_fullsized":"2024-09-14T22:38:12.619229+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:08:05.131342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010495459999999999,"stat_sum":{"num_bytes":30715546,"num_objects":67,"num_object_clones":17,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":24,"num_read":3560,"num_read_kb":14527,"num_write":495,"num_write_kb":11884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":173,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:11.991302+0000","last_change":"2024-09-14T22:38:03.977889+0000","last_active":"2024-09-14T22:38:11.991302+0000","last_peered":"2024-09-14T22:38:11.991302+0000","last_clean":"2024-09-14T22:38:11.991302+0000","last_became_active":"2024-09-14T22:38:03.976899+0000","last_became_peered":"2024-09-14T22:38:03.976899+0000","last_unstale":"2024-09-14T22:38:11.991302+0000","last_undegraded":"2024-09-14T22:38:11.991302+0000","last_fullsized":"2024-09-14T22:38:11.991302+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:00:41.381700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"119'410","reported_seq":5187,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.670868+0000","last_change":"2024-09-14T22:36:21.722517+0000","last_active":"2024-09-14T22:38:12.670868+0000","last_peered":"2024-09-14T22:38:12.670868+0000","last_clean":"2024-09-14T22:38:12.670868+0000","last_became_active":"2024-09-14T22:36:21.722085+0000","last_became_peered":"2024-09-14T22:36:21.722085+0000","last_unstale":"2024-09-14T22:38:12.670868+0000","last_undegraded":"2024-09-14T22:38:12.670868+0000","last_fullsized":"2024-09-14T22:38:12.670868+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:50:44.432096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020774109999999999,"stat_sum":{"num_bytes":52168094,"num_objects":52,"num_object_clones":11,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":21,"num_read":5685,"num_read_kb":28635,"num_write":477,"num_write_kb":8482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":214,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.004909+0000","last_change":"2024-09-14T22:36:08.054606+0000","last_active":"2024-09-14T22:38:12.004909+0000","last_peered":"2024-09-14T22:38:12.004909+0000","last_clean":"2024-09-14T22:38:12.004909+0000","last_became_active":"2024-09-14T22:36:06.266908+0000","last_became_peered":"2024-09-14T22:36:06.266908+0000","last_unstale":"2024-09-14T22:38:12.004909+0000","last_undegraded":"2024-09-14T22:38:12.004909+0000","last_fullsized":"2024-09-14T22:38:12.004909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:36.738643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098249300000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"118'1320","reported_seq":4881,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-14T22:38:10.736563+0000","last_change":"2024-09-14T22:37:17.676125+0000","last_active":"2024-09-14T22:38:10.736563+0000","last_peered":"2024-09-14T22:38:10.736563+0000","last_clean":"2024-09-14T22:38:10.736563+0000","last_became_active":"2024-09-14T22:37:17.674322+0000","last_became_peered":"2024-09-14T22:37:17.674322+0000","last_unstale":"2024-09-14T22:38:10.736563+0000","last_undegraded":"2024-09-14T22:38:10.736563+0000","last_fullsized":"2024-09-14T22:38:10.736563+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1320,"log_dups_size":0,"ondisk_log_size":1320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:29:16.921603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012564830000000001,"stat_sum":{"num_bytes":27908088,"num_objects":62,"num_object_clones":15,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":26,"num_read":9456,"num_read_kb":23522,"num_write":2357,"num_write_kb":17901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":4264139,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":163,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:11.988088+0000","last_change":"2024-09-14T22:36:41.930961+0000","last_active":"2024-09-14T22:38:11.988088+0000","last_peered":"2024-09-14T22:38:11.988088+0000","last_clean":"2024-09-14T22:38:11.988088+0000","last_became_active":"2024-09-14T22:36:41.930619+0000","last_became_peered":"2024-09-14T22:36:41.930619+0000","last_unstale":"2024-09-14T22:38:11.988088+0000","last_undegraded":"2024-09-14T22:38:11.988088+0000","last_fullsized":"2024-09-14T22:38:11.988088+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:04:40.582999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"111'439","reported_seq":3942,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.355326+0000","last_change":"2024-09-14T22:38:09.741451+0000","last_active":"2024-09-14T22:38:12.355326+0000","last_peered":"2024-09-14T22:38:12.355326+0000","last_clean":"2024-09-14T22:38:12.355326+0000","last_became_active":"2024-09-14T22:38:09.740623+0000","last_became_peered":"2024-09-14T22:38:09.740623+0000","last_unstale":"2024-09-14T22:38:12.355326+0000","last_undegraded":"2024-09-14T22:38:12.355326+0000","last_fullsized":"2024-09-14T22:38:12.355326+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:03:29.131235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071220099999999998,"stat_sum":{"num_bytes":28456380,"num_objects":53,"num_object_clones":14,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":21,"num_read":4394,"num_read_kb":13382,"num_write":480,"num_write_kb":5683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":5797306,"num_keys_recovered":72,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"119'321","reported_seq":4774,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.355910+0000","last_change":"2024-09-14T22:36:17.711068+0000","last_active":"2024-09-14T22:38:12.355910+0000","last_peered":"2024-09-14T22:38:12.355910+0000","last_clean":"2024-09-14T22:38:12.355910+0000","last_became_active":"2024-09-14T22:36:15.706147+0000","last_became_peered":"2024-09-14T22:36:15.706147+0000","last_unstale":"2024-09-14T22:38:12.355910+0000","last_undegraded":"2024-09-14T22:38:12.355910+0000","last_fullsized":"2024-09-14T22:38:12.355910+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:04:01.813199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082625500000000002,"stat_sum":{"num_bytes":52759702,"num_objects":54,"num_object_clones":16,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":22,"num_read":4331,"num_read_kb":25513,"num_write":390,"num_write_kb":13888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":216,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.023799+0000","last_change":"2024-09-14T22:36:08.055938+0000","last_active":"2024-09-14T22:38:12.023799+0000","last_peered":"2024-09-14T22:38:12.023799+0000","last_clean":"2024-09-14T22:38:12.023799+0000","last_became_active":"2024-09-14T22:36:06.039600+0000","last_became_peered":"2024-09-14T22:36:06.039600+0000","last_unstale":"2024-09-14T22:38:12.023799+0000","last_undegraded":"2024-09-14T22:38:12.023799+0000","last_fullsized":"2024-09-14T22:38:12.023799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:49:09.245563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021617870000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"118'827","reported_seq":8676,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-14T22:38:10.986050+0000","last_change":"2024-09-14T22:37:16.669227+0000","last_active":"2024-09-14T22:38:10.986050+0000","last_peered":"2024-09-14T22:38:10.986050+0000","last_clean":"2024-09-14T22:38:10.986050+0000","last_became_active":"2024-09-14T22:37:16.662230+0000","last_became_peered":"2024-09-14T22:37:16.662230+0000","last_unstale":"2024-09-14T22:38:10.986050+0000","last_undegraded":"2024-09-14T22:38:10.986050+0000","last_fullsized":"2024-09-14T22:38:10.986050+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":827,"log_dups_size":0,"ondisk_log_size":827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:38:34.241178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021500769999999998,"stat_sum":{"num_bytes":79368880,"num_objects":62,"num_object_clones":13,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":18,"num_read":9986,"num_read_kb":29516,"num_write":1160,"num_write_kb":14916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":4851034,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":147,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-14T22:38:10.736384+0000","last_change":"2024-09-14T22:38:05.724182+0000","last_active":"2024-09-14T22:38:10.736384+0000","last_peered":"2024-09-14T22:38:10.736384+0000","last_clean":"2024-09-14T22:38:10.736384+0000","last_became_active":"2024-09-14T22:38:05.723636+0000","last_became_peered":"2024-09-14T22:38:05.723636+0000","last_unstale":"2024-09-14T22:38:10.736384+0000","last_undegraded":"2024-09-14T22:38:10.736384+0000","last_fullsized":"2024-09-14T22:38:10.736384+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:14:09.687576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"119'341","reported_seq":5025,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.355946+0000","last_change":"2024-09-14T22:38:08.744952+0000","last_active":"2024-09-14T22:38:12.355946+0000","last_peered":"2024-09-14T22:38:12.355946+0000","last_clean":"2024-09-14T22:38:12.355946+0000","last_became_active":"2024-09-14T22:38:08.743679+0000","last_became_peered":"2024-09-14T22:38:08.743679+0000","last_unstale":"2024-09-14T22:38:12.355946+0000","last_undegraded":"2024-09-14T22:38:12.355946+0000","last_fullsized":"2024-09-14T22:38:12.355946+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:06:41.843049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085549400000000005,"stat_sum":{"num_bytes":63327300,"num_objects":50,"num_object_clones":13,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":17,"num_read":4244,"num_read_kb":21096,"num_write":357,"num_write_kb":7081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":3679238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":229,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.004152+0000","last_change":"2024-09-14T22:38:03.976161+0000","last_active":"2024-09-14T22:38:12.004152+0000","last_peered":"2024-09-14T22:38:12.004152+0000","last_clean":"2024-09-14T22:38:12.004152+0000","last_became_active":"2024-09-14T22:38:03.975240+0000","last_became_peered":"2024-09-14T22:38:03.975240+0000","last_unstale":"2024-09-14T22:38:12.004152+0000","last_undegraded":"2024-09-14T22:38:12.004152+0000","last_fullsized":"2024-09-14T22:38:12.004152+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:34:10.534483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010817520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"119'998","reported_seq":4189,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.023938+0000","last_change":"2024-09-14T22:38:09.742778+0000","last_active":"2024-09-14T22:38:12.023938+0000","last_peered":"2024-09-14T22:38:12.023938+0000","last_clean":"2024-09-14T22:38:12.023938+0000","last_became_active":"2024-09-14T22:38:09.741015+0000","last_became_peered":"2024-09-14T22:38:09.741015+0000","last_unstale":"2024-09-14T22:38:12.023938+0000","last_undegraded":"2024-09-14T22:38:12.023938+0000","last_fullsized":"2024-09-14T22:38:12.023938+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":998,"log_dups_size":0,"ondisk_log_size":998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:41:12.099183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57348378,"num_objects":67,"num_object_clones":14,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":24,"num_read":7203,"num_read_kb":15917,"num_write":1504,"num_write_kb":11829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":2936757,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":222,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.023772+0000","last_change":"2024-09-14T22:37:13.625072+0000","last_active":"2024-09-14T22:38:12.023772+0000","last_peered":"2024-09-14T22:38:12.023772+0000","last_clean":"2024-09-14T22:38:12.023772+0000","last_became_active":"2024-09-14T22:37:13.624669+0000","last_became_peered":"2024-09-14T22:37:13.624669+0000","last_unstale":"2024-09-14T22:38:12.023772+0000","last_undegraded":"2024-09-14T22:38:12.023772+0000","last_fullsized":"2024-09-14T22:38:12.023772+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:54:20.545098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011401880000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":365,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.004179+0000","last_change":"2024-09-14T22:36:00.402862+0000","last_active":"2024-09-14T22:38:12.004179+0000","last_peered":"2024-09-14T22:38:12.004179+0000","last_clean":"2024-09-14T22:38:12.004179+0000","last_became_active":"2024-09-14T22:36:00.400529+0000","last_became_peered":"2024-09-14T22:36:00.400529+0000","last_unstale":"2024-09-14T22:38:12.004179+0000","last_undegraded":"2024-09-14T22:38:12.004179+0000","last_fullsized":"2024-09-14T22:38:12.004179+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_clean_scrub_stamp":"2024-09-14T22:35:58.989025+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:07:12.951728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"119'561","reported_seq":7095,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.269979+0000","last_change":"2024-09-14T22:38:08.741268+0000","last_active":"2024-09-14T22:38:12.269979+0000","last_peered":"2024-09-14T22:38:12.269979+0000","last_clean":"2024-09-14T22:38:12.269979+0000","last_became_active":"2024-09-14T22:38:08.739090+0000","last_became_peered":"2024-09-14T22:38:08.739090+0000","last_unstale":"2024-09-14T22:38:12.269979+0000","last_undegraded":"2024-09-14T22:38:12.269979+0000","last_fullsized":"2024-09-14T22:38:12.269979+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:04:49.078208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001381511,"stat_sum":{"num_bytes":39365030,"num_objects":59,"num_object_clones":15,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":21,"num_read":7944,"num_read_kb":21366,"num_write":882,"num_write_kb":14459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":18690892,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"118'334","reported_seq":6958,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.277460+0000","last_change":"2024-09-14T22:38:09.737459+0000","last_active":"2024-09-14T22:38:12.277460+0000","last_peered":"2024-09-14T22:38:12.277460+0000","last_clean":"2024-09-14T22:38:12.277460+0000","last_became_active":"2024-09-14T22:38:09.736445+0000","last_became_peered":"2024-09-14T22:38:09.736445+0000","last_unstale":"2024-09-14T22:38:12.277460+0000","last_undegraded":"2024-09-14T22:38:12.277460+0000","last_fullsized":"2024-09-14T22:38:12.277460+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:38:15.357704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060272699999999995,"stat_sum":{"num_bytes":62873388,"num_objects":80,"num_object_clones":20,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":32,"num_read":6782,"num_read_kb":18461,"num_write":357,"num_write_kb":6976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":317,"num_bytes_recovered":16527933,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"119'629","reported_seq":3708,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.079686+0000","last_change":"2024-09-14T22:36:17.776247+0000","last_active":"2024-09-14T22:38:12.079686+0000","last_peered":"2024-09-14T22:38:12.079686+0000","last_clean":"2024-09-14T22:38:12.079686+0000","last_became_active":"2024-09-14T22:36:15.718794+0000","last_became_peered":"2024-09-14T22:36:15.718794+0000","last_unstale":"2024-09-14T22:38:12.079686+0000","last_undegraded":"2024-09-14T22:38:12.079686+0000","last_fullsized":"2024-09-14T22:38:12.079686+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:28.867934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088662699999999997,"stat_sum":{"num_bytes":29108656,"num_objects":58,"num_object_clones":12,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":25,"num_read":5745,"num_read_kb":24933,"num_write":628,"num_write_kb":14488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"119'382","reported_seq":2234,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.572410+0000","last_change":"2024-09-14T22:36:17.733002+0000","last_active":"2024-09-14T22:38:12.572410+0000","last_peered":"2024-09-14T22:38:12.572410+0000","last_clean":"2024-09-14T22:38:12.572410+0000","last_became_active":"2024-09-14T22:36:15.702885+0000","last_became_peered":"2024-09-14T22:36:15.702885+0000","last_unstale":"2024-09-14T22:38:12.572410+0000","last_undegraded":"2024-09-14T22:38:12.572410+0000","last_fullsized":"2024-09-14T22:38:12.572410+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:25:08.258188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015180580000000001,"stat_sum":{"num_bytes":36984245,"num_objects":57,"num_object_clones":17,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":25,"num_read":2775,"num_read_kb":11024,"num_write":350,"num_write_kb":6754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"118'350","reported_seq":6481,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.356011+0000","last_change":"2024-09-14T22:38:08.740002+0000","last_active":"2024-09-14T22:38:12.356011+0000","last_peered":"2024-09-14T22:38:12.356011+0000","last_clean":"2024-09-14T22:38:12.356011+0000","last_became_active":"2024-09-14T22:38:08.738830+0000","last_became_peered":"2024-09-14T22:38:08.738830+0000","last_unstale":"2024-09-14T22:38:12.356011+0000","last_undegraded":"2024-09-14T22:38:12.356011+0000","last_fullsized":"2024-09-14T22:38:12.356011+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:24:20.095876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082782099999999996,"stat_sum":{"num_bytes":50022508,"num_objects":57,"num_object_clones":17,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":24,"num_read":5745,"num_read_kb":33424,"num_write":436,"num_write_kb":17783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":23218688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"119'494","reported_seq":3705,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.581100+0000","last_change":"2024-09-14T22:36:18.009776+0000","last_active":"2024-09-14T22:38:12.581100+0000","last_peered":"2024-09-14T22:38:12.581100+0000","last_clean":"2024-09-14T22:38:12.581100+0000","last_became_active":"2024-09-14T22:36:15.723205+0000","last_became_peered":"2024-09-14T22:36:15.723205+0000","last_unstale":"2024-09-14T22:38:12.581100+0000","last_undegraded":"2024-09-14T22:38:12.581100+0000","last_fullsized":"2024-09-14T22:38:12.581100+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:41:33.559201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087783999999999996,"stat_sum":{"num_bytes":26214933,"num_objects":56,"num_object_clones":15,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":18,"num_read":4045,"num_read_kb":15969,"num_write":635,"num_write_kb":5854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"119'400","reported_seq":2269,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.038773+0000","last_change":"2024-09-14T22:38:09.739942+0000","last_active":"2024-09-14T22:38:12.038773+0000","last_peered":"2024-09-14T22:38:12.038773+0000","last_clean":"2024-09-14T22:38:12.038773+0000","last_became_active":"2024-09-14T22:38:09.738915+0000","last_became_peered":"2024-09-14T22:38:09.738915+0000","last_unstale":"2024-09-14T22:38:12.038773+0000","last_undegraded":"2024-09-14T22:38:12.038773+0000","last_fullsized":"2024-09-14T22:38:12.038773+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:55:44.728937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000776448,"stat_sum":{"num_bytes":28271172,"num_objects":47,"num_object_clones":10,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":19,"num_read":2294,"num_read_kb":8129,"num_write":486,"num_write_kb":3705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":4944418,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"110'349","reported_seq":7872,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.004885+0000","last_change":"2024-09-14T22:36:18.010182+0000","last_active":"2024-09-14T22:38:12.004885+0000","last_peered":"2024-09-14T22:38:12.004885+0000","last_clean":"2024-09-14T22:38:12.004885+0000","last_became_active":"2024-09-14T22:36:15.719693+0000","last_became_peered":"2024-09-14T22:36:15.719693+0000","last_unstale":"2024-09-14T22:38:12.004885+0000","last_undegraded":"2024-09-14T22:38:12.004885+0000","last_fullsized":"2024-09-14T22:38:12.004885+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:38.843525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001045011,"stat_sum":{"num_bytes":25859668,"num_objects":54,"num_object_clones":12,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":16,"num_read":8145,"num_read_kb":23241,"num_write":432,"num_write_kb":9514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"119'376","reported_seq":3585,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:11.991625+0000","last_change":"2024-09-14T22:37:26.718624+0000","last_active":"2024-09-14T22:38:11.991625+0000","last_peered":"2024-09-14T22:38:11.991625+0000","last_clean":"2024-09-14T22:38:11.991625+0000","last_became_active":"2024-09-14T22:37:26.717088+0000","last_became_peered":"2024-09-14T22:37:26.717088+0000","last_unstale":"2024-09-14T22:38:11.991625+0000","last_undegraded":"2024-09-14T22:38:11.991625+0000","last_fullsized":"2024-09-14T22:38:11.991625+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:04:23.164813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094227299999999998,"stat_sum":{"num_bytes":42221837,"num_objects":60,"num_object_clones":17,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":24,"num_read":3287,"num_read_kb":29016,"num_write":475,"num_write_kb":14729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"119'570","reported_seq":2158,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.677675+0000","last_change":"2024-09-14T22:36:18.009347+0000","last_active":"2024-09-14T22:38:12.677675+0000","last_peered":"2024-09-14T22:38:12.677675+0000","last_clean":"2024-09-14T22:38:12.677675+0000","last_became_active":"2024-09-14T22:36:15.715569+0000","last_became_peered":"2024-09-14T22:36:15.715569+0000","last_unstale":"2024-09-14T22:38:12.677675+0000","last_undegraded":"2024-09-14T22:38:12.677675+0000","last_fullsized":"2024-09-14T22:38:12.677675+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:49:48.308579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054921600000000005,"stat_sum":{"num_bytes":31469256,"num_objects":46,"num_object_clones":9,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":21,"num_read":3278,"num_read_kb":11514,"num_write":798,"num_write_kb":10917,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"119'404","reported_seq":7857,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.827674+0000","last_change":"2024-09-14T22:36:17.720448+0000","last_active":"2024-09-14T22:38:12.827674+0000","last_peered":"2024-09-14T22:38:12.827674+0000","last_clean":"2024-09-14T22:38:12.827674+0000","last_became_active":"2024-09-14T22:36:15.716696+0000","last_became_peered":"2024-09-14T22:36:15.716696+0000","last_unstale":"2024-09-14T22:38:12.827674+0000","last_undegraded":"2024-09-14T22:38:12.827674+0000","last_fullsized":"2024-09-14T22:38:12.827674+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:02:57.573600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086547600000000001,"stat_sum":{"num_bytes":64820745,"num_objects":61,"num_object_clones":11,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":22,"num_read":8971,"num_read_kb":49931,"num_write":453,"num_write_kb":13097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"111'432","reported_seq":6982,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.004646+0000","last_change":"2024-09-14T22:38:10.747323+0000","last_active":"2024-09-14T22:38:12.004646+0000","last_peered":"2024-09-14T22:38:12.004646+0000","last_clean":"2024-09-14T22:38:12.004646+0000","last_became_active":"2024-09-14T22:38:10.744013+0000","last_became_peered":"2024-09-14T22:38:10.744013+0000","last_unstale":"2024-09-14T22:38:12.004646+0000","last_undegraded":"2024-09-14T22:38:12.004646+0000","last_fullsized":"2024-09-14T22:38:12.004646+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:28:18.109354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010286189999999999,"stat_sum":{"num_bytes":55135358,"num_objects":67,"num_object_clones":16,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":26,"num_read":6705,"num_read_kb":34970,"num_write":509,"num_write_kb":15408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":14511090,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1384310174,"num_objects":1855,"num_object_clones":465,"num_object_copies":3710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1855,"num_whiteouts":734,"num_read":188806,"num_read_kb":782271,"num_write":21965,"num_write_kb":378596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3299,"num_bytes_recovered":173602381,"num_keys_recovered":342,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53800960,"data_stored":500243010,"data_compressed":14235300,"data_compressed_allocated":43307008,"data_compressed_original":494040043,"omap_allocated":4238,"internal_metadata":0},"log_size":16721,"ondisk_log_size":16721,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542174,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62248,"kb_used_data":8848,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94309592,"statfs":{"total":96636764160,"available":96573022208,"internally_reserved":0,"allocated":9060352,"data_stored":67098004,"data_compressed":2177226,"data_compressed_allocated":5918720,"data_compressed_original":65424589,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542175,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43556,"kb_used_data":4048,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328284,"statfs":{"total":96636764160,"available":96592162816,"internally_reserved":0,"allocated":4145152,"data_stored":34577091,"data_compressed":831425,"data_compressed_allocated":2396160,"data_compressed_original":33731651,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542175,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57420,"kb_used_data":8188,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94314420,"statfs":{"total":96636764160,"available":96577966080,"internally_reserved":0,"allocated":8384512,"data_stored":61568834,"data_compressed":1773874,"data_compressed_allocated":5619712,"data_compressed_original":60098770,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542174,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67836,"kb_used_data":10284,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94304004,"statfs":{"total":96636764160,"available":96567300096,"internally_reserved":0,"allocated":10530816,"data_stored":87237733,"data_compressed":2540736,"data_compressed_allocated":7221248,"data_compressed_original":85465266,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542175,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66064,"kb_used_data":8508,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94305776,"statfs":{"total":96636764160,"available":96569114624,"internally_reserved":0,"allocated":8712192,"data_stored":66173292,"data_compressed":2075267,"data_compressed_allocated":5791744,"data_compressed_original":64612085,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":66,"seq":283467841549,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43004,"kb_used_data":6176,"kb_used_omap":6,"kb_used_meta":36793,"kb_avail":94328836,"statfs":{"total":96636764160,"available":96592728064,"internally_reserved":0,"allocated":6324224,"data_stored":46361629,"data_compressed":1024509,"data_compressed_allocated":4083712,"data_compressed_original":45194103,"omap_allocated":7025,"internal_metadata":37676175},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61168,"kb_used_data":7764,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94310672,"statfs":{"total":96636764160,"available":96574128128,"internally_reserved":0,"allocated":7950336,"data_stored":58881442,"data_compressed":1659920,"data_compressed_allocated":5287936,"data_compressed_original":57466459,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542174,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41056,"kb_used_data":9868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330784,"statfs":{"total":96636764160,"available":96594722816,"internally_reserved":0,"allocated":10104832,"data_stored":84816203,"data_compressed":2161236,"data_compressed_allocated":7045120,"data_compressed_original":83185808,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8732672,"data_stored":83708538,"data_compressed":2216917,"data_compressed_allocated":7086080,"data_compressed_original":82747536,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6541312,"data_stored":58223033,"data_compressed":1659920,"data_compressed_allocated":5287936,"data_compressed_original":57466459,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4775936,"data_stored":45475998,"data_compressed":922200,"data_compressed_allocated":3969024,"data_compressed_original":44981111,"omap_allocated":4238,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7294976,"data_stored":65508730,"data_compressed":2075267,"data_compressed_allocated":5791744,"data_compressed_original":64612085,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":9138176,"data_stored":86617576,"data_compressed":2556118,"data_compressed_allocated":7241728,"data_compressed_original":85506226,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6979584,"data_stored":60923025,"data_compressed":1784145,"data_compressed_allocated":5632000,"data_compressed_original":60115154,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2752512,"data_stored":33943036,"data_compressed":849316,"data_compressed_allocated":2416640,"data_compressed_original":33760323,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":7585792,"data_stored":65843074,"data_compressed":2171417,"data_compressed_allocated":5881856,"data_compressed_original":64851149,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:38:13.678 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-14T22:38:13.678 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:38:13.679 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-14T22:38:13.679 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T22:38:13.739 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:13.739 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:13.747 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:13.742+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:13.754 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:38:13.761 INFO:teuthology.orchestra.run.smithi001.stdout:5835 writesame 0xdb3e1ce thru 0xdb3fa97 data_size 0x13(0x18ca bytes) 2024-09-14T22:38:13.761 INFO:teuthology.orchestra.run.smithi001.stdout:5836 clone 40 order 24 su 8388608 sc 3 2024-09-14T22:38:13.849 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:13.842+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:13.912 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:13.912 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:13.926 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:38:13.950 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:13.946+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:14.017 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T22:38:14.030 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:38:14.030 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:14.030 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 121, 2024-09-14T22:38:14.030 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "f07f407a-b17d-42c4-bf73-92dd80114a4f", 2024-09-14T22:38:14.030 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-09-14T22:35:45.983055+0000", 2024-09-14T22:38:14.031 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-09-14T22:38:14.002738+0000", 2024-09-14T22:38:14.031 INFO:teuthology.orchestra.run.smithi001.stdout: "last_up_change": "2024-09-14T22:37:12.616042+0000", 2024-09-14T22:38:14.031 INFO:teuthology.orchestra.run.smithi001.stdout: "last_in_change": "2024-09-14T22:35:48.283160+0000", 2024-09-14T22:38:14.031 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T22:38:14.031 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_num": 5799936, 2024-09-14T22:38:14.031 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_set": [ 2024-09-14T22:38:14.031 INFO:teuthology.orchestra.run.smithi001.stdout: "pglog_hardlimit", 2024-09-14T22:38:14.031 INFO:teuthology.orchestra.run.smithi001.stdout: "purged_snapdirs", 2024-09-14T22:38:14.031 INFO:teuthology.orchestra.run.smithi001.stdout: "recovery_deletes", 2024-09-14T22:38:14.031 INFO:teuthology.orchestra.run.smithi001.stdout: "sortbitwise" 2024-09-14T22:38:14.031 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:38:14.031 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_version": 6, 2024-09-14T22:38:14.031 INFO:teuthology.orchestra.run.smithi001.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T22:38:14.032 INFO:teuthology.orchestra.run.smithi001.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T22:38:14.032 INFO:teuthology.orchestra.run.smithi001.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T22:38:14.032 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_snapshot": "", 2024-09-14T22:38:14.032 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_max": 3, 2024-09-14T22:38:14.032 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd": 8, 2024-09-14T22:38:14.032 INFO:teuthology.orchestra.run.smithi001.stdout: "require_min_compat_client": "luminous", 2024-09-14T22:38:14.032 INFO:teuthology.orchestra.run.smithi001.stdout: "min_compat_client": "luminous", 2024-09-14T22:38:14.032 INFO:teuthology.orchestra.run.smithi001.stdout: "require_osd_release": "squid", 2024-09-14T22:38:14.032 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_crimson": false, 2024-09-14T22:38:14.033 INFO:teuthology.orchestra.run.smithi001.stdout: "pools": [ 2024-09-14T22:38:14.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.033 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 1, 2024-09-14T22:38:14.033 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": ".mgr", 2024-09-14T22:38:14.033 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-14T22:35:58.938784+0000", 2024-09-14T22:38:14.033 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-14T22:38:14.033 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool", 2024-09-14T22:38:14.033 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-14T22:38:14.033 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-14T22:38:14.033 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-14T22:38:14.033 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-14T22:38:14.033 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:38:14.034 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:38:14.034 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:38:14.034 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:38:14.034 INFO:teuthology.orchestra.run.smithi001.stdout: "is_stretch_pool": false, 2024-09-14T22:38:14.034 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-14T22:38:14.034 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:38:14.034 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 1, 2024-09-14T22:38:14.034 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 1, 2024-09-14T22:38:14.034 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 1, 2024-09-14T22:38:14.034 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 1, 2024-09-14T22:38:14.034 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 1, 2024-09-14T22:38:14.035 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-14T22:38:14.035 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-14T22:38:14.035 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-14T22:38:14.035 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-14T22:38:14.035 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-14T22:38:14.035 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-14T22:38:14.035 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-14T22:38:14.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.035 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "18", 2024-09-14T22:38:14.035 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-14T22:38:14.035 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:38:14.036 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:38:14.036 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-14T22:38:14.036 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:38:14.036 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 0, 2024-09-14T22:38:14.036 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 0, 2024-09-14T22:38:14.036 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-14T22:38:14.036 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-14T22:38:14.036 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-14T22:38:14.037 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-14T22:38:14.037 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-14T22:38:14.037 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-14T22:38:14.037 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-14T22:38:14.037 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-14T22:38:14.037 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-14T22:38:14.037 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-14T22:38:14.037 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-14T22:38:14.037 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:38:14.037 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:38:14.037 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:38:14.037 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-14T22:38:14.038 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-14T22:38:14.038 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-14T22:38:14.038 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-14T22:38:14.038 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-14T22:38:14.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.038 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-14T22:38:14.038 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-14T22:38:14.038 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-14T22:38:14.038 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:38:14.038 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:38:14.038 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:38:14.038 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:38:14.039 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-14T22:38:14.039 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-14T22:38:14.039 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-14T22:38:14.039 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-14T22:38:14.039 INFO:teuthology.orchestra.run.smithi001.stdout: "options": { 2024-09-14T22:38:14.039 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_max": 32, 2024-09-14T22:38:14.039 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_min": 1 2024-09-14T22:38:14.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.039 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-14T22:38:14.039 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": {} 2024-09-14T22:38:14.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.039 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-14T22:38:14.039 INFO:teuthology.orchestra.run.smithi001.stdout: "score_type": "Fair distribution", 2024-09-14T22:38:14.040 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 8, 2024-09-14T22:38:14.040 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 8, 2024-09-14T22:38:14.040 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 0.25, 2024-09-14T22:38:14.040 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 2, 2024-09-14T22:38:14.040 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 2, 2024-09-14T22:38:14.040 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 0.87667274475097656, 2024-09-14T22:38:14.040 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T22:38:14.040 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T22:38:14.040 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.040 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 2, 2024-09-14T22:38:14.040 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "rbd", 2024-09-14T22:38:14.041 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-14T22:36:04.304242+0000", 2024-09-14T22:38:14.041 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 8193, 2024-09-14T22:38:14.041 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T22:38:14.041 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-14T22:38:14.041 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-14T22:38:14.041 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-14T22:38:14.041 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-14T22:38:14.041 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:38:14.041 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:38:14.041 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:38:14.041 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:38:14.041 INFO:teuthology.orchestra.run.smithi001.stdout: "is_stretch_pool": false, 2024-09-14T22:38:14.041 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-14T22:38:14.041 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:38:14.042 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 8, 2024-09-14T22:38:14.042 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 8, 2024-09-14T22:38:14.042 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 8, 2024-09-14T22:38:14.042 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 8, 2024-09-14T22:38:14.042 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 8, 2024-09-14T22:38:14.042 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-14T22:38:14.042 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-14T22:38:14.042 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-14T22:38:14.042 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-14T22:38:14.042 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-14T22:38:14.042 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-14T22:38:14.043 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-14T22:38:14.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.043 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "22", 2024-09-14T22:38:14.043 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-14T22:38:14.043 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:38:14.043 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:38:14.043 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-14T22:38:14.043 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:38:14.043 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 2, 2024-09-14T22:38:14.043 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 22, 2024-09-14T22:38:14.043 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-14T22:38:14.044 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-14T22:38:14.044 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-14T22:38:14.044 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-14T22:38:14.044 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-14T22:38:14.044 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-14T22:38:14.044 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-14T22:38:14.044 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-14T22:38:14.044 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-14T22:38:14.044 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-14T22:38:14.044 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-14T22:38:14.044 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:38:14.044 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:38:14.044 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:38:14.045 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-14T22:38:14.045 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-14T22:38:14.045 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-14T22:38:14.045 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-14T22:38:14.045 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-14T22:38:14.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.045 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-14T22:38:14.045 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-14T22:38:14.045 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-14T22:38:14.045 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:38:14.045 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:38:14.045 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:38:14.045 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:38:14.045 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-14T22:38:14.046 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-14T22:38:14.046 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-14T22:38:14.046 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-14T22:38:14.046 INFO:teuthology.orchestra.run.smithi001.stdout: "options": {}, 2024-09-14T22:38:14.046 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-14T22:38:14.046 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd": {} 2024-09-14T22:38:14.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.046 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-14T22:38:14.046 INFO:teuthology.orchestra.run.smithi001.stdout: "score_type": "Fair distribution", 2024-09-14T22:38:14.046 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 1.9900000095367432, 2024-09-14T22:38:14.046 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 1.9900000095367432, 2024-09-14T22:38:14.046 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 0.87999999523162842, 2024-09-14T22:38:14.046 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 1.75, 2024-09-14T22:38:14.047 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 1.75, 2024-09-14T22:38:14.047 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 0.87667274475097656, 2024-09-14T22:38:14.047 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T22:38:14.047 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T22:38:14.047 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.047 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 3, 2024-09-14T22:38:14.047 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "pool_client.0", 2024-09-14T22:38:14.047 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-14T22:36:14.374249+0000", 2024-09-14T22:38:14.047 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 8193, 2024-09-14T22:38:14.047 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T22:38:14.048 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-14T22:38:14.048 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-14T22:38:14.048 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-14T22:38:14.048 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-14T22:38:14.048 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:38:14.048 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:38:14.048 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:38:14.048 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:38:14.048 INFO:teuthology.orchestra.run.smithi001.stdout: "is_stretch_pool": false, 2024-09-14T22:38:14.048 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-14T22:38:14.048 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:38:14.048 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 32, 2024-09-14T22:38:14.048 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 32, 2024-09-14T22:38:14.048 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 32, 2024-09-14T22:38:14.049 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 32, 2024-09-14T22:38:14.049 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 32, 2024-09-14T22:38:14.049 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-14T22:38:14.049 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-14T22:38:14.049 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-14T22:38:14.049 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-14T22:38:14.049 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-14T22:38:14.049 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-14T22:38:14.049 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-14T22:38:14.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.049 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "121", 2024-09-14T22:38:14.050 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-14T22:38:14.050 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:38:14.050 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:38:14.050 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-14T22:38:14.050 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:38:14.050 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 42, 2024-09-14T22:38:14.050 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 121, 2024-09-14T22:38:14.050 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-14T22:38:14.050 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-14T22:38:14.050 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-14T22:38:14.050 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-14T22:38:14.050 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-14T22:38:14.050 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-14T22:38:14.051 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-14T22:38:14.051 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-14T22:38:14.051 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-14T22:38:14.051 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-14T22:38:14.051 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-14T22:38:14.051 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:38:14.051 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:38:14.051 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:38:14.051 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-14T22:38:14.051 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-14T22:38:14.051 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-14T22:38:14.051 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-14T22:38:14.051 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-14T22:38:14.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.052 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-14T22:38:14.052 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-14T22:38:14.052 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-14T22:38:14.052 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:38:14.052 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:38:14.052 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:38:14.052 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:38:14.052 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-14T22:38:14.052 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-14T22:38:14.052 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-14T22:38:14.052 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-14T22:38:14.052 INFO:teuthology.orchestra.run.smithi001.stdout: "options": {}, 2024-09-14T22:38:14.053 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-14T22:38:14.053 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd": {} 2024-09-14T22:38:14.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.053 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-14T22:38:14.053 INFO:teuthology.orchestra.run.smithi001.stdout: "score_type": "Fair distribution", 2024-09-14T22:38:14.053 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 1.75, 2024-09-14T22:38:14.053 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 1.75, 2024-09-14T22:38:14.053 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 1, 2024-09-14T22:38:14.053 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 1.75, 2024-09-14T22:38:14.053 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 1.75, 2024-09-14T22:38:14.053 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 0.87667274475097656, 2024-09-14T22:38:14.053 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T22:38:14.054 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T22:38:14.054 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.054 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.054 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:38:14.054 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2024-09-14T22:38:14.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.054 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 0, 2024-09-14T22:38:14.054 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "9831fcc5-9040-4a80-99d4-838225e65139", 2024-09-14T22:38:14.054 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:38:14.054 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:38:14.054 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:38:14.054 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:38:14.054 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:38:14.054 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:38:14.054 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:38:14.055 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 115, 2024-09-14T22:38:14.055 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:38:14.055 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:38:14.055 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:38:14.055 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:14.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.055 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:14.055 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6824", 2024-09-14T22:38:14.055 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:38:14.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.055 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:14.055 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6825", 2024-09-14T22:38:14.055 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:38:14.056 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.056 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.056 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:38:14.056 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:14.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.056 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:14.056 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6826", 2024-09-14T22:38:14.056 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:38:14.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.056 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:14.056 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6827", 2024-09-14T22:38:14.057 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:38:14.057 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.057 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.057 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:38:14.057 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:14.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.057 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:14.057 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6830", 2024-09-14T22:38:14.057 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:38:14.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.057 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:14.058 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6831", 2024-09-14T22:38:14.058 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:38:14.058 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.058 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.058 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:38:14.058 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:14.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.058 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:14.058 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6828", 2024-09-14T22:38:14.058 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:38:14.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.059 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:14.059 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6829", 2024-09-14T22:38:14.059 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:38:14.059 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.059 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.059 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6825/1702929600", 2024-09-14T22:38:14.059 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6827/1702929600", 2024-09-14T22:38:14.059 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6831/1702929600", 2024-09-14T22:38:14.059 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6829/1702929600", 2024-09-14T22:38:14.059 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:38:14.059 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:38:14.059 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:38:14.060 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.060 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 1, 2024-09-14T22:38:14.060 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "7f79abe8-115d-46b2-8677-72fcb0c8ab7d", 2024-09-14T22:38:14.060 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:38:14.060 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:38:14.060 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:38:14.060 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 0.1606292724609375, 2024-09-14T22:38:14.060 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:38:14.060 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:38:14.060 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2024-09-14T22:38:14.060 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 116, 2024-09-14T22:38:14.060 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:38:14.060 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:38:14.061 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:38:14.061 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:14.061 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.061 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:14.061 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6808", 2024-09-14T22:38:14.061 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:38:14.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.061 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.061 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:14.061 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6809", 2024-09-14T22:38:14.061 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:38:14.061 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.061 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.062 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:38:14.062 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:14.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.062 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:14.062 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6810", 2024-09-14T22:38:14.062 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:38:14.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.062 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:14.062 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6811", 2024-09-14T22:38:14.062 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:38:14.062 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.062 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.063 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:38:14.063 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:14.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.063 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:14.063 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6814", 2024-09-14T22:38:14.063 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:38:14.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.063 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:14.063 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6815", 2024-09-14T22:38:14.063 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:38:14.063 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.064 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.064 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:38:14.064 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:14.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.064 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:14.064 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6812", 2024-09-14T22:38:14.064 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:38:14.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.064 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:14.064 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6813", 2024-09-14T22:38:14.064 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:38:14.065 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.065 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.065 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6809/719990653", 2024-09-14T22:38:14.065 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6811/719990653", 2024-09-14T22:38:14.065 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6815/719990653", 2024-09-14T22:38:14.065 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6813/719990653", 2024-09-14T22:38:14.065 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:38:14.065 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:38:14.065 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:38:14.065 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.066 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 2, 2024-09-14T22:38:14.066 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "1f5768b2-4db4-4b2e-8d5e-5206f8bde28d", 2024-09-14T22:38:14.066 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:38:14.066 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:38:14.066 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:38:14.066 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:38:14.066 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 14, 2024-09-14T22:38:14.066 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 30, 2024-09-14T22:38:14.066 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 66, 2024-09-14T22:38:14.066 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 115, 2024-09-14T22:38:14.066 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 31, 2024-09-14T22:38:14.066 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:38:14.066 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:38:14.066 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:14.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.067 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:14.067 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6816", 2024-09-14T22:38:14.067 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:38:14.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.067 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.067 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:14.067 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6817", 2024-09-14T22:38:14.067 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:38:14.067 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.067 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.067 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:38:14.067 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:14.067 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.068 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:14.068 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6818", 2024-09-14T22:38:14.068 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:38:14.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.068 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:14.068 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6819", 2024-09-14T22:38:14.068 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:38:14.068 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.068 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.068 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:38:14.068 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:14.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.069 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:14.069 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6822", 2024-09-14T22:38:14.069 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:38:14.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.069 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:14.069 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6823", 2024-09-14T22:38:14.069 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:38:14.069 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.069 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.069 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:38:14.070 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:14.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.070 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:14.070 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6820", 2024-09-14T22:38:14.070 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:38:14.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.070 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:14.070 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6821", 2024-09-14T22:38:14.070 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:38:14.070 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.070 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.071 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6817/1006730601", 2024-09-14T22:38:14.071 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6819/1006730601", 2024-09-14T22:38:14.071 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6823/1006730601", 2024-09-14T22:38:14.071 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6821/1006730601", 2024-09-14T22:38:14.071 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:38:14.071 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:38:14.071 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:38:14.071 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.071 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 3, 2024-09-14T22:38:14.071 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "8103f3c7-7899-4fc3-bb02-dd2273120138", 2024-09-14T22:38:14.071 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:38:14.072 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:38:14.072 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:38:14.072 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 0.852752685546875, 2024-09-14T22:38:14.072 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:38:14.072 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:38:14.072 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:38:14.072 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 115, 2024-09-14T22:38:14.072 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:38:14.072 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:38:14.072 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:38:14.072 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:14.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.072 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:14.073 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6800", 2024-09-14T22:38:14.073 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:38:14.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.073 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:14.073 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6801", 2024-09-14T22:38:14.073 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:38:14.073 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.073 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.073 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:38:14.073 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:14.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.074 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:14.074 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6802", 2024-09-14T22:38:14.074 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:38:14.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.074 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:14.074 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6803", 2024-09-14T22:38:14.074 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:38:14.074 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.074 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.074 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:38:14.074 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:14.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.075 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:14.075 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6806", 2024-09-14T22:38:14.075 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:38:14.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.075 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:14.075 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6807", 2024-09-14T22:38:14.075 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:38:14.075 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.075 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.075 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:38:14.076 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:14.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.076 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:14.076 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6804", 2024-09-14T22:38:14.076 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:38:14.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.076 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:14.076 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6805", 2024-09-14T22:38:14.076 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:38:14.076 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.076 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.077 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6801/750403671", 2024-09-14T22:38:14.077 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6803/750403671", 2024-09-14T22:38:14.077 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6807/750403671", 2024-09-14T22:38:14.077 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6805/750403671", 2024-09-14T22:38:14.077 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:38:14.077 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:38:14.078 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:38:14.078 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.078 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 4, 2024-09-14T22:38:14.078 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "23cce5cb-726e-44c2-8edb-c2567a89dfbc", 2024-09-14T22:38:14.078 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:38:14.078 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:38:14.078 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:38:14.078 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:38:14.078 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:38:14.078 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:38:14.078 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:38:14.078 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 117, 2024-09-14T22:38:14.079 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:38:14.079 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:38:14.079 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:38:14.079 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:14.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.079 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:14.079 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6816", 2024-09-14T22:38:14.079 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:38:14.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.079 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:14.079 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6817", 2024-09-14T22:38:14.079 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:38:14.080 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.080 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.080 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:38:14.080 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:14.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.080 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:14.080 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6818", 2024-09-14T22:38:14.080 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:38:14.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.080 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:14.080 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6819", 2024-09-14T22:38:14.081 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:38:14.081 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.081 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.081 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:38:14.081 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:14.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.081 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:14.081 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6822", 2024-09-14T22:38:14.081 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:38:14.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.081 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:14.082 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6823", 2024-09-14T22:38:14.082 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:38:14.082 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.082 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.082 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:38:14.082 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:14.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.082 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:14.082 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6820", 2024-09-14T22:38:14.082 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:38:14.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.082 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:14.083 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6821", 2024-09-14T22:38:14.083 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:38:14.083 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.083 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.083 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.176:6817/1408471829", 2024-09-14T22:38:14.083 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.176:6819/1408471829", 2024-09-14T22:38:14.083 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.176:6823/1408471829", 2024-09-14T22:38:14.083 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.176:6821/1408471829", 2024-09-14T22:38:14.083 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:38:14.083 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:38:14.083 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:38:14.084 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.084 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 5, 2024-09-14T22:38:14.084 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "eb2f187b-d2f1-49ab-ad5e-ea6dc441a28e", 2024-09-14T22:38:14.084 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:38:14.084 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:38:14.084 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:38:14.084 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:38:14.084 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:38:14.084 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:38:14.084 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:38:14.084 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 116, 2024-09-14T22:38:14.085 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:38:14.085 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:38:14.085 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:38:14.085 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:14.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.085 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:14.085 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6805", 2024-09-14T22:38:14.085 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:38:14.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.085 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:14.085 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6808", 2024-09-14T22:38:14.085 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:38:14.085 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.086 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.086 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:38:14.086 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:14.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.086 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:14.086 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6810", 2024-09-14T22:38:14.086 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:38:14.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.086 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:14.086 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6811", 2024-09-14T22:38:14.086 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:38:14.086 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.087 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.087 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:38:14.087 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:14.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.087 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:14.087 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6814", 2024-09-14T22:38:14.087 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:38:14.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.087 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:14.087 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6815", 2024-09-14T22:38:14.087 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:38:14.088 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.088 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.088 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:38:14.088 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:14.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.088 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:14.088 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6812", 2024-09-14T22:38:14.088 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:38:14.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.088 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:14.088 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6813", 2024-09-14T22:38:14.088 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:38:14.089 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.089 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.089 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.176:6808/1945902326", 2024-09-14T22:38:14.089 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.176:6811/1945902326", 2024-09-14T22:38:14.089 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.176:6815/1945902326", 2024-09-14T22:38:14.089 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.176:6813/1945902326", 2024-09-14T22:38:14.089 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:38:14.089 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:38:14.089 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:38:14.089 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.090 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 6, 2024-09-14T22:38:14.090 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "d371aed8-2ad2-49a5-8ece-4b7fd37b4822", 2024-09-14T22:38:14.090 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:38:14.090 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:38:14.090 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:38:14.090 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:38:14.090 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:38:14.090 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:38:14.090 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:38:14.090 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 116, 2024-09-14T22:38:14.090 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:38:14.090 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:38:14.090 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:38:14.090 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:14.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.091 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:14.091 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6800", 2024-09-14T22:38:14.091 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:38:14.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.091 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:14.091 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6801", 2024-09-14T22:38:14.091 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:38:14.091 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.091 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.091 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:38:14.092 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:14.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.092 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:14.092 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6802", 2024-09-14T22:38:14.092 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:38:14.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.092 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:14.092 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6803", 2024-09-14T22:38:14.092 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:38:14.092 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.092 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.092 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:38:14.093 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:14.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.093 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:14.093 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6807", 2024-09-14T22:38:14.093 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:38:14.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.093 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:14.093 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6809", 2024-09-14T22:38:14.093 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:38:14.093 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.094 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.094 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:38:14.094 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:14.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.094 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:14.094 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6804", 2024-09-14T22:38:14.094 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:38:14.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.094 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:14.094 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6806", 2024-09-14T22:38:14.094 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:38:14.095 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.095 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.095 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.176:6801/4116085586", 2024-09-14T22:38:14.095 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.176:6803/4116085586", 2024-09-14T22:38:14.095 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.176:6809/4116085586", 2024-09-14T22:38:14.095 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.176:6806/4116085586", 2024-09-14T22:38:14.095 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:38:14.095 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:38:14.095 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:38:14.095 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.095 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 7, 2024-09-14T22:38:14.096 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "a012c819-8d49-4310-a840-c2752d9314ab", 2024-09-14T22:38:14.096 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:38:14.096 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:38:14.096 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:38:14.096 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:38:14.096 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:38:14.096 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:38:14.096 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:38:14.096 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 117, 2024-09-14T22:38:14.096 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:38:14.096 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:38:14.096 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:38:14.096 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:14.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.097 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:14.097 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6824", 2024-09-14T22:38:14.097 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:38:14.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.097 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:14.097 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6825", 2024-09-14T22:38:14.097 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:38:14.097 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.097 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.098 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:38:14.098 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:14.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.098 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:14.098 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6826", 2024-09-14T22:38:14.098 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:38:14.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.098 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:14.098 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6827", 2024-09-14T22:38:14.098 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:38:14.098 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.099 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.099 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:38:14.099 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:14.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.099 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:14.099 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6830", 2024-09-14T22:38:14.099 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:38:14.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.099 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:14.099 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6831", 2024-09-14T22:38:14.099 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:38:14.099 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.100 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.100 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:38:14.100 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:14.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.100 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:14.100 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6828", 2024-09-14T22:38:14.100 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:38:14.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.100 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:14.100 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6829", 2024-09-14T22:38:14.100 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:38:14.101 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.101 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.101 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.176:6825/877760244", 2024-09-14T22:38:14.101 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.176:6827/877760244", 2024-09-14T22:38:14.101 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.176:6831/877760244", 2024-09-14T22:38:14.101 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.176:6829/877760244", 2024-09-14T22:38:14.101 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:38:14.101 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:38:14.101 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:38:14.101 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.101 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.101 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:38:14.101 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_xinfo": [ 2024-09-14T22:38:14.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.102 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 0, 2024-09-14T22:38:14.102 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:38:14.102 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:38:14.102 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:38:14.102 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:38:14.102 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:38:14.102 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:55.511688+0000", 2024-09-14T22:38:14.102 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:38:14.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.102 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 1, 2024-09-14T22:38:14.102 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:38:14.102 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:38:14.102 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:38:14.103 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:38:14.103 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:38:14.103 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.742414+0000", 2024-09-14T22:38:14.103 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:38:14.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.103 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 2, 2024-09-14T22:38:14.103 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:38:14.103 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:38:14.103 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:38:14.103 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:38:14.103 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:38:14.103 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.871355+0000", 2024-09-14T22:38:14.103 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 31 2024-09-14T22:38:14.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.104 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 3, 2024-09-14T22:38:14.104 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:38:14.104 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:38:14.104 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:38:14.104 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:38:14.104 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:38:14.104 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.504553+0000", 2024-09-14T22:38:14.104 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:38:14.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.104 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 4, 2024-09-14T22:38:14.104 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:38:14.104 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:38:14.104 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:38:14.104 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:38:14.105 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:38:14.105 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.589640+0000", 2024-09-14T22:38:14.105 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:38:14.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.105 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 5, 2024-09-14T22:38:14.105 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:38:14.105 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:38:14.105 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:38:14.105 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:38:14.105 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:38:14.105 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.518845+0000", 2024-09-14T22:38:14.105 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:38:14.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.106 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 6, 2024-09-14T22:38:14.106 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:38:14.106 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:38:14.106 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:38:14.106 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:38:14.106 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:38:14.106 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.731632+0000", 2024-09-14T22:38:14.106 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:38:14.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.106 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 7, 2024-09-14T22:38:14.106 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:38:14.106 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:38:14.107 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:38:14.107 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:38:14.107 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:38:14.107 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.952634+0000", 2024-09-14T22:38:14.107 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:38:14.107 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.107 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:38:14.107 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap": [ 2024-09-14T22:38:14.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:14.107 INFO:teuthology.orchestra.run.smithi001.stdout: "pgid": "3.1d", 2024-09-14T22:38:14.107 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2024-09-14T22:38:14.107 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-09-14T22:38:14.107 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-09-14T22:38:14.108 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:14.108 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.108 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:38:14.108 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap_items": [], 2024-09-14T22:38:14.108 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap_primaries": [], 2024-09-14T22:38:14.108 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_temp": [], 2024-09-14T22:38:14.108 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_temp": [], 2024-09-14T22:38:14.108 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklist": {}, 2024-09-14T22:38:14.108 INFO:teuthology.orchestra.run.smithi001.stdout: "range_blocklist": {}, 2024-09-14T22:38:14.108 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profiles": { 2024-09-14T22:38:14.108 INFO:teuthology.orchestra.run.smithi001.stdout: "default": { 2024-09-14T22:38:14.108 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-failure-domain": "osd", 2024-09-14T22:38:14.108 INFO:teuthology.orchestra.run.smithi001.stdout: "k": "2", 2024-09-14T22:38:14.108 INFO:teuthology.orchestra.run.smithi001.stdout: "m": "1", 2024-09-14T22:38:14.108 INFO:teuthology.orchestra.run.smithi001.stdout: "plugin": "jerasure", 2024-09-14T22:38:14.109 INFO:teuthology.orchestra.run.smithi001.stdout: "technique": "reed_sol_van" 2024-09-14T22:38:14.109 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:14.109 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps_queue": [], 2024-09-14T22:38:14.109 INFO:teuthology.orchestra.run.smithi001.stdout: "new_removed_snaps": [], 2024-09-14T22:38:14.109 INFO:teuthology.orchestra.run.smithi001.stdout: "new_purged_snaps": [], 2024-09-14T22:38:14.109 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_node_flags": {}, 2024-09-14T22:38:14.109 INFO:teuthology.orchestra.run.smithi001.stdout: "device_class_flags": {}, 2024-09-14T22:38:14.109 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": { 2024-09-14T22:38:14.109 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode_enabled": false, 2024-09-14T22:38:14.109 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_bucket_count": 0, 2024-09-14T22:38:14.109 INFO:teuthology.orchestra.run.smithi001.stdout: "degraded_stretch_mode": 0, 2024-09-14T22:38:14.109 INFO:teuthology.orchestra.run.smithi001.stdout: "recovering_stretch_mode": 0, 2024-09-14T22:38:14.110 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode_bucket": 0 2024-09-14T22:38:14.110 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:14.110 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:14.111 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:14.111 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:14.112 INFO:tasks.thrashosds.thrasher:j is {'epoch': 121, 'fsid': 'f07f407a-b17d-42c4-bf73-92dd80114a4f', 'created': '2024-09-14T22:35:45.983055+0000', 'modified': '2024-09-14T22:38:14.002738+0000', 'last_up_change': '2024-09-14T22:37:12.616042+0000', 'last_in_change': '2024-09-14T22:35:48.283160+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T22:35:58.938784+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8766727447509766, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T22:36:04.304242+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.8766727447509766, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T22:36:14.374249+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 42, 'snap_epoch': 121, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.8766727447509766, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': '9831fcc5-9040-4a80-99d4-838225e65139', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 115, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6824', 'nonce': 1702929600}, {'type': 'v1', 'addr': '172.21.15.1:6825', 'nonce': 1702929600}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6826', 'nonce': 1702929600}, {'type': 'v1', 'addr': '172.21.15.1:6827', 'nonce': 1702929600}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6830', 'nonce': 1702929600}, {'type': 'v1', 'addr': '172.21.15.1:6831', 'nonce': 1702929600}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6828', 'nonce': 1702929600}, {'type': 'v1', 'addr': '172.21.15.1:6829', 'nonce': 1702929600}]}, 'public_addr': '172.21.15.1:6825/1702929600', 'cluster_addr': '172.21.15.1:6827/1702929600', 'heartbeat_back_addr': '172.21.15.1:6831/1702929600', 'heartbeat_front_addr': '172.21.15.1:6829/1702929600', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '7f79abe8-115d-46b2-8677-72fcb0c8ab7d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.1606292724609375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 116, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6808', 'nonce': 719990653}, {'type': 'v1', 'addr': '172.21.15.1:6809', 'nonce': 719990653}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6810', 'nonce': 719990653}, {'type': 'v1', 'addr': '172.21.15.1:6811', 'nonce': 719990653}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6814', 'nonce': 719990653}, {'type': 'v1', 'addr': '172.21.15.1:6815', 'nonce': 719990653}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6812', 'nonce': 719990653}, {'type': 'v1', 'addr': '172.21.15.1:6813', 'nonce': 719990653}]}, 'public_addr': '172.21.15.1:6809/719990653', 'cluster_addr': '172.21.15.1:6811/719990653', 'heartbeat_back_addr': '172.21.15.1:6815/719990653', 'heartbeat_front_addr': '172.21.15.1:6813/719990653', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '1f5768b2-4db4-4b2e-8d5e-5206f8bde28d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 14, 'last_clean_end': 30, 'up_from': 66, 'up_thru': 115, 'down_at': 31, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6816', 'nonce': 1006730601}, {'type': 'v1', 'addr': '172.21.15.1:6817', 'nonce': 1006730601}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6818', 'nonce': 1006730601}, {'type': 'v1', 'addr': '172.21.15.1:6819', 'nonce': 1006730601}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6822', 'nonce': 1006730601}, {'type': 'v1', 'addr': '172.21.15.1:6823', 'nonce': 1006730601}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6820', 'nonce': 1006730601}, {'type': 'v1', 'addr': '172.21.15.1:6821', 'nonce': 1006730601}]}, 'public_addr': '172.21.15.1:6817/1006730601', 'cluster_addr': '172.21.15.1:6819/1006730601', 'heartbeat_back_addr': '172.21.15.1:6823/1006730601', 'heartbeat_front_addr': '172.21.15.1:6821/1006730601', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '8103f3c7-7899-4fc3-bb02-dd2273120138', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.852752685546875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 115, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6800', 'nonce': 750403671}, {'type': 'v1', 'addr': '172.21.15.1:6801', 'nonce': 750403671}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6802', 'nonce': 750403671}, {'type': 'v1', 'addr': '172.21.15.1:6803', 'nonce': 750403671}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6806', 'nonce': 750403671}, {'type': 'v1', 'addr': '172.21.15.1:6807', 'nonce': 750403671}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6804', 'nonce': 750403671}, {'type': 'v1', 'addr': '172.21.15.1:6805', 'nonce': 750403671}]}, 'public_addr': '172.21.15.1:6801/750403671', 'cluster_addr': '172.21.15.1:6803/750403671', 'heartbeat_back_addr': '172.21.15.1:6807/750403671', 'heartbeat_front_addr': '172.21.15.1:6805/750403671', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '23cce5cb-726e-44c2-8edb-c2567a89dfbc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 117, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6816', 'nonce': 1408471829}, {'type': 'v1', 'addr': '172.21.15.176:6817', 'nonce': 1408471829}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6818', 'nonce': 1408471829}, {'type': 'v1', 'addr': '172.21.15.176:6819', 'nonce': 1408471829}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6822', 'nonce': 1408471829}, {'type': 'v1', 'addr': '172.21.15.176:6823', 'nonce': 1408471829}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6820', 'nonce': 1408471829}, {'type': 'v1', 'addr': '172.21.15.176:6821', 'nonce': 1408471829}]}, 'public_addr': '172.21.15.176:6817/1408471829', 'cluster_addr': '172.21.15.176:6819/1408471829', 'heartbeat_back_addr': '172.21.15.176:6823/1408471829', 'heartbeat_front_addr': '172.21.15.176:6821/1408471829', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'eb2f187b-d2f1-49ab-ad5e-ea6dc441a28e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 116, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6805', 'nonce': 1945902326}, {'type': 'v1', 'addr': '172.21.15.176:6808', 'nonce': 1945902326}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6810', 'nonce': 1945902326}, {'type': 'v1', 'addr': '172.21.15.176:6811', 'nonce': 1945902326}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6814', 'nonce': 1945902326}, {'type': 'v1', 'addr': '172.21.15.176:6815', 'nonce': 1945902326}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6812', 'nonce': 1945902326}, {'type': 'v1', 'addr': '172.21.15.176:6813', 'nonce': 1945902326}]}, 'public_addr': '172.21.15.176:6808/1945902326', 'cluster_addr': '172.21.15.176:6811/1945902326', 'heartbeat_back_addr': '172.21.15.176:6815/1945902326', 'heartbeat_front_addr': '172.21.15.176:6813/1945902326', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'd371aed8-2ad2-49a5-8ece-4b7fd37b4822', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 116, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6800', 'nonce': 4116085586}, {'type': 'v1', 'addr': '172.21.15.176:6801', 'nonce': 4116085586}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6802', 'nonce': 4116085586}, {'type': 'v1', 'addr': '172.21.15.176:6803', 'nonce': 4116085586}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6807', 'nonce': 4116085586}, {'type': 'v1', 'addr': '172.21.15.176:6809', 'nonce': 4116085586}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6804', 'nonce': 4116085586}, {'type': 'v1', 'addr': '172.21.15.176:6806', 'nonce': 4116085586}]}, 'public_addr': '172.21.15.176:6801/4116085586', 'cluster_addr': '172.21.15.176:6803/4116085586', 'heartbeat_back_addr': '172.21.15.176:6809/4116085586', 'heartbeat_front_addr': '172.21.15.176:6806/4116085586', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'a012c819-8d49-4310-a840-c2752d9314ab', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 117, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6824', 'nonce': 877760244}, {'type': 'v1', 'addr': '172.21.15.176:6825', 'nonce': 877760244}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6826', 'nonce': 877760244}, {'type': 'v1', 'addr': '172.21.15.176:6827', 'nonce': 877760244}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6830', 'nonce': 877760244}, {'type': 'v1', 'addr': '172.21.15.176:6831', 'nonce': 877760244}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6828', 'nonce': 877760244}, {'type': 'v1', 'addr': '172.21.15.176:6829', 'nonce': 877760244}]}, 'public_addr': '172.21.15.176:6825/877760244', 'cluster_addr': '172.21.15.176:6827/877760244', 'heartbeat_back_addr': '172.21.15.176:6831/877760244', 'heartbeat_front_addr': '172.21.15.176:6829/877760244', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:55.511688+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.742414+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.871355+0000', 'dead_epoch': 31}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.504553+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.589640+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.518845+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.731632+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.952634+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.1d', 'osds': [1, 2]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T22:38:14.112 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:38:14.113 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:14.106+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:14.113 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:38:14.137 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone39 from 0xdb3fa98 (overlap 0xac4b17a) to 0x22756b2 2024-09-14T22:38:14.212 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:14.206+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:14.282 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:14.283 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:14.303 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:38:14.313 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:14.306+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:14.414 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:14.410+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:14.486 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:38:14.508 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:14.509 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:14.516 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:14.510+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:14.531 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:38:14.589 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:38:14.590 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-14T22:38:14.607 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2024-09-14T22:38:14.011473+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1384900561,"num_objects":1859,"num_object_clones":465,"num_object_copies":3718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1859,"num_whiteouts":734,"num_read":188922,"num_read_kb":782371,"num_write":22126,"num_write_kb":381014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3299,"num_bytes_recovered":173602381,"num_keys_recovered":342,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16841,"ondisk_log_size":16841,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":442352,"kb_used_data":63684,"kb_used_omap":17,"kb_used_meta":378478,"kb_avail":754532368,"statfs":{"total":773094113280,"available":772641144832,"internally_reserved":0,"allocated":65212416,"data_stored":506714228,"data_compressed":14244193,"data_compressed_allocated":43364352,"data_compressed_original":495178731,"omap_allocated":18180,"internal_metadata":387561724},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17658631,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-131,"num_objects_misplaced":-216,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":1270,"num_read_kb":20047,"num_write":294,"num_write_kb":7459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":649,"num_bytes_recovered":31769304,"num_keys_recovered":57,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.280317"},"pg_stats":[{"pgid":"3.1f","version":"119'367","reported_seq":4752,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.590916+0000","last_change":"2024-09-14T22:37:25.739233+0000","last_active":"2024-09-14T22:38:12.590916+0000","last_peered":"2024-09-14T22:38:12.590916+0000","last_clean":"2024-09-14T22:38:12.590916+0000","last_became_active":"2024-09-14T22:37:25.718906+0000","last_became_peered":"2024-09-14T22:37:25.718906+0000","last_unstale":"2024-09-14T22:38:12.590916+0000","last_undegraded":"2024-09-14T22:38:12.590916+0000","last_fullsized":"2024-09-14T22:38:12.590916+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:47:47.368174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017190879999999999,"stat_sum":{"num_bytes":55797417,"num_objects":60,"num_object_clones":16,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":26,"num_read":4759,"num_read_kb":16316,"num_write":373,"num_write_kb":7115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":3277632,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"118'1040","reported_seq":5819,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:11.992258+0000","last_change":"2024-09-14T22:38:08.744177+0000","last_active":"2024-09-14T22:38:11.992258+0000","last_peered":"2024-09-14T22:38:11.992258+0000","last_clean":"2024-09-14T22:38:11.992258+0000","last_became_active":"2024-09-14T22:38:08.741149+0000","last_became_peered":"2024-09-14T22:38:08.741149+0000","last_unstale":"2024-09-14T22:38:11.992258+0000","last_undegraded":"2024-09-14T22:38:11.992258+0000","last_fullsized":"2024-09-14T22:38:11.992258+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1040,"log_dups_size":0,"ondisk_log_size":1040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:20:39.731232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001723619,"stat_sum":{"num_bytes":76601511,"num_objects":56,"num_object_clones":15,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":22,"num_read":10210,"num_read_kb":24720,"num_write":1588,"num_write_kb":10683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":7412288,"num_keys_recovered":90,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"118'583","reported_seq":9977,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-14T22:38:10.736644+0000","last_change":"2024-09-14T22:37:51.741030+0000","last_active":"2024-09-14T22:38:10.736644+0000","last_peered":"2024-09-14T22:38:10.736644+0000","last_clean":"2024-09-14T22:38:10.736644+0000","last_became_active":"2024-09-14T22:37:51.730971+0000","last_became_peered":"2024-09-14T22:37:51.730971+0000","last_unstale":"2024-09-14T22:38:10.736644+0000","last_undegraded":"2024-09-14T22:38:10.736644+0000","last_fullsized":"2024-09-14T22:38:10.736644+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:50:14.998252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053914200000000005,"stat_sum":{"num_bytes":76488346,"num_objects":68,"num_object_clones":22,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":31,"num_read":10660,"num_read_kb":63051,"num_write":799,"num_write_kb":13692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":8030815,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"119'557","reported_seq":5730,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.618057+0000","last_change":"2024-09-14T22:36:17.702413+0000","last_active":"2024-09-14T22:38:12.618057+0000","last_peered":"2024-09-14T22:38:12.618057+0000","last_clean":"2024-09-14T22:38:12.618057+0000","last_became_active":"2024-09-14T22:36:15.719470+0000","last_became_peered":"2024-09-14T22:36:15.719470+0000","last_unstale":"2024-09-14T22:38:12.618057+0000","last_undegraded":"2024-09-14T22:38:12.618057+0000","last_fullsized":"2024-09-14T22:38:12.618057+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":557,"log_dups_size":0,"ondisk_log_size":557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:58:05.414233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055079499999999997,"stat_sum":{"num_bytes":35950567,"num_objects":59,"num_object_clones":15,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":26,"num_read":6008,"num_read_kb":26114,"num_write":733,"num_write_kb":8524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"118'471","reported_seq":4882,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:11.989100+0000","last_change":"2024-09-14T22:37:25.736579+0000","last_active":"2024-09-14T22:38:11.989100+0000","last_peered":"2024-09-14T22:38:11.989100+0000","last_clean":"2024-09-14T22:38:11.989100+0000","last_became_active":"2024-09-14T22:37:25.718519+0000","last_became_peered":"2024-09-14T22:37:25.718519+0000","last_unstale":"2024-09-14T22:38:11.989100+0000","last_undegraded":"2024-09-14T22:38:11.989100+0000","last_fullsized":"2024-09-14T22:38:11.989100+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:09:56.374762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092291499999999996,"stat_sum":{"num_bytes":30938989,"num_objects":50,"num_object_clones":14,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":21,"num_read":5129,"num_read_kb":32004,"num_write":571,"num_write_kb":13306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":7858104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"118'406","reported_seq":3347,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.849338+0000","last_change":"2024-09-14T22:38:10.745128+0000","last_active":"2024-09-14T22:38:12.849338+0000","last_peered":"2024-09-14T22:38:12.849338+0000","last_clean":"2024-09-14T22:38:12.849338+0000","last_became_active":"2024-09-14T22:38:10.742092+0000","last_became_peered":"2024-09-14T22:38:10.742092+0000","last_unstale":"2024-09-14T22:38:12.849338+0000","last_undegraded":"2024-09-14T22:38:12.849338+0000","last_fullsized":"2024-09-14T22:38:12.849338+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:44:24.632361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012502780000000001,"stat_sum":{"num_bytes":11058244,"num_objects":68,"num_object_clones":19,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":34,"num_read":3864,"num_read_kb":12494,"num_write":410,"num_write_kb":6721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":10036538,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"110'363","reported_seq":5112,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-14T22:38:10.742406+0000","last_change":"2024-09-14T22:36:17.705423+0000","last_active":"2024-09-14T22:38:10.742406+0000","last_peered":"2024-09-14T22:38:10.742406+0000","last_clean":"2024-09-14T22:38:10.742406+0000","last_became_active":"2024-09-14T22:36:15.700990+0000","last_became_peered":"2024-09-14T22:36:15.700990+0000","last_unstale":"2024-09-14T22:38:10.742406+0000","last_undegraded":"2024-09-14T22:38:10.742406+0000","last_fullsized":"2024-09-14T22:38:10.742406+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:55:28.951912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010520620000000001,"stat_sum":{"num_bytes":22178140,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":23,"num_read":5058,"num_read_kb":15226,"num_write":369,"num_write_kb":5409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"118'405","reported_seq":4451,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:11.992236+0000","last_change":"2024-09-14T22:38:07.735828+0000","last_active":"2024-09-14T22:38:11.992236+0000","last_peered":"2024-09-14T22:38:11.992236+0000","last_clean":"2024-09-14T22:38:11.992236+0000","last_became_active":"2024-09-14T22:38:07.734308+0000","last_became_peered":"2024-09-14T22:38:07.734308+0000","last_unstale":"2024-09-14T22:38:11.992236+0000","last_undegraded":"2024-09-14T22:38:11.992236+0000","last_fullsized":"2024-09-14T22:38:11.992236+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:17:54.535972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002063484,"stat_sum":{"num_bytes":36374590,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":14,"num_read":4705,"num_read_kb":35743,"num_write":569,"num_write_kb":27694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":20067293,"num_keys_recovered":21,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"119'349","reported_seq":3848,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.855763+0000","last_change":"2024-09-14T22:36:17.721195+0000","last_active":"2024-09-14T22:38:12.855763+0000","last_peered":"2024-09-14T22:38:12.855763+0000","last_clean":"2024-09-14T22:38:12.855763+0000","last_became_active":"2024-09-14T22:36:15.715869+0000","last_became_peered":"2024-09-14T22:36:15.715869+0000","last_unstale":"2024-09-14T22:38:12.855763+0000","last_undegraded":"2024-09-14T22:38:12.855763+0000","last_fullsized":"2024-09-14T22:38:12.855763+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:08:39.994970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001160672,"stat_sum":{"num_bytes":23103658,"num_objects":56,"num_object_clones":10,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":24,"num_read":3873,"num_read_kb":17259,"num_write":437,"num_write_kb":21983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"119'474","reported_seq":6175,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.670227+0000","last_change":"2024-09-14T22:36:17.703048+0000","last_active":"2024-09-14T22:38:12.670227+0000","last_peered":"2024-09-14T22:38:12.670227+0000","last_clean":"2024-09-14T22:38:12.670227+0000","last_became_active":"2024-09-14T22:36:15.725078+0000","last_became_peered":"2024-09-14T22:36:15.725078+0000","last_unstale":"2024-09-14T22:38:12.670227+0000","last_undegraded":"2024-09-14T22:38:12.670227+0000","last_fullsized":"2024-09-14T22:38:12.670227+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:47:39.115115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010500290000000001,"stat_sum":{"num_bytes":72882446,"num_objects":66,"num_object_clones":18,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":23,"num_read":6586,"num_read_kb":51137,"num_write":655,"num_write_kb":18943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"118'287","reported_seq":3350,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.004653+0000","last_change":"2024-09-14T22:36:18.009371+0000","last_active":"2024-09-14T22:38:12.004653+0000","last_peered":"2024-09-14T22:38:12.004653+0000","last_clean":"2024-09-14T22:38:12.004653+0000","last_became_active":"2024-09-14T22:36:15.717341+0000","last_became_peered":"2024-09-14T22:36:15.717341+0000","last_unstale":"2024-09-14T22:38:12.004653+0000","last_undegraded":"2024-09-14T22:38:12.004653+0000","last_fullsized":"2024-09-14T22:38:12.004653+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:18:04.062775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054571999999999997,"stat_sum":{"num_bytes":14418034,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":17,"num_read":3100,"num_read_kb":5383,"num_write":300,"num_write_kb":7316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"119'1020","reported_seq":8336,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.859757+0000","last_change":"2024-09-14T22:38:10.748696+0000","last_active":"2024-09-14T22:38:12.859757+0000","last_peered":"2024-09-14T22:38:12.859757+0000","last_clean":"2024-09-14T22:38:12.859757+0000","last_became_active":"2024-09-14T22:38:10.741866+0000","last_became_peered":"2024-09-14T22:38:10.741866+0000","last_unstale":"2024-09-14T22:38:12.859757+0000","last_undegraded":"2024-09-14T22:38:12.859757+0000","last_fullsized":"2024-09-14T22:38:12.859757+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1020,"log_dups_size":0,"ondisk_log_size":1020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:38:06.421661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010804009999999999,"stat_sum":{"num_bytes":44119068,"num_objects":72,"num_object_clones":19,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":27,"num_read":10279,"num_read_kb":28738,"num_write":1500,"num_write_kb":11862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":17498216,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":213,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:11.991311+0000","last_change":"2024-09-14T22:36:08.042860+0000","last_active":"2024-09-14T22:38:11.991311+0000","last_peered":"2024-09-14T22:38:11.991311+0000","last_clean":"2024-09-14T22:38:11.991311+0000","last_became_active":"2024-09-14T22:36:06.040961+0000","last_became_peered":"2024-09-14T22:36:06.040961+0000","last_unstale":"2024-09-14T22:38:11.991311+0000","last_undegraded":"2024-09-14T22:38:11.991311+0000","last_fullsized":"2024-09-14T22:38:11.991311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:03:48.547834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010780950000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"119'462","reported_seq":2550,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.619229+0000","last_change":"2024-09-14T22:36:17.720897+0000","last_active":"2024-09-14T22:38:12.619229+0000","last_peered":"2024-09-14T22:38:12.619229+0000","last_clean":"2024-09-14T22:38:12.619229+0000","last_became_active":"2024-09-14T22:36:15.700982+0000","last_became_peered":"2024-09-14T22:36:15.700982+0000","last_unstale":"2024-09-14T22:38:12.619229+0000","last_undegraded":"2024-09-14T22:38:12.619229+0000","last_fullsized":"2024-09-14T22:38:12.619229+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:08:05.131342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010495459999999999,"stat_sum":{"num_bytes":30715546,"num_objects":67,"num_object_clones":17,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":24,"num_read":3560,"num_read_kb":14527,"num_write":495,"num_write_kb":11884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":173,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:11.991302+0000","last_change":"2024-09-14T22:38:03.977889+0000","last_active":"2024-09-14T22:38:11.991302+0000","last_peered":"2024-09-14T22:38:11.991302+0000","last_clean":"2024-09-14T22:38:11.991302+0000","last_became_active":"2024-09-14T22:38:03.976899+0000","last_became_peered":"2024-09-14T22:38:03.976899+0000","last_unstale":"2024-09-14T22:38:11.991302+0000","last_undegraded":"2024-09-14T22:38:11.991302+0000","last_fullsized":"2024-09-14T22:38:11.991302+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:00:41.381700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"119'410","reported_seq":5187,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.670868+0000","last_change":"2024-09-14T22:36:21.722517+0000","last_active":"2024-09-14T22:38:12.670868+0000","last_peered":"2024-09-14T22:38:12.670868+0000","last_clean":"2024-09-14T22:38:12.670868+0000","last_became_active":"2024-09-14T22:36:21.722085+0000","last_became_peered":"2024-09-14T22:36:21.722085+0000","last_unstale":"2024-09-14T22:38:12.670868+0000","last_undegraded":"2024-09-14T22:38:12.670868+0000","last_fullsized":"2024-09-14T22:38:12.670868+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:50:44.432096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020774109999999999,"stat_sum":{"num_bytes":52168094,"num_objects":52,"num_object_clones":11,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":21,"num_read":5685,"num_read_kb":28635,"num_write":477,"num_write_kb":8482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":214,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.004909+0000","last_change":"2024-09-14T22:36:08.054606+0000","last_active":"2024-09-14T22:38:12.004909+0000","last_peered":"2024-09-14T22:38:12.004909+0000","last_clean":"2024-09-14T22:38:12.004909+0000","last_became_active":"2024-09-14T22:36:06.266908+0000","last_became_peered":"2024-09-14T22:36:06.266908+0000","last_unstale":"2024-09-14T22:38:12.004909+0000","last_undegraded":"2024-09-14T22:38:12.004909+0000","last_fullsized":"2024-09-14T22:38:12.004909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:36.738643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098249300000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"118'1320","reported_seq":4881,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-14T22:38:10.736563+0000","last_change":"2024-09-14T22:37:17.676125+0000","last_active":"2024-09-14T22:38:10.736563+0000","last_peered":"2024-09-14T22:38:10.736563+0000","last_clean":"2024-09-14T22:38:10.736563+0000","last_became_active":"2024-09-14T22:37:17.674322+0000","last_became_peered":"2024-09-14T22:37:17.674322+0000","last_unstale":"2024-09-14T22:38:10.736563+0000","last_undegraded":"2024-09-14T22:38:10.736563+0000","last_fullsized":"2024-09-14T22:38:10.736563+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1320,"log_dups_size":0,"ondisk_log_size":1320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:29:16.921603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012564830000000001,"stat_sum":{"num_bytes":27908088,"num_objects":62,"num_object_clones":15,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":26,"num_read":9456,"num_read_kb":23522,"num_write":2357,"num_write_kb":17901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":4264139,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":163,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:11.988088+0000","last_change":"2024-09-14T22:36:41.930961+0000","last_active":"2024-09-14T22:38:11.988088+0000","last_peered":"2024-09-14T22:38:11.988088+0000","last_clean":"2024-09-14T22:38:11.988088+0000","last_became_active":"2024-09-14T22:36:41.930619+0000","last_became_peered":"2024-09-14T22:36:41.930619+0000","last_unstale":"2024-09-14T22:38:11.988088+0000","last_undegraded":"2024-09-14T22:38:11.988088+0000","last_fullsized":"2024-09-14T22:38:11.988088+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:04:40.582999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"111'439","reported_seq":3942,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.355326+0000","last_change":"2024-09-14T22:38:09.741451+0000","last_active":"2024-09-14T22:38:12.355326+0000","last_peered":"2024-09-14T22:38:12.355326+0000","last_clean":"2024-09-14T22:38:12.355326+0000","last_became_active":"2024-09-14T22:38:09.740623+0000","last_became_peered":"2024-09-14T22:38:09.740623+0000","last_unstale":"2024-09-14T22:38:12.355326+0000","last_undegraded":"2024-09-14T22:38:12.355326+0000","last_fullsized":"2024-09-14T22:38:12.355326+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:03:29.131235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071220099999999998,"stat_sum":{"num_bytes":28456380,"num_objects":53,"num_object_clones":14,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":21,"num_read":4394,"num_read_kb":13382,"num_write":480,"num_write_kb":5683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":5797306,"num_keys_recovered":72,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"119'321","reported_seq":4774,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.355910+0000","last_change":"2024-09-14T22:36:17.711068+0000","last_active":"2024-09-14T22:38:12.355910+0000","last_peered":"2024-09-14T22:38:12.355910+0000","last_clean":"2024-09-14T22:38:12.355910+0000","last_became_active":"2024-09-14T22:36:15.706147+0000","last_became_peered":"2024-09-14T22:36:15.706147+0000","last_unstale":"2024-09-14T22:38:12.355910+0000","last_undegraded":"2024-09-14T22:38:12.355910+0000","last_fullsized":"2024-09-14T22:38:12.355910+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:04:01.813199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082625500000000002,"stat_sum":{"num_bytes":52759702,"num_objects":54,"num_object_clones":16,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":22,"num_read":4331,"num_read_kb":25513,"num_write":390,"num_write_kb":13888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":216,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.023799+0000","last_change":"2024-09-14T22:36:08.055938+0000","last_active":"2024-09-14T22:38:12.023799+0000","last_peered":"2024-09-14T22:38:12.023799+0000","last_clean":"2024-09-14T22:38:12.023799+0000","last_became_active":"2024-09-14T22:36:06.039600+0000","last_became_peered":"2024-09-14T22:36:06.039600+0000","last_unstale":"2024-09-14T22:38:12.023799+0000","last_undegraded":"2024-09-14T22:38:12.023799+0000","last_fullsized":"2024-09-14T22:38:12.023799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:49:09.245563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021617870000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"118'827","reported_seq":8676,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-14T22:38:10.986050+0000","last_change":"2024-09-14T22:37:16.669227+0000","last_active":"2024-09-14T22:38:10.986050+0000","last_peered":"2024-09-14T22:38:10.986050+0000","last_clean":"2024-09-14T22:38:10.986050+0000","last_became_active":"2024-09-14T22:37:16.662230+0000","last_became_peered":"2024-09-14T22:37:16.662230+0000","last_unstale":"2024-09-14T22:38:10.986050+0000","last_undegraded":"2024-09-14T22:38:10.986050+0000","last_fullsized":"2024-09-14T22:38:10.986050+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":827,"log_dups_size":0,"ondisk_log_size":827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:38:34.241178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021500769999999998,"stat_sum":{"num_bytes":79368880,"num_objects":62,"num_object_clones":13,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":18,"num_read":9986,"num_read_kb":29516,"num_write":1160,"num_write_kb":14916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":4851034,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":147,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-14T22:38:10.736384+0000","last_change":"2024-09-14T22:38:05.724182+0000","last_active":"2024-09-14T22:38:10.736384+0000","last_peered":"2024-09-14T22:38:10.736384+0000","last_clean":"2024-09-14T22:38:10.736384+0000","last_became_active":"2024-09-14T22:38:05.723636+0000","last_became_peered":"2024-09-14T22:38:05.723636+0000","last_unstale":"2024-09-14T22:38:10.736384+0000","last_undegraded":"2024-09-14T22:38:10.736384+0000","last_fullsized":"2024-09-14T22:38:10.736384+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:14:09.687576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"119'341","reported_seq":5025,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.355946+0000","last_change":"2024-09-14T22:38:08.744952+0000","last_active":"2024-09-14T22:38:12.355946+0000","last_peered":"2024-09-14T22:38:12.355946+0000","last_clean":"2024-09-14T22:38:12.355946+0000","last_became_active":"2024-09-14T22:38:08.743679+0000","last_became_peered":"2024-09-14T22:38:08.743679+0000","last_unstale":"2024-09-14T22:38:12.355946+0000","last_undegraded":"2024-09-14T22:38:12.355946+0000","last_fullsized":"2024-09-14T22:38:12.355946+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:06:41.843049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085549400000000005,"stat_sum":{"num_bytes":63327300,"num_objects":50,"num_object_clones":13,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":17,"num_read":4244,"num_read_kb":21096,"num_write":357,"num_write_kb":7081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":3679238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":229,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.004152+0000","last_change":"2024-09-14T22:38:03.976161+0000","last_active":"2024-09-14T22:38:12.004152+0000","last_peered":"2024-09-14T22:38:12.004152+0000","last_clean":"2024-09-14T22:38:12.004152+0000","last_became_active":"2024-09-14T22:38:03.975240+0000","last_became_peered":"2024-09-14T22:38:03.975240+0000","last_unstale":"2024-09-14T22:38:12.004152+0000","last_undegraded":"2024-09-14T22:38:12.004152+0000","last_fullsized":"2024-09-14T22:38:12.004152+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:34:10.534483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010817520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"119'998","reported_seq":4189,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.023938+0000","last_change":"2024-09-14T22:38:09.742778+0000","last_active":"2024-09-14T22:38:12.023938+0000","last_peered":"2024-09-14T22:38:12.023938+0000","last_clean":"2024-09-14T22:38:12.023938+0000","last_became_active":"2024-09-14T22:38:09.741015+0000","last_became_peered":"2024-09-14T22:38:09.741015+0000","last_unstale":"2024-09-14T22:38:12.023938+0000","last_undegraded":"2024-09-14T22:38:12.023938+0000","last_fullsized":"2024-09-14T22:38:12.023938+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":998,"log_dups_size":0,"ondisk_log_size":998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:41:12.099183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57348378,"num_objects":67,"num_object_clones":14,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":24,"num_read":7203,"num_read_kb":15917,"num_write":1504,"num_write_kb":11829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":2936757,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":222,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.023772+0000","last_change":"2024-09-14T22:37:13.625072+0000","last_active":"2024-09-14T22:38:12.023772+0000","last_peered":"2024-09-14T22:38:12.023772+0000","last_clean":"2024-09-14T22:38:12.023772+0000","last_became_active":"2024-09-14T22:37:13.624669+0000","last_became_peered":"2024-09-14T22:37:13.624669+0000","last_unstale":"2024-09-14T22:38:12.023772+0000","last_undegraded":"2024-09-14T22:38:12.023772+0000","last_fullsized":"2024-09-14T22:38:12.023772+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:54:20.545098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011401880000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":365,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.004179+0000","last_change":"2024-09-14T22:36:00.402862+0000","last_active":"2024-09-14T22:38:12.004179+0000","last_peered":"2024-09-14T22:38:12.004179+0000","last_clean":"2024-09-14T22:38:12.004179+0000","last_became_active":"2024-09-14T22:36:00.400529+0000","last_became_peered":"2024-09-14T22:36:00.400529+0000","last_unstale":"2024-09-14T22:38:12.004179+0000","last_undegraded":"2024-09-14T22:38:12.004179+0000","last_fullsized":"2024-09-14T22:38:12.004179+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_clean_scrub_stamp":"2024-09-14T22:35:58.989025+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:07:12.951728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"119'561","reported_seq":7095,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.269979+0000","last_change":"2024-09-14T22:38:08.741268+0000","last_active":"2024-09-14T22:38:12.269979+0000","last_peered":"2024-09-14T22:38:12.269979+0000","last_clean":"2024-09-14T22:38:12.269979+0000","last_became_active":"2024-09-14T22:38:08.739090+0000","last_became_peered":"2024-09-14T22:38:08.739090+0000","last_unstale":"2024-09-14T22:38:12.269979+0000","last_undegraded":"2024-09-14T22:38:12.269979+0000","last_fullsized":"2024-09-14T22:38:12.269979+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:04:49.078208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001381511,"stat_sum":{"num_bytes":39365030,"num_objects":59,"num_object_clones":15,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":21,"num_read":7944,"num_read_kb":21366,"num_write":882,"num_write_kb":14459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":18690892,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"118'334","reported_seq":6958,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.277460+0000","last_change":"2024-09-14T22:38:09.737459+0000","last_active":"2024-09-14T22:38:12.277460+0000","last_peered":"2024-09-14T22:38:12.277460+0000","last_clean":"2024-09-14T22:38:12.277460+0000","last_became_active":"2024-09-14T22:38:09.736445+0000","last_became_peered":"2024-09-14T22:38:09.736445+0000","last_unstale":"2024-09-14T22:38:12.277460+0000","last_undegraded":"2024-09-14T22:38:12.277460+0000","last_fullsized":"2024-09-14T22:38:12.277460+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:38:15.357704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060272699999999995,"stat_sum":{"num_bytes":62873388,"num_objects":80,"num_object_clones":20,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":32,"num_read":6782,"num_read_kb":18461,"num_write":357,"num_write_kb":6976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":317,"num_bytes_recovered":16527933,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"119'629","reported_seq":3708,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.079686+0000","last_change":"2024-09-14T22:36:17.776247+0000","last_active":"2024-09-14T22:38:12.079686+0000","last_peered":"2024-09-14T22:38:12.079686+0000","last_clean":"2024-09-14T22:38:12.079686+0000","last_became_active":"2024-09-14T22:36:15.718794+0000","last_became_peered":"2024-09-14T22:36:15.718794+0000","last_unstale":"2024-09-14T22:38:12.079686+0000","last_undegraded":"2024-09-14T22:38:12.079686+0000","last_fullsized":"2024-09-14T22:38:12.079686+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:28.867934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088662699999999997,"stat_sum":{"num_bytes":29108656,"num_objects":58,"num_object_clones":12,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":25,"num_read":5745,"num_read_kb":24933,"num_write":628,"num_write_kb":14488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"119'382","reported_seq":2234,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.572410+0000","last_change":"2024-09-14T22:36:17.733002+0000","last_active":"2024-09-14T22:38:12.572410+0000","last_peered":"2024-09-14T22:38:12.572410+0000","last_clean":"2024-09-14T22:38:12.572410+0000","last_became_active":"2024-09-14T22:36:15.702885+0000","last_became_peered":"2024-09-14T22:36:15.702885+0000","last_unstale":"2024-09-14T22:38:12.572410+0000","last_undegraded":"2024-09-14T22:38:12.572410+0000","last_fullsized":"2024-09-14T22:38:12.572410+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:25:08.258188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015180580000000001,"stat_sum":{"num_bytes":36984245,"num_objects":57,"num_object_clones":17,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":25,"num_read":2775,"num_read_kb":11024,"num_write":350,"num_write_kb":6754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"118'350","reported_seq":6481,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.356011+0000","last_change":"2024-09-14T22:38:08.740002+0000","last_active":"2024-09-14T22:38:12.356011+0000","last_peered":"2024-09-14T22:38:12.356011+0000","last_clean":"2024-09-14T22:38:12.356011+0000","last_became_active":"2024-09-14T22:38:08.738830+0000","last_became_peered":"2024-09-14T22:38:08.738830+0000","last_unstale":"2024-09-14T22:38:12.356011+0000","last_undegraded":"2024-09-14T22:38:12.356011+0000","last_fullsized":"2024-09-14T22:38:12.356011+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:24:20.095876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082782099999999996,"stat_sum":{"num_bytes":50022508,"num_objects":57,"num_object_clones":17,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":24,"num_read":5745,"num_read_kb":33424,"num_write":436,"num_write_kb":17783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":23218688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"119'494","reported_seq":3705,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.581100+0000","last_change":"2024-09-14T22:36:18.009776+0000","last_active":"2024-09-14T22:38:12.581100+0000","last_peered":"2024-09-14T22:38:12.581100+0000","last_clean":"2024-09-14T22:38:12.581100+0000","last_became_active":"2024-09-14T22:36:15.723205+0000","last_became_peered":"2024-09-14T22:36:15.723205+0000","last_unstale":"2024-09-14T22:38:12.581100+0000","last_undegraded":"2024-09-14T22:38:12.581100+0000","last_fullsized":"2024-09-14T22:38:12.581100+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:41:33.559201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087783999999999996,"stat_sum":{"num_bytes":26214933,"num_objects":56,"num_object_clones":15,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":18,"num_read":4045,"num_read_kb":15969,"num_write":635,"num_write_kb":5854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"119'400","reported_seq":2269,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.038773+0000","last_change":"2024-09-14T22:38:09.739942+0000","last_active":"2024-09-14T22:38:12.038773+0000","last_peered":"2024-09-14T22:38:12.038773+0000","last_clean":"2024-09-14T22:38:12.038773+0000","last_became_active":"2024-09-14T22:38:09.738915+0000","last_became_peered":"2024-09-14T22:38:09.738915+0000","last_unstale":"2024-09-14T22:38:12.038773+0000","last_undegraded":"2024-09-14T22:38:12.038773+0000","last_fullsized":"2024-09-14T22:38:12.038773+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:55:44.728937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000776448,"stat_sum":{"num_bytes":28271172,"num_objects":47,"num_object_clones":10,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":19,"num_read":2294,"num_read_kb":8129,"num_write":486,"num_write_kb":3705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":4944418,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"110'349","reported_seq":7872,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.004885+0000","last_change":"2024-09-14T22:36:18.010182+0000","last_active":"2024-09-14T22:38:12.004885+0000","last_peered":"2024-09-14T22:38:12.004885+0000","last_clean":"2024-09-14T22:38:12.004885+0000","last_became_active":"2024-09-14T22:36:15.719693+0000","last_became_peered":"2024-09-14T22:36:15.719693+0000","last_unstale":"2024-09-14T22:38:12.004885+0000","last_undegraded":"2024-09-14T22:38:12.004885+0000","last_fullsized":"2024-09-14T22:38:12.004885+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:38.843525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001045011,"stat_sum":{"num_bytes":25859668,"num_objects":54,"num_object_clones":12,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":16,"num_read":8145,"num_read_kb":23241,"num_write":432,"num_write_kb":9514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"119'376","reported_seq":3585,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:11.991625+0000","last_change":"2024-09-14T22:37:26.718624+0000","last_active":"2024-09-14T22:38:11.991625+0000","last_peered":"2024-09-14T22:38:11.991625+0000","last_clean":"2024-09-14T22:38:11.991625+0000","last_became_active":"2024-09-14T22:37:26.717088+0000","last_became_peered":"2024-09-14T22:37:26.717088+0000","last_unstale":"2024-09-14T22:38:11.991625+0000","last_undegraded":"2024-09-14T22:38:11.991625+0000","last_fullsized":"2024-09-14T22:38:11.991625+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:04:23.164813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094227299999999998,"stat_sum":{"num_bytes":42221837,"num_objects":60,"num_object_clones":17,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":24,"num_read":3287,"num_read_kb":29016,"num_write":475,"num_write_kb":14729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"119'570","reported_seq":2158,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.677675+0000","last_change":"2024-09-14T22:36:18.009347+0000","last_active":"2024-09-14T22:38:12.677675+0000","last_peered":"2024-09-14T22:38:12.677675+0000","last_clean":"2024-09-14T22:38:12.677675+0000","last_became_active":"2024-09-14T22:36:15.715569+0000","last_became_peered":"2024-09-14T22:36:15.715569+0000","last_unstale":"2024-09-14T22:38:12.677675+0000","last_undegraded":"2024-09-14T22:38:12.677675+0000","last_fullsized":"2024-09-14T22:38:12.677675+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:49:48.308579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054921600000000005,"stat_sum":{"num_bytes":31469256,"num_objects":46,"num_object_clones":9,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":21,"num_read":3278,"num_read_kb":11514,"num_write":798,"num_write_kb":10917,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"119'404","reported_seq":7857,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.827674+0000","last_change":"2024-09-14T22:36:17.720448+0000","last_active":"2024-09-14T22:38:12.827674+0000","last_peered":"2024-09-14T22:38:12.827674+0000","last_clean":"2024-09-14T22:38:12.827674+0000","last_became_active":"2024-09-14T22:36:15.716696+0000","last_became_peered":"2024-09-14T22:36:15.716696+0000","last_unstale":"2024-09-14T22:38:12.827674+0000","last_undegraded":"2024-09-14T22:38:12.827674+0000","last_fullsized":"2024-09-14T22:38:12.827674+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:02:57.573600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086547600000000001,"stat_sum":{"num_bytes":64820745,"num_objects":61,"num_object_clones":11,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":22,"num_read":8971,"num_read_kb":49931,"num_write":453,"num_write_kb":13097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"111'432","reported_seq":6982,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T22:38:12.004646+0000","last_change":"2024-09-14T22:38:10.747323+0000","last_active":"2024-09-14T22:38:12.004646+0000","last_peered":"2024-09-14T22:38:12.004646+0000","last_clean":"2024-09-14T22:38:12.004646+0000","last_became_active":"2024-09-14T22:38:10.744013+0000","last_became_peered":"2024-09-14T22:38:10.744013+0000","last_unstale":"2024-09-14T22:38:12.004646+0000","last_undegraded":"2024-09-14T22:38:12.004646+0000","last_fullsized":"2024-09-14T22:38:12.004646+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:28:18.109354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010286189999999999,"stat_sum":{"num_bytes":55135358,"num_objects":67,"num_object_clones":16,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":26,"num_read":6705,"num_read_kb":34970,"num_write":509,"num_write_kb":15408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":14511090,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1384310174,"num_objects":1855,"num_object_clones":465,"num_object_copies":3710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1855,"num_whiteouts":734,"num_read":188806,"num_read_kb":782271,"num_write":21965,"num_write_kb":378596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3299,"num_bytes_recovered":173602381,"num_keys_recovered":342,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53800960,"data_stored":500243010,"data_compressed":14235300,"data_compressed_allocated":43307008,"data_compressed_original":494040043,"omap_allocated":4238,"internal_metadata":0},"log_size":16721,"ondisk_log_size":16721,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542174,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62248,"kb_used_data":8848,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94309592,"statfs":{"total":96636764160,"available":96573022208,"internally_reserved":0,"allocated":9060352,"data_stored":67098004,"data_compressed":2177226,"data_compressed_allocated":5918720,"data_compressed_original":65424589,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542175,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43556,"kb_used_data":4048,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328284,"statfs":{"total":96636764160,"available":96592162816,"internally_reserved":0,"allocated":4145152,"data_stored":34577091,"data_compressed":831425,"data_compressed_allocated":2396160,"data_compressed_original":33731651,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542175,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57420,"kb_used_data":8188,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94314420,"statfs":{"total":96636764160,"available":96577966080,"internally_reserved":0,"allocated":8384512,"data_stored":61568834,"data_compressed":1773874,"data_compressed_allocated":5619712,"data_compressed_original":60098770,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542174,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67836,"kb_used_data":10284,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94304004,"statfs":{"total":96636764160,"available":96567300096,"internally_reserved":0,"allocated":10530816,"data_stored":87237733,"data_compressed":2540736,"data_compressed_allocated":7221248,"data_compressed_original":85465266,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542175,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66064,"kb_used_data":8508,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94305776,"statfs":{"total":96636764160,"available":96569114624,"internally_reserved":0,"allocated":8712192,"data_stored":66173292,"data_compressed":2075267,"data_compressed_allocated":5791744,"data_compressed_original":64612085,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":66,"seq":283467841549,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43004,"kb_used_data":6176,"kb_used_omap":6,"kb_used_meta":36793,"kb_avail":94328836,"statfs":{"total":96636764160,"available":96592728064,"internally_reserved":0,"allocated":6324224,"data_stored":46361629,"data_compressed":1024509,"data_compressed_allocated":4083712,"data_compressed_original":45194103,"omap_allocated":7025,"internal_metadata":37676175},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61168,"kb_used_data":7764,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94310672,"statfs":{"total":96636764160,"available":96574128128,"internally_reserved":0,"allocated":7950336,"data_stored":58881442,"data_compressed":1659920,"data_compressed_allocated":5287936,"data_compressed_original":57466459,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542174,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41056,"kb_used_data":9868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330784,"statfs":{"total":96636764160,"available":96594722816,"internally_reserved":0,"allocated":10104832,"data_stored":84816203,"data_compressed":2161236,"data_compressed_allocated":7045120,"data_compressed_original":83185808,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8732672,"data_stored":83708538,"data_compressed":2216917,"data_compressed_allocated":7086080,"data_compressed_original":82747536,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6541312,"data_stored":58223033,"data_compressed":1659920,"data_compressed_allocated":5287936,"data_compressed_original":57466459,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4775936,"data_stored":45475998,"data_compressed":922200,"data_compressed_allocated":3969024,"data_compressed_original":44981111,"omap_allocated":4238,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7294976,"data_stored":65508730,"data_compressed":2075267,"data_compressed_allocated":5791744,"data_compressed_original":64612085,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":9138176,"data_stored":86617576,"data_compressed":2556118,"data_compressed_allocated":7241728,"data_compressed_original":85506226,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6979584,"data_stored":60923025,"data_compressed":1784145,"data_compressed_allocated":5632000,"data_compressed_original":60115154,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2752512,"data_stored":33943036,"data_compressed":849316,"data_compressed_allocated":2416640,"data_compressed_original":33760323,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":7585792,"data_stored":65843074,"data_compressed":2171417,"data_compressed_allocated":5881856,"data_compressed_original":64851149,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:38:14.609 INFO:tasks.thrashosds.thrasher:Setting 2.0 to [4, 2] 2024-09-14T22:38:14.609 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '2.0', '4', '2'] 2024-09-14T22:38:14.609 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 2.0 4 2 2024-09-14T22:38:14.623 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:14.618+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:14.646 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-14T22:38:14.662 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:38:14.662 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:38:14.685 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:38:14.686 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:38:14.693 INFO:teuthology.orchestra.run.smithi001.stdout:5837 write 0xcd8b9fd thru 0xcd97e6e (0xc472 bytes) 2024-09-14T22:38:14.701 INFO:teuthology.orchestra.run.smithi001.stdout:5838 read 0x578d221 thru 0x57962e8 (0x90c8 bytes) 2024-09-14T22:38:14.702 INFO:teuthology.orchestra.run.smithi001.stdout:5840 punch from 0x488994f to 0x488ccbe, (0x336f bytes) 2024-09-14T22:38:14.702 INFO:teuthology.orchestra.run.smithi001.stdout:5841 writesame 0x6ebf414 thru 0x6ecc585 data_size 0x53(0xd172 bytes) 2024-09-14T22:38:14.710 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:14.711 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:14.715 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:38:14.715 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:38:14.716 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:14.710+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:14.724 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:38:14.725 INFO:teuthology.orchestra.run.smithi001.stdout:5842 trunc from 0xdb3fa98 to 0x5f739 2024-09-14T22:38:14.737 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:38:14.737 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:38:14.780 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:38:14.780 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:38:14.783 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:14.800 INFO:teuthology.orchestra.run.smithi001.stdout:5845 write 0x837cbbf thru 0x8389ad1 (0xcf13 bytes) 2024-09-14T22:38:14.802 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:38:14.802 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:38:14.804 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:14.804 INFO:teuthology.orchestra.run.smithi001.stdout:5847 write 0x75c885d thru 0x75c900c (0x7b0 bytes) 2024-09-14T22:38:14.808 INFO:teuthology.orchestra.run.smithi001.stdout:5848 read 0x1e08942 thru 0x1e14316 (0xb9d5 bytes) 2024-09-14T22:38:14.817 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:14.810+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:14.820 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:38:14.820 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:38:14.821 INFO:teuthology.orchestra.run.smithi001.stdout:5849 write 0xc1363af thru 0xc13a091 (0x3ce3 bytes) 2024-09-14T22:38:14.824 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:14.824 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:14.825 INFO:teuthology.orchestra.run.smithi001.stdout:5852 write 0xa65898f thru 0xa66315f (0xa7d1 bytes) 2024-09-14T22:38:14.835 INFO:teuthology.orchestra.run.smithi001.stdout:5853 trunc from 0xc13a092 to 0x667c7b5 2024-09-14T22:38:14.839 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:38:14.839 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:38:14.863 INFO:teuthology.orchestra.run.smithi001.stdout:5854 write 0x67b4839 thru 0x67c053b (0xbd03 bytes) 2024-09-14T22:38:14.866 INFO:teuthology.orchestra.run.smithi001.stdout:5857 punch from 0x414e2bc to 0x414f068, (0xdac bytes) 2024-09-14T22:38:14.867 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:14.870 INFO:teuthology.orchestra.run.smithi001.stdout:5859 trunc from 0x67c053c to 0x92d35a9 2024-09-14T22:38:14.878 INFO:teuthology.orchestra.run.smithi001.stdout:5860 read 0x74e71d3 thru 0x74f1e96 (0xacc4 bytes) 2024-09-14T22:38:14.878 INFO:teuthology.orchestra.run.smithi001.stdout:5861 read 0x37c673c thru 0x37c8f3e (0x2803 bytes) 2024-09-14T22:38:14.878 INFO:teuthology.orchestra.run.smithi001.stdout:5862 write 0x5066c66 thru 0x506c9e0 (0x5d7b bytes) 2024-09-14T22:38:14.890 INFO:teuthology.orchestra.run.smithi001.stdout:5863 write 0xa077a2c thru 0xa08577c (0xdd51 bytes) 2024-09-14T22:38:14.897 INFO:teuthology.orchestra.run.smithi001.stdout:5864 trunc from 0xa08577d to 0x6968e3d 2024-09-14T22:38:14.899 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:14.899 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:14.899 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:38:14.899 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:38:14.899 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:14.914 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:38:14.918 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:14.914+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:14.928 INFO:teuthology.orchestra.run.smithi001.stdout:5865 writesame 0xe1a19b1 thru 0xe1ac454 data_size 0x86(0xaaa4 bytes) 2024-09-14T22:38:14.928 INFO:teuthology.orchestra.run.smithi001.stdout:5866 writesame 0x6ea0ec thru 0x6ef4d9 data_size 0x3(0x53ee bytes) 2024-09-14T22:38:14.934 INFO:teuthology.orchestra.run.smithi001.stdout:5867 write 0x1542a4e thru 0x154820e (0x57c1 bytes) 2024-09-14T22:38:14.937 INFO:teuthology.orchestra.run.smithi001.stdout:5868 read 0x37a8995 thru 0x37b011f (0x778b bytes) 2024-09-14T22:38:14.937 INFO:teuthology.orchestra.run.smithi001.stdout:5870 write 0x2d07810 thru 0x2d097f5 (0x1fe6 bytes) 2024-09-14T22:38:14.938 INFO:teuthology.orchestra.run.smithi001.stdout:5871 punch from 0x97c7116 to 0x97d6da8, (0xfc92 bytes) 2024-09-14T22:38:14.938 INFO:teuthology.orchestra.run.smithi001.stdout:5872 read 0x6a5a987 thru 0x6a638dd (0x8f57 bytes) 2024-09-14T22:38:14.938 INFO:teuthology.orchestra.run.smithi001.stdout:5873 write 0x141b682 thru 0x141be8f (0x80e bytes) 2024-09-14T22:38:14.954 INFO:teuthology.orchestra.run.smithi001.stdout:5874 trunc from 0xe1ac455 to 0x96017b 2024-09-14T22:38:15.004 INFO:teuthology.orchestra.run.smithi001.stdout:5875 writesame 0xe87b736 thru 0xe88a407 data_size 0x1(0xecd2 bytes) 2024-09-14T22:38:15.004 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:15.004 INFO:teuthology.orchestra.run.smithi001.stdout:5877 read 0xb856bcb thru 0xb864638 (0xda6e bytes) 2024-09-14T22:38:15.004 INFO:teuthology.orchestra.run.smithi001.stdout:5880 clone 41 order 23 su 262144 sc 15 2024-09-14T22:38:15.014 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2024-09-14T22:38:15.017 INFO:teuthology.orchestra.run.smithi001.stderr:set 2.0 pg_upmap mapping to [4,2] 2024-09-14T22:38:15.018 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:15.010+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:15.040 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:15.040 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:38:15.040 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:38:15.040 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:38:15.040 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.040 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:38:15.040 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:38:15.041 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 135.03135684700001, 2024-09-14T22:38:15.041 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.398029152, 2024-09-14T22:38:15.041 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:15.041 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:15.041 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:15.041 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:15.041 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:38:15.041 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:38:15.041 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 1 2024-09-14T22:38:15.041 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.041 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:15.041 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.041 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:15.041 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:38:15.042 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.042 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.042 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.042 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:15.042 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:38:15.042 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.042 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.042 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.042 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:15.042 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006183+0000", 2024-09-14T22:38:15.042 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9059999999999998e-06 2024-09-14T22:38:15.042 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.042 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.042 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:15.043 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006198+0000", 2024-09-14T22:38:15.043 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.523e-05 2024-09-14T22:38:15.043 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.043 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.043 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:15.043 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006219+0000", 2024-09-14T22:38:15.043 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1342999999999999e-05 2024-09-14T22:38:15.043 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.043 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.043 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.044 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006240+0000", 2024-09-14T22:38:15.044 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1186999999999999e-05 2024-09-14T22:38:15.044 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.044 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.044 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:15.044 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006363+0000", 2024-09-14T22:38:15.044 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012258200000000001 2024-09-14T22:38:15.044 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.044 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.044 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:38:15.044 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006392+0000", 2024-09-14T22:38:15.044 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9607e-05 2024-09-14T22:38:15.045 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.045 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.045 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:15.045 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401084+0000", 2024-09-14T22:38:15.045 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.39469142699999998 2024-09-14T22:38:15.045 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.045 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.045 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:15.045 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401394+0000", 2024-09-14T22:38:15.045 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030987799999999998 2024-09-14T22:38:15.045 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.045 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.045 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:38:15.046 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401860+0000", 2024-09-14T22:38:15.046 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00046654299999999998 2024-09-14T22:38:15.046 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.046 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.046 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:38:15.046 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.403386+0000", 2024-09-14T22:38:15.046 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001525322 2024-09-14T22:38:15.046 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.046 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.046 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:15.046 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404042+0000", 2024-09-14T22:38:15.046 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065676099999999996 2024-09-14T22:38:15.046 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.046 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.047 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:38:15.047 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404136+0000", 2024-09-14T22:38:15.047 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4032999999999999e-05 2024-09-14T22:38:15.047 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.047 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.047 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:15.047 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404209+0000", 2024-09-14T22:38:15.047 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2333e-05 2024-09-14T22:38:15.047 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.047 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:15.047 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.047 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.047 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.047 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104182 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14352384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:15.048 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:38:15.048 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.548502861000003, 2024-09-14T22:38:15.048 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071642496999999999, 2024-09-14T22:38:15.048 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:15.048 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:15.048 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:15.048 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:15.048 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:15.048 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.048 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104182 2024-09-14T22:38:15.048 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.048 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:15.048 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.049 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:15.049 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:38:15.049 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.049 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.049 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.049 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:15.049 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:38:15.049 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.049 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.049 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.049 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:15.049 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489035+0000", 2024-09-14T22:38:15.049 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3430000000000001e-06 2024-09-14T22:38:15.050 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.050 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.050 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:15.050 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489041+0000", 2024-09-14T22:38:15.050 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8139999999999997e-06 2024-09-14T22:38:15.050 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.050 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.050 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:15.050 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489059+0000", 2024-09-14T22:38:15.050 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8193e-05 2024-09-14T22:38:15.050 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.050 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.050 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489073+0000", 2024-09-14T22:38:15.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:38:15.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:15.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560434+0000", 2024-09-14T22:38:15.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071361463999999999 2024-09-14T22:38:15.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:15.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560545+0000", 2024-09-14T22:38:15.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110263 2024-09-14T22:38:15.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.052 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:15.052 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560676+0000", 2024-09-14T22:38:15.052 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013138000000000001 2024-09-14T22:38:15.052 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.052 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:15.052 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.052 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.052 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104183 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14483456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:15.052 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:38:15.052 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.548379541999999, 2024-09-14T22:38:15.052 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071781943000000001, 2024-09-14T22:38:15.053 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:15.053 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:15.053 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:15.053 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:15.053 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:15.053 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.053 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104183 2024-09-14T22:38:15.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.053 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:15.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:15.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:38:15.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.054 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:15.054 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:38:15.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.054 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:15.054 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489159+0000", 2024-09-14T22:38:15.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5629999999999999e-06 2024-09-14T22:38:15.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:15.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489165+0000", 2024-09-14T22:38:15.055 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7070000000000003e-06 2024-09-14T22:38:15.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:15.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489186+0000", 2024-09-14T22:38:15.055 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.088e-05 2024-09-14T22:38:15.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.056 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489201+0000", 2024-09-14T22:38:15.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5203e-05 2024-09-14T22:38:15.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.056 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:15.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560713+0000", 2024-09-14T22:38:15.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071511447000000006 2024-09-14T22:38:15.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.056 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:15.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560819+0000", 2024-09-14T22:38:15.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106579 2024-09-14T22:38:15.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.057 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:15.057 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560939+0000", 2024-09-14T22:38:15.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119564 2024-09-14T22:38:15.057 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.057 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:15.057 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.057 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104184 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:15.057 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:38:15.058 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.548315099, 2024-09-14T22:38:15.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071970107000000005, 2024-09-14T22:38:15.058 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:15.058 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:15.058 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:15.058 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:15.058 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:15.058 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.058 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104184 2024-09-14T22:38:15.058 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.058 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:15.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.058 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:15.058 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:38:15.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.059 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:15.059 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:38:15.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.059 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:15.059 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489223+0000", 2024-09-14T22:38:15.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4470000000000001e-06 2024-09-14T22:38:15.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:15.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489229+0000", 2024-09-14T22:38:15.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8200000000000002e-06 2024-09-14T22:38:15.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:15.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489247+0000", 2024-09-14T22:38:15.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8150000000000001e-05 2024-09-14T22:38:15.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489260+0000", 2024-09-14T22:38:15.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3570000000000001e-05 2024-09-14T22:38:15.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:15.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560979+0000", 2024-09-14T22:38:15.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071718831999999996 2024-09-14T22:38:15.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:15.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561075+0000", 2024-09-14T22:38:15.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5848999999999997e-05 2024-09-14T22:38:15.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:15.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561191+0000", 2024-09-14T22:38:15.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116439 2024-09-14T22:38:15.061 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.061 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:15.061 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104185 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14745600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:38:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.548255515999998, 2024-09-14T22:38:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072173750999999994, 2024-09-14T22:38:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104185 2024-09-14T22:38:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:38:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:38:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489282+0000", 2024-09-14T22:38:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2500000000000001e-06 2024-09-14T22:38:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489287+0000", 2024-09-14T22:38:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1359999999999996e-06 2024-09-14T22:38:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489304+0000", 2024-09-14T22:38:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7033000000000001e-05 2024-09-14T22:38:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489318+0000", 2024-09-14T22:38:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3934000000000001e-05 2024-09-14T22:38:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561232+0000", 2024-09-14T22:38:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071913759999999993 2024-09-14T22:38:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561340+0000", 2024-09-14T22:38:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107921 2024-09-14T22:38:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561455+0000", 2024-09-14T22:38:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011471700000000001 2024-09-14T22:38:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104186 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14876672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:38:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.548199607000001, 2024-09-14T22:38:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072393374999999996, 2024-09-14T22:38:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104186 2024-09-14T22:38:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:38:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:38:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489338+0000", 2024-09-14T22:38:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.48e-06 2024-09-14T22:38:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489366+0000", 2024-09-14T22:38:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7307000000000001e-05 2024-09-14T22:38:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489385+0000", 2024-09-14T22:38:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9283000000000002e-05 2024-09-14T22:38:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489399+0000", 2024-09-14T22:38:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.452e-05 2024-09-14T22:38:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561494+0000", 2024-09-14T22:38:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072094885999999997 2024-09-14T22:38:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561604+0000", 2024-09-14T22:38:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010925900000000001 2024-09-14T22:38:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561730+0000", 2024-09-14T22:38:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012663999999999999 2024-09-14T22:38:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104187 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15007744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:38:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.548116237999999, 2024-09-14T22:38:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072599143000000005, 2024-09-14T22:38:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104187 2024-09-14T22:38:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:38:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:38:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489422+0000", 2024-09-14T22:38:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5340000000000001e-06 2024-09-14T22:38:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489428+0000", 2024-09-14T22:38:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1430000000000003e-06 2024-09-14T22:38:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489445+0000", 2024-09-14T22:38:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7042999999999999e-05 2024-09-14T22:38:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489458+0000", 2024-09-14T22:38:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3474e-05 2024-09-14T22:38:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.074 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:15.074 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561771+0000", 2024-09-14T22:38:15.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072312600000000005 2024-09-14T22:38:15.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.074 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:15.074 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561885+0000", 2024-09-14T22:38:15.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011411799999999999 2024-09-14T22:38:15.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.074 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:15.074 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562019+0000", 2024-09-14T22:38:15.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013423100000000001 2024-09-14T22:38:15.075 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.075 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:15.075 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.075 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.075 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104188 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15138816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:15.075 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:38:15.075 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.548056314, 2024-09-14T22:38:15.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072824624000000004, 2024-09-14T22:38:15.075 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:15.075 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:15.075 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:15.075 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:15.076 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:15.076 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.076 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104188 2024-09-14T22:38:15.076 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.076 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:15.076 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.076 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:15.076 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:38:15.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.076 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.076 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.076 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:15.076 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:38:15.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.077 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.077 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:15.077 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489481+0000", 2024-09-14T22:38:15.077 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.22e-06 2024-09-14T22:38:15.077 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.077 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:15.077 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489487+0000", 2024-09-14T22:38:15.077 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6930000000000004e-06 2024-09-14T22:38:15.077 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.077 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:15.078 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489508+0000", 2024-09-14T22:38:15.078 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1135999999999999e-05 2024-09-14T22:38:15.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.078 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.078 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.078 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489522+0000", 2024-09-14T22:38:15.078 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3817e-05 2024-09-14T22:38:15.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.078 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.078 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:15.078 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562066+0000", 2024-09-14T22:38:15.078 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072543813999999998 2024-09-14T22:38:15.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.078 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.078 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:15.079 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562177+0000", 2024-09-14T22:38:15.079 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000111264 2024-09-14T22:38:15.079 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.079 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.079 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:15.079 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562305+0000", 2024-09-14T22:38:15.079 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012768 2024-09-14T22:38:15.079 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.079 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:15.079 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.079 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.079 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.079 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104189 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15269888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:15.079 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:38:15.079 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.547995864999997, 2024-09-14T22:38:15.080 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073061300999999995, 2024-09-14T22:38:15.080 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:15.080 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:15.080 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:15.080 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:15.080 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:15.080 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.080 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104189 2024-09-14T22:38:15.080 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.080 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:15.080 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.080 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:15.080 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:38:15.081 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.081 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.081 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.081 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:15.081 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:38:15.081 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.081 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.081 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.081 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:15.081 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489542+0000", 2024-09-14T22:38:15.081 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1659999999999999e-06 2024-09-14T22:38:15.082 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.082 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.082 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:15.082 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489561+0000", 2024-09-14T22:38:15.082 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.908e-05 2024-09-14T22:38:15.082 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.082 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.082 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:15.082 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489579+0000", 2024-09-14T22:38:15.082 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8536999999999999e-05 2024-09-14T22:38:15.082 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.082 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.082 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.082 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489593+0000", 2024-09-14T22:38:15.083 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3977e-05 2024-09-14T22:38:15.083 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.083 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.083 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:15.083 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562347+0000", 2024-09-14T22:38:15.083 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072754092000000006 2024-09-14T22:38:15.083 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.083 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.083 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:15.083 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562458+0000", 2024-09-14T22:38:15.083 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110722 2024-09-14T22:38:15.083 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.083 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.083 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562602+0000", 2024-09-14T22:38:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014372699999999999 2024-09-14T22:38:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104190 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:38:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.547923568999998, 2024-09-14T22:38:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.07326133, 2024-09-14T22:38:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104190 2024-09-14T22:38:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:38:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:38:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489614+0000", 2024-09-14T22:38:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1999999999999999e-06 2024-09-14T22:38:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489620+0000", 2024-09-14T22:38:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.524e-06 2024-09-14T22:38:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489637+0000", 2024-09-14T22:38:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7825999999999998e-05 2024-09-14T22:38:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489671+0000", 2024-09-14T22:38:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3726000000000002e-05 2024-09-14T22:38:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562644+0000", 2024-09-14T22:38:15.088 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072973089000000005 2024-09-14T22:38:15.088 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.088 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.088 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:15.088 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562762+0000", 2024-09-14T22:38:15.088 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011743600000000001 2024-09-14T22:38:15.088 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.088 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.088 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:15.088 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562874+0000", 2024-09-14T22:38:15.088 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011252900000000001 2024-09-14T22:38:15.088 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.088 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:15.088 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104191 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15532032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:38:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.547815145999998, 2024-09-14T22:38:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073311166999999997, 2024-09-14T22:38:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104191 2024-09-14T22:38:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:38:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:38:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489723+0000", 2024-09-14T22:38:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6330000000000001e-06 2024-09-14T22:38:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489730+0000", 2024-09-14T22:38:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2300000000000002e-06 2024-09-14T22:38:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:15.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489751+0000", 2024-09-14T22:38:15.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0849999999999999e-05 2024-09-14T22:38:15.092 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.092 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.092 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489766+0000", 2024-09-14T22:38:15.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.524e-05 2024-09-14T22:38:15.092 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.092 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.092 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:15.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562896+0000", 2024-09-14T22:38:15.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073129662999999998 2024-09-14T22:38:15.092 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.093 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.093 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:15.093 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562960+0000", 2024-09-14T22:38:15.093 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3689999999999995e-05 2024-09-14T22:38:15.093 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.093 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.093 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:15.093 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563033+0000", 2024-09-14T22:38:15.093 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2861000000000006e-05 2024-09-14T22:38:15.093 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.093 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:15.093 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.093 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.094 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.094 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104192 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:15.094 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:38:15.094 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.547719340999997, 2024-09-14T22:38:15.094 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073463632000000001, 2024-09-14T22:38:15.094 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:15.094 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:15.094 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:15.094 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:15.094 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:15.094 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.094 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104192 2024-09-14T22:38:15.094 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.094 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:38:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:38:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489819+0000", 2024-09-14T22:38:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.424e-06 2024-09-14T22:38:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.096 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:15.096 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489825+0000", 2024-09-14T22:38:15.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.02e-06 2024-09-14T22:38:15.096 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.096 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:15.096 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489844+0000", 2024-09-14T22:38:15.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9760000000000001e-05 2024-09-14T22:38:15.096 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.096 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.096 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489860+0000", 2024-09-14T22:38:15.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5563e-05 2024-09-14T22:38:15.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.097 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:15.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563070+0000", 2024-09-14T22:38:15.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073209724000000004 2024-09-14T22:38:15.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.097 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:15.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563175+0000", 2024-09-14T22:38:15.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105183 2024-09-14T22:38:15.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563281+0000", 2024-09-14T22:38:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105958 2024-09-14T22:38:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104193 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15794176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:38:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.547657174000001, 2024-09-14T22:38:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073600040000000005, 2024-09-14T22:38:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:15.099 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:15.099 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:15.099 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:15.099 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.099 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104193 2024-09-14T22:38:15.099 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.099 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:15.099 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.099 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:15.099 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:38:15.099 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.099 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.099 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:38:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489881+0000", 2024-09-14T22:38:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5460000000000001e-06 2024-09-14T22:38:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489886+0000", 2024-09-14T22:38:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1739999999999999e-06 2024-09-14T22:38:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489912+0000", 2024-09-14T22:38:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6461999999999998e-05 2024-09-14T22:38:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489927+0000", 2024-09-14T22:38:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:38:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563310+0000", 2024-09-14T22:38:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073382977000000002 2024-09-14T22:38:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563385+0000", 2024-09-14T22:38:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5218999999999999e-05 2024-09-14T22:38:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563479+0000", 2024-09-14T22:38:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4622000000000003e-05 2024-09-14T22:38:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104194 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15925248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:38:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.547590581999998, 2024-09-14T22:38:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073745434999999998, 2024-09-14T22:38:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104194 2024-09-14T22:38:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:38:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:38:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489947+0000", 2024-09-14T22:38:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.327e-06 2024-09-14T22:38:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489953+0000", 2024-09-14T22:38:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9329999999999998e-06 2024-09-14T22:38:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489971+0000", 2024-09-14T22:38:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7994000000000001e-05 2024-09-14T22:38:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489985+0000", 2024-09-14T22:38:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4226000000000001e-05 2024-09-14T22:38:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563510+0000", 2024-09-14T22:38:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073524667000000002 2024-09-14T22:38:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563597+0000", 2024-09-14T22:38:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6550000000000003e-05 2024-09-14T22:38:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563691+0000", 2024-09-14T22:38:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4738e-05 2024-09-14T22:38:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104195 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:38:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.547532384999997, 2024-09-14T22:38:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073876598000000002, 2024-09-14T22:38:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104195 2024-09-14T22:38:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:38:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:38:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490006+0000", 2024-09-14T22:38:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4699999999999999e-06 2024-09-14T22:38:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490011+0000", 2024-09-14T22:38:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.366e-06 2024-09-14T22:38:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490046+0000", 2024-09-14T22:38:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5496e-05 2024-09-14T22:38:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490061+0000", 2024-09-14T22:38:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5054e-05 2024-09-14T22:38:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563719+0000", 2024-09-14T22:38:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073657412000000005 2024-09-14T22:38:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563792+0000", 2024-09-14T22:38:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3319999999999996e-05 2024-09-14T22:38:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563881+0000", 2024-09-14T22:38:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8480000000000007e-05 2024-09-14T22:38:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104196 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16187392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:38:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.547455235999998, 2024-09-14T22:38:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073997496999999995, 2024-09-14T22:38:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104196 2024-09-14T22:38:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:38:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:38:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490083+0000", 2024-09-14T22:38:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.347e-06 2024-09-14T22:38:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490089+0000", 2024-09-14T22:38:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9630000000000003e-06 2024-09-14T22:38:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490107+0000", 2024-09-14T22:38:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8859999999999999e-05 2024-09-14T22:38:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490122+0000", 2024-09-14T22:38:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.432e-05 2024-09-14T22:38:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563910+0000", 2024-09-14T22:38:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073788753999999998 2024-09-14T22:38:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563988+0000", 2024-09-14T22:38:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7862999999999996e-05 2024-09-14T22:38:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564079+0000", 2024-09-14T22:38:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0389999999999993e-05 2024-09-14T22:38:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104197 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16318464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:38:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.547363607000001, 2024-09-14T22:38:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074103547000000006, 2024-09-14T22:38:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104197 2024-09-14T22:38:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:38:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:38:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490174+0000", 2024-09-14T22:38:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.37e-06 2024-09-14T22:38:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490197+0000", 2024-09-14T22:38:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3017e-05 2024-09-14T22:38:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490219+0000", 2024-09-14T22:38:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.181e-05 2024-09-14T22:38:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490240+0000", 2024-09-14T22:38:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0786000000000001e-05 2024-09-14T22:38:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564109+0000", 2024-09-14T22:38:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073869031000000002 2024-09-14T22:38:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564186+0000", 2024-09-14T22:38:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6823999999999995e-05 2024-09-14T22:38:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564276+0000", 2024-09-14T22:38:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0709000000000002e-05 2024-09-14T22:38:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104198 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16449536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:38:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.547271371000001, 2024-09-14T22:38:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074167693000000007, 2024-09-14T22:38:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104198 2024-09-14T22:38:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:38:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:38:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490266+0000", 2024-09-14T22:38:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3599999999999999e-06 2024-09-14T22:38:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490272+0000", 2024-09-14T22:38:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9599999999999997e-06 2024-09-14T22:38:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490291+0000", 2024-09-14T22:38:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8485999999999999e-05 2024-09-14T22:38:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490305+0000", 2024-09-14T22:38:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4327e-05 2024-09-14T22:38:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564306+0000", 2024-09-14T22:38:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074000745000000007 2024-09-14T22:38:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564367+0000", 2024-09-14T22:38:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1501999999999995e-05 2024-09-14T22:38:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564433+0000", 2024-09-14T22:38:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5313000000000003e-05 2024-09-14T22:38:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104199 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16580608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:38:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.547189818, 2024-09-14T22:38:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074231631000000006, 2024-09-14T22:38:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104199 2024-09-14T22:38:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:38:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:38:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490349+0000", 2024-09-14T22:38:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9429999999999999e-06 2024-09-14T22:38:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490355+0000", 2024-09-14T22:38:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1670000000000004e-06 2024-09-14T22:38:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490374+0000", 2024-09-14T22:38:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9613e-05 2024-09-14T22:38:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490405+0000", 2024-09-14T22:38:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0936000000000003e-05 2024-09-14T22:38:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564451+0000", 2024-09-14T22:38:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074045266999999998 2024-09-14T22:38:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564494+0000", 2024-09-14T22:38:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3133000000000002e-05 2024-09-14T22:38:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564578+0000", 2024-09-14T22:38:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4572000000000003e-05 2024-09-14T22:38:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104200 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:38:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.545394590000001, 2024-09-14T22:38:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072582435000000001, 2024-09-14T22:38:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104200 2024-09-14T22:38:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:38:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:38:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492144+0000", 2024-09-14T22:38:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3070000000000001e-06 2024-09-14T22:38:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492155+0000", 2024-09-14T22:38:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0573e-05 2024-09-14T22:38:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492185+0000", 2024-09-14T22:38:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.008e-05 2024-09-14T22:38:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492205+0000", 2024-09-14T22:38:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9882999999999999e-05 2024-09-14T22:38:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564609+0000", 2024-09-14T22:38:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072404181999999997 2024-09-14T22:38:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564667+0000", 2024-09-14T22:38:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8102e-05 2024-09-14T22:38:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564724+0000", 2024-09-14T22:38:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7308000000000001e-05 2024-09-14T22:38:15.135 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.135 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:15.135 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.135 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:15.135 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:15.135 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:15.136 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:38:15.137 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:15.130+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:15.161 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone40 from 0xe88a408 (overlap 0x5f739) to 0x1ca2a 2024-09-14T22:38:15.237 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:15.230+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:15.260 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:15.260 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:15.260 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:38:15.260 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:15.273 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:38:15.346 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-14T22:38:15.346 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:15.342+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:15.439 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:15.430+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:15.473 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:15.474 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:15.474 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:38:15.474 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:38:15.474 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:15.488 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:38:15.501 INFO:teuthology.orchestra.run.smithi001.stdout:5883 writesame 0x75c12d thru 0x7670f0 data_size 0xe(0xafc4 bytes) 2024-09-14T22:38:15.522 INFO:teuthology.orchestra.run.smithi001.stdout:5884 write 0x8a63b4 thru 0x8a7303 (0xf50 bytes) 2024-09-14T22:38:15.531 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:15.531 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:15.531 INFO:teuthology.orchestra.run.smithi001.stdout:5888 read 0x55fc8ee thru 0x55febdb (0x22ee bytes) 2024-09-14T22:38:15.540 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:15.534+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:15.541 INFO:teuthology.orchestra.run.smithi001.stdout:5889 trunc from 0xe88a408 to 0x7b2fd18 2024-09-14T22:38:15.627 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:15.627 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:38:15.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:38:15.627 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:38:15.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.627 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144987 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3424256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:15.627 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549297+0000", 2024-09-14T22:38:15.627 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.075918400999999, 2024-09-14T22:38:15.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.079931556000000001, 2024-09-14T22:38:15.627 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:15.627 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:15.627 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:15.628 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:15.628 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:15.628 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.628 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144987 2024-09-14T22:38:15.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.628 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:15.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:15.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549297+0000", 2024-09-14T22:38:15.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:15.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549297+0000", 2024-09-14T22:38:15.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.629 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:15.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549298+0000", 2024-09-14T22:38:15.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2699999999999999e-07 2024-09-14T22:38:15.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.629 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:15.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549301+0000", 2024-09-14T22:38:15.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2959999999999999e-06 2024-09-14T22:38:15.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:15.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549314+0000", 2024-09-14T22:38:15.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2588999999999999e-05 2024-09-14T22:38:15.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549323+0000", 2024-09-14T22:38:15.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.927e-06 2024-09-14T22:38:15.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:15.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629038+0000", 2024-09-14T22:38:15.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.079714220000000002 2024-09-14T22:38:15.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:15.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629116+0000", 2024-09-14T22:38:15.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7966000000000005e-05 2024-09-14T22:38:15.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:15.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629228+0000", 2024-09-14T22:38:15.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112831 2024-09-14T22:38:15.631 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.631 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:15.631 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.632 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144988 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3325952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:15.632 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549335+0000", 2024-09-14T22:38:15.632 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.075880175999998, 2024-09-14T22:38:15.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080179116999999994, 2024-09-14T22:38:15.632 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:15.632 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:15.632 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:15.632 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:15.632 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:15.632 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.632 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144988 2024-09-14T22:38:15.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.633 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:15.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:15.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549335+0000", 2024-09-14T22:38:15.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:15.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549335+0000", 2024-09-14T22:38:15.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:15.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549336+0000", 2024-09-14T22:38:15.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7999999999999995e-07 2024-09-14T22:38:15.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:15.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549339+0000", 2024-09-14T22:38:15.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.836e-06 2024-09-14T22:38:15.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:15.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549352+0000", 2024-09-14T22:38:15.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3069e-05 2024-09-14T22:38:15.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549359+0000", 2024-09-14T22:38:15.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.357e-06 2024-09-14T22:38:15.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:15.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629288+0000", 2024-09-14T22:38:15.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.079928492000000004 2024-09-14T22:38:15.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:15.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629412+0000", 2024-09-14T22:38:15.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012413699999999999 2024-09-14T22:38:15.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:15.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629514+0000", 2024-09-14T22:38:15.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000102546 2024-09-14T22:38:15.636 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.636 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:15.636 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.636 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144989 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3522560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:15.636 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:38:15.636 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.075845768000001, 2024-09-14T22:38:15.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080393122999999997, 2024-09-14T22:38:15.636 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:15.636 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:15.637 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:15.637 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:15.637 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:15.637 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.637 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144989 2024-09-14T22:38:15.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.637 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:15.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:15.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:38:15.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:15.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:38:15.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:15.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:38:15.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3600000000000003e-07 2024-09-14T22:38:15.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:15.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549373+0000", 2024-09-14T22:38:15.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8169999999999999e-06 2024-09-14T22:38:15.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:15.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549385+0000", 2024-09-14T22:38:15.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1803e-05 2024-09-14T22:38:15.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549394+0000", 2024-09-14T22:38:15.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4290000000000007e-06 2024-09-14T22:38:15.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:15.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629576+0000", 2024-09-14T22:38:15.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080181551000000004 2024-09-14T22:38:15.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:15.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629654+0000", 2024-09-14T22:38:15.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7633999999999996e-05 2024-09-14T22:38:15.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:15.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629763+0000", 2024-09-14T22:38:15.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109153 2024-09-14T22:38:15.640 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.640 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:15.640 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.641 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144990 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:15.641 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:38:15.641 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.075802744000001, 2024-09-14T22:38:15.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080614496999999993, 2024-09-14T22:38:15.641 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:15.641 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:15.641 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:15.641 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:15.641 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:15.641 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.641 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144990 2024-09-14T22:38:15.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.642 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:15.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:15.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:38:15.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:15.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:38:15.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:15.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:38:15.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0999999999999997e-07 2024-09-14T22:38:15.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:15.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549419+0000", 2024-09-14T22:38:15.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5339999999999999e-06 2024-09-14T22:38:15.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:15.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549439+0000", 2024-09-14T22:38:15.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0035e-05 2024-09-14T22:38:15.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549449+0000", 2024-09-14T22:38:15.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0429e-05 2024-09-14T22:38:15.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.644 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:15.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629815+0000", 2024-09-14T22:38:15.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080365467999999995 2024-09-14T22:38:15.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.644 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:15.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629922+0000", 2024-09-14T22:38:15.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106914 2024-09-14T22:38:15.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:15.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630027+0000", 2024-09-14T22:38:15.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105207 2024-09-14T22:38:15.645 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.645 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:15.645 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.645 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144991 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 2998272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:15.645 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549463+0000", 2024-09-14T22:38:15.645 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.075752439999999, 2024-09-14T22:38:15.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080847637, 2024-09-14T22:38:15.645 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:15.646 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:15.646 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:15.646 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:15.646 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:15.646 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.646 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144991 2024-09-14T22:38:15.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.646 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:15.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:15.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549463+0000", 2024-09-14T22:38:15.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:15.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549463+0000", 2024-09-14T22:38:15.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:15.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549464+0000", 2024-09-14T22:38:15.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9999999999999997e-07 2024-09-14T22:38:15.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:15.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549467+0000", 2024-09-14T22:38:15.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0800000000000002e-06 2024-09-14T22:38:15.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:15.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549478+0000", 2024-09-14T22:38:15.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1813e-05 2024-09-14T22:38:15.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549488+0000", 2024-09-14T22:38:15.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7030000000000001e-06 2024-09-14T22:38:15.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:15.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630090+0000", 2024-09-14T22:38:15.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080601962999999999 2024-09-14T22:38:15.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:15.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630187+0000", 2024-09-14T22:38:15.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6552000000000007e-05 2024-09-14T22:38:15.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:15.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630311+0000", 2024-09-14T22:38:15.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000123826 2024-09-14T22:38:15.649 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.650 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:15.650 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.650 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144992 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3620864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:15.650 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:38:15.650 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.075716631999999, 2024-09-14T22:38:15.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081097875999999999, 2024-09-14T22:38:15.650 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:15.650 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:15.650 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:15.650 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:15.650 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:15.650 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.651 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144992 2024-09-14T22:38:15.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.651 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:15.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:15.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:38:15.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:15.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:38:15.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:15.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:38:15.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5400000000000001e-07 2024-09-14T22:38:15.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:15.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549502+0000", 2024-09-14T22:38:15.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7889999999999999e-06 2024-09-14T22:38:15.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:15.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549513+0000", 2024-09-14T22:38:15.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1204e-05 2024-09-14T22:38:15.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549523+0000", 2024-09-14T22:38:15.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5990000000000008e-06 2024-09-14T22:38:15.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:15.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630362+0000", 2024-09-14T22:38:15.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080839272000000004 2024-09-14T22:38:15.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:15.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630469+0000", 2024-09-14T22:38:15.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107285 2024-09-14T22:38:15.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:15.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630597+0000", 2024-09-14T22:38:15.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012707300000000001 2024-09-14T22:38:15.654 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.654 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:15.654 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.654 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144993 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:15.654 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:38:15.654 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.075683556000001, 2024-09-14T22:38:15.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081338411999999999, 2024-09-14T22:38:15.655 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:15.655 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:15.655 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:15.655 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:15.655 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:15.655 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.655 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144993 2024-09-14T22:38:15.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.655 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:15.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:15.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:38:15.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:15.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:38:15.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:15.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:38:15.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5300000000000004e-07 2024-09-14T22:38:15.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:15.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549544+0000", 2024-09-14T22:38:15.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1912e-05 2024-09-14T22:38:15.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:15.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549557+0000", 2024-09-14T22:38:15.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2413e-05 2024-09-14T22:38:15.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549567+0000", 2024-09-14T22:38:15.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8660000000000001e-06 2024-09-14T22:38:15.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:15.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630651+0000", 2024-09-14T22:38:15.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081084229999999993 2024-09-14T22:38:15.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:15.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630729+0000", 2024-09-14T22:38:15.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8265999999999999e-05 2024-09-14T22:38:15.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:15.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630870+0000", 2024-09-14T22:38:15.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014107200000000001 2024-09-14T22:38:15.659 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.659 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:15.659 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.659 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144994 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:15.660 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:38:15.660 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.075636588999998, 2024-09-14T22:38:15.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081547958000000004, 2024-09-14T22:38:15.660 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:15.660 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:15.660 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:15.660 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:15.660 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:15.660 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.660 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144994 2024-09-14T22:38:15.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.660 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:15.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:15.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:38:15.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:15.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:38:15.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:15.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:38:15.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6599999999999996e-07 2024-09-14T22:38:15.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:15.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549582+0000", 2024-09-14T22:38:15.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.004e-06 2024-09-14T22:38:15.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:15.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549594+0000", 2024-09-14T22:38:15.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1459e-05 2024-09-14T22:38:15.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549603+0000", 2024-09-14T22:38:15.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6129999999999993e-06 2024-09-14T22:38:15.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:15.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630932+0000", 2024-09-14T22:38:15.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081328558999999995 2024-09-14T22:38:15.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:15.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631027+0000", 2024-09-14T22:38:15.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4742999999999997e-05 2024-09-14T22:38:15.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:15.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631127+0000", 2024-09-14T22:38:15.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9914000000000006e-05 2024-09-14T22:38:15.664 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.664 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:15.664 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.664 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144995 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3391488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:15.664 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549615+0000", 2024-09-14T22:38:15.664 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.075600451, 2024-09-14T22:38:15.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081734251999999993, 2024-09-14T22:38:15.664 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:15.664 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:15.665 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:15.665 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:15.665 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:15.665 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.665 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144995 2024-09-14T22:38:15.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.665 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:15.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:15.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549615+0000", 2024-09-14T22:38:15.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:15.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549615+0000", 2024-09-14T22:38:15.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.666 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:15.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549616+0000", 2024-09-14T22:38:15.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9299999999999997e-07 2024-09-14T22:38:15.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.666 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:15.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549618+0000", 2024-09-14T22:38:15.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7269999999999999e-06 2024-09-14T22:38:15.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:15.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549630+0000", 2024-09-14T22:38:15.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1313e-05 2024-09-14T22:38:15.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549639+0000", 2024-09-14T22:38:15.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4159999999999993e-06 2024-09-14T22:38:15.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:15.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631183+0000", 2024-09-14T22:38:15.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081543633000000004 2024-09-14T22:38:15.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:15.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631252+0000", 2024-09-14T22:38:15.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9567999999999995e-05 2024-09-14T22:38:15.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:15.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631349+0000", 2024-09-14T22:38:15.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6801999999999999e-05 2024-09-14T22:38:15.668 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.668 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:15.668 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.669 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144996 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3194880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:15.669 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:38:15.669 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.075564773, 2024-09-14T22:38:15.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081947213000000005, 2024-09-14T22:38:15.669 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:15.669 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:15.669 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:15.669 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:15.669 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:15.669 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.669 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144996 2024-09-14T22:38:15.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.669 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:15.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:15.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:38:15.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:15.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:38:15.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:15.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:38:15.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3e-07 2024-09-14T22:38:15.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:15.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549654+0000", 2024-09-14T22:38:15.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6900000000000001e-06 2024-09-14T22:38:15.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:15.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549665+0000", 2024-09-14T22:38:15.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1392999999999999e-05 2024-09-14T22:38:15.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549675+0000", 2024-09-14T22:38:15.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5459999999999997e-06 2024-09-14T22:38:15.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:15.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631401+0000", 2024-09-14T22:38:15.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081726036000000002 2024-09-14T22:38:15.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:15.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631492+0000", 2024-09-14T22:38:15.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0886999999999999e-05 2024-09-14T22:38:15.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:15.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631598+0000", 2024-09-14T22:38:15.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106031 2024-09-14T22:38:15.673 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.673 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:15.673 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.673 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144997 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:15.673 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549692+0000", 2024-09-14T22:38:15.673 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.075523108999999, 2024-09-14T22:38:15.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082149925999999998, 2024-09-14T22:38:15.673 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:15.674 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:15.674 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:15.674 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:15.674 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:15.674 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.674 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144997 2024-09-14T22:38:15.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.674 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:15.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:15.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549692+0000", 2024-09-14T22:38:15.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:15.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549692+0000", 2024-09-14T22:38:15.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:15.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549693+0000", 2024-09-14T22:38:15.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.540000000000001e-07 2024-09-14T22:38:15.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:15.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549698+0000", 2024-09-14T22:38:15.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1460000000000004e-06 2024-09-14T22:38:15.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:15.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549715+0000", 2024-09-14T22:38:15.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6983e-05 2024-09-14T22:38:15.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549728+0000", 2024-09-14T22:38:15.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2326e-05 2024-09-14T22:38:15.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:15.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631649+0000", 2024-09-14T22:38:15.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081921830000000001 2024-09-14T22:38:15.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:15.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631726+0000", 2024-09-14T22:38:15.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6759000000000001e-05 2024-09-14T22:38:15.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:15.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631842+0000", 2024-09-14T22:38:15.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011592800000000001 2024-09-14T22:38:15.677 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.677 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:15.678 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.678 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144998 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 2965504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:15.678 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549741+0000", 2024-09-14T22:38:15.678 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.075473816999999, 2024-09-14T22:38:15.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082394832000000001, 2024-09-14T22:38:15.678 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:15.678 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:15.678 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:15.678 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:15.678 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:15.678 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.678 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144998 2024-09-14T22:38:15.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.679 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:15.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:15.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549741+0000", 2024-09-14T22:38:15.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:15.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549741+0000", 2024-09-14T22:38:15.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:15.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549742+0000", 2024-09-14T22:38:15.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.06e-07 2024-09-14T22:38:15.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:15.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549757+0000", 2024-09-14T22:38:15.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4839e-05 2024-09-14T22:38:15.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:15.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549770+0000", 2024-09-14T22:38:15.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2879e-05 2024-09-14T22:38:15.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.681 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.681 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549780+0000", 2024-09-14T22:38:15.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0304e-05 2024-09-14T22:38:15.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.681 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:15.681 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631904+0000", 2024-09-14T22:38:15.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082124168999999997 2024-09-14T22:38:15.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.681 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:15.681 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631995+0000", 2024-09-14T22:38:15.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0993000000000001e-05 2024-09-14T22:38:15.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:15.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632136+0000", 2024-09-14T22:38:15.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014094199999999999 2024-09-14T22:38:15.682 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.682 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:15.682 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.682 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144999 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3588096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:15.682 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:38:15.682 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.075423607000001, 2024-09-14T22:38:15.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082593742999999997, 2024-09-14T22:38:15.683 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:15.683 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:15.683 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:15.683 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:15.683 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:15.683 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.683 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144999 2024-09-14T22:38:15.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.683 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:15.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:15.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:38:15.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:15.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:38:15.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:15.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:38:15.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1999999999999999e-07 2024-09-14T22:38:15.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:15.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549795+0000", 2024-09-14T22:38:15.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9569999999999998e-06 2024-09-14T22:38:15.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:15.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549807+0000", 2024-09-14T22:38:15.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1691999999999999e-05 2024-09-14T22:38:15.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549817+0000", 2024-09-14T22:38:15.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5100000000000004e-06 2024-09-14T22:38:15.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:15.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632190+0000", 2024-09-14T22:38:15.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082373374999999999 2024-09-14T22:38:15.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:15.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632264+0000", 2024-09-14T22:38:15.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3972000000000003e-05 2024-09-14T22:38:15.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:15.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632385+0000", 2024-09-14T22:38:15.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121517 2024-09-14T22:38:15.686 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.687 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:15.687 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.687 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145000 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3063808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:15.687 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549828+0000", 2024-09-14T22:38:15.687 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.075387329000002, 2024-09-14T22:38:15.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082860235000000004, 2024-09-14T22:38:15.687 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:15.687 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:15.687 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:15.687 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:15.687 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:15.687 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.688 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145000 2024-09-14T22:38:15.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.688 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:15.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.688 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:15.688 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549828+0000", 2024-09-14T22:38:15.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.688 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:15.688 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549828+0000", 2024-09-14T22:38:15.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:15.689 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549829+0000", 2024-09-14T22:38:15.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3300000000000002e-07 2024-09-14T22:38:15.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:15.689 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549831+0000", 2024-09-14T22:38:15.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6740000000000001e-06 2024-09-14T22:38:15.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:15.689 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549842+0000", 2024-09-14T22:38:15.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0316000000000001e-05 2024-09-14T22:38:15.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549851+0000", 2024-09-14T22:38:15.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7229999999999999e-06 2024-09-14T22:38:15.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:15.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632448+0000", 2024-09-14T22:38:15.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082596648999999994 2024-09-14T22:38:15.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:15.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632568+0000", 2024-09-14T22:38:15.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119592 2024-09-14T22:38:15.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:15.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632688+0000", 2024-09-14T22:38:15.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120648 2024-09-14T22:38:15.691 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.691 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:15.691 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.691 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145001 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3162112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:15.692 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549863+0000", 2024-09-14T22:38:15.692 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.075352101, 2024-09-14T22:38:15.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083105534999999994, 2024-09-14T22:38:15.692 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:15.692 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:15.692 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:15.692 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:15.692 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:15.692 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.692 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145001 2024-09-14T22:38:15.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.692 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:15.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549863+0000", 2024-09-14T22:38:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549863+0000", 2024-09-14T22:38:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549864+0000", 2024-09-14T22:38:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4300000000000003e-07 2024-09-14T22:38:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549867+0000", 2024-09-14T22:38:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.757e-06 2024-09-14T22:38:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549878+0000", 2024-09-14T22:38:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1345999999999999e-05 2024-09-14T22:38:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549888+0000", 2024-09-14T22:38:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6460000000000005e-06 2024-09-14T22:38:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632754+0000", 2024-09-14T22:38:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082866258999999998 2024-09-14T22:38:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632841+0000", 2024-09-14T22:38:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6971000000000005e-05 2024-09-14T22:38:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:15.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632969+0000", 2024-09-14T22:38:15.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012791300000000001 2024-09-14T22:38:15.696 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.696 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:15.696 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.696 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145002 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3260416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:15.696 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549933+0000", 2024-09-14T22:38:15.696 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.075282598000001, 2024-09-14T22:38:15.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083291027000000004, 2024-09-14T22:38:15.696 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:15.696 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145002 2024-09-14T22:38:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549933+0000", 2024-09-14T22:38:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:15.698 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549933+0000", 2024-09-14T22:38:15.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.698 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:15.698 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549934+0000", 2024-09-14T22:38:15.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8700000000000004e-07 2024-09-14T22:38:15.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.698 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:15.698 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549938+0000", 2024-09-14T22:38:15.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5029999999999999e-06 2024-09-14T22:38:15.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549952+0000", 2024-09-14T22:38:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4382999999999999e-05 2024-09-14T22:38:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549963+0000", 2024-09-14T22:38:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0665999999999999e-05 2024-09-14T22:38:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633025+0000", 2024-09-14T22:38:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083061727000000002 2024-09-14T22:38:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633117+0000", 2024-09-14T22:38:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1812999999999997e-05 2024-09-14T22:38:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633224+0000", 2024-09-14T22:38:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107048 2024-09-14T22:38:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145003 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3457024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549985+0000", 2024-09-14T22:38:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.075230179999998, 2024-09-14T22:38:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083465871999999997, 2024-09-14T22:38:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145003 2024-09-14T22:38:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549985+0000", 2024-09-14T22:38:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549985+0000", 2024-09-14T22:38:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549986+0000", 2024-09-14T22:38:15.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7299999999999995e-07 2024-09-14T22:38:15.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:15.703 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549989+0000", 2024-09-14T22:38:15.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8169999999999999e-06 2024-09-14T22:38:15.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:15.703 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549998+0000", 2024-09-14T22:38:15.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4290000000000007e-06 2024-09-14T22:38:15.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550009+0000", 2024-09-14T22:38:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0633e-05 2024-09-14T22:38:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633275+0000", 2024-09-14T22:38:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083266397000000006 2024-09-14T22:38:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633353+0000", 2024-09-14T22:38:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8129999999999999e-05 2024-09-14T22:38:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633451+0000", 2024-09-14T22:38:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7793000000000004e-05 2024-09-14T22:38:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145004 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3358720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.550029+0000", 2024-09-14T22:38:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.075186608999999, 2024-09-14T22:38:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083662089999999995, 2024-09-14T22:38:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145004 2024-09-14T22:38:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550029+0000", 2024-09-14T22:38:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550029+0000", 2024-09-14T22:38:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550030+0000", 2024-09-14T22:38:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.33e-07 2024-09-14T22:38:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550035+0000", 2024-09-14T22:38:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3430000000000003e-06 2024-09-14T22:38:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550053+0000", 2024-09-14T22:38:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8485e-05 2024-09-14T22:38:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550064+0000", 2024-09-14T22:38:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0264e-05 2024-09-14T22:38:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633502+0000", 2024-09-14T22:38:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083438498999999999 2024-09-14T22:38:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633592+0000", 2024-09-14T22:38:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.022e-05 2024-09-14T22:38:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633691+0000", 2024-09-14T22:38:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8345999999999997e-05 2024-09-14T22:38:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145005 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3555328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.550077+0000", 2024-09-14T22:38:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.075138149000001, 2024-09-14T22:38:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083854340999999999, 2024-09-14T22:38:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145005 2024-09-14T22:38:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550077+0000", 2024-09-14T22:38:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550077+0000", 2024-09-14T22:38:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550078+0000", 2024-09-14T22:38:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8700000000000005e-07 2024-09-14T22:38:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550101+0000", 2024-09-14T22:38:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3632e-05 2024-09-14T22:38:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550115+0000", 2024-09-14T22:38:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3893000000000001e-05 2024-09-14T22:38:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550126+0000", 2024-09-14T22:38:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0426e-05 2024-09-14T22:38:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633742+0000", 2024-09-14T22:38:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083616633999999995 2024-09-14T22:38:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633816+0000", 2024-09-14T22:38:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3690999999999996e-05 2024-09-14T22:38:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633932+0000", 2024-09-14T22:38:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011537800000000001 2024-09-14T22:38:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145006 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3653632~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.550236+0000", 2024-09-14T22:38:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.074979642999999, 2024-09-14T22:38:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083924027999999998, 2024-09-14T22:38:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145006 2024-09-14T22:38:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550236+0000", 2024-09-14T22:38:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550236+0000", 2024-09-14T22:38:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550237+0000", 2024-09-14T22:38:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2929999999999999e-06 2024-09-14T22:38:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550243+0000", 2024-09-14T22:38:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0390000000000001e-06 2024-09-14T22:38:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550256+0000", 2024-09-14T22:38:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3393e-05 2024-09-14T22:38:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550271+0000", 2024-09-14T22:38:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4166e-05 2024-09-14T22:38:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633974+0000", 2024-09-14T22:38:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083703825999999995 2024-09-14T22:38:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.634047+0000", 2024-09-14T22:38:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2935999999999994e-05 2024-09-14T22:38:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.634160+0000", 2024-09-14T22:38:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011237499999999999 2024-09-14T22:38:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.719 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:15.719 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:15.720 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:38:15.721 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:15.714+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:15.795 INFO:teuthology.orchestra.run.smithi001.stdout:5890 punch from 0x438dbf5 to 0x438eed6, (0x12e1 bytes) 2024-09-14T22:38:15.795 INFO:teuthology.orchestra.run.smithi001.stdout:5891 read 0x5ed3ae4 thru 0x5ed3d72 (0x28f bytes) 2024-09-14T22:38:15.796 INFO:teuthology.orchestra.run.smithi001.stdout:5893 read 0x429e592 thru 0x42a4e6a (0x68d9 bytes) 2024-09-14T22:38:15.808 INFO:teuthology.orchestra.run.smithi001.stdout:5894 trunc from 0x7b2fd18 to 0xdff02a2 2024-09-14T22:38:15.821 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:15.814+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:15.822 INFO:teuthology.orchestra.run.smithi001.stdout:5896 write 0x14155bd thru 0x141f813 (0xa257 bytes) 2024-09-14T22:38:15.832 INFO:teuthology.orchestra.run.smithi001.stdout:5897 writesame 0x21e18e9 thru 0x21e66c0 data_size 0x5e(0x4dd8 bytes) 2024-09-14T22:38:15.847 INFO:teuthology.orchestra.run.smithi001.stdout:5900 punch from 0x6bcab9c to 0x6bd938a, (0xe7ee bytes) 2024-09-14T22:38:15.869 INFO:teuthology.orchestra.run.smithi001.stdout:5901 trunc from 0xdff02a2 to 0x7e2aef 2024-09-14T22:38:15.903 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:15.903 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:38:15.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.904 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:156047 3.6 3:67caca63:::rbd_data.113c2d66d993.0000000000000002:head [call rbd.sparse_copyup in=24b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-14T22:38:15.904 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:38:15.901398+0000", 2024-09-14T22:38:15.904 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0022521139999999999, 2024-09-14T22:38:15.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00227378, 2024-09-14T22:38:15.904 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:15.904 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:15.904 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "waiting for sub ops", 2024-09-14T22:38:15.904 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:15.904 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:15.904 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:15.904 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 156047 2024-09-14T22:38:15.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.905 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:15.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:15.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:15.901398+0000", 2024-09-14T22:38:15.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:15.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:15.901398+0000", 2024-09-14T22:38:15.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:15.906 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:15.901400+0000", 2024-09-14T22:38:15.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8500000000000001e-06 2024-09-14T22:38:15.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.906 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:15.906 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:15.901410+0000", 2024-09-14T22:38:15.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0403e-05 2024-09-14T22:38:15.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.906 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:15.906 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:15.901437+0000", 2024-09-14T22:38:15.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6152e-05 2024-09-14T22:38:15.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:15.901450+0000", 2024-09-14T22:38:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.382e-05 2024-09-14T22:38:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:15.901572+0000", 2024-09-14T22:38:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121363 2024-09-14T22:38:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:15.901825+0000", 2024-09-14T22:38:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025320900000000001 2024-09-14T22:38:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 1", 2024-09-14T22:38:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:15.902206+0000", 2024-09-14T22:38:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00038102900000000001 2024-09-14T22:38:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:38:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:15.903526+0000", 2024-09-14T22:38:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0013195329999999999 2024-09-14T22:38:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.909 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop_reply(client.4412.0:156047 3.6 e122/23 ondisk, result = 0)", 2024-09-14T22:38:15.909 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:38:15.903568+0000", 2024-09-14T22:38:15.909 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.2608999999999995e-05, 2024-09-14T22:38:15.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015597000000000001, 2024-09-14T22:38:15.909 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:15.909 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:15.909 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:15.909 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:15.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:15.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:15.903568+0000", 2024-09-14T22:38:15.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:15.903568+0000", 2024-09-14T22:38:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:15.903569+0000", 2024-09-14T22:38:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3769999999999999e-06 2024-09-14T22:38:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:15.903577+0000", 2024-09-14T22:38:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.276e-06 2024-09-14T22:38:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:15.903601+0000", 2024-09-14T22:38:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3730000000000001e-05 2024-09-14T22:38:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:15.903616+0000", 2024-09-14T22:38:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4491999999999999e-05 2024-09-14T22:38:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:15.903672+0000", 2024-09-14T22:38:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6026999999999997e-05 2024-09-14T22:38:15.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:15.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:15.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:15.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:15.903679+0000", 2024-09-14T22:38:15.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0870000000000001e-06 2024-09-14T22:38:15.912 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.912 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:15.912 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.912 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:15.912 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:38:15.912 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 2 2024-09-14T22:38:15.912 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:15.918 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:38:15.939 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:15.940 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:15.934+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:15.965 INFO:teuthology.orchestra.run.smithi001.stdout:5905 writesame 0xb20155f thru 0xb2052e0 data_size 0x1(0x3d82 bytes) 2024-09-14T22:38:15.966 INFO:teuthology.orchestra.run.smithi001.stdout:5906 writesame 0x35a6e82 thru 0x35b5ea3 data_size 0x7(0xf022 bytes) 2024-09-14T22:38:15.979 INFO:teuthology.orchestra.run.smithi001.stdout:5907 punch from 0x62788ed to 0x627c3c7, (0x3ada bytes) 2024-09-14T22:38:15.995 INFO:teuthology.orchestra.run.smithi001.stdout:5908 writesame 0xc3ca074 thru 0xc3d8a3a data_size 0x1(0xe9c7 bytes) 2024-09-14T22:38:15.995 INFO:teuthology.orchestra.run.smithi001.stdout:5910 write 0x5e0b402 thru 0x5e0e53f (0x313e bytes) 2024-09-14T22:38:15.997 INFO:teuthology.orchestra.run.smithi001.stdout:5911 write 0x1aa5811 thru 0x1aafac0 (0xa2b0 bytes) 2024-09-14T22:38:16.000 INFO:teuthology.orchestra.run.smithi001.stdout:5913 punch from 0x15cb124 to 0x15cf6b1, (0x458d bytes) 2024-09-14T22:38:16.000 INFO:teuthology.orchestra.run.smithi001.stdout:5914 read 0x2746c4a thru 0x275081b (0x9bd2 bytes) 2024-09-14T22:38:16.001 INFO:teuthology.orchestra.run.smithi001.stdout:5915 write 0x71401c1 thru 0x71424bb (0x22fb bytes) 2024-09-14T22:38:16.005 INFO:teuthology.orchestra.run.smithi001.stdout:5916 read 0x3b7603a thru 0x3b7c02e (0x5ff5 bytes) 2024-09-14T22:38:16.006 INFO:teuthology.orchestra.run.smithi001.stdout:5917 read 0x68968e thru 0x691f4f (0x88c2 bytes) 2024-09-14T22:38:16.007 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:16.007 INFO:teuthology.orchestra.run.smithi001.stdout:5920 write 0x4405add thru 0x44085cc (0x2af0 bytes) 2024-09-14T22:38:16.009 INFO:teuthology.orchestra.run.smithi001.stdout:5921 write 0x1cbb937 thru 0x1cc598e (0xa058 bytes) 2024-09-14T22:38:16.015 INFO:teuthology.orchestra.run.smithi001.stdout:5922 clone 42 order 20 su 524288 sc 5 2024-09-14T22:38:16.023 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:16.018+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:16.073 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:16.074 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:16.074 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:38:16.074 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:38:16.074 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:16.088 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:38:16.130 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:16.126+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:16.225 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:16.218+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:16.260 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:16.261 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:38:16.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:38:16.261 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:38:16.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.261 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:122960 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 2347008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:16.261 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:41.998325+0000", 2024-09-14T22:38:16.261 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.258784890999998, 2024-09-14T22:38:16.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056570400999999999, 2024-09-14T22:38:16.261 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.261 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.261 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.261 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.262 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.262 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.262 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 122960 2024-09-14T22:38:16.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.262 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998325+0000", 2024-09-14T22:38:16.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998325+0000", 2024-09-14T22:38:16.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998326+0000", 2024-09-14T22:38:16.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9299999999999998e-07 2024-09-14T22:38:16.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998329+0000", 2024-09-14T22:38:16.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1370000000000002e-06 2024-09-14T22:38:16.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998340+0000", 2024-09-14T22:38:16.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1039999999999999e-05 2024-09-14T22:38:16.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998348+0000", 2024-09-14T22:38:16.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1920000000000005e-06 2024-09-14T22:38:16.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054700+0000", 2024-09-14T22:38:16.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056351714999999997 2024-09-14T22:38:16.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.265 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054771+0000", 2024-09-14T22:38:16.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1009999999999999e-05 2024-09-14T22:38:16.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.265 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054896+0000", 2024-09-14T22:38:16.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000124714 2024-09-14T22:38:16.265 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.265 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.265 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.266 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:122961 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 2351104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:16.266 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:41.998357+0000", 2024-09-14T22:38:16.266 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.258752936, 2024-09-14T22:38:16.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056974912000000003, 2024-09-14T22:38:16.266 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.266 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.266 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.266 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.266 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.266 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.266 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 122961 2024-09-14T22:38:16.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.266 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998357+0000", 2024-09-14T22:38:16.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998357+0000", 2024-09-14T22:38:16.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998358+0000", 2024-09-14T22:38:16.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9699999999999996e-07 2024-09-14T22:38:16.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998361+0000", 2024-09-14T22:38:16.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1370000000000002e-06 2024-09-14T22:38:16.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998371+0000", 2024-09-14T22:38:16.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5789999999999993e-06 2024-09-14T22:38:16.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998379+0000", 2024-09-14T22:38:16.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0129999999999993e-06 2024-09-14T22:38:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054998+0000", 2024-09-14T22:38:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056618961000000002 2024-09-14T22:38:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.055126+0000", 2024-09-14T22:38:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012795100000000001 2024-09-14T22:38:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.055332+0000", 2024-09-14T22:38:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020667400000000001 2024-09-14T22:38:16.270 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.270 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.270 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.270 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123259 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 741376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:16.270 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:38:16.270 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.245027419000003, 2024-09-14T22:38:16.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056672581999999999, 2024-09-14T22:38:16.270 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.270 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.270 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.270 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.270 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123259 2024-09-14T22:38:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:38:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:38:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:38:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1600000000000001e-07 2024-09-14T22:38:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012086+0000", 2024-09-14T22:38:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8499999999999998e-06 2024-09-14T22:38:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012097+0000", 2024-09-14T22:38:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0943e-05 2024-09-14T22:38:16.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.273 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012105+0000", 2024-09-14T22:38:16.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0830000000000004e-06 2024-09-14T22:38:16.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.273 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068584+0000", 2024-09-14T22:38:16.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056478387999999997 2024-09-14T22:38:16.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.273 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068644+0000", 2024-09-14T22:38:16.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9984e-05 2024-09-14T22:38:16.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068755+0000", 2024-09-14T22:38:16.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111718 2024-09-14T22:38:16.274 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.274 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.274 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.274 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123260 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 778240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:16.274 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012118+0000", 2024-09-14T22:38:16.275 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.244992193999998, 2024-09-14T22:38:16.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056825887999999998, 2024-09-14T22:38:16.275 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.275 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.275 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.275 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.275 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.275 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.275 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123260 2024-09-14T22:38:16.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.275 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012118+0000", 2024-09-14T22:38:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012118+0000", 2024-09-14T22:38:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012119+0000", 2024-09-14T22:38:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.13e-07 2024-09-14T22:38:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.277 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012130+0000", 2024-09-14T22:38:16.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1552999999999999e-05 2024-09-14T22:38:16.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.277 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.277 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012141+0000", 2024-09-14T22:38:16.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.03e-05 2024-09-14T22:38:16.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.277 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.277 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012149+0000", 2024-09-14T22:38:16.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1419999999999993e-06 2024-09-14T22:38:16.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.278 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.278 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068800+0000", 2024-09-14T22:38:16.278 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056651574000000003 2024-09-14T22:38:16.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.278 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.278 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068858+0000", 2024-09-14T22:38:16.278 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8047000000000002e-05 2024-09-14T22:38:16.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.278 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.278 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068944+0000", 2024-09-14T22:38:16.278 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5659000000000001e-05 2024-09-14T22:38:16.278 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.279 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.279 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.279 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123261 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 761856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:16.279 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012161+0000", 2024-09-14T22:38:16.279 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.244948792999999, 2024-09-14T22:38:16.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057113753000000003, 2024-09-14T22:38:16.279 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.279 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.279 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.279 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.279 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.280 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.280 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123261 2024-09-14T22:38:16.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.280 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012161+0000", 2024-09-14T22:38:16.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012161+0000", 2024-09-14T22:38:16.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012162+0000", 2024-09-14T22:38:16.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4399999999999999e-07 2024-09-14T22:38:16.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012177+0000", 2024-09-14T22:38:16.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5099000000000001e-05 2024-09-14T22:38:16.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012191+0000", 2024-09-14T22:38:16.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3631999999999999e-05 2024-09-14T22:38:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012199+0000", 2024-09-14T22:38:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5769999999999994e-06 2024-09-14T22:38:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069004+0000", 2024-09-14T22:38:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056804504999999998 2024-09-14T22:38:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069137+0000", 2024-09-14T22:38:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013302600000000001 2024-09-14T22:38:16.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.283 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.283 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069275+0000", 2024-09-14T22:38:16.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013816999999999999 2024-09-14T22:38:16.283 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.283 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.283 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.283 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123262 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 724992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:16.283 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012224+0000", 2024-09-14T22:38:16.283 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.244886346000001, 2024-09-14T22:38:16.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057417959999999997, 2024-09-14T22:38:16.284 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.284 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.284 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.284 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.284 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.284 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.284 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123262 2024-09-14T22:38:16.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.284 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012224+0000", 2024-09-14T22:38:16.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012224+0000", 2024-09-14T22:38:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012225+0000", 2024-09-14T22:38:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4000000000000001e-07 2024-09-14T22:38:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012228+0000", 2024-09-14T22:38:16.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5729999999999999e-06 2024-09-14T22:38:16.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012241+0000", 2024-09-14T22:38:16.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3210000000000001e-05 2024-09-14T22:38:16.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012255+0000", 2024-09-14T22:38:16.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3852000000000001e-05 2024-09-14T22:38:16.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069341+0000", 2024-09-14T22:38:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057086139000000001 2024-09-14T22:38:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069477+0000", 2024-09-14T22:38:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013595 2024-09-14T22:38:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069642+0000", 2024-09-14T22:38:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016459600000000001 2024-09-14T22:38:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123263 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 782336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012265+0000", 2024-09-14T22:38:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.244844804000003, 2024-09-14T22:38:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057665095, 2024-09-14T22:38:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123263 2024-09-14T22:38:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012265+0000", 2024-09-14T22:38:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012265+0000", 2024-09-14T22:38:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012266+0000", 2024-09-14T22:38:16.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8999999999999996e-07 2024-09-14T22:38:16.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012269+0000", 2024-09-14T22:38:16.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.796e-06 2024-09-14T22:38:16.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012279+0000", 2024-09-14T22:38:16.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7389999999999994e-06 2024-09-14T22:38:16.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012286+0000", 2024-09-14T22:38:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8199999999999997e-06 2024-09-14T22:38:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069705+0000", 2024-09-14T22:38:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057418590999999998 2024-09-14T22:38:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069799+0000", 2024-09-14T22:38:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4419999999999994e-05 2024-09-14T22:38:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.292 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.292 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069931+0000", 2024-09-14T22:38:16.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013113900000000001 2024-09-14T22:38:16.292 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.292 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.292 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.292 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123264 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 745472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:16.292 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012295+0000", 2024-09-14T22:38:16.292 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.244814802999997, 2024-09-14T22:38:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057959498999999998, 2024-09-14T22:38:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123264 2024-09-14T22:38:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012295+0000", 2024-09-14T22:38:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012295+0000", 2024-09-14T22:38:16.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012296+0000", 2024-09-14T22:38:16.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9699999999999996e-07 2024-09-14T22:38:16.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.295 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012299+0000", 2024-09-14T22:38:16.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.79e-06 2024-09-14T22:38:16.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.295 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012320+0000", 2024-09-14T22:38:16.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1242000000000001e-05 2024-09-14T22:38:16.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012338+0000", 2024-09-14T22:38:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7768999999999999e-05 2024-09-14T22:38:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069995+0000", 2024-09-14T22:38:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057657 2024-09-14T22:38:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070132+0000", 2024-09-14T22:38:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013727800000000001 2024-09-14T22:38:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.297 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070255+0000", 2024-09-14T22:38:16.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012282300000000001 2024-09-14T22:38:16.297 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.297 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.297 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.297 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123265 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1056768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:16.297 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012348+0000", 2024-09-14T22:38:16.297 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.244761848000003, 2024-09-14T22:38:16.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058145087999999998, 2024-09-14T22:38:16.297 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.297 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.298 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.298 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.298 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.298 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.298 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123265 2024-09-14T22:38:16.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.298 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012348+0000", 2024-09-14T22:38:16.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012348+0000", 2024-09-14T22:38:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012349+0000", 2024-09-14T22:38:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6599999999999996e-07 2024-09-14T22:38:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012352+0000", 2024-09-14T22:38:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3139999999999999e-06 2024-09-14T22:38:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012365+0000", 2024-09-14T22:38:16.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2625e-05 2024-09-14T22:38:16.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012373+0000", 2024-09-14T22:38:16.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2470000000000008e-06 2024-09-14T22:38:16.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070286+0000", 2024-09-14T22:38:16.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057912547000000002 2024-09-14T22:38:16.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070366+0000", 2024-09-14T22:38:16.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0468000000000003e-05 2024-09-14T22:38:16.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070493+0000", 2024-09-14T22:38:16.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012722100000000001 2024-09-14T22:38:16.301 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.302 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.302 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.302 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123266 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 765952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:16.302 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:38:16.302 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.244727599999997, 2024-09-14T22:38:16.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058386949, 2024-09-14T22:38:16.302 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.302 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.302 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.302 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.302 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.302 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.302 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123266 2024-09-14T22:38:16.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.303 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:38:16.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:38:16.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:38:16.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9299999999999998e-07 2024-09-14T22:38:16.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012386+0000", 2024-09-14T22:38:16.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8729999999999999e-06 2024-09-14T22:38:16.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012406+0000", 2024-09-14T22:38:16.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0103e-05 2024-09-14T22:38:16.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012419+0000", 2024-09-14T22:38:16.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.311e-05 2024-09-14T22:38:16.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070550+0000", 2024-09-14T22:38:16.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058130902999999998 2024-09-14T22:38:16.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070650+0000", 2024-09-14T22:38:16.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010005000000000001 2024-09-14T22:38:16.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.306 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070770+0000", 2024-09-14T22:38:16.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119317 2024-09-14T22:38:16.306 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.306 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.306 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.306 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123267 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1077248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:16.306 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012429+0000", 2024-09-14T22:38:16.306 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.244681686, 2024-09-14T22:38:16.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058620343999999998, 2024-09-14T22:38:16.307 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.307 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.307 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.307 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.307 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.307 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.307 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123267 2024-09-14T22:38:16.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.307 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012429+0000", 2024-09-14T22:38:16.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012429+0000", 2024-09-14T22:38:16.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013488+0000", 2024-09-14T22:38:16.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001059061 2024-09-14T22:38:16.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013499+0000", 2024-09-14T22:38:16.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1348999999999999e-05 2024-09-14T22:38:16.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013534+0000", 2024-09-14T22:38:16.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5206000000000001e-05 2024-09-14T22:38:16.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013558+0000", 2024-09-14T22:38:16.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4324999999999998e-05 2024-09-14T22:38:16.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070834+0000", 2024-09-14T22:38:16.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057275110999999997 2024-09-14T22:38:16.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070937+0000", 2024-09-14T22:38:16.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010306099999999999 2024-09-14T22:38:16.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071049+0000", 2024-09-14T22:38:16.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112231 2024-09-14T22:38:16.310 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.310 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.310 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.311 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123268 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1097728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:16.311 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013593+0000", 2024-09-14T22:38:16.311 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.243516962999998, 2024-09-14T22:38:16.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057751793000000003, 2024-09-14T22:38:16.311 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.311 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.311 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.311 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.311 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.311 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.311 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123268 2024-09-14T22:38:16.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.312 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.312 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.312 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013593+0000", 2024-09-14T22:38:16.312 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.312 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.312 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013593+0000", 2024-09-14T22:38:16.312 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.312 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013595+0000", 2024-09-14T22:38:16.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.59e-06 2024-09-14T22:38:16.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013602+0000", 2024-09-14T22:38:16.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3560000000000004e-06 2024-09-14T22:38:16.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013625+0000", 2024-09-14T22:38:16.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2419e-05 2024-09-14T22:38:16.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013650+0000", 2024-09-14T22:38:16.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5151999999999999e-05 2024-09-14T22:38:16.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071096+0000", 2024-09-14T22:38:16.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057445877999999999 2024-09-14T22:38:16.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071212+0000", 2024-09-14T22:38:16.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116215 2024-09-14T22:38:16.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.315 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071345+0000", 2024-09-14T22:38:16.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013318299999999999 2024-09-14T22:38:16.315 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.315 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.315 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.316 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123269 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1060864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:16.316 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013676+0000", 2024-09-14T22:38:16.316 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.243434204000003, 2024-09-14T22:38:16.316 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058027982999999998, 2024-09-14T22:38:16.316 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.316 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.316 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.316 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.316 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.316 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.316 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123269 2024-09-14T22:38:16.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.316 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013676+0000", 2024-09-14T22:38:16.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013676+0000", 2024-09-14T22:38:16.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013677+0000", 2024-09-14T22:38:16.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.387e-06 2024-09-14T22:38:16.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013684+0000", 2024-09-14T22:38:16.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0530000000000001e-06 2024-09-14T22:38:16.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013706+0000", 2024-09-14T22:38:16.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1121999999999999e-05 2024-09-14T22:38:16.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013722+0000", 2024-09-14T22:38:16.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6249999999999999e-05 2024-09-14T22:38:16.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071402+0000", 2024-09-14T22:38:16.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057679945000000003 2024-09-14T22:38:16.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071547+0000", 2024-09-14T22:38:16.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000145632 2024-09-14T22:38:16.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071704+0000", 2024-09-14T22:38:16.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000156594 2024-09-14T22:38:16.320 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.320 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.320 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.320 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123270 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1064960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:16.320 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013744+0000", 2024-09-14T22:38:16.320 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.243366252999998, 2024-09-14T22:38:16.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058355872000000003, 2024-09-14T22:38:16.320 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.320 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.320 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.320 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.320 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.320 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.321 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123270 2024-09-14T22:38:16.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.321 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.321 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.321 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013744+0000", 2024-09-14T22:38:16.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.321 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.321 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013744+0000", 2024-09-14T22:38:16.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.321 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013745+0000", 2024-09-14T22:38:16.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2589999999999999e-06 2024-09-14T22:38:16.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013751+0000", 2024-09-14T22:38:16.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1929999999999998e-06 2024-09-14T22:38:16.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013778+0000", 2024-09-14T22:38:16.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6472e-05 2024-09-14T22:38:16.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013795+0000", 2024-09-14T22:38:16.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6626e-05 2024-09-14T22:38:16.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071775+0000", 2024-09-14T22:38:16.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057980232 2024-09-14T22:38:16.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071911+0000", 2024-09-14T22:38:16.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000136129 2024-09-14T22:38:16.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072100+0000", 2024-09-14T22:38:16.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000188961 2024-09-14T22:38:16.324 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.324 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.324 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.324 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123271 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1081344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:16.324 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013818+0000", 2024-09-14T22:38:16.324 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.243292056999998, 2024-09-14T22:38:16.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058549565999999997, 2024-09-14T22:38:16.325 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.325 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.325 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.325 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.325 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.325 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.325 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123271 2024-09-14T22:38:16.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.325 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.325 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013818+0000", 2024-09-14T22:38:16.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013818+0000", 2024-09-14T22:38:16.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013958+0000", 2024-09-14T22:38:16.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013934 2024-09-14T22:38:16.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013968+0000", 2024-09-14T22:38:16.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0329000000000001e-05 2024-09-14T22:38:16.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013995+0000", 2024-09-14T22:38:16.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7613000000000001e-05 2024-09-14T22:38:16.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014014+0000", 2024-09-14T22:38:16.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8338999999999998e-05 2024-09-14T22:38:16.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072155+0000", 2024-09-14T22:38:16.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058140989999999997 2024-09-14T22:38:16.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072247+0000", 2024-09-14T22:38:16.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1774999999999995e-05 2024-09-14T22:38:16.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072368+0000", 2024-09-14T22:38:16.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012118 2024-09-14T22:38:16.329 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.329 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.329 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.329 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123272 3.1 3.4b235a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:16.329 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.014046+0000", 2024-09-14T22:38:16.329 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.243064381000003, 2024-09-14T22:38:16.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058364077, 2024-09-14T22:38:16.330 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.330 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.330 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-14T22:38:16.330 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.330 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.330 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.330 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123272 2024-09-14T22:38:16.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.330 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014046+0000", 2024-09-14T22:38:16.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014046+0000", 2024-09-14T22:38:16.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014047+0000", 2024-09-14T22:38:16.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.533e-06 2024-09-14T22:38:16.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014085+0000", 2024-09-14T22:38:16.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7734999999999997e-05 2024-09-14T22:38:16.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014111+0000", 2024-09-14T22:38:16.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6055999999999999e-05 2024-09-14T22:38:16.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014136+0000", 2024-09-14T22:38:16.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4592000000000001e-05 2024-09-14T22:38:16.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072402+0000", 2024-09-14T22:38:16.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058266251999999998 2024-09-14T22:38:16.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072410+0000", 2024-09-14T22:38:16.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.909e-06 2024-09-14T22:38:16.334 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.334 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.334 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.334 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123273 3.1 3.4b235a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:16.334 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.014161+0000", 2024-09-14T22:38:16.334 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.242949353, 2024-09-14T22:38:16.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058284415999999999, 2024-09-14T22:38:16.334 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.334 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.334 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-14T22:38:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123273 2024-09-14T22:38:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014161+0000", 2024-09-14T22:38:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014161+0000", 2024-09-14T22:38:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.017996+0000", 2024-09-14T22:38:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0038350670000000002 2024-09-14T22:38:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.018009+0000", 2024-09-14T22:38:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3305999999999999e-05 2024-09-14T22:38:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.018044+0000", 2024-09-14T22:38:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4792000000000002e-05 2024-09-14T22:38:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.018067+0000", 2024-09-14T22:38:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2595e-05 2024-09-14T22:38:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072439+0000", 2024-09-14T22:38:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054372192 2024-09-14T22:38:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072445+0000", 2024-09-14T22:38:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4640000000000003e-06 2024-09-14T22:38:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:148469 3.18 3:1f6f7244:::rbd_header.113c827b523f:head [watch reconnect cookie 140250011614800 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-14T22:38:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:55.746449+0000", 2024-09-14T22:38:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 20.510660740999999, 2024-09-14T22:38:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.996484173, 2024-09-14T22:38:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 148469 2024-09-14T22:38:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746449+0000", 2024-09-14T22:38:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746449+0000", 2024-09-14T22:38:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746452+0000", 2024-09-14T22:38:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0019999999999998e-06 2024-09-14T22:38:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746461+0000", 2024-09-14T22:38:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0919999999999998e-06 2024-09-14T22:38:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746475+0000", 2024-09-14T22:38:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4853e-05 2024-09-14T22:38:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746484+0000", 2024-09-14T22:38:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3780000000000001e-06 2024-09-14T22:38:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746559+0000", 2024-09-14T22:38:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5259000000000006e-05 2024-09-14T22:38:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746572+0000", 2024-09-14T22:38:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.271e-05 2024-09-14T22:38:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727807+0000", 2024-09-14T22:38:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98123555900000003 2024-09-14T22:38:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727826+0000", 2024-09-14T22:38:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8828e-05 2024-09-14T22:38:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.739033+0000", 2024-09-14T22:38:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.011207062 2024-09-14T22:38:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.739479+0000", 2024-09-14T22:38:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044563200000000001 2024-09-14T22:38:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.742934+0000", 2024-09-14T22:38:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0034547979999999998 2024-09-14T22:38:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:148473 3.18 3:1f6f7244:::rbd_header.113c827b523f:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-14T22:38:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:55.755969+0000", 2024-09-14T22:38:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 20.501140974999998, 2024-09-14T22:38:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9872532889999999, 2024-09-14T22:38:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 148473 2024-09-14T22:38:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755969+0000", 2024-09-14T22:38:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755969+0000", 2024-09-14T22:38:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755971+0000", 2024-09-14T22:38:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.674e-06 2024-09-14T22:38:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755979+0000", 2024-09-14T22:38:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8599999999999993e-06 2024-09-14T22:38:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755992+0000", 2024-09-14T22:38:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.361e-05 2024-09-14T22:38:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.756001+0000", 2024-09-14T22:38:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.422e-06 2024-09-14T22:38:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.756069+0000", 2024-09-14T22:38:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8209999999999999e-05 2024-09-14T22:38:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.756077+0000", 2024-09-14T22:38:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.994e-06 2024-09-14T22:38:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727872+0000", 2024-09-14T22:38:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.97179484999999999 2024-09-14T22:38:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727885+0000", 2024-09-14T22:38:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3101e-05 2024-09-14T22:38:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.740559+0000", 2024-09-14T22:38:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0126736300000001 2024-09-14T22:38:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.740777+0000", 2024-09-14T22:38:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021822000000000001 2024-09-14T22:38:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 2", 2024-09-14T22:38:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.741114+0000", 2024-09-14T22:38:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000336793 2024-09-14T22:38:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.742668+0000", 2024-09-14T22:38:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001554061 2024-09-14T22:38:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:38:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.742947+0000", 2024-09-14T22:38:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027937200000000001 2024-09-14T22:38:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:38:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743030+0000", 2024-09-14T22:38:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3385000000000002e-05 2024-09-14T22:38:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743223+0000", 2024-09-14T22:38:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00019210700000000001 2024-09-14T22:38:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:148474 3.18 3:1f6f7244:::rbd_header.113c827b523f:head [watch reconnect cookie 140250011614800 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104)", 2024-09-14T22:38:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:56.728273+0000", 2024-09-14T22:38:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.528837727999999, 2024-09-14T22:38:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0180497770000001, 2024-09-14T22:38:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 148474 2024-09-14T22:38:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728273+0000", 2024-09-14T22:38:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728273+0000", 2024-09-14T22:38:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728274+0000", 2024-09-14T22:38:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7570000000000001e-06 2024-09-14T22:38:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728281+0000", 2024-09-14T22:38:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5089999999999999e-06 2024-09-14T22:38:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728298+0000", 2024-09-14T22:38:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6727000000000001e-05 2024-09-14T22:38:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728309+0000", 2024-09-14T22:38:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1021e-05 2024-09-14T22:38:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.730078+0000", 2024-09-14T22:38:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017694010000000001 2024-09-14T22:38:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.730096+0000", 2024-09-14T22:38:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8039000000000001e-05 2024-09-14T22:38:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.741672+0000", 2024-09-14T22:38:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.011575817 2024-09-14T22:38:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2024-09-14T22:38:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.741827+0000", 2024-09-14T22:38:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000155563 2024-09-14T22:38:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743299+0000", 2024-09-14T22:38:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0014712729999999999 2024-09-14T22:38:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743415+0000", 2024-09-14T22:38:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011594700000000001 2024-09-14T22:38:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.746322+0000", 2024-09-14T22:38:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0029077230000000001 2024-09-14T22:38:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.358 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.358 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.358 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.358 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.358 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:16.359 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:38:16.360 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:16.354+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:16.460 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:16.458+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:16.491 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:16.491 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:16.491 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:38:16.491 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:16.505 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:38:16.562 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:16.558+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:16.630 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:16.630 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:16.631 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:38:16.631 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:38:16.631 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:16.645 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:38:16.686 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:16.681+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:16.764 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:16.762+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:16.818 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:16.818 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:38:16.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:38:16.818 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:38:16.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.818 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:31370 3.0 3:02f48f58:::rbd_data.113cda0183a0.0000000000000000:6 [read 813056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:38:16.818 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:38:16.818 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 103.452171908, 2024-09-14T22:38:16.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047519796000000003, 2024-09-14T22:38:16.818 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 31370 2024-09-14T22:38:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:38:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:38:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:38:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0999999999999999e-07 2024-09-14T22:38:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362401+0000", 2024-09-14T22:38:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2900000000000001e-06 2024-09-14T22:38:16.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362406+0000", 2024-09-14T22:38:16.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6960000000000002e-06 2024-09-14T22:38:16.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362412+0000", 2024-09-14T22:38:16.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.823e-06 2024-09-14T22:38:16.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.400270+0000", 2024-09-14T22:38:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037857737000000002 2024-09-14T22:38:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.400342+0000", 2024-09-14T22:38:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1797000000000004e-05 2024-09-14T22:38:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.409918+0000", 2024-09-14T22:38:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0095760429999999994 2024-09-14T22:38:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:113947 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 5603328~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.923453+0000", 2024-09-14T22:38:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.891116775999997, 2024-09-14T22:38:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047310420999999998, 2024-09-14T22:38:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 113947 2024-09-14T22:38:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.923453+0000", 2024-09-14T22:38:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.923453+0000", 2024-09-14T22:38:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926004+0000", 2024-09-14T22:38:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002551382 2024-09-14T22:38:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926009+0000", 2024-09-14T22:38:16.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5639999999999998e-06 2024-09-14T22:38:16.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926022+0000", 2024-09-14T22:38:16.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2734999999999999e-05 2024-09-14T22:38:16.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926029+0000", 2024-09-14T22:38:16.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2520000000000002e-06 2024-09-14T22:38:16.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.970650+0000", 2024-09-14T22:38:16.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044621277000000001 2024-09-14T22:38:16.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.970700+0000", 2024-09-14T22:38:16.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9713000000000003e-05 2024-09-14T22:38:16.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.970763+0000", 2024-09-14T22:38:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3497999999999994e-05 2024-09-14T22:38:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114136 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3448832~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932179+0000", 2024-09-14T22:38:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.882390409999999, 2024-09-14T22:38:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047236017999999998, 2024-09-14T22:38:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114136 2024-09-14T22:38:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932179+0000", 2024-09-14T22:38:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932179+0000", 2024-09-14T22:38:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932324+0000", 2024-09-14T22:38:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014451000000000001 2024-09-14T22:38:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932328+0000", 2024-09-14T22:38:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5889999999999996e-06 2024-09-14T22:38:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932338+0000", 2024-09-14T22:38:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3000000000000007e-06 2024-09-14T22:38:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932344+0000", 2024-09-14T22:38:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7150000000000001e-06 2024-09-14T22:38:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979233+0000", 2024-09-14T22:38:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046888711 2024-09-14T22:38:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979313+0000", 2024-09-14T22:38:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9487000000000007e-05 2024-09-14T22:38:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979415+0000", 2024-09-14T22:38:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000102706 2024-09-14T22:38:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114138 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3497984~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932356+0000", 2024-09-14T22:38:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.882213524000001, 2024-09-14T22:38:16.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047285958000000003, 2024-09-14T22:38:16.832 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.832 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.832 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.832 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.832 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.832 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.832 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114138 2024-09-14T22:38:16.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.832 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932356+0000", 2024-09-14T22:38:16.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932356+0000", 2024-09-14T22:38:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932498+0000", 2024-09-14T22:38:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000142182 2024-09-14T22:38:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932503+0000", 2024-09-14T22:38:16.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.301e-06 2024-09-14T22:38:16.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932514+0000", 2024-09-14T22:38:16.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1484000000000001e-05 2024-09-14T22:38:16.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932521+0000", 2024-09-14T22:38:16.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0740000000000004e-06 2024-09-14T22:38:16.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979451+0000", 2024-09-14T22:38:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046929379 2024-09-14T22:38:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979535+0000", 2024-09-14T22:38:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4423000000000001e-05 2024-09-14T22:38:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979642+0000", 2024-09-14T22:38:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010711499999999999 2024-09-14T22:38:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.836 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.836 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114139 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3547136~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:16.836 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932554+0000", 2024-09-14T22:38:16.836 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.882015756999998, 2024-09-14T22:38:16.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047292230999999997, 2024-09-14T22:38:16.836 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.836 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.836 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.836 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.836 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.836 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.836 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114139 2024-09-14T22:38:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932554+0000", 2024-09-14T22:38:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932554+0000", 2024-09-14T22:38:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932819+0000", 2024-09-14T22:38:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026538400000000002 2024-09-14T22:38:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932825+0000", 2024-09-14T22:38:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3469999999999998e-06 2024-09-14T22:38:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932836+0000", 2024-09-14T22:38:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1e-05 2024-09-14T22:38:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932843+0000", 2024-09-14T22:38:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1790000000000002e-06 2024-09-14T22:38:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979673+0000", 2024-09-14T22:38:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046830389 2024-09-14T22:38:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979750+0000", 2024-09-14T22:38:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6312999999999999e-05 2024-09-14T22:38:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.840 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979846+0000", 2024-09-14T22:38:16.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6619000000000004e-05 2024-09-14T22:38:16.840 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.840 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.840 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.840 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114141 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3596288~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:16.840 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932855+0000", 2024-09-14T22:38:16.840 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.881715159000002, 2024-09-14T22:38:16.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047201870999999999, 2024-09-14T22:38:16.840 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.840 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114141 2024-09-14T22:38:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932855+0000", 2024-09-14T22:38:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932855+0000", 2024-09-14T22:38:16.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934033+0000", 2024-09-14T22:38:16.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001178264 2024-09-14T22:38:16.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934038+0000", 2024-09-14T22:38:16.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4820000000000002e-06 2024-09-14T22:38:16.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934048+0000", 2024-09-14T22:38:16.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9890000000000004e-06 2024-09-14T22:38:16.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934055+0000", 2024-09-14T22:38:16.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8329999999999996e-06 2024-09-14T22:38:16.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979878+0000", 2024-09-14T22:38:16.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045822773999999997 2024-09-14T22:38:16.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979958+0000", 2024-09-14T22:38:16.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0477999999999998e-05 2024-09-14T22:38:16.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.980056+0000", 2024-09-14T22:38:16.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8051e-05 2024-09-14T22:38:16.844 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.844 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.844 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114159 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3940352~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.934179+0000", 2024-09-14T22:38:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.880391123000003, 2024-09-14T22:38:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047341439999999999, 2024-09-14T22:38:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114159 2024-09-14T22:38:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.846 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934179+0000", 2024-09-14T22:38:16.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934179+0000", 2024-09-14T22:38:16.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935021+0000", 2024-09-14T22:38:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00084276800000000003 2024-09-14T22:38:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935027+0000", 2024-09-14T22:38:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2959999999999998e-06 2024-09-14T22:38:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935038+0000", 2024-09-14T22:38:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1198e-05 2024-09-14T22:38:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935054+0000", 2024-09-14T22:38:16.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5764e-05 2024-09-14T22:38:16.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.981341+0000", 2024-09-14T22:38:16.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.04628782 2024-09-14T22:38:16.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.981415+0000", 2024-09-14T22:38:16.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3349999999999994e-05 2024-09-14T22:38:16.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.981520+0000", 2024-09-14T22:38:16.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105244 2024-09-14T22:38:16.849 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.849 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.849 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.849 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114180 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4136960~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:16.849 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:38:16.849 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.879391587000001, 2024-09-14T22:38:16.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047190621000000002, 2024-09-14T22:38:16.849 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.850 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.850 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.850 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.850 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.850 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.850 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114180 2024-09-14T22:38:16.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.850 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:38:16.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:38:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:38:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6800000000000002e-07 2024-09-14T22:38:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935180+0000", 2024-09-14T22:38:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.42e-06 2024-09-14T22:38:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935185+0000", 2024-09-14T22:38:16.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1359999999999996e-06 2024-09-14T22:38:16.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935188+0000", 2024-09-14T22:38:16.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.711e-06 2024-09-14T22:38:16.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982201+0000", 2024-09-14T22:38:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047013309000000003 2024-09-14T22:38:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982278+0000", 2024-09-14T22:38:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6705000000000005e-05 2024-09-14T22:38:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982369+0000", 2024-09-14T22:38:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1071999999999998e-05 2024-09-14T22:38:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.854 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114182 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4186112~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:16.854 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:38:16.854 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.879362964999999, 2024-09-14T22:38:16.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047368990999999999, 2024-09-14T22:38:16.854 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.854 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.854 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.854 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.854 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.854 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.854 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114182 2024-09-14T22:38:16.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:38:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:38:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:38:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7099999999999998e-07 2024-09-14T22:38:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935209+0000", 2024-09-14T22:38:16.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0260000000000002e-06 2024-09-14T22:38:16.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935213+0000", 2024-09-14T22:38:16.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3429999999999998e-06 2024-09-14T22:38:16.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935217+0000", 2024-09-14T22:38:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2480000000000001e-06 2024-09-14T22:38:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982406+0000", 2024-09-14T22:38:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047188920000000002 2024-09-14T22:38:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982483+0000", 2024-09-14T22:38:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7008999999999994e-05 2024-09-14T22:38:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982576+0000", 2024-09-14T22:38:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3173999999999999e-05 2024-09-14T22:38:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114185 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4202496~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.935221+0000", 2024-09-14T22:38:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.879348800000002, 2024-09-14T22:38:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047601128999999999, 2024-09-14T22:38:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114185 2024-09-14T22:38:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935221+0000", 2024-09-14T22:38:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935221+0000", 2024-09-14T22:38:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935970+0000", 2024-09-14T22:38:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074930899999999998 2024-09-14T22:38:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935976+0000", 2024-09-14T22:38:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5439999999999998e-06 2024-09-14T22:38:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935988+0000", 2024-09-14T22:38:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1779e-05 2024-09-14T22:38:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935996+0000", 2024-09-14T22:38:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.106e-06 2024-09-14T22:38:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982607+0000", 2024-09-14T22:38:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046611504999999998 2024-09-14T22:38:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982686+0000", 2024-09-14T22:38:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8805000000000002e-05 2024-09-14T22:38:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982822+0000", 2024-09-14T22:38:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000136081 2024-09-14T22:38:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114189 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4300800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.936048+0000", 2024-09-14T22:38:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.878521667999998, 2024-09-14T22:38:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047231539000000003, 2024-09-14T22:38:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114189 2024-09-14T22:38:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936048+0000", 2024-09-14T22:38:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936048+0000", 2024-09-14T22:38:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936049+0000", 2024-09-14T22:38:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4099999999999999e-07 2024-09-14T22:38:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936052+0000", 2024-09-14T22:38:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0029999999999999e-06 2024-09-14T22:38:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936058+0000", 2024-09-14T22:38:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7880000000000001e-06 2024-09-14T22:38:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936064+0000", 2024-09-14T22:38:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2279999999999998e-06 2024-09-14T22:38:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983120+0000", 2024-09-14T22:38:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047056384999999999 2024-09-14T22:38:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983194+0000", 2024-09-14T22:38:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3811000000000002e-05 2024-09-14T22:38:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983280+0000", 2024-09-14T22:38:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5883000000000004e-05 2024-09-14T22:38:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114191 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4349952~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:38:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.878497928000002, 2024-09-14T22:38:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047365365, 2024-09-14T22:38:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114191 2024-09-14T22:38:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:38:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:38:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:38:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0499999999999999e-07 2024-09-14T22:38:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936075+0000", 2024-09-14T22:38:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.734e-06 2024-09-14T22:38:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936082+0000", 2024-09-14T22:38:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8360000000000003e-06 2024-09-14T22:38:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936087+0000", 2024-09-14T22:38:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3129999999999998e-06 2024-09-14T22:38:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983319+0000", 2024-09-14T22:38:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047232140999999998 2024-09-14T22:38:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983365+0000", 2024-09-14T22:38:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5776999999999999e-05 2024-09-14T22:38:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983437+0000", 2024-09-14T22:38:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2158999999999998e-05 2024-09-14T22:38:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114193 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4399104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.936095+0000", 2024-09-14T22:38:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.878474992000001, 2024-09-14T22:38:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047505915000000003, 2024-09-14T22:38:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114193 2024-09-14T22:38:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936095+0000", 2024-09-14T22:38:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936095+0000", 2024-09-14T22:38:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938021+0000", 2024-09-14T22:38:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001926178 2024-09-14T22:38:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938027+0000", 2024-09-14T22:38:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3110000000000002e-06 2024-09-14T22:38:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938041+0000", 2024-09-14T22:38:16.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3344e-05 2024-09-14T22:38:16.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938049+0000", 2024-09-14T22:38:16.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2279999999999998e-06 2024-09-14T22:38:16.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983468+0000", 2024-09-14T22:38:16.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045419494999999997 2024-09-14T22:38:16.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983524+0000", 2024-09-14T22:38:16.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5952000000000002e-05 2024-09-14T22:38:16.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983601+0000", 2024-09-14T22:38:16.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6407000000000002e-05 2024-09-14T22:38:16.875 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.875 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.875 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.875 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114710 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4857856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:16.875 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.962571+0000", 2024-09-14T22:38:16.876 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.851998719999997, 2024-09-14T22:38:16.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047253314999999997, 2024-09-14T22:38:16.876 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.876 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.876 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.876 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.876 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.876 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.876 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114710 2024-09-14T22:38:16.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.876 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962571+0000", 2024-09-14T22:38:16.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962571+0000", 2024-09-14T22:38:16.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962572+0000", 2024-09-14T22:38:16.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1399999999999997e-07 2024-09-14T22:38:16.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962574+0000", 2024-09-14T22:38:16.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8679999999999999e-06 2024-09-14T22:38:16.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962583+0000", 2024-09-14T22:38:16.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8310000000000007e-06 2024-09-14T22:38:16.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962589+0000", 2024-09-14T22:38:16.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9529999999999996e-06 2024-09-14T22:38:16.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009692+0000", 2024-09-14T22:38:16.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047103249999999999 2024-09-14T22:38:16.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009767+0000", 2024-09-14T22:38:16.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4672000000000006e-05 2024-09-14T22:38:16.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009824+0000", 2024-09-14T22:38:16.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7226999999999999e-05 2024-09-14T22:38:16.880 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.880 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.880 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.880 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114712 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4907008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:16.880 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:38:16.880 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.85197324, 2024-09-14T22:38:16.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047421087000000001, 2024-09-14T22:38:16.880 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.880 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.880 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.880 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.881 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.881 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.881 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114712 2024-09-14T22:38:16.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.881 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:38:16.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:38:16.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:38:16.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.03e-07 2024-09-14T22:38:16.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962599+0000", 2024-09-14T22:38:16.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3199999999999998e-06 2024-09-14T22:38:16.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962606+0000", 2024-09-14T22:38:16.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5289999999999997e-06 2024-09-14T22:38:16.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962612+0000", 2024-09-14T22:38:16.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.947e-06 2024-09-14T22:38:16.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009866+0000", 2024-09-14T22:38:16.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047254681999999999 2024-09-14T22:38:16.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009924+0000", 2024-09-14T22:38:16.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7556000000000003e-05 2024-09-14T22:38:16.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010018+0000", 2024-09-14T22:38:16.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.365e-05 2024-09-14T22:38:16.884 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.884 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.884 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.884 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114714 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4956160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:16.885 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.962618+0000", 2024-09-14T22:38:16.885 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.851951722000003, 2024-09-14T22:38:16.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.04759302, 2024-09-14T22:38:16.885 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.885 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.885 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.885 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.885 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.885 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.885 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114714 2024-09-14T22:38:16.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.885 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962618+0000", 2024-09-14T22:38:16.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.886 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962618+0000", 2024-09-14T22:38:16.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.886 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966011+0000", 2024-09-14T22:38:16.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003393067 2024-09-14T22:38:16.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966016+0000", 2024-09-14T22:38:16.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3580000000000002e-06 2024-09-14T22:38:16.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966032+0000", 2024-09-14T22:38:16.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5356e-05 2024-09-14T22:38:16.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966040+0000", 2024-09-14T22:38:16.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1980000000000001e-06 2024-09-14T22:38:16.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010055+0000", 2024-09-14T22:38:16.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044014676000000003 2024-09-14T22:38:16.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010127+0000", 2024-09-14T22:38:16.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2206000000000006e-05 2024-09-14T22:38:16.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010211+0000", 2024-09-14T22:38:16.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4159000000000005e-05 2024-09-14T22:38:16.889 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.889 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.889 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.889 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114754 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 6283264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:16.889 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:38:16.889 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.847776809000003, 2024-09-14T22:38:16.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047228150000000003, 2024-09-14T22:38:16.889 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.889 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.889 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:16.889 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.890 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.890 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.890 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114754 2024-09-14T22:38:16.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.890 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:38:16.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:38:16.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:38:16.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7399999999999999e-07 2024-09-14T22:38:16.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966796+0000", 2024-09-14T22:38:16.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3630000000000001e-06 2024-09-14T22:38:16.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966802+0000", 2024-09-14T22:38:16.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4679999999999997e-06 2024-09-14T22:38:16.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966807+0000", 2024-09-14T22:38:16.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1109999999999999e-06 2024-09-14T22:38:16.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.013929+0000", 2024-09-14T22:38:16.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047121720999999998 2024-09-14T22:38:16.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.013968+0000", 2024-09-14T22:38:16.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8877000000000001e-05 2024-09-14T22:38:16.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014021+0000", 2024-09-14T22:38:16.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3236000000000003e-05 2024-09-14T22:38:16.893 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.893 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.893 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.893 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114789 3.19 3:9836bb25:::rbd_object_map.113c14c67864.000000000000001b:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:16.894 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.966813+0000", 2024-09-14T22:38:16.894 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.847756347000001, 2024-09-14T22:38:16.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05001311, 2024-09-14T22:38:16.894 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.894 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.894 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:16.894 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.894 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.894 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.894 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114789 2024-09-14T22:38:16.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.894 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966813+0000", 2024-09-14T22:38:16.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966813+0000", 2024-09-14T22:38:16.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966814+0000", 2024-09-14T22:38:16.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9200000000000002e-07 2024-09-14T22:38:16.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966817+0000", 2024-09-14T22:38:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8380000000000002e-06 2024-09-14T22:38:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966831+0000", 2024-09-14T22:38:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4209999999999999e-05 2024-09-14T22:38:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966845+0000", 2024-09-14T22:38:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3799e-05 2024-09-14T22:38:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014046+0000", 2024-09-14T22:38:16.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047201604000000001 2024-09-14T22:38:16.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014116+0000", 2024-09-14T22:38:16.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9387000000000005e-05 2024-09-14T22:38:16.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-14T22:38:16.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014469+0000", 2024-09-14T22:38:16.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00035355800000000002 2024-09-14T22:38:16.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:16.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016719+0000", 2024-09-14T22:38:16.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0022498459999999998 2024-09-14T22:38:16.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:38:16.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016735+0000", 2024-09-14T22:38:16.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5792999999999999e-05 2024-09-14T22:38:16.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:38:16.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016790+0000", 2024-09-14T22:38:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4715e-05 2024-09-14T22:38:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016827+0000", 2024-09-14T22:38:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6968000000000002e-05 2024-09-14T22:38:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114803 3.19 3:9836bb25:::rbd_object_map.113c14c67864.000000000000001b:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.966853+0000", 2024-09-14T22:38:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.847716917, 2024-09-14T22:38:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050656093999999999, 2024-09-14T22:38:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114803 2024-09-14T22:38:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966853+0000", 2024-09-14T22:38:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966853+0000", 2024-09-14T22:38:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969930+0000", 2024-09-14T22:38:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030769759999999999 2024-09-14T22:38:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969936+0000", 2024-09-14T22:38:16.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7150000000000003e-06 2024-09-14T22:38:16.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969949+0000", 2024-09-14T22:38:16.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3166000000000001e-05 2024-09-14T22:38:16.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969957+0000", 2024-09-14T22:38:16.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8269999999999998e-06 2024-09-14T22:38:16.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014996+0000", 2024-09-14T22:38:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045039202 2024-09-14T22:38:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.015085+0000", 2024-09-14T22:38:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8987999999999996e-05 2024-09-14T22:38:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-14T22:38:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.015318+0000", 2024-09-14T22:38:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023284999999999999 2024-09-14T22:38:16.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.904 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:16.904 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016898+0000", 2024-09-14T22:38:16.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0015807379999999999 2024-09-14T22:38:16.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.904 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:38:16.904 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.017455+0000", 2024-09-14T22:38:16.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00055664600000000005 2024-09-14T22:38:16.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.904 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:38:16.904 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.017483+0000", 2024-09-14T22:38:16.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8238e-05 2024-09-14T22:38:16.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.017509+0000", 2024-09-14T22:38:16.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5748000000000001e-05 2024-09-14T22:38:16.905 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.905 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.905 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.905 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:150759 3.8 3:15dc94e5:::rbd_header.113c88f88d4a:head [create,call rbd.create in=9b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e113)", 2024-09-14T22:38:16.905 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:38:05.726302+0000", 2024-09-14T22:38:16.905 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 11.088268225, 2024-09-14T22:38:16.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0241160819999999, 2024-09-14T22:38:16.906 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:16.906 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:16.906 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:16.906 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:16.906 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:16.906 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:16.906 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 150759 2024-09-14T22:38:16.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.906 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:16.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.906 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:16.906 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.726302+0000", 2024-09-14T22:38:16.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:16.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.726302+0000", 2024-09-14T22:38:16.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:16.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:16.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.726304+0000", 2024-09-14T22:38:16.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.075e-06 2024-09-14T22:38:16.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:16.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.726312+0000", 2024-09-14T22:38:16.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2050000000000002e-06 2024-09-14T22:38:16.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:16.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.727969+0000", 2024-09-14T22:38:16.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001657499 2024-09-14T22:38:16.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:16.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.727985+0000", 2024-09-14T22:38:16.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6082999999999998e-05 2024-09-14T22:38:16.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.728193+0000", 2024-09-14T22:38:16.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000208082 2024-09-14T22:38:16.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:16.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.728205+0000", 2024-09-14T22:38:16.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1897e-05 2024-09-14T22:38:16.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:16.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.741926+0000", 2024-09-14T22:38:16.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.013720591 2024-09-14T22:38:16.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:16.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.742076+0000", 2024-09-14T22:38:16.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015024099999999999 2024-09-14T22:38:16.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-14T22:38:16.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.742415+0000", 2024-09-14T22:38:16.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00033887700000000002 2024-09-14T22:38:16.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:38:16.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.743436+0000", 2024-09-14T22:38:16.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010210639999999999 2024-09-14T22:38:16.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:16.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.750216+0000", 2024-09-14T22:38:16.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0067798950000000002 2024-09-14T22:38:16.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:38:16.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.750288+0000", 2024-09-14T22:38:16.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2192000000000003e-05 2024-09-14T22:38:16.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:16.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:16.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:16.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.750418+0000", 2024-09-14T22:38:16.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000129381 2024-09-14T22:38:16.911 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.911 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.911 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.912 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:16.912 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:16.912 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:16.912 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:38:16.914 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:38:16.914 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:16.909+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:17.014 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:17.006+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:17.038 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T22:38:17.110 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:17.110 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:17.110 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:38:17.110 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:17.114 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:17.109+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:17.115 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:38:17.115 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:38:17.127 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:38:17.134 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:38:17.135 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:38:17.155 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:38:17.155 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:38:17.176 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:38:17.176 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:38:17.198 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:38:17.199 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:38:17.201 INFO:teuthology.orchestra.run.smithi001.stdout:leaving image image_client.0-clone41 intact 2024-09-14T22:38:17.215 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:17.210+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:17.220 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:38:17.220 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:38:17.242 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:38:17.242 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:38:17.264 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:38:17.264 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:38:17.314 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:17.315 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:17.316 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:17.309+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:17.332 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:38:17.417 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:17.410+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:17.467 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-14T22:38:17.475 INFO:teuthology.orchestra.run.smithi001.stdout:5923 trunc from 0xc3d8a3b to 0xb63005b 2024-09-14T22:38:17.498 INFO:teuthology.orchestra.run.smithi001.stdout:5925 write 0xa50b42 thru 0xa51622 (0xae1 bytes) 2024-09-14T22:38:17.503 INFO:teuthology.orchestra.run.smithi001.stdout:5926 punch from 0xb339393 to 0xb348e7b, (0xfae8 bytes) 2024-09-14T22:38:17.503 INFO:teuthology.orchestra.run.smithi001.stdout:5927 read 0xb2c3117 thru 0xb2cdd5d (0xac47 bytes) 2024-09-14T22:38:17.506 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:17.506 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:17.515 INFO:teuthology.orchestra.run.smithi001.stdout:5929 writesame 0xda53718 thru 0xda619fd data_size 0xe(0xe2e6 bytes) 2024-09-14T22:38:17.515 INFO:teuthology.orchestra.run.smithi001.stdout:5930 writesame 0x4e90222 thru 0x4e9eea8 data_size 0x1(0xec87 bytes) 2024-09-14T22:38:17.518 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:17.513+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:17.522 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:38:17.551 INFO:teuthology.orchestra.run.smithi001.stdout:5932 punch from 0x25f3aa2 to 0x25f8994, (0x4ef2 bytes) 2024-09-14T22:38:17.551 INFO:teuthology.orchestra.run.smithi001.stdout:5934 punch from 0x4bd503f to 0x4be09fb, (0xb9bc bytes) 2024-09-14T22:38:17.551 INFO:teuthology.orchestra.run.smithi001.stdout:5935 writesame 0x544dbbc thru 0x5457d41 data_size 0x19(0xa186 bytes) 2024-09-14T22:38:17.559 INFO:teuthology.orchestra.run.smithi001.stdout:5936 write 0x78dfdd5 thru 0x78e5f10 (0x613c bytes) 2024-09-14T22:38:17.564 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:17.565 INFO:teuthology.orchestra.run.smithi001.stdout:5939 write 0x113eeff thru 0x114c42f (0xd531 bytes) 2024-09-14T22:38:17.570 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:17.570 INFO:teuthology.orchestra.run.smithi001.stdout:5941 read 0x5475fb0 thru 0x5485625 (0xf676 bytes) 2024-09-14T22:38:17.571 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:17.571 INFO:teuthology.orchestra.run.smithi001.stdout:5944 writesame 0xc4940dc thru 0xc4990f7 data_size 0x6(0x501c bytes) 2024-09-14T22:38:17.579 INFO:teuthology.orchestra.run.smithi001.stdout:5945 read 0x873757e thru 0x8738354 (0xdd7 bytes) 2024-09-14T22:38:17.579 INFO:teuthology.orchestra.run.smithi001.stdout:5946 read 0x350e539 thru 0x350f861 (0x1329 bytes) 2024-09-14T22:38:17.579 INFO:teuthology.orchestra.run.smithi001.stdout:5947 write 0x2c89595 thru 0x2c94d2b (0xb797 bytes) 2024-09-14T22:38:17.586 INFO:teuthology.orchestra.run.smithi001.stdout:5948 write 0x6e3bf1b thru 0x6e478e5 (0xb9cb bytes) 2024-09-14T22:38:17.601 INFO:teuthology.orchestra.run.smithi001.stdout:5949 writesame 0xea75db1 thru 0xea81fe7 data_size 0x1(0xc237 bytes) 2024-09-14T22:38:17.618 INFO:teuthology.orchestra.run.smithi001.stdout:5950 trunc from 0xea81fe8 to 0x85f118 2024-09-14T22:38:17.619 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:17.613+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:17.653 INFO:teuthology.orchestra.run.smithi001.stdout:5951 read 0x317fdc thru 0x327a58 (0xfa7d bytes) 2024-09-14T22:38:17.658 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:17.658 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:17.671 INFO:teuthology.orchestra.run.smithi001.stdout:5952 write 0x5659789 thru 0x566100d (0x7885 bytes) 2024-09-14T22:38:17.673 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:38:17.677 INFO:teuthology.orchestra.run.smithi001.stdout:5954 writesame 0x256097b thru 0x2564965 data_size 0x1(0x3feb bytes) 2024-09-14T22:38:17.691 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:17.691 INFO:teuthology.orchestra.run.smithi001.stdout:5956 punch from 0x3bf0a2e to 0x3bf8f84, (0x8556 bytes) 2024-09-14T22:38:17.691 INFO:teuthology.orchestra.run.smithi001.stdout:5957 punch from 0x533b3c7 to 0x5344d35, (0x996e bytes) 2024-09-14T22:38:17.692 INFO:teuthology.orchestra.run.smithi001.stdout:5958 read 0x45b8da0 thru 0x45bc0f7 (0x3358 bytes) 2024-09-14T22:38:17.692 INFO:teuthology.orchestra.run.smithi001.stdout:5960 writesame 0x12df387 thru 0x12e29d1 data_size 0x29(0x364b bytes) 2024-09-14T22:38:17.695 INFO:teuthology.orchestra.run.smithi001.stdout:5961 read 0x1477053 thru 0x1480df1 (0x9d9f bytes) 2024-09-14T22:38:17.695 INFO:teuthology.orchestra.run.smithi001.stdout:5962 write 0x4a255f2 thru 0x4a3178e (0xc19d bytes) 2024-09-14T22:38:17.709 INFO:teuthology.orchestra.run.smithi001.stdout:5963 trunc from 0x566100e to 0x196b9e 2024-09-14T22:38:17.722 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:17.714+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:17.742 INFO:teuthology.orchestra.run.smithi001.stdout:5964 read 0x3975c thru 0x45fa5 (0xc84a bytes) 2024-09-14T22:38:17.787 INFO:teuthology.orchestra.run.smithi001.stdout:5966 write 0xd742dc7 thru 0xd749273 (0x64ad bytes) 2024-09-14T22:38:17.795 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-14T22:38:17.795 INFO:teuthology.orchestra.run.smithi001.stdout:5969 read 0x6995e97 thru 0x69a54aa (0xf614 bytes) 2024-09-14T22:38:17.801 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:17.801 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:17.815 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:38:17.815 INFO:teuthology.orchestra.run.smithi001.stdout:5970 trunc from 0xd749274 to 0x61f59c1 2024-09-14T22:38:17.821 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:17.813+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:17.839 INFO:teuthology.orchestra.run.smithi001.stdout:5971 read 0x2f5b764 thru 0x2f6a209 (0xeaa6 bytes) 2024-09-14T22:38:17.875 INFO:teuthology.orchestra.run.smithi001.stdout:5973 write 0xdf207ee thru 0xdf28468 (0x7c7b bytes) 2024-09-14T22:38:17.881 INFO:teuthology.orchestra.run.smithi001.stdout:5974 read 0x7457aa3 thru 0x746720c (0xf76a bytes) 2024-09-14T22:38:17.882 INFO:teuthology.orchestra.run.smithi001.stdout:5975 write 0xd3c3183 thru 0xd3c6da1 (0x3c1f bytes) 2024-09-14T22:38:17.890 INFO:teuthology.orchestra.run.smithi001.stdout:5977 read 0x1fb7dfc thru 0x1fc6f09 (0xf10e bytes) 2024-09-14T22:38:17.913 INFO:teuthology.orchestra.run.smithi001.stdout:5980 trunc from 0xdf28469 to 0x5831829 2024-09-14T22:38:17.922 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:17.917+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:17.960 INFO:teuthology.orchestra.run.smithi001.stdout:5981 trunc from 0x5831829 to 0xe7e994d 2024-09-14T22:38:17.982 INFO:teuthology.orchestra.run.smithi001.stdout:5982 trunc from 0xe7e994d to 0xb996320 2024-09-14T22:38:17.983 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:17.983 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:17.997 INFO:teuthology.orchestra.run.smithi001.stdout:5984 punch from 0x7ac9dc7 to 0x7acea5a, (0x4c93 bytes) 2024-09-14T22:38:17.997 INFO:teuthology.orchestra.run.smithi001.stdout:5985 read 0x5411425 thru 0x541f7c4 (0xe3a0 bytes) 2024-09-14T22:38:17.998 INFO:teuthology.orchestra.run.smithi001.stdout:5986 punch from 0x8be577a to 0x8bf5351, (0xfbd7 bytes) 2024-09-14T22:38:17.998 INFO:teuthology.orchestra.run.smithi001.stdout:5987 write 0xba124a thru 0xba839c (0x7153 bytes) 2024-09-14T22:38:17.999 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:38:18.005 INFO:teuthology.orchestra.run.smithi001.stdout:5989 read 0x7f78e47 thru 0x7f8804b (0xf205 bytes) 2024-09-14T22:38:18.018 INFO:teuthology.orchestra.run.smithi001.stdout:5992 trunc from 0xb996320 to 0x6fd5517 2024-09-14T22:38:18.025 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:18.017+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:18.033 INFO:teuthology.orchestra.run.smithi001.stdout:5993 write 0x3523739 thru 0x352d249 (0x9b11 bytes) 2024-09-14T22:38:18.035 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2024-09-14T22:38:18.055 INFO:teuthology.orchestra.run.smithi001.stdout:5994 trunc from 0x6fd5517 to 0x1eed0a6 2024-09-14T22:38:18.100 INFO:teuthology.orchestra.run.smithi001.stdout:5995 write 0xe170192 thru 0xe172b35 (0x29a4 bytes) 2024-09-14T22:38:18.104 INFO:teuthology.orchestra.run.smithi001.stdout:5996 write 0xa25ff5 thru 0xa2ca1c (0x6a28 bytes) 2024-09-14T22:38:18.109 INFO:teuthology.orchestra.run.smithi001.stdout:5998 write 0xc6d2eac thru 0xc6db710 (0x8865 bytes) 2024-09-14T22:38:18.113 INFO:teuthology.orchestra.run.smithi001.stdout:5999 read 0x52609dd thru 0x526a08f (0x96b3 bytes) 2024-09-14T22:38:18.124 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:18.124 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:18.125 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:18.118+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:18.137 INFO:teuthology.orchestra.run.smithi001.stdout:6000 trunc from 0xe172b36 to 0x104421c 2024-09-14T22:38:18.138 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:38:18.226 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:18.221+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:18.258 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-09-14T22:38:18.310 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:18.310 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:18.324 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:38:18.327 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:18.321+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:18.428 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:18.422+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:18.443 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:18.443 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:18.457 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:38:18.528 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:18.521+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:18.624 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:18.624 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:18.629 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:18.622+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:18.638 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:38:18.730 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:18.725+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:18.808 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:18.808 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:18.822 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:38:18.831 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:18.825+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:18.932 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:18.926+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:18.949 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:18.950 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:18.964 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:38:19.051 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:19.046+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:19.134 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:19.126+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:19.137 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:19.137 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:19.152 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:38:19.247 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:19.242+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:19.286 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:38:19.329 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:19.329 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:19.336 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:19.330+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:19.343 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:38:19.436 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:19.433+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:19.442 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:38:19.442 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:38:19.458 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:38:19.459 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:38:19.475 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:38:19.475 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:38:19.491 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:38:19.491 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:38:19.511 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:19.511 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:19.512 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:38:19.512 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:38:19.525 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:38:19.531 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:38:19.531 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:38:19.538 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:19.530+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:19.553 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:38:19.554 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:38:19.572 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:38:19.572 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:38:19.639 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:19.634+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:19.643 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:19.643 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:19.643 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:38:19.643 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:19.657 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:38:19.740 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:19.734+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:19.773 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:19.774 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:19.774 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:38:19.774 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:38:19.774 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:19.788 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:38:19.841 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:19.834+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:19.919 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:19.920 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:38:19.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:38:19.920 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:38:19.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.920 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144987 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3424256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:19.920 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549297+0000", 2024-09-14T22:38:19.920 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.36738759, 2024-09-14T22:38:19.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.079931556000000001, 2024-09-14T22:38:19.920 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:19.920 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:19.920 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:19.920 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:19.921 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:19.921 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:19.921 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144987 2024-09-14T22:38:19.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.921 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:19.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:19.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549297+0000", 2024-09-14T22:38:19.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:19.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549297+0000", 2024-09-14T22:38:19.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:19.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549298+0000", 2024-09-14T22:38:19.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2699999999999999e-07 2024-09-14T22:38:19.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:19.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549301+0000", 2024-09-14T22:38:19.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2959999999999999e-06 2024-09-14T22:38:19.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:19.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549314+0000", 2024-09-14T22:38:19.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2588999999999999e-05 2024-09-14T22:38:19.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:19.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549323+0000", 2024-09-14T22:38:19.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.927e-06 2024-09-14T22:38:19.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:19.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629038+0000", 2024-09-14T22:38:19.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.079714220000000002 2024-09-14T22:38:19.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:19.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629116+0000", 2024-09-14T22:38:19.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7966000000000005e-05 2024-09-14T22:38:19.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:19.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629228+0000", 2024-09-14T22:38:19.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112831 2024-09-14T22:38:19.924 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.924 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:19.924 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.924 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144988 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3325952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:19.925 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549335+0000", 2024-09-14T22:38:19.925 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.367349364999999, 2024-09-14T22:38:19.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080179116999999994, 2024-09-14T22:38:19.925 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:19.925 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:19.925 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:19.925 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:19.925 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:19.925 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:19.925 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144988 2024-09-14T22:38:19.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.925 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:19.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:19.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549335+0000", 2024-09-14T22:38:19.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:19.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549335+0000", 2024-09-14T22:38:19.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:19.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549336+0000", 2024-09-14T22:38:19.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7999999999999995e-07 2024-09-14T22:38:19.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:19.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549339+0000", 2024-09-14T22:38:19.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.836e-06 2024-09-14T22:38:19.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:19.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549352+0000", 2024-09-14T22:38:19.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3069e-05 2024-09-14T22:38:19.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:19.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549359+0000", 2024-09-14T22:38:19.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.357e-06 2024-09-14T22:38:19.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:19.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629288+0000", 2024-09-14T22:38:19.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.079928492000000004 2024-09-14T22:38:19.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:19.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629412+0000", 2024-09-14T22:38:19.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012413699999999999 2024-09-14T22:38:19.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:19.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629514+0000", 2024-09-14T22:38:19.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000102546 2024-09-14T22:38:19.928 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.929 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:19.929 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.929 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144989 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3522560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:19.929 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:38:19.929 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.367314957000001, 2024-09-14T22:38:19.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080393122999999997, 2024-09-14T22:38:19.929 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:19.929 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:19.929 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:19.929 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:19.929 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:19.929 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:19.930 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144989 2024-09-14T22:38:19.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.930 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:19.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:19.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:38:19.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:19.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:38:19.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:19.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:38:19.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3600000000000003e-07 2024-09-14T22:38:19.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:19.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549373+0000", 2024-09-14T22:38:19.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8169999999999999e-06 2024-09-14T22:38:19.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:19.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549385+0000", 2024-09-14T22:38:19.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1803e-05 2024-09-14T22:38:19.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:19.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549394+0000", 2024-09-14T22:38:19.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4290000000000007e-06 2024-09-14T22:38:19.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:19.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629576+0000", 2024-09-14T22:38:19.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080181551000000004 2024-09-14T22:38:19.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:19.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629654+0000", 2024-09-14T22:38:19.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7633999999999996e-05 2024-09-14T22:38:19.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:19.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629763+0000", 2024-09-14T22:38:19.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109153 2024-09-14T22:38:19.933 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.933 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:19.933 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.933 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144990 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:19.933 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:38:19.933 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.367271933000001, 2024-09-14T22:38:19.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080614496999999993, 2024-09-14T22:38:19.934 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:19.934 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:19.934 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:19.934 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:19.934 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:19.934 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:19.934 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144990 2024-09-14T22:38:19.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.934 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:19.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:19.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:38:19.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:19.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:38:19.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:19.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:38:19.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0999999999999997e-07 2024-09-14T22:38:19.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:19.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549419+0000", 2024-09-14T22:38:19.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5339999999999999e-06 2024-09-14T22:38:19.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:19.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549439+0000", 2024-09-14T22:38:19.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0035e-05 2024-09-14T22:38:19.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:19.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549449+0000", 2024-09-14T22:38:19.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0429e-05 2024-09-14T22:38:19.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:19.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629815+0000", 2024-09-14T22:38:19.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080365467999999995 2024-09-14T22:38:19.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:19.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629922+0000", 2024-09-14T22:38:19.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106914 2024-09-14T22:38:19.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:19.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630027+0000", 2024-09-14T22:38:19.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105207 2024-09-14T22:38:19.938 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.938 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:19.938 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.939 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144991 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 2998272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:19.939 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549463+0000", 2024-09-14T22:38:19.939 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.367221628999999, 2024-09-14T22:38:19.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080847637, 2024-09-14T22:38:19.939 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:19.939 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:19.939 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:19.939 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:19.939 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:19.939 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:19.939 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144991 2024-09-14T22:38:19.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.939 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:19.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:19.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549463+0000", 2024-09-14T22:38:19.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:19.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549463+0000", 2024-09-14T22:38:19.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:19.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549464+0000", 2024-09-14T22:38:19.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9999999999999997e-07 2024-09-14T22:38:19.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:19.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549467+0000", 2024-09-14T22:38:19.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0800000000000002e-06 2024-09-14T22:38:19.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:19.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549478+0000", 2024-09-14T22:38:19.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1813e-05 2024-09-14T22:38:19.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:19.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549488+0000", 2024-09-14T22:38:19.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7030000000000001e-06 2024-09-14T22:38:19.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:19.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630090+0000", 2024-09-14T22:38:19.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080601962999999999 2024-09-14T22:38:19.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:19.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630187+0000", 2024-09-14T22:38:19.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6552000000000007e-05 2024-09-14T22:38:19.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:19.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630311+0000", 2024-09-14T22:38:19.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000123826 2024-09-14T22:38:19.943 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.943 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:19.943 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.943 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144992 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3620864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:19.943 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:38:19.943 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.367185821, 2024-09-14T22:38:19.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081097875999999999, 2024-09-14T22:38:19.944 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:19.944 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:19.944 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:19.944 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:19.944 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:19.944 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:19.944 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144992 2024-09-14T22:38:19.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.944 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:19.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:19.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:38:19.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:19.945 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:38:19.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:19.945 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:38:19.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5400000000000001e-07 2024-09-14T22:38:19.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:19.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549502+0000", 2024-09-14T22:38:19.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7889999999999999e-06 2024-09-14T22:38:19.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:19.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549513+0000", 2024-09-14T22:38:19.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1204e-05 2024-09-14T22:38:19.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:19.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549523+0000", 2024-09-14T22:38:19.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5990000000000008e-06 2024-09-14T22:38:19.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:19.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630362+0000", 2024-09-14T22:38:19.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080839272000000004 2024-09-14T22:38:19.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:19.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630469+0000", 2024-09-14T22:38:19.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107285 2024-09-14T22:38:19.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:19.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630597+0000", 2024-09-14T22:38:19.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012707300000000001 2024-09-14T22:38:19.947 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.948 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:19.948 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.948 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144993 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:19.948 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:38:19.948 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.367152744999999, 2024-09-14T22:38:19.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081338411999999999, 2024-09-14T22:38:19.948 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:19.948 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:19.948 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:19.948 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:19.948 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:19.949 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:19.949 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144993 2024-09-14T22:38:19.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.949 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:19.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:19.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:38:19.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:19.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:38:19.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:19.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:38:19.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5300000000000004e-07 2024-09-14T22:38:19.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:19.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549544+0000", 2024-09-14T22:38:19.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1912e-05 2024-09-14T22:38:19.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:19.951 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549557+0000", 2024-09-14T22:38:19.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2413e-05 2024-09-14T22:38:19.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.951 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:19.951 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549567+0000", 2024-09-14T22:38:19.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8660000000000001e-06 2024-09-14T22:38:19.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.951 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:19.951 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630651+0000", 2024-09-14T22:38:19.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081084229999999993 2024-09-14T22:38:19.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:19.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630729+0000", 2024-09-14T22:38:19.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8265999999999999e-05 2024-09-14T22:38:19.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:19.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630870+0000", 2024-09-14T22:38:19.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014107200000000001 2024-09-14T22:38:19.952 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.952 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:19.952 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.953 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144994 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:19.953 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:38:19.953 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.367105777999999, 2024-09-14T22:38:19.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081547958000000004, 2024-09-14T22:38:19.953 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:19.953 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:19.953 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:19.953 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:19.953 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:19.953 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:19.953 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144994 2024-09-14T22:38:19.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.953 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:19.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.954 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:19.954 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:38:19.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.954 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:19.954 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:38:19.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.954 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:19.954 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:38:19.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6599999999999996e-07 2024-09-14T22:38:19.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:19.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549582+0000", 2024-09-14T22:38:19.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.004e-06 2024-09-14T22:38:19.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:19.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549594+0000", 2024-09-14T22:38:19.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1459e-05 2024-09-14T22:38:19.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:19.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549603+0000", 2024-09-14T22:38:19.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6129999999999993e-06 2024-09-14T22:38:19.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:19.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630932+0000", 2024-09-14T22:38:19.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081328558999999995 2024-09-14T22:38:19.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:19.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631027+0000", 2024-09-14T22:38:19.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4742999999999997e-05 2024-09-14T22:38:19.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:19.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631127+0000", 2024-09-14T22:38:19.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9914000000000006e-05 2024-09-14T22:38:19.957 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.957 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:19.957 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.957 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144995 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3391488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:19.957 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549615+0000", 2024-09-14T22:38:19.957 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.36706964, 2024-09-14T22:38:19.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081734251999999993, 2024-09-14T22:38:19.957 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:19.957 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:19.957 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:19.958 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:19.958 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:19.958 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:19.958 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144995 2024-09-14T22:38:19.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.958 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:19.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:19.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549615+0000", 2024-09-14T22:38:19.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:19.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549615+0000", 2024-09-14T22:38:19.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:19.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549616+0000", 2024-09-14T22:38:19.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9299999999999997e-07 2024-09-14T22:38:19.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:19.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549618+0000", 2024-09-14T22:38:19.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7269999999999999e-06 2024-09-14T22:38:19.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:19.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549630+0000", 2024-09-14T22:38:19.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1313e-05 2024-09-14T22:38:19.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:19.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549639+0000", 2024-09-14T22:38:19.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4159999999999993e-06 2024-09-14T22:38:19.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:19.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631183+0000", 2024-09-14T22:38:19.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081543633000000004 2024-09-14T22:38:19.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:19.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631252+0000", 2024-09-14T22:38:19.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9567999999999995e-05 2024-09-14T22:38:19.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:19.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631349+0000", 2024-09-14T22:38:19.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6801999999999999e-05 2024-09-14T22:38:19.961 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.961 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:19.961 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.962 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144996 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3194880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:19.962 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:38:19.962 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.367033962000001, 2024-09-14T22:38:19.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081947213000000005, 2024-09-14T22:38:19.962 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:19.962 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:19.962 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:19.962 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:19.962 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:19.962 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:19.962 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144996 2024-09-14T22:38:19.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.962 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:19.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:19.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:38:19.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:19.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:38:19.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:19.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:38:19.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3e-07 2024-09-14T22:38:19.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:19.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549654+0000", 2024-09-14T22:38:19.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6900000000000001e-06 2024-09-14T22:38:19.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:19.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549665+0000", 2024-09-14T22:38:19.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1392999999999999e-05 2024-09-14T22:38:19.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:19.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549675+0000", 2024-09-14T22:38:19.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5459999999999997e-06 2024-09-14T22:38:19.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:19.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631401+0000", 2024-09-14T22:38:19.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081726036000000002 2024-09-14T22:38:19.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:19.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631492+0000", 2024-09-14T22:38:19.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0886999999999999e-05 2024-09-14T22:38:19.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:19.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631598+0000", 2024-09-14T22:38:19.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106031 2024-09-14T22:38:19.966 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.966 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:19.966 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.966 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144997 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:19.966 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549692+0000", 2024-09-14T22:38:19.966 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.366992298, 2024-09-14T22:38:19.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082149925999999998, 2024-09-14T22:38:19.966 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:19.966 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:19.967 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:19.967 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:19.967 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:19.967 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:19.967 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144997 2024-09-14T22:38:19.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.967 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:19.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:19.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549692+0000", 2024-09-14T22:38:19.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:19.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549692+0000", 2024-09-14T22:38:19.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:19.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549693+0000", 2024-09-14T22:38:19.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.540000000000001e-07 2024-09-14T22:38:19.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:19.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549698+0000", 2024-09-14T22:38:19.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1460000000000004e-06 2024-09-14T22:38:19.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:19.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549715+0000", 2024-09-14T22:38:19.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6983e-05 2024-09-14T22:38:19.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:19.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549728+0000", 2024-09-14T22:38:19.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2326e-05 2024-09-14T22:38:19.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:19.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631649+0000", 2024-09-14T22:38:19.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081921830000000001 2024-09-14T22:38:19.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:19.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631726+0000", 2024-09-14T22:38:19.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6759000000000001e-05 2024-09-14T22:38:19.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:19.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631842+0000", 2024-09-14T22:38:19.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011592800000000001 2024-09-14T22:38:19.971 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.971 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:19.971 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.971 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144998 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 2965504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:19.971 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549741+0000", 2024-09-14T22:38:19.971 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.366943006, 2024-09-14T22:38:19.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082394832000000001, 2024-09-14T22:38:19.971 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:19.971 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:19.971 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:19.971 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:19.972 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:19.972 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:19.972 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144998 2024-09-14T22:38:19.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.972 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:19.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:19.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549741+0000", 2024-09-14T22:38:19.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:19.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549741+0000", 2024-09-14T22:38:19.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.973 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:19.973 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549742+0000", 2024-09-14T22:38:19.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.06e-07 2024-09-14T22:38:19.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.973 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:19.973 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549757+0000", 2024-09-14T22:38:19.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4839e-05 2024-09-14T22:38:19.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.973 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:19.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549770+0000", 2024-09-14T22:38:19.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2879e-05 2024-09-14T22:38:19.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:19.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549780+0000", 2024-09-14T22:38:19.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0304e-05 2024-09-14T22:38:19.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:19.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631904+0000", 2024-09-14T22:38:19.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082124168999999997 2024-09-14T22:38:19.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:19.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631995+0000", 2024-09-14T22:38:19.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0993000000000001e-05 2024-09-14T22:38:19.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:19.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632136+0000", 2024-09-14T22:38:19.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014094199999999999 2024-09-14T22:38:19.975 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.975 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:19.975 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.976 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144999 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3588096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:19.976 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:38:19.976 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.366892795999998, 2024-09-14T22:38:19.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082593742999999997, 2024-09-14T22:38:19.976 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:19.976 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:19.976 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:19.976 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:19.976 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:19.976 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:19.976 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144999 2024-09-14T22:38:19.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.976 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:19.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:19.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:38:19.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:19.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:38:19.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:19.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:38:19.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1999999999999999e-07 2024-09-14T22:38:19.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:19.978 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549795+0000", 2024-09-14T22:38:19.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9569999999999998e-06 2024-09-14T22:38:19.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:19.978 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549807+0000", 2024-09-14T22:38:19.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1691999999999999e-05 2024-09-14T22:38:19.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:19.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549817+0000", 2024-09-14T22:38:19.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5100000000000004e-06 2024-09-14T22:38:19.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:19.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632190+0000", 2024-09-14T22:38:19.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082373374999999999 2024-09-14T22:38:19.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:19.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632264+0000", 2024-09-14T22:38:19.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3972000000000003e-05 2024-09-14T22:38:19.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:19.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632385+0000", 2024-09-14T22:38:19.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121517 2024-09-14T22:38:19.980 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.980 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:19.980 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.980 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145000 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3063808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:19.980 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549828+0000", 2024-09-14T22:38:19.981 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.366856517999999, 2024-09-14T22:38:19.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082860235000000004, 2024-09-14T22:38:19.981 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:19.981 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:19.981 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:19.981 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:19.981 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:19.981 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:19.981 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145000 2024-09-14T22:38:19.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.981 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:19.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:19.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549828+0000", 2024-09-14T22:38:19.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:19.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549828+0000", 2024-09-14T22:38:19.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:19.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549829+0000", 2024-09-14T22:38:19.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3300000000000002e-07 2024-09-14T22:38:19.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:19.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549831+0000", 2024-09-14T22:38:19.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6740000000000001e-06 2024-09-14T22:38:19.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:19.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549842+0000", 2024-09-14T22:38:19.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0316000000000001e-05 2024-09-14T22:38:19.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:19.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549851+0000", 2024-09-14T22:38:19.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7229999999999999e-06 2024-09-14T22:38:19.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:19.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632448+0000", 2024-09-14T22:38:19.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082596648999999994 2024-09-14T22:38:19.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:19.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632568+0000", 2024-09-14T22:38:19.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119592 2024-09-14T22:38:19.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:19.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632688+0000", 2024-09-14T22:38:19.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120648 2024-09-14T22:38:19.985 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.985 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:19.985 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.985 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145001 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3162112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:19.985 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549863+0000", 2024-09-14T22:38:19.985 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.366821290000001, 2024-09-14T22:38:19.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083105534999999994, 2024-09-14T22:38:19.986 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:19.986 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:19.986 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:19.986 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:19.987 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:19.987 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:19.987 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145001 2024-09-14T22:38:19.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.987 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:19.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:19.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549863+0000", 2024-09-14T22:38:19.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:19.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549863+0000", 2024-09-14T22:38:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549864+0000", 2024-09-14T22:38:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4300000000000003e-07 2024-09-14T22:38:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549867+0000", 2024-09-14T22:38:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.757e-06 2024-09-14T22:38:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549878+0000", 2024-09-14T22:38:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1345999999999999e-05 2024-09-14T22:38:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549888+0000", 2024-09-14T22:38:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6460000000000005e-06 2024-09-14T22:38:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632754+0000", 2024-09-14T22:38:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082866258999999998 2024-09-14T22:38:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:19.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632841+0000", 2024-09-14T22:38:19.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6971000000000005e-05 2024-09-14T22:38:19.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:19.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632969+0000", 2024-09-14T22:38:19.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012791300000000001 2024-09-14T22:38:19.990 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.990 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:19.990 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.991 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145002 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3260416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:19.991 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549933+0000", 2024-09-14T22:38:19.991 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.366751786999998, 2024-09-14T22:38:19.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083291027000000004, 2024-09-14T22:38:19.991 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:19.991 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:19.991 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:19.991 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:19.991 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:19.991 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:19.991 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145002 2024-09-14T22:38:19.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.991 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:19.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:19.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549933+0000", 2024-09-14T22:38:19.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:19.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549933+0000", 2024-09-14T22:38:19.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:19.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549934+0000", 2024-09-14T22:38:19.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8700000000000004e-07 2024-09-14T22:38:19.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:19.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549938+0000", 2024-09-14T22:38:19.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5029999999999999e-06 2024-09-14T22:38:19.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:19.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549952+0000", 2024-09-14T22:38:19.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4382999999999999e-05 2024-09-14T22:38:19.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:19.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549963+0000", 2024-09-14T22:38:19.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0665999999999999e-05 2024-09-14T22:38:19.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:19.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633025+0000", 2024-09-14T22:38:19.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083061727000000002 2024-09-14T22:38:19.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:19.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633117+0000", 2024-09-14T22:38:19.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1812999999999997e-05 2024-09-14T22:38:19.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:19.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633224+0000", 2024-09-14T22:38:19.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107048 2024-09-14T22:38:19.995 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.995 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:19.995 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.995 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145003 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3457024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:19.995 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549985+0000", 2024-09-14T22:38:19.995 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.366699368999999, 2024-09-14T22:38:19.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083465871999999997, 2024-09-14T22:38:19.995 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:19.995 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:19.995 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:19.996 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:19.996 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:19.996 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:19.996 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145003 2024-09-14T22:38:19.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.996 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:19.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:19.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549985+0000", 2024-09-14T22:38:19.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:19.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549985+0000", 2024-09-14T22:38:19.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:19.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:19.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549986+0000", 2024-09-14T22:38:19.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7299999999999995e-07 2024-09-14T22:38:19.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:19.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549989+0000", 2024-09-14T22:38:19.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8169999999999999e-06 2024-09-14T22:38:19.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:19.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549998+0000", 2024-09-14T22:38:19.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4290000000000007e-06 2024-09-14T22:38:19.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:19.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550009+0000", 2024-09-14T22:38:19.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0633e-05 2024-09-14T22:38:19.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:19.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633275+0000", 2024-09-14T22:38:19.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083266397000000006 2024-09-14T22:38:19.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:19.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633353+0000", 2024-09-14T22:38:19.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8129999999999999e-05 2024-09-14T22:38:19.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:19.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:19.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:19.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633451+0000", 2024-09-14T22:38:19.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7793000000000004e-05 2024-09-14T22:38:19.999 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.999 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:19.999 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:19.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.000 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145004 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3358720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:20.000 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.550029+0000", 2024-09-14T22:38:20.000 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.366655798, 2024-09-14T22:38:20.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083662089999999995, 2024-09-14T22:38:20.000 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:20.000 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:20.000 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:20.000 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:20.000 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:20.000 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:20.000 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145004 2024-09-14T22:38:20.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.001 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:20.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:20.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550029+0000", 2024-09-14T22:38:20.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:20.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550029+0000", 2024-09-14T22:38:20.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:20.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550030+0000", 2024-09-14T22:38:20.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.33e-07 2024-09-14T22:38:20.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.002 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:20.002 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550035+0000", 2024-09-14T22:38:20.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3430000000000003e-06 2024-09-14T22:38:20.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.002 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:20.002 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550053+0000", 2024-09-14T22:38:20.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8485e-05 2024-09-14T22:38:20.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.002 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:20.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550064+0000", 2024-09-14T22:38:20.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0264e-05 2024-09-14T22:38:20.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:20.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633502+0000", 2024-09-14T22:38:20.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083438498999999999 2024-09-14T22:38:20.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:20.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633592+0000", 2024-09-14T22:38:20.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.022e-05 2024-09-14T22:38:20.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:20.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633691+0000", 2024-09-14T22:38:20.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8345999999999997e-05 2024-09-14T22:38:20.004 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.004 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:20.004 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.004 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145005 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3555328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:20.004 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.550077+0000", 2024-09-14T22:38:20.004 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.366607338000001, 2024-09-14T22:38:20.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083854340999999999, 2024-09-14T22:38:20.004 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:20.004 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:20.005 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:20.005 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:20.005 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:20.005 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:20.005 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145005 2024-09-14T22:38:20.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.005 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:20.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:20.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550077+0000", 2024-09-14T22:38:20.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:20.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550077+0000", 2024-09-14T22:38:20.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:20.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550078+0000", 2024-09-14T22:38:20.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8700000000000005e-07 2024-09-14T22:38:20.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:20.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550101+0000", 2024-09-14T22:38:20.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3632e-05 2024-09-14T22:38:20.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:20.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550115+0000", 2024-09-14T22:38:20.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3893000000000001e-05 2024-09-14T22:38:20.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:20.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550126+0000", 2024-09-14T22:38:20.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0426e-05 2024-09-14T22:38:20.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:20.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633742+0000", 2024-09-14T22:38:20.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083616633999999995 2024-09-14T22:38:20.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:20.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633816+0000", 2024-09-14T22:38:20.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3690999999999996e-05 2024-09-14T22:38:20.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:20.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633932+0000", 2024-09-14T22:38:20.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011537800000000001 2024-09-14T22:38:20.008 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.008 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:20.008 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.008 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145006 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3653632~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:20.009 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.550236+0000", 2024-09-14T22:38:20.009 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.366448832, 2024-09-14T22:38:20.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083924027999999998, 2024-09-14T22:38:20.009 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:20.009 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:20.009 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:20.009 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:20.009 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:20.009 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:20.009 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145006 2024-09-14T22:38:20.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.009 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:20.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:20.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550236+0000", 2024-09-14T22:38:20.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:20.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550236+0000", 2024-09-14T22:38:20.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:20.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550237+0000", 2024-09-14T22:38:20.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2929999999999999e-06 2024-09-14T22:38:20.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:20.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550243+0000", 2024-09-14T22:38:20.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0390000000000001e-06 2024-09-14T22:38:20.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:20.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550256+0000", 2024-09-14T22:38:20.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3393e-05 2024-09-14T22:38:20.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:20.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550271+0000", 2024-09-14T22:38:20.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4166e-05 2024-09-14T22:38:20.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:20.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633974+0000", 2024-09-14T22:38:20.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083703825999999995 2024-09-14T22:38:20.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:20.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.634047+0000", 2024-09-14T22:38:20.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2935999999999994e-05 2024-09-14T22:38:20.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:20.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.634160+0000", 2024-09-14T22:38:20.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011237499999999999 2024-09-14T22:38:20.013 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.013 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:20.013 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.013 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.013 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:20.013 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:20.015 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:38:20.015 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:20.010+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:20.040 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-14T22:38:20.040 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:38:20.040 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-14T22:38:20.040 INFO:tasks.thrashosds.thrasher:Setting osd 5 primary_affinity to 0.297132 2024-09-14T22:38:20.040 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 0.2971319225787109 2024-09-14T22:38:20.068 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T22:38:20.116 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:20.110+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:20.158 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:20.158 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:20.158 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:38:20.159 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:20.178 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:38:20.217 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:20.210+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:20.313 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:20.313 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:20.313 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:38:20.313 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:38:20.314 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:20.318 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:20.317+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:20.328 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:38:20.419 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:20.414+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:20.488 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:20.488 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:38:20.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:38:20.488 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:38:20.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.488 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:122960 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 2347008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:20.488 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:41.998325+0000", 2024-09-14T22:38:20.488 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 38.487175487000002, 2024-09-14T22:38:20.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056570400999999999, 2024-09-14T22:38:20.488 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 122960 2024-09-14T22:38:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998325+0000", 2024-09-14T22:38:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:20.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998325+0000", 2024-09-14T22:38:20.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:20.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998326+0000", 2024-09-14T22:38:20.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9299999999999998e-07 2024-09-14T22:38:20.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:20.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998329+0000", 2024-09-14T22:38:20.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1370000000000002e-06 2024-09-14T22:38:20.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.492 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.492 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:20.492 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998340+0000", 2024-09-14T22:38:20.492 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1039999999999999e-05 2024-09-14T22:38:20.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.492 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.492 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:20.492 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998348+0000", 2024-09-14T22:38:20.492 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1920000000000005e-06 2024-09-14T22:38:20.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.492 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.492 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:20.493 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054700+0000", 2024-09-14T22:38:20.493 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056351714999999997 2024-09-14T22:38:20.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.493 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:20.493 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054771+0000", 2024-09-14T22:38:20.493 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1009999999999999e-05 2024-09-14T22:38:20.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.493 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:20.493 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054896+0000", 2024-09-14T22:38:20.493 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000124714 2024-09-14T22:38:20.493 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.494 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:20.494 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.494 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:122961 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 2351104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:20.494 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:41.998357+0000", 2024-09-14T22:38:20.494 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 38.487143531999997, 2024-09-14T22:38:20.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056974912000000003, 2024-09-14T22:38:20.494 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:20.494 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:20.494 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:20.494 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:20.494 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:20.494 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:20.495 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 122961 2024-09-14T22:38:20.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.495 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:20.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:20.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998357+0000", 2024-09-14T22:38:20.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:20.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998357+0000", 2024-09-14T22:38:20.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:20.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998358+0000", 2024-09-14T22:38:20.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9699999999999996e-07 2024-09-14T22:38:20.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.496 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:20.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998361+0000", 2024-09-14T22:38:20.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1370000000000002e-06 2024-09-14T22:38:20.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.496 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:20.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998371+0000", 2024-09-14T22:38:20.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5789999999999993e-06 2024-09-14T22:38:20.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:20.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998379+0000", 2024-09-14T22:38:20.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0129999999999993e-06 2024-09-14T22:38:20.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:20.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054998+0000", 2024-09-14T22:38:20.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056618961000000002 2024-09-14T22:38:20.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:20.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.055126+0000", 2024-09-14T22:38:20.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012795100000000001 2024-09-14T22:38:20.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.498 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:20.498 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.055332+0000", 2024-09-14T22:38:20.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020667400000000001 2024-09-14T22:38:20.498 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.498 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:20.498 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.498 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123259 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 741376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:20.498 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:38:20.498 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 38.473418015, 2024-09-14T22:38:20.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056672581999999999, 2024-09-14T22:38:20.499 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:20.499 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:20.499 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:20.499 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:20.499 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:20.499 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:20.499 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123259 2024-09-14T22:38:20.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.499 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:20.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:20.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:38:20.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:20.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:38:20.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:20.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:38:20.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1600000000000001e-07 2024-09-14T22:38:20.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:20.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012086+0000", 2024-09-14T22:38:20.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8499999999999998e-06 2024-09-14T22:38:20.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:20.501 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012097+0000", 2024-09-14T22:38:20.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0943e-05 2024-09-14T22:38:20.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:20.501 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012105+0000", 2024-09-14T22:38:20.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0830000000000004e-06 2024-09-14T22:38:20.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.502 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:20.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068584+0000", 2024-09-14T22:38:20.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056478387999999997 2024-09-14T22:38:20.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.502 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:20.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068644+0000", 2024-09-14T22:38:20.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9984e-05 2024-09-14T22:38:20.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.502 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:20.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068755+0000", 2024-09-14T22:38:20.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111718 2024-09-14T22:38:20.503 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.503 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:20.503 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.503 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123260 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 778240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:20.503 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012118+0000", 2024-09-14T22:38:20.503 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 38.473382790000002, 2024-09-14T22:38:20.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056825887999999998, 2024-09-14T22:38:20.504 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:20.504 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:20.504 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:20.504 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:20.504 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:20.504 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:20.504 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123260 2024-09-14T22:38:20.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.504 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:20.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.504 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:20.504 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012118+0000", 2024-09-14T22:38:20.504 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:20.505 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012118+0000", 2024-09-14T22:38:20.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:20.505 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012119+0000", 2024-09-14T22:38:20.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.13e-07 2024-09-14T22:38:20.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:20.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012130+0000", 2024-09-14T22:38:20.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1552999999999999e-05 2024-09-14T22:38:20.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:20.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012141+0000", 2024-09-14T22:38:20.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.03e-05 2024-09-14T22:38:20.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:20.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012149+0000", 2024-09-14T22:38:20.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1419999999999993e-06 2024-09-14T22:38:20.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:20.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068800+0000", 2024-09-14T22:38:20.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056651574000000003 2024-09-14T22:38:20.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:20.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068858+0000", 2024-09-14T22:38:20.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8047000000000002e-05 2024-09-14T22:38:20.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:20.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068944+0000", 2024-09-14T22:38:20.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5659000000000001e-05 2024-09-14T22:38:20.508 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.508 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:20.508 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.508 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123261 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 761856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:20.508 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012161+0000", 2024-09-14T22:38:20.508 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 38.473339389000003, 2024-09-14T22:38:20.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057113753000000003, 2024-09-14T22:38:20.508 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:20.508 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:20.508 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:20.508 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:20.509 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:20.509 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:20.509 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123261 2024-09-14T22:38:20.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.509 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:20.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:20.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012161+0000", 2024-09-14T22:38:20.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:20.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012161+0000", 2024-09-14T22:38:20.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:20.510 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012162+0000", 2024-09-14T22:38:20.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4399999999999999e-07 2024-09-14T22:38:20.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:20.510 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012177+0000", 2024-09-14T22:38:20.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5099000000000001e-05 2024-09-14T22:38:20.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:20.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012191+0000", 2024-09-14T22:38:20.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3631999999999999e-05 2024-09-14T22:38:20.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:20.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012199+0000", 2024-09-14T22:38:20.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5769999999999994e-06 2024-09-14T22:38:20.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:20.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069004+0000", 2024-09-14T22:38:20.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056804504999999998 2024-09-14T22:38:20.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:20.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069137+0000", 2024-09-14T22:38:20.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013302600000000001 2024-09-14T22:38:20.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:20.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069275+0000", 2024-09-14T22:38:20.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013816999999999999 2024-09-14T22:38:20.512 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.512 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:20.513 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.513 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123262 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 724992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:20.513 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012224+0000", 2024-09-14T22:38:20.513 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 38.473276941999998, 2024-09-14T22:38:20.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057417959999999997, 2024-09-14T22:38:20.513 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:20.513 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:20.513 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:20.513 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:20.513 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:20.513 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:20.514 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123262 2024-09-14T22:38:20.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.514 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:20.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.514 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:20.514 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012224+0000", 2024-09-14T22:38:20.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.514 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:20.514 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012224+0000", 2024-09-14T22:38:20.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:20.515 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012225+0000", 2024-09-14T22:38:20.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4000000000000001e-07 2024-09-14T22:38:20.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:20.515 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012228+0000", 2024-09-14T22:38:20.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5729999999999999e-06 2024-09-14T22:38:20.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:20.515 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012241+0000", 2024-09-14T22:38:20.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3210000000000001e-05 2024-09-14T22:38:20.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:20.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012255+0000", 2024-09-14T22:38:20.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3852000000000001e-05 2024-09-14T22:38:20.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:20.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069341+0000", 2024-09-14T22:38:20.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057086139000000001 2024-09-14T22:38:20.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:20.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069477+0000", 2024-09-14T22:38:20.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013595 2024-09-14T22:38:20.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:20.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069642+0000", 2024-09-14T22:38:20.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016459600000000001 2024-09-14T22:38:20.517 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.517 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:20.517 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.517 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123263 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 782336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:20.517 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012265+0000", 2024-09-14T22:38:20.518 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 38.4732354, 2024-09-14T22:38:20.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057665095, 2024-09-14T22:38:20.518 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:20.518 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:20.518 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:20.518 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:20.518 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:20.518 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:20.518 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123263 2024-09-14T22:38:20.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.518 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:20.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:20.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012265+0000", 2024-09-14T22:38:20.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:20.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012265+0000", 2024-09-14T22:38:20.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:20.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012266+0000", 2024-09-14T22:38:20.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8999999999999996e-07 2024-09-14T22:38:20.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:20.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012269+0000", 2024-09-14T22:38:20.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.796e-06 2024-09-14T22:38:20.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:20.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012279+0000", 2024-09-14T22:38:20.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7389999999999994e-06 2024-09-14T22:38:20.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:20.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012286+0000", 2024-09-14T22:38:20.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8199999999999997e-06 2024-09-14T22:38:20.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:20.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069705+0000", 2024-09-14T22:38:20.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057418590999999998 2024-09-14T22:38:20.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:20.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069799+0000", 2024-09-14T22:38:20.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4419999999999994e-05 2024-09-14T22:38:20.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:20.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069931+0000", 2024-09-14T22:38:20.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013113900000000001 2024-09-14T22:38:20.522 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.522 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:20.522 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.522 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123264 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 745472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:20.522 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012295+0000", 2024-09-14T22:38:20.522 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 38.473205399000001, 2024-09-14T22:38:20.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057959498999999998, 2024-09-14T22:38:20.522 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:20.522 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:20.522 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:20.523 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:20.523 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:20.523 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:20.523 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123264 2024-09-14T22:38:20.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.523 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:20.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.523 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:20.523 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012295+0000", 2024-09-14T22:38:20.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.523 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:20.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012295+0000", 2024-09-14T22:38:20.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:20.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012296+0000", 2024-09-14T22:38:20.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9699999999999996e-07 2024-09-14T22:38:20.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:20.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012299+0000", 2024-09-14T22:38:20.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.79e-06 2024-09-14T22:38:20.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:20.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012320+0000", 2024-09-14T22:38:20.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1242000000000001e-05 2024-09-14T22:38:20.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:20.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012338+0000", 2024-09-14T22:38:20.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7768999999999999e-05 2024-09-14T22:38:20.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:20.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069995+0000", 2024-09-14T22:38:20.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057657 2024-09-14T22:38:20.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:20.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070132+0000", 2024-09-14T22:38:20.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013727800000000001 2024-09-14T22:38:20.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:20.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070255+0000", 2024-09-14T22:38:20.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012282300000000001 2024-09-14T22:38:20.526 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.526 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:20.526 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123265 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1056768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012348+0000", 2024-09-14T22:38:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 38.473152444, 2024-09-14T22:38:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058145087999999998, 2024-09-14T22:38:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123265 2024-09-14T22:38:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:20.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.528 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:20.528 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012348+0000", 2024-09-14T22:38:20.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.528 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:20.528 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012348+0000", 2024-09-14T22:38:20.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.528 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:20.528 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012349+0000", 2024-09-14T22:38:20.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6599999999999996e-07 2024-09-14T22:38:20.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:20.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012352+0000", 2024-09-14T22:38:20.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3139999999999999e-06 2024-09-14T22:38:20.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:20.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012365+0000", 2024-09-14T22:38:20.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2625e-05 2024-09-14T22:38:20.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:20.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012373+0000", 2024-09-14T22:38:20.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2470000000000008e-06 2024-09-14T22:38:20.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:20.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070286+0000", 2024-09-14T22:38:20.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057912547000000002 2024-09-14T22:38:20.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:20.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070366+0000", 2024-09-14T22:38:20.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0468000000000003e-05 2024-09-14T22:38:20.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:20.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070493+0000", 2024-09-14T22:38:20.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012722100000000001 2024-09-14T22:38:20.531 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.531 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:20.531 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.531 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123266 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 765952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:20.531 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:38:20.531 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 38.473118196000001, 2024-09-14T22:38:20.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058386949, 2024-09-14T22:38:20.531 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:20.531 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:20.532 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:20.532 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:20.532 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:20.532 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:20.532 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123266 2024-09-14T22:38:20.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.532 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:20.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.532 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:20.532 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:38:20.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.533 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:20.533 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:38:20.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.533 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:20.533 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:38:20.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9299999999999998e-07 2024-09-14T22:38:20.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.533 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:20.533 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012386+0000", 2024-09-14T22:38:20.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8729999999999999e-06 2024-09-14T22:38:20.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:20.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012406+0000", 2024-09-14T22:38:20.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0103e-05 2024-09-14T22:38:20.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:20.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012419+0000", 2024-09-14T22:38:20.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.311e-05 2024-09-14T22:38:20.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:20.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070550+0000", 2024-09-14T22:38:20.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058130902999999998 2024-09-14T22:38:20.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:20.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070650+0000", 2024-09-14T22:38:20.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010005000000000001 2024-09-14T22:38:20.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:20.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070770+0000", 2024-09-14T22:38:20.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119317 2024-09-14T22:38:20.535 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.535 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:20.535 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123267 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1077248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012429+0000", 2024-09-14T22:38:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 38.473072281999997, 2024-09-14T22:38:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058620343999999998, 2024-09-14T22:38:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123267 2024-09-14T22:38:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:20.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.537 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:20.537 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012429+0000", 2024-09-14T22:38:20.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.537 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:20.537 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012429+0000", 2024-09-14T22:38:20.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.537 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:20.537 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013488+0000", 2024-09-14T22:38:20.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001059061 2024-09-14T22:38:20.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:20.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013499+0000", 2024-09-14T22:38:20.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1348999999999999e-05 2024-09-14T22:38:20.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:20.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013534+0000", 2024-09-14T22:38:20.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5206000000000001e-05 2024-09-14T22:38:20.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:20.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013558+0000", 2024-09-14T22:38:20.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4324999999999998e-05 2024-09-14T22:38:20.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:20.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070834+0000", 2024-09-14T22:38:20.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057275110999999997 2024-09-14T22:38:20.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:20.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070937+0000", 2024-09-14T22:38:20.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010306099999999999 2024-09-14T22:38:20.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:20.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071049+0000", 2024-09-14T22:38:20.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112231 2024-09-14T22:38:20.540 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.540 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:20.540 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.540 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123268 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1097728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:20.540 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013593+0000", 2024-09-14T22:38:20.540 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 38.471907559000002, 2024-09-14T22:38:20.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057751793000000003, 2024-09-14T22:38:20.540 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:20.540 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:20.540 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:20.541 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:20.541 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:20.541 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:20.541 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123268 2024-09-14T22:38:20.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.541 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:20.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.541 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:20.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013593+0000", 2024-09-14T22:38:20.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.541 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:20.542 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013593+0000", 2024-09-14T22:38:20.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.542 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:20.542 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013595+0000", 2024-09-14T22:38:20.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.59e-06 2024-09-14T22:38:20.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.542 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:20.542 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013602+0000", 2024-09-14T22:38:20.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3560000000000004e-06 2024-09-14T22:38:20.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:20.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013625+0000", 2024-09-14T22:38:20.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2419e-05 2024-09-14T22:38:20.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:20.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013650+0000", 2024-09-14T22:38:20.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5151999999999999e-05 2024-09-14T22:38:20.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:20.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071096+0000", 2024-09-14T22:38:20.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057445877999999999 2024-09-14T22:38:20.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:20.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071212+0000", 2024-09-14T22:38:20.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116215 2024-09-14T22:38:20.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:20.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071345+0000", 2024-09-14T22:38:20.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013318299999999999 2024-09-14T22:38:20.544 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.544 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:20.544 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.545 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123269 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1060864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:20.545 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013676+0000", 2024-09-14T22:38:20.545 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 38.4718248, 2024-09-14T22:38:20.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058027982999999998, 2024-09-14T22:38:20.545 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:20.545 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:20.545 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:20.545 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:20.545 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:20.545 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:20.546 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123269 2024-09-14T22:38:20.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.546 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:20.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.546 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:20.546 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013676+0000", 2024-09-14T22:38:20.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.546 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:20.546 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013676+0000", 2024-09-14T22:38:20.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:20.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013677+0000", 2024-09-14T22:38:20.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.387e-06 2024-09-14T22:38:20.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:20.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013684+0000", 2024-09-14T22:38:20.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0530000000000001e-06 2024-09-14T22:38:20.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:20.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013706+0000", 2024-09-14T22:38:20.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1121999999999999e-05 2024-09-14T22:38:20.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:20.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013722+0000", 2024-09-14T22:38:20.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6249999999999999e-05 2024-09-14T22:38:20.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:20.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071402+0000", 2024-09-14T22:38:20.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057679945000000003 2024-09-14T22:38:20.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:20.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071547+0000", 2024-09-14T22:38:20.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000145632 2024-09-14T22:38:20.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:20.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071704+0000", 2024-09-14T22:38:20.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000156594 2024-09-14T22:38:20.549 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.549 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:20.549 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.549 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123270 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1064960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:20.549 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013744+0000", 2024-09-14T22:38:20.550 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 38.471756849000002, 2024-09-14T22:38:20.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058355872000000003, 2024-09-14T22:38:20.550 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:20.550 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:20.550 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:20.550 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:20.550 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:20.550 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:20.550 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123270 2024-09-14T22:38:20.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.550 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:20.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.550 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:20.550 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013744+0000", 2024-09-14T22:38:20.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.551 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:20.551 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013744+0000", 2024-09-14T22:38:20.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.551 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:20.551 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013745+0000", 2024-09-14T22:38:20.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2589999999999999e-06 2024-09-14T22:38:20.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:20.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013751+0000", 2024-09-14T22:38:20.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1929999999999998e-06 2024-09-14T22:38:20.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:20.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013778+0000", 2024-09-14T22:38:20.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6472e-05 2024-09-14T22:38:20.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:20.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013795+0000", 2024-09-14T22:38:20.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6626e-05 2024-09-14T22:38:20.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:20.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071775+0000", 2024-09-14T22:38:20.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057980232 2024-09-14T22:38:20.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:20.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071911+0000", 2024-09-14T22:38:20.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000136129 2024-09-14T22:38:20.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:20.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072100+0000", 2024-09-14T22:38:20.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000188961 2024-09-14T22:38:20.554 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.554 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:20.554 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.554 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123271 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1081344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:20.554 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013818+0000", 2024-09-14T22:38:20.554 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 38.471682653000002, 2024-09-14T22:38:20.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058549565999999997, 2024-09-14T22:38:20.554 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:20.554 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:20.555 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:20.555 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:20.555 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:20.555 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:20.555 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123271 2024-09-14T22:38:20.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.555 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:20.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.555 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:20.555 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013818+0000", 2024-09-14T22:38:20.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.555 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:20.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013818+0000", 2024-09-14T22:38:20.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:20.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013958+0000", 2024-09-14T22:38:20.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013934 2024-09-14T22:38:20.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:20.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013968+0000", 2024-09-14T22:38:20.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0329000000000001e-05 2024-09-14T22:38:20.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:20.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013995+0000", 2024-09-14T22:38:20.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7613000000000001e-05 2024-09-14T22:38:20.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:20.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014014+0000", 2024-09-14T22:38:20.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8338999999999998e-05 2024-09-14T22:38:20.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:20.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072155+0000", 2024-09-14T22:38:20.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058140989999999997 2024-09-14T22:38:20.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:20.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072247+0000", 2024-09-14T22:38:20.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1774999999999995e-05 2024-09-14T22:38:20.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:20.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072368+0000", 2024-09-14T22:38:20.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012118 2024-09-14T22:38:20.558 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.558 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:20.559 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.559 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123272 3.1 3.4b235a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:20.559 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.014046+0000", 2024-09-14T22:38:20.559 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 38.471454977, 2024-09-14T22:38:20.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058364077, 2024-09-14T22:38:20.559 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:20.559 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:20.559 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-14T22:38:20.559 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:20.559 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:20.559 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:20.560 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123272 2024-09-14T22:38:20.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.560 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:20.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.560 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:20.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014046+0000", 2024-09-14T22:38:20.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.560 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:20.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014046+0000", 2024-09-14T22:38:20.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:20.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014047+0000", 2024-09-14T22:38:20.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.533e-06 2024-09-14T22:38:20.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:20.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014085+0000", 2024-09-14T22:38:20.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7734999999999997e-05 2024-09-14T22:38:20.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:20.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014111+0000", 2024-09-14T22:38:20.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6055999999999999e-05 2024-09-14T22:38:20.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:20.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014136+0000", 2024-09-14T22:38:20.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4592000000000001e-05 2024-09-14T22:38:20.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:20.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072402+0000", 2024-09-14T22:38:20.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058266251999999998 2024-09-14T22:38:20.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:20.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072410+0000", 2024-09-14T22:38:20.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.909e-06 2024-09-14T22:38:20.563 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.563 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:20.563 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.563 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123273 3.1 3.4b235a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:20.563 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.014161+0000", 2024-09-14T22:38:20.563 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 38.471339948999997, 2024-09-14T22:38:20.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058284415999999999, 2024-09-14T22:38:20.563 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:20.563 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:20.563 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-14T22:38:20.564 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:20.564 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:20.564 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:20.564 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123273 2024-09-14T22:38:20.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.564 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:20.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.564 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:20.564 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014161+0000", 2024-09-14T22:38:20.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.564 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:20.565 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014161+0000", 2024-09-14T22:38:20.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:20.565 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.017996+0000", 2024-09-14T22:38:20.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0038350670000000002 2024-09-14T22:38:20.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:20.565 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.018009+0000", 2024-09-14T22:38:20.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3305999999999999e-05 2024-09-14T22:38:20.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:20.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.018044+0000", 2024-09-14T22:38:20.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4792000000000002e-05 2024-09-14T22:38:20.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:20.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.018067+0000", 2024-09-14T22:38:20.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2595e-05 2024-09-14T22:38:20.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:20.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072439+0000", 2024-09-14T22:38:20.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054372192 2024-09-14T22:38:20.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:20.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072445+0000", 2024-09-14T22:38:20.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4640000000000003e-06 2024-09-14T22:38:20.567 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.567 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:20.567 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.568 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:148469 3.18 3:1f6f7244:::rbd_header.113c827b523f:head [watch reconnect cookie 140250011614800 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-14T22:38:20.568 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:55.746449+0000", 2024-09-14T22:38:20.568 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 24.739051336999999, 2024-09-14T22:38:20.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.996484173, 2024-09-14T22:38:20.568 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:20.568 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:20.568 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:20.568 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:20.568 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:20.568 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:20.568 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 148469 2024-09-14T22:38:20.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.568 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:20.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:20.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746449+0000", 2024-09-14T22:38:20.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:20.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746449+0000", 2024-09-14T22:38:20.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:20.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746452+0000", 2024-09-14T22:38:20.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0019999999999998e-06 2024-09-14T22:38:20.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:20.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746461+0000", 2024-09-14T22:38:20.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0919999999999998e-06 2024-09-14T22:38:20.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:20.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746475+0000", 2024-09-14T22:38:20.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4853e-05 2024-09-14T22:38:20.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:20.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746484+0000", 2024-09-14T22:38:20.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3780000000000001e-06 2024-09-14T22:38:20.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:20.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746559+0000", 2024-09-14T22:38:20.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5259000000000006e-05 2024-09-14T22:38:20.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:20.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746572+0000", 2024-09-14T22:38:20.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.271e-05 2024-09-14T22:38:20.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:20.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727807+0000", 2024-09-14T22:38:20.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98123555900000003 2024-09-14T22:38:20.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:20.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727826+0000", 2024-09-14T22:38:20.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8828e-05 2024-09-14T22:38:20.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:20.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.739033+0000", 2024-09-14T22:38:20.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.011207062 2024-09-14T22:38:20.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:20.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.739479+0000", 2024-09-14T22:38:20.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044563200000000001 2024-09-14T22:38:20.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:20.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.742934+0000", 2024-09-14T22:38:20.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0034547979999999998 2024-09-14T22:38:20.573 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.573 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:20.573 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.574 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:148473 3.18 3:1f6f7244:::rbd_header.113c827b523f:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-14T22:38:20.574 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:55.755969+0000", 2024-09-14T22:38:20.574 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 24.729531570999999, 2024-09-14T22:38:20.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9872532889999999, 2024-09-14T22:38:20.574 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:20.574 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:20.574 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:20.574 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:20.574 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:20.574 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:20.574 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 148473 2024-09-14T22:38:20.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.574 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:20.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:20.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755969+0000", 2024-09-14T22:38:20.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:20.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755969+0000", 2024-09-14T22:38:20.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:20.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755971+0000", 2024-09-14T22:38:20.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.674e-06 2024-09-14T22:38:20.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:20.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755979+0000", 2024-09-14T22:38:20.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8599999999999993e-06 2024-09-14T22:38:20.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:20.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755992+0000", 2024-09-14T22:38:20.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.361e-05 2024-09-14T22:38:20.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:20.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.756001+0000", 2024-09-14T22:38:20.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.422e-06 2024-09-14T22:38:20.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:20.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.756069+0000", 2024-09-14T22:38:20.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8209999999999999e-05 2024-09-14T22:38:20.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:20.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.756077+0000", 2024-09-14T22:38:20.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.994e-06 2024-09-14T22:38:20.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:20.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727872+0000", 2024-09-14T22:38:20.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.97179484999999999 2024-09-14T22:38:20.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:20.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727885+0000", 2024-09-14T22:38:20.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3101e-05 2024-09-14T22:38:20.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:20.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.740559+0000", 2024-09-14T22:38:20.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0126736300000001 2024-09-14T22:38:20.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:20.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.740777+0000", 2024-09-14T22:38:20.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021822000000000001 2024-09-14T22:38:20.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 2", 2024-09-14T22:38:20.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.741114+0000", 2024-09-14T22:38:20.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000336793 2024-09-14T22:38:20.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:20.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.742668+0000", 2024-09-14T22:38:20.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001554061 2024-09-14T22:38:20.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:38:20.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.742947+0000", 2024-09-14T22:38:20.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027937200000000001 2024-09-14T22:38:20.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:38:20.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743030+0000", 2024-09-14T22:38:20.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3385000000000002e-05 2024-09-14T22:38:20.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:20.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743223+0000", 2024-09-14T22:38:20.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00019210700000000001 2024-09-14T22:38:20.581 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.581 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:20.581 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.581 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:148474 3.18 3:1f6f7244:::rbd_header.113c827b523f:head [watch reconnect cookie 140250011614800 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104)", 2024-09-14T22:38:20.581 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:56.728273+0000", 2024-09-14T22:38:20.582 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.757228324, 2024-09-14T22:38:20.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0180497770000001, 2024-09-14T22:38:20.582 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:20.582 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:20.582 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:20.582 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:20.582 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:20.582 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:20.582 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 148474 2024-09-14T22:38:20.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.582 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:20.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:20.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728273+0000", 2024-09-14T22:38:20.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:20.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728273+0000", 2024-09-14T22:38:20.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:20.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:20.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728274+0000", 2024-09-14T22:38:20.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7570000000000001e-06 2024-09-14T22:38:20.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:20.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728281+0000", 2024-09-14T22:38:20.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5089999999999999e-06 2024-09-14T22:38:20.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:20.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728298+0000", 2024-09-14T22:38:20.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6727000000000001e-05 2024-09-14T22:38:20.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:20.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728309+0000", 2024-09-14T22:38:20.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1021e-05 2024-09-14T22:38:20.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:20.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.730078+0000", 2024-09-14T22:38:20.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017694010000000001 2024-09-14T22:38:20.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:20.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.730096+0000", 2024-09-14T22:38:20.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8039000000000001e-05 2024-09-14T22:38:20.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:20.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.741672+0000", 2024-09-14T22:38:20.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.011575817 2024-09-14T22:38:20.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2024-09-14T22:38:20.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.741827+0000", 2024-09-14T22:38:20.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000155563 2024-09-14T22:38:20.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:20.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743299+0000", 2024-09-14T22:38:20.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0014712729999999999 2024-09-14T22:38:20.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:20.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743415+0000", 2024-09-14T22:38:20.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011594700000000001 2024-09-14T22:38:20.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:20.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:20.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:20.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.746322+0000", 2024-09-14T22:38:20.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0029077230000000001 2024-09-14T22:38:20.587 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.587 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:20.588 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.588 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:20.588 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:20.588 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:20.589 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:38:20.590 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:20.585+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:20.690 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:20.689+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:20.768 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:20.768 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:20.768 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:38:20.769 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:20.782 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:38:20.791 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:20.786+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:20.892 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:20.889+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:20.955 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:20.955 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:20.956 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:38:20.956 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:38:20.956 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:20.971 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:38:20.993 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:20.986+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:21.066 INFO:teuthology.orchestra.run.smithi001.stderr:set osd.5 primary-affinity to 0.297132 (8194722) 2024-09-14T22:38:21.067 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2024-09-14T22:38:21.094 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:21.086+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:21.159 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:21.159 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:38:21.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:38:21.160 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:38:21.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.160 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:31370 3.0 3:02f48f58:::rbd_data.113cda0183a0.0000000000000000:6 [read 813056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:38:21.160 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:38:21.160 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 107.794728616, 2024-09-14T22:38:21.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047519796000000003, 2024-09-14T22:38:21.160 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.160 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.160 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.160 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.160 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.161 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.161 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 31370 2024-09-14T22:38:21.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.161 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.161 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.161 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:38:21.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.161 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.161 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:38:21.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:38:21.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0999999999999999e-07 2024-09-14T22:38:21.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362401+0000", 2024-09-14T22:38:21.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2900000000000001e-06 2024-09-14T22:38:21.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362406+0000", 2024-09-14T22:38:21.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6960000000000002e-06 2024-09-14T22:38:21.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362412+0000", 2024-09-14T22:38:21.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.823e-06 2024-09-14T22:38:21.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.400270+0000", 2024-09-14T22:38:21.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037857737000000002 2024-09-14T22:38:21.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.400342+0000", 2024-09-14T22:38:21.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1797000000000004e-05 2024-09-14T22:38:21.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.409918+0000", 2024-09-14T22:38:21.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0095760429999999994 2024-09-14T22:38:21.164 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.164 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.164 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.164 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:113947 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 5603328~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:21.164 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.923453+0000", 2024-09-14T22:38:21.165 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.233673484000001, 2024-09-14T22:38:21.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047310420999999998, 2024-09-14T22:38:21.165 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.165 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.165 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.165 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.165 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.165 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.165 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 113947 2024-09-14T22:38:21.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.165 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.165 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.166 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.923453+0000", 2024-09-14T22:38:21.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.166 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.166 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.923453+0000", 2024-09-14T22:38:21.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.166 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.166 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926004+0000", 2024-09-14T22:38:21.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002551382 2024-09-14T22:38:21.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.166 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926009+0000", 2024-09-14T22:38:21.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5639999999999998e-06 2024-09-14T22:38:21.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926022+0000", 2024-09-14T22:38:21.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2734999999999999e-05 2024-09-14T22:38:21.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926029+0000", 2024-09-14T22:38:21.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2520000000000002e-06 2024-09-14T22:38:21.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.970650+0000", 2024-09-14T22:38:21.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044621277000000001 2024-09-14T22:38:21.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.970700+0000", 2024-09-14T22:38:21.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9713000000000003e-05 2024-09-14T22:38:21.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.970763+0000", 2024-09-14T22:38:21.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3497999999999994e-05 2024-09-14T22:38:21.169 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.169 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.169 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.169 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114136 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3448832~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:21.169 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932179+0000", 2024-09-14T22:38:21.169 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.224947118000003, 2024-09-14T22:38:21.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047236017999999998, 2024-09-14T22:38:21.170 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.170 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.170 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.170 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.170 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.170 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.170 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114136 2024-09-14T22:38:21.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.170 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.171 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.171 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932179+0000", 2024-09-14T22:38:21.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.171 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.171 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932179+0000", 2024-09-14T22:38:21.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.171 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932324+0000", 2024-09-14T22:38:21.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014451000000000001 2024-09-14T22:38:21.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932328+0000", 2024-09-14T22:38:21.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5889999999999996e-06 2024-09-14T22:38:21.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932338+0000", 2024-09-14T22:38:21.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3000000000000007e-06 2024-09-14T22:38:21.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932344+0000", 2024-09-14T22:38:21.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7150000000000001e-06 2024-09-14T22:38:21.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979233+0000", 2024-09-14T22:38:21.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046888711 2024-09-14T22:38:21.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979313+0000", 2024-09-14T22:38:21.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9487000000000007e-05 2024-09-14T22:38:21.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.174 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.174 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979415+0000", 2024-09-14T22:38:21.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000102706 2024-09-14T22:38:21.174 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.174 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.174 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.174 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114138 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3497984~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:21.174 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932356+0000", 2024-09-14T22:38:21.174 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.224770231999997, 2024-09-14T22:38:21.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047285958000000003, 2024-09-14T22:38:21.174 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.174 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.175 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.175 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.175 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.175 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.175 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114138 2024-09-14T22:38:21.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.175 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.175 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.175 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932356+0000", 2024-09-14T22:38:21.175 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932356+0000", 2024-09-14T22:38:21.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932498+0000", 2024-09-14T22:38:21.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000142182 2024-09-14T22:38:21.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932503+0000", 2024-09-14T22:38:21.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.301e-06 2024-09-14T22:38:21.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932514+0000", 2024-09-14T22:38:21.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1484000000000001e-05 2024-09-14T22:38:21.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932521+0000", 2024-09-14T22:38:21.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0740000000000004e-06 2024-09-14T22:38:21.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979451+0000", 2024-09-14T22:38:21.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046929379 2024-09-14T22:38:21.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979535+0000", 2024-09-14T22:38:21.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4423000000000001e-05 2024-09-14T22:38:21.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979642+0000", 2024-09-14T22:38:21.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010711499999999999 2024-09-14T22:38:21.179 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.179 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.179 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.179 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114139 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3547136~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:21.179 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932554+0000", 2024-09-14T22:38:21.180 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.224572465000001, 2024-09-14T22:38:21.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047292230999999997, 2024-09-14T22:38:21.180 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.180 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.180 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.180 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.180 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.180 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.180 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114139 2024-09-14T22:38:21.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.180 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932554+0000", 2024-09-14T22:38:21.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932554+0000", 2024-09-14T22:38:21.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932819+0000", 2024-09-14T22:38:21.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026538400000000002 2024-09-14T22:38:21.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932825+0000", 2024-09-14T22:38:21.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3469999999999998e-06 2024-09-14T22:38:21.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932836+0000", 2024-09-14T22:38:21.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1e-05 2024-09-14T22:38:21.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932843+0000", 2024-09-14T22:38:21.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1790000000000002e-06 2024-09-14T22:38:21.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.184 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.184 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979673+0000", 2024-09-14T22:38:21.184 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046830389 2024-09-14T22:38:21.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.184 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.184 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979750+0000", 2024-09-14T22:38:21.184 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6312999999999999e-05 2024-09-14T22:38:21.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.185 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.185 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979846+0000", 2024-09-14T22:38:21.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6619000000000004e-05 2024-09-14T22:38:21.185 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.185 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.185 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.185 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114141 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3596288~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:21.185 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932855+0000", 2024-09-14T22:38:21.185 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.224271866999999, 2024-09-14T22:38:21.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047201870999999999, 2024-09-14T22:38:21.185 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.186 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.186 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.186 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.186 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.186 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.186 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114141 2024-09-14T22:38:21.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.186 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.186 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932855+0000", 2024-09-14T22:38:21.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932855+0000", 2024-09-14T22:38:21.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934033+0000", 2024-09-14T22:38:21.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001178264 2024-09-14T22:38:21.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934038+0000", 2024-09-14T22:38:21.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4820000000000002e-06 2024-09-14T22:38:21.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934048+0000", 2024-09-14T22:38:21.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9890000000000004e-06 2024-09-14T22:38:21.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934055+0000", 2024-09-14T22:38:21.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8329999999999996e-06 2024-09-14T22:38:21.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979878+0000", 2024-09-14T22:38:21.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045822773999999997 2024-09-14T22:38:21.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979958+0000", 2024-09-14T22:38:21.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0477999999999998e-05 2024-09-14T22:38:21.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.980056+0000", 2024-09-14T22:38:21.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8051e-05 2024-09-14T22:38:21.189 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.189 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.189 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.190 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114159 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3940352~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:21.190 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.934179+0000", 2024-09-14T22:38:21.190 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.222947830999999, 2024-09-14T22:38:21.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047341439999999999, 2024-09-14T22:38:21.190 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.190 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.190 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.190 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.190 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.190 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.190 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114159 2024-09-14T22:38:21.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.191 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.191 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.191 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934179+0000", 2024-09-14T22:38:21.191 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.191 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.191 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934179+0000", 2024-09-14T22:38:21.191 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.191 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.191 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935021+0000", 2024-09-14T22:38:21.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00084276800000000003 2024-09-14T22:38:21.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.192 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935027+0000", 2024-09-14T22:38:21.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2959999999999998e-06 2024-09-14T22:38:21.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.192 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935038+0000", 2024-09-14T22:38:21.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1198e-05 2024-09-14T22:38:21.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.193 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935054+0000", 2024-09-14T22:38:21.193 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5764e-05 2024-09-14T22:38:21.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.193 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.193 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.981341+0000", 2024-09-14T22:38:21.193 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.04628782 2024-09-14T22:38:21.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.193 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.193 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.981415+0000", 2024-09-14T22:38:21.193 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3349999999999994e-05 2024-09-14T22:38:21.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.981520+0000", 2024-09-14T22:38:21.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105244 2024-09-14T22:38:21.194 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.194 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.194 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.194 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114180 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4136960~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:21.194 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:38:21.194 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.221948294999997, 2024-09-14T22:38:21.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047190621000000002, 2024-09-14T22:38:21.194 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.195 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.195 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.195 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.195 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.195 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.195 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114180 2024-09-14T22:38:21.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.195 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:38:21.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:38:21.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:38:21.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6800000000000002e-07 2024-09-14T22:38:21.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935180+0000", 2024-09-14T22:38:21.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.42e-06 2024-09-14T22:38:21.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.197 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.197 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935185+0000", 2024-09-14T22:38:21.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1359999999999996e-06 2024-09-14T22:38:21.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.197 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.197 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935188+0000", 2024-09-14T22:38:21.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.711e-06 2024-09-14T22:38:21.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.197 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.197 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982201+0000", 2024-09-14T22:38:21.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047013309000000003 2024-09-14T22:38:21.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982278+0000", 2024-09-14T22:38:21.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6705000000000005e-05 2024-09-14T22:38:21.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982369+0000", 2024-09-14T22:38:21.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1071999999999998e-05 2024-09-14T22:38:21.198 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.198 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.198 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.199 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114182 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4186112~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:21.199 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:38:21.199 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.221919673000002, 2024-09-14T22:38:21.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047368990999999999, 2024-09-14T22:38:21.199 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.199 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.199 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.199 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.199 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.199 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.199 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114182 2024-09-14T22:38:21.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.200 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:38:21.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:38:21.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:38:21.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7099999999999998e-07 2024-09-14T22:38:21.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935209+0000", 2024-09-14T22:38:21.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0260000000000002e-06 2024-09-14T22:38:21.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935213+0000", 2024-09-14T22:38:21.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3429999999999998e-06 2024-09-14T22:38:21.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935217+0000", 2024-09-14T22:38:21.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2480000000000001e-06 2024-09-14T22:38:21.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982406+0000", 2024-09-14T22:38:21.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047188920000000002 2024-09-14T22:38:21.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982483+0000", 2024-09-14T22:38:21.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7008999999999994e-05 2024-09-14T22:38:21.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982576+0000", 2024-09-14T22:38:21.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3173999999999999e-05 2024-09-14T22:38:21.203 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.203 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.203 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.203 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114185 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4202496~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:21.203 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.935221+0000", 2024-09-14T22:38:21.203 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.221905507999999, 2024-09-14T22:38:21.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047601128999999999, 2024-09-14T22:38:21.203 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.204 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.204 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.204 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.204 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.204 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.204 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114185 2024-09-14T22:38:21.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.204 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935221+0000", 2024-09-14T22:38:21.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935221+0000", 2024-09-14T22:38:21.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935970+0000", 2024-09-14T22:38:21.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074930899999999998 2024-09-14T22:38:21.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935976+0000", 2024-09-14T22:38:21.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5439999999999998e-06 2024-09-14T22:38:21.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.206 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.206 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935988+0000", 2024-09-14T22:38:21.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1779e-05 2024-09-14T22:38:21.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.206 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.206 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935996+0000", 2024-09-14T22:38:21.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.106e-06 2024-09-14T22:38:21.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.206 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.206 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982607+0000", 2024-09-14T22:38:21.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046611504999999998 2024-09-14T22:38:21.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.207 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982686+0000", 2024-09-14T22:38:21.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8805000000000002e-05 2024-09-14T22:38:21.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.207 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982822+0000", 2024-09-14T22:38:21.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000136081 2024-09-14T22:38:21.207 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.207 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.207 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.208 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114189 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4300800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:21.208 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.936048+0000", 2024-09-14T22:38:21.208 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.221078376000001, 2024-09-14T22:38:21.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047231539000000003, 2024-09-14T22:38:21.208 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.208 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.208 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.208 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.208 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.208 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.208 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114189 2024-09-14T22:38:21.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.209 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936048+0000", 2024-09-14T22:38:21.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936048+0000", 2024-09-14T22:38:21.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936049+0000", 2024-09-14T22:38:21.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4099999999999999e-07 2024-09-14T22:38:21.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936052+0000", 2024-09-14T22:38:21.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0029999999999999e-06 2024-09-14T22:38:21.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936058+0000", 2024-09-14T22:38:21.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7880000000000001e-06 2024-09-14T22:38:21.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.211 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936064+0000", 2024-09-14T22:38:21.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2279999999999998e-06 2024-09-14T22:38:21.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.211 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.211 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.211 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983120+0000", 2024-09-14T22:38:21.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047056384999999999 2024-09-14T22:38:21.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.211 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.211 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.211 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983194+0000", 2024-09-14T22:38:21.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3811000000000002e-05 2024-09-14T22:38:21.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.211 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.212 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.212 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983280+0000", 2024-09-14T22:38:21.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5883000000000004e-05 2024-09-14T22:38:21.212 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.212 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.212 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.212 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114191 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4349952~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:21.212 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:38:21.212 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.221054635999998, 2024-09-14T22:38:21.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047365365, 2024-09-14T22:38:21.212 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.212 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.213 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.213 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.213 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.213 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.213 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114191 2024-09-14T22:38:21.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.213 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:38:21.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:38:21.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:38:21.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0499999999999999e-07 2024-09-14T22:38:21.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936075+0000", 2024-09-14T22:38:21.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.734e-06 2024-09-14T22:38:21.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936082+0000", 2024-09-14T22:38:21.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8360000000000003e-06 2024-09-14T22:38:21.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936087+0000", 2024-09-14T22:38:21.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3129999999999998e-06 2024-09-14T22:38:21.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983319+0000", 2024-09-14T22:38:21.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047232140999999998 2024-09-14T22:38:21.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983365+0000", 2024-09-14T22:38:21.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5776999999999999e-05 2024-09-14T22:38:21.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983437+0000", 2024-09-14T22:38:21.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2158999999999998e-05 2024-09-14T22:38:21.216 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.216 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.216 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.216 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114193 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4399104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:21.217 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.936095+0000", 2024-09-14T22:38:21.217 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.221031699999997, 2024-09-14T22:38:21.217 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047505915000000003, 2024-09-14T22:38:21.217 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.217 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.217 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.217 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.217 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.217 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.217 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114193 2024-09-14T22:38:21.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.217 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.217 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936095+0000", 2024-09-14T22:38:21.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936095+0000", 2024-09-14T22:38:21.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938021+0000", 2024-09-14T22:38:21.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001926178 2024-09-14T22:38:21.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938027+0000", 2024-09-14T22:38:21.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3110000000000002e-06 2024-09-14T22:38:21.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938041+0000", 2024-09-14T22:38:21.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3344e-05 2024-09-14T22:38:21.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938049+0000", 2024-09-14T22:38:21.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2279999999999998e-06 2024-09-14T22:38:21.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983468+0000", 2024-09-14T22:38:21.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045419494999999997 2024-09-14T22:38:21.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983524+0000", 2024-09-14T22:38:21.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5952000000000002e-05 2024-09-14T22:38:21.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983601+0000", 2024-09-14T22:38:21.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6407000000000002e-05 2024-09-14T22:38:21.221 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.221 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.221 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.221 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114710 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4857856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:21.221 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.962571+0000", 2024-09-14T22:38:21.221 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.194555428000001, 2024-09-14T22:38:21.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047253314999999997, 2024-09-14T22:38:21.221 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.221 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.221 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.221 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.221 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.222 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.222 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114710 2024-09-14T22:38:21.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.222 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962571+0000", 2024-09-14T22:38:21.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962571+0000", 2024-09-14T22:38:21.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.223 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962572+0000", 2024-09-14T22:38:21.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1399999999999997e-07 2024-09-14T22:38:21.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.223 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962574+0000", 2024-09-14T22:38:21.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8679999999999999e-06 2024-09-14T22:38:21.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.224 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962583+0000", 2024-09-14T22:38:21.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8310000000000007e-06 2024-09-14T22:38:21.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.224 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.224 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962589+0000", 2024-09-14T22:38:21.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9529999999999996e-06 2024-09-14T22:38:21.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.224 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.224 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009692+0000", 2024-09-14T22:38:21.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047103249999999999 2024-09-14T22:38:21.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009767+0000", 2024-09-14T22:38:21.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4672000000000006e-05 2024-09-14T22:38:21.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009824+0000", 2024-09-14T22:38:21.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7226999999999999e-05 2024-09-14T22:38:21.225 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.225 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.225 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.226 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114712 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4907008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:21.226 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:38:21.226 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.194529948000003, 2024-09-14T22:38:21.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047421087000000001, 2024-09-14T22:38:21.226 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.226 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.226 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.226 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.226 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.226 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.226 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114712 2024-09-14T22:38:21.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.226 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:38:21.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:38:21.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:38:21.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.03e-07 2024-09-14T22:38:21.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.228 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962599+0000", 2024-09-14T22:38:21.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3199999999999998e-06 2024-09-14T22:38:21.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.228 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962606+0000", 2024-09-14T22:38:21.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5289999999999997e-06 2024-09-14T22:38:21.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.228 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962612+0000", 2024-09-14T22:38:21.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.947e-06 2024-09-14T22:38:21.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009866+0000", 2024-09-14T22:38:21.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047254681999999999 2024-09-14T22:38:21.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009924+0000", 2024-09-14T22:38:21.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7556000000000003e-05 2024-09-14T22:38:21.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010018+0000", 2024-09-14T22:38:21.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.365e-05 2024-09-14T22:38:21.230 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.230 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.230 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.230 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114714 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4956160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:21.230 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.962618+0000", 2024-09-14T22:38:21.230 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.194508429999999, 2024-09-14T22:38:21.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.04759302, 2024-09-14T22:38:21.230 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.230 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.230 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.230 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.230 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.230 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.230 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114714 2024-09-14T22:38:21.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.231 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.231 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962618+0000", 2024-09-14T22:38:21.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.231 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962618+0000", 2024-09-14T22:38:21.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966011+0000", 2024-09-14T22:38:21.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003393067 2024-09-14T22:38:21.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966016+0000", 2024-09-14T22:38:21.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3580000000000002e-06 2024-09-14T22:38:21.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966032+0000", 2024-09-14T22:38:21.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5356e-05 2024-09-14T22:38:21.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.233 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966040+0000", 2024-09-14T22:38:21.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1980000000000001e-06 2024-09-14T22:38:21.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.233 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010055+0000", 2024-09-14T22:38:21.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044014676000000003 2024-09-14T22:38:21.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010127+0000", 2024-09-14T22:38:21.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2206000000000006e-05 2024-09-14T22:38:21.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010211+0000", 2024-09-14T22:38:21.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4159000000000005e-05 2024-09-14T22:38:21.234 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.234 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.234 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.234 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114754 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 6283264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:21.235 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:38:21.235 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.190333516999999, 2024-09-14T22:38:21.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047228150000000003, 2024-09-14T22:38:21.235 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.235 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.235 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.235 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.235 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.235 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.235 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114754 2024-09-14T22:38:21.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.235 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.236 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:38:21.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.236 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:38:21.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.236 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:38:21.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7399999999999999e-07 2024-09-14T22:38:21.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966796+0000", 2024-09-14T22:38:21.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3630000000000001e-06 2024-09-14T22:38:21.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966802+0000", 2024-09-14T22:38:21.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4679999999999997e-06 2024-09-14T22:38:21.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966807+0000", 2024-09-14T22:38:21.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1109999999999999e-06 2024-09-14T22:38:21.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.013929+0000", 2024-09-14T22:38:21.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047121720999999998 2024-09-14T22:38:21.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.013968+0000", 2024-09-14T22:38:21.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8877000000000001e-05 2024-09-14T22:38:21.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014021+0000", 2024-09-14T22:38:21.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3236000000000003e-05 2024-09-14T22:38:21.239 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.239 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.239 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.239 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114789 3.19 3:9836bb25:::rbd_object_map.113c14c67864.000000000000001b:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:21.239 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.966813+0000", 2024-09-14T22:38:21.239 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.190313054999997, 2024-09-14T22:38:21.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05001311, 2024-09-14T22:38:21.239 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.239 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.240 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:21.240 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.240 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.240 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.240 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114789 2024-09-14T22:38:21.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.240 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966813+0000", 2024-09-14T22:38:21.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966813+0000", 2024-09-14T22:38:21.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966814+0000", 2024-09-14T22:38:21.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9200000000000002e-07 2024-09-14T22:38:21.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966817+0000", 2024-09-14T22:38:21.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8380000000000002e-06 2024-09-14T22:38:21.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966831+0000", 2024-09-14T22:38:21.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4209999999999999e-05 2024-09-14T22:38:21.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966845+0000", 2024-09-14T22:38:21.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3799e-05 2024-09-14T22:38:21.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014046+0000", 2024-09-14T22:38:21.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047201604000000001 2024-09-14T22:38:21.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.243 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.243 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014116+0000", 2024-09-14T22:38:21.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9387000000000005e-05 2024-09-14T22:38:21.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.243 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-14T22:38:21.243 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014469+0000", 2024-09-14T22:38:21.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00035355800000000002 2024-09-14T22:38:21.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.243 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:21.243 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016719+0000", 2024-09-14T22:38:21.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0022498459999999998 2024-09-14T22:38:21.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:38:21.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016735+0000", 2024-09-14T22:38:21.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5792999999999999e-05 2024-09-14T22:38:21.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:38:21.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016790+0000", 2024-09-14T22:38:21.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4715e-05 2024-09-14T22:38:21.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016827+0000", 2024-09-14T22:38:21.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6968000000000002e-05 2024-09-14T22:38:21.245 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.245 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.245 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.245 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114803 3.19 3:9836bb25:::rbd_object_map.113c14c67864.000000000000001b:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:21.245 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.966853+0000", 2024-09-14T22:38:21.245 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.190273625000003, 2024-09-14T22:38:21.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050656093999999999, 2024-09-14T22:38:21.245 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.245 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.246 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:21.246 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.246 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.246 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.246 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114803 2024-09-14T22:38:21.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.246 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.246 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.246 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966853+0000", 2024-09-14T22:38:21.246 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.247 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966853+0000", 2024-09-14T22:38:21.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.247 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969930+0000", 2024-09-14T22:38:21.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030769759999999999 2024-09-14T22:38:21.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.247 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969936+0000", 2024-09-14T22:38:21.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7150000000000003e-06 2024-09-14T22:38:21.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969949+0000", 2024-09-14T22:38:21.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3166000000000001e-05 2024-09-14T22:38:21.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969957+0000", 2024-09-14T22:38:21.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8269999999999998e-06 2024-09-14T22:38:21.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014996+0000", 2024-09-14T22:38:21.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045039202 2024-09-14T22:38:21.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.015085+0000", 2024-09-14T22:38:21.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8987999999999996e-05 2024-09-14T22:38:21.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-14T22:38:21.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.015318+0000", 2024-09-14T22:38:21.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023284999999999999 2024-09-14T22:38:21.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.250 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:21.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016898+0000", 2024-09-14T22:38:21.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0015807379999999999 2024-09-14T22:38:21.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.250 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:38:21.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.017455+0000", 2024-09-14T22:38:21.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00055664600000000005 2024-09-14T22:38:21.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.250 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:38:21.251 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.017483+0000", 2024-09-14T22:38:21.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8238e-05 2024-09-14T22:38:21.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.251 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.251 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.017509+0000", 2024-09-14T22:38:21.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5748000000000001e-05 2024-09-14T22:38:21.251 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.251 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.251 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.252 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:150759 3.8 3:15dc94e5:::rbd_header.113c88f88d4a:head [create,call rbd.create in=9b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e113)", 2024-09-14T22:38:21.252 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:38:05.726302+0000", 2024-09-14T22:38:21.252 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.430824933, 2024-09-14T22:38:21.252 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0241160819999999, 2024-09-14T22:38:21.252 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.252 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.252 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:21.252 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.252 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.252 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.253 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 150759 2024-09-14T22:38:21.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.253 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.726302+0000", 2024-09-14T22:38:21.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.726302+0000", 2024-09-14T22:38:21.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.726304+0000", 2024-09-14T22:38:21.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.075e-06 2024-09-14T22:38:21.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.726312+0000", 2024-09-14T22:38:21.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2050000000000002e-06 2024-09-14T22:38:21.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.727969+0000", 2024-09-14T22:38:21.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001657499 2024-09-14T22:38:21.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.255 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.727985+0000", 2024-09-14T22:38:21.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6082999999999998e-05 2024-09-14T22:38:21.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.255 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.728193+0000", 2024-09-14T22:38:21.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000208082 2024-09-14T22:38:21.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:21.255 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.728205+0000", 2024-09-14T22:38:21.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1897e-05 2024-09-14T22:38:21.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.741926+0000", 2024-09-14T22:38:21.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.013720591 2024-09-14T22:38:21.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.742076+0000", 2024-09-14T22:38:21.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015024099999999999 2024-09-14T22:38:21.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-14T22:38:21.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.742415+0000", 2024-09-14T22:38:21.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00033887700000000002 2024-09-14T22:38:21.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:38:21.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.743436+0000", 2024-09-14T22:38:21.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010210639999999999 2024-09-14T22:38:21.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:21.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.750216+0000", 2024-09-14T22:38:21.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0067798950000000002 2024-09-14T22:38:21.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:38:21.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.750288+0000", 2024-09-14T22:38:21.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2192000000000003e-05 2024-09-14T22:38:21.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.750418+0000", 2024-09-14T22:38:21.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000129381 2024-09-14T22:38:21.258 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.258 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.258 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.258 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.258 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.258 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:21.260 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:38:21.260 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:21.257+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:21.360 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:21.357+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:21.434 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:21.434 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:21.434 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:38:21.434 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:21.448 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:38:21.490 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:21.489+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:21.561 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:21.558+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:21.591 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:38:21.609 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:21.609 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:21.609 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:38:21.609 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:38:21.609 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:21.624 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:38:21.663 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:21.658+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:21.763 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:38:21.763 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:38:21.763 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:21.758+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:21.799 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:38:21.799 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:38:21.834 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:21.834 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:38:21.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:38:21.835 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:38:21.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.835 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145249 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 15745024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:21.835 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.693850+0000", 2024-09-14T22:38:21.835 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.137299897999998, 2024-09-14T22:38:21.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.060850297999999997, 2024-09-14T22:38:21.835 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.835 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.835 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.835 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.835 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.835 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.835 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145249 2024-09-14T22:38:21.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.836 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693850+0000", 2024-09-14T22:38:21.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693850+0000", 2024-09-14T22:38:21.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693851+0000", 2024-09-14T22:38:21.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.46e-07 2024-09-14T22:38:21.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693853+0000", 2024-09-14T22:38:21.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0940000000000002e-06 2024-09-14T22:38:21.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693858+0000", 2024-09-14T22:38:21.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2100000000000001e-06 2024-09-14T22:38:21.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693863+0000", 2024-09-14T22:38:21.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5549999999999996e-06 2024-09-14T22:38:21.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754503+0000", 2024-09-14T22:38:21.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.060640558999999997 2024-09-14T22:38:21.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754613+0000", 2024-09-14T22:38:21.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109966 2024-09-14T22:38:21.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754701+0000", 2024-09-14T22:38:21.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7567999999999999e-05 2024-09-14T22:38:21.839 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.839 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.839 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.839 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145250 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16039936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:21.839 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.693868+0000", 2024-09-14T22:38:21.840 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.137282169999999, 2024-09-14T22:38:21.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061182667000000003, 2024-09-14T22:38:21.840 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.840 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.840 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.840 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.840 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.840 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.840 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145250 2024-09-14T22:38:21.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.841 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693868+0000", 2024-09-14T22:38:21.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693868+0000", 2024-09-14T22:38:21.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693869+0000", 2024-09-14T22:38:21.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4400000000000001e-07 2024-09-14T22:38:21.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693870+0000", 2024-09-14T22:38:21.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9130000000000001e-06 2024-09-14T22:38:21.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693876+0000", 2024-09-14T22:38:21.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1270000000000002e-06 2024-09-14T22:38:21.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693880+0000", 2024-09-14T22:38:21.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.673e-06 2024-09-14T22:38:21.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754780+0000", 2024-09-14T22:38:21.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.060899495999999997 2024-09-14T22:38:21.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754925+0000", 2024-09-14T22:38:21.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014511799999999999 2024-09-14T22:38:21.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755051+0000", 2024-09-14T22:38:21.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012599599999999999 2024-09-14T22:38:21.844 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.844 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.844 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.845 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145251 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16236544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:21.845 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.693886+0000", 2024-09-14T22:38:21.845 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.137264516999998, 2024-09-14T22:38:21.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061498816999999997, 2024-09-14T22:38:21.845 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.845 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.845 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.845 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.845 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.845 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.845 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145251 2024-09-14T22:38:21.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.846 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693886+0000", 2024-09-14T22:38:21.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693886+0000", 2024-09-14T22:38:21.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693886+0000", 2024-09-14T22:38:21.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1199999999999998e-07 2024-09-14T22:38:21.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693888+0000", 2024-09-14T22:38:21.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.877e-06 2024-09-14T22:38:21.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693894+0000", 2024-09-14T22:38:21.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5350000000000004e-06 2024-09-14T22:38:21.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693917+0000", 2024-09-14T22:38:21.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.296e-05 2024-09-14T22:38:21.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755127+0000", 2024-09-14T22:38:21.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061210286000000003 2024-09-14T22:38:21.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755266+0000", 2024-09-14T22:38:21.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013871599999999999 2024-09-14T22:38:21.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755385+0000", 2024-09-14T22:38:21.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011903100000000001 2024-09-14T22:38:21.850 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.850 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.850 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.850 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145252 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16302080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:21.850 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.693985+0000", 2024-09-14T22:38:21.850 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.137165835000001, 2024-09-14T22:38:21.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061743767999999997, 2024-09-14T22:38:21.850 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.850 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.850 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.851 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.851 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.851 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.851 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145252 2024-09-14T22:38:21.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.851 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693985+0000", 2024-09-14T22:38:21.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693985+0000", 2024-09-14T22:38:21.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693986+0000", 2024-09-14T22:38:21.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0640000000000001e-06 2024-09-14T22:38:21.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693991+0000", 2024-09-14T22:38:21.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7599999999999999e-06 2024-09-14T22:38:21.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694005+0000", 2024-09-14T22:38:21.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3271999999999999e-05 2024-09-14T22:38:21.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694015+0000", 2024-09-14T22:38:21.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0617e-05 2024-09-14T22:38:21.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755459+0000", 2024-09-14T22:38:21.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061443820000000003 2024-09-14T22:38:21.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755605+0000", 2024-09-14T22:38:21.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014569 2024-09-14T22:38:21.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755728+0000", 2024-09-14T22:38:21.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000123545 2024-09-14T22:38:21.855 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.855 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.855 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.856 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145253 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16138240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:21.856 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694031+0000", 2024-09-14T22:38:21.856 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.137119281, 2024-09-14T22:38:21.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062032836000000001, 2024-09-14T22:38:21.856 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.856 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.856 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.856 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.856 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.856 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.857 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145253 2024-09-14T22:38:21.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.857 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694031+0000", 2024-09-14T22:38:21.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694031+0000", 2024-09-14T22:38:21.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694032+0000", 2024-09-14T22:38:21.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7999999999999995e-07 2024-09-14T22:38:21.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694036+0000", 2024-09-14T22:38:21.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8800000000000001e-06 2024-09-14T22:38:21.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694047+0000", 2024-09-14T22:38:21.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0835999999999999e-05 2024-09-14T22:38:21.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694055+0000", 2024-09-14T22:38:21.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6619999999999994e-06 2024-09-14T22:38:21.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755804+0000", 2024-09-14T22:38:21.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061748448999999997 2024-09-14T22:38:21.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755941+0000", 2024-09-14T22:38:21.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013751599999999999 2024-09-14T22:38:21.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756064+0000", 2024-09-14T22:38:21.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012281299999999999 2024-09-14T22:38:21.860 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.860 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.860 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.861 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145254 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16400384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:21.861 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694067+0000", 2024-09-14T22:38:21.861 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.137083115999999, 2024-09-14T22:38:21.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062280549999999997, 2024-09-14T22:38:21.861 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.861 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.861 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.861 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.861 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.861 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.861 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145254 2024-09-14T22:38:21.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.862 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694067+0000", 2024-09-14T22:38:21.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694067+0000", 2024-09-14T22:38:21.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694068+0000", 2024-09-14T22:38:21.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2200000000000003e-07 2024-09-14T22:38:21.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694072+0000", 2024-09-14T22:38:21.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5729999999999999e-06 2024-09-14T22:38:21.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694082+0000", 2024-09-14T22:38:21.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0703999999999999e-05 2024-09-14T22:38:21.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694091+0000", 2024-09-14T22:38:21.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8030000000000003e-06 2024-09-14T22:38:21.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756155+0000", 2024-09-14T22:38:21.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062063937 2024-09-14T22:38:21.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756263+0000", 2024-09-14T22:38:21.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108229 2024-09-14T22:38:21.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756348+0000", 2024-09-14T22:38:21.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4581999999999998e-05 2024-09-14T22:38:21.865 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.865 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.865 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.866 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145255 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16596992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:21.866 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694103+0000", 2024-09-14T22:38:21.866 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.137047634999998, 2024-09-14T22:38:21.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062580620000000003, 2024-09-14T22:38:21.866 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.866 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.866 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.866 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.866 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.866 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.866 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145255 2024-09-14T22:38:21.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.867 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694103+0000", 2024-09-14T22:38:21.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694103+0000", 2024-09-14T22:38:21.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694103+0000", 2024-09-14T22:38:21.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7800000000000001e-07 2024-09-14T22:38:21.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694107+0000", 2024-09-14T22:38:21.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5939999999999998e-06 2024-09-14T22:38:21.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694121+0000", 2024-09-14T22:38:21.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3913999999999999e-05 2024-09-14T22:38:21.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694130+0000", 2024-09-14T22:38:21.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7690000000000002e-06 2024-09-14T22:38:21.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756431+0000", 2024-09-14T22:38:21.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062301593000000002 2024-09-14T22:38:21.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756564+0000", 2024-09-14T22:38:21.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000132693 2024-09-14T22:38:21.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756683+0000", 2024-09-14T22:38:21.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119479 2024-09-14T22:38:21.870 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.870 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.870 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.871 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145256 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16695296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:21.871 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694142+0000", 2024-09-14T22:38:21.871 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.137008561999998, 2024-09-14T22:38:21.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062806034999999996, 2024-09-14T22:38:21.871 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.871 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.871 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.871 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.871 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.871 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.871 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145256 2024-09-14T22:38:21.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.872 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694142+0000", 2024-09-14T22:38:21.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694142+0000", 2024-09-14T22:38:21.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694142+0000", 2024-09-14T22:38:21.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1699999999999998e-07 2024-09-14T22:38:21.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694146+0000", 2024-09-14T22:38:21.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7450000000000001e-06 2024-09-14T22:38:21.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694157+0000", 2024-09-14T22:38:21.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1161e-05 2024-09-14T22:38:21.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694166+0000", 2024-09-14T22:38:21.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0450000000000006e-06 2024-09-14T22:38:21.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756768+0000", 2024-09-14T22:38:21.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062601554000000004 2024-09-14T22:38:21.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756867+0000", 2024-09-14T22:38:21.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8741999999999998e-05 2024-09-14T22:38:21.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756948+0000", 2024-09-14T22:38:21.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1170999999999999e-05 2024-09-14T22:38:21.875 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.875 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.876 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.876 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145257 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16498688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:21.876 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694179+0000", 2024-09-14T22:38:21.876 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.136971263, 2024-09-14T22:38:21.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063118469999999996, 2024-09-14T22:38:21.876 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.876 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.876 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.876 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.876 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.876 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.876 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145257 2024-09-14T22:38:21.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.877 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694179+0000", 2024-09-14T22:38:21.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694179+0000", 2024-09-14T22:38:21.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694180+0000", 2024-09-14T22:38:21.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1099999999999995e-07 2024-09-14T22:38:21.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694196+0000", 2024-09-14T22:38:21.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5996000000000001e-05 2024-09-14T22:38:21.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694208+0000", 2024-09-14T22:38:21.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2584e-05 2024-09-14T22:38:21.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694219+0000", 2024-09-14T22:38:21.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0462000000000001e-05 2024-09-14T22:38:21.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757033+0000", 2024-09-14T22:38:21.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062814485000000003 2024-09-14T22:38:21.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757183+0000", 2024-09-14T22:38:21.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014914099999999999 2024-09-14T22:38:21.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757298+0000", 2024-09-14T22:38:21.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115091 2024-09-14T22:38:21.880 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.880 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.880 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.880 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145258 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16269312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:21.880 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694257+0000", 2024-09-14T22:38:21.880 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.136893068999999, 2024-09-14T22:38:21.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063297279999999997, 2024-09-14T22:38:21.881 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.881 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.881 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.881 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.881 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.881 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.881 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145258 2024-09-14T22:38:21.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.881 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694257+0000", 2024-09-14T22:38:21.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694257+0000", 2024-09-14T22:38:21.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694258+0000", 2024-09-14T22:38:21.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7700000000000004e-07 2024-09-14T22:38:21.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694263+0000", 2024-09-14T22:38:21.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6609999999999999e-06 2024-09-14T22:38:21.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694271+0000", 2024-09-14T22:38:21.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7809999999999994e-06 2024-09-14T22:38:21.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694277+0000", 2024-09-14T22:38:21.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.823e-06 2024-09-14T22:38:21.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757377+0000", 2024-09-14T22:38:21.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063100081000000002 2024-09-14T22:38:21.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757476+0000", 2024-09-14T22:38:21.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8507999999999999e-05 2024-09-14T22:38:21.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757555+0000", 2024-09-14T22:38:21.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8749000000000001e-05 2024-09-14T22:38:21.884 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.885 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.885 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.885 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145259 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16465920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:21.885 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694286+0000", 2024-09-14T22:38:21.885 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.136864872, 2024-09-14T22:38:21.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063608168000000007, 2024-09-14T22:38:21.885 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.885 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.885 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.885 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.885 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.886 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.886 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145259 2024-09-14T22:38:21.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.886 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.886 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694286+0000", 2024-09-14T22:38:21.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.886 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694286+0000", 2024-09-14T22:38:21.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694286+0000", 2024-09-14T22:38:21.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9999999999999998e-07 2024-09-14T22:38:21.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694288+0000", 2024-09-14T22:38:21.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3760000000000002e-06 2024-09-14T22:38:21.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694294+0000", 2024-09-14T22:38:21.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6779999999999997e-06 2024-09-14T22:38:21.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694298+0000", 2024-09-14T22:38:21.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3309999999999997e-06 2024-09-14T22:38:21.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757638+0000", 2024-09-14T22:38:21.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063339464999999998 2024-09-14T22:38:21.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757769+0000", 2024-09-14T22:38:21.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013148700000000001 2024-09-14T22:38:21.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757894+0000", 2024-09-14T22:38:21.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012443099999999999 2024-09-14T22:38:21.889 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.889 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.889 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.889 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145260 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16367616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:21.889 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694304+0000", 2024-09-14T22:38:21.890 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.136846078000001, 2024-09-14T22:38:21.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063838521999999995, 2024-09-14T22:38:21.890 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.890 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.890 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.890 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.890 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.890 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.890 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145260 2024-09-14T22:38:21.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.890 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694304+0000", 2024-09-14T22:38:21.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694304+0000", 2024-09-14T22:38:21.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694305+0000", 2024-09-14T22:38:21.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8299999999999997e-07 2024-09-14T22:38:21.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694307+0000", 2024-09-14T22:38:21.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9989999999999998e-06 2024-09-14T22:38:21.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694312+0000", 2024-09-14T22:38:21.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2689999999999999e-06 2024-09-14T22:38:21.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694317+0000", 2024-09-14T22:38:21.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9359999999999998e-06 2024-09-14T22:38:21.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757944+0000", 2024-09-14T22:38:21.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063626951000000001 2024-09-14T22:38:21.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758065+0000", 2024-09-14T22:38:21.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120945 2024-09-14T22:38:21.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758143+0000", 2024-09-14T22:38:21.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7838999999999998e-05 2024-09-14T22:38:21.894 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.894 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.894 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.894 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145261 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16564224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:21.894 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694323+0000", 2024-09-14T22:38:21.894 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.136827374999999, 2024-09-14T22:38:21.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064150180000000001, 2024-09-14T22:38:21.894 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.894 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.894 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.894 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.894 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.895 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.895 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145261 2024-09-14T22:38:21.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.895 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694323+0000", 2024-09-14T22:38:21.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694323+0000", 2024-09-14T22:38:21.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694323+0000", 2024-09-14T22:38:21.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4900000000000001e-07 2024-09-14T22:38:21.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694325+0000", 2024-09-14T22:38:21.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.04e-06 2024-09-14T22:38:21.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694331+0000", 2024-09-14T22:38:21.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3449999999999996e-06 2024-09-14T22:38:21.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694335+0000", 2024-09-14T22:38:21.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.566e-06 2024-09-14T22:38:21.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758225+0000", 2024-09-14T22:38:21.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063889535999999997 2024-09-14T22:38:21.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758357+0000", 2024-09-14T22:38:21.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013181 2024-09-14T22:38:21.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758473+0000", 2024-09-14T22:38:21.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011653400000000001 2024-09-14T22:38:21.898 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.898 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.898 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.898 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145262 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16760832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:21.898 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694341+0000", 2024-09-14T22:38:21.899 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.136809517, 2024-09-14T22:38:21.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064424631999999996, 2024-09-14T22:38:21.899 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.899 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.899 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.899 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.899 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.899 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.899 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145262 2024-09-14T22:38:21.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.899 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694341+0000", 2024-09-14T22:38:21.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694341+0000", 2024-09-14T22:38:21.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694341+0000", 2024-09-14T22:38:21.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5499999999999999e-07 2024-09-14T22:38:21.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694343+0000", 2024-09-14T22:38:21.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9630000000000001e-06 2024-09-14T22:38:21.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694349+0000", 2024-09-14T22:38:21.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3120000000000001e-06 2024-09-14T22:38:21.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694353+0000", 2024-09-14T22:38:21.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6500000000000004e-06 2024-09-14T22:38:21.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758561+0000", 2024-09-14T22:38:21.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064207934999999994 2024-09-14T22:38:21.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758677+0000", 2024-09-14T22:38:21.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116241 2024-09-14T22:38:21.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758766+0000", 2024-09-14T22:38:21.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8176000000000004e-05 2024-09-14T22:38:21.903 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.903 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.903 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.903 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145263 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16662528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:21.903 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694359+0000", 2024-09-14T22:38:21.903 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.136791674000001, 2024-09-14T22:38:21.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064743189000000007, 2024-09-14T22:38:21.903 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.903 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.904 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.904 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.904 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.904 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.904 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145263 2024-09-14T22:38:21.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.904 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.904 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.904 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694359+0000", 2024-09-14T22:38:21.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694359+0000", 2024-09-14T22:38:21.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694359+0000", 2024-09-14T22:38:21.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5100000000000001e-07 2024-09-14T22:38:21.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694361+0000", 2024-09-14T22:38:21.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9429999999999999e-06 2024-09-14T22:38:21.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694366+0000", 2024-09-14T22:38:21.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3489999999999999e-06 2024-09-14T22:38:21.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694371+0000", 2024-09-14T22:38:21.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4399999999999998e-06 2024-09-14T22:38:21.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758848+0000", 2024-09-14T22:38:21.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064477396000000006 2024-09-14T22:38:21.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758980+0000", 2024-09-14T22:38:21.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013162199999999999 2024-09-14T22:38:21.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759102+0000", 2024-09-14T22:38:21.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012208799999999999 2024-09-14T22:38:21.910 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.910 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.910 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.911 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145264 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16334848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:21.911 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694377+0000", 2024-09-14T22:38:21.911 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.136773640000001, 2024-09-14T22:38:21.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065048054999999994, 2024-09-14T22:38:21.911 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.911 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.911 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.911 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.911 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.911 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.911 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145264 2024-09-14T22:38:21.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.911 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694377+0000", 2024-09-14T22:38:21.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694377+0000", 2024-09-14T22:38:21.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694377+0000", 2024-09-14T22:38:21.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.39e-07 2024-09-14T22:38:21.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694379+0000", 2024-09-14T22:38:21.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0190000000000001e-06 2024-09-14T22:38:21.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694385+0000", 2024-09-14T22:38:21.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4399999999999996e-06 2024-09-14T22:38:21.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694389+0000", 2024-09-14T22:38:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.639e-06 2024-09-14T22:38:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759175+0000", 2024-09-14T22:38:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064785899999999993 2024-09-14T22:38:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759309+0000", 2024-09-14T22:38:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013374400000000001 2024-09-14T22:38:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759425+0000", 2024-09-14T22:38:21.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115974 2024-09-14T22:38:21.915 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.915 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.915 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.915 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145265 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16531456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:21.915 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694395+0000", 2024-09-14T22:38:21.915 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.136755518000001, 2024-09-14T22:38:21.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065298325000000004, 2024-09-14T22:38:21.915 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.916 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.916 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.916 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.916 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.916 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.916 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145265 2024-09-14T22:38:21.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.916 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694395+0000", 2024-09-14T22:38:21.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694395+0000", 2024-09-14T22:38:21.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694395+0000", 2024-09-14T22:38:21.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6699999999999999e-07 2024-09-14T22:38:21.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694397+0000", 2024-09-14T22:38:21.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.984e-06 2024-09-14T22:38:21.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694403+0000", 2024-09-14T22:38:21.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3519999999999997e-06 2024-09-14T22:38:21.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694407+0000", 2024-09-14T22:38:21.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3239999999999996e-06 2024-09-14T22:38:21.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759516+0000", 2024-09-14T22:38:21.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065109279000000006 2024-09-14T22:38:21.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759610+0000", 2024-09-14T22:38:21.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3566000000000005e-05 2024-09-14T22:38:21.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759693+0000", 2024-09-14T22:38:21.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3453000000000002e-05 2024-09-14T22:38:21.919 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.919 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.920 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.920 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145266 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16433152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:21.920 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694413+0000", 2024-09-14T22:38:21.920 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.136737718999999, 2024-09-14T22:38:21.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065618308, 2024-09-14T22:38:21.920 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.920 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.920 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.920 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.920 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.920 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.921 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145266 2024-09-14T22:38:21.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.921 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694413+0000", 2024-09-14T22:38:21.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694413+0000", 2024-09-14T22:38:21.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694413+0000", 2024-09-14T22:38:21.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4799999999999999e-07 2024-09-14T22:38:21.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694415+0000", 2024-09-14T22:38:21.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.08e-06 2024-09-14T22:38:21.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694420+0000", 2024-09-14T22:38:21.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.203e-06 2024-09-14T22:38:21.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694425+0000", 2024-09-14T22:38:21.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4649999999999996e-06 2024-09-14T22:38:21.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759768+0000", 2024-09-14T22:38:21.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065343613999999994 2024-09-14T22:38:21.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759909+0000", 2024-09-14T22:38:21.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014054099999999999 2024-09-14T22:38:21.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760031+0000", 2024-09-14T22:38:21.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122057 2024-09-14T22:38:21.924 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.924 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.924 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.924 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145267 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16629760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:21.925 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694431+0000", 2024-09-14T22:38:21.925 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.136719804999998, 2024-09-14T22:38:21.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065850237000000006, 2024-09-14T22:38:21.925 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.925 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.925 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.925 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.925 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.925 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.925 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145267 2024-09-14T22:38:21.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.925 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694431+0000", 2024-09-14T22:38:21.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694431+0000", 2024-09-14T22:38:21.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694431+0000", 2024-09-14T22:38:21.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4799999999999999e-07 2024-09-14T22:38:21.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694433+0000", 2024-09-14T22:38:21.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8470000000000001e-06 2024-09-14T22:38:21.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694438+0000", 2024-09-14T22:38:21.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3249999999999998e-06 2024-09-14T22:38:21.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694442+0000", 2024-09-14T22:38:21.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3490000000000002e-06 2024-09-14T22:38:21.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760113+0000", 2024-09-14T22:38:21.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065670872000000005 2024-09-14T22:38:21.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760199+0000", 2024-09-14T22:38:21.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5982000000000004e-05 2024-09-14T22:38:21.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760281+0000", 2024-09-14T22:38:21.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1514000000000006e-05 2024-09-14T22:38:21.928 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.928 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.929 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.929 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145268 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16728064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:21.929 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694448+0000", 2024-09-14T22:38:21.929 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.136702221, 2024-09-14T22:38:21.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066157585000000005, 2024-09-14T22:38:21.929 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:21.929 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:21.929 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:21.929 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:21.929 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:21.929 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:21.929 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145268 2024-09-14T22:38:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694448+0000", 2024-09-14T22:38:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694448+0000", 2024-09-14T22:38:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694449+0000", 2024-09-14T22:38:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.39e-07 2024-09-14T22:38:21.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:21.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694450+0000", 2024-09-14T22:38:21.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.906e-06 2024-09-14T22:38:21.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:21.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694456+0000", 2024-09-14T22:38:21.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3820000000000003e-06 2024-09-14T22:38:21.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:21.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694460+0000", 2024-09-14T22:38:21.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6290000000000001e-06 2024-09-14T22:38:21.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:21.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760345+0000", 2024-09-14T22:38:21.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065884540000000005 2024-09-14T22:38:21.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:21.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760482+0000", 2024-09-14T22:38:21.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013731600000000001 2024-09-14T22:38:21.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:21.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:21.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:21.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760606+0000", 2024-09-14T22:38:21.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000123473 2024-09-14T22:38:21.933 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.933 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.933 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.933 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:21.933 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:21.933 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:21.933 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:38:21.934 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:38:21.934 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:38:21.935 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:38:21.935 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:38:21.936 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:38:21.936 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:38:21.936 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:38:21.937 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:38:21.937 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:38:21.937 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:38:21.938 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:38:21.938 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:38:21.938 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:21.930+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:22.040 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:22.037+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:22.068 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:22.068 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:22.087 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:38:22.140 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:22.134+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:22.220 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:22.221 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:22.236 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:38:22.241 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:22.241+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:22.343 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:22.338+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:22.399 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:22.399 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:22.412 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:38:22.455 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:22.450+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:22.545 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:22.538+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:22.579 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:22.579 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:22.593 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:38:22.679 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:22.674+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:22.746 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:22.745+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:22.761 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:22.761 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:22.777 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:38:22.863 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:22.858+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:22.946 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:22.946 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:22.948 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:22.945+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:22.962 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:38:23.051 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:23.046+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:23.099 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T22:38:23.141 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:23.141 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:23.150 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:23.149+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:23.157 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:38:23.250 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:23.249+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:23.333 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:23.333 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:23.348 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:38:23.351 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:23.349+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:23.451 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:23.446+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:23.467 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:23.467 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:23.481 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:38:23.571 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:23.566+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:23.646 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:23.646 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:23.653 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:23.646+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:23.660 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:38:23.753 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:23.746+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:23.826 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:23.827 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:23.841 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:38:23.854 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:23.853+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:23.954 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:38:23.955 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:23.953+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:24.007 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:24.007 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:24.021 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:38:24.066 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:24.065+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:24.086 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2024-09-14T22:38:24.124 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:38:24.124 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:38:24.143 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:38:24.143 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:38:24.156 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:24.153+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:24.160 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:24.160 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:24.164 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:38:24.164 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:38:24.177 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:38:24.182 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:38:24.182 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:38:24.209 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:38:24.210 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:38:24.233 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:38:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:38:24.258 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:38:24.258 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:38:24.258 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:24.250+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:24.281 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:38:24.282 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:38:24.308 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:24.308 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:24.308 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:38:24.308 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:38:24.308 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:24.323 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:38:24.359 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:24.350+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:24.459 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:24.454+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:24.494 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:24.494 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:38:24.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:38:24.494 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:38:24.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.494 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144987 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3424256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:24.494 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549297+0000", 2024-09-14T22:38:24.494 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.941690980000001, 2024-09-14T22:38:24.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.079931556000000001, 2024-09-14T22:38:24.494 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:24.495 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:24.495 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:24.495 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:24.495 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:24.495 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:24.495 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144987 2024-09-14T22:38:24.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.495 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:24.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:24.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549297+0000", 2024-09-14T22:38:24.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.496 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:24.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549297+0000", 2024-09-14T22:38:24.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.496 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:24.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549298+0000", 2024-09-14T22:38:24.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2699999999999999e-07 2024-09-14T22:38:24.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.496 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:24.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549301+0000", 2024-09-14T22:38:24.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2959999999999999e-06 2024-09-14T22:38:24.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:24.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549314+0000", 2024-09-14T22:38:24.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2588999999999999e-05 2024-09-14T22:38:24.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:24.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549323+0000", 2024-09-14T22:38:24.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.927e-06 2024-09-14T22:38:24.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:24.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629038+0000", 2024-09-14T22:38:24.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.079714220000000002 2024-09-14T22:38:24.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.498 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:24.498 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629116+0000", 2024-09-14T22:38:24.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7966000000000005e-05 2024-09-14T22:38:24.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.498 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:24.498 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629228+0000", 2024-09-14T22:38:24.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112831 2024-09-14T22:38:24.498 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.498 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:24.499 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.499 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144988 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3325952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:24.499 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549335+0000", 2024-09-14T22:38:24.499 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.941652755, 2024-09-14T22:38:24.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080179116999999994, 2024-09-14T22:38:24.499 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:24.499 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:24.499 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:24.499 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:24.499 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:24.499 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:24.500 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144988 2024-09-14T22:38:24.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.500 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:24.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:24.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549335+0000", 2024-09-14T22:38:24.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:24.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549335+0000", 2024-09-14T22:38:24.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:24.501 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549336+0000", 2024-09-14T22:38:24.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7999999999999995e-07 2024-09-14T22:38:24.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:24.501 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549339+0000", 2024-09-14T22:38:24.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.836e-06 2024-09-14T22:38:24.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.502 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:24.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549352+0000", 2024-09-14T22:38:24.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3069e-05 2024-09-14T22:38:24.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.502 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:24.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549359+0000", 2024-09-14T22:38:24.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.357e-06 2024-09-14T22:38:24.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:24.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629288+0000", 2024-09-14T22:38:24.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.079928492000000004 2024-09-14T22:38:24.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:24.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629412+0000", 2024-09-14T22:38:24.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012413699999999999 2024-09-14T22:38:24.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:24.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629514+0000", 2024-09-14T22:38:24.504 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000102546 2024-09-14T22:38:24.504 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.504 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:24.504 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.504 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144989 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3522560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:24.504 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:38:24.504 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.941618346999999, 2024-09-14T22:38:24.504 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080393122999999997, 2024-09-14T22:38:24.505 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:24.505 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:24.505 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:24.505 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:24.505 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:24.505 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:24.505 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144989 2024-09-14T22:38:24.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.505 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:24.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:24.505 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:38:24.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:24.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:38:24.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:24.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:38:24.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3600000000000003e-07 2024-09-14T22:38:24.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:24.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549373+0000", 2024-09-14T22:38:24.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8169999999999999e-06 2024-09-14T22:38:24.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:24.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549385+0000", 2024-09-14T22:38:24.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1803e-05 2024-09-14T22:38:24.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:24.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549394+0000", 2024-09-14T22:38:24.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4290000000000007e-06 2024-09-14T22:38:24.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:24.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629576+0000", 2024-09-14T22:38:24.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080181551000000004 2024-09-14T22:38:24.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:24.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629654+0000", 2024-09-14T22:38:24.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7633999999999996e-05 2024-09-14T22:38:24.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:24.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629763+0000", 2024-09-14T22:38:24.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109153 2024-09-14T22:38:24.509 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.509 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:24.509 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.509 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144990 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:24.509 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:38:24.509 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.941575322999999, 2024-09-14T22:38:24.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080614496999999993, 2024-09-14T22:38:24.509 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:24.509 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:24.510 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:24.510 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:24.510 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:24.510 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:24.510 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144990 2024-09-14T22:38:24.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.510 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:24.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:24.510 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:38:24.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:24.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:38:24.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:24.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:38:24.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0999999999999997e-07 2024-09-14T22:38:24.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:24.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549419+0000", 2024-09-14T22:38:24.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5339999999999999e-06 2024-09-14T22:38:24.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:24.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549439+0000", 2024-09-14T22:38:24.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0035e-05 2024-09-14T22:38:24.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:24.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549449+0000", 2024-09-14T22:38:24.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0429e-05 2024-09-14T22:38:24.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:24.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629815+0000", 2024-09-14T22:38:24.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080365467999999995 2024-09-14T22:38:24.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:24.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629922+0000", 2024-09-14T22:38:24.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106914 2024-09-14T22:38:24.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:24.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630027+0000", 2024-09-14T22:38:24.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105207 2024-09-14T22:38:24.514 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.514 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:24.514 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.514 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144991 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 2998272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:24.514 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549463+0000", 2024-09-14T22:38:24.514 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.941525019, 2024-09-14T22:38:24.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080847637, 2024-09-14T22:38:24.514 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:24.514 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:24.514 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:24.514 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:24.515 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:24.515 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:24.515 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144991 2024-09-14T22:38:24.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.515 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:24.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:24.515 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549463+0000", 2024-09-14T22:38:24.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:24.515 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549463+0000", 2024-09-14T22:38:24.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:24.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549464+0000", 2024-09-14T22:38:24.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9999999999999997e-07 2024-09-14T22:38:24.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:24.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549467+0000", 2024-09-14T22:38:24.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0800000000000002e-06 2024-09-14T22:38:24.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:24.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549478+0000", 2024-09-14T22:38:24.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1813e-05 2024-09-14T22:38:24.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:24.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549488+0000", 2024-09-14T22:38:24.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7030000000000001e-06 2024-09-14T22:38:24.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:24.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630090+0000", 2024-09-14T22:38:24.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080601962999999999 2024-09-14T22:38:24.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:24.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630187+0000", 2024-09-14T22:38:24.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6552000000000007e-05 2024-09-14T22:38:24.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:24.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630311+0000", 2024-09-14T22:38:24.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000123826 2024-09-14T22:38:24.518 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.518 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:24.518 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.519 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144992 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3620864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:24.519 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:38:24.519 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.941489211, 2024-09-14T22:38:24.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081097875999999999, 2024-09-14T22:38:24.519 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:24.519 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:24.519 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:24.519 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:24.519 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:24.519 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:24.519 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144992 2024-09-14T22:38:24.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.519 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:24.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:24.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:38:24.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:24.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:38:24.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:24.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:38:24.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5400000000000001e-07 2024-09-14T22:38:24.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:24.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549502+0000", 2024-09-14T22:38:24.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7889999999999999e-06 2024-09-14T22:38:24.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:24.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549513+0000", 2024-09-14T22:38:24.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1204e-05 2024-09-14T22:38:24.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:24.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549523+0000", 2024-09-14T22:38:24.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5990000000000008e-06 2024-09-14T22:38:24.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.522 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:24.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630362+0000", 2024-09-14T22:38:24.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080839272000000004 2024-09-14T22:38:24.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.522 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:24.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630469+0000", 2024-09-14T22:38:24.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107285 2024-09-14T22:38:24.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.522 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:24.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630597+0000", 2024-09-14T22:38:24.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012707300000000001 2024-09-14T22:38:24.523 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.523 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:24.523 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.523 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144993 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:24.523 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:38:24.523 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.941456134999999, 2024-09-14T22:38:24.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081338411999999999, 2024-09-14T22:38:24.523 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:24.523 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:24.523 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:24.523 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:24.523 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:24.524 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:24.524 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144993 2024-09-14T22:38:24.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.524 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:24.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:24.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:38:24.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:24.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:38:24.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:24.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:38:24.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5300000000000004e-07 2024-09-14T22:38:24.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:24.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549544+0000", 2024-09-14T22:38:24.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1912e-05 2024-09-14T22:38:24.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:24.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549557+0000", 2024-09-14T22:38:24.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2413e-05 2024-09-14T22:38:24.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:24.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549567+0000", 2024-09-14T22:38:24.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8660000000000001e-06 2024-09-14T22:38:24.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:24.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630651+0000", 2024-09-14T22:38:24.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081084229999999993 2024-09-14T22:38:24.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.527 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:24.527 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630729+0000", 2024-09-14T22:38:24.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8265999999999999e-05 2024-09-14T22:38:24.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.527 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:24.527 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630870+0000", 2024-09-14T22:38:24.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014107200000000001 2024-09-14T22:38:24.527 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.527 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:24.527 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.527 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144994 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:24.528 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:38:24.528 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.941409168, 2024-09-14T22:38:24.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081547958000000004, 2024-09-14T22:38:24.528 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:24.528 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:24.528 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:24.528 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:24.528 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:24.528 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:24.528 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144994 2024-09-14T22:38:24.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.528 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:24.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:24.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:38:24.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:24.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:38:24.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:24.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:38:24.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6599999999999996e-07 2024-09-14T22:38:24.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:24.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549582+0000", 2024-09-14T22:38:24.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.004e-06 2024-09-14T22:38:24.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:24.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549594+0000", 2024-09-14T22:38:24.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1459e-05 2024-09-14T22:38:24.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:24.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549603+0000", 2024-09-14T22:38:24.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6129999999999993e-06 2024-09-14T22:38:24.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:24.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630932+0000", 2024-09-14T22:38:24.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081328558999999995 2024-09-14T22:38:24.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:24.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631027+0000", 2024-09-14T22:38:24.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4742999999999997e-05 2024-09-14T22:38:24.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.532 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:24.532 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631127+0000", 2024-09-14T22:38:24.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9914000000000006e-05 2024-09-14T22:38:24.532 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.532 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:24.532 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.532 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144995 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3391488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:24.532 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549615+0000", 2024-09-14T22:38:24.532 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.941373030000001, 2024-09-14T22:38:24.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081734251999999993, 2024-09-14T22:38:24.533 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:24.533 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:24.533 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:24.533 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:24.533 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:24.533 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:24.533 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144995 2024-09-14T22:38:24.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.533 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:24.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.533 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:24.533 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549615+0000", 2024-09-14T22:38:24.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:24.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549615+0000", 2024-09-14T22:38:24.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:24.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549616+0000", 2024-09-14T22:38:24.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9299999999999997e-07 2024-09-14T22:38:24.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:24.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549618+0000", 2024-09-14T22:38:24.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7269999999999999e-06 2024-09-14T22:38:24.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:24.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549630+0000", 2024-09-14T22:38:24.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1313e-05 2024-09-14T22:38:24.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:24.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549639+0000", 2024-09-14T22:38:24.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4159999999999993e-06 2024-09-14T22:38:24.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:24.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631183+0000", 2024-09-14T22:38:24.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081543633000000004 2024-09-14T22:38:24.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:24.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631252+0000", 2024-09-14T22:38:24.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9567999999999995e-05 2024-09-14T22:38:24.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.537 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:24.537 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631349+0000", 2024-09-14T22:38:24.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6801999999999999e-05 2024-09-14T22:38:24.537 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.537 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:24.537 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.537 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144996 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3194880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:24.537 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:38:24.537 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.941337352000001, 2024-09-14T22:38:24.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081947213000000005, 2024-09-14T22:38:24.538 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:24.538 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:24.538 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:24.538 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:24.538 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:24.538 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:24.538 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144996 2024-09-14T22:38:24.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.538 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:24.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:24.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:38:24.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:24.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:38:24.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:24.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:38:24.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3e-07 2024-09-14T22:38:24.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:24.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549654+0000", 2024-09-14T22:38:24.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6900000000000001e-06 2024-09-14T22:38:24.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:24.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549665+0000", 2024-09-14T22:38:24.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1392999999999999e-05 2024-09-14T22:38:24.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:24.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549675+0000", 2024-09-14T22:38:24.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5459999999999997e-06 2024-09-14T22:38:24.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:24.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631401+0000", 2024-09-14T22:38:24.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081726036000000002 2024-09-14T22:38:24.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.541 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:24.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631492+0000", 2024-09-14T22:38:24.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0886999999999999e-05 2024-09-14T22:38:24.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.541 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:24.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631598+0000", 2024-09-14T22:38:24.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106031 2024-09-14T22:38:24.541 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.541 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:24.541 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.542 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144997 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:24.542 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549692+0000", 2024-09-14T22:38:24.542 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.941295688, 2024-09-14T22:38:24.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082149925999999998, 2024-09-14T22:38:24.542 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:24.542 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:24.542 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:24.542 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:24.542 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:24.542 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:24.542 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144997 2024-09-14T22:38:24.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.543 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:24.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:24.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549692+0000", 2024-09-14T22:38:24.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:24.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549692+0000", 2024-09-14T22:38:24.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:24.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549693+0000", 2024-09-14T22:38:24.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.540000000000001e-07 2024-09-14T22:38:24.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:24.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549698+0000", 2024-09-14T22:38:24.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1460000000000004e-06 2024-09-14T22:38:24.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:24.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549715+0000", 2024-09-14T22:38:24.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6983e-05 2024-09-14T22:38:24.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:24.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549728+0000", 2024-09-14T22:38:24.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2326e-05 2024-09-14T22:38:24.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.545 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:24.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631649+0000", 2024-09-14T22:38:24.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081921830000000001 2024-09-14T22:38:24.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.545 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:24.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631726+0000", 2024-09-14T22:38:24.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6759000000000001e-05 2024-09-14T22:38:24.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.546 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:24.546 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631842+0000", 2024-09-14T22:38:24.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011592800000000001 2024-09-14T22:38:24.546 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.546 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:24.546 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.546 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144998 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 2965504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:24.546 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549741+0000", 2024-09-14T22:38:24.546 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.941246396, 2024-09-14T22:38:24.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082394832000000001, 2024-09-14T22:38:24.547 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:24.547 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:24.547 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:24.547 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:24.547 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:24.547 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:24.547 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144998 2024-09-14T22:38:24.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.547 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:24.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:24.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549741+0000", 2024-09-14T22:38:24.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:24.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549741+0000", 2024-09-14T22:38:24.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:24.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549742+0000", 2024-09-14T22:38:24.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.06e-07 2024-09-14T22:38:24.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:24.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549757+0000", 2024-09-14T22:38:24.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4839e-05 2024-09-14T22:38:24.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:24.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549770+0000", 2024-09-14T22:38:24.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2879e-05 2024-09-14T22:38:24.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:24.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549780+0000", 2024-09-14T22:38:24.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0304e-05 2024-09-14T22:38:24.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.550 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:24.550 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631904+0000", 2024-09-14T22:38:24.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082124168999999997 2024-09-14T22:38:24.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.550 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:24.550 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631995+0000", 2024-09-14T22:38:24.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0993000000000001e-05 2024-09-14T22:38:24.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.550 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:24.550 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632136+0000", 2024-09-14T22:38:24.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014094199999999999 2024-09-14T22:38:24.550 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.551 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:24.551 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.551 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144999 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3588096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:24.551 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:38:24.551 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.941196185999999, 2024-09-14T22:38:24.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082593742999999997, 2024-09-14T22:38:24.551 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:24.551 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:24.551 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:24.551 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:24.551 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:24.551 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:24.552 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144999 2024-09-14T22:38:24.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.552 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:24.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:24.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:38:24.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:24.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:38:24.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:24.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:38:24.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1999999999999999e-07 2024-09-14T22:38:24.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:24.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549795+0000", 2024-09-14T22:38:24.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9569999999999998e-06 2024-09-14T22:38:24.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:24.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549807+0000", 2024-09-14T22:38:24.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1691999999999999e-05 2024-09-14T22:38:24.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:24.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549817+0000", 2024-09-14T22:38:24.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5100000000000004e-06 2024-09-14T22:38:24.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:24.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632190+0000", 2024-09-14T22:38:24.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082373374999999999 2024-09-14T22:38:24.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.555 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:24.555 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632264+0000", 2024-09-14T22:38:24.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3972000000000003e-05 2024-09-14T22:38:24.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.555 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:24.555 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632385+0000", 2024-09-14T22:38:24.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121517 2024-09-14T22:38:24.555 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.555 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:24.555 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.556 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145000 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3063808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:24.556 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549828+0000", 2024-09-14T22:38:24.556 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.941159907999999, 2024-09-14T22:38:24.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082860235000000004, 2024-09-14T22:38:24.556 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:24.556 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:24.556 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:24.556 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:24.556 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:24.556 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:24.556 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145000 2024-09-14T22:38:24.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.557 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:24.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:24.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549828+0000", 2024-09-14T22:38:24.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:24.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549828+0000", 2024-09-14T22:38:24.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:24.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549829+0000", 2024-09-14T22:38:24.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3300000000000002e-07 2024-09-14T22:38:24.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:24.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549831+0000", 2024-09-14T22:38:24.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6740000000000001e-06 2024-09-14T22:38:24.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:24.559 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549842+0000", 2024-09-14T22:38:24.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0316000000000001e-05 2024-09-14T22:38:24.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:24.559 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549851+0000", 2024-09-14T22:38:24.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7229999999999999e-06 2024-09-14T22:38:24.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:24.559 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632448+0000", 2024-09-14T22:38:24.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082596648999999994 2024-09-14T22:38:24.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.560 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:24.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632568+0000", 2024-09-14T22:38:24.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119592 2024-09-14T22:38:24.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.560 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:24.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632688+0000", 2024-09-14T22:38:24.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120648 2024-09-14T22:38:24.560 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.560 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:24.561 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.561 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145001 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3162112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:24.561 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549863+0000", 2024-09-14T22:38:24.561 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.941124680000001, 2024-09-14T22:38:24.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083105534999999994, 2024-09-14T22:38:24.561 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:24.561 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:24.561 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:24.561 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:24.561 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:24.562 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:24.562 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145001 2024-09-14T22:38:24.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.562 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:24.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:24.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549863+0000", 2024-09-14T22:38:24.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:24.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549863+0000", 2024-09-14T22:38:24.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:24.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549864+0000", 2024-09-14T22:38:24.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4300000000000003e-07 2024-09-14T22:38:24.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:24.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549867+0000", 2024-09-14T22:38:24.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.757e-06 2024-09-14T22:38:24.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:24.564 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549878+0000", 2024-09-14T22:38:24.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1345999999999999e-05 2024-09-14T22:38:24.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.564 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:24.564 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549888+0000", 2024-09-14T22:38:24.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6460000000000005e-06 2024-09-14T22:38:24.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.564 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:24.564 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632754+0000", 2024-09-14T22:38:24.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082866258999999998 2024-09-14T22:38:24.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:24.565 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632841+0000", 2024-09-14T22:38:24.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6971000000000005e-05 2024-09-14T22:38:24.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:24.565 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632969+0000", 2024-09-14T22:38:24.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012791300000000001 2024-09-14T22:38:24.565 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.565 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:24.565 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.566 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145002 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3260416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:24.566 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549933+0000", 2024-09-14T22:38:24.566 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.941055176999999, 2024-09-14T22:38:24.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083291027000000004, 2024-09-14T22:38:24.566 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:24.566 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:24.566 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:24.566 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:24.566 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:24.566 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:24.566 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145002 2024-09-14T22:38:24.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.566 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:24.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:24.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549933+0000", 2024-09-14T22:38:24.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:24.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549933+0000", 2024-09-14T22:38:24.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:24.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549934+0000", 2024-09-14T22:38:24.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8700000000000004e-07 2024-09-14T22:38:24.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:24.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549938+0000", 2024-09-14T22:38:24.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5029999999999999e-06 2024-09-14T22:38:24.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:24.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549952+0000", 2024-09-14T22:38:24.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4382999999999999e-05 2024-09-14T22:38:24.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:24.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549963+0000", 2024-09-14T22:38:24.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0665999999999999e-05 2024-09-14T22:38:24.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:24.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633025+0000", 2024-09-14T22:38:24.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083061727000000002 2024-09-14T22:38:24.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:24.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633117+0000", 2024-09-14T22:38:24.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1812999999999997e-05 2024-09-14T22:38:24.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:24.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633224+0000", 2024-09-14T22:38:24.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107048 2024-09-14T22:38:24.570 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.570 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:24.570 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.570 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145003 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3457024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:24.570 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549985+0000", 2024-09-14T22:38:24.570 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.941002759, 2024-09-14T22:38:24.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083465871999999997, 2024-09-14T22:38:24.570 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:24.571 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:24.571 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:24.571 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:24.571 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:24.571 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:24.571 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145003 2024-09-14T22:38:24.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.571 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:24.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:24.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549985+0000", 2024-09-14T22:38:24.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:24.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549985+0000", 2024-09-14T22:38:24.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:24.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549986+0000", 2024-09-14T22:38:24.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7299999999999995e-07 2024-09-14T22:38:24.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:24.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549989+0000", 2024-09-14T22:38:24.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8169999999999999e-06 2024-09-14T22:38:24.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:24.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549998+0000", 2024-09-14T22:38:24.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4290000000000007e-06 2024-09-14T22:38:24.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:24.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550009+0000", 2024-09-14T22:38:24.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0633e-05 2024-09-14T22:38:24.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:24.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633275+0000", 2024-09-14T22:38:24.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083266397000000006 2024-09-14T22:38:24.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:24.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633353+0000", 2024-09-14T22:38:24.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8129999999999999e-05 2024-09-14T22:38:24.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:24.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633451+0000", 2024-09-14T22:38:24.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7793000000000004e-05 2024-09-14T22:38:24.574 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.575 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:24.575 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.575 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145004 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3358720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:24.575 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.550029+0000", 2024-09-14T22:38:24.575 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.940959188000001, 2024-09-14T22:38:24.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083662089999999995, 2024-09-14T22:38:24.575 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:24.575 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:24.575 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:24.575 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:24.575 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:24.575 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:24.576 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145004 2024-09-14T22:38:24.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.576 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:24.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:24.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550029+0000", 2024-09-14T22:38:24.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:24.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550029+0000", 2024-09-14T22:38:24.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:24.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550030+0000", 2024-09-14T22:38:24.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.33e-07 2024-09-14T22:38:24.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:24.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550035+0000", 2024-09-14T22:38:24.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3430000000000003e-06 2024-09-14T22:38:24.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:24.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550053+0000", 2024-09-14T22:38:24.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8485e-05 2024-09-14T22:38:24.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:24.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550064+0000", 2024-09-14T22:38:24.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0264e-05 2024-09-14T22:38:24.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:24.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633502+0000", 2024-09-14T22:38:24.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083438498999999999 2024-09-14T22:38:24.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:24.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633592+0000", 2024-09-14T22:38:24.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.022e-05 2024-09-14T22:38:24.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:24.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633691+0000", 2024-09-14T22:38:24.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8345999999999997e-05 2024-09-14T22:38:24.579 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.579 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:24.579 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.579 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145005 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3555328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:24.579 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.550077+0000", 2024-09-14T22:38:24.580 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.940910727999999, 2024-09-14T22:38:24.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083854340999999999, 2024-09-14T22:38:24.580 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:24.580 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:24.580 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:24.580 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:24.580 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:24.580 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:24.580 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145005 2024-09-14T22:38:24.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.580 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:24.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:24.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550077+0000", 2024-09-14T22:38:24.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:24.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550077+0000", 2024-09-14T22:38:24.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:24.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550078+0000", 2024-09-14T22:38:24.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8700000000000005e-07 2024-09-14T22:38:24.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:24.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550101+0000", 2024-09-14T22:38:24.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3632e-05 2024-09-14T22:38:24.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:24.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550115+0000", 2024-09-14T22:38:24.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3893000000000001e-05 2024-09-14T22:38:24.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:24.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550126+0000", 2024-09-14T22:38:24.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0426e-05 2024-09-14T22:38:24.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:24.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633742+0000", 2024-09-14T22:38:24.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083616633999999995 2024-09-14T22:38:24.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:24.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633816+0000", 2024-09-14T22:38:24.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3690999999999996e-05 2024-09-14T22:38:24.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:24.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633932+0000", 2024-09-14T22:38:24.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011537800000000001 2024-09-14T22:38:24.584 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.584 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:24.584 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.584 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145006 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3653632~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:24.584 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.550236+0000", 2024-09-14T22:38:24.584 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.940752222, 2024-09-14T22:38:24.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083924027999999998, 2024-09-14T22:38:24.584 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:24.584 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:24.584 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:24.584 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:24.585 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:24.585 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:24.585 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145006 2024-09-14T22:38:24.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.585 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:24.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:24.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550236+0000", 2024-09-14T22:38:24.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:24.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550236+0000", 2024-09-14T22:38:24.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:24.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:24.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550237+0000", 2024-09-14T22:38:24.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2929999999999999e-06 2024-09-14T22:38:24.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:24.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550243+0000", 2024-09-14T22:38:24.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0390000000000001e-06 2024-09-14T22:38:24.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:24.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550256+0000", 2024-09-14T22:38:24.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3393e-05 2024-09-14T22:38:24.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:24.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550271+0000", 2024-09-14T22:38:24.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4166e-05 2024-09-14T22:38:24.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:24.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633974+0000", 2024-09-14T22:38:24.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083703825999999995 2024-09-14T22:38:24.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:24.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.634047+0000", 2024-09-14T22:38:24.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2935999999999994e-05 2024-09-14T22:38:24.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:24.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:24.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:24.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.634160+0000", 2024-09-14T22:38:24.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011237499999999999 2024-09-14T22:38:24.588 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.588 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:24.588 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.588 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:24.588 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:24.589 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:24.590 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:38:24.591 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:24.582+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:24.691 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:24.689+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:24.721 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:24.722 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:24.722 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:38:24.722 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:24.736 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:38:24.826 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:24.825+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:24.892 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:24.889+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:24.905 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:24.905 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:24.906 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:38:24.906 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:38:24.906 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:24.920 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:38:25.006 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:25.005+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:25.082 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:25.082 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:38:25.082 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:38:25.082 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:38:25.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.082 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:122960 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 2347008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:25.082 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:41.998325+0000", 2024-09-14T22:38:25.082 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.081945679, 2024-09-14T22:38:25.083 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056570400999999999, 2024-09-14T22:38:25.083 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.083 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.083 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.083 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.083 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.083 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.083 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 122960 2024-09-14T22:38:25.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.083 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.083 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.083 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998325+0000", 2024-09-14T22:38:25.083 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.084 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.084 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998325+0000", 2024-09-14T22:38:25.084 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.084 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.084 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998326+0000", 2024-09-14T22:38:25.084 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9299999999999998e-07 2024-09-14T22:38:25.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.084 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.085 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998329+0000", 2024-09-14T22:38:25.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1370000000000002e-06 2024-09-14T22:38:25.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.085 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.085 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998340+0000", 2024-09-14T22:38:25.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1039999999999999e-05 2024-09-14T22:38:25.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.085 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.085 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998348+0000", 2024-09-14T22:38:25.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1920000000000005e-06 2024-09-14T22:38:25.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.086 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054700+0000", 2024-09-14T22:38:25.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056351714999999997 2024-09-14T22:38:25.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.086 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054771+0000", 2024-09-14T22:38:25.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1009999999999999e-05 2024-09-14T22:38:25.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.086 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054896+0000", 2024-09-14T22:38:25.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000124714 2024-09-14T22:38:25.087 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.087 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.087 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.087 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:122961 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 2351104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:25.087 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:41.998357+0000", 2024-09-14T22:38:25.087 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.081913724000003, 2024-09-14T22:38:25.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056974912000000003, 2024-09-14T22:38:25.087 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.087 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.087 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.087 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.088 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.088 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.088 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 122961 2024-09-14T22:38:25.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.088 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.088 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.088 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998357+0000", 2024-09-14T22:38:25.088 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.088 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.089 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998357+0000", 2024-09-14T22:38:25.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.089 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998358+0000", 2024-09-14T22:38:25.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9699999999999996e-07 2024-09-14T22:38:25.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.089 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998361+0000", 2024-09-14T22:38:25.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1370000000000002e-06 2024-09-14T22:38:25.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998371+0000", 2024-09-14T22:38:25.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5789999999999993e-06 2024-09-14T22:38:25.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998379+0000", 2024-09-14T22:38:25.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0129999999999993e-06 2024-09-14T22:38:25.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054998+0000", 2024-09-14T22:38:25.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056618961000000002 2024-09-14T22:38:25.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.055126+0000", 2024-09-14T22:38:25.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012795100000000001 2024-09-14T22:38:25.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.055332+0000", 2024-09-14T22:38:25.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020667400000000001 2024-09-14T22:38:25.091 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.091 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.092 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.092 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123259 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 741376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:25.092 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:38:25.092 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.068188206999999, 2024-09-14T22:38:25.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056672581999999999, 2024-09-14T22:38:25.092 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.092 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.092 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.092 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.092 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.092 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.092 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123259 2024-09-14T22:38:25.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.093 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.093 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:38:25.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.093 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:38:25.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:38:25.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1600000000000001e-07 2024-09-14T22:38:25.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012086+0000", 2024-09-14T22:38:25.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8499999999999998e-06 2024-09-14T22:38:25.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012097+0000", 2024-09-14T22:38:25.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0943e-05 2024-09-14T22:38:25.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012105+0000", 2024-09-14T22:38:25.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0830000000000004e-06 2024-09-14T22:38:25.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068584+0000", 2024-09-14T22:38:25.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056478387999999997 2024-09-14T22:38:25.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068644+0000", 2024-09-14T22:38:25.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9984e-05 2024-09-14T22:38:25.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068755+0000", 2024-09-14T22:38:25.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111718 2024-09-14T22:38:25.096 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.096 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.096 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.096 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123260 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 778240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:25.096 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012118+0000", 2024-09-14T22:38:25.097 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.068152982000001, 2024-09-14T22:38:25.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056825887999999998, 2024-09-14T22:38:25.097 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.097 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.097 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.097 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.097 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.097 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.097 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123260 2024-09-14T22:38:25.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.097 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012118+0000", 2024-09-14T22:38:25.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.098 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012118+0000", 2024-09-14T22:38:25.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.098 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012119+0000", 2024-09-14T22:38:25.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.13e-07 2024-09-14T22:38:25.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.099 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012130+0000", 2024-09-14T22:38:25.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1552999999999999e-05 2024-09-14T22:38:25.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.099 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012141+0000", 2024-09-14T22:38:25.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.03e-05 2024-09-14T22:38:25.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.099 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012149+0000", 2024-09-14T22:38:25.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1419999999999993e-06 2024-09-14T22:38:25.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.100 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068800+0000", 2024-09-14T22:38:25.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056651574000000003 2024-09-14T22:38:25.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.100 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068858+0000", 2024-09-14T22:38:25.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8047000000000002e-05 2024-09-14T22:38:25.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.100 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068944+0000", 2024-09-14T22:38:25.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5659000000000001e-05 2024-09-14T22:38:25.101 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.101 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.101 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.101 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123261 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 761856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:25.101 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012161+0000", 2024-09-14T22:38:25.101 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.068109581000002, 2024-09-14T22:38:25.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057113753000000003, 2024-09-14T22:38:25.101 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.101 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.101 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.102 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.102 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.102 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.102 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123261 2024-09-14T22:38:25.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.102 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.102 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012161+0000", 2024-09-14T22:38:25.102 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012161+0000", 2024-09-14T22:38:25.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012162+0000", 2024-09-14T22:38:25.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4399999999999999e-07 2024-09-14T22:38:25.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012177+0000", 2024-09-14T22:38:25.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5099000000000001e-05 2024-09-14T22:38:25.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012191+0000", 2024-09-14T22:38:25.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3631999999999999e-05 2024-09-14T22:38:25.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012199+0000", 2024-09-14T22:38:25.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5769999999999994e-06 2024-09-14T22:38:25.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.105 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.105 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069004+0000", 2024-09-14T22:38:25.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056804504999999998 2024-09-14T22:38:25.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.105 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.105 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069137+0000", 2024-09-14T22:38:25.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013302600000000001 2024-09-14T22:38:25.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.105 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069275+0000", 2024-09-14T22:38:25.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013816999999999999 2024-09-14T22:38:25.106 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.106 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.106 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.106 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123262 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 724992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:25.106 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012224+0000", 2024-09-14T22:38:25.106 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.068047133999997, 2024-09-14T22:38:25.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057417959999999997, 2024-09-14T22:38:25.106 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.106 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.106 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.107 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.107 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.107 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.107 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123262 2024-09-14T22:38:25.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.107 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012224+0000", 2024-09-14T22:38:25.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012224+0000", 2024-09-14T22:38:25.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012225+0000", 2024-09-14T22:38:25.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4000000000000001e-07 2024-09-14T22:38:25.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012228+0000", 2024-09-14T22:38:25.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5729999999999999e-06 2024-09-14T22:38:25.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012241+0000", 2024-09-14T22:38:25.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3210000000000001e-05 2024-09-14T22:38:25.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012255+0000", 2024-09-14T22:38:25.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3852000000000001e-05 2024-09-14T22:38:25.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069341+0000", 2024-09-14T22:38:25.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057086139000000001 2024-09-14T22:38:25.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069477+0000", 2024-09-14T22:38:25.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013595 2024-09-14T22:38:25.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069642+0000", 2024-09-14T22:38:25.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016459600000000001 2024-09-14T22:38:25.110 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.110 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.110 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.111 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123263 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 782336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:25.111 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012265+0000", 2024-09-14T22:38:25.111 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.068005591999999, 2024-09-14T22:38:25.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057665095, 2024-09-14T22:38:25.111 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.111 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.111 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.111 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.111 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.111 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.111 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123263 2024-09-14T22:38:25.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.111 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012265+0000", 2024-09-14T22:38:25.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012265+0000", 2024-09-14T22:38:25.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012266+0000", 2024-09-14T22:38:25.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8999999999999996e-07 2024-09-14T22:38:25.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012269+0000", 2024-09-14T22:38:25.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.796e-06 2024-09-14T22:38:25.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012279+0000", 2024-09-14T22:38:25.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7389999999999994e-06 2024-09-14T22:38:25.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012286+0000", 2024-09-14T22:38:25.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8199999999999997e-06 2024-09-14T22:38:25.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069705+0000", 2024-09-14T22:38:25.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057418590999999998 2024-09-14T22:38:25.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069799+0000", 2024-09-14T22:38:25.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4419999999999994e-05 2024-09-14T22:38:25.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069931+0000", 2024-09-14T22:38:25.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013113900000000001 2024-09-14T22:38:25.115 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.115 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.115 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.115 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123264 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 745472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:25.115 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012295+0000", 2024-09-14T22:38:25.115 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.067975591, 2024-09-14T22:38:25.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057959498999999998, 2024-09-14T22:38:25.115 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.116 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.116 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.116 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.116 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.116 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.116 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123264 2024-09-14T22:38:25.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.116 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012295+0000", 2024-09-14T22:38:25.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012295+0000", 2024-09-14T22:38:25.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012296+0000", 2024-09-14T22:38:25.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9699999999999996e-07 2024-09-14T22:38:25.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012299+0000", 2024-09-14T22:38:25.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.79e-06 2024-09-14T22:38:25.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012320+0000", 2024-09-14T22:38:25.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1242000000000001e-05 2024-09-14T22:38:25.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012338+0000", 2024-09-14T22:38:25.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7768999999999999e-05 2024-09-14T22:38:25.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069995+0000", 2024-09-14T22:38:25.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057657 2024-09-14T22:38:25.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070132+0000", 2024-09-14T22:38:25.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013727800000000001 2024-09-14T22:38:25.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070255+0000", 2024-09-14T22:38:25.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012282300000000001 2024-09-14T22:38:25.119 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.119 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.120 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.120 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123265 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1056768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:25.120 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012348+0000", 2024-09-14T22:38:25.120 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.067922635999999, 2024-09-14T22:38:25.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058145087999999998, 2024-09-14T22:38:25.120 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.120 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.120 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.120 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.120 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.120 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.120 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123265 2024-09-14T22:38:25.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.121 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012348+0000", 2024-09-14T22:38:25.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012348+0000", 2024-09-14T22:38:25.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.122 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012349+0000", 2024-09-14T22:38:25.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6599999999999996e-07 2024-09-14T22:38:25.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.122 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012352+0000", 2024-09-14T22:38:25.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3139999999999999e-06 2024-09-14T22:38:25.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.122 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012365+0000", 2024-09-14T22:38:25.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2625e-05 2024-09-14T22:38:25.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012373+0000", 2024-09-14T22:38:25.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2470000000000008e-06 2024-09-14T22:38:25.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070286+0000", 2024-09-14T22:38:25.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057912547000000002 2024-09-14T22:38:25.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070366+0000", 2024-09-14T22:38:25.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0468000000000003e-05 2024-09-14T22:38:25.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070493+0000", 2024-09-14T22:38:25.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012722100000000001 2024-09-14T22:38:25.124 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.124 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.124 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.124 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123266 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 765952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:25.124 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:38:25.124 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.067888388, 2024-09-14T22:38:25.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058386949, 2024-09-14T22:38:25.125 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.125 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.125 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.125 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.125 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.125 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.125 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123266 2024-09-14T22:38:25.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.125 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.125 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:38:25.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:38:25.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:38:25.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9299999999999998e-07 2024-09-14T22:38:25.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012386+0000", 2024-09-14T22:38:25.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8729999999999999e-06 2024-09-14T22:38:25.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012406+0000", 2024-09-14T22:38:25.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0103e-05 2024-09-14T22:38:25.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012419+0000", 2024-09-14T22:38:25.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.311e-05 2024-09-14T22:38:25.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070550+0000", 2024-09-14T22:38:25.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058130902999999998 2024-09-14T22:38:25.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070650+0000", 2024-09-14T22:38:25.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010005000000000001 2024-09-14T22:38:25.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070770+0000", 2024-09-14T22:38:25.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119317 2024-09-14T22:38:25.128 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.129 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.129 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.129 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123267 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1077248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:25.129 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012429+0000", 2024-09-14T22:38:25.129 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.067842474000003, 2024-09-14T22:38:25.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058620343999999998, 2024-09-14T22:38:25.129 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.129 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.129 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.129 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.129 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.129 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.130 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123267 2024-09-14T22:38:25.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.130 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012429+0000", 2024-09-14T22:38:25.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012429+0000", 2024-09-14T22:38:25.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013488+0000", 2024-09-14T22:38:25.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001059061 2024-09-14T22:38:25.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013499+0000", 2024-09-14T22:38:25.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1348999999999999e-05 2024-09-14T22:38:25.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013534+0000", 2024-09-14T22:38:25.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5206000000000001e-05 2024-09-14T22:38:25.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013558+0000", 2024-09-14T22:38:25.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4324999999999998e-05 2024-09-14T22:38:25.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070834+0000", 2024-09-14T22:38:25.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057275110999999997 2024-09-14T22:38:25.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.133 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070937+0000", 2024-09-14T22:38:25.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010306099999999999 2024-09-14T22:38:25.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.133 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.133 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071049+0000", 2024-09-14T22:38:25.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112231 2024-09-14T22:38:25.133 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.133 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.133 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.133 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123268 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1097728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:25.133 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013593+0000", 2024-09-14T22:38:25.134 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.066677751, 2024-09-14T22:38:25.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057751793000000003, 2024-09-14T22:38:25.134 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.134 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.134 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.134 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.134 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.134 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.134 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123268 2024-09-14T22:38:25.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.134 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.134 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013593+0000", 2024-09-14T22:38:25.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013593+0000", 2024-09-14T22:38:25.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013595+0000", 2024-09-14T22:38:25.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.59e-06 2024-09-14T22:38:25.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013602+0000", 2024-09-14T22:38:25.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3560000000000004e-06 2024-09-14T22:38:25.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013625+0000", 2024-09-14T22:38:25.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2419e-05 2024-09-14T22:38:25.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013650+0000", 2024-09-14T22:38:25.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5151999999999999e-05 2024-09-14T22:38:25.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071096+0000", 2024-09-14T22:38:25.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057445877999999999 2024-09-14T22:38:25.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071212+0000", 2024-09-14T22:38:25.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116215 2024-09-14T22:38:25.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071345+0000", 2024-09-14T22:38:25.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013318299999999999 2024-09-14T22:38:25.137 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.138 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.138 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.138 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123269 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1060864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:25.138 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013676+0000", 2024-09-14T22:38:25.138 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.066594991999999, 2024-09-14T22:38:25.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058027982999999998, 2024-09-14T22:38:25.138 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.138 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.138 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.138 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.138 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.138 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.138 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123269 2024-09-14T22:38:25.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.139 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.139 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.139 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013676+0000", 2024-09-14T22:38:25.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.139 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.139 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013676+0000", 2024-09-14T22:38:25.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.139 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.139 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013677+0000", 2024-09-14T22:38:25.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.387e-06 2024-09-14T22:38:25.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013684+0000", 2024-09-14T22:38:25.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0530000000000001e-06 2024-09-14T22:38:25.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013706+0000", 2024-09-14T22:38:25.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1121999999999999e-05 2024-09-14T22:38:25.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013722+0000", 2024-09-14T22:38:25.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6249999999999999e-05 2024-09-14T22:38:25.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071402+0000", 2024-09-14T22:38:25.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057679945000000003 2024-09-14T22:38:25.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071547+0000", 2024-09-14T22:38:25.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000145632 2024-09-14T22:38:25.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071704+0000", 2024-09-14T22:38:25.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000156594 2024-09-14T22:38:25.142 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.142 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.142 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.142 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123270 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1064960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:25.142 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013744+0000", 2024-09-14T22:38:25.142 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.066527041000001, 2024-09-14T22:38:25.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058355872000000003, 2024-09-14T22:38:25.142 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.142 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.143 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.143 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.143 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.143 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.143 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123270 2024-09-14T22:38:25.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.143 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013744+0000", 2024-09-14T22:38:25.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013744+0000", 2024-09-14T22:38:25.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013745+0000", 2024-09-14T22:38:25.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2589999999999999e-06 2024-09-14T22:38:25.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013751+0000", 2024-09-14T22:38:25.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1929999999999998e-06 2024-09-14T22:38:25.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013778+0000", 2024-09-14T22:38:25.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6472e-05 2024-09-14T22:38:25.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013795+0000", 2024-09-14T22:38:25.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6626e-05 2024-09-14T22:38:25.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071775+0000", 2024-09-14T22:38:25.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057980232 2024-09-14T22:38:25.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071911+0000", 2024-09-14T22:38:25.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000136129 2024-09-14T22:38:25.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072100+0000", 2024-09-14T22:38:25.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000188961 2024-09-14T22:38:25.146 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.146 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.146 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.147 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123271 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1081344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:25.147 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013818+0000", 2024-09-14T22:38:25.147 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.066452845000001, 2024-09-14T22:38:25.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058549565999999997, 2024-09-14T22:38:25.147 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.147 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.147 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.147 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.148 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.148 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.148 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123271 2024-09-14T22:38:25.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.148 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.148 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013818+0000", 2024-09-14T22:38:25.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.148 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013818+0000", 2024-09-14T22:38:25.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013958+0000", 2024-09-14T22:38:25.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013934 2024-09-14T22:38:25.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013968+0000", 2024-09-14T22:38:25.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0329000000000001e-05 2024-09-14T22:38:25.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013995+0000", 2024-09-14T22:38:25.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7613000000000001e-05 2024-09-14T22:38:25.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014014+0000", 2024-09-14T22:38:25.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8338999999999998e-05 2024-09-14T22:38:25.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072155+0000", 2024-09-14T22:38:25.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058140989999999997 2024-09-14T22:38:25.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072247+0000", 2024-09-14T22:38:25.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1774999999999995e-05 2024-09-14T22:38:25.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072368+0000", 2024-09-14T22:38:25.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012118 2024-09-14T22:38:25.151 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.151 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.151 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.151 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123272 3.1 3.4b235a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:25.151 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.014046+0000", 2024-09-14T22:38:25.151 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.066225168999999, 2024-09-14T22:38:25.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058364077, 2024-09-14T22:38:25.152 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.152 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.152 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-14T22:38:25.152 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.152 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.152 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.152 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123272 2024-09-14T22:38:25.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.152 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.152 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014046+0000", 2024-09-14T22:38:25.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014046+0000", 2024-09-14T22:38:25.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014047+0000", 2024-09-14T22:38:25.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.533e-06 2024-09-14T22:38:25.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014085+0000", 2024-09-14T22:38:25.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7734999999999997e-05 2024-09-14T22:38:25.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014111+0000", 2024-09-14T22:38:25.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6055999999999999e-05 2024-09-14T22:38:25.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014136+0000", 2024-09-14T22:38:25.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4592000000000001e-05 2024-09-14T22:38:25.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072402+0000", 2024-09-14T22:38:25.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058266251999999998 2024-09-14T22:38:25.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072410+0000", 2024-09-14T22:38:25.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.909e-06 2024-09-14T22:38:25.155 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.155 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.155 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.155 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123273 3.1 3.4b235a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:25.155 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.014161+0000", 2024-09-14T22:38:25.156 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.066110141000003, 2024-09-14T22:38:25.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058284415999999999, 2024-09-14T22:38:25.156 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.156 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.156 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-14T22:38:25.156 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.156 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.156 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.156 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123273 2024-09-14T22:38:25.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.156 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014161+0000", 2024-09-14T22:38:25.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.157 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.157 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014161+0000", 2024-09-14T22:38:25.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.157 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.157 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.017996+0000", 2024-09-14T22:38:25.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0038350670000000002 2024-09-14T22:38:25.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.018009+0000", 2024-09-14T22:38:25.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3305999999999999e-05 2024-09-14T22:38:25.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.018044+0000", 2024-09-14T22:38:25.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4792000000000002e-05 2024-09-14T22:38:25.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.018067+0000", 2024-09-14T22:38:25.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2595e-05 2024-09-14T22:38:25.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072439+0000", 2024-09-14T22:38:25.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054372192 2024-09-14T22:38:25.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072445+0000", 2024-09-14T22:38:25.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4640000000000003e-06 2024-09-14T22:38:25.159 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.159 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.159 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.160 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:148469 3.18 3:1f6f7244:::rbd_header.113c827b523f:head [watch reconnect cookie 140250011614800 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-14T22:38:25.160 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:55.746449+0000", 2024-09-14T22:38:25.160 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.333821529000002, 2024-09-14T22:38:25.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.996484173, 2024-09-14T22:38:25.160 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.160 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.160 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.160 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.160 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.160 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.161 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 148469 2024-09-14T22:38:25.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.161 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.161 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.161 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746449+0000", 2024-09-14T22:38:25.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746449+0000", 2024-09-14T22:38:25.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746452+0000", 2024-09-14T22:38:25.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0019999999999998e-06 2024-09-14T22:38:25.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746461+0000", 2024-09-14T22:38:25.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0919999999999998e-06 2024-09-14T22:38:25.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746475+0000", 2024-09-14T22:38:25.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4853e-05 2024-09-14T22:38:25.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746484+0000", 2024-09-14T22:38:25.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3780000000000001e-06 2024-09-14T22:38:25.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746559+0000", 2024-09-14T22:38:25.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5259000000000006e-05 2024-09-14T22:38:25.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:25.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746572+0000", 2024-09-14T22:38:25.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.271e-05 2024-09-14T22:38:25.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727807+0000", 2024-09-14T22:38:25.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98123555900000003 2024-09-14T22:38:25.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.165 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:25.165 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727826+0000", 2024-09-14T22:38:25.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8828e-05 2024-09-14T22:38:25.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.165 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.165 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.739033+0000", 2024-09-14T22:38:25.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.011207062 2024-09-14T22:38:25.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.165 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.165 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.739479+0000", 2024-09-14T22:38:25.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044563200000000001 2024-09-14T22:38:25.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.166 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.166 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.742934+0000", 2024-09-14T22:38:25.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0034547979999999998 2024-09-14T22:38:25.166 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.166 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.166 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.166 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:148473 3.18 3:1f6f7244:::rbd_header.113c827b523f:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-14T22:38:25.166 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:55.755969+0000", 2024-09-14T22:38:25.166 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.324301763000001, 2024-09-14T22:38:25.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9872532889999999, 2024-09-14T22:38:25.167 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.167 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.167 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:25.167 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.167 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.167 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.167 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 148473 2024-09-14T22:38:25.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.167 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755969+0000", 2024-09-14T22:38:25.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755969+0000", 2024-09-14T22:38:25.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755971+0000", 2024-09-14T22:38:25.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.674e-06 2024-09-14T22:38:25.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755979+0000", 2024-09-14T22:38:25.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8599999999999993e-06 2024-09-14T22:38:25.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755992+0000", 2024-09-14T22:38:25.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.361e-05 2024-09-14T22:38:25.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.756001+0000", 2024-09-14T22:38:25.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.422e-06 2024-09-14T22:38:25.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.756069+0000", 2024-09-14T22:38:25.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8209999999999999e-05 2024-09-14T22:38:25.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:25.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.756077+0000", 2024-09-14T22:38:25.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.994e-06 2024-09-14T22:38:25.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727872+0000", 2024-09-14T22:38:25.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.97179484999999999 2024-09-14T22:38:25.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.171 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:25.171 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727885+0000", 2024-09-14T22:38:25.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3101e-05 2024-09-14T22:38:25.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.171 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.171 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.740559+0000", 2024-09-14T22:38:25.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0126736300000001 2024-09-14T22:38:25.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.171 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.740777+0000", 2024-09-14T22:38:25.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021822000000000001 2024-09-14T22:38:25.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 2", 2024-09-14T22:38:25.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.741114+0000", 2024-09-14T22:38:25.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000336793 2024-09-14T22:38:25.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:25.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.742668+0000", 2024-09-14T22:38:25.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001554061 2024-09-14T22:38:25.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:38:25.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.742947+0000", 2024-09-14T22:38:25.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027937200000000001 2024-09-14T22:38:25.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:38:25.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743030+0000", 2024-09-14T22:38:25.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3385000000000002e-05 2024-09-14T22:38:25.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743223+0000", 2024-09-14T22:38:25.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00019210700000000001 2024-09-14T22:38:25.174 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.174 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.174 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.174 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:148474 3.18 3:1f6f7244:::rbd_header.113c827b523f:head [watch reconnect cookie 140250011614800 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104)", 2024-09-14T22:38:25.174 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:56.728273+0000", 2024-09-14T22:38:25.174 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.351998515999998, 2024-09-14T22:38:25.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0180497770000001, 2024-09-14T22:38:25.174 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.174 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.174 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.175 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.175 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.175 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.175 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 148474 2024-09-14T22:38:25.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.175 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.175 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.175 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728273+0000", 2024-09-14T22:38:25.175 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.175 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.175 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728273+0000", 2024-09-14T22:38:25.175 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728274+0000", 2024-09-14T22:38:25.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7570000000000001e-06 2024-09-14T22:38:25.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728281+0000", 2024-09-14T22:38:25.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5089999999999999e-06 2024-09-14T22:38:25.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728298+0000", 2024-09-14T22:38:25.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6727000000000001e-05 2024-09-14T22:38:25.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728309+0000", 2024-09-14T22:38:25.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1021e-05 2024-09-14T22:38:25.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.730078+0000", 2024-09-14T22:38:25.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017694010000000001 2024-09-14T22:38:25.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:25.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.730096+0000", 2024-09-14T22:38:25.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8039000000000001e-05 2024-09-14T22:38:25.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.741672+0000", 2024-09-14T22:38:25.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.011575817 2024-09-14T22:38:25.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2024-09-14T22:38:25.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.741827+0000", 2024-09-14T22:38:25.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000155563 2024-09-14T22:38:25.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743299+0000", 2024-09-14T22:38:25.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0014712729999999999 2024-09-14T22:38:25.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743415+0000", 2024-09-14T22:38:25.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011594700000000001 2024-09-14T22:38:25.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.746322+0000", 2024-09-14T22:38:25.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0029077230000000001 2024-09-14T22:38:25.181 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.181 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.181 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.181 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.181 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.181 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:25.183 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:38:25.183 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:25.181+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:25.284 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:25.281+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:25.379 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:25.380 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:25.380 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:38:25.380 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:25.385 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:25.381+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:25.394 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:38:25.485 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:25.478+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:25.570 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:25.570 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:25.570 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:38:25.570 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:38:25.570 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:25.584 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:38:25.586 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:25.585+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:25.687 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:25.682+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:25.708 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:25.708 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:38:25.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:38:25.708 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:38:25.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.708 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:31370 3.0 3:02f48f58:::rbd_data.113cda0183a0.0000000000000000:6 [read 813056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:38:25.708 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:38:25.708 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 112.342248782, 2024-09-14T22:38:25.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047519796000000003, 2024-09-14T22:38:25.709 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.709 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.709 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.709 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.709 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.709 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.709 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 31370 2024-09-14T22:38:25.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.709 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:38:25.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:38:25.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:38:25.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0999999999999999e-07 2024-09-14T22:38:25.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362401+0000", 2024-09-14T22:38:25.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2900000000000001e-06 2024-09-14T22:38:25.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362406+0000", 2024-09-14T22:38:25.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6960000000000002e-06 2024-09-14T22:38:25.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362412+0000", 2024-09-14T22:38:25.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.823e-06 2024-09-14T22:38:25.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.400270+0000", 2024-09-14T22:38:25.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037857737000000002 2024-09-14T22:38:25.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.400342+0000", 2024-09-14T22:38:25.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1797000000000004e-05 2024-09-14T22:38:25.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.409918+0000", 2024-09-14T22:38:25.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0095760429999999994 2024-09-14T22:38:25.713 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.713 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.713 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.713 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:113947 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 5603328~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:25.713 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.923453+0000", 2024-09-14T22:38:25.713 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 56.781193649999999, 2024-09-14T22:38:25.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047310420999999998, 2024-09-14T22:38:25.713 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.713 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.713 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.713 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.714 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.714 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.714 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 113947 2024-09-14T22:38:25.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.714 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.923453+0000", 2024-09-14T22:38:25.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.923453+0000", 2024-09-14T22:38:25.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926004+0000", 2024-09-14T22:38:25.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002551382 2024-09-14T22:38:25.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926009+0000", 2024-09-14T22:38:25.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5639999999999998e-06 2024-09-14T22:38:25.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926022+0000", 2024-09-14T22:38:25.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2734999999999999e-05 2024-09-14T22:38:25.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926029+0000", 2024-09-14T22:38:25.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2520000000000002e-06 2024-09-14T22:38:25.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.970650+0000", 2024-09-14T22:38:25.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044621277000000001 2024-09-14T22:38:25.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.970700+0000", 2024-09-14T22:38:25.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9713000000000003e-05 2024-09-14T22:38:25.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.717 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.970763+0000", 2024-09-14T22:38:25.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3497999999999994e-05 2024-09-14T22:38:25.717 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.717 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.717 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.717 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114136 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3448832~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:25.718 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932179+0000", 2024-09-14T22:38:25.718 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 56.772467284000001, 2024-09-14T22:38:25.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047236017999999998, 2024-09-14T22:38:25.718 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.718 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.718 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.718 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.718 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.718 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.718 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114136 2024-09-14T22:38:25.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.718 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932179+0000", 2024-09-14T22:38:25.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932179+0000", 2024-09-14T22:38:25.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932324+0000", 2024-09-14T22:38:25.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014451000000000001 2024-09-14T22:38:25.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932328+0000", 2024-09-14T22:38:25.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5889999999999996e-06 2024-09-14T22:38:25.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932338+0000", 2024-09-14T22:38:25.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3000000000000007e-06 2024-09-14T22:38:25.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932344+0000", 2024-09-14T22:38:25.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7150000000000001e-06 2024-09-14T22:38:25.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979233+0000", 2024-09-14T22:38:25.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046888711 2024-09-14T22:38:25.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979313+0000", 2024-09-14T22:38:25.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9487000000000007e-05 2024-09-14T22:38:25.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979415+0000", 2024-09-14T22:38:25.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000102706 2024-09-14T22:38:25.722 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.722 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.722 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.722 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114138 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3497984~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:25.722 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932356+0000", 2024-09-14T22:38:25.722 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 56.772290398000003, 2024-09-14T22:38:25.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047285958000000003, 2024-09-14T22:38:25.722 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.722 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.722 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.722 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.723 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.723 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.723 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114138 2024-09-14T22:38:25.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.723 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932356+0000", 2024-09-14T22:38:25.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932356+0000", 2024-09-14T22:38:25.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932498+0000", 2024-09-14T22:38:25.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000142182 2024-09-14T22:38:25.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932503+0000", 2024-09-14T22:38:25.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.301e-06 2024-09-14T22:38:25.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932514+0000", 2024-09-14T22:38:25.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1484000000000001e-05 2024-09-14T22:38:25.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932521+0000", 2024-09-14T22:38:25.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0740000000000004e-06 2024-09-14T22:38:25.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979451+0000", 2024-09-14T22:38:25.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046929379 2024-09-14T22:38:25.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979535+0000", 2024-09-14T22:38:25.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4423000000000001e-05 2024-09-14T22:38:25.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979642+0000", 2024-09-14T22:38:25.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010711499999999999 2024-09-14T22:38:25.726 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.726 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.726 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.727 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114139 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3547136~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:25.727 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932554+0000", 2024-09-14T22:38:25.727 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 56.772092631, 2024-09-14T22:38:25.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047292230999999997, 2024-09-14T22:38:25.727 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.727 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.727 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.727 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.727 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.727 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.727 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114139 2024-09-14T22:38:25.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.727 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932554+0000", 2024-09-14T22:38:25.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932554+0000", 2024-09-14T22:38:25.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932819+0000", 2024-09-14T22:38:25.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026538400000000002 2024-09-14T22:38:25.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932825+0000", 2024-09-14T22:38:25.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3469999999999998e-06 2024-09-14T22:38:25.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932836+0000", 2024-09-14T22:38:25.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1e-05 2024-09-14T22:38:25.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932843+0000", 2024-09-14T22:38:25.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1790000000000002e-06 2024-09-14T22:38:25.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979673+0000", 2024-09-14T22:38:25.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046830389 2024-09-14T22:38:25.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979750+0000", 2024-09-14T22:38:25.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6312999999999999e-05 2024-09-14T22:38:25.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.731 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.731 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979846+0000", 2024-09-14T22:38:25.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6619000000000004e-05 2024-09-14T22:38:25.731 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.731 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.731 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.731 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114141 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3596288~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:25.731 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932855+0000", 2024-09-14T22:38:25.731 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 56.771792032999997, 2024-09-14T22:38:25.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047201870999999999, 2024-09-14T22:38:25.731 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.731 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.732 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.732 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.732 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.732 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.732 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114141 2024-09-14T22:38:25.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.732 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932855+0000", 2024-09-14T22:38:25.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932855+0000", 2024-09-14T22:38:25.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934033+0000", 2024-09-14T22:38:25.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001178264 2024-09-14T22:38:25.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934038+0000", 2024-09-14T22:38:25.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4820000000000002e-06 2024-09-14T22:38:25.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934048+0000", 2024-09-14T22:38:25.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9890000000000004e-06 2024-09-14T22:38:25.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934055+0000", 2024-09-14T22:38:25.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8329999999999996e-06 2024-09-14T22:38:25.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979878+0000", 2024-09-14T22:38:25.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045822773999999997 2024-09-14T22:38:25.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979958+0000", 2024-09-14T22:38:25.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0477999999999998e-05 2024-09-14T22:38:25.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.980056+0000", 2024-09-14T22:38:25.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8051e-05 2024-09-14T22:38:25.735 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.735 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.735 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.736 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114159 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3940352~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:25.736 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.934179+0000", 2024-09-14T22:38:25.736 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 56.770467996999997, 2024-09-14T22:38:25.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047341439999999999, 2024-09-14T22:38:25.736 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.736 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.736 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.736 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.736 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.736 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.736 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114159 2024-09-14T22:38:25.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.737 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934179+0000", 2024-09-14T22:38:25.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934179+0000", 2024-09-14T22:38:25.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935021+0000", 2024-09-14T22:38:25.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00084276800000000003 2024-09-14T22:38:25.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935027+0000", 2024-09-14T22:38:25.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2959999999999998e-06 2024-09-14T22:38:25.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935038+0000", 2024-09-14T22:38:25.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1198e-05 2024-09-14T22:38:25.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935054+0000", 2024-09-14T22:38:25.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5764e-05 2024-09-14T22:38:25.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.981341+0000", 2024-09-14T22:38:25.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.04628782 2024-09-14T22:38:25.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.981415+0000", 2024-09-14T22:38:25.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3349999999999994e-05 2024-09-14T22:38:25.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.981520+0000", 2024-09-14T22:38:25.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105244 2024-09-14T22:38:25.740 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.740 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.740 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.740 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114180 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4136960~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:25.740 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:38:25.740 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 56.769468461000002, 2024-09-14T22:38:25.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047190621000000002, 2024-09-14T22:38:25.740 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.741 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.741 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.741 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.741 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.741 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.741 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114180 2024-09-14T22:38:25.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.741 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:38:25.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:38:25.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:38:25.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6800000000000002e-07 2024-09-14T22:38:25.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935180+0000", 2024-09-14T22:38:25.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.42e-06 2024-09-14T22:38:25.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935185+0000", 2024-09-14T22:38:25.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1359999999999996e-06 2024-09-14T22:38:25.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935188+0000", 2024-09-14T22:38:25.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.711e-06 2024-09-14T22:38:25.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982201+0000", 2024-09-14T22:38:25.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047013309000000003 2024-09-14T22:38:25.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982278+0000", 2024-09-14T22:38:25.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6705000000000005e-05 2024-09-14T22:38:25.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982369+0000", 2024-09-14T22:38:25.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1071999999999998e-05 2024-09-14T22:38:25.744 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.744 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.745 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.745 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114182 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4186112~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:25.745 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:38:25.745 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 56.769439839, 2024-09-14T22:38:25.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047368990999999999, 2024-09-14T22:38:25.745 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.745 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.745 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.745 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.745 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.745 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.746 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114182 2024-09-14T22:38:25.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.746 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:38:25.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:38:25.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:38:25.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7099999999999998e-07 2024-09-14T22:38:25.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935209+0000", 2024-09-14T22:38:25.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0260000000000002e-06 2024-09-14T22:38:25.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935213+0000", 2024-09-14T22:38:25.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3429999999999998e-06 2024-09-14T22:38:25.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935217+0000", 2024-09-14T22:38:25.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2480000000000001e-06 2024-09-14T22:38:25.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982406+0000", 2024-09-14T22:38:25.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047188920000000002 2024-09-14T22:38:25.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982483+0000", 2024-09-14T22:38:25.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7008999999999994e-05 2024-09-14T22:38:25.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982576+0000", 2024-09-14T22:38:25.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3173999999999999e-05 2024-09-14T22:38:25.749 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.749 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.749 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.749 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114185 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4202496~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:25.750 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.935221+0000", 2024-09-14T22:38:25.750 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 56.769425673999997, 2024-09-14T22:38:25.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047601128999999999, 2024-09-14T22:38:25.750 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.750 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.750 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.750 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.750 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.750 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.750 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114185 2024-09-14T22:38:25.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.750 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935221+0000", 2024-09-14T22:38:25.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935221+0000", 2024-09-14T22:38:25.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935970+0000", 2024-09-14T22:38:25.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074930899999999998 2024-09-14T22:38:25.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935976+0000", 2024-09-14T22:38:25.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5439999999999998e-06 2024-09-14T22:38:25.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935988+0000", 2024-09-14T22:38:25.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1779e-05 2024-09-14T22:38:25.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935996+0000", 2024-09-14T22:38:25.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.106e-06 2024-09-14T22:38:25.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982607+0000", 2024-09-14T22:38:25.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046611504999999998 2024-09-14T22:38:25.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982686+0000", 2024-09-14T22:38:25.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8805000000000002e-05 2024-09-14T22:38:25.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982822+0000", 2024-09-14T22:38:25.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000136081 2024-09-14T22:38:25.754 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.754 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.754 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.754 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114189 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4300800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:25.754 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.936048+0000", 2024-09-14T22:38:25.754 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 56.768598541999999, 2024-09-14T22:38:25.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047231539000000003, 2024-09-14T22:38:25.754 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.754 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.754 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.755 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.755 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.755 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.755 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114189 2024-09-14T22:38:25.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.755 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936048+0000", 2024-09-14T22:38:25.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936048+0000", 2024-09-14T22:38:25.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936049+0000", 2024-09-14T22:38:25.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4099999999999999e-07 2024-09-14T22:38:25.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936052+0000", 2024-09-14T22:38:25.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0029999999999999e-06 2024-09-14T22:38:25.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936058+0000", 2024-09-14T22:38:25.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7880000000000001e-06 2024-09-14T22:38:25.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936064+0000", 2024-09-14T22:38:25.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2279999999999998e-06 2024-09-14T22:38:25.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983120+0000", 2024-09-14T22:38:25.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047056384999999999 2024-09-14T22:38:25.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983194+0000", 2024-09-14T22:38:25.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3811000000000002e-05 2024-09-14T22:38:25.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983280+0000", 2024-09-14T22:38:25.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5883000000000004e-05 2024-09-14T22:38:25.758 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.758 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.758 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.759 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114191 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4349952~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:25.759 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:38:25.759 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 56.768574802000003, 2024-09-14T22:38:25.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047365365, 2024-09-14T22:38:25.759 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.759 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.759 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.759 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.759 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.759 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.759 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114191 2024-09-14T22:38:25.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.760 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:38:25.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:38:25.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:38:25.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0499999999999999e-07 2024-09-14T22:38:25.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936075+0000", 2024-09-14T22:38:25.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.734e-06 2024-09-14T22:38:25.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936082+0000", 2024-09-14T22:38:25.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8360000000000003e-06 2024-09-14T22:38:25.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936087+0000", 2024-09-14T22:38:25.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3129999999999998e-06 2024-09-14T22:38:25.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983319+0000", 2024-09-14T22:38:25.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047232140999999998 2024-09-14T22:38:25.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983365+0000", 2024-09-14T22:38:25.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5776999999999999e-05 2024-09-14T22:38:25.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983437+0000", 2024-09-14T22:38:25.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2158999999999998e-05 2024-09-14T22:38:25.763 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.763 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.763 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.763 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114193 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4399104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:25.763 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.936095+0000", 2024-09-14T22:38:25.763 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 56.768551866000003, 2024-09-14T22:38:25.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047505915000000003, 2024-09-14T22:38:25.764 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.764 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.764 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.764 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.764 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.764 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.764 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114193 2024-09-14T22:38:25.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.764 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936095+0000", 2024-09-14T22:38:25.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936095+0000", 2024-09-14T22:38:25.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938021+0000", 2024-09-14T22:38:25.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001926178 2024-09-14T22:38:25.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938027+0000", 2024-09-14T22:38:25.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3110000000000002e-06 2024-09-14T22:38:25.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938041+0000", 2024-09-14T22:38:25.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3344e-05 2024-09-14T22:38:25.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938049+0000", 2024-09-14T22:38:25.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2279999999999998e-06 2024-09-14T22:38:25.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983468+0000", 2024-09-14T22:38:25.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045419494999999997 2024-09-14T22:38:25.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983524+0000", 2024-09-14T22:38:25.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5952000000000002e-05 2024-09-14T22:38:25.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983601+0000", 2024-09-14T22:38:25.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6407000000000002e-05 2024-09-14T22:38:25.767 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.767 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.768 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.768 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114710 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4857856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:25.768 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.962571+0000", 2024-09-14T22:38:25.768 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 56.742075593999999, 2024-09-14T22:38:25.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047253314999999997, 2024-09-14T22:38:25.768 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.768 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.768 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.768 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.768 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.768 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.769 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114710 2024-09-14T22:38:25.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.769 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962571+0000", 2024-09-14T22:38:25.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962571+0000", 2024-09-14T22:38:25.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962572+0000", 2024-09-14T22:38:25.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1399999999999997e-07 2024-09-14T22:38:25.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962574+0000", 2024-09-14T22:38:25.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8679999999999999e-06 2024-09-14T22:38:25.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962583+0000", 2024-09-14T22:38:25.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8310000000000007e-06 2024-09-14T22:38:25.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962589+0000", 2024-09-14T22:38:25.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9529999999999996e-06 2024-09-14T22:38:25.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009692+0000", 2024-09-14T22:38:25.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047103249999999999 2024-09-14T22:38:25.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009767+0000", 2024-09-14T22:38:25.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4672000000000006e-05 2024-09-14T22:38:25.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009824+0000", 2024-09-14T22:38:25.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7226999999999999e-05 2024-09-14T22:38:25.772 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.772 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.772 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.772 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114712 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4907008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:25.772 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:38:25.773 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 56.742050114000001, 2024-09-14T22:38:25.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047421087000000001, 2024-09-14T22:38:25.773 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.773 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.773 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.773 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.773 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.773 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.773 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114712 2024-09-14T22:38:25.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.773 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:38:25.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:38:25.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:38:25.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.03e-07 2024-09-14T22:38:25.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962599+0000", 2024-09-14T22:38:25.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3199999999999998e-06 2024-09-14T22:38:25.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962606+0000", 2024-09-14T22:38:25.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5289999999999997e-06 2024-09-14T22:38:25.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962612+0000", 2024-09-14T22:38:25.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.947e-06 2024-09-14T22:38:25.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009866+0000", 2024-09-14T22:38:25.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047254681999999999 2024-09-14T22:38:25.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009924+0000", 2024-09-14T22:38:25.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7556000000000003e-05 2024-09-14T22:38:25.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010018+0000", 2024-09-14T22:38:25.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.365e-05 2024-09-14T22:38:25.776 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.777 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.777 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.777 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114714 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4956160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:25.777 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.962618+0000", 2024-09-14T22:38:25.777 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 56.742028595999997, 2024-09-14T22:38:25.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.04759302, 2024-09-14T22:38:25.777 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.777 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.777 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.777 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.777 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.778 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.778 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114714 2024-09-14T22:38:25.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.778 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962618+0000", 2024-09-14T22:38:25.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962618+0000", 2024-09-14T22:38:25.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966011+0000", 2024-09-14T22:38:25.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003393067 2024-09-14T22:38:25.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966016+0000", 2024-09-14T22:38:25.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3580000000000002e-06 2024-09-14T22:38:25.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966032+0000", 2024-09-14T22:38:25.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5356e-05 2024-09-14T22:38:25.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966040+0000", 2024-09-14T22:38:25.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1980000000000001e-06 2024-09-14T22:38:25.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010055+0000", 2024-09-14T22:38:25.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044014676000000003 2024-09-14T22:38:25.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010127+0000", 2024-09-14T22:38:25.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2206000000000006e-05 2024-09-14T22:38:25.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010211+0000", 2024-09-14T22:38:25.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4159000000000005e-05 2024-09-14T22:38:25.781 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.781 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.781 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.781 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114754 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 6283264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:25.782 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:38:25.782 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 56.737853682999997, 2024-09-14T22:38:25.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047228150000000003, 2024-09-14T22:38:25.782 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.782 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.782 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:25.782 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.782 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.782 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.782 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114754 2024-09-14T22:38:25.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.782 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:38:25.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:38:25.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:38:25.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7399999999999999e-07 2024-09-14T22:38:25.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966796+0000", 2024-09-14T22:38:25.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3630000000000001e-06 2024-09-14T22:38:25.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966802+0000", 2024-09-14T22:38:25.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4679999999999997e-06 2024-09-14T22:38:25.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966807+0000", 2024-09-14T22:38:25.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1109999999999999e-06 2024-09-14T22:38:25.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.013929+0000", 2024-09-14T22:38:25.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047121720999999998 2024-09-14T22:38:25.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.013968+0000", 2024-09-14T22:38:25.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8877000000000001e-05 2024-09-14T22:38:25.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014021+0000", 2024-09-14T22:38:25.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3236000000000003e-05 2024-09-14T22:38:25.786 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.786 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.786 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.786 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114789 3.19 3:9836bb25:::rbd_object_map.113c14c67864.000000000000001b:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:25.786 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.966813+0000", 2024-09-14T22:38:25.786 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 56.737833221000002, 2024-09-14T22:38:25.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05001311, 2024-09-14T22:38:25.786 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.786 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.786 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:25.786 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.787 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.787 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.787 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114789 2024-09-14T22:38:25.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.787 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966813+0000", 2024-09-14T22:38:25.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966813+0000", 2024-09-14T22:38:25.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966814+0000", 2024-09-14T22:38:25.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9200000000000002e-07 2024-09-14T22:38:25.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966817+0000", 2024-09-14T22:38:25.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8380000000000002e-06 2024-09-14T22:38:25.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966831+0000", 2024-09-14T22:38:25.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4209999999999999e-05 2024-09-14T22:38:25.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.789 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966845+0000", 2024-09-14T22:38:25.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3799e-05 2024-09-14T22:38:25.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.789 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014046+0000", 2024-09-14T22:38:25.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047201604000000001 2024-09-14T22:38:25.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014116+0000", 2024-09-14T22:38:25.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9387000000000005e-05 2024-09-14T22:38:25.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-14T22:38:25.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014469+0000", 2024-09-14T22:38:25.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00035355800000000002 2024-09-14T22:38:25.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:25.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016719+0000", 2024-09-14T22:38:25.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0022498459999999998 2024-09-14T22:38:25.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:38:25.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016735+0000", 2024-09-14T22:38:25.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5792999999999999e-05 2024-09-14T22:38:25.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:38:25.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016790+0000", 2024-09-14T22:38:25.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4715e-05 2024-09-14T22:38:25.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016827+0000", 2024-09-14T22:38:25.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6968000000000002e-05 2024-09-14T22:38:25.792 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.792 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.792 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.792 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114803 3.19 3:9836bb25:::rbd_object_map.113c14c67864.000000000000001b:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:25.793 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.966853+0000", 2024-09-14T22:38:25.793 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 56.737793791000001, 2024-09-14T22:38:25.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050656093999999999, 2024-09-14T22:38:25.793 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.793 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.793 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:25.793 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.793 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.793 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.793 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114803 2024-09-14T22:38:25.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.793 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966853+0000", 2024-09-14T22:38:25.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966853+0000", 2024-09-14T22:38:25.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969930+0000", 2024-09-14T22:38:25.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030769759999999999 2024-09-14T22:38:25.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.795 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969936+0000", 2024-09-14T22:38:25.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7150000000000003e-06 2024-09-14T22:38:25.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.795 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969949+0000", 2024-09-14T22:38:25.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3166000000000001e-05 2024-09-14T22:38:25.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.795 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969957+0000", 2024-09-14T22:38:25.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8269999999999998e-06 2024-09-14T22:38:25.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014996+0000", 2024-09-14T22:38:25.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045039202 2024-09-14T22:38:25.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.015085+0000", 2024-09-14T22:38:25.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8987999999999996e-05 2024-09-14T22:38:25.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-14T22:38:25.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.015318+0000", 2024-09-14T22:38:25.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023284999999999999 2024-09-14T22:38:25.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:25.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016898+0000", 2024-09-14T22:38:25.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0015807379999999999 2024-09-14T22:38:25.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:38:25.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.017455+0000", 2024-09-14T22:38:25.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00055664600000000005 2024-09-14T22:38:25.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:38:25.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.017483+0000", 2024-09-14T22:38:25.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8238e-05 2024-09-14T22:38:25.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.017509+0000", 2024-09-14T22:38:25.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5748000000000001e-05 2024-09-14T22:38:25.801 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.801 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.801 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.801 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:150759 3.8 3:15dc94e5:::rbd_header.113c88f88d4a:head [create,call rbd.create in=9b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e113)", 2024-09-14T22:38:25.801 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:38:05.726302+0000", 2024-09-14T22:38:25.801 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.978345098999998, 2024-09-14T22:38:25.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0241160819999999, 2024-09-14T22:38:25.801 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:25.801 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:25.802 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:25.802 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:25.802 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:25.802 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:25.802 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 150759 2024-09-14T22:38:25.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.802 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:25.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:25.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.726302+0000", 2024-09-14T22:38:25.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:25.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.726302+0000", 2024-09-14T22:38:25.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:25.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:25.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.726304+0000", 2024-09-14T22:38:25.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.075e-06 2024-09-14T22:38:25.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:25.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.726312+0000", 2024-09-14T22:38:25.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2050000000000002e-06 2024-09-14T22:38:25.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:25.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.727969+0000", 2024-09-14T22:38:25.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001657499 2024-09-14T22:38:25.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:25.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.727985+0000", 2024-09-14T22:38:25.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6082999999999998e-05 2024-09-14T22:38:25.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.728193+0000", 2024-09-14T22:38:25.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000208082 2024-09-14T22:38:25.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:25.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.728205+0000", 2024-09-14T22:38:25.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1897e-05 2024-09-14T22:38:25.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:25.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.741926+0000", 2024-09-14T22:38:25.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.013720591 2024-09-14T22:38:25.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:25.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.742076+0000", 2024-09-14T22:38:25.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015024099999999999 2024-09-14T22:38:25.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-14T22:38:25.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.742415+0000", 2024-09-14T22:38:25.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00033887700000000002 2024-09-14T22:38:25.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:38:25.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.743436+0000", 2024-09-14T22:38:25.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010210639999999999 2024-09-14T22:38:25.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:25.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.750216+0000", 2024-09-14T22:38:25.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0067798950000000002 2024-09-14T22:38:25.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:38:25.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.750288+0000", 2024-09-14T22:38:25.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2192000000000003e-05 2024-09-14T22:38:25.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:25.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:25.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:25.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.750418+0000", 2024-09-14T22:38:25.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000129381 2024-09-14T22:38:25.807 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.807 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.807 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.808 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:25.808 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:25.808 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:25.809 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:38:25.810 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:25.802+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:25.910 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:25.909+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:25.948 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:25.949 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:25.949 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:38:25.949 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:25.963 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:38:26.010 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:26.002+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:26.107 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-14T22:38:26.107 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:38:26.107 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-14T22:38:26.107 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-14T22:38:26.111 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:26.109+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:26.116 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T22:38:26.133 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:26.133 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:26.133 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:38:26.133 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:38:26.133 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:26.149 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:38:26.212 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:26.206+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:26.289 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-14T22:38:26.300 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:38:26.313 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:26.306+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:26.329 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:26.329 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:38:26.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:38:26.329 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:38:26.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.329 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145249 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 15745024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:26.329 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.693850+0000", 2024-09-14T22:38:26.329 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.633597694000002, 2024-09-14T22:38:26.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.060850297999999997, 2024-09-14T22:38:26.330 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:26.330 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:26.330 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:26.330 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:26.330 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:26.330 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:26.330 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145249 2024-09-14T22:38:26.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.330 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:26.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:26.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693850+0000", 2024-09-14T22:38:26.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:26.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693850+0000", 2024-09-14T22:38:26.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:26.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693851+0000", 2024-09-14T22:38:26.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.46e-07 2024-09-14T22:38:26.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:26.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693853+0000", 2024-09-14T22:38:26.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0940000000000002e-06 2024-09-14T22:38:26.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:26.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693858+0000", 2024-09-14T22:38:26.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2100000000000001e-06 2024-09-14T22:38:26.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:26.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693863+0000", 2024-09-14T22:38:26.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5549999999999996e-06 2024-09-14T22:38:26.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:26.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754503+0000", 2024-09-14T22:38:26.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.060640558999999997 2024-09-14T22:38:26.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:26.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754613+0000", 2024-09-14T22:38:26.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109966 2024-09-14T22:38:26.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:26.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754701+0000", 2024-09-14T22:38:26.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7567999999999999e-05 2024-09-14T22:38:26.333 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.333 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:26.334 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.334 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145250 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16039936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:26.334 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.693868+0000", 2024-09-14T22:38:26.334 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.633579965999999, 2024-09-14T22:38:26.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061182667000000003, 2024-09-14T22:38:26.334 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:26.334 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:26.334 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:26.334 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:26.334 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:26.334 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:26.334 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145250 2024-09-14T22:38:26.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.335 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:26.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:26.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693868+0000", 2024-09-14T22:38:26.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:26.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693868+0000", 2024-09-14T22:38:26.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:26.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693869+0000", 2024-09-14T22:38:26.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4400000000000001e-07 2024-09-14T22:38:26.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:26.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693870+0000", 2024-09-14T22:38:26.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9130000000000001e-06 2024-09-14T22:38:26.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:26.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693876+0000", 2024-09-14T22:38:26.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1270000000000002e-06 2024-09-14T22:38:26.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:26.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693880+0000", 2024-09-14T22:38:26.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.673e-06 2024-09-14T22:38:26.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:26.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754780+0000", 2024-09-14T22:38:26.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.060899495999999997 2024-09-14T22:38:26.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:26.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754925+0000", 2024-09-14T22:38:26.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014511799999999999 2024-09-14T22:38:26.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:26.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755051+0000", 2024-09-14T22:38:26.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012599599999999999 2024-09-14T22:38:26.338 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.338 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:26.338 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.338 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145251 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16236544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:26.338 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.693886+0000", 2024-09-14T22:38:26.338 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.633562312999999, 2024-09-14T22:38:26.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061498816999999997, 2024-09-14T22:38:26.339 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:26.339 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:26.339 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:26.339 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:26.339 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:26.339 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:26.339 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145251 2024-09-14T22:38:26.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.339 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:26.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:26.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693886+0000", 2024-09-14T22:38:26.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:26.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693886+0000", 2024-09-14T22:38:26.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:26.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693886+0000", 2024-09-14T22:38:26.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1199999999999998e-07 2024-09-14T22:38:26.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:26.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693888+0000", 2024-09-14T22:38:26.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.877e-06 2024-09-14T22:38:26.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:26.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693894+0000", 2024-09-14T22:38:26.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5350000000000004e-06 2024-09-14T22:38:26.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:26.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693917+0000", 2024-09-14T22:38:26.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.296e-05 2024-09-14T22:38:26.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:26.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755127+0000", 2024-09-14T22:38:26.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061210286000000003 2024-09-14T22:38:26.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:26.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755266+0000", 2024-09-14T22:38:26.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013871599999999999 2024-09-14T22:38:26.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:26.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755385+0000", 2024-09-14T22:38:26.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011903100000000001 2024-09-14T22:38:26.342 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.342 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:26.342 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.343 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145252 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16302080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:26.343 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.693985+0000", 2024-09-14T22:38:26.343 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.633463630999998, 2024-09-14T22:38:26.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061743767999999997, 2024-09-14T22:38:26.343 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:26.343 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:26.343 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:26.343 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:26.343 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:26.343 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:26.343 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145252 2024-09-14T22:38:26.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.344 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:26.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:26.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693985+0000", 2024-09-14T22:38:26.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:26.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693985+0000", 2024-09-14T22:38:26.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:26.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693986+0000", 2024-09-14T22:38:26.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0640000000000001e-06 2024-09-14T22:38:26.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:26.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693991+0000", 2024-09-14T22:38:26.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7599999999999999e-06 2024-09-14T22:38:26.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:26.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694005+0000", 2024-09-14T22:38:26.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3271999999999999e-05 2024-09-14T22:38:26.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:26.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694015+0000", 2024-09-14T22:38:26.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0617e-05 2024-09-14T22:38:26.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:26.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755459+0000", 2024-09-14T22:38:26.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061443820000000003 2024-09-14T22:38:26.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:26.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755605+0000", 2024-09-14T22:38:26.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014569 2024-09-14T22:38:26.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:26.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755728+0000", 2024-09-14T22:38:26.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000123545 2024-09-14T22:38:26.347 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.347 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:26.347 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.347 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145253 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16138240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:26.347 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694031+0000", 2024-09-14T22:38:26.347 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.633417076999997, 2024-09-14T22:38:26.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062032836000000001, 2024-09-14T22:38:26.348 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:26.348 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:26.348 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:26.348 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:26.348 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:26.348 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:26.348 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145253 2024-09-14T22:38:26.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.348 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:26.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:26.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694031+0000", 2024-09-14T22:38:26.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:26.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694031+0000", 2024-09-14T22:38:26.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:26.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694032+0000", 2024-09-14T22:38:26.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7999999999999995e-07 2024-09-14T22:38:26.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:26.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694036+0000", 2024-09-14T22:38:26.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8800000000000001e-06 2024-09-14T22:38:26.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:26.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694047+0000", 2024-09-14T22:38:26.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0835999999999999e-05 2024-09-14T22:38:26.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:26.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694055+0000", 2024-09-14T22:38:26.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6619999999999994e-06 2024-09-14T22:38:26.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:26.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755804+0000", 2024-09-14T22:38:26.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061748448999999997 2024-09-14T22:38:26.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:26.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755941+0000", 2024-09-14T22:38:26.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013751599999999999 2024-09-14T22:38:26.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:26.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756064+0000", 2024-09-14T22:38:26.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012281299999999999 2024-09-14T22:38:26.351 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.351 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:26.351 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.352 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145254 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16400384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:26.352 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694067+0000", 2024-09-14T22:38:26.352 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.633380912, 2024-09-14T22:38:26.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062280549999999997, 2024-09-14T22:38:26.352 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:26.352 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:26.352 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:26.352 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:26.352 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:26.352 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:26.352 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145254 2024-09-14T22:38:26.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.353 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:26.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:26.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694067+0000", 2024-09-14T22:38:26.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:26.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694067+0000", 2024-09-14T22:38:26.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:26.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694068+0000", 2024-09-14T22:38:26.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2200000000000003e-07 2024-09-14T22:38:26.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:26.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694072+0000", 2024-09-14T22:38:26.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5729999999999999e-06 2024-09-14T22:38:26.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:26.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694082+0000", 2024-09-14T22:38:26.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0703999999999999e-05 2024-09-14T22:38:26.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:26.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694091+0000", 2024-09-14T22:38:26.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8030000000000003e-06 2024-09-14T22:38:26.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:26.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756155+0000", 2024-09-14T22:38:26.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062063937 2024-09-14T22:38:26.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:26.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756263+0000", 2024-09-14T22:38:26.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108229 2024-09-14T22:38:26.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:26.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756348+0000", 2024-09-14T22:38:26.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4581999999999998e-05 2024-09-14T22:38:26.356 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.356 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:26.356 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.356 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145255 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16596992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:26.357 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694103+0000", 2024-09-14T22:38:26.357 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.633345431000002, 2024-09-14T22:38:26.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062580620000000003, 2024-09-14T22:38:26.357 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:26.357 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:26.357 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:26.357 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:26.357 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:26.357 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:26.357 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145255 2024-09-14T22:38:26.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.357 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:26.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:26.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694103+0000", 2024-09-14T22:38:26.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:26.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694103+0000", 2024-09-14T22:38:26.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:26.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694103+0000", 2024-09-14T22:38:26.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7800000000000001e-07 2024-09-14T22:38:26.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:26.359 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694107+0000", 2024-09-14T22:38:26.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5939999999999998e-06 2024-09-14T22:38:26.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:26.359 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694121+0000", 2024-09-14T22:38:26.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3913999999999999e-05 2024-09-14T22:38:26.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:26.359 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694130+0000", 2024-09-14T22:38:26.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7690000000000002e-06 2024-09-14T22:38:26.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:26.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756431+0000", 2024-09-14T22:38:26.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062301593000000002 2024-09-14T22:38:26.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:26.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756564+0000", 2024-09-14T22:38:26.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000132693 2024-09-14T22:38:26.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:26.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756683+0000", 2024-09-14T22:38:26.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119479 2024-09-14T22:38:26.361 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.361 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:26.361 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.361 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145256 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16695296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:26.361 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694142+0000", 2024-09-14T22:38:26.361 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.633306357999999, 2024-09-14T22:38:26.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062806034999999996, 2024-09-14T22:38:26.361 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:26.361 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:26.361 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:26.361 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:26.362 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:26.362 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:26.362 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145256 2024-09-14T22:38:26.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.362 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:26.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:26.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694142+0000", 2024-09-14T22:38:26.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:26.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694142+0000", 2024-09-14T22:38:26.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:26.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694142+0000", 2024-09-14T22:38:26.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1699999999999998e-07 2024-09-14T22:38:26.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:26.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694146+0000", 2024-09-14T22:38:26.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7450000000000001e-06 2024-09-14T22:38:26.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:26.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694157+0000", 2024-09-14T22:38:26.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1161e-05 2024-09-14T22:38:26.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:26.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694166+0000", 2024-09-14T22:38:26.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0450000000000006e-06 2024-09-14T22:38:26.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:26.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756768+0000", 2024-09-14T22:38:26.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062601554000000004 2024-09-14T22:38:26.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:26.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756867+0000", 2024-09-14T22:38:26.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8741999999999998e-05 2024-09-14T22:38:26.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:26.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756948+0000", 2024-09-14T22:38:26.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1170999999999999e-05 2024-09-14T22:38:26.365 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.365 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:26.365 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.366 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145257 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16498688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:26.366 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694179+0000", 2024-09-14T22:38:26.366 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.633269059, 2024-09-14T22:38:26.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063118469999999996, 2024-09-14T22:38:26.366 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:26.366 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:26.366 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:26.366 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:26.366 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:26.366 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:26.366 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145257 2024-09-14T22:38:26.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.366 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:26.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:26.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694179+0000", 2024-09-14T22:38:26.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:26.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694179+0000", 2024-09-14T22:38:26.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:26.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694180+0000", 2024-09-14T22:38:26.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1099999999999995e-07 2024-09-14T22:38:26.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:26.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694196+0000", 2024-09-14T22:38:26.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5996000000000001e-05 2024-09-14T22:38:26.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:26.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694208+0000", 2024-09-14T22:38:26.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2584e-05 2024-09-14T22:38:26.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:26.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694219+0000", 2024-09-14T22:38:26.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0462000000000001e-05 2024-09-14T22:38:26.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:26.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757033+0000", 2024-09-14T22:38:26.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062814485000000003 2024-09-14T22:38:26.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:26.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757183+0000", 2024-09-14T22:38:26.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014914099999999999 2024-09-14T22:38:26.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:26.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757298+0000", 2024-09-14T22:38:26.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115091 2024-09-14T22:38:26.370 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.370 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:26.370 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.370 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145258 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16269312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:26.370 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694257+0000", 2024-09-14T22:38:26.370 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.633190865000003, 2024-09-14T22:38:26.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063297279999999997, 2024-09-14T22:38:26.370 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:26.370 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:26.371 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:26.371 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:26.371 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:26.371 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:26.371 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145258 2024-09-14T22:38:26.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.371 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:26.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.371 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:26.371 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694257+0000", 2024-09-14T22:38:26.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.371 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:26.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694257+0000", 2024-09-14T22:38:26.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:26.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694258+0000", 2024-09-14T22:38:26.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7700000000000004e-07 2024-09-14T22:38:26.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:26.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694263+0000", 2024-09-14T22:38:26.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6609999999999999e-06 2024-09-14T22:38:26.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:26.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694271+0000", 2024-09-14T22:38:26.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7809999999999994e-06 2024-09-14T22:38:26.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:26.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694277+0000", 2024-09-14T22:38:26.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.823e-06 2024-09-14T22:38:26.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:26.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757377+0000", 2024-09-14T22:38:26.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063100081000000002 2024-09-14T22:38:26.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:26.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757476+0000", 2024-09-14T22:38:26.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8507999999999999e-05 2024-09-14T22:38:26.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:26.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757555+0000", 2024-09-14T22:38:26.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8749000000000001e-05 2024-09-14T22:38:26.374 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.374 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:26.374 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.375 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145259 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16465920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:26.375 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694286+0000", 2024-09-14T22:38:26.375 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.633162667999997, 2024-09-14T22:38:26.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063608168000000007, 2024-09-14T22:38:26.375 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:26.375 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:26.375 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:26.375 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:26.375 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:26.375 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:26.375 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145259 2024-09-14T22:38:26.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.375 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:26.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:26.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694286+0000", 2024-09-14T22:38:26.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:26.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694286+0000", 2024-09-14T22:38:26.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:26.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694286+0000", 2024-09-14T22:38:26.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9999999999999998e-07 2024-09-14T22:38:26.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:26.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694288+0000", 2024-09-14T22:38:26.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3760000000000002e-06 2024-09-14T22:38:26.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:26.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694294+0000", 2024-09-14T22:38:26.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6779999999999997e-06 2024-09-14T22:38:26.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:26.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694298+0000", 2024-09-14T22:38:26.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3309999999999997e-06 2024-09-14T22:38:26.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:26.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757638+0000", 2024-09-14T22:38:26.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063339464999999998 2024-09-14T22:38:26.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:26.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757769+0000", 2024-09-14T22:38:26.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013148700000000001 2024-09-14T22:38:26.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:26.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757894+0000", 2024-09-14T22:38:26.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012443099999999999 2024-09-14T22:38:26.379 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.379 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:26.379 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.379 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145260 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16367616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:26.379 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694304+0000", 2024-09-14T22:38:26.379 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.633143873999998, 2024-09-14T22:38:26.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063838521999999995, 2024-09-14T22:38:26.379 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:26.379 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:26.379 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:26.380 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:26.380 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:26.380 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:26.380 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145260 2024-09-14T22:38:26.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.380 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:26.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:26.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694304+0000", 2024-09-14T22:38:26.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:26.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694304+0000", 2024-09-14T22:38:26.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:26.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694305+0000", 2024-09-14T22:38:26.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8299999999999997e-07 2024-09-14T22:38:26.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:26.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694307+0000", 2024-09-14T22:38:26.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9989999999999998e-06 2024-09-14T22:38:26.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:26.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694312+0000", 2024-09-14T22:38:26.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2689999999999999e-06 2024-09-14T22:38:26.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:26.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694317+0000", 2024-09-14T22:38:26.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9359999999999998e-06 2024-09-14T22:38:26.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:26.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757944+0000", 2024-09-14T22:38:26.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063626951000000001 2024-09-14T22:38:26.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:26.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758065+0000", 2024-09-14T22:38:26.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120945 2024-09-14T22:38:26.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:26.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758143+0000", 2024-09-14T22:38:26.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7838999999999998e-05 2024-09-14T22:38:26.383 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.383 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:26.383 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.383 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145261 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16564224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:26.384 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694323+0000", 2024-09-14T22:38:26.384 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.633125171000003, 2024-09-14T22:38:26.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064150180000000001, 2024-09-14T22:38:26.384 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:26.384 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:26.384 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:26.384 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:26.384 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:26.384 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:26.384 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145261 2024-09-14T22:38:26.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.384 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:26.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:26.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694323+0000", 2024-09-14T22:38:26.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.385 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:26.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694323+0000", 2024-09-14T22:38:26.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.385 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:26.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694323+0000", 2024-09-14T22:38:26.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4900000000000001e-07 2024-09-14T22:38:26.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:26.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694325+0000", 2024-09-14T22:38:26.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.04e-06 2024-09-14T22:38:26.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:26.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694331+0000", 2024-09-14T22:38:26.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3449999999999996e-06 2024-09-14T22:38:26.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:26.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694335+0000", 2024-09-14T22:38:26.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.566e-06 2024-09-14T22:38:26.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:26.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758225+0000", 2024-09-14T22:38:26.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063889535999999997 2024-09-14T22:38:26.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:26.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758357+0000", 2024-09-14T22:38:26.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013181 2024-09-14T22:38:26.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:26.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758473+0000", 2024-09-14T22:38:26.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011653400000000001 2024-09-14T22:38:26.388 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.388 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:26.388 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.388 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145262 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16760832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:26.388 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694341+0000", 2024-09-14T22:38:26.388 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.633107312999996, 2024-09-14T22:38:26.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064424631999999996, 2024-09-14T22:38:26.388 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:26.388 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:26.388 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:26.388 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:26.388 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:26.389 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:26.389 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145262 2024-09-14T22:38:26.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.389 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:26.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:26.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694341+0000", 2024-09-14T22:38:26.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:26.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694341+0000", 2024-09-14T22:38:26.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:26.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694341+0000", 2024-09-14T22:38:26.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5499999999999999e-07 2024-09-14T22:38:26.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:26.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694343+0000", 2024-09-14T22:38:26.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9630000000000001e-06 2024-09-14T22:38:26.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:26.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694349+0000", 2024-09-14T22:38:26.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3120000000000001e-06 2024-09-14T22:38:26.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:26.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694353+0000", 2024-09-14T22:38:26.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6500000000000004e-06 2024-09-14T22:38:26.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:26.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758561+0000", 2024-09-14T22:38:26.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064207934999999994 2024-09-14T22:38:26.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:26.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758677+0000", 2024-09-14T22:38:26.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116241 2024-09-14T22:38:26.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:26.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758766+0000", 2024-09-14T22:38:26.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8176000000000004e-05 2024-09-14T22:38:26.392 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.392 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:26.392 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.392 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145263 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16662528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:26.392 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694359+0000", 2024-09-14T22:38:26.393 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.633089470000002, 2024-09-14T22:38:26.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064743189000000007, 2024-09-14T22:38:26.393 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:26.393 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:26.393 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:26.393 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:26.393 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:26.393 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:26.393 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145263 2024-09-14T22:38:26.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.393 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:26.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:26.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694359+0000", 2024-09-14T22:38:26.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:26.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694359+0000", 2024-09-14T22:38:26.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:26.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694359+0000", 2024-09-14T22:38:26.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5100000000000001e-07 2024-09-14T22:38:26.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:26.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694361+0000", 2024-09-14T22:38:26.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9429999999999999e-06 2024-09-14T22:38:26.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:26.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694366+0000", 2024-09-14T22:38:26.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3489999999999999e-06 2024-09-14T22:38:26.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:26.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694371+0000", 2024-09-14T22:38:26.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4399999999999998e-06 2024-09-14T22:38:26.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:26.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758848+0000", 2024-09-14T22:38:26.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064477396000000006 2024-09-14T22:38:26.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:26.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758980+0000", 2024-09-14T22:38:26.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013162199999999999 2024-09-14T22:38:26.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:26.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759102+0000", 2024-09-14T22:38:26.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012208799999999999 2024-09-14T22:38:26.396 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.397 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:26.397 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.397 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145264 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16334848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:26.397 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694377+0000", 2024-09-14T22:38:26.397 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.633071436000002, 2024-09-14T22:38:26.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065048054999999994, 2024-09-14T22:38:26.397 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:26.397 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:26.397 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:26.397 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:26.397 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:26.397 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:26.398 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145264 2024-09-14T22:38:26.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.398 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:26.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:26.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694377+0000", 2024-09-14T22:38:26.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:26.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694377+0000", 2024-09-14T22:38:26.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:26.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694377+0000", 2024-09-14T22:38:26.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.39e-07 2024-09-14T22:38:26.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:26.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694379+0000", 2024-09-14T22:38:26.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0190000000000001e-06 2024-09-14T22:38:26.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:26.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694385+0000", 2024-09-14T22:38:26.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4399999999999996e-06 2024-09-14T22:38:26.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:26.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694389+0000", 2024-09-14T22:38:26.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.639e-06 2024-09-14T22:38:26.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:26.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759175+0000", 2024-09-14T22:38:26.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064785899999999993 2024-09-14T22:38:26.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:26.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759309+0000", 2024-09-14T22:38:26.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013374400000000001 2024-09-14T22:38:26.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:26.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759425+0000", 2024-09-14T22:38:26.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115974 2024-09-14T22:38:26.401 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.401 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:26.401 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.401 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145265 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16531456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:26.401 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694395+0000", 2024-09-14T22:38:26.401 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.633053314000001, 2024-09-14T22:38:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065298325000000004, 2024-09-14T22:38:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145265 2024-09-14T22:38:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694395+0000", 2024-09-14T22:38:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694395+0000", 2024-09-14T22:38:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694395+0000", 2024-09-14T22:38:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6699999999999999e-07 2024-09-14T22:38:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694397+0000", 2024-09-14T22:38:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.984e-06 2024-09-14T22:38:26.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:26.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694403+0000", 2024-09-14T22:38:26.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3519999999999997e-06 2024-09-14T22:38:26.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:26.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694407+0000", 2024-09-14T22:38:26.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3239999999999996e-06 2024-09-14T22:38:26.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:26.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759516+0000", 2024-09-14T22:38:26.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065109279000000006 2024-09-14T22:38:26.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:26.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759610+0000", 2024-09-14T22:38:26.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3566000000000005e-05 2024-09-14T22:38:26.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:26.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759693+0000", 2024-09-14T22:38:26.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3453000000000002e-05 2024-09-14T22:38:26.405 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.405 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:26.405 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.406 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145266 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16433152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:26.406 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694413+0000", 2024-09-14T22:38:26.406 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.633035515000003, 2024-09-14T22:38:26.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065618308, 2024-09-14T22:38:26.406 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:26.406 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:26.406 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:26.406 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:26.406 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:26.406 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:26.406 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145266 2024-09-14T22:38:26.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.406 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:26.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:26.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694413+0000", 2024-09-14T22:38:26.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:26.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694413+0000", 2024-09-14T22:38:26.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:26.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694413+0000", 2024-09-14T22:38:26.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4799999999999999e-07 2024-09-14T22:38:26.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:26.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694415+0000", 2024-09-14T22:38:26.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.08e-06 2024-09-14T22:38:26.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:26.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694420+0000", 2024-09-14T22:38:26.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.203e-06 2024-09-14T22:38:26.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:26.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694425+0000", 2024-09-14T22:38:26.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4649999999999996e-06 2024-09-14T22:38:26.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:26.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759768+0000", 2024-09-14T22:38:26.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065343613999999994 2024-09-14T22:38:26.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:26.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759909+0000", 2024-09-14T22:38:26.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014054099999999999 2024-09-14T22:38:26.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:26.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760031+0000", 2024-09-14T22:38:26.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122057 2024-09-14T22:38:26.410 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.410 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:26.410 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.410 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145267 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16629760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:26.410 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694431+0000", 2024-09-14T22:38:26.410 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.633017600999999, 2024-09-14T22:38:26.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065850237000000006, 2024-09-14T22:38:26.410 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:26.410 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:26.410 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:26.410 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:26.410 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:26.411 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:26.411 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145267 2024-09-14T22:38:26.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.411 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:26.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:26.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694431+0000", 2024-09-14T22:38:26.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:26.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694431+0000", 2024-09-14T22:38:26.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:26.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694431+0000", 2024-09-14T22:38:26.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4799999999999999e-07 2024-09-14T22:38:26.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:26.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694433+0000", 2024-09-14T22:38:26.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8470000000000001e-06 2024-09-14T22:38:26.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:26.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694438+0000", 2024-09-14T22:38:26.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3249999999999998e-06 2024-09-14T22:38:26.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:26.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694442+0000", 2024-09-14T22:38:26.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3490000000000002e-06 2024-09-14T22:38:26.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:26.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760113+0000", 2024-09-14T22:38:26.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065670872000000005 2024-09-14T22:38:26.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:26.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760199+0000", 2024-09-14T22:38:26.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5982000000000004e-05 2024-09-14T22:38:26.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:26.414 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760281+0000", 2024-09-14T22:38:26.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1514000000000006e-05 2024-09-14T22:38:26.414 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.414 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:26.414 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.414 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145268 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16728064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:26.414 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694448+0000", 2024-09-14T22:38:26.414 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.633000017000001, 2024-09-14T22:38:26.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066157585000000005, 2024-09-14T22:38:26.415 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:26.415 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:26.415 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:26.415 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:26.415 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:26.415 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:26.415 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145268 2024-09-14T22:38:26.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.415 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:26.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:26.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694448+0000", 2024-09-14T22:38:26.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:26.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694448+0000", 2024-09-14T22:38:26.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:26.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:26.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694449+0000", 2024-09-14T22:38:26.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.39e-07 2024-09-14T22:38:26.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:26.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694450+0000", 2024-09-14T22:38:26.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.906e-06 2024-09-14T22:38:26.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:26.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694456+0000", 2024-09-14T22:38:26.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3820000000000003e-06 2024-09-14T22:38:26.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:26.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694460+0000", 2024-09-14T22:38:26.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6290000000000001e-06 2024-09-14T22:38:26.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:26.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760345+0000", 2024-09-14T22:38:26.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065884540000000005 2024-09-14T22:38:26.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:26.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760482+0000", 2024-09-14T22:38:26.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013731600000000001 2024-09-14T22:38:26.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:26.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:26.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:26.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760606+0000", 2024-09-14T22:38:26.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000123473 2024-09-14T22:38:26.418 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.418 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:26.418 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.419 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:26.419 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:26.419 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:26.420 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:38:26.421 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:26.414+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:26.448 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-14T22:38:26.521 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:26.514+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:26.553 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:26.554 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:26.554 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:38:26.554 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:26.568 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:38:26.572 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:38:26.572 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:38:26.590 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:38:26.590 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:38:26.607 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:38:26.607 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:38:26.624 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:38:26.624 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:38:26.642 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:38:26.643 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:38:26.655 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:26.650+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:26.662 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-14T22:38:26.664 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:38:26.664 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:38:26.685 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:38:26.685 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:38:26.705 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:38:26.705 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:38:26.722 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:26.721+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:26.725 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2024-09-14T22:38:26.742 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:26.742 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:26.760 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:38:26.825 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:26.821+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:26.841 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-14T22:38:26.930 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:26.930 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:26.930 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:26.929+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:26.944 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:38:27.027 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:27.022+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:27.049 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-14T22:38:27.069 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:27.070 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:27.084 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:38:27.134 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:27.133+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:27.228 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:27.225+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:27.244 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:27.245 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:27.258 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-14T22:38:27.260 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:38:27.346 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:27.345+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:27.383 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:27.383 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:27.398 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:38:27.443 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:27.438+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:27.465 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-14T22:38:27.520 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:27.520 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:27.532 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:27.526+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:27.534 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:38:27.633 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:27.629+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:27.695 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:27.695 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:27.709 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:38:27.733 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:27.733+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:27.834 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:27.826+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:27.842 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:27.842 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:27.856 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:38:27.947 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:27.942+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:28.029 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:28.029 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:28.036 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:28.030+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:28.044 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:38:28.137 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:28.137+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:28.213 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:28.213 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:28.227 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:38:28.238 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:28.237+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:28.339 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:28.337+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:28.357 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:28.357 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:28.371 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:38:28.440 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:28.434+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:28.507 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:28.508 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:28.523 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:38:28.541 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:28.534+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:28.642 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:28.634+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:28.649 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:28.650 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:28.665 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:38:28.725 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:38:28.742 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:28.734+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:28.756 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T22:38:28.809 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:28.809 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:28.828 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:38:28.843 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:28.838+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:28.904 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:38:28.904 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:38:28.928 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:38:28.928 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:38:28.944 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:28.938+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:28.946 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:38:28.946 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:38:28.963 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:38:28.963 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:38:28.986 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:38:28.986 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:38:29.007 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:38:29.007 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:38:29.020 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:29.020 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:29.020 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:38:29.020 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:29.028 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:38:29.028 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:38:29.035 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:38:29.050 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:38:29.050 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:38:29.051 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:29.049+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:29.144 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:29.141+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:29.166 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:29.166 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:29.167 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:38:29.167 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:38:29.167 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:29.181 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:38:29.245 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:29.238+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:29.346 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:29.345+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:29.349 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:29.349 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:38:29.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:38:29.349 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:38:29.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.349 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:122960 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 2347008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:29.349 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:41.998325+0000", 2024-09-14T22:38:29.350 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.348163882999998, 2024-09-14T22:38:29.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056570400999999999, 2024-09-14T22:38:29.350 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:29.350 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:29.350 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:29.350 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:29.350 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:29.350 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:29.350 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 122960 2024-09-14T22:38:29.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.350 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:29.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:29.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998325+0000", 2024-09-14T22:38:29.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:29.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998325+0000", 2024-09-14T22:38:29.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:29.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998326+0000", 2024-09-14T22:38:29.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9299999999999998e-07 2024-09-14T22:38:29.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:29.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998329+0000", 2024-09-14T22:38:29.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1370000000000002e-06 2024-09-14T22:38:29.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:29.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998340+0000", 2024-09-14T22:38:29.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1039999999999999e-05 2024-09-14T22:38:29.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:29.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998348+0000", 2024-09-14T22:38:29.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1920000000000005e-06 2024-09-14T22:38:29.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054700+0000", 2024-09-14T22:38:29.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056351714999999997 2024-09-14T22:38:29.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:29.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054771+0000", 2024-09-14T22:38:29.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1009999999999999e-05 2024-09-14T22:38:29.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:29.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054896+0000", 2024-09-14T22:38:29.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000124714 2024-09-14T22:38:29.353 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.353 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:29.354 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.354 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:122961 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 2351104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:29.354 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:41.998357+0000", 2024-09-14T22:38:29.354 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.348131928000001, 2024-09-14T22:38:29.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056974912000000003, 2024-09-14T22:38:29.354 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:29.354 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:29.354 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:29.354 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:29.354 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:29.354 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:29.355 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 122961 2024-09-14T22:38:29.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.355 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:29.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:29.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998357+0000", 2024-09-14T22:38:29.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:29.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998357+0000", 2024-09-14T22:38:29.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:29.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998358+0000", 2024-09-14T22:38:29.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9699999999999996e-07 2024-09-14T22:38:29.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:29.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998361+0000", 2024-09-14T22:38:29.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1370000000000002e-06 2024-09-14T22:38:29.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:29.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998371+0000", 2024-09-14T22:38:29.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5789999999999993e-06 2024-09-14T22:38:29.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:29.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998379+0000", 2024-09-14T22:38:29.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0129999999999993e-06 2024-09-14T22:38:29.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054998+0000", 2024-09-14T22:38:29.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056618961000000002 2024-09-14T22:38:29.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:29.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.055126+0000", 2024-09-14T22:38:29.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012795100000000001 2024-09-14T22:38:29.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:29.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.055332+0000", 2024-09-14T22:38:29.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020667400000000001 2024-09-14T22:38:29.358 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.358 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:29.358 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.358 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123259 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 741376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:29.358 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:38:29.358 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.334406411000003, 2024-09-14T22:38:29.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056672581999999999, 2024-09-14T22:38:29.358 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:29.359 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:29.359 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:29.359 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:29.359 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:29.359 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:29.359 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123259 2024-09-14T22:38:29.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.359 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:29.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:29.359 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:38:29.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:29.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:38:29.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:29.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:38:29.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1600000000000001e-07 2024-09-14T22:38:29.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:29.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012086+0000", 2024-09-14T22:38:29.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8499999999999998e-06 2024-09-14T22:38:29.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:29.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012097+0000", 2024-09-14T22:38:29.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0943e-05 2024-09-14T22:38:29.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:29.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012105+0000", 2024-09-14T22:38:29.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0830000000000004e-06 2024-09-14T22:38:29.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068584+0000", 2024-09-14T22:38:29.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056478387999999997 2024-09-14T22:38:29.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:29.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068644+0000", 2024-09-14T22:38:29.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9984e-05 2024-09-14T22:38:29.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:29.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068755+0000", 2024-09-14T22:38:29.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111718 2024-09-14T22:38:29.362 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.362 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:29.362 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.362 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123260 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 778240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:29.362 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012118+0000", 2024-09-14T22:38:29.363 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.334371185999998, 2024-09-14T22:38:29.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056825887999999998, 2024-09-14T22:38:29.363 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:29.363 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:29.363 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:29.363 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:29.363 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:29.363 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:29.363 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123260 2024-09-14T22:38:29.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.363 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:29.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:29.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012118+0000", 2024-09-14T22:38:29.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:29.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012118+0000", 2024-09-14T22:38:29.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:29.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012119+0000", 2024-09-14T22:38:29.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.13e-07 2024-09-14T22:38:29.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:29.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012130+0000", 2024-09-14T22:38:29.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1552999999999999e-05 2024-09-14T22:38:29.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:29.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012141+0000", 2024-09-14T22:38:29.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.03e-05 2024-09-14T22:38:29.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:29.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012149+0000", 2024-09-14T22:38:29.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1419999999999993e-06 2024-09-14T22:38:29.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068800+0000", 2024-09-14T22:38:29.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056651574000000003 2024-09-14T22:38:29.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:29.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068858+0000", 2024-09-14T22:38:29.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8047000000000002e-05 2024-09-14T22:38:29.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:29.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068944+0000", 2024-09-14T22:38:29.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5659000000000001e-05 2024-09-14T22:38:29.366 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.366 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:29.366 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.367 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123261 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 761856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:29.367 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012161+0000", 2024-09-14T22:38:29.367 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.334327784999999, 2024-09-14T22:38:29.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057113753000000003, 2024-09-14T22:38:29.367 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:29.367 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:29.367 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:29.367 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:29.367 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:29.367 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:29.367 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123261 2024-09-14T22:38:29.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.367 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:29.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:29.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012161+0000", 2024-09-14T22:38:29.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:29.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012161+0000", 2024-09-14T22:38:29.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:29.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012162+0000", 2024-09-14T22:38:29.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4399999999999999e-07 2024-09-14T22:38:29.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:29.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012177+0000", 2024-09-14T22:38:29.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5099000000000001e-05 2024-09-14T22:38:29.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:29.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012191+0000", 2024-09-14T22:38:29.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3631999999999999e-05 2024-09-14T22:38:29.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:29.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012199+0000", 2024-09-14T22:38:29.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5769999999999994e-06 2024-09-14T22:38:29.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069004+0000", 2024-09-14T22:38:29.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056804504999999998 2024-09-14T22:38:29.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:29.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069137+0000", 2024-09-14T22:38:29.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013302600000000001 2024-09-14T22:38:29.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:29.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069275+0000", 2024-09-14T22:38:29.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013816999999999999 2024-09-14T22:38:29.370 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.371 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:29.371 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.371 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123262 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 724992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:29.371 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012224+0000", 2024-09-14T22:38:29.371 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.334265338000002, 2024-09-14T22:38:29.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057417959999999997, 2024-09-14T22:38:29.371 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:29.371 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:29.371 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:29.371 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:29.371 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:29.371 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:29.372 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123262 2024-09-14T22:38:29.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.372 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:29.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:29.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012224+0000", 2024-09-14T22:38:29.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:29.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012224+0000", 2024-09-14T22:38:29.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:29.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012225+0000", 2024-09-14T22:38:29.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4000000000000001e-07 2024-09-14T22:38:29.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:29.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012228+0000", 2024-09-14T22:38:29.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5729999999999999e-06 2024-09-14T22:38:29.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:29.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012241+0000", 2024-09-14T22:38:29.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3210000000000001e-05 2024-09-14T22:38:29.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:29.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012255+0000", 2024-09-14T22:38:29.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3852000000000001e-05 2024-09-14T22:38:29.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069341+0000", 2024-09-14T22:38:29.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057086139000000001 2024-09-14T22:38:29.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:29.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069477+0000", 2024-09-14T22:38:29.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013595 2024-09-14T22:38:29.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:29.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069642+0000", 2024-09-14T22:38:29.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016459600000000001 2024-09-14T22:38:29.375 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.375 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:29.375 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.375 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123263 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 782336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:29.375 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012265+0000", 2024-09-14T22:38:29.375 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.334223796000003, 2024-09-14T22:38:29.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057665095, 2024-09-14T22:38:29.375 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:29.375 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:29.376 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:29.376 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:29.376 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:29.376 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:29.376 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123263 2024-09-14T22:38:29.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.376 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:29.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:29.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012265+0000", 2024-09-14T22:38:29.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:29.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012265+0000", 2024-09-14T22:38:29.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:29.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012266+0000", 2024-09-14T22:38:29.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8999999999999996e-07 2024-09-14T22:38:29.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:29.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012269+0000", 2024-09-14T22:38:29.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.796e-06 2024-09-14T22:38:29.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012279+0000", 2024-09-14T22:38:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7389999999999994e-06 2024-09-14T22:38:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012286+0000", 2024-09-14T22:38:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8199999999999997e-06 2024-09-14T22:38:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069705+0000", 2024-09-14T22:38:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057418590999999998 2024-09-14T22:38:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069799+0000", 2024-09-14T22:38:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4419999999999994e-05 2024-09-14T22:38:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069931+0000", 2024-09-14T22:38:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013113900000000001 2024-09-14T22:38:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123264 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 745472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012295+0000", 2024-09-14T22:38:29.380 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.334193794999997, 2024-09-14T22:38:29.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057959498999999998, 2024-09-14T22:38:29.380 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:29.380 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:29.380 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:29.380 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:29.380 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:29.380 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:29.380 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123264 2024-09-14T22:38:29.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.380 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:29.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:29.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012295+0000", 2024-09-14T22:38:29.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:29.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012295+0000", 2024-09-14T22:38:29.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:29.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012296+0000", 2024-09-14T22:38:29.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9699999999999996e-07 2024-09-14T22:38:29.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:29.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012299+0000", 2024-09-14T22:38:29.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.79e-06 2024-09-14T22:38:29.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:29.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012320+0000", 2024-09-14T22:38:29.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1242000000000001e-05 2024-09-14T22:38:29.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:29.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012338+0000", 2024-09-14T22:38:29.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7768999999999999e-05 2024-09-14T22:38:29.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069995+0000", 2024-09-14T22:38:29.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057657 2024-09-14T22:38:29.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:29.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070132+0000", 2024-09-14T22:38:29.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013727800000000001 2024-09-14T22:38:29.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:29.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070255+0000", 2024-09-14T22:38:29.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012282300000000001 2024-09-14T22:38:29.383 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.383 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:29.383 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.384 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123265 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1056768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:29.384 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012348+0000", 2024-09-14T22:38:29.384 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.334140840000003, 2024-09-14T22:38:29.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058145087999999998, 2024-09-14T22:38:29.384 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:29.384 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:29.384 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:29.384 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:29.384 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:29.384 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:29.384 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123265 2024-09-14T22:38:29.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.384 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:29.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.385 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:29.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012348+0000", 2024-09-14T22:38:29.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.385 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:29.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012348+0000", 2024-09-14T22:38:29.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.385 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:29.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012349+0000", 2024-09-14T22:38:29.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6599999999999996e-07 2024-09-14T22:38:29.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:29.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012352+0000", 2024-09-14T22:38:29.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3139999999999999e-06 2024-09-14T22:38:29.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:29.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012365+0000", 2024-09-14T22:38:29.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2625e-05 2024-09-14T22:38:29.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:29.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012373+0000", 2024-09-14T22:38:29.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2470000000000008e-06 2024-09-14T22:38:29.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070286+0000", 2024-09-14T22:38:29.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057912547000000002 2024-09-14T22:38:29.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:29.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070366+0000", 2024-09-14T22:38:29.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0468000000000003e-05 2024-09-14T22:38:29.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:29.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070493+0000", 2024-09-14T22:38:29.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012722100000000001 2024-09-14T22:38:29.387 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.388 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:29.388 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.388 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123266 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 765952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:29.388 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:38:29.388 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.334106591999998, 2024-09-14T22:38:29.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058386949, 2024-09-14T22:38:29.388 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:29.388 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:29.388 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:29.388 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:29.388 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:29.388 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:29.389 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123266 2024-09-14T22:38:29.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.389 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:29.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:29.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:38:29.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:29.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:38:29.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:29.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:38:29.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9299999999999998e-07 2024-09-14T22:38:29.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:29.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012386+0000", 2024-09-14T22:38:29.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8729999999999999e-06 2024-09-14T22:38:29.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:29.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012406+0000", 2024-09-14T22:38:29.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0103e-05 2024-09-14T22:38:29.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:29.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012419+0000", 2024-09-14T22:38:29.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.311e-05 2024-09-14T22:38:29.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070550+0000", 2024-09-14T22:38:29.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058130902999999998 2024-09-14T22:38:29.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:29.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070650+0000", 2024-09-14T22:38:29.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010005000000000001 2024-09-14T22:38:29.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:29.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070770+0000", 2024-09-14T22:38:29.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119317 2024-09-14T22:38:29.392 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.392 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:29.392 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.392 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123267 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1077248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:29.392 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012429+0000", 2024-09-14T22:38:29.393 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.334060678, 2024-09-14T22:38:29.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058620343999999998, 2024-09-14T22:38:29.393 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:29.393 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:29.393 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:29.393 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:29.393 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:29.393 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:29.393 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123267 2024-09-14T22:38:29.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.393 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:29.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:29.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012429+0000", 2024-09-14T22:38:29.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:29.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012429+0000", 2024-09-14T22:38:29.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:29.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013488+0000", 2024-09-14T22:38:29.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001059061 2024-09-14T22:38:29.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:29.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013499+0000", 2024-09-14T22:38:29.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1348999999999999e-05 2024-09-14T22:38:29.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:29.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013534+0000", 2024-09-14T22:38:29.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5206000000000001e-05 2024-09-14T22:38:29.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:29.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013558+0000", 2024-09-14T22:38:29.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4324999999999998e-05 2024-09-14T22:38:29.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070834+0000", 2024-09-14T22:38:29.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057275110999999997 2024-09-14T22:38:29.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:29.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070937+0000", 2024-09-14T22:38:29.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010306099999999999 2024-09-14T22:38:29.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:29.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071049+0000", 2024-09-14T22:38:29.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112231 2024-09-14T22:38:29.396 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.397 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:29.397 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.397 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123268 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1097728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:29.397 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013593+0000", 2024-09-14T22:38:29.397 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.332895954999998, 2024-09-14T22:38:29.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057751793000000003, 2024-09-14T22:38:29.397 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:29.397 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:29.397 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:29.397 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:29.397 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:29.397 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:29.397 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123268 2024-09-14T22:38:29.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.398 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:29.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:29.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013593+0000", 2024-09-14T22:38:29.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:29.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013593+0000", 2024-09-14T22:38:29.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:29.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013595+0000", 2024-09-14T22:38:29.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.59e-06 2024-09-14T22:38:29.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:29.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013602+0000", 2024-09-14T22:38:29.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3560000000000004e-06 2024-09-14T22:38:29.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:29.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013625+0000", 2024-09-14T22:38:29.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2419e-05 2024-09-14T22:38:29.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:29.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013650+0000", 2024-09-14T22:38:29.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5151999999999999e-05 2024-09-14T22:38:29.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071096+0000", 2024-09-14T22:38:29.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057445877999999999 2024-09-14T22:38:29.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:29.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071212+0000", 2024-09-14T22:38:29.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116215 2024-09-14T22:38:29.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:29.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071345+0000", 2024-09-14T22:38:29.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013318299999999999 2024-09-14T22:38:29.401 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.401 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:29.401 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.401 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123269 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1060864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:29.401 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013676+0000", 2024-09-14T22:38:29.401 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.332813195999996, 2024-09-14T22:38:29.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058027982999999998, 2024-09-14T22:38:29.401 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:29.401 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:29.401 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:29.401 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:29.401 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:29.401 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:29.402 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123269 2024-09-14T22:38:29.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.402 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:29.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:29.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013676+0000", 2024-09-14T22:38:29.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:29.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013676+0000", 2024-09-14T22:38:29.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:29.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013677+0000", 2024-09-14T22:38:29.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.387e-06 2024-09-14T22:38:29.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:29.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013684+0000", 2024-09-14T22:38:29.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0530000000000001e-06 2024-09-14T22:38:29.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:29.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013706+0000", 2024-09-14T22:38:29.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1121999999999999e-05 2024-09-14T22:38:29.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:29.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013722+0000", 2024-09-14T22:38:29.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6249999999999999e-05 2024-09-14T22:38:29.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071402+0000", 2024-09-14T22:38:29.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057679945000000003 2024-09-14T22:38:29.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:29.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071547+0000", 2024-09-14T22:38:29.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000145632 2024-09-14T22:38:29.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:29.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071704+0000", 2024-09-14T22:38:29.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000156594 2024-09-14T22:38:29.405 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.405 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:29.405 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.405 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123270 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1064960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:29.405 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013744+0000", 2024-09-14T22:38:29.405 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.332745244999998, 2024-09-14T22:38:29.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058355872000000003, 2024-09-14T22:38:29.405 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:29.405 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:29.406 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:29.406 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:29.406 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:29.406 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:29.406 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123270 2024-09-14T22:38:29.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.406 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:29.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:29.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013744+0000", 2024-09-14T22:38:29.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:29.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013744+0000", 2024-09-14T22:38:29.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:29.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013745+0000", 2024-09-14T22:38:29.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2589999999999999e-06 2024-09-14T22:38:29.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:29.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013751+0000", 2024-09-14T22:38:29.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1929999999999998e-06 2024-09-14T22:38:29.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:29.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013778+0000", 2024-09-14T22:38:29.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6472e-05 2024-09-14T22:38:29.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:29.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013795+0000", 2024-09-14T22:38:29.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6626e-05 2024-09-14T22:38:29.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071775+0000", 2024-09-14T22:38:29.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057980232 2024-09-14T22:38:29.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:29.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071911+0000", 2024-09-14T22:38:29.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000136129 2024-09-14T22:38:29.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:29.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072100+0000", 2024-09-14T22:38:29.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000188961 2024-09-14T22:38:29.409 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.409 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:29.409 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.410 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123271 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1081344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:29.410 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013818+0000", 2024-09-14T22:38:29.410 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.332671048999998, 2024-09-14T22:38:29.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058549565999999997, 2024-09-14T22:38:29.410 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:29.410 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:29.410 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:29.410 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:29.410 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:29.410 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:29.410 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123271 2024-09-14T22:38:29.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.410 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:29.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:29.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013818+0000", 2024-09-14T22:38:29.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:29.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013818+0000", 2024-09-14T22:38:29.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:29.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013958+0000", 2024-09-14T22:38:29.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013934 2024-09-14T22:38:29.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:29.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013968+0000", 2024-09-14T22:38:29.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0329000000000001e-05 2024-09-14T22:38:29.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:29.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013995+0000", 2024-09-14T22:38:29.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7613000000000001e-05 2024-09-14T22:38:29.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:29.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014014+0000", 2024-09-14T22:38:29.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8338999999999998e-05 2024-09-14T22:38:29.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072155+0000", 2024-09-14T22:38:29.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058140989999999997 2024-09-14T22:38:29.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:29.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072247+0000", 2024-09-14T22:38:29.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1774999999999995e-05 2024-09-14T22:38:29.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:29.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072368+0000", 2024-09-14T22:38:29.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012118 2024-09-14T22:38:29.413 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.414 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:29.414 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.414 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123272 3.1 3.4b235a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:29.414 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.014046+0000", 2024-09-14T22:38:29.414 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.332443372999997, 2024-09-14T22:38:29.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058364077, 2024-09-14T22:38:29.414 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:29.414 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:29.414 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-14T22:38:29.414 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:29.414 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:29.414 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:29.414 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123272 2024-09-14T22:38:29.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.415 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:29.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:29.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014046+0000", 2024-09-14T22:38:29.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:29.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014046+0000", 2024-09-14T22:38:29.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:29.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014047+0000", 2024-09-14T22:38:29.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.533e-06 2024-09-14T22:38:29.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:29.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014085+0000", 2024-09-14T22:38:29.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7734999999999997e-05 2024-09-14T22:38:29.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:29.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014111+0000", 2024-09-14T22:38:29.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6055999999999999e-05 2024-09-14T22:38:29.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:29.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014136+0000", 2024-09-14T22:38:29.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4592000000000001e-05 2024-09-14T22:38:29.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072402+0000", 2024-09-14T22:38:29.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058266251999999998 2024-09-14T22:38:29.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:29.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072410+0000", 2024-09-14T22:38:29.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.909e-06 2024-09-14T22:38:29.417 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.418 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:29.418 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.418 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123273 3.1 3.4b235a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:29.418 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.014161+0000", 2024-09-14T22:38:29.418 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.332328345000001, 2024-09-14T22:38:29.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058284415999999999, 2024-09-14T22:38:29.418 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:29.418 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:29.418 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-14T22:38:29.418 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:29.418 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:29.418 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:29.418 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123273 2024-09-14T22:38:29.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.419 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:29.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:29.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014161+0000", 2024-09-14T22:38:29.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:29.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014161+0000", 2024-09-14T22:38:29.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:29.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.017996+0000", 2024-09-14T22:38:29.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0038350670000000002 2024-09-14T22:38:29.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:29.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.018009+0000", 2024-09-14T22:38:29.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3305999999999999e-05 2024-09-14T22:38:29.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:29.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.018044+0000", 2024-09-14T22:38:29.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4792000000000002e-05 2024-09-14T22:38:29.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.018067+0000", 2024-09-14T22:38:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2595e-05 2024-09-14T22:38:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072439+0000", 2024-09-14T22:38:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054372192 2024-09-14T22:38:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072445+0000", 2024-09-14T22:38:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4640000000000003e-06 2024-09-14T22:38:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:148469 3.18 3:1f6f7244:::rbd_header.113c827b523f:head [watch reconnect cookie 140250011614800 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-14T22:38:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:55.746449+0000", 2024-09-14T22:38:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.600039733000003, 2024-09-14T22:38:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.996484173, 2024-09-14T22:38:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 148469 2024-09-14T22:38:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746449+0000", 2024-09-14T22:38:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746449+0000", 2024-09-14T22:38:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746452+0000", 2024-09-14T22:38:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0019999999999998e-06 2024-09-14T22:38:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746461+0000", 2024-09-14T22:38:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0919999999999998e-06 2024-09-14T22:38:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746475+0000", 2024-09-14T22:38:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4853e-05 2024-09-14T22:38:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746484+0000", 2024-09-14T22:38:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3780000000000001e-06 2024-09-14T22:38:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746559+0000", 2024-09-14T22:38:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5259000000000006e-05 2024-09-14T22:38:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746572+0000", 2024-09-14T22:38:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.271e-05 2024-09-14T22:38:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727807+0000", 2024-09-14T22:38:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98123555900000003 2024-09-14T22:38:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727826+0000", 2024-09-14T22:38:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8828e-05 2024-09-14T22:38:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.739033+0000", 2024-09-14T22:38:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.011207062 2024-09-14T22:38:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.739479+0000", 2024-09-14T22:38:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044563200000000001 2024-09-14T22:38:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.742934+0000", 2024-09-14T22:38:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0034547979999999998 2024-09-14T22:38:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:148473 3.18 3:1f6f7244:::rbd_header.113c827b523f:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-14T22:38:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:55.755969+0000", 2024-09-14T22:38:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.590519966999999, 2024-09-14T22:38:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9872532889999999, 2024-09-14T22:38:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 148473 2024-09-14T22:38:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755969+0000", 2024-09-14T22:38:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755969+0000", 2024-09-14T22:38:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755971+0000", 2024-09-14T22:38:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.674e-06 2024-09-14T22:38:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755979+0000", 2024-09-14T22:38:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8599999999999993e-06 2024-09-14T22:38:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755992+0000", 2024-09-14T22:38:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.361e-05 2024-09-14T22:38:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.756001+0000", 2024-09-14T22:38:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.422e-06 2024-09-14T22:38:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.756069+0000", 2024-09-14T22:38:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8209999999999999e-05 2024-09-14T22:38:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.756077+0000", 2024-09-14T22:38:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.994e-06 2024-09-14T22:38:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727872+0000", 2024-09-14T22:38:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.97179484999999999 2024-09-14T22:38:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727885+0000", 2024-09-14T22:38:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3101e-05 2024-09-14T22:38:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.740559+0000", 2024-09-14T22:38:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0126736300000001 2024-09-14T22:38:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.740777+0000", 2024-09-14T22:38:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021822000000000001 2024-09-14T22:38:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 2", 2024-09-14T22:38:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.741114+0000", 2024-09-14T22:38:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000336793 2024-09-14T22:38:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.742668+0000", 2024-09-14T22:38:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001554061 2024-09-14T22:38:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:38:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.742947+0000", 2024-09-14T22:38:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027937200000000001 2024-09-14T22:38:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:38:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743030+0000", 2024-09-14T22:38:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3385000000000002e-05 2024-09-14T22:38:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743223+0000", 2024-09-14T22:38:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00019210700000000001 2024-09-14T22:38:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:148474 3.18 3:1f6f7244:::rbd_header.113c827b523f:head [watch reconnect cookie 140250011614800 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104)", 2024-09-14T22:38:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:56.728273+0000", 2024-09-14T22:38:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.618216719999999, 2024-09-14T22:38:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0180497770000001, 2024-09-14T22:38:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 148474 2024-09-14T22:38:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728273+0000", 2024-09-14T22:38:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728273+0000", 2024-09-14T22:38:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728274+0000", 2024-09-14T22:38:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7570000000000001e-06 2024-09-14T22:38:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728281+0000", 2024-09-14T22:38:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5089999999999999e-06 2024-09-14T22:38:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728298+0000", 2024-09-14T22:38:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6727000000000001e-05 2024-09-14T22:38:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728309+0000", 2024-09-14T22:38:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1021e-05 2024-09-14T22:38:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.730078+0000", 2024-09-14T22:38:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017694010000000001 2024-09-14T22:38:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.730096+0000", 2024-09-14T22:38:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8039000000000001e-05 2024-09-14T22:38:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.741672+0000", 2024-09-14T22:38:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.011575817 2024-09-14T22:38:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2024-09-14T22:38:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.741827+0000", 2024-09-14T22:38:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000155563 2024-09-14T22:38:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743299+0000", 2024-09-14T22:38:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0014712729999999999 2024-09-14T22:38:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743415+0000", 2024-09-14T22:38:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011594700000000001 2024-09-14T22:38:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.746322+0000", 2024-09-14T22:38:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0029077230000000001 2024-09-14T22:38:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:29.439 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:29.440 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:38:29.447 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:29.442+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:29.547 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:29.545+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:29.611 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:29.612 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:29.612 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:38:29.612 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:29.626 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:38:29.649 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:29.642+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:29.745 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2024-09-14T22:38:29.750 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:29.742+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:29.806 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:29.806 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:29.806 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:38:29.806 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:38:29.806 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:29.822 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:38:29.869 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:29.869+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:29.951 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:29.949+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:29.954 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:29.954 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:38:29.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:38:29.954 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:38:29.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.954 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:31370 3.0 3:02f48f58:::rbd_data.113cda0183a0.0000000000000000:6 [read 813056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:38:29.955 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:38:29.955 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 116.587078003, 2024-09-14T22:38:29.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047519796000000003, 2024-09-14T22:38:29.955 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:29.955 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:29.955 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:29.955 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:29.955 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:29.955 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:29.955 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 31370 2024-09-14T22:38:29.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.956 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:29.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:29.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:38:29.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:29.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:38:29.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:29.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:38:29.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0999999999999999e-07 2024-09-14T22:38:29.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:29.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362401+0000", 2024-09-14T22:38:29.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2900000000000001e-06 2024-09-14T22:38:29.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:29.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362406+0000", 2024-09-14T22:38:29.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6960000000000002e-06 2024-09-14T22:38:29.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:29.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362412+0000", 2024-09-14T22:38:29.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.823e-06 2024-09-14T22:38:29.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.400270+0000", 2024-09-14T22:38:29.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037857737000000002 2024-09-14T22:38:29.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:29.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.400342+0000", 2024-09-14T22:38:29.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1797000000000004e-05 2024-09-14T22:38:29.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:29.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.409918+0000", 2024-09-14T22:38:29.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0095760429999999994 2024-09-14T22:38:29.959 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.959 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:29.959 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.959 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:113947 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 5603328~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:29.959 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.923453+0000", 2024-09-14T22:38:29.959 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.026022871000002, 2024-09-14T22:38:29.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047310420999999998, 2024-09-14T22:38:29.960 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:29.960 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:29.960 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:29.960 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:29.960 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:29.960 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:29.960 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 113947 2024-09-14T22:38:29.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.960 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:29.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:29.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.923453+0000", 2024-09-14T22:38:29.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:29.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.923453+0000", 2024-09-14T22:38:29.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:29.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926004+0000", 2024-09-14T22:38:29.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002551382 2024-09-14T22:38:29.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:29.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926009+0000", 2024-09-14T22:38:29.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5639999999999998e-06 2024-09-14T22:38:29.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:29.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926022+0000", 2024-09-14T22:38:29.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2734999999999999e-05 2024-09-14T22:38:29.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:29.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926029+0000", 2024-09-14T22:38:29.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2520000000000002e-06 2024-09-14T22:38:29.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.970650+0000", 2024-09-14T22:38:29.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044621277000000001 2024-09-14T22:38:29.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:29.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.970700+0000", 2024-09-14T22:38:29.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9713000000000003e-05 2024-09-14T22:38:29.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:29.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.970763+0000", 2024-09-14T22:38:29.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3497999999999994e-05 2024-09-14T22:38:29.963 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.964 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:29.964 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.964 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114136 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3448832~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:29.964 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932179+0000", 2024-09-14T22:38:29.964 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.017296504999997, 2024-09-14T22:38:29.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047236017999999998, 2024-09-14T22:38:29.964 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:29.964 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:29.965 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:29.965 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:29.965 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:29.965 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:29.965 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114136 2024-09-14T22:38:29.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.965 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:29.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:29.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932179+0000", 2024-09-14T22:38:29.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:29.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932179+0000", 2024-09-14T22:38:29.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:29.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932324+0000", 2024-09-14T22:38:29.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014451000000000001 2024-09-14T22:38:29.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:29.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932328+0000", 2024-09-14T22:38:29.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5889999999999996e-06 2024-09-14T22:38:29.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:29.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932338+0000", 2024-09-14T22:38:29.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3000000000000007e-06 2024-09-14T22:38:29.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:29.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932344+0000", 2024-09-14T22:38:29.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7150000000000001e-06 2024-09-14T22:38:29.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979233+0000", 2024-09-14T22:38:29.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046888711 2024-09-14T22:38:29.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:29.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979313+0000", 2024-09-14T22:38:29.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9487000000000007e-05 2024-09-14T22:38:29.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:29.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979415+0000", 2024-09-14T22:38:29.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000102706 2024-09-14T22:38:29.968 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.968 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:29.968 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.969 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114138 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3497984~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:29.969 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932356+0000", 2024-09-14T22:38:29.969 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.017119618999999, 2024-09-14T22:38:29.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047285958000000003, 2024-09-14T22:38:29.969 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:29.969 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:29.969 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:29.969 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:29.969 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:29.969 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:29.969 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114138 2024-09-14T22:38:29.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.970 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:29.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:29.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932356+0000", 2024-09-14T22:38:29.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:29.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932356+0000", 2024-09-14T22:38:29.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:29.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932498+0000", 2024-09-14T22:38:29.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000142182 2024-09-14T22:38:29.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:29.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932503+0000", 2024-09-14T22:38:29.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.301e-06 2024-09-14T22:38:29.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:29.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932514+0000", 2024-09-14T22:38:29.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1484000000000001e-05 2024-09-14T22:38:29.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:29.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932521+0000", 2024-09-14T22:38:29.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0740000000000004e-06 2024-09-14T22:38:29.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979451+0000", 2024-09-14T22:38:29.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046929379 2024-09-14T22:38:29.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:29.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979535+0000", 2024-09-14T22:38:29.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4423000000000001e-05 2024-09-14T22:38:29.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:29.973 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979642+0000", 2024-09-14T22:38:29.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010711499999999999 2024-09-14T22:38:29.973 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.973 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:29.973 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.973 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114139 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3547136~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:29.973 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932554+0000", 2024-09-14T22:38:29.973 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.016921852000003, 2024-09-14T22:38:29.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047292230999999997, 2024-09-14T22:38:29.973 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:29.973 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:29.974 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:29.974 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:29.974 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:29.974 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:29.974 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114139 2024-09-14T22:38:29.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.974 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:29.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:29.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932554+0000", 2024-09-14T22:38:29.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:29.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932554+0000", 2024-09-14T22:38:29.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:29.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932819+0000", 2024-09-14T22:38:29.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026538400000000002 2024-09-14T22:38:29.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:29.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932825+0000", 2024-09-14T22:38:29.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3469999999999998e-06 2024-09-14T22:38:29.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:29.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932836+0000", 2024-09-14T22:38:29.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1e-05 2024-09-14T22:38:29.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:29.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932843+0000", 2024-09-14T22:38:29.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1790000000000002e-06 2024-09-14T22:38:29.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979673+0000", 2024-09-14T22:38:29.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046830389 2024-09-14T22:38:29.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:29.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979750+0000", 2024-09-14T22:38:29.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6312999999999999e-05 2024-09-14T22:38:29.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:29.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979846+0000", 2024-09-14T22:38:29.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6619000000000004e-05 2024-09-14T22:38:29.978 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.978 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:29.978 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.978 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114141 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3596288~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:29.978 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932855+0000", 2024-09-14T22:38:29.978 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.016621254, 2024-09-14T22:38:29.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047201870999999999, 2024-09-14T22:38:29.978 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:29.978 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:29.978 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:29.979 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:29.979 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:29.979 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:29.979 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114141 2024-09-14T22:38:29.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.979 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:29.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:29.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932855+0000", 2024-09-14T22:38:29.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:29.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932855+0000", 2024-09-14T22:38:29.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:29.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934033+0000", 2024-09-14T22:38:29.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001178264 2024-09-14T22:38:29.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:29.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934038+0000", 2024-09-14T22:38:29.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4820000000000002e-06 2024-09-14T22:38:29.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:29.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934048+0000", 2024-09-14T22:38:29.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9890000000000004e-06 2024-09-14T22:38:29.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:29.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934055+0000", 2024-09-14T22:38:29.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8329999999999996e-06 2024-09-14T22:38:29.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979878+0000", 2024-09-14T22:38:29.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045822773999999997 2024-09-14T22:38:29.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:29.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979958+0000", 2024-09-14T22:38:29.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0477999999999998e-05 2024-09-14T22:38:29.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:29.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.980056+0000", 2024-09-14T22:38:29.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8051e-05 2024-09-14T22:38:29.983 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.983 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:29.983 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.983 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114159 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3940352~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:29.983 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.934179+0000", 2024-09-14T22:38:29.983 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.015297218000001, 2024-09-14T22:38:29.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047341439999999999, 2024-09-14T22:38:29.983 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:29.983 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:29.983 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:29.984 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:29.984 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:29.984 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:29.984 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114159 2024-09-14T22:38:29.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.984 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:29.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:29.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934179+0000", 2024-09-14T22:38:29.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:29.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934179+0000", 2024-09-14T22:38:29.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:29.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935021+0000", 2024-09-14T22:38:29.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00084276800000000003 2024-09-14T22:38:29.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:29.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935027+0000", 2024-09-14T22:38:29.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2959999999999998e-06 2024-09-14T22:38:29.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:29.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935038+0000", 2024-09-14T22:38:29.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1198e-05 2024-09-14T22:38:29.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:29.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935054+0000", 2024-09-14T22:38:29.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5764e-05 2024-09-14T22:38:29.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.981341+0000", 2024-09-14T22:38:29.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.04628782 2024-09-14T22:38:29.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:29.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.981415+0000", 2024-09-14T22:38:29.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3349999999999994e-05 2024-09-14T22:38:29.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:29.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.981520+0000", 2024-09-14T22:38:29.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105244 2024-09-14T22:38:29.988 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.988 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:29.989 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.989 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114180 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4136960~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:29.989 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:38:29.989 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.014297681999999, 2024-09-14T22:38:29.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047190621000000002, 2024-09-14T22:38:29.989 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:29.989 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:29.990 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:29.990 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:29.990 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:29.990 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:29.990 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114180 2024-09-14T22:38:29.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.990 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:29.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:29.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:38:29.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:29.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:38:29.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:29.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:38:29.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6800000000000002e-07 2024-09-14T22:38:29.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:29.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935180+0000", 2024-09-14T22:38:29.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.42e-06 2024-09-14T22:38:29.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:29.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935185+0000", 2024-09-14T22:38:29.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1359999999999996e-06 2024-09-14T22:38:29.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:29.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935188+0000", 2024-09-14T22:38:29.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.711e-06 2024-09-14T22:38:29.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982201+0000", 2024-09-14T22:38:29.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047013309000000003 2024-09-14T22:38:29.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:29.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982278+0000", 2024-09-14T22:38:29.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6705000000000005e-05 2024-09-14T22:38:29.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:29.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982369+0000", 2024-09-14T22:38:29.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1071999999999998e-05 2024-09-14T22:38:29.994 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.994 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:29.994 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.994 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114182 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4186112~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:29.994 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:38:29.994 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.014269059999997, 2024-09-14T22:38:29.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047368990999999999, 2024-09-14T22:38:29.994 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:29.994 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:29.994 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:29.994 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:29.995 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:29.995 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:29.995 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114182 2024-09-14T22:38:29.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.995 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:29.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:29.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:38:29.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:29.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:38:29.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:29.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:29.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:38:29.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7099999999999998e-07 2024-09-14T22:38:29.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:29.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935209+0000", 2024-09-14T22:38:29.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0260000000000002e-06 2024-09-14T22:38:29.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:29.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935213+0000", 2024-09-14T22:38:29.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3429999999999998e-06 2024-09-14T22:38:29.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:29.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935217+0000", 2024-09-14T22:38:29.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2480000000000001e-06 2024-09-14T22:38:29.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:29.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982406+0000", 2024-09-14T22:38:29.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047188920000000002 2024-09-14T22:38:29.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:29.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982483+0000", 2024-09-14T22:38:29.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7008999999999994e-05 2024-09-14T22:38:29.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:29.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982576+0000", 2024-09-14T22:38:29.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3173999999999999e-05 2024-09-14T22:38:29.998 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.998 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:29.999 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:29.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:29.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:29.999 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114185 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4202496~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:29.999 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.935221+0000", 2024-09-14T22:38:29.999 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.014254895000001, 2024-09-14T22:38:29.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047601128999999999, 2024-09-14T22:38:29.999 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:29.999 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:29.999 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:29.999 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:29.999 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:29.999 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:29.999 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114185 2024-09-14T22:38:29.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.000 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:30.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:30.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935221+0000", 2024-09-14T22:38:30.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:30.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935221+0000", 2024-09-14T22:38:30.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:30.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935970+0000", 2024-09-14T22:38:30.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074930899999999998 2024-09-14T22:38:30.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:30.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935976+0000", 2024-09-14T22:38:30.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5439999999999998e-06 2024-09-14T22:38:30.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:30.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935988+0000", 2024-09-14T22:38:30.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1779e-05 2024-09-14T22:38:30.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.002 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:30.002 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935996+0000", 2024-09-14T22:38:30.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.106e-06 2024-09-14T22:38:30.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.002 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:30.002 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982607+0000", 2024-09-14T22:38:30.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046611504999999998 2024-09-14T22:38:30.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.002 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:30.002 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982686+0000", 2024-09-14T22:38:30.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8805000000000002e-05 2024-09-14T22:38:30.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:30.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982822+0000", 2024-09-14T22:38:30.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000136081 2024-09-14T22:38:30.003 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.003 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:30.003 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.003 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114189 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4300800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:30.003 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.936048+0000", 2024-09-14T22:38:30.003 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.013427763000003, 2024-09-14T22:38:30.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047231539000000003, 2024-09-14T22:38:30.003 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:30.003 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:30.004 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:30.004 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:30.004 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:30.004 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:30.004 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114189 2024-09-14T22:38:30.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.004 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:30.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:30.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936048+0000", 2024-09-14T22:38:30.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:30.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936048+0000", 2024-09-14T22:38:30.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:30.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936049+0000", 2024-09-14T22:38:30.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4099999999999999e-07 2024-09-14T22:38:30.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:30.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936052+0000", 2024-09-14T22:38:30.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0029999999999999e-06 2024-09-14T22:38:30.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:30.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936058+0000", 2024-09-14T22:38:30.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7880000000000001e-06 2024-09-14T22:38:30.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:30.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936064+0000", 2024-09-14T22:38:30.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2279999999999998e-06 2024-09-14T22:38:30.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:30.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983120+0000", 2024-09-14T22:38:30.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047056384999999999 2024-09-14T22:38:30.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:30.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983194+0000", 2024-09-14T22:38:30.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3811000000000002e-05 2024-09-14T22:38:30.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:30.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983280+0000", 2024-09-14T22:38:30.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5883000000000004e-05 2024-09-14T22:38:30.007 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.007 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:30.007 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.008 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114191 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4349952~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:30.008 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:38:30.008 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.013404023, 2024-09-14T22:38:30.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047365365, 2024-09-14T22:38:30.008 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:30.008 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:30.008 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:30.008 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:30.008 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:30.008 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:30.008 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114191 2024-09-14T22:38:30.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.009 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:30.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:30.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:38:30.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:30.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:38:30.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:30.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:38:30.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0499999999999999e-07 2024-09-14T22:38:30.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:30.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936075+0000", 2024-09-14T22:38:30.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.734e-06 2024-09-14T22:38:30.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:30.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936082+0000", 2024-09-14T22:38:30.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8360000000000003e-06 2024-09-14T22:38:30.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:30.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936087+0000", 2024-09-14T22:38:30.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3129999999999998e-06 2024-09-14T22:38:30.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:30.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983319+0000", 2024-09-14T22:38:30.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047232140999999998 2024-09-14T22:38:30.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:30.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983365+0000", 2024-09-14T22:38:30.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5776999999999999e-05 2024-09-14T22:38:30.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:30.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983437+0000", 2024-09-14T22:38:30.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2158999999999998e-05 2024-09-14T22:38:30.012 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.012 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:30.012 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.012 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114193 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4399104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:30.012 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.936095+0000", 2024-09-14T22:38:30.012 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.013381086999999, 2024-09-14T22:38:30.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047505915000000003, 2024-09-14T22:38:30.013 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:30.013 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:30.013 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:30.013 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:30.013 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:30.013 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:30.013 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114193 2024-09-14T22:38:30.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.013 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:30.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:30.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936095+0000", 2024-09-14T22:38:30.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:30.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936095+0000", 2024-09-14T22:38:30.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:30.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938021+0000", 2024-09-14T22:38:30.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001926178 2024-09-14T22:38:30.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:30.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938027+0000", 2024-09-14T22:38:30.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3110000000000002e-06 2024-09-14T22:38:30.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:30.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938041+0000", 2024-09-14T22:38:30.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3344e-05 2024-09-14T22:38:30.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:30.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938049+0000", 2024-09-14T22:38:30.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2279999999999998e-06 2024-09-14T22:38:30.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:30.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983468+0000", 2024-09-14T22:38:30.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045419494999999997 2024-09-14T22:38:30.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:30.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983524+0000", 2024-09-14T22:38:30.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5952000000000002e-05 2024-09-14T22:38:30.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:30.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983601+0000", 2024-09-14T22:38:30.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6407000000000002e-05 2024-09-14T22:38:30.017 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.017 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:30.017 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.018 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114710 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4857856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:30.018 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.962571+0000", 2024-09-14T22:38:30.018 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 60.986904815000003, 2024-09-14T22:38:30.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047253314999999997, 2024-09-14T22:38:30.018 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:30.018 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:30.018 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:30.018 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:30.018 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:30.018 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:30.018 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114710 2024-09-14T22:38:30.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.018 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:30.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:30.019 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962571+0000", 2024-09-14T22:38:30.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:30.019 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962571+0000", 2024-09-14T22:38:30.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:30.020 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962572+0000", 2024-09-14T22:38:30.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1399999999999997e-07 2024-09-14T22:38:30.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.020 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:30.020 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962574+0000", 2024-09-14T22:38:30.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8679999999999999e-06 2024-09-14T22:38:30.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.020 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:30.020 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962583+0000", 2024-09-14T22:38:30.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8310000000000007e-06 2024-09-14T22:38:30.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:30.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962589+0000", 2024-09-14T22:38:30.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9529999999999996e-06 2024-09-14T22:38:30.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:30.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009692+0000", 2024-09-14T22:38:30.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047103249999999999 2024-09-14T22:38:30.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:30.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009767+0000", 2024-09-14T22:38:30.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4672000000000006e-05 2024-09-14T22:38:30.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:30.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009824+0000", 2024-09-14T22:38:30.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7226999999999999e-05 2024-09-14T22:38:30.022 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.022 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:30.022 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.023 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114712 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4907008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:30.023 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:38:30.023 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 60.986879334999998, 2024-09-14T22:38:30.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047421087000000001, 2024-09-14T22:38:30.023 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:30.023 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:30.023 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:30.023 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:30.023 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:30.023 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:30.023 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114712 2024-09-14T22:38:30.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.024 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:30.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.024 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:30.024 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:38:30.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.024 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:30.024 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:38:30.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:30.025 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:38:30.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.03e-07 2024-09-14T22:38:30.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:30.025 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962599+0000", 2024-09-14T22:38:30.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3199999999999998e-06 2024-09-14T22:38:30.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:30.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962606+0000", 2024-09-14T22:38:30.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5289999999999997e-06 2024-09-14T22:38:30.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:30.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962612+0000", 2024-09-14T22:38:30.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.947e-06 2024-09-14T22:38:30.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:30.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009866+0000", 2024-09-14T22:38:30.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047254681999999999 2024-09-14T22:38:30.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:30.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009924+0000", 2024-09-14T22:38:30.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7556000000000003e-05 2024-09-14T22:38:30.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:30.028 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010018+0000", 2024-09-14T22:38:30.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.365e-05 2024-09-14T22:38:30.028 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.028 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:30.028 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.028 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114714 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4956160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:30.029 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.962618+0000", 2024-09-14T22:38:30.029 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 60.986857817000001, 2024-09-14T22:38:30.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.04759302, 2024-09-14T22:38:30.029 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:30.029 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:30.029 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:30.029 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:30.029 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:30.029 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:30.030 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114714 2024-09-14T22:38:30.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.030 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:30.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:30.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962618+0000", 2024-09-14T22:38:30.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:30.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962618+0000", 2024-09-14T22:38:30.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:30.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966011+0000", 2024-09-14T22:38:30.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003393067 2024-09-14T22:38:30.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:30.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966016+0000", 2024-09-14T22:38:30.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3580000000000002e-06 2024-09-14T22:38:30.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:30.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966032+0000", 2024-09-14T22:38:30.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5356e-05 2024-09-14T22:38:30.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:30.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966040+0000", 2024-09-14T22:38:30.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1980000000000001e-06 2024-09-14T22:38:30.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:30.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010055+0000", 2024-09-14T22:38:30.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044014676000000003 2024-09-14T22:38:30.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.033 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:30.033 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010127+0000", 2024-09-14T22:38:30.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2206000000000006e-05 2024-09-14T22:38:30.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.033 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:30.033 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010211+0000", 2024-09-14T22:38:30.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4159000000000005e-05 2024-09-14T22:38:30.033 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.033 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:30.033 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.034 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114754 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 6283264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:30.034 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:38:30.034 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 60.982682904000001, 2024-09-14T22:38:30.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047228150000000003, 2024-09-14T22:38:30.034 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:30.034 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:30.034 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:30.034 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:30.034 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:30.034 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:30.034 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114754 2024-09-14T22:38:30.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.035 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:30.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:30.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:38:30.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:30.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:38:30.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:30.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:38:30.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7399999999999999e-07 2024-09-14T22:38:30.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:30.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966796+0000", 2024-09-14T22:38:30.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3630000000000001e-06 2024-09-14T22:38:30.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:30.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966802+0000", 2024-09-14T22:38:30.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4679999999999997e-06 2024-09-14T22:38:30.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.037 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:30.037 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966807+0000", 2024-09-14T22:38:30.037 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1109999999999999e-06 2024-09-14T22:38:30.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.037 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:30.037 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.013929+0000", 2024-09-14T22:38:30.037 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047121720999999998 2024-09-14T22:38:30.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.037 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:30.037 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.013968+0000", 2024-09-14T22:38:30.037 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8877000000000001e-05 2024-09-14T22:38:30.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.038 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:30.038 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014021+0000", 2024-09-14T22:38:30.038 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3236000000000003e-05 2024-09-14T22:38:30.038 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.038 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:30.038 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.038 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114789 3.19 3:9836bb25:::rbd_object_map.113c14c67864.000000000000001b:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:30.038 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.966813+0000", 2024-09-14T22:38:30.038 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 60.982662441999999, 2024-09-14T22:38:30.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05001311, 2024-09-14T22:38:30.039 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:30.039 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:30.039 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:30.039 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:30.039 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:30.039 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:30.039 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114789 2024-09-14T22:38:30.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.039 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:30.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.039 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:30.039 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966813+0000", 2024-09-14T22:38:30.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:30.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966813+0000", 2024-09-14T22:38:30.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:30.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966814+0000", 2024-09-14T22:38:30.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9200000000000002e-07 2024-09-14T22:38:30.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.041 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:30.041 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966817+0000", 2024-09-14T22:38:30.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8380000000000002e-06 2024-09-14T22:38:30.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.041 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:30.041 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966831+0000", 2024-09-14T22:38:30.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4209999999999999e-05 2024-09-14T22:38:30.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.041 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:30.041 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966845+0000", 2024-09-14T22:38:30.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3799e-05 2024-09-14T22:38:30.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.042 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:30.042 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014046+0000", 2024-09-14T22:38:30.042 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047201604000000001 2024-09-14T22:38:30.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.042 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:30.042 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014116+0000", 2024-09-14T22:38:30.042 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9387000000000005e-05 2024-09-14T22:38:30.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.042 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-14T22:38:30.042 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014469+0000", 2024-09-14T22:38:30.042 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00035355800000000002 2024-09-14T22:38:30.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:30.043 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016719+0000", 2024-09-14T22:38:30.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0022498459999999998 2024-09-14T22:38:30.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:38:30.043 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016735+0000", 2024-09-14T22:38:30.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5792999999999999e-05 2024-09-14T22:38:30.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:38:30.044 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016790+0000", 2024-09-14T22:38:30.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4715e-05 2024-09-14T22:38:30.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.044 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:30.044 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016827+0000", 2024-09-14T22:38:30.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6968000000000002e-05 2024-09-14T22:38:30.044 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.044 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:30.044 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.044 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114803 3.19 3:9836bb25:::rbd_object_map.113c14c67864.000000000000001b:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:30.045 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.966853+0000", 2024-09-14T22:38:30.045 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 60.982623011999998, 2024-09-14T22:38:30.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050656093999999999, 2024-09-14T22:38:30.045 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:30.045 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:30.045 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:30.045 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:30.045 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:30.045 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:30.045 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114803 2024-09-14T22:38:30.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.045 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:30.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:30.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966853+0000", 2024-09-14T22:38:30.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:30.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966853+0000", 2024-09-14T22:38:30.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:30.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969930+0000", 2024-09-14T22:38:30.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030769759999999999 2024-09-14T22:38:30.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:30.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969936+0000", 2024-09-14T22:38:30.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7150000000000003e-06 2024-09-14T22:38:30.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:30.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969949+0000", 2024-09-14T22:38:30.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3166000000000001e-05 2024-09-14T22:38:30.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:30.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969957+0000", 2024-09-14T22:38:30.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8269999999999998e-06 2024-09-14T22:38:30.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:30.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014996+0000", 2024-09-14T22:38:30.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045039202 2024-09-14T22:38:30.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:30.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.015085+0000", 2024-09-14T22:38:30.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8987999999999996e-05 2024-09-14T22:38:30.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-14T22:38:30.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.015318+0000", 2024-09-14T22:38:30.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023284999999999999 2024-09-14T22:38:30.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:30.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016898+0000", 2024-09-14T22:38:30.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0015807379999999999 2024-09-14T22:38:30.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:38:30.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.017455+0000", 2024-09-14T22:38:30.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00055664600000000005 2024-09-14T22:38:30.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.050 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:38:30.050 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.017483+0000", 2024-09-14T22:38:30.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8238e-05 2024-09-14T22:38:30.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.050 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:30.050 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.017509+0000", 2024-09-14T22:38:30.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5748000000000001e-05 2024-09-14T22:38:30.050 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.050 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:30.050 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.051 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:150759 3.8 3:15dc94e5:::rbd_header.113c88f88d4a:head [create,call rbd.create in=9b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e113)", 2024-09-14T22:38:30.051 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:38:05.726302+0000", 2024-09-14T22:38:30.051 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 24.223174319999998, 2024-09-14T22:38:30.051 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0241160819999999, 2024-09-14T22:38:30.051 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:30.051 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:30.051 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:30.051 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:30.051 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:30.051 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:30.051 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 150759 2024-09-14T22:38:30.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.052 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:30.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.052 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:30.052 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.726302+0000", 2024-09-14T22:38:30.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.052 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:30.052 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.726302+0000", 2024-09-14T22:38:30.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.052 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:30.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.726304+0000", 2024-09-14T22:38:30.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.075e-06 2024-09-14T22:38:30.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:30.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.726312+0000", 2024-09-14T22:38:30.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2050000000000002e-06 2024-09-14T22:38:30.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:30.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.727969+0000", 2024-09-14T22:38:30.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001657499 2024-09-14T22:38:30.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:30.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.727985+0000", 2024-09-14T22:38:30.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6082999999999998e-05 2024-09-14T22:38:30.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:30.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.728193+0000", 2024-09-14T22:38:30.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000208082 2024-09-14T22:38:30.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:30.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.728205+0000", 2024-09-14T22:38:30.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1897e-05 2024-09-14T22:38:30.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:30.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.741926+0000", 2024-09-14T22:38:30.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.013720591 2024-09-14T22:38:30.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.056 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:30.056 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.742076+0000", 2024-09-14T22:38:30.056 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015024099999999999 2024-09-14T22:38:30.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.056 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-14T22:38:30.056 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.742415+0000", 2024-09-14T22:38:30.056 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00033887700000000002 2024-09-14T22:38:30.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.056 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:38:30.057 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.743436+0000", 2024-09-14T22:38:30.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010210639999999999 2024-09-14T22:38:30.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.057 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:30.057 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.750216+0000", 2024-09-14T22:38:30.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0067798950000000002 2024-09-14T22:38:30.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.057 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:38:30.058 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.750288+0000", 2024-09-14T22:38:30.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2192000000000003e-05 2024-09-14T22:38:30.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.058 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:30.058 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.750418+0000", 2024-09-14T22:38:30.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000129381 2024-09-14T22:38:30.058 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.058 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:30.058 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.058 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.058 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:30.058 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:30.060 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:38:30.061 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:30.057+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:30.161 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:30.157+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:30.235 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:30.235 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:30.235 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:38:30.235 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:30.249 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:38:30.294 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:30.293+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:30.363 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:30.361+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:30.422 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:30.422 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:30.422 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:38:30.422 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:38:30.422 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:30.437 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:38:30.464 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:30.462+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:30.565 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:30.558+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:30.602 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:30.603 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:38:30.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:38:30.603 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:38:30.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.603 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145249 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 15745024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:30.603 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.693850+0000", 2024-09-14T22:38:30.603 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.905817349000003, 2024-09-14T22:38:30.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.060850297999999997, 2024-09-14T22:38:30.603 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:30.603 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:30.603 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:30.603 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:30.604 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:30.604 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:30.604 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145249 2024-09-14T22:38:30.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.604 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:30.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:30.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693850+0000", 2024-09-14T22:38:30.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:30.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693850+0000", 2024-09-14T22:38:30.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:30.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693851+0000", 2024-09-14T22:38:30.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.46e-07 2024-09-14T22:38:30.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:30.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693853+0000", 2024-09-14T22:38:30.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0940000000000002e-06 2024-09-14T22:38:30.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:30.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693858+0000", 2024-09-14T22:38:30.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2100000000000001e-06 2024-09-14T22:38:30.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:30.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693863+0000", 2024-09-14T22:38:30.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5549999999999996e-06 2024-09-14T22:38:30.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:30.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754503+0000", 2024-09-14T22:38:30.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.060640558999999997 2024-09-14T22:38:30.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:30.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754613+0000", 2024-09-14T22:38:30.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109966 2024-09-14T22:38:30.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:30.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754701+0000", 2024-09-14T22:38:30.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7567999999999999e-05 2024-09-14T22:38:30.607 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.607 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:30.607 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.607 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145250 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16039936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:30.608 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.693868+0000", 2024-09-14T22:38:30.608 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.905799621, 2024-09-14T22:38:30.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061182667000000003, 2024-09-14T22:38:30.608 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:30.608 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:30.608 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:30.608 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:30.608 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:30.608 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:30.608 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145250 2024-09-14T22:38:30.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.608 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:30.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:30.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693868+0000", 2024-09-14T22:38:30.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:30.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693868+0000", 2024-09-14T22:38:30.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:30.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693869+0000", 2024-09-14T22:38:30.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4400000000000001e-07 2024-09-14T22:38:30.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:30.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693870+0000", 2024-09-14T22:38:30.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9130000000000001e-06 2024-09-14T22:38:30.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:30.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693876+0000", 2024-09-14T22:38:30.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1270000000000002e-06 2024-09-14T22:38:30.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:30.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693880+0000", 2024-09-14T22:38:30.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.673e-06 2024-09-14T22:38:30.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:30.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754780+0000", 2024-09-14T22:38:30.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.060899495999999997 2024-09-14T22:38:30.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:30.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754925+0000", 2024-09-14T22:38:30.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014511799999999999 2024-09-14T22:38:30.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:30.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755051+0000", 2024-09-14T22:38:30.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012599599999999999 2024-09-14T22:38:30.612 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.612 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:30.612 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.612 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145251 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16236544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:30.612 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.693886+0000", 2024-09-14T22:38:30.612 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.905781967999999, 2024-09-14T22:38:30.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061498816999999997, 2024-09-14T22:38:30.612 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:30.612 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:30.612 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:30.612 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:30.613 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:30.613 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:30.613 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145251 2024-09-14T22:38:30.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.613 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:30.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:30.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693886+0000", 2024-09-14T22:38:30.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:30.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693886+0000", 2024-09-14T22:38:30.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:30.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693886+0000", 2024-09-14T22:38:30.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1199999999999998e-07 2024-09-14T22:38:30.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:30.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693888+0000", 2024-09-14T22:38:30.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.877e-06 2024-09-14T22:38:30.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:30.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693894+0000", 2024-09-14T22:38:30.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5350000000000004e-06 2024-09-14T22:38:30.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:30.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693917+0000", 2024-09-14T22:38:30.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.296e-05 2024-09-14T22:38:30.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:30.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755127+0000", 2024-09-14T22:38:30.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061210286000000003 2024-09-14T22:38:30.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.616 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:30.616 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755266+0000", 2024-09-14T22:38:30.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013871599999999999 2024-09-14T22:38:30.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.616 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:30.616 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755385+0000", 2024-09-14T22:38:30.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011903100000000001 2024-09-14T22:38:30.616 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.616 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:30.616 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.616 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145252 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16302080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:30.617 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.693985+0000", 2024-09-14T22:38:30.617 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.905683285999999, 2024-09-14T22:38:30.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061743767999999997, 2024-09-14T22:38:30.617 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:30.617 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:30.617 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:30.617 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:30.617 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:30.617 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:30.617 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145252 2024-09-14T22:38:30.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.617 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:30.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:30.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693985+0000", 2024-09-14T22:38:30.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:30.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693985+0000", 2024-09-14T22:38:30.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:30.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693986+0000", 2024-09-14T22:38:30.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0640000000000001e-06 2024-09-14T22:38:30.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:30.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693991+0000", 2024-09-14T22:38:30.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7599999999999999e-06 2024-09-14T22:38:30.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:30.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694005+0000", 2024-09-14T22:38:30.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3271999999999999e-05 2024-09-14T22:38:30.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:30.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694015+0000", 2024-09-14T22:38:30.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0617e-05 2024-09-14T22:38:30.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:30.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755459+0000", 2024-09-14T22:38:30.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061443820000000003 2024-09-14T22:38:30.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:30.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755605+0000", 2024-09-14T22:38:30.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014569 2024-09-14T22:38:30.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:30.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755728+0000", 2024-09-14T22:38:30.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000123545 2024-09-14T22:38:30.620 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.620 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:30.621 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.621 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145253 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16138240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:30.621 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694031+0000", 2024-09-14T22:38:30.621 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.905636731999998, 2024-09-14T22:38:30.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062032836000000001, 2024-09-14T22:38:30.621 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:30.621 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:30.621 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:30.621 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:30.621 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:30.621 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:30.621 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145253 2024-09-14T22:38:30.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.622 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:30.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:30.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694031+0000", 2024-09-14T22:38:30.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:30.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694031+0000", 2024-09-14T22:38:30.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:30.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694032+0000", 2024-09-14T22:38:30.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7999999999999995e-07 2024-09-14T22:38:30.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:30.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694036+0000", 2024-09-14T22:38:30.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8800000000000001e-06 2024-09-14T22:38:30.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:30.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694047+0000", 2024-09-14T22:38:30.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0835999999999999e-05 2024-09-14T22:38:30.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:30.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694055+0000", 2024-09-14T22:38:30.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6619999999999994e-06 2024-09-14T22:38:30.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:30.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755804+0000", 2024-09-14T22:38:30.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061748448999999997 2024-09-14T22:38:30.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:30.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755941+0000", 2024-09-14T22:38:30.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013751599999999999 2024-09-14T22:38:30.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:30.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756064+0000", 2024-09-14T22:38:30.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012281299999999999 2024-09-14T22:38:30.625 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.625 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:30.625 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.625 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145254 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16400384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:30.625 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694067+0000", 2024-09-14T22:38:30.625 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.905600567, 2024-09-14T22:38:30.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062280549999999997, 2024-09-14T22:38:30.625 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:30.625 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:30.625 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:30.625 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:30.625 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:30.626 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:30.626 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145254 2024-09-14T22:38:30.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.626 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:30.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:30.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694067+0000", 2024-09-14T22:38:30.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:30.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694067+0000", 2024-09-14T22:38:30.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:30.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694068+0000", 2024-09-14T22:38:30.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2200000000000003e-07 2024-09-14T22:38:30.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:30.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694072+0000", 2024-09-14T22:38:30.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5729999999999999e-06 2024-09-14T22:38:30.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:30.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694082+0000", 2024-09-14T22:38:30.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0703999999999999e-05 2024-09-14T22:38:30.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:30.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694091+0000", 2024-09-14T22:38:30.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8030000000000003e-06 2024-09-14T22:38:30.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:30.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756155+0000", 2024-09-14T22:38:30.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062063937 2024-09-14T22:38:30.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:30.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756263+0000", 2024-09-14T22:38:30.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108229 2024-09-14T22:38:30.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.629 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:30.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756348+0000", 2024-09-14T22:38:30.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4581999999999998e-05 2024-09-14T22:38:30.629 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.629 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:30.629 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.629 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145255 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16596992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:30.629 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694103+0000", 2024-09-14T22:38:30.629 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.905565086000003, 2024-09-14T22:38:30.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062580620000000003, 2024-09-14T22:38:30.629 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:30.630 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:30.630 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:30.630 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:30.630 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:30.630 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:30.630 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145255 2024-09-14T22:38:30.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.630 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:30.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:30.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694103+0000", 2024-09-14T22:38:30.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:30.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694103+0000", 2024-09-14T22:38:30.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:30.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694103+0000", 2024-09-14T22:38:30.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7800000000000001e-07 2024-09-14T22:38:30.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:30.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694107+0000", 2024-09-14T22:38:30.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5939999999999998e-06 2024-09-14T22:38:30.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:30.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694121+0000", 2024-09-14T22:38:30.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3913999999999999e-05 2024-09-14T22:38:30.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:30.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694130+0000", 2024-09-14T22:38:30.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7690000000000002e-06 2024-09-14T22:38:30.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:30.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756431+0000", 2024-09-14T22:38:30.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062301593000000002 2024-09-14T22:38:30.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:30.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756564+0000", 2024-09-14T22:38:30.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000132693 2024-09-14T22:38:30.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:30.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756683+0000", 2024-09-14T22:38:30.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119479 2024-09-14T22:38:30.634 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.634 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:30.634 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.634 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145256 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16695296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:30.634 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694142+0000", 2024-09-14T22:38:30.634 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.905526012999999, 2024-09-14T22:38:30.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062806034999999996, 2024-09-14T22:38:30.634 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:30.635 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:30.635 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:30.635 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:30.635 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:30.635 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:30.635 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145256 2024-09-14T22:38:30.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.635 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:30.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:30.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694142+0000", 2024-09-14T22:38:30.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:30.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694142+0000", 2024-09-14T22:38:30.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:30.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694142+0000", 2024-09-14T22:38:30.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1699999999999998e-07 2024-09-14T22:38:30.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:30.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694146+0000", 2024-09-14T22:38:30.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7450000000000001e-06 2024-09-14T22:38:30.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:30.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694157+0000", 2024-09-14T22:38:30.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1161e-05 2024-09-14T22:38:30.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:30.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694166+0000", 2024-09-14T22:38:30.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0450000000000006e-06 2024-09-14T22:38:30.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:30.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756768+0000", 2024-09-14T22:38:30.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062601554000000004 2024-09-14T22:38:30.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:30.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756867+0000", 2024-09-14T22:38:30.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8741999999999998e-05 2024-09-14T22:38:30.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:30.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756948+0000", 2024-09-14T22:38:30.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1170999999999999e-05 2024-09-14T22:38:30.638 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.639 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:30.639 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.639 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145257 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16498688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:30.639 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694179+0000", 2024-09-14T22:38:30.639 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.905488714000001, 2024-09-14T22:38:30.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063118469999999996, 2024-09-14T22:38:30.639 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:30.639 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:30.639 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:30.639 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:30.639 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:30.640 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:30.640 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145257 2024-09-14T22:38:30.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.640 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:30.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:30.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694179+0000", 2024-09-14T22:38:30.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:30.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694179+0000", 2024-09-14T22:38:30.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:30.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694180+0000", 2024-09-14T22:38:30.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1099999999999995e-07 2024-09-14T22:38:30.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:30.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694196+0000", 2024-09-14T22:38:30.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5996000000000001e-05 2024-09-14T22:38:30.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:30.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694208+0000", 2024-09-14T22:38:30.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2584e-05 2024-09-14T22:38:30.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:30.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694219+0000", 2024-09-14T22:38:30.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0462000000000001e-05 2024-09-14T22:38:30.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:30.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757033+0000", 2024-09-14T22:38:30.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062814485000000003 2024-09-14T22:38:30.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:30.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757183+0000", 2024-09-14T22:38:30.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014914099999999999 2024-09-14T22:38:30.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:30.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757298+0000", 2024-09-14T22:38:30.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115091 2024-09-14T22:38:30.643 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.643 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:30.643 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.643 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145258 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16269312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:30.643 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694257+0000", 2024-09-14T22:38:30.644 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.905410519999997, 2024-09-14T22:38:30.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063297279999999997, 2024-09-14T22:38:30.644 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:30.644 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:30.644 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:30.644 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:30.644 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:30.644 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:30.644 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145258 2024-09-14T22:38:30.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.644 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:30.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.644 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:30.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694257+0000", 2024-09-14T22:38:30.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:30.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694257+0000", 2024-09-14T22:38:30.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:30.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694258+0000", 2024-09-14T22:38:30.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7700000000000004e-07 2024-09-14T22:38:30.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:30.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694263+0000", 2024-09-14T22:38:30.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6609999999999999e-06 2024-09-14T22:38:30.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:30.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694271+0000", 2024-09-14T22:38:30.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7809999999999994e-06 2024-09-14T22:38:30.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:30.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694277+0000", 2024-09-14T22:38:30.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.823e-06 2024-09-14T22:38:30.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:30.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757377+0000", 2024-09-14T22:38:30.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063100081000000002 2024-09-14T22:38:30.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:30.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757476+0000", 2024-09-14T22:38:30.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8507999999999999e-05 2024-09-14T22:38:30.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:30.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757555+0000", 2024-09-14T22:38:30.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8749000000000001e-05 2024-09-14T22:38:30.647 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.647 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:30.648 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.648 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145259 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16465920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:30.648 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694286+0000", 2024-09-14T22:38:30.648 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.905382322999998, 2024-09-14T22:38:30.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063608168000000007, 2024-09-14T22:38:30.648 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:30.648 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:30.648 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:30.648 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:30.648 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:30.648 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:30.649 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145259 2024-09-14T22:38:30.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.649 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:30.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:30.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694286+0000", 2024-09-14T22:38:30.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:30.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694286+0000", 2024-09-14T22:38:30.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:30.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694286+0000", 2024-09-14T22:38:30.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9999999999999998e-07 2024-09-14T22:38:30.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:30.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694288+0000", 2024-09-14T22:38:30.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3760000000000002e-06 2024-09-14T22:38:30.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:30.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694294+0000", 2024-09-14T22:38:30.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6779999999999997e-06 2024-09-14T22:38:30.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:30.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694298+0000", 2024-09-14T22:38:30.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3309999999999997e-06 2024-09-14T22:38:30.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:30.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757638+0000", 2024-09-14T22:38:30.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063339464999999998 2024-09-14T22:38:30.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:30.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757769+0000", 2024-09-14T22:38:30.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013148700000000001 2024-09-14T22:38:30.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:30.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757894+0000", 2024-09-14T22:38:30.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012443099999999999 2024-09-14T22:38:30.652 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.652 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:30.652 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.652 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145260 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16367616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:30.653 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694304+0000", 2024-09-14T22:38:30.653 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.905363528999999, 2024-09-14T22:38:30.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063838521999999995, 2024-09-14T22:38:30.653 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:30.653 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:30.653 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:30.653 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:30.653 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:30.653 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:30.653 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145260 2024-09-14T22:38:30.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.653 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:30.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:30.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694304+0000", 2024-09-14T22:38:30.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:30.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694304+0000", 2024-09-14T22:38:30.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:30.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694305+0000", 2024-09-14T22:38:30.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8299999999999997e-07 2024-09-14T22:38:30.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:30.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694307+0000", 2024-09-14T22:38:30.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9989999999999998e-06 2024-09-14T22:38:30.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:30.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694312+0000", 2024-09-14T22:38:30.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2689999999999999e-06 2024-09-14T22:38:30.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:30.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694317+0000", 2024-09-14T22:38:30.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9359999999999998e-06 2024-09-14T22:38:30.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:30.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757944+0000", 2024-09-14T22:38:30.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063626951000000001 2024-09-14T22:38:30.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:30.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758065+0000", 2024-09-14T22:38:30.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120945 2024-09-14T22:38:30.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:30.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758143+0000", 2024-09-14T22:38:30.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7838999999999998e-05 2024-09-14T22:38:30.656 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.657 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:30.657 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.657 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145261 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16564224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:30.657 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694323+0000", 2024-09-14T22:38:30.657 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.905344825999997, 2024-09-14T22:38:30.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064150180000000001, 2024-09-14T22:38:30.657 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:30.657 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:30.657 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:30.657 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:30.657 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:30.658 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:30.658 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145261 2024-09-14T22:38:30.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.658 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:30.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:30.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694323+0000", 2024-09-14T22:38:30.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:30.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694323+0000", 2024-09-14T22:38:30.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:30.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694323+0000", 2024-09-14T22:38:30.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4900000000000001e-07 2024-09-14T22:38:30.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:30.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694325+0000", 2024-09-14T22:38:30.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.04e-06 2024-09-14T22:38:30.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:30.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694331+0000", 2024-09-14T22:38:30.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3449999999999996e-06 2024-09-14T22:38:30.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:30.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694335+0000", 2024-09-14T22:38:30.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.566e-06 2024-09-14T22:38:30.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:30.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758225+0000", 2024-09-14T22:38:30.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063889535999999997 2024-09-14T22:38:30.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:30.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758357+0000", 2024-09-14T22:38:30.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013181 2024-09-14T22:38:30.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:30.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758473+0000", 2024-09-14T22:38:30.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011653400000000001 2024-09-14T22:38:30.661 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.661 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:30.661 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.661 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145262 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16760832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:30.661 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694341+0000", 2024-09-14T22:38:30.661 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.905326967999997, 2024-09-14T22:38:30.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064424631999999996, 2024-09-14T22:38:30.661 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:30.661 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:30.662 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:30.662 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:30.662 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:30.662 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:30.662 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145262 2024-09-14T22:38:30.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.662 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:30.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:30.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694341+0000", 2024-09-14T22:38:30.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:30.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694341+0000", 2024-09-14T22:38:30.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:30.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694341+0000", 2024-09-14T22:38:30.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5499999999999999e-07 2024-09-14T22:38:30.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:30.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694343+0000", 2024-09-14T22:38:30.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9630000000000001e-06 2024-09-14T22:38:30.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:30.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694349+0000", 2024-09-14T22:38:30.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3120000000000001e-06 2024-09-14T22:38:30.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:30.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694353+0000", 2024-09-14T22:38:30.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6500000000000004e-06 2024-09-14T22:38:30.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:30.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758561+0000", 2024-09-14T22:38:30.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064207934999999994 2024-09-14T22:38:30.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:30.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758677+0000", 2024-09-14T22:38:30.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116241 2024-09-14T22:38:30.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:30.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758766+0000", 2024-09-14T22:38:30.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8176000000000004e-05 2024-09-14T22:38:30.665 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.666 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:30.666 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.666 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145263 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16662528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:30.666 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694359+0000", 2024-09-14T22:38:30.666 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.905309125000002, 2024-09-14T22:38:30.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064743189000000007, 2024-09-14T22:38:30.666 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:30.666 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:30.666 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:30.667 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:30.667 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:30.667 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:30.667 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145263 2024-09-14T22:38:30.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.667 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:30.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:30.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694359+0000", 2024-09-14T22:38:30.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:30.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694359+0000", 2024-09-14T22:38:30.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:30.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694359+0000", 2024-09-14T22:38:30.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5100000000000001e-07 2024-09-14T22:38:30.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:30.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694361+0000", 2024-09-14T22:38:30.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9429999999999999e-06 2024-09-14T22:38:30.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:30.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694366+0000", 2024-09-14T22:38:30.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3489999999999999e-06 2024-09-14T22:38:30.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:30.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694371+0000", 2024-09-14T22:38:30.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4399999999999998e-06 2024-09-14T22:38:30.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:30.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758848+0000", 2024-09-14T22:38:30.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064477396000000006 2024-09-14T22:38:30.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:30.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758980+0000", 2024-09-14T22:38:30.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013162199999999999 2024-09-14T22:38:30.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:30.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759102+0000", 2024-09-14T22:38:30.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012208799999999999 2024-09-14T22:38:30.671 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.671 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:30.671 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.671 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145264 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16334848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:30.671 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694377+0000", 2024-09-14T22:38:30.671 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.905291091000002, 2024-09-14T22:38:30.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065048054999999994, 2024-09-14T22:38:30.671 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:30.671 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:30.671 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:30.672 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:30.672 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:30.672 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:30.672 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145264 2024-09-14T22:38:30.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.672 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:30.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:30.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694377+0000", 2024-09-14T22:38:30.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:30.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694377+0000", 2024-09-14T22:38:30.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:30.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694377+0000", 2024-09-14T22:38:30.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.39e-07 2024-09-14T22:38:30.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:30.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694379+0000", 2024-09-14T22:38:30.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0190000000000001e-06 2024-09-14T22:38:30.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:30.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694385+0000", 2024-09-14T22:38:30.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4399999999999996e-06 2024-09-14T22:38:30.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:30.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694389+0000", 2024-09-14T22:38:30.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.639e-06 2024-09-14T22:38:30.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:30.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759175+0000", 2024-09-14T22:38:30.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064785899999999993 2024-09-14T22:38:30.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:30.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759309+0000", 2024-09-14T22:38:30.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013374400000000001 2024-09-14T22:38:30.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:30.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759425+0000", 2024-09-14T22:38:30.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115974 2024-09-14T22:38:30.675 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.675 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:30.675 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.675 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145265 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16531456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:30.676 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694395+0000", 2024-09-14T22:38:30.676 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.905272969000002, 2024-09-14T22:38:30.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065298325000000004, 2024-09-14T22:38:30.676 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:30.676 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:30.676 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:30.676 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:30.676 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:30.676 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:30.676 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145265 2024-09-14T22:38:30.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.676 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:30.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:30.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694395+0000", 2024-09-14T22:38:30.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:30.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694395+0000", 2024-09-14T22:38:30.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:30.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694395+0000", 2024-09-14T22:38:30.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6699999999999999e-07 2024-09-14T22:38:30.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:30.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694397+0000", 2024-09-14T22:38:30.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.984e-06 2024-09-14T22:38:30.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:30.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694403+0000", 2024-09-14T22:38:30.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3519999999999997e-06 2024-09-14T22:38:30.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:30.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694407+0000", 2024-09-14T22:38:30.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3239999999999996e-06 2024-09-14T22:38:30.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:30.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759516+0000", 2024-09-14T22:38:30.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065109279000000006 2024-09-14T22:38:30.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:30.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759610+0000", 2024-09-14T22:38:30.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3566000000000005e-05 2024-09-14T22:38:30.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:30.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759693+0000", 2024-09-14T22:38:30.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3453000000000002e-05 2024-09-14T22:38:30.679 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.679 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:30.680 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.680 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145266 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16433152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:30.680 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694413+0000", 2024-09-14T22:38:30.680 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.905255169999997, 2024-09-14T22:38:30.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065618308, 2024-09-14T22:38:30.680 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:30.680 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:30.680 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:30.680 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:30.680 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:30.680 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:30.680 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145266 2024-09-14T22:38:30.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.681 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:30.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.681 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:30.681 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694413+0000", 2024-09-14T22:38:30.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.681 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:30.681 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694413+0000", 2024-09-14T22:38:30.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:30.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694413+0000", 2024-09-14T22:38:30.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4799999999999999e-07 2024-09-14T22:38:30.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:30.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694415+0000", 2024-09-14T22:38:30.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.08e-06 2024-09-14T22:38:30.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:30.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694420+0000", 2024-09-14T22:38:30.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.203e-06 2024-09-14T22:38:30.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:30.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694425+0000", 2024-09-14T22:38:30.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4649999999999996e-06 2024-09-14T22:38:30.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:30.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759768+0000", 2024-09-14T22:38:30.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065343613999999994 2024-09-14T22:38:30.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:30.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759909+0000", 2024-09-14T22:38:30.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014054099999999999 2024-09-14T22:38:30.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:30.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760031+0000", 2024-09-14T22:38:30.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122057 2024-09-14T22:38:30.684 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.684 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:30.684 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.684 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145267 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16629760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:30.685 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694431+0000", 2024-09-14T22:38:30.685 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.905237255999999, 2024-09-14T22:38:30.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065850237000000006, 2024-09-14T22:38:30.685 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:30.685 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:30.685 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:30.685 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:30.685 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:30.685 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:30.685 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145267 2024-09-14T22:38:30.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.685 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:30.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:30.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694431+0000", 2024-09-14T22:38:30.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:30.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694431+0000", 2024-09-14T22:38:30.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:30.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694431+0000", 2024-09-14T22:38:30.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4799999999999999e-07 2024-09-14T22:38:30.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:30.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694433+0000", 2024-09-14T22:38:30.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8470000000000001e-06 2024-09-14T22:38:30.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:30.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694438+0000", 2024-09-14T22:38:30.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3249999999999998e-06 2024-09-14T22:38:30.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:30.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694442+0000", 2024-09-14T22:38:30.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3490000000000002e-06 2024-09-14T22:38:30.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:30.688 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760113+0000", 2024-09-14T22:38:30.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065670872000000005 2024-09-14T22:38:30.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.688 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:30.688 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760199+0000", 2024-09-14T22:38:30.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5982000000000004e-05 2024-09-14T22:38:30.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.688 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:30.688 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760281+0000", 2024-09-14T22:38:30.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1514000000000006e-05 2024-09-14T22:38:30.688 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.688 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:30.688 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.689 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145268 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16728064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:30.689 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694448+0000", 2024-09-14T22:38:30.689 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.905219672000001, 2024-09-14T22:38:30.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066157585000000005, 2024-09-14T22:38:30.689 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:30.689 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:30.689 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:30.689 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:30.689 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:30.689 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:30.689 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145268 2024-09-14T22:38:30.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.690 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:30.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:30.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694448+0000", 2024-09-14T22:38:30.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:30.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694448+0000", 2024-09-14T22:38:30.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:30.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:30.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694449+0000", 2024-09-14T22:38:30.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.39e-07 2024-09-14T22:38:30.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:30.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694450+0000", 2024-09-14T22:38:30.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.906e-06 2024-09-14T22:38:30.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:30.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694456+0000", 2024-09-14T22:38:30.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3820000000000003e-06 2024-09-14T22:38:30.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:30.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694460+0000", 2024-09-14T22:38:30.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6290000000000001e-06 2024-09-14T22:38:30.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.692 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:30.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760345+0000", 2024-09-14T22:38:30.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065884540000000005 2024-09-14T22:38:30.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.692 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:30.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760482+0000", 2024-09-14T22:38:30.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013731600000000001 2024-09-14T22:38:30.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:30.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:30.692 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:30.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760606+0000", 2024-09-14T22:38:30.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000123473 2024-09-14T22:38:30.693 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.693 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:30.693 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.693 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:30.693 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:30.693 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:30.694 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:38:30.695 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:30.693+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:30.796 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:30.793+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:30.838 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:30.838 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:30.839 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:38:30.839 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:30.855 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:38:30.897 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:30.893+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:30.985 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:30.985 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:30.985 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:38:30.986 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:38:30.986 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:30.997 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:30.993+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:31.000 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:38:31.069 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:38:31.118 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:31.117+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:31.123 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:31.123 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:38:31.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:38:31.124 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:38:31.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.124 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6395 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8386560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:38:31.124 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:38:31.124 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 127.185122394, 2024-09-14T22:38:31.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054534379000000001, 2024-09-14T22:38:31.124 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:31.124 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:31.124 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:31.124 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:31.124 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:31.124 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:31.124 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6395 2024-09-14T22:38:31.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.125 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:31.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:31.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:38:31.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:31.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:38:31.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.126 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:31.126 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933833+0000", 2024-09-14T22:38:31.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3859999999999999e-06 2024-09-14T22:38:31.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.126 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:31.126 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933838+0000", 2024-09-14T22:38:31.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8709999999999996e-06 2024-09-14T22:38:31.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.126 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:31.126 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933854+0000", 2024-09-14T22:38:31.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6189e-05 2024-09-14T22:38:31.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:31.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933871+0000", 2024-09-14T22:38:31.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.626e-05 2024-09-14T22:38:31.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:31.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988198+0000", 2024-09-14T22:38:31.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054327581999999999 2024-09-14T22:38:31.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:31.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988274+0000", 2024-09-14T22:38:31.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5759000000000004e-05 2024-09-14T22:38:31.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:31.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988366+0000", 2024-09-14T22:38:31.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2331999999999996e-05 2024-09-14T22:38:31.128 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.128 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:31.128 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.128 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104944 3.3 3:c3780f3b:::rbd_data.113ca561f7a9.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:31.128 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:38:31.128 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 72.085520521000007, 2024-09-14T22:38:31.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073041087000000005, 2024-09-14T22:38:31.128 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:31.129 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:31.129 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:31.129 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:31.129 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:31.129 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:31.129 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104944 2024-09-14T22:38:31.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.129 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:31.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:31.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:38:31.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:31.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:38:31.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:31.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033436+0000", 2024-09-14T22:38:31.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2060000000000001e-06 2024-09-14T22:38:31.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:31.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033448+0000", 2024-09-14T22:38:31.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.169e-05 2024-09-14T22:38:31.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:31.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033474+0000", 2024-09-14T22:38:31.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6400000000000001e-05 2024-09-14T22:38:31.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:31.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033489+0000", 2024-09-14T22:38:31.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5067e-05 2024-09-14T22:38:31.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:31.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033658+0000", 2024-09-14T22:38:31.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000168565 2024-09-14T22:38:31.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:31.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033793+0000", 2024-09-14T22:38:31.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013536900000000001 2024-09-14T22:38:31.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:38:31.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.047559+0000", 2024-09-14T22:38:31.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013766294 2024-09-14T22:38:31.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:38:31.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106157+0000", 2024-09-14T22:38:31.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058597110000000001 2024-09-14T22:38:31.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:31.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106290+0000", 2024-09-14T22:38:31.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000133244 2024-09-14T22:38:31.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:38:31.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106396+0000", 2024-09-14T22:38:31.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106246 2024-09-14T22:38:31.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:31.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106475+0000", 2024-09-14T22:38:31.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8895999999999998e-05 2024-09-14T22:38:31.134 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.134 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:31.134 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.134 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:111858 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 10403840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:31.134 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:38:31.134 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 63.295067400000001, 2024-09-14T22:38:31.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054398798999999998, 2024-09-14T22:38:31.134 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:31.134 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:31.135 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:31.135 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:31.135 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:31.135 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:31.135 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 111858 2024-09-14T22:38:31.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.135 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:31.135 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.135 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:31.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:38:31.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:31.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:38:31.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:31.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823888+0000", 2024-09-14T22:38:31.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0200000000000001e-07 2024-09-14T22:38:31.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:31.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823927+0000", 2024-09-14T22:38:31.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8940999999999999e-05 2024-09-14T22:38:31.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:31.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823940+0000", 2024-09-14T22:38:31.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3682e-05 2024-09-14T22:38:31.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:31.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823950+0000", 2024-09-14T22:38:31.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7110000000000007e-06 2024-09-14T22:38:31.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:31.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878202+0000", 2024-09-14T22:38:31.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054251969999999997 2024-09-14T22:38:31.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:31.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878249+0000", 2024-09-14T22:38:31.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6854999999999998e-05 2024-09-14T22:38:31.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:31.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878286+0000", 2024-09-14T22:38:31.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6838000000000002e-05 2024-09-14T22:38:31.138 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.139 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:31.139 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.139 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:111859 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 10452992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:31.139 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:38:31.139 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 63.294978280999999, 2024-09-14T22:38:31.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054435292000000003, 2024-09-14T22:38:31.139 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:31.139 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:31.139 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:31.139 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:31.139 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:31.140 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:31.140 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 111859 2024-09-14T22:38:31.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.140 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:31.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.140 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:31.140 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:38:31.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.140 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:31.140 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:38:31.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:31.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823977+0000", 2024-09-14T22:38:31.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3500000000000005e-07 2024-09-14T22:38:31.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:31.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823981+0000", 2024-09-14T22:38:31.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2050000000000004e-06 2024-09-14T22:38:31.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:31.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824007+0000", 2024-09-14T22:38:31.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.62e-05 2024-09-14T22:38:31.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:31.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824018+0000", 2024-09-14T22:38:31.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0389e-05 2024-09-14T22:38:31.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:31.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878312+0000", 2024-09-14T22:38:31.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054294754000000001 2024-09-14T22:38:31.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.143 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:31.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878350+0000", 2024-09-14T22:38:31.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7982000000000003e-05 2024-09-14T22:38:31.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.143 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:31.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878411+0000", 2024-09-14T22:38:31.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0927000000000001e-05 2024-09-14T22:38:31.143 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.143 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:31.143 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.144 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112030 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6316032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:31.144 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:38:31.144 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 63.294923939, 2024-09-14T22:38:31.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054597826000000002, 2024-09-14T22:38:31.144 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:31.144 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:31.144 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:31.144 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:31.144 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:31.144 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:31.144 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112030 2024-09-14T22:38:31.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.144 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:31.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:31.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:38:31.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:31.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:38:31.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:31.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824031+0000", 2024-09-14T22:38:31.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3600000000000003e-07 2024-09-14T22:38:31.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:31.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824036+0000", 2024-09-14T22:38:31.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7350000000000004e-06 2024-09-14T22:38:31.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:31.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824049+0000", 2024-09-14T22:38:31.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3329000000000001e-05 2024-09-14T22:38:31.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:31.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824057+0000", 2024-09-14T22:38:31.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2460000000000003e-06 2024-09-14T22:38:31.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:31.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878464+0000", 2024-09-14T22:38:31.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054406207999999998 2024-09-14T22:38:31.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:31.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878554+0000", 2024-09-14T22:38:31.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0199999999999997e-05 2024-09-14T22:38:31.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.148 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:31.148 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878628+0000", 2024-09-14T22:38:31.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4371999999999999e-05 2024-09-14T22:38:31.148 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.148 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:31.148 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.148 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112031 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6365184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:31.148 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:38:31.148 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 63.294885620999999, 2024-09-14T22:38:31.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054741868999999999, 2024-09-14T22:38:31.148 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:31.149 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:31.149 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:31.149 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:31.149 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:31.149 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:31.149 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112031 2024-09-14T22:38:31.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.149 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:31.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.149 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:31.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:38:31.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:31.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:38:31.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:31.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824070+0000", 2024-09-14T22:38:31.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5099999999999998e-07 2024-09-14T22:38:31.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:31.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824073+0000", 2024-09-14T22:38:31.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6940000000000001e-06 2024-09-14T22:38:31.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:31.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824084+0000", 2024-09-14T22:38:31.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0380000000000001e-05 2024-09-14T22:38:31.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:31.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824091+0000", 2024-09-14T22:38:31.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5730000000000002e-06 2024-09-14T22:38:31.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:31.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878670+0000", 2024-09-14T22:38:31.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054579176 2024-09-14T22:38:31.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:31.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878742+0000", 2024-09-14T22:38:31.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T22:38:31.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.152 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:31.152 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878811+0000", 2024-09-14T22:38:31.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8170999999999995e-05 2024-09-14T22:38:31.152 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.152 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:31.152 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.152 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112032 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6414336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:31.152 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:38:31.152 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 63.294851755000003, 2024-09-14T22:38:31.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054931916999999997, 2024-09-14T22:38:31.153 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:31.153 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:31.153 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:31.153 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:31.153 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:31.153 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:31.153 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112032 2024-09-14T22:38:31.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.153 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:31.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:31.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:38:31.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:31.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:38:31.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:31.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:38:31.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:38:31.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:31.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824107+0000", 2024-09-14T22:38:31.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5640000000000001e-06 2024-09-14T22:38:31.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:31.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824117+0000", 2024-09-14T22:38:31.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0366e-05 2024-09-14T22:38:31.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:31.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824158+0000", 2024-09-14T22:38:31.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0670000000000002e-05 2024-09-14T22:38:31.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:31.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878853+0000", 2024-09-14T22:38:31.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054694853000000002 2024-09-14T22:38:31.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:31.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878934+0000", 2024-09-14T22:38:31.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1754000000000004e-05 2024-09-14T22:38:31.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:31.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879034+0000", 2024-09-14T22:38:31.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000100115 2024-09-14T22:38:31.156 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.156 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:31.156 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.156 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112033 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6463488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:31.157 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:38:31.157 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 63.294783097, 2024-09-14T22:38:31.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054991076999999999, 2024-09-14T22:38:31.157 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:31.157 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:31.157 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:31.157 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:31.157 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:31.157 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:31.157 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112033 2024-09-14T22:38:31.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.157 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:31.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:31.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:38:31.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:31.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:38:31.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:31.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824172+0000", 2024-09-14T22:38:31.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9999999999999996e-07 2024-09-14T22:38:31.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:31.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824177+0000", 2024-09-14T22:38:31.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5310000000000003e-06 2024-09-14T22:38:31.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:31.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824188+0000", 2024-09-14T22:38:31.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1235e-05 2024-09-14T22:38:31.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:31.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824197+0000", 2024-09-14T22:38:31.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7069999999999998e-06 2024-09-14T22:38:31.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:31.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879075+0000", 2024-09-14T22:38:31.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054878749999999997 2024-09-14T22:38:31.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:31.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879118+0000", 2024-09-14T22:38:31.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2749e-05 2024-09-14T22:38:31.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:31.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879162+0000", 2024-09-14T22:38:31.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4205000000000001e-05 2024-09-14T22:38:31.160 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.160 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:31.160 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.161 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112034 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6512640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:31.161 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:38:31.161 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 63.294746146000001, 2024-09-14T22:38:31.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055181246000000003, 2024-09-14T22:38:31.161 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:31.161 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:31.161 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:31.161 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:31.161 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:31.161 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:31.161 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112034 2024-09-14T22:38:31.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.161 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:31.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:38:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:38:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824209+0000", 2024-09-14T22:38:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5499999999999998e-07 2024-09-14T22:38:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824212+0000", 2024-09-14T22:38:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6119999999999999e-06 2024-09-14T22:38:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824224+0000", 2024-09-14T22:38:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1127999999999999e-05 2024-09-14T22:38:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824232+0000", 2024-09-14T22:38:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4700000000000002e-06 2024-09-14T22:38:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879209+0000", 2024-09-14T22:38:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054977201000000003 2024-09-14T22:38:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879308+0000", 2024-09-14T22:38:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8419999999999996e-05 2024-09-14T22:38:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879389+0000", 2024-09-14T22:38:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1760000000000003e-05 2024-09-14T22:38:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112035 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6561792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:38:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 63.294652417999998, 2024-09-14T22:38:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055300522999999997, 2024-09-14T22:38:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112035 2024-09-14T22:38:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:38:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:38:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824303+0000", 2024-09-14T22:38:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5099999999999998e-07 2024-09-14T22:38:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824308+0000", 2024-09-14T22:38:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6689999999999997e-06 2024-09-14T22:38:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824744+0000", 2024-09-14T22:38:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00043601600000000001 2024-09-14T22:38:31.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:31.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824756+0000", 2024-09-14T22:38:31.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2476999999999999e-05 2024-09-14T22:38:31.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:31.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879434+0000", 2024-09-14T22:38:31.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054677630999999997 2024-09-14T22:38:31.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879518+0000", 2024-09-14T22:38:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4220000000000003e-05 2024-09-14T22:38:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879602+0000", 2024-09-14T22:38:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4559000000000001e-05 2024-09-14T22:38:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112036 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6610944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:38:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 63.294160679999997, 2024-09-14T22:38:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055036320999999999, 2024-09-14T22:38:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112036 2024-09-14T22:38:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:38:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:38:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824795+0000", 2024-09-14T22:38:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-06 2024-09-14T22:38:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824802+0000", 2024-09-14T22:38:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6680000000000004e-06 2024-09-14T22:38:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824814+0000", 2024-09-14T22:38:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2102e-05 2024-09-14T22:38:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824823+0000", 2024-09-14T22:38:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5550000000000008e-06 2024-09-14T22:38:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879648+0000", 2024-09-14T22:38:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054825167000000001 2024-09-14T22:38:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879747+0000", 2024-09-14T22:38:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8968999999999994e-05 2024-09-14T22:38:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879830+0000", 2024-09-14T22:38:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2633999999999995e-05 2024-09-14T22:38:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112037 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6660096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:38:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 63.294117970999999, 2024-09-14T22:38:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055122926000000003, 2024-09-14T22:38:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112037 2024-09-14T22:38:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:38:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:38:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824837+0000", 2024-09-14T22:38:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5499999999999998e-07 2024-09-14T22:38:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824840+0000", 2024-09-14T22:38:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0429999999999999e-06 2024-09-14T22:38:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824851+0000", 2024-09-14T22:38:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1046999999999999e-05 2024-09-14T22:38:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824859+0000", 2024-09-14T22:38:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1200000000000002e-06 2024-09-14T22:38:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879867+0000", 2024-09-14T22:38:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055007697000000001 2024-09-14T22:38:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879910+0000", 2024-09-14T22:38:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3402000000000002e-05 2024-09-14T22:38:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879959+0000", 2024-09-14T22:38:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9061999999999999e-05 2024-09-14T22:38:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112038 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6709248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:38:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 63.294083274999998, 2024-09-14T22:38:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055312828000000001, 2024-09-14T22:38:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112038 2024-09-14T22:38:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:38:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:38:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824872+0000", 2024-09-14T22:38:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T22:38:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824875+0000", 2024-09-14T22:38:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7519999999999998e-06 2024-09-14T22:38:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824917+0000", 2024-09-14T22:38:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1047999999999997e-05 2024-09-14T22:38:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824926+0000", 2024-09-14T22:38:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3749999999999992e-06 2024-09-14T22:38:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880006+0000", 2024-09-14T22:38:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055080525999999998 2024-09-14T22:38:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880102+0000", 2024-09-14T22:38:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5995999999999994e-05 2024-09-14T22:38:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880184+0000", 2024-09-14T22:38:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1478999999999997e-05 2024-09-14T22:38:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112039 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6758400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:38:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 63.294014605000001, 2024-09-14T22:38:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055449709999999999, 2024-09-14T22:38:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112039 2024-09-14T22:38:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:38:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:38:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824941+0000", 2024-09-14T22:38:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3699999999999999e-07 2024-09-14T22:38:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824946+0000", 2024-09-14T22:38:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0209999999999999e-06 2024-09-14T22:38:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824972+0000", 2024-09-14T22:38:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6750999999999998e-05 2024-09-14T22:38:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824982+0000", 2024-09-14T22:38:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5019999999999998e-06 2024-09-14T22:38:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880227+0000", 2024-09-14T22:38:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055245257999999998 2024-09-14T22:38:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880305+0000", 2024-09-14T22:38:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7424999999999998e-05 2024-09-14T22:38:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880389+0000", 2024-09-14T22:38:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4815999999999996e-05 2024-09-14T22:38:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.187 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:31.187 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.187 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112040 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6807552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:31.187 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:38:31.187 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 63.293957888999998, 2024-09-14T22:38:31.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055534615000000002, 2024-09-14T22:38:31.187 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:31.187 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:31.187 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:31.187 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:31.187 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:31.187 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:31.187 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112040 2024-09-14T22:38:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:38:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:38:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824997+0000", 2024-09-14T22:38:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7199999999999998e-07 2024-09-14T22:38:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825042+0000", 2024-09-14T22:38:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5065999999999998e-05 2024-09-14T22:38:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825060+0000", 2024-09-14T22:38:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7520000000000002e-05 2024-09-14T22:38:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825070+0000", 2024-09-14T22:38:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8230000000000006e-06 2024-09-14T22:38:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880427+0000", 2024-09-14T22:38:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055357244 2024-09-14T22:38:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880483+0000", 2024-09-14T22:38:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5807000000000003e-05 2024-09-14T22:38:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880531+0000", 2024-09-14T22:38:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8383000000000001e-05 2024-09-14T22:38:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112041 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6856704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:38:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 63.293868715000002, 2024-09-14T22:38:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055573887000000002, 2024-09-14T22:38:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112041 2024-09-14T22:38:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:38:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:38:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825087+0000", 2024-09-14T22:38:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2399999999999996e-07 2024-09-14T22:38:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825091+0000", 2024-09-14T22:38:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9729999999999999e-06 2024-09-14T22:38:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825101+0000", 2024-09-14T22:38:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.093e-05 2024-09-14T22:38:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825110+0000", 2024-09-14T22:38:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1450000000000008e-06 2024-09-14T22:38:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880559+0000", 2024-09-14T22:38:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055449684999999999 2024-09-14T22:38:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880601+0000", 2024-09-14T22:38:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1248000000000002e-05 2024-09-14T22:38:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880659+0000", 2024-09-14T22:38:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8981999999999999e-05 2024-09-14T22:38:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112042 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6905856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:38:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 63.293833980999999, 2024-09-14T22:38:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055781415000000001, 2024-09-14T22:38:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112042 2024-09-14T22:38:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:38:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:38:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825478+0000", 2024-09-14T22:38:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035775 2024-09-14T22:38:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825494+0000", 2024-09-14T22:38:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5994999999999998e-05 2024-09-14T22:38:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825520+0000", 2024-09-14T22:38:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5738999999999999e-05 2024-09-14T22:38:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825547+0000", 2024-09-14T22:38:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7667000000000001e-05 2024-09-14T22:38:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880725+0000", 2024-09-14T22:38:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055177962999999997 2024-09-14T22:38:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880816+0000", 2024-09-14T22:38:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0315000000000005e-05 2024-09-14T22:38:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880902+0000", 2024-09-14T22:38:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5986e-05 2024-09-14T22:38:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112043 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:38:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 63.293385977, 2024-09-14T22:38:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055537912000000002, 2024-09-14T22:38:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112043 2024-09-14T22:38:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:38:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:38:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825911+0000", 2024-09-14T22:38:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00034306899999999999 2024-09-14T22:38:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825918+0000", 2024-09-14T22:38:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.002e-06 2024-09-14T22:38:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825931+0000", 2024-09-14T22:38:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2401e-05 2024-09-14T22:38:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825942+0000", 2024-09-14T22:38:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0903e-05 2024-09-14T22:38:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880951+0000", 2024-09-14T22:38:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055009379999999997 2024-09-14T22:38:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881027+0000", 2024-09-14T22:38:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5944000000000003e-05 2024-09-14T22:38:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881106+0000", 2024-09-14T22:38:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9213000000000002e-05 2024-09-14T22:38:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112044 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 7004160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:38:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 63.292992308000002, 2024-09-14T22:38:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055265416999999997, 2024-09-14T22:38:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112044 2024-09-14T22:38:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:38:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:38:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826239+0000", 2024-09-14T22:38:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027676000000000003 2024-09-14T22:38:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826247+0000", 2024-09-14T22:38:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7440000000000007e-06 2024-09-14T22:38:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826275+0000", 2024-09-14T22:38:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8053000000000001e-05 2024-09-14T22:38:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826286+0000", 2024-09-14T22:38:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1780000000000001e-05 2024-09-14T22:38:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881137+0000", 2024-09-14T22:38:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054850500000000003 2024-09-14T22:38:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881181+0000", 2024-09-14T22:38:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4153000000000002e-05 2024-09-14T22:38:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881227+0000", 2024-09-14T22:38:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6427000000000001e-05 2024-09-14T22:38:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112045 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 7053312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:38:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 63.292630074999998, 2024-09-14T22:38:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055045050999999998, 2024-09-14T22:38:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112045 2024-09-14T22:38:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:38:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:38:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830250+0000", 2024-09-14T22:38:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0039258670000000004 2024-09-14T22:38:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830260+0000", 2024-09-14T22:38:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0088000000000001e-05 2024-09-14T22:38:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830281+0000", 2024-09-14T22:38:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0720000000000002e-05 2024-09-14T22:38:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830292+0000", 2024-09-14T22:38:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1455e-05 2024-09-14T22:38:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881255+0000", 2024-09-14T22:38:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.050962247000000002 2024-09-14T22:38:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881296+0000", 2024-09-14T22:38:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1557999999999998e-05 2024-09-14T22:38:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881369+0000", 2024-09-14T22:38:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3115999999999996e-05 2024-09-14T22:38:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.212 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:31.212 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:31.212 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:31.213 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:38:31.213 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:31.206+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:31.227 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:38:31.227 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:38:31.244 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:38:31.244 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:38:31.260 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:38:31.260 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:38:31.276 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:38:31.276 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:38:31.295 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:38:31.296 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:38:31.314 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:31.313+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:31.314 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:38:31.314 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:38:31.332 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:38:31.333 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:38:31.353 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:38:31.354 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:38:31.382 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:31.382 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:31.396 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:38:31.414 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:31.413+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:31.513 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:31.514 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:31.515 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:31.513+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:31.528 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:38:31.616 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:31.613+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:31.649 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:31.649 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:31.663 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:38:31.717 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:31.710+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:31.769 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T22:38:31.790 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:31.790 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:31.809 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:38:31.851 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:31.846+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:31.919 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:31.917+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:31.976 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:31.976 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:31.990 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:38:32.019 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:32.017+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:32.113 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:32.113 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:32.120 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:32.114+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:32.127 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:38:32.221 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:32.217+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:32.288 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:32.288 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:32.303 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:38:32.347 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:32.342+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:32.421 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:32.422 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:32.423 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:32.421+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:32.435 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:38:32.523 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:32.521+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:32.596 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:32.597 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:32.610 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:38:32.624 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:32.618+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:32.683 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-14T22:38:32.683 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:38:32.683 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T22:38:32.684 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T22:38:32.725 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:32.718+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:32.747 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:32.747 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:32.764 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:38:32.772 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2024-09-14T22:38:32.850 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:32.849+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:32.919 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:32.919 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:32.927 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:32.922+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:32.937 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:38:33.028 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:33.022+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:33.035 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:38:33.035 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:33.035 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 140, 2024-09-14T22:38:33.035 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "f07f407a-b17d-42c4-bf73-92dd80114a4f", 2024-09-14T22:38:33.036 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-09-14T22:35:45.983055+0000", 2024-09-14T22:38:33.036 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-09-14T22:38:32.759271+0000", 2024-09-14T22:38:33.036 INFO:teuthology.orchestra.run.smithi001.stdout: "last_up_change": "2024-09-14T22:37:12.616042+0000", 2024-09-14T22:38:33.036 INFO:teuthology.orchestra.run.smithi001.stdout: "last_in_change": "2024-09-14T22:35:48.283160+0000", 2024-09-14T22:38:33.036 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T22:38:33.036 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_num": 5804032, 2024-09-14T22:38:33.036 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_set": [ 2024-09-14T22:38:33.036 INFO:teuthology.orchestra.run.smithi001.stdout: "nodeep-scrub", 2024-09-14T22:38:33.036 INFO:teuthology.orchestra.run.smithi001.stdout: "pglog_hardlimit", 2024-09-14T22:38:33.037 INFO:teuthology.orchestra.run.smithi001.stdout: "purged_snapdirs", 2024-09-14T22:38:33.037 INFO:teuthology.orchestra.run.smithi001.stdout: "recovery_deletes", 2024-09-14T22:38:33.037 INFO:teuthology.orchestra.run.smithi001.stdout: "sortbitwise" 2024-09-14T22:38:33.037 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:38:33.037 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_version": 6, 2024-09-14T22:38:33.037 INFO:teuthology.orchestra.run.smithi001.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T22:38:33.037 INFO:teuthology.orchestra.run.smithi001.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T22:38:33.037 INFO:teuthology.orchestra.run.smithi001.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T22:38:33.037 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_snapshot": "", 2024-09-14T22:38:33.038 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_max": 3, 2024-09-14T22:38:33.038 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd": 8, 2024-09-14T22:38:33.038 INFO:teuthology.orchestra.run.smithi001.stdout: "require_min_compat_client": "luminous", 2024-09-14T22:38:33.038 INFO:teuthology.orchestra.run.smithi001.stdout: "min_compat_client": "luminous", 2024-09-14T22:38:33.038 INFO:teuthology.orchestra.run.smithi001.stdout: "require_osd_release": "squid", 2024-09-14T22:38:33.038 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_crimson": false, 2024-09-14T22:38:33.039 INFO:teuthology.orchestra.run.smithi001.stdout: "pools": [ 2024-09-14T22:38:33.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.039 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 1, 2024-09-14T22:38:33.039 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": ".mgr", 2024-09-14T22:38:33.039 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-14T22:35:58.938784+0000", 2024-09-14T22:38:33.039 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-14T22:38:33.040 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool", 2024-09-14T22:38:33.040 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-14T22:38:33.040 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-14T22:38:33.040 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-14T22:38:33.040 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-14T22:38:33.040 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:38:33.040 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:38:33.040 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:38:33.040 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:38:33.041 INFO:teuthology.orchestra.run.smithi001.stdout: "is_stretch_pool": false, 2024-09-14T22:38:33.041 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-14T22:38:33.041 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:38:33.041 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 1, 2024-09-14T22:38:33.041 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 1, 2024-09-14T22:38:33.041 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 1, 2024-09-14T22:38:33.041 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 1, 2024-09-14T22:38:33.041 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 1, 2024-09-14T22:38:33.041 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-14T22:38:33.041 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-14T22:38:33.041 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-14T22:38:33.042 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-14T22:38:33.042 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-14T22:38:33.042 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-14T22:38:33.042 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-14T22:38:33.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.042 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "18", 2024-09-14T22:38:33.042 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-14T22:38:33.043 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:38:33.043 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:38:33.043 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-14T22:38:33.043 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:38:33.043 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 0, 2024-09-14T22:38:33.043 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 0, 2024-09-14T22:38:33.043 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-14T22:38:33.043 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-14T22:38:33.043 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-14T22:38:33.043 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-14T22:38:33.044 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-14T22:38:33.044 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-14T22:38:33.044 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-14T22:38:33.044 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-14T22:38:33.044 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-14T22:38:33.044 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-14T22:38:33.044 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-14T22:38:33.044 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:38:33.044 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:38:33.045 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:38:33.045 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-14T22:38:33.045 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-14T22:38:33.045 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-14T22:38:33.045 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-14T22:38:33.046 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-14T22:38:33.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.046 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-14T22:38:33.046 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-14T22:38:33.046 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-14T22:38:33.046 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:38:33.046 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:38:33.046 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:38:33.046 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:38:33.047 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-14T22:38:33.047 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-14T22:38:33.047 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-14T22:38:33.047 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-14T22:38:33.047 INFO:teuthology.orchestra.run.smithi001.stdout: "options": { 2024-09-14T22:38:33.047 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_max": 32, 2024-09-14T22:38:33.047 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_min": 1 2024-09-14T22:38:33.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.047 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-14T22:38:33.047 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": {} 2024-09-14T22:38:33.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.048 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-14T22:38:33.048 INFO:teuthology.orchestra.run.smithi001.stdout: "score_type": "Fair distribution", 2024-09-14T22:38:33.048 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 8, 2024-09-14T22:38:33.048 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 8, 2024-09-14T22:38:33.048 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 0.25, 2024-09-14T22:38:33.048 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 2, 2024-09-14T22:38:33.048 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 2, 2024-09-14T22:38:33.048 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 0.78881263732910156, 2024-09-14T22:38:33.048 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 0.79000002145767212, 2024-09-14T22:38:33.049 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 0.79000002145767212 2024-09-14T22:38:33.049 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.049 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 2, 2024-09-14T22:38:33.049 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "rbd", 2024-09-14T22:38:33.049 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-14T22:36:04.304242+0000", 2024-09-14T22:38:33.050 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 8193, 2024-09-14T22:38:33.050 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T22:38:33.050 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-14T22:38:33.050 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-14T22:38:33.050 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-14T22:38:33.050 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-14T22:38:33.050 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:38:33.050 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:38:33.050 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:38:33.051 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:38:33.051 INFO:teuthology.orchestra.run.smithi001.stdout: "is_stretch_pool": false, 2024-09-14T22:38:33.051 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-14T22:38:33.051 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:38:33.051 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 8, 2024-09-14T22:38:33.051 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 8, 2024-09-14T22:38:33.051 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 8, 2024-09-14T22:38:33.052 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 8, 2024-09-14T22:38:33.052 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 8, 2024-09-14T22:38:33.052 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-14T22:38:33.052 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-14T22:38:33.052 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-14T22:38:33.052 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-14T22:38:33.052 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-14T22:38:33.052 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-14T22:38:33.052 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-14T22:38:33.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.053 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "22", 2024-09-14T22:38:33.053 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-14T22:38:33.053 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:38:33.053 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:38:33.053 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-14T22:38:33.053 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:38:33.053 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 2, 2024-09-14T22:38:33.053 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 22, 2024-09-14T22:38:33.053 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-14T22:38:33.053 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-14T22:38:33.054 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-14T22:38:33.054 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-14T22:38:33.054 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-14T22:38:33.054 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-14T22:38:33.054 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-14T22:38:33.054 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-14T22:38:33.054 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-14T22:38:33.054 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-14T22:38:33.054 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-14T22:38:33.054 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:38:33.054 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:38:33.054 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:38:33.054 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-14T22:38:33.055 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-14T22:38:33.055 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-14T22:38:33.055 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-14T22:38:33.055 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-14T22:38:33.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.055 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-14T22:38:33.055 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-14T22:38:33.055 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-14T22:38:33.055 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:38:33.055 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:38:33.055 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:38:33.055 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:38:33.055 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-14T22:38:33.055 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-14T22:38:33.056 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-14T22:38:33.056 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-14T22:38:33.056 INFO:teuthology.orchestra.run.smithi001.stdout: "options": {}, 2024-09-14T22:38:33.056 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-14T22:38:33.056 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd": {} 2024-09-14T22:38:33.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.056 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-14T22:38:33.056 INFO:teuthology.orchestra.run.smithi001.stdout: "score_type": "Fair distribution", 2024-09-14T22:38:33.056 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 2, 2024-09-14T22:38:33.056 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 2, 2024-09-14T22:38:33.056 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 1, 2024-09-14T22:38:33.056 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 2, 2024-09-14T22:38:33.056 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 2, 2024-09-14T22:38:33.057 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 0.78881263732910156, 2024-09-14T22:38:33.057 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 0.79000002145767212, 2024-09-14T22:38:33.057 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 0.79000002145767212 2024-09-14T22:38:33.057 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.057 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 3, 2024-09-14T22:38:33.057 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "pool_client.0", 2024-09-14T22:38:33.057 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-14T22:36:14.374249+0000", 2024-09-14T22:38:33.057 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 8193, 2024-09-14T22:38:33.057 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T22:38:33.057 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-14T22:38:33.057 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-14T22:38:33.057 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-14T22:38:33.058 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-14T22:38:33.058 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:38:33.058 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:38:33.058 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:38:33.058 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:38:33.058 INFO:teuthology.orchestra.run.smithi001.stdout: "is_stretch_pool": false, 2024-09-14T22:38:33.058 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-14T22:38:33.058 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:38:33.058 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 32, 2024-09-14T22:38:33.058 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 32, 2024-09-14T22:38:33.058 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 32, 2024-09-14T22:38:33.058 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 32, 2024-09-14T22:38:33.058 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 32, 2024-09-14T22:38:33.058 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-14T22:38:33.058 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-14T22:38:33.059 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-14T22:38:33.059 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-14T22:38:33.059 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-14T22:38:33.059 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-14T22:38:33.059 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-14T22:38:33.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.059 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "140", 2024-09-14T22:38:33.059 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-14T22:38:33.059 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:38:33.059 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:38:33.059 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-14T22:38:33.059 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:38:33.059 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 44, 2024-09-14T22:38:33.059 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 140, 2024-09-14T22:38:33.060 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-14T22:38:33.060 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-14T22:38:33.060 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-14T22:38:33.060 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-14T22:38:33.060 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-14T22:38:33.060 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-14T22:38:33.060 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-14T22:38:33.060 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-14T22:38:33.060 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-14T22:38:33.060 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-14T22:38:33.060 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-14T22:38:33.060 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:38:33.060 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:38:33.061 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:38:33.061 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-14T22:38:33.061 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-14T22:38:33.061 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-14T22:38:33.061 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-14T22:38:33.061 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-14T22:38:33.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.061 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-14T22:38:33.061 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-14T22:38:33.061 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-14T22:38:33.061 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:38:33.061 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:38:33.061 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:38:33.062 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:38:33.062 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-14T22:38:33.062 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-14T22:38:33.062 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-14T22:38:33.062 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-14T22:38:33.062 INFO:teuthology.orchestra.run.smithi001.stdout: "options": {}, 2024-09-14T22:38:33.062 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-14T22:38:33.062 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd": {} 2024-09-14T22:38:33.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.062 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-14T22:38:33.062 INFO:teuthology.orchestra.run.smithi001.stdout: "score_type": "Fair distribution", 2024-09-14T22:38:33.062 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 1.75, 2024-09-14T22:38:33.062 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 1.75, 2024-09-14T22:38:33.062 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 1, 2024-09-14T22:38:33.062 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 1.75, 2024-09-14T22:38:33.063 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 1.75, 2024-09-14T22:38:33.063 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 0.78881263732910156, 2024-09-14T22:38:33.063 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 0.79000002145767212, 2024-09-14T22:38:33.063 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 0.79000002145767212 2024-09-14T22:38:33.063 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.063 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.063 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:38:33.063 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2024-09-14T22:38:33.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.063 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 0, 2024-09-14T22:38:33.063 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "9831fcc5-9040-4a80-99d4-838225e65139", 2024-09-14T22:38:33.063 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:38:33.063 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:38:33.063 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:38:33.063 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:38:33.064 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:38:33.064 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:38:33.064 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:38:33.064 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 115, 2024-09-14T22:38:33.064 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:38:33.064 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:38:33.064 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:38:33.064 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:33.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.064 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:33.064 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6824", 2024-09-14T22:38:33.064 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:38:33.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.064 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:33.065 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6825", 2024-09-14T22:38:33.065 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:38:33.065 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.065 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.065 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:38:33.065 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:33.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.065 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:33.065 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6826", 2024-09-14T22:38:33.065 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:38:33.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.066 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:33.066 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6827", 2024-09-14T22:38:33.066 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:38:33.066 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.066 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.066 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:38:33.066 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:33.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.066 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:33.066 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6830", 2024-09-14T22:38:33.066 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:38:33.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.066 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:33.067 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6831", 2024-09-14T22:38:33.067 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:38:33.067 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.067 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.067 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:38:33.067 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:33.067 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.067 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:33.067 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6828", 2024-09-14T22:38:33.067 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:38:33.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.067 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.067 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:33.067 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6829", 2024-09-14T22:38:33.068 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:38:33.068 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.068 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.068 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6825/1702929600", 2024-09-14T22:38:33.068 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6827/1702929600", 2024-09-14T22:38:33.068 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6831/1702929600", 2024-09-14T22:38:33.068 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6829/1702929600", 2024-09-14T22:38:33.068 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:38:33.068 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:38:33.068 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:38:33.068 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.069 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 1, 2024-09-14T22:38:33.069 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "7f79abe8-115d-46b2-8677-72fcb0c8ab7d", 2024-09-14T22:38:33.069 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:38:33.069 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:38:33.069 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:38:33.069 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 0.1606292724609375, 2024-09-14T22:38:33.069 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:38:33.069 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:38:33.069 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2024-09-14T22:38:33.069 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 128, 2024-09-14T22:38:33.069 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:38:33.069 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:38:33.069 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:38:33.069 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:33.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.070 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:33.070 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6808", 2024-09-14T22:38:33.070 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:38:33.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.070 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:33.070 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6809", 2024-09-14T22:38:33.070 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:38:33.070 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.070 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.070 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:38:33.070 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:33.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.071 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:33.071 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6810", 2024-09-14T22:38:33.071 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:38:33.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.071 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:33.071 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6811", 2024-09-14T22:38:33.071 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:38:33.071 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.071 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.071 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:38:33.071 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:33.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.072 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:33.072 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6814", 2024-09-14T22:38:33.072 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:38:33.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.072 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:33.072 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6815", 2024-09-14T22:38:33.072 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:38:33.072 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.072 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.072 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:38:33.072 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:33.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.073 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:33.073 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6812", 2024-09-14T22:38:33.073 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:38:33.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.073 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:33.073 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6813", 2024-09-14T22:38:33.073 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:38:33.073 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.073 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.073 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6809/719990653", 2024-09-14T22:38:33.074 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6811/719990653", 2024-09-14T22:38:33.074 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6815/719990653", 2024-09-14T22:38:33.074 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6813/719990653", 2024-09-14T22:38:33.074 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:38:33.074 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:38:33.074 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:38:33.074 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.074 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 2, 2024-09-14T22:38:33.074 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "1f5768b2-4db4-4b2e-8d5e-5206f8bde28d", 2024-09-14T22:38:33.074 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:38:33.074 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:38:33.074 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:38:33.075 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:38:33.075 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 14, 2024-09-14T22:38:33.075 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 30, 2024-09-14T22:38:33.075 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 66, 2024-09-14T22:38:33.075 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 128, 2024-09-14T22:38:33.075 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 31, 2024-09-14T22:38:33.075 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:38:33.075 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:38:33.075 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:33.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.075 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:33.075 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6816", 2024-09-14T22:38:33.075 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:38:33.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.076 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:33.076 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6817", 2024-09-14T22:38:33.076 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:38:33.076 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.076 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.076 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:38:33.076 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:33.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.076 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:33.076 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6818", 2024-09-14T22:38:33.076 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:38:33.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.077 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:33.077 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6819", 2024-09-14T22:38:33.077 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:38:33.077 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.077 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.077 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:38:33.077 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:33.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.077 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:33.077 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6822", 2024-09-14T22:38:33.078 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:38:33.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.078 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:33.078 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6823", 2024-09-14T22:38:33.078 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:38:33.078 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.078 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.078 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:38:33.078 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:33.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.079 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:33.079 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6820", 2024-09-14T22:38:33.079 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:38:33.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.079 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:33.079 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6821", 2024-09-14T22:38:33.079 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:38:33.079 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.079 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.079 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6817/1006730601", 2024-09-14T22:38:33.079 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6819/1006730601", 2024-09-14T22:38:33.080 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6823/1006730601", 2024-09-14T22:38:33.080 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6821/1006730601", 2024-09-14T22:38:33.080 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:38:33.080 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:38:33.080 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:38:33.080 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.080 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 3, 2024-09-14T22:38:33.080 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "8103f3c7-7899-4fc3-bb02-dd2273120138", 2024-09-14T22:38:33.080 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:38:33.080 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:38:33.080 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:38:33.081 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 0.852752685546875, 2024-09-14T22:38:33.081 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:38:33.081 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:38:33.081 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:38:33.081 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 128, 2024-09-14T22:38:33.081 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:38:33.081 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:38:33.081 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:38:33.081 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:33.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.081 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:33.081 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6800", 2024-09-14T22:38:33.082 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:38:33.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.082 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:33.082 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6801", 2024-09-14T22:38:33.082 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:38:33.082 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.082 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.082 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:38:33.082 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:33.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.082 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:33.082 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6802", 2024-09-14T22:38:33.083 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:38:33.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.083 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:33.083 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6803", 2024-09-14T22:38:33.083 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:38:33.083 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.083 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.083 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:38:33.083 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:33.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.083 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:33.084 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6806", 2024-09-14T22:38:33.084 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:38:33.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.084 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:33.084 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6807", 2024-09-14T22:38:33.084 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:38:33.084 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.084 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.084 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:38:33.084 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:33.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.085 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:33.085 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6804", 2024-09-14T22:38:33.085 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:38:33.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.085 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:33.085 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6805", 2024-09-14T22:38:33.085 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:38:33.085 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.085 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.085 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6801/750403671", 2024-09-14T22:38:33.086 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6803/750403671", 2024-09-14T22:38:33.086 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6807/750403671", 2024-09-14T22:38:33.086 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6805/750403671", 2024-09-14T22:38:33.086 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:38:33.086 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:38:33.086 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:38:33.086 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.086 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 4, 2024-09-14T22:38:33.086 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "23cce5cb-726e-44c2-8edb-c2567a89dfbc", 2024-09-14T22:38:33.086 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:38:33.086 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:38:33.087 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:38:33.087 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:38:33.087 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:38:33.087 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:38:33.087 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:38:33.087 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 128, 2024-09-14T22:38:33.087 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:38:33.087 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:38:33.087 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:38:33.087 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:33.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.087 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:33.087 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6816", 2024-09-14T22:38:33.088 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:38:33.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.088 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:33.088 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6817", 2024-09-14T22:38:33.088 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:38:33.088 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.088 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.088 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:38:33.088 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:33.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.089 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:33.089 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6818", 2024-09-14T22:38:33.089 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:38:33.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.089 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:33.089 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6819", 2024-09-14T22:38:33.089 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:38:33.089 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.089 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.089 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:38:33.090 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:33.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.090 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:33.090 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6822", 2024-09-14T22:38:33.090 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:38:33.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.090 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:33.090 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6823", 2024-09-14T22:38:33.090 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:38:33.090 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.090 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.090 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:38:33.091 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:33.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.091 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:33.091 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6820", 2024-09-14T22:38:33.091 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:38:33.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.091 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:33.091 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6821", 2024-09-14T22:38:33.091 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:38:33.091 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.091 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.092 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.176:6817/1408471829", 2024-09-14T22:38:33.092 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.176:6819/1408471829", 2024-09-14T22:38:33.092 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.176:6823/1408471829", 2024-09-14T22:38:33.092 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.176:6821/1408471829", 2024-09-14T22:38:33.092 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:38:33.092 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:38:33.092 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:38:33.092 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.092 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 5, 2024-09-14T22:38:33.092 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "eb2f187b-d2f1-49ab-ad5e-ea6dc441a28e", 2024-09-14T22:38:33.093 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:38:33.093 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:38:33.093 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:38:33.093 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 0.297119140625, 2024-09-14T22:38:33.093 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:38:33.093 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:38:33.093 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:38:33.093 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 116, 2024-09-14T22:38:33.093 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:38:33.093 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:38:33.093 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:38:33.093 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:33.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.093 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:33.094 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6805", 2024-09-14T22:38:33.094 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:38:33.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.094 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:33.094 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6808", 2024-09-14T22:38:33.094 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:38:33.094 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.094 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.094 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:38:33.094 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:33.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.094 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:33.095 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6810", 2024-09-14T22:38:33.095 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:38:33.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.095 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:33.095 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6811", 2024-09-14T22:38:33.095 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:38:33.095 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.095 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.095 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:38:33.095 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:33.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.096 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:33.096 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6814", 2024-09-14T22:38:33.096 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:38:33.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.096 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:33.096 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6815", 2024-09-14T22:38:33.096 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:38:33.096 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.096 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.097 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:38:33.097 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:33.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.097 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:33.097 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6812", 2024-09-14T22:38:33.097 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:38:33.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.097 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:33.097 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6813", 2024-09-14T22:38:33.097 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:38:33.097 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.098 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.098 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.176:6808/1945902326", 2024-09-14T22:38:33.098 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.176:6811/1945902326", 2024-09-14T22:38:33.098 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.176:6815/1945902326", 2024-09-14T22:38:33.098 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.176:6813/1945902326", 2024-09-14T22:38:33.098 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:38:33.098 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:38:33.098 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:38:33.098 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.098 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 6, 2024-09-14T22:38:33.098 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "d371aed8-2ad2-49a5-8ece-4b7fd37b4822", 2024-09-14T22:38:33.099 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:38:33.099 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:38:33.099 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:38:33.099 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:38:33.099 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:38:33.099 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:38:33.099 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:38:33.099 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 116, 2024-09-14T22:38:33.099 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:38:33.099 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:38:33.099 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:38:33.099 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:33.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.099 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:33.100 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6800", 2024-09-14T22:38:33.100 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:38:33.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.100 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:33.100 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6801", 2024-09-14T22:38:33.100 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:38:33.100 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.100 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.100 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:38:33.100 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:33.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.101 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:33.101 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6802", 2024-09-14T22:38:33.101 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:38:33.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.101 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:33.101 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6803", 2024-09-14T22:38:33.101 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:38:33.101 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.101 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.101 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:38:33.101 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:33.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.102 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:33.102 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6807", 2024-09-14T22:38:33.102 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:38:33.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.102 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:33.102 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6809", 2024-09-14T22:38:33.102 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:38:33.102 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.102 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.102 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:38:33.103 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:33.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.103 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:33.103 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6804", 2024-09-14T22:38:33.103 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:38:33.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.103 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:33.103 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6806", 2024-09-14T22:38:33.103 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:38:33.103 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.103 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.103 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.176:6801/4116085586", 2024-09-14T22:38:33.104 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.176:6803/4116085586", 2024-09-14T22:38:33.104 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.176:6809/4116085586", 2024-09-14T22:38:33.104 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.176:6806/4116085586", 2024-09-14T22:38:33.104 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:38:33.104 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:38:33.104 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:38:33.104 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.104 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 7, 2024-09-14T22:38:33.104 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "a012c819-8d49-4310-a840-c2752d9314ab", 2024-09-14T22:38:33.104 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:38:33.104 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:38:33.104 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:38:33.104 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:38:33.105 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:38:33.105 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:38:33.105 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:38:33.105 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 117, 2024-09-14T22:38:33.105 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:38:33.105 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:38:33.105 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:38:33.105 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:33.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.105 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:33.105 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6824", 2024-09-14T22:38:33.105 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:38:33.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.106 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:33.106 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6825", 2024-09-14T22:38:33.106 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:38:33.106 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.106 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.106 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:38:33.106 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:33.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.106 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:33.106 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6826", 2024-09-14T22:38:33.106 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:38:33.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.107 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:33.107 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6827", 2024-09-14T22:38:33.107 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:38:33.107 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.107 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.107 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:38:33.107 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:33.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.107 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:33.107 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6830", 2024-09-14T22:38:33.107 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:38:33.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.108 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:33.108 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6831", 2024-09-14T22:38:33.108 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:38:33.108 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.108 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.108 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:38:33.108 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:33.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.108 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:33.108 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6828", 2024-09-14T22:38:33.108 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:38:33.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.109 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:33.109 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6829", 2024-09-14T22:38:33.109 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:38:33.109 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.109 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.109 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.176:6825/877760244", 2024-09-14T22:38:33.109 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.176:6827/877760244", 2024-09-14T22:38:33.109 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.176:6831/877760244", 2024-09-14T22:38:33.109 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.176:6829/877760244", 2024-09-14T22:38:33.109 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:38:33.109 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:38:33.109 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:38:33.110 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.110 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.110 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:38:33.110 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_xinfo": [ 2024-09-14T22:38:33.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.110 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 0, 2024-09-14T22:38:33.110 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:38:33.110 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:38:33.110 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:38:33.110 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:38:33.110 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:38:33.110 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:55.511688+0000", 2024-09-14T22:38:33.110 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:38:33.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.111 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 1, 2024-09-14T22:38:33.111 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:38:33.111 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:38:33.111 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:38:33.111 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:38:33.111 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:38:33.111 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.742414+0000", 2024-09-14T22:38:33.111 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:38:33.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.111 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 2, 2024-09-14T22:38:33.111 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:38:33.111 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:38:33.112 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:38:33.112 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:38:33.112 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:38:33.112 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.871355+0000", 2024-09-14T22:38:33.112 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 31 2024-09-14T22:38:33.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.112 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 3, 2024-09-14T22:38:33.112 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:38:33.112 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:38:33.112 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:38:33.112 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:38:33.113 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:38:33.113 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.504553+0000", 2024-09-14T22:38:33.113 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:38:33.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.113 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 4, 2024-09-14T22:38:33.113 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:38:33.113 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:38:33.113 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:38:33.113 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:38:33.113 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:38:33.113 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.589640+0000", 2024-09-14T22:38:33.113 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:38:33.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.114 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 5, 2024-09-14T22:38:33.114 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:38:33.114 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:38:33.114 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:38:33.114 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:38:33.114 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:38:33.114 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.518845+0000", 2024-09-14T22:38:33.114 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:38:33.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.114 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 6, 2024-09-14T22:38:33.115 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:38:33.115 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:38:33.115 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:38:33.115 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:38:33.115 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:38:33.115 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.731632+0000", 2024-09-14T22:38:33.115 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:38:33.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.115 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 7, 2024-09-14T22:38:33.115 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:38:33.115 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:38:33.116 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:38:33.116 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:38:33.116 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:38:33.116 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.952634+0000", 2024-09-14T22:38:33.116 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:38:33.116 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.116 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:38:33.116 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap": [ 2024-09-14T22:38:33.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.116 INFO:teuthology.orchestra.run.smithi001.stdout: "pgid": "2.0", 2024-09-14T22:38:33.116 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2024-09-14T22:38:33.116 INFO:teuthology.orchestra.run.smithi001.stdout: 4, 2024-09-14T22:38:33.116 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-09-14T22:38:33.117 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.117 INFO:teuthology.orchestra.run.smithi001.stdout: "pgid": "3.1d", 2024-09-14T22:38:33.117 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2024-09-14T22:38:33.117 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-09-14T22:38:33.117 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-09-14T22:38:33.117 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.117 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.117 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:38:33.117 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap_items": [], 2024-09-14T22:38:33.117 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap_primaries": [], 2024-09-14T22:38:33.117 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_temp": [], 2024-09-14T22:38:33.117 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_temp": [], 2024-09-14T22:38:33.118 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklist": {}, 2024-09-14T22:38:33.118 INFO:teuthology.orchestra.run.smithi001.stdout: "range_blocklist": {}, 2024-09-14T22:38:33.118 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profiles": { 2024-09-14T22:38:33.118 INFO:teuthology.orchestra.run.smithi001.stdout: "default": { 2024-09-14T22:38:33.118 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-failure-domain": "osd", 2024-09-14T22:38:33.118 INFO:teuthology.orchestra.run.smithi001.stdout: "k": "2", 2024-09-14T22:38:33.118 INFO:teuthology.orchestra.run.smithi001.stdout: "m": "1", 2024-09-14T22:38:33.118 INFO:teuthology.orchestra.run.smithi001.stdout: "plugin": "jerasure", 2024-09-14T22:38:33.118 INFO:teuthology.orchestra.run.smithi001.stdout: "technique": "reed_sol_van" 2024-09-14T22:38:33.118 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.118 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps_queue": [ 2024-09-14T22:38:33.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.118 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 3, 2024-09-14T22:38:33.119 INFO:teuthology.orchestra.run.smithi001.stdout: "snaps": [ 2024-09-14T22:38:33.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.119 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 31, 2024-09-14T22:38:33.119 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 7 2024-09-14T22:38:33.119 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.119 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.119 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.119 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:38:33.119 INFO:teuthology.orchestra.run.smithi001.stdout: "new_removed_snaps": [ 2024-09-14T22:38:33.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.119 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 3, 2024-09-14T22:38:33.119 INFO:teuthology.orchestra.run.smithi001.stdout: "snaps": [ 2024-09-14T22:38:33.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.120 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 31, 2024-09-14T22:38:33.120 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2024-09-14T22:38:33.120 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.120 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.120 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.120 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:38:33.120 INFO:teuthology.orchestra.run.smithi001.stdout: "new_purged_snaps": [], 2024-09-14T22:38:33.120 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_node_flags": {}, 2024-09-14T22:38:33.120 INFO:teuthology.orchestra.run.smithi001.stdout: "device_class_flags": {}, 2024-09-14T22:38:33.120 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": { 2024-09-14T22:38:33.120 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode_enabled": false, 2024-09-14T22:38:33.120 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_bucket_count": 0, 2024-09-14T22:38:33.120 INFO:teuthology.orchestra.run.smithi001.stdout: "degraded_stretch_mode": 0, 2024-09-14T22:38:33.121 INFO:teuthology.orchestra.run.smithi001.stdout: "recovering_stretch_mode": 0, 2024-09-14T22:38:33.121 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode_bucket": 0 2024-09-14T22:38:33.121 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.121 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:33.123 INFO:tasks.thrashosds.thrasher:j is {'epoch': 140, 'fsid': 'f07f407a-b17d-42c4-bf73-92dd80114a4f', 'created': '2024-09-14T22:35:45.983055+0000', 'modified': '2024-09-14T22:38:32.759271+0000', 'last_up_change': '2024-09-14T22:37:12.616042+0000', 'last_in_change': '2024-09-14T22:35:48.283160+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T22:35:58.938784+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7888126373291016, 'average_primary_affinity': 0.7900000214576721, 'average_primary_affinity_weighted': 0.7900000214576721}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T22:36:04.304242+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7888126373291016, 'average_primary_affinity': 0.7900000214576721, 'average_primary_affinity_weighted': 0.7900000214576721}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T22:36:14.374249+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '140', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 44, 'snap_epoch': 140, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.7888126373291016, 'average_primary_affinity': 0.7900000214576721, 'average_primary_affinity_weighted': 0.7900000214576721}}], 'osds': [{'osd': 0, 'uuid': '9831fcc5-9040-4a80-99d4-838225e65139', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 115, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6824', 'nonce': 1702929600}, {'type': 'v1', 'addr': '172.21.15.1:6825', 'nonce': 1702929600}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6826', 'nonce': 1702929600}, {'type': 'v1', 'addr': '172.21.15.1:6827', 'nonce': 1702929600}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6830', 'nonce': 1702929600}, {'type': 'v1', 'addr': '172.21.15.1:6831', 'nonce': 1702929600}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6828', 'nonce': 1702929600}, {'type': 'v1', 'addr': '172.21.15.1:6829', 'nonce': 1702929600}]}, 'public_addr': '172.21.15.1:6825/1702929600', 'cluster_addr': '172.21.15.1:6827/1702929600', 'heartbeat_back_addr': '172.21.15.1:6831/1702929600', 'heartbeat_front_addr': '172.21.15.1:6829/1702929600', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '7f79abe8-115d-46b2-8677-72fcb0c8ab7d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.1606292724609375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 128, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6808', 'nonce': 719990653}, {'type': 'v1', 'addr': '172.21.15.1:6809', 'nonce': 719990653}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6810', 'nonce': 719990653}, {'type': 'v1', 'addr': '172.21.15.1:6811', 'nonce': 719990653}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6814', 'nonce': 719990653}, {'type': 'v1', 'addr': '172.21.15.1:6815', 'nonce': 719990653}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6812', 'nonce': 719990653}, {'type': 'v1', 'addr': '172.21.15.1:6813', 'nonce': 719990653}]}, 'public_addr': '172.21.15.1:6809/719990653', 'cluster_addr': '172.21.15.1:6811/719990653', 'heartbeat_back_addr': '172.21.15.1:6815/719990653', 'heartbeat_front_addr': '172.21.15.1:6813/719990653', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '1f5768b2-4db4-4b2e-8d5e-5206f8bde28d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 14, 'last_clean_end': 30, 'up_from': 66, 'up_thru': 128, 'down_at': 31, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6816', 'nonce': 1006730601}, {'type': 'v1', 'addr': '172.21.15.1:6817', 'nonce': 1006730601}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6818', 'nonce': 1006730601}, {'type': 'v1', 'addr': '172.21.15.1:6819', 'nonce': 1006730601}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6822', 'nonce': 1006730601}, {'type': 'v1', 'addr': '172.21.15.1:6823', 'nonce': 1006730601}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6820', 'nonce': 1006730601}, {'type': 'v1', 'addr': '172.21.15.1:6821', 'nonce': 1006730601}]}, 'public_addr': '172.21.15.1:6817/1006730601', 'cluster_addr': '172.21.15.1:6819/1006730601', 'heartbeat_back_addr': '172.21.15.1:6823/1006730601', 'heartbeat_front_addr': '172.21.15.1:6821/1006730601', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '8103f3c7-7899-4fc3-bb02-dd2273120138', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.852752685546875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 128, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6800', 'nonce': 750403671}, {'type': 'v1', 'addr': '172.21.15.1:6801', 'nonce': 750403671}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6802', 'nonce': 750403671}, {'type': 'v1', 'addr': '172.21.15.1:6803', 'nonce': 750403671}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6806', 'nonce': 750403671}, {'type': 'v1', 'addr': '172.21.15.1:6807', 'nonce': 750403671}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6804', 'nonce': 750403671}, {'type': 'v1', 'addr': '172.21.15.1:6805', 'nonce': 750403671}]}, 'public_addr': '172.21.15.1:6801/750403671', 'cluster_addr': '172.21.15.1:6803/750403671', 'heartbeat_back_addr': '172.21.15.1:6807/750403671', 'heartbeat_front_addr': '172.21.15.1:6805/750403671', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '23cce5cb-726e-44c2-8edb-c2567a89dfbc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 128, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6816', 'nonce': 1408471829}, {'type': 'v1', 'addr': '172.21.15.176:6817', 'nonce': 1408471829}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6818', 'nonce': 1408471829}, {'type': 'v1', 'addr': '172.21.15.176:6819', 'nonce': 1408471829}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6822', 'nonce': 1408471829}, {'type': 'v1', 'addr': '172.21.15.176:6823', 'nonce': 1408471829}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6820', 'nonce': 1408471829}, {'type': 'v1', 'addr': '172.21.15.176:6821', 'nonce': 1408471829}]}, 'public_addr': '172.21.15.176:6817/1408471829', 'cluster_addr': '172.21.15.176:6819/1408471829', 'heartbeat_back_addr': '172.21.15.176:6823/1408471829', 'heartbeat_front_addr': '172.21.15.176:6821/1408471829', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'eb2f187b-d2f1-49ab-ad5e-ea6dc441a28e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.297119140625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 116, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6805', 'nonce': 1945902326}, {'type': 'v1', 'addr': '172.21.15.176:6808', 'nonce': 1945902326}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6810', 'nonce': 1945902326}, {'type': 'v1', 'addr': '172.21.15.176:6811', 'nonce': 1945902326}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6814', 'nonce': 1945902326}, {'type': 'v1', 'addr': '172.21.15.176:6815', 'nonce': 1945902326}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6812', 'nonce': 1945902326}, {'type': 'v1', 'addr': '172.21.15.176:6813', 'nonce': 1945902326}]}, 'public_addr': '172.21.15.176:6808/1945902326', 'cluster_addr': '172.21.15.176:6811/1945902326', 'heartbeat_back_addr': '172.21.15.176:6815/1945902326', 'heartbeat_front_addr': '172.21.15.176:6813/1945902326', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'd371aed8-2ad2-49a5-8ece-4b7fd37b4822', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 116, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6800', 'nonce': 4116085586}, {'type': 'v1', 'addr': '172.21.15.176:6801', 'nonce': 4116085586}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6802', 'nonce': 4116085586}, {'type': 'v1', 'addr': '172.21.15.176:6803', 'nonce': 4116085586}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6807', 'nonce': 4116085586}, {'type': 'v1', 'addr': '172.21.15.176:6809', 'nonce': 4116085586}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6804', 'nonce': 4116085586}, {'type': 'v1', 'addr': '172.21.15.176:6806', 'nonce': 4116085586}]}, 'public_addr': '172.21.15.176:6801/4116085586', 'cluster_addr': '172.21.15.176:6803/4116085586', 'heartbeat_back_addr': '172.21.15.176:6809/4116085586', 'heartbeat_front_addr': '172.21.15.176:6806/4116085586', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'a012c819-8d49-4310-a840-c2752d9314ab', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 117, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6824', 'nonce': 877760244}, {'type': 'v1', 'addr': '172.21.15.176:6825', 'nonce': 877760244}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6826', 'nonce': 877760244}, {'type': 'v1', 'addr': '172.21.15.176:6827', 'nonce': 877760244}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6830', 'nonce': 877760244}, {'type': 'v1', 'addr': '172.21.15.176:6831', 'nonce': 877760244}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6828', 'nonce': 877760244}, {'type': 'v1', 'addr': '172.21.15.176:6829', 'nonce': 877760244}]}, 'public_addr': '172.21.15.176:6825/877760244', 'cluster_addr': '172.21.15.176:6827/877760244', 'heartbeat_back_addr': '172.21.15.176:6831/877760244', 'heartbeat_front_addr': '172.21.15.176:6829/877760244', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:55.511688+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.742414+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.871355+0000', 'dead_epoch': 31}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.504553+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.589640+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.518845+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.731632+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.952634+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.0', 'osds': [4, 2]}, {'pgid': '3.1d', 'osds': [1, 2]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 31, 'length': 7}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 31, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T22:38:33.123 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:38:33.124 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:33.124 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:33.124 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:38:33.128 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:33.122+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:33.229 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:33.225+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:33.293 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:33.293 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:33.309 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:38:33.329 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:33.322+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:33.372 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:38:33.430 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:33.422+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:33.465 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:38:33.465 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-14T22:38:33.468 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:33.468 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:33.485 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:38:33.486 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":211,"stamp":"2024-09-14T22:38:32.900609+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1288488327,"num_objects":2291,"num_object_clones":441,"num_object_copies":4582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2291,"num_whiteouts":1267,"num_read":198822,"num_read_kb":822251,"num_write":23677,"num_write_kb":383835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3299,"num_bytes_recovered":173602381,"num_keys_recovered":342,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18986,"ondisk_log_size":18986,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":416156,"kb_used_data":58176,"kb_used_omap":17,"kb_used_meta":357678,"kb_avail":754558564,"statfs":{"total":773094113280,"available":772667969536,"internally_reserved":0,"allocated":59572224,"data_stored":442895179,"data_compressed":12672729,"data_compressed_allocated":36921344,"data_compressed_original":430994853,"omap_allocated":18180,"internal_metadata":366262524},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-32882772,"num_objects":-24,"num_object_clones":-8,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":-8,"num_read":267,"num_read_kb":228,"num_write":37,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001546"},"pg_stats":[{"pgid":"3.1f","version":"135'427","reported_seq":4898,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-14T22:38:27.879829+0000","last_change":"2024-09-14T22:38:27.741370+0000","last_active":"2024-09-14T22:38:27.879829+0000","last_peered":"2024-09-14T22:38:27.879829+0000","last_clean":"2024-09-14T22:38:27.879829+0000","last_became_active":"2024-09-14T22:37:25.718906+0000","last_became_peered":"2024-09-14T22:37:25.718906+0000","last_unstale":"2024-09-14T22:38:27.879829+0000","last_undegraded":"2024-09-14T22:38:27.879829+0000","last_fullsized":"2024-09-14T22:38:27.879829+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:04:17.784320+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.006298789,"stat_sum":{"num_bytes":53705826,"num_objects":72,"num_object_clones":13,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":42,"num_read":4801,"num_read_kb":16330,"num_write":406,"num_write_kb":7115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":3277632,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"24","length":"6"}]},{"pgid":"3.1e","version":"134'1093","reported_seq":6611,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-14T22:38:26.741583+0000","last_change":"2024-09-14T22:38:26.741871+0000","last_active":"2024-09-14T22:38:26.741583+0000","last_peered":"2024-09-14T22:38:26.741583+0000","last_clean":"2024-09-14T22:38:26.741583+0000","last_became_active":"2024-09-14T22:38:08.741149+0000","last_became_peered":"2024-09-14T22:38:08.741149+0000","last_unstale":"2024-09-14T22:38:26.741583+0000","last_undegraded":"2024-09-14T22:38:26.741583+0000","last_fullsized":"2024-09-14T22:38:26.741583+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1093,"log_dups_size":0,"ondisk_log_size":1093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:43.649915+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0079064370000000005,"stat_sum":{"num_bytes":73544589,"num_objects":63,"num_object_clones":14,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":32,"num_read":10982,"num_read_kb":33928,"num_write":1633,"num_write_kb":10739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":7412288,"num_keys_recovered":90,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"25","length":"5"}]},{"pgid":"3.1d","version":"135'637","reported_seq":10179,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-14T22:38:30.751381+0000","last_change":"2024-09-14T22:38:30.758570+0000","last_active":"2024-09-14T22:38:30.751381+0000","last_peered":"2024-09-14T22:38:30.751381+0000","last_clean":"2024-09-14T22:38:30.751381+0000","last_became_active":"2024-09-14T22:37:51.730971+0000","last_became_peered":"2024-09-14T22:37:51.730971+0000","last_unstale":"2024-09-14T22:38:30.751381+0000","last_undegraded":"2024-09-14T22:38:30.751381+0000","last_fullsized":"2024-09-14T22:38:30.751381+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:24:10.271087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0087517770000000005,"stat_sum":{"num_bytes":64691866,"num_objects":82,"num_object_clones":20,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":47,"num_read":10733,"num_read_kb":63055,"num_write":825,"num_write_kb":13693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":8030815,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"21","length":"5"}]},{"pgid":"3.1c","version":"135'628","reported_seq":6038,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-14T22:38:27.788251+0000","last_change":"2024-09-14T22:38:27.728546+0000","last_active":"2024-09-14T22:38:27.788251+0000","last_peered":"2024-09-14T22:38:27.788251+0000","last_clean":"2024-09-14T22:38:27.788251+0000","last_became_active":"2024-09-14T22:38:22.078879+0000","last_became_peered":"2024-09-14T22:38:22.078879+0000","last_unstale":"2024-09-14T22:38:27.788251+0000","last_undegraded":"2024-09-14T22:38:27.788251+0000","last_fullsized":"2024-09-14T22:38:27.788251+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:02:54.264093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058421099999999995,"stat_sum":{"num_bytes":34533408,"num_objects":81,"num_object_clones":15,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":49,"num_read":6217,"num_read_kb":26747,"num_write":789,"num_write_kb":8977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"24","length":"6"}]},{"pgid":"3.1b","version":"135'551","reported_seq":5236,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-14T22:38:27.736041+0000","last_change":"2024-09-14T22:38:27.736041+0000","last_active":"2024-09-14T22:38:27.736041+0000","last_peered":"2024-09-14T22:38:27.736041+0000","last_clean":"2024-09-14T22:38:27.736041+0000","last_became_active":"2024-09-14T22:37:25.718519+0000","last_became_peered":"2024-09-14T22:37:25.718519+0000","last_unstale":"2024-09-14T22:38:27.736041+0000","last_undegraded":"2024-09-14T22:38:27.736041+0000","last_fullsized":"2024-09-14T22:38:27.736041+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:49:17.645708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010612600000000001,"stat_sum":{"num_bytes":30938973,"num_objects":65,"num_object_clones":14,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":37,"num_read":5636,"num_read_kb":32416,"num_write":626,"num_write_kb":13330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":7858104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"24","length":"6"}]},{"pgid":"3.1a","version":"134'445","reported_seq":3651,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-14T22:38:27.796735+0000","last_change":"2024-09-14T22:38:27.727976+0000","last_active":"2024-09-14T22:38:27.796735+0000","last_peered":"2024-09-14T22:38:27.796735+0000","last_clean":"2024-09-14T22:38:27.796735+0000","last_became_active":"2024-09-14T22:38:10.742092+0000","last_became_peered":"2024-09-14T22:38:10.742092+0000","last_unstale":"2024-09-14T22:38:27.796735+0000","last_undegraded":"2024-09-14T22:38:27.796735+0000","last_fullsized":"2024-09-14T22:38:27.796735+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:17:44.104556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000436873,"stat_sum":{"num_bytes":10878020,"num_objects":82,"num_object_clones":18,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":49,"num_read":4065,"num_read_kb":12728,"num_write":432,"num_write_kb":7025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":10036538,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"24","length":"6"}]},{"pgid":"3.19","version":"139'412","reported_seq":5282,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T22:38:31.776291+0000","last_change":"2024-09-14T22:38:31.776980+0000","last_active":"2024-09-14T22:38:31.776291+0000","last_peered":"2024-09-14T22:38:31.776291+0000","last_clean":"2024-09-14T22:38:31.776291+0000","last_became_active":"2024-09-14T22:36:15.700990+0000","last_became_peered":"2024-09-14T22:36:15.700990+0000","last_unstale":"2024-09-14T22:38:31.776291+0000","last_undegraded":"2024-09-14T22:38:31.776291+0000","last_fullsized":"2024-09-14T22:38:31.776291+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:53:43.175793+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0087881450000000007,"stat_sum":{"num_bytes":20479200,"num_objects":63,"num_object_clones":10,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":41,"num_read":5094,"num_read_kb":15230,"num_write":397,"num_write_kb":5480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"20","length":"6"}]},{"pgid":"3.18","version":"134'465","reported_seq":4663,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-14T22:38:26.745086+0000","last_change":"2024-09-14T22:38:26.745293+0000","last_active":"2024-09-14T22:38:26.745086+0000","last_peered":"2024-09-14T22:38:26.745086+0000","last_clean":"2024-09-14T22:38:26.745086+0000","last_became_active":"2024-09-14T22:38:07.734308+0000","last_became_peered":"2024-09-14T22:38:07.734308+0000","last_unstale":"2024-09-14T22:38:26.745086+0000","last_undegraded":"2024-09-14T22:38:26.745086+0000","last_fullsized":"2024-09-14T22:38:26.745086+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:48:54.433841+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.011030602,"stat_sum":{"num_bytes":35850302,"num_objects":52,"num_object_clones":9,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":26,"num_read":4907,"num_read_kb":35898,"num_write":606,"num_write_kb":27731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":20067293,"num_keys_recovered":21,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"25","length":"5"}]},{"pgid":"3.17","version":"133'411","reported_seq":4459,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-14T22:38:27.736659+0000","last_change":"2024-09-14T22:38:27.737317+0000","last_active":"2024-09-14T22:38:27.736659+0000","last_peered":"2024-09-14T22:38:27.736659+0000","last_clean":"2024-09-14T22:38:27.736659+0000","last_became_active":"2024-09-14T22:36:15.715869+0000","last_became_peered":"2024-09-14T22:36:15.715869+0000","last_unstale":"2024-09-14T22:38:27.736659+0000","last_undegraded":"2024-09-14T22:38:27.736659+0000","last_fullsized":"2024-09-14T22:38:27.736659+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:57:34.736987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018870790000000001,"stat_sum":{"num_bytes":16937992,"num_objects":64,"num_object_clones":10,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":39,"num_read":4417,"num_read_kb":24992,"num_write":485,"num_write_kb":21997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"24","length":"6"}]},{"pgid":"3.16","version":"135'528","reported_seq":6312,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-14T22:38:27.742356+0000","last_change":"2024-09-14T22:38:27.742618+0000","last_active":"2024-09-14T22:38:27.742356+0000","last_peered":"2024-09-14T22:38:27.742356+0000","last_clean":"2024-09-14T22:38:27.742356+0000","last_became_active":"2024-09-14T22:36:15.725078+0000","last_became_peered":"2024-09-14T22:36:15.725078+0000","last_unstale":"2024-09-14T22:38:27.742356+0000","last_undegraded":"2024-09-14T22:38:27.742356+0000","last_fullsized":"2024-09-14T22:38:27.742356+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:12:43.611680+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0041551289999999996,"stat_sum":{"num_bytes":60847960,"num_objects":84,"num_object_clones":16,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":45,"num_read":6613,"num_read_kb":51176,"num_write":681,"num_write_kb":18943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"24","length":"6"}]},{"pgid":"3.15","version":"134'425","reported_seq":4482,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-14T22:38:27.737116+0000","last_change":"2024-09-14T22:38:27.737621+0000","last_active":"2024-09-14T22:38:27.737116+0000","last_peered":"2024-09-14T22:38:27.737116+0000","last_clean":"2024-09-14T22:38:27.737116+0000","last_became_active":"2024-09-14T22:36:15.717341+0000","last_became_peered":"2024-09-14T22:36:15.717341+0000","last_unstale":"2024-09-14T22:38:27.737116+0000","last_undegraded":"2024-09-14T22:38:27.737116+0000","last_fullsized":"2024-09-14T22:38:27.737116+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:00:30.402326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011966679999999999,"stat_sum":{"num_bytes":12058738,"num_objects":59,"num_object_clones":10,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":38,"num_read":5055,"num_read_kb":12763,"num_write":395,"num_write_kb":7370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"24","length":"6"}]},{"pgid":"3.14","version":"132'1096","reported_seq":8760,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-14T22:38:27.728023+0000","last_change":"2024-09-14T22:38:27.728339+0000","last_active":"2024-09-14T22:38:27.728023+0000","last_peered":"2024-09-14T22:38:27.728023+0000","last_clean":"2024-09-14T22:38:27.728023+0000","last_became_active":"2024-09-14T22:38:10.741866+0000","last_became_peered":"2024-09-14T22:38:10.741866+0000","last_unstale":"2024-09-14T22:38:27.728023+0000","last_undegraded":"2024-09-14T22:38:27.728023+0000","last_fullsized":"2024-09-14T22:38:27.728023+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:35:37.212243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058014700000000004,"stat_sum":{"num_bytes":43414556,"num_objects":89,"num_object_clones":19,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":46,"num_read":11120,"num_read_kb":29521,"num_write":1550,"num_write_kb":11933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":17498216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"24","length":"6"}]},{"pgid":"2.7","version":"0'0","reported_seq":246,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-14T22:38:27.870549+0000","last_change":"2024-09-14T22:36:08.042860+0000","last_active":"2024-09-14T22:38:27.870549+0000","last_peered":"2024-09-14T22:38:27.870549+0000","last_clean":"2024-09-14T22:38:27.870549+0000","last_became_active":"2024-09-14T22:36:06.040961+0000","last_became_peered":"2024-09-14T22:36:06.040961+0000","last_unstale":"2024-09-14T22:38:27.870549+0000","last_undegraded":"2024-09-14T22:38:27.870549+0000","last_fullsized":"2024-09-14T22:38:27.870549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:03:48.547834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010780950000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"135'586","reported_seq":3205,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-14T22:38:27.909278+0000","last_change":"2024-09-14T22:38:27.736597+0000","last_active":"2024-09-14T22:38:27.909278+0000","last_peered":"2024-09-14T22:38:27.909278+0000","last_clean":"2024-09-14T22:38:27.909278+0000","last_became_active":"2024-09-14T22:36:15.700982+0000","last_became_peered":"2024-09-14T22:36:15.700982+0000","last_unstale":"2024-09-14T22:38:27.909278+0000","last_undegraded":"2024-09-14T22:38:27.909278+0000","last_fullsized":"2024-09-14T22:38:27.909278+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:18:47.077108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001631457,"stat_sum":{"num_bytes":27041772,"num_objects":82,"num_object_clones":17,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":43,"num_read":4264,"num_read_kb":22334,"num_write":616,"num_write_kb":12157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"24","length":"6"}]},{"pgid":"2.6","version":"0'0","reported_seq":205,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-14T22:38:27.870586+0000","last_change":"2024-09-14T22:38:03.977889+0000","last_active":"2024-09-14T22:38:27.870586+0000","last_peered":"2024-09-14T22:38:27.870586+0000","last_clean":"2024-09-14T22:38:27.870586+0000","last_became_active":"2024-09-14T22:38:03.976899+0000","last_became_peered":"2024-09-14T22:38:03.976899+0000","last_unstale":"2024-09-14T22:38:27.870586+0000","last_undegraded":"2024-09-14T22:38:27.870586+0000","last_fullsized":"2024-09-14T22:38:27.870586+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:00:41.381700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"132'445","reported_seq":5309,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-14T22:38:27.736521+0000","last_change":"2024-09-14T22:38:27.736772+0000","last_active":"2024-09-14T22:38:27.736521+0000","last_peered":"2024-09-14T22:38:27.736521+0000","last_clean":"2024-09-14T22:38:27.736521+0000","last_became_active":"2024-09-14T22:36:21.722085+0000","last_became_peered":"2024-09-14T22:36:21.722085+0000","last_unstale":"2024-09-14T22:38:27.736521+0000","last_undegraded":"2024-09-14T22:38:27.736521+0000","last_fullsized":"2024-09-14T22:38:27.736521+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:28:48.125407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050504200000000003,"stat_sum":{"num_bytes":51578270,"num_objects":66,"num_object_clones":11,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":36,"num_read":5712,"num_read_kb":28638,"num_write":497,"num_write_kb":8483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"24","length":"6"}]},{"pgid":"2.5","version":"0'0","reported_seq":246,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-14T22:38:27.736646+0000","last_change":"2024-09-14T22:36:08.054606+0000","last_active":"2024-09-14T22:38:27.736646+0000","last_peered":"2024-09-14T22:38:27.736646+0000","last_clean":"2024-09-14T22:38:27.736646+0000","last_became_active":"2024-09-14T22:36:06.266908+0000","last_became_peered":"2024-09-14T22:36:06.266908+0000","last_unstale":"2024-09-14T22:38:27.736646+0000","last_undegraded":"2024-09-14T22:38:27.736646+0000","last_fullsized":"2024-09-14T22:38:27.736646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:36.738643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098249300000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"138'1396","reported_seq":5182,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-14T22:38:31.003719+0000","last_change":"2024-09-14T22:38:30.770364+0000","last_active":"2024-09-14T22:38:31.003719+0000","last_peered":"2024-09-14T22:38:31.003719+0000","last_clean":"2024-09-14T22:38:31.003719+0000","last_became_active":"2024-09-14T22:37:17.674322+0000","last_became_peered":"2024-09-14T22:37:17.674322+0000","last_unstale":"2024-09-14T22:38:31.003719+0000","last_undegraded":"2024-09-14T22:38:31.003719+0000","last_fullsized":"2024-09-14T22:38:31.003719+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1396,"log_dups_size":0,"ondisk_log_size":1396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:30:21.927059+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.020677187999999999,"stat_sum":{"num_bytes":16528446,"num_objects":62,"num_object_clones":11,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":36,"num_read":9792,"num_read_kb":23788,"num_write":2405,"num_write_kb":17934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":4264139,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"21","length":"5"}]},{"pgid":"2.4","version":"0'0","reported_seq":195,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-14T22:38:27.734744+0000","last_change":"2024-09-14T22:36:41.930961+0000","last_active":"2024-09-14T22:38:27.734744+0000","last_peered":"2024-09-14T22:38:27.734744+0000","last_clean":"2024-09-14T22:38:27.734744+0000","last_became_active":"2024-09-14T22:36:41.930619+0000","last_became_peered":"2024-09-14T22:36:41.930619+0000","last_unstale":"2024-09-14T22:38:27.734744+0000","last_undegraded":"2024-09-14T22:38:27.734744+0000","last_fullsized":"2024-09-14T22:38:27.734744+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:04:40.582999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"132'519","reported_seq":4280,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-14T22:38:26.741947+0000","last_change":"2024-09-14T22:38:26.742329+0000","last_active":"2024-09-14T22:38:26.741947+0000","last_peered":"2024-09-14T22:38:26.741947+0000","last_clean":"2024-09-14T22:38:26.741947+0000","last_became_active":"2024-09-14T22:38:22.080243+0000","last_became_peered":"2024-09-14T22:38:22.080243+0000","last_unstale":"2024-09-14T22:38:26.741947+0000","last_undegraded":"2024-09-14T22:38:26.741947+0000","last_fullsized":"2024-09-14T22:38:26.741947+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:08:31.846710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082140779999999997,"stat_sum":{"num_bytes":25799104,"num_objects":65,"num_object_clones":14,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":36,"num_read":4940,"num_read_kb":13834,"num_write":536,"num_write_kb":5794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":5797306,"num_keys_recovered":72,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"25","length":"5"}]},{"pgid":"3.3","version":"135'419","reported_seq":5001,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-14T22:38:27.899560+0000","last_change":"2024-09-14T22:38:27.729185+0000","last_active":"2024-09-14T22:38:27.899560+0000","last_peered":"2024-09-14T22:38:27.899560+0000","last_clean":"2024-09-14T22:38:27.899560+0000","last_became_active":"2024-09-14T22:36:15.706147+0000","last_became_peered":"2024-09-14T22:36:15.706147+0000","last_unstale":"2024-09-14T22:38:27.899560+0000","last_undegraded":"2024-09-14T22:38:27.899560+0000","last_fullsized":"2024-09-14T22:38:27.899560+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:58:09.448499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030558400000000002,"stat_sum":{"num_bytes":52284550,"num_objects":74,"num_object_clones":16,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":44,"num_read":4425,"num_read_kb":25591,"num_write":463,"num_write_kb":14016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"24","length":"6"}]},{"pgid":"2.2","version":"22'2","reported_seq":49,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T22:38:31.768630+0000","last_change":"2024-09-14T22:38:22.084850+0000","last_active":"2024-09-14T22:38:31.768630+0000","last_peered":"2024-09-14T22:38:31.768630+0000","last_clean":"2024-09-14T22:38:31.768630+0000","last_became_active":"2024-09-14T22:38:22.083760+0000","last_became_peered":"2024-09-14T22:38:22.083760+0000","last_unstale":"2024-09-14T22:38:31.768630+0000","last_undegraded":"2024-09-14T22:38:31.768630+0000","last_fullsized":"2024-09-14T22:38:31.768630+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:05:19.354830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"138'889","reported_seq":8851,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-14T22:38:30.762420+0000","last_change":"2024-09-14T22:38:30.762420+0000","last_active":"2024-09-14T22:38:30.762420+0000","last_peered":"2024-09-14T22:38:30.762420+0000","last_clean":"2024-09-14T22:38:30.762420+0000","last_became_active":"2024-09-14T22:37:16.662230+0000","last_became_peered":"2024-09-14T22:37:16.662230+0000","last_unstale":"2024-09-14T22:38:30.762420+0000","last_undegraded":"2024-09-14T22:38:30.762420+0000","last_fullsized":"2024-09-14T22:38:30.762420+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":889,"log_dups_size":0,"ondisk_log_size":889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:56:26.349334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.012027166000000001,"stat_sum":{"num_bytes":65779683,"num_objects":70,"num_object_clones":12,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":34,"num_read":10040,"num_read_kb":29619,"num_write":1204,"num_write_kb":14996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":4851034,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"21","length":"5"}]},{"pgid":"2.1","version":"0'0","reported_seq":187,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-14T22:38:30.750933+0000","last_change":"2024-09-14T22:38:05.724182+0000","last_active":"2024-09-14T22:38:30.750933+0000","last_peered":"2024-09-14T22:38:30.750933+0000","last_clean":"2024-09-14T22:38:30.750933+0000","last_became_active":"2024-09-14T22:38:05.723636+0000","last_became_peered":"2024-09-14T22:38:05.723636+0000","last_unstale":"2024-09-14T22:38:30.750933+0000","last_undegraded":"2024-09-14T22:38:30.750933+0000","last_fullsized":"2024-09-14T22:38:30.750933+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:14:09.687576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"134'399","reported_seq":5455,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-14T22:38:27.735116+0000","last_change":"2024-09-14T22:38:27.735487+0000","last_active":"2024-09-14T22:38:27.735116+0000","last_peered":"2024-09-14T22:38:27.735116+0000","last_clean":"2024-09-14T22:38:27.735116+0000","last_became_active":"2024-09-14T22:38:08.743679+0000","last_became_peered":"2024-09-14T22:38:08.743679+0000","last_unstale":"2024-09-14T22:38:27.735116+0000","last_undegraded":"2024-09-14T22:38:27.735116+0000","last_fullsized":"2024-09-14T22:38:27.735116+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:22:34.785824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064930800000000002,"stat_sum":{"num_bytes":57986116,"num_objects":61,"num_object_clones":12,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":34,"num_read":4558,"num_read_kb":23172,"num_write":392,"num_write_kb":7226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":3679238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"24","length":"6"}]},{"pgid":"2.0","version":"0'0","reported_seq":34,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-14T22:38:27.727345+0000","last_change":"2024-09-14T22:38:16.113520+0000","last_active":"2024-09-14T22:38:27.727345+0000","last_peered":"2024-09-14T22:38:27.727345+0000","last_clean":"2024-09-14T22:38:27.727345+0000","last_became_active":"2024-09-14T22:38:16.112705+0000","last_became_peered":"2024-09-14T22:38:16.112705+0000","last_unstale":"2024-09-14T22:38:27.727345+0000","last_undegraded":"2024-09-14T22:38:27.727345+0000","last_fullsized":"2024-09-14T22:38:27.727345+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:52:31.713617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010817520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"134'1030","reported_seq":4341,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-14T22:38:27.739829+0000","last_change":"2024-09-14T22:38:27.739829+0000","last_active":"2024-09-14T22:38:27.739829+0000","last_peered":"2024-09-14T22:38:27.739829+0000","last_clean":"2024-09-14T22:38:27.739829+0000","last_became_active":"2024-09-14T22:38:09.741015+0000","last_became_peered":"2024-09-14T22:38:09.741015+0000","last_unstale":"2024-09-14T22:38:27.739829+0000","last_undegraded":"2024-09-14T22:38:27.739829+0000","last_fullsized":"2024-09-14T22:38:27.739829+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1030,"log_dups_size":0,"ondisk_log_size":1030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:19:56.836912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010544510000000001,"stat_sum":{"num_bytes":56086942,"num_objects":75,"num_object_clones":13,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":34,"num_read":7260,"num_read_kb":15923,"num_write":1520,"num_write_kb":11844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":2936757,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"24","length":"6"}]},{"pgid":"2.3","version":"20'1","reported_seq":33,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-14T22:38:30.749327+0000","last_change":"2024-09-14T22:38:22.082065+0000","last_active":"2024-09-14T22:38:30.749327+0000","last_peered":"2024-09-14T22:38:30.749327+0000","last_clean":"2024-09-14T22:38:30.749327+0000","last_became_active":"2024-09-14T22:38:22.081064+0000","last_became_peered":"2024-09-14T22:38:22.081064+0000","last_unstale":"2024-09-14T22:38:30.749327+0000","last_undegraded":"2024-09-14T22:38:30.749327+0000","last_fullsized":"2024-09-14T22:38:30.749327+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:54:57.636584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":397,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-14T22:38:27.736013+0000","last_change":"2024-09-14T22:36:00.402862+0000","last_active":"2024-09-14T22:38:27.736013+0000","last_peered":"2024-09-14T22:38:27.736013+0000","last_clean":"2024-09-14T22:38:27.736013+0000","last_became_active":"2024-09-14T22:36:00.400529+0000","last_became_peered":"2024-09-14T22:36:00.400529+0000","last_unstale":"2024-09-14T22:38:27.736013+0000","last_undegraded":"2024-09-14T22:38:27.736013+0000","last_fullsized":"2024-09-14T22:38:27.736013+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_clean_scrub_stamp":"2024-09-14T22:35:58.989025+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:07:12.951728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"131'604","reported_seq":7463,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-14T22:38:26.734110+0000","last_change":"2024-09-14T22:38:26.734600+0000","last_active":"2024-09-14T22:38:26.734110+0000","last_peered":"2024-09-14T22:38:26.734110+0000","last_clean":"2024-09-14T22:38:26.734110+0000","last_became_active":"2024-09-14T22:38:08.739090+0000","last_became_peered":"2024-09-14T22:38:08.739090+0000","last_unstale":"2024-09-14T22:38:26.734110+0000","last_undegraded":"2024-09-14T22:38:26.734110+0000","last_fullsized":"2024-09-14T22:38:26.734110+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:56:19.653164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070837199999999997,"stat_sum":{"num_bytes":39365030,"num_objects":67,"num_object_clones":15,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":30,"num_read":8756,"num_read_kb":22057,"num_write":908,"num_write_kb":14465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":18690892,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"25","length":"5"}]},{"pgid":"3.9","version":"132'373","reported_seq":7080,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-14T22:38:27.727702+0000","last_change":"2024-09-14T22:38:27.727940+0000","last_active":"2024-09-14T22:38:27.727702+0000","last_peered":"2024-09-14T22:38:27.727702+0000","last_clean":"2024-09-14T22:38:27.727702+0000","last_became_active":"2024-09-14T22:38:09.736445+0000","last_became_peered":"2024-09-14T22:38:09.736445+0000","last_unstale":"2024-09-14T22:38:27.727702+0000","last_undegraded":"2024-09-14T22:38:27.727702+0000","last_fullsized":"2024-09-14T22:38:27.727702+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:27:12.256325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045812000000000001,"stat_sum":{"num_bytes":62873372,"num_objects":96,"num_object_clones":20,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":49,"num_read":6803,"num_read_kb":18462,"num_write":378,"num_write_kb":7002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":317,"num_bytes_recovered":16527933,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"24","length":"6"}]},{"pgid":"3.a","version":"135'678","reported_seq":3852,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-14T22:38:27.875187+0000","last_change":"2024-09-14T22:38:27.872129+0000","last_active":"2024-09-14T22:38:27.875187+0000","last_peered":"2024-09-14T22:38:27.875187+0000","last_clean":"2024-09-14T22:38:27.875187+0000","last_became_active":"2024-09-14T22:36:15.718794+0000","last_became_peered":"2024-09-14T22:36:15.718794+0000","last_unstale":"2024-09-14T22:38:27.875187+0000","last_undegraded":"2024-09-14T22:38:27.875187+0000","last_fullsized":"2024-09-14T22:38:27.875187+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:16:36.549986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001490077,"stat_sum":{"num_bytes":29108595,"num_objects":67,"num_object_clones":12,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":35,"num_read":5810,"num_read_kb":25498,"num_write":670,"num_write_kb":14791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"24","length":"6"}]},{"pgid":"3.b","version":"134'493","reported_seq":2423,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-14T22:38:26.736592+0000","last_change":"2024-09-14T22:38:26.736968+0000","last_active":"2024-09-14T22:38:26.736592+0000","last_peered":"2024-09-14T22:38:26.736592+0000","last_clean":"2024-09-14T22:38:26.736592+0000","last_became_active":"2024-09-14T22:36:15.702885+0000","last_became_peered":"2024-09-14T22:36:15.702885+0000","last_unstale":"2024-09-14T22:38:26.736592+0000","last_undegraded":"2024-09-14T22:38:26.736592+0000","last_fullsized":"2024-09-14T22:38:26.736592+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:16:57.355083+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0030001839999999999,"stat_sum":{"num_bytes":36831357,"num_objects":72,"num_object_clones":16,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":41,"num_read":3140,"num_read_kb":11302,"num_write":489,"num_write_kb":6865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"25","length":"5"}]},{"pgid":"3.c","version":"134'393","reported_seq":6607,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-14T22:38:26.748127+0000","last_change":"2024-09-14T22:38:26.748127+0000","last_active":"2024-09-14T22:38:26.748127+0000","last_peered":"2024-09-14T22:38:26.748127+0000","last_clean":"2024-09-14T22:38:26.748127+0000","last_became_active":"2024-09-14T22:38:22.076411+0000","last_became_peered":"2024-09-14T22:38:22.076411+0000","last_unstale":"2024-09-14T22:38:26.748127+0000","last_undegraded":"2024-09-14T22:38:26.748127+0000","last_fullsized":"2024-09-14T22:38:26.748127+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:17:57.859119+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.014129345,"stat_sum":{"num_bytes":43749615,"num_objects":65,"num_object_clones":16,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":38,"num_read":5777,"num_read_kb":33485,"num_write":462,"num_write_kb":17848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":23218688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"25","length":"5"}]},{"pgid":"3.d","version":"133'596","reported_seq":3904,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-14T22:38:27.737720+0000","last_change":"2024-09-14T22:38:27.737928+0000","last_active":"2024-09-14T22:38:27.737720+0000","last_peered":"2024-09-14T22:38:27.737720+0000","last_clean":"2024-09-14T22:38:27.737720+0000","last_became_active":"2024-09-14T22:36:15.723205+0000","last_became_peered":"2024-09-14T22:36:15.723205+0000","last_unstale":"2024-09-14T22:38:27.737720+0000","last_undegraded":"2024-09-14T22:38:27.737720+0000","last_fullsized":"2024-09-14T22:38:27.737720+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:49:45.674526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013810649999999999,"stat_sum":{"num_bytes":26214845,"num_objects":70,"num_object_clones":15,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":34,"num_read":4336,"num_read_kb":16206,"num_write":754,"num_write_kb":5956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"24","length":"6"}]},{"pgid":"3.e","version":"134'470","reported_seq":2468,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-14T22:38:27.736831+0000","last_change":"2024-09-14T22:38:27.737086+0000","last_active":"2024-09-14T22:38:27.736831+0000","last_peered":"2024-09-14T22:38:27.736831+0000","last_clean":"2024-09-14T22:38:27.736831+0000","last_became_active":"2024-09-14T22:38:09.738915+0000","last_became_peered":"2024-09-14T22:38:09.738915+0000","last_unstale":"2024-09-14T22:38:27.736831+0000","last_undegraded":"2024-09-14T22:38:27.736831+0000","last_fullsized":"2024-09-14T22:38:27.736831+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:50:09.401902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075126900000000005,"stat_sum":{"num_bytes":28271070,"num_objects":66,"num_object_clones":10,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":40,"num_read":2431,"num_read_kb":8294,"num_write":536,"num_write_kb":3814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":4944418,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"24","length":"6"}]},{"pgid":"3.f","version":"134'397","reported_seq":8004,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-14T22:38:27.737177+0000","last_change":"2024-09-14T22:38:27.737428+0000","last_active":"2024-09-14T22:38:27.737177+0000","last_peered":"2024-09-14T22:38:27.737177+0000","last_clean":"2024-09-14T22:38:27.737177+0000","last_became_active":"2024-09-14T22:36:15.719693+0000","last_became_peered":"2024-09-14T22:36:15.719693+0000","last_unstale":"2024-09-14T22:38:27.737177+0000","last_undegraded":"2024-09-14T22:38:27.737177+0000","last_fullsized":"2024-09-14T22:38:27.737177+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:24:26.906058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090649000000000003,"stat_sum":{"num_bytes":24577780,"num_objects":74,"num_object_clones":12,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":38,"num_read":8172,"num_read_kb":23242,"num_write":458,"num_write_kb":9514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"24","length":"6"}]},{"pgid":"3.10","version":"134'431","reported_seq":3768,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-14T22:38:27.870992+0000","last_change":"2024-09-14T22:38:27.871483+0000","last_active":"2024-09-14T22:38:27.870992+0000","last_peered":"2024-09-14T22:38:27.870992+0000","last_clean":"2024-09-14T22:38:27.870992+0000","last_became_active":"2024-09-14T22:37:26.717088+0000","last_became_peered":"2024-09-14T22:37:26.717088+0000","last_unstale":"2024-09-14T22:38:27.870992+0000","last_undegraded":"2024-09-14T22:38:27.870992+0000","last_fullsized":"2024-09-14T22:38:27.870992+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:27:54.606955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072148799999999995,"stat_sum":{"num_bytes":40194809,"num_objects":78,"num_object_clones":17,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":44,"num_read":3369,"num_read_kb":29055,"num_write":505,"num_write_kb":14812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"24","length":"6"}]},{"pgid":"3.11","version":"134'635","reported_seq":2319,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-14T22:38:27.736425+0000","last_change":"2024-09-14T22:38:27.737015+0000","last_active":"2024-09-14T22:38:27.736425+0000","last_peered":"2024-09-14T22:38:27.736425+0000","last_clean":"2024-09-14T22:38:27.736425+0000","last_became_active":"2024-09-14T22:36:15.715569+0000","last_became_peered":"2024-09-14T22:36:15.715569+0000","last_unstale":"2024-09-14T22:38:27.736425+0000","last_undegraded":"2024-09-14T22:38:27.736425+0000","last_fullsized":"2024-09-14T22:38:27.736425+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":635,"log_dups_size":0,"ondisk_log_size":635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:33:34.969285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080338700000000002,"stat_sum":{"num_bytes":30135901,"num_objects":63,"num_object_clones":8,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":42,"num_read":3345,"num_read_kb":11582,"num_write":841,"num_write_kb":10931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"24","length":"6"}]},{"pgid":"3.12","version":"135'503","reported_seq":8222,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-14T22:38:27.887967+0000","last_change":"2024-09-14T22:38:27.736383+0000","last_active":"2024-09-14T22:38:27.887967+0000","last_peered":"2024-09-14T22:38:27.887967+0000","last_clean":"2024-09-14T22:38:27.887967+0000","last_became_active":"2024-09-14T22:36:15.716696+0000","last_became_peered":"2024-09-14T22:36:15.716696+0000","last_unstale":"2024-09-14T22:38:27.887967+0000","last_undegraded":"2024-09-14T22:38:27.887967+0000","last_fullsized":"2024-09-14T22:38:27.887967+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:22:37.556373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001549747,"stat_sum":{"num_bytes":61489703,"num_objects":73,"num_object_clones":11,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":41,"num_read":9403,"num_read_kb":50315,"num_write":515,"num_write_kb":13159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"24","length":"6"}]},{"pgid":"3.13","version":"134'492","reported_seq":7124,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-14T22:38:27.737826+0000","last_change":"2024-09-14T22:38:27.738407+0000","last_active":"2024-09-14T22:38:27.737826+0000","last_peered":"2024-09-14T22:38:27.737826+0000","last_clean":"2024-09-14T22:38:27.737826+0000","last_became_active":"2024-09-14T22:38:10.744013+0000","last_became_peered":"2024-09-14T22:38:10.744013+0000","last_unstale":"2024-09-14T22:38:27.737826+0000","last_undegraded":"2024-09-14T22:38:27.737826+0000","last_fullsized":"2024-09-14T22:38:27.737826+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:56:25.480516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012045809999999999,"stat_sum":{"num_bytes":54119550,"num_objects":85,"num_object_clones":15,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":47,"num_read":6733,"num_read_kb":34970,"num_write":542,"num_write_kb":15477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":14511090,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"24","length":"6"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1287897940,"num_objects":2287,"num_object_clones":441,"num_object_copies":4574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2287,"num_whiteouts":1267,"num_read":198706,"num_read_kb":822151,"num_write":23516,"num_write_kb":381417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3299,"num_bytes_recovered":173602381,"num_keys_recovered":342,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45563904,"data_stored":430791961,"data_compressed":12550072,"data_compressed_allocated":35983360,"data_compressed_original":425096613,"omap_allocated":4238,"internal_metadata":0},"log_size":18866,"ondisk_log_size":18866,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542177,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62012,"kb_used_data":8584,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94309828,"statfs":{"total":96636764160,"available":96573263872,"internally_reserved":0,"allocated":8790016,"data_stored":64144190,"data_compressed":2044951,"data_compressed_allocated":5586944,"data_compressed_original":62430413,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542178,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43588,"kb_used_data":4068,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328252,"statfs":{"total":96636764160,"available":96592130048,"internally_reserved":0,"allocated":4165632,"data_stored":32703726,"data_compressed":793833,"data_compressed_allocated":2236416,"data_compressed_original":31765571,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542178,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56288,"kb_used_data":7032,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94315552,"statfs":{"total":96636764160,"available":96579125248,"internally_reserved":0,"allocated":7200768,"data_stored":47472759,"data_compressed":1481189,"data_compressed_allocated":4349952,"data_compressed_original":45967502,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542177,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55340,"kb_used_data":10224,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94316500,"statfs":{"total":96636764160,"available":96580096000,"internally_reserved":0,"allocated":10469376,"data_stored":85746564,"data_compressed":2482968,"data_compressed_allocated":7049216,"data_compressed_original":83912882,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542178,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52808,"kb_used_data":7724,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94319032,"statfs":{"total":96636764160,"available":96582688768,"internally_reserved":0,"allocated":7909376,"data_stored":55206820,"data_compressed":1822035,"data_compressed_allocated":4870144,"data_compressed_original":53597941,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":66,"seq":283467841553,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42068,"kb_used_data":5244,"kb_used_omap":6,"kb_used_meta":36793,"kb_avail":94329772,"statfs":{"total":96636764160,"available":96593686528,"internally_reserved":0,"allocated":5369856,"data_stored":35668867,"data_compressed":750289,"data_compressed_allocated":3002368,"data_compressed_original":34429098,"omap_allocated":7025,"internal_metadata":37676175},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59436,"kb_used_data":6040,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94312404,"statfs":{"total":96636764160,"available":96575901696,"internally_reserved":0,"allocated":6184960,"data_stored":43955850,"data_compressed":1388018,"data_compressed_allocated":3477504,"data_compressed_original":42547034,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542177,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44616,"kb_used_data":9260,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94327224,"statfs":{"total":96636764160,"available":96591077376,"internally_reserved":0,"allocated":9482240,"data_stored":77996403,"data_compressed":1909446,"data_compressed_allocated":6348800,"data_compressed_original":76344412,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7778304,"data_stored":76602545,"data_compressed":1878088,"data_compressed_allocated":6283264,"data_compressed_original":75730012,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4464640,"data_stored":43056606,"data_compressed":1366094,"data_compressed_allocated":3440640,"data_compressed_original":42452826,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3256320,"data_stored":33983917,"data_compressed":726694,"data_compressed_allocated":2568192,"data_compressed_original":33544362,"omap_allocated":4238,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6307840,"data_stored":54449482,"data_compressed":1822035,"data_compressed_allocated":4870144,"data_compressed_original":53597941,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8855552,"data_stored":84982941,"data_compressed":2482968,"data_compressed_allocated":7049216,"data_compressed_original":83912882,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5578752,"data_stored":46709087,"data_compressed":1481189,"data_compressed_allocated":4349952,"data_compressed_original":45967502,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2564096,"data_stored":31946388,"data_compressed":793833,"data_compressed_allocated":2236416,"data_compressed_original":31765571,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":6758400,"data_stored":59060995,"data_compressed":1999171,"data_compressed_allocated":5185536,"data_compressed_original":58125517,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:38:33.488 INFO:tasks.thrashosds.thrasher:Setting 3.9 to [2, 4, 0, 7] 2024-09-14T22:38:33.488 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.9', '2', '4', '0', '7'] 2024-09-14T22:38:33.488 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.9 2 4 0 7 2024-09-14T22:38:33.525 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:38:33.525 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:38:33.543 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:38:33.543 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:38:33.543 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:33.541+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:33.559 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:38:33.560 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:38:33.575 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:38:33.576 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:38:33.598 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:38:33.599 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:38:33.620 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:38:33.620 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:38:33.632 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:33.629+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:33.637 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:33.637 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:38:33.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:38:33.637 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:38:33.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.637 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:122960 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 2347008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:33.637 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:41.998325+0000", 2024-09-14T22:38:33.637 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.637158145000001, 2024-09-14T22:38:33.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056570400999999999, 2024-09-14T22:38:33.637 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:33.637 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:33.638 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:33.638 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:33.638 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:33.638 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:33.638 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 122960 2024-09-14T22:38:33.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.638 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:33.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:33.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998325+0000", 2024-09-14T22:38:33.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:33.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998325+0000", 2024-09-14T22:38:33.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:33.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998326+0000", 2024-09-14T22:38:33.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9299999999999998e-07 2024-09-14T22:38:33.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:33.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998329+0000", 2024-09-14T22:38:33.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1370000000000002e-06 2024-09-14T22:38:33.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:33.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998340+0000", 2024-09-14T22:38:33.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1039999999999999e-05 2024-09-14T22:38:33.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:33.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998348+0000", 2024-09-14T22:38:33.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1920000000000005e-06 2024-09-14T22:38:33.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:33.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054700+0000", 2024-09-14T22:38:33.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056351714999999997 2024-09-14T22:38:33.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:33.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054771+0000", 2024-09-14T22:38:33.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1009999999999999e-05 2024-09-14T22:38:33.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:33.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054896+0000", 2024-09-14T22:38:33.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000124714 2024-09-14T22:38:33.641 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.641 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.641 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.642 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:122961 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 2351104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:33.642 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:41.998357+0000", 2024-09-14T22:38:33.642 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.637126190000004, 2024-09-14T22:38:33.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056974912000000003, 2024-09-14T22:38:33.642 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:33.642 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:33.642 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:33.642 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:33.642 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:33.642 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:33.642 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 122961 2024-09-14T22:38:33.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.642 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:33.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:33.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998357+0000", 2024-09-14T22:38:33.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:33.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998357+0000", 2024-09-14T22:38:33.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:33.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998358+0000", 2024-09-14T22:38:33.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9699999999999996e-07 2024-09-14T22:38:33.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:33.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998361+0000", 2024-09-14T22:38:33.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1370000000000002e-06 2024-09-14T22:38:33.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.644 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:33.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998371+0000", 2024-09-14T22:38:33.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5789999999999993e-06 2024-09-14T22:38:33.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.644 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:33.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998379+0000", 2024-09-14T22:38:33.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0129999999999993e-06 2024-09-14T22:38:33.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.644 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:33.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054998+0000", 2024-09-14T22:38:33.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056618961000000002 2024-09-14T22:38:33.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:33.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.055126+0000", 2024-09-14T22:38:33.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012795100000000001 2024-09-14T22:38:33.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:33.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.055332+0000", 2024-09-14T22:38:33.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020667400000000001 2024-09-14T22:38:33.645 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.645 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.645 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.645 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123259 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 741376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:33.646 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:38:33.646 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.623400672999999, 2024-09-14T22:38:33.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056672581999999999, 2024-09-14T22:38:33.646 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:33.646 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:33.646 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:33.646 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:33.646 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:33.646 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:33.646 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123259 2024-09-14T22:38:33.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.646 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:33.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:33.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:38:33.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:33.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:38:33.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:33.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:38:33.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1600000000000001e-07 2024-09-14T22:38:33.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:33.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012086+0000", 2024-09-14T22:38:33.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8499999999999998e-06 2024-09-14T22:38:33.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:33.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012097+0000", 2024-09-14T22:38:33.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0943e-05 2024-09-14T22:38:33.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:33.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012105+0000", 2024-09-14T22:38:33.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0830000000000004e-06 2024-09-14T22:38:33.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:33.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068584+0000", 2024-09-14T22:38:33.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056478387999999997 2024-09-14T22:38:33.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:33.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068644+0000", 2024-09-14T22:38:33.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9984e-05 2024-09-14T22:38:33.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:33.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068755+0000", 2024-09-14T22:38:33.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111718 2024-09-14T22:38:33.650 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.650 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.650 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.650 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123260 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 778240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:33.650 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012118+0000", 2024-09-14T22:38:33.650 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.623365448000001, 2024-09-14T22:38:33.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056825887999999998, 2024-09-14T22:38:33.650 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:33.650 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:33.650 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:33.650 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:33.650 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:33.650 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:33.651 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123260 2024-09-14T22:38:33.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.651 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:33.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:33.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012118+0000", 2024-09-14T22:38:33.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:33.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012118+0000", 2024-09-14T22:38:33.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:33.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012119+0000", 2024-09-14T22:38:33.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.13e-07 2024-09-14T22:38:33.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:33.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012130+0000", 2024-09-14T22:38:33.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1552999999999999e-05 2024-09-14T22:38:33.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:33.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012141+0000", 2024-09-14T22:38:33.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.03e-05 2024-09-14T22:38:33.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:33.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012149+0000", 2024-09-14T22:38:33.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1419999999999993e-06 2024-09-14T22:38:33.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:33.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068800+0000", 2024-09-14T22:38:33.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056651574000000003 2024-09-14T22:38:33.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:33.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068858+0000", 2024-09-14T22:38:33.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8047000000000002e-05 2024-09-14T22:38:33.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:33.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068944+0000", 2024-09-14T22:38:33.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5659000000000001e-05 2024-09-14T22:38:33.654 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.654 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.654 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.654 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123261 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 761856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:33.654 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012161+0000", 2024-09-14T22:38:33.654 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.623322047000002, 2024-09-14T22:38:33.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057113753000000003, 2024-09-14T22:38:33.655 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:33.655 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:33.655 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:33.655 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:33.655 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:33.655 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:33.655 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123261 2024-09-14T22:38:33.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.655 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:33.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:33.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012161+0000", 2024-09-14T22:38:33.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:33.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012161+0000", 2024-09-14T22:38:33.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:33.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012162+0000", 2024-09-14T22:38:33.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4399999999999999e-07 2024-09-14T22:38:33.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:33.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012177+0000", 2024-09-14T22:38:33.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5099000000000001e-05 2024-09-14T22:38:33.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:33.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012191+0000", 2024-09-14T22:38:33.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3631999999999999e-05 2024-09-14T22:38:33.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:33.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012199+0000", 2024-09-14T22:38:33.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5769999999999994e-06 2024-09-14T22:38:33.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:33.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069004+0000", 2024-09-14T22:38:33.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056804504999999998 2024-09-14T22:38:33.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:33.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069137+0000", 2024-09-14T22:38:33.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013302600000000001 2024-09-14T22:38:33.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:33.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069275+0000", 2024-09-14T22:38:33.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013816999999999999 2024-09-14T22:38:33.658 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.658 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.658 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.659 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123262 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 724992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:33.659 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012224+0000", 2024-09-14T22:38:33.659 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.623259599999997, 2024-09-14T22:38:33.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057417959999999997, 2024-09-14T22:38:33.659 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:33.659 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:33.659 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:33.659 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:33.659 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:33.659 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:33.659 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123262 2024-09-14T22:38:33.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.659 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:33.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:33.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012224+0000", 2024-09-14T22:38:33.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:33.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012224+0000", 2024-09-14T22:38:33.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:33.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012225+0000", 2024-09-14T22:38:33.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4000000000000001e-07 2024-09-14T22:38:33.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:33.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012228+0000", 2024-09-14T22:38:33.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5729999999999999e-06 2024-09-14T22:38:33.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:33.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012241+0000", 2024-09-14T22:38:33.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3210000000000001e-05 2024-09-14T22:38:33.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:33.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012255+0000", 2024-09-14T22:38:33.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3852000000000001e-05 2024-09-14T22:38:33.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:33.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069341+0000", 2024-09-14T22:38:33.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057086139000000001 2024-09-14T22:38:33.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:33.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069477+0000", 2024-09-14T22:38:33.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013595 2024-09-14T22:38:33.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:33.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069642+0000", 2024-09-14T22:38:33.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016459600000000001 2024-09-14T22:38:33.662 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.663 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.663 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.663 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123263 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 782336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:33.663 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012265+0000", 2024-09-14T22:38:33.663 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.623218057999999, 2024-09-14T22:38:33.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057665095, 2024-09-14T22:38:33.663 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:33.663 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:33.663 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:33.663 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:33.663 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:33.663 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:33.663 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123263 2024-09-14T22:38:33.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.664 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:33.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:33.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012265+0000", 2024-09-14T22:38:33.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:33.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012265+0000", 2024-09-14T22:38:33.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:33.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012266+0000", 2024-09-14T22:38:33.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8999999999999996e-07 2024-09-14T22:38:33.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:33.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012269+0000", 2024-09-14T22:38:33.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.796e-06 2024-09-14T22:38:33.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:33.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012279+0000", 2024-09-14T22:38:33.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7389999999999994e-06 2024-09-14T22:38:33.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:33.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012286+0000", 2024-09-14T22:38:33.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8199999999999997e-06 2024-09-14T22:38:33.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.666 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:33.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069705+0000", 2024-09-14T22:38:33.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057418590999999998 2024-09-14T22:38:33.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.666 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:33.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069799+0000", 2024-09-14T22:38:33.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4419999999999994e-05 2024-09-14T22:38:33.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.666 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:33.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069931+0000", 2024-09-14T22:38:33.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013113900000000001 2024-09-14T22:38:33.667 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.667 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.667 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.667 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123264 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 745472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:33.667 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012295+0000", 2024-09-14T22:38:33.667 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.623188057, 2024-09-14T22:38:33.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057959498999999998, 2024-09-14T22:38:33.667 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:33.667 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:33.668 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:33.668 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:33.668 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:33.668 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:33.668 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123264 2024-09-14T22:38:33.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.668 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:33.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:33.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012295+0000", 2024-09-14T22:38:33.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:33.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012295+0000", 2024-09-14T22:38:33.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:33.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012296+0000", 2024-09-14T22:38:33.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9699999999999996e-07 2024-09-14T22:38:33.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:33.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012299+0000", 2024-09-14T22:38:33.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.79e-06 2024-09-14T22:38:33.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:33.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012320+0000", 2024-09-14T22:38:33.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1242000000000001e-05 2024-09-14T22:38:33.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:33.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012338+0000", 2024-09-14T22:38:33.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7768999999999999e-05 2024-09-14T22:38:33.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:33.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069995+0000", 2024-09-14T22:38:33.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057657 2024-09-14T22:38:33.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:33.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070132+0000", 2024-09-14T22:38:33.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013727800000000001 2024-09-14T22:38:33.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:33.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070255+0000", 2024-09-14T22:38:33.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012282300000000001 2024-09-14T22:38:33.671 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.671 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.671 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.671 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123265 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1056768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:33.672 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012348+0000", 2024-09-14T22:38:33.672 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.623135101999999, 2024-09-14T22:38:33.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058145087999999998, 2024-09-14T22:38:33.672 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:33.672 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:33.672 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:33.672 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:33.672 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:33.672 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:33.672 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123265 2024-09-14T22:38:33.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.672 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:33.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:33.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012348+0000", 2024-09-14T22:38:33.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:33.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012348+0000", 2024-09-14T22:38:33.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:33.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012349+0000", 2024-09-14T22:38:33.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6599999999999996e-07 2024-09-14T22:38:33.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:33.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012352+0000", 2024-09-14T22:38:33.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3139999999999999e-06 2024-09-14T22:38:33.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:33.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012365+0000", 2024-09-14T22:38:33.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2625e-05 2024-09-14T22:38:33.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:33.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012373+0000", 2024-09-14T22:38:33.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2470000000000008e-06 2024-09-14T22:38:33.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:33.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070286+0000", 2024-09-14T22:38:33.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057912547000000002 2024-09-14T22:38:33.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:33.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070366+0000", 2024-09-14T22:38:33.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0468000000000003e-05 2024-09-14T22:38:33.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:33.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070493+0000", 2024-09-14T22:38:33.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012722100000000001 2024-09-14T22:38:33.676 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.676 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.676 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.676 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123266 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 765952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:33.676 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:38:33.676 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.623100854, 2024-09-14T22:38:33.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058386949, 2024-09-14T22:38:33.676 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:33.676 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:33.676 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:33.677 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:33.677 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:33.677 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:33.677 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123266 2024-09-14T22:38:33.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.677 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:33.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:33.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:38:33.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:33.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:38:33.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:33.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:38:33.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9299999999999998e-07 2024-09-14T22:38:33.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:33.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012386+0000", 2024-09-14T22:38:33.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8729999999999999e-06 2024-09-14T22:38:33.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:33.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012406+0000", 2024-09-14T22:38:33.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0103e-05 2024-09-14T22:38:33.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:33.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012419+0000", 2024-09-14T22:38:33.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.311e-05 2024-09-14T22:38:33.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:33.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070550+0000", 2024-09-14T22:38:33.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058130902999999998 2024-09-14T22:38:33.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:33.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070650+0000", 2024-09-14T22:38:33.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010005000000000001 2024-09-14T22:38:33.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:33.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070770+0000", 2024-09-14T22:38:33.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119317 2024-09-14T22:38:33.680 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.680 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.680 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.681 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123267 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1077248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:33.681 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012429+0000", 2024-09-14T22:38:33.681 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.623054940000003, 2024-09-14T22:38:33.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058620343999999998, 2024-09-14T22:38:33.681 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:33.681 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:33.681 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:33.681 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:33.681 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:33.681 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:33.681 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123267 2024-09-14T22:38:33.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.681 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:33.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:33.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012429+0000", 2024-09-14T22:38:33.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:33.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012429+0000", 2024-09-14T22:38:33.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:33.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013488+0000", 2024-09-14T22:38:33.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001059061 2024-09-14T22:38:33.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:33.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013499+0000", 2024-09-14T22:38:33.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1348999999999999e-05 2024-09-14T22:38:33.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:33.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013534+0000", 2024-09-14T22:38:33.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5206000000000001e-05 2024-09-14T22:38:33.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:33.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013558+0000", 2024-09-14T22:38:33.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4324999999999998e-05 2024-09-14T22:38:33.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:33.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070834+0000", 2024-09-14T22:38:33.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057275110999999997 2024-09-14T22:38:33.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:33.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070937+0000", 2024-09-14T22:38:33.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010306099999999999 2024-09-14T22:38:33.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:33.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071049+0000", 2024-09-14T22:38:33.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112231 2024-09-14T22:38:33.685 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.685 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.685 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.685 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123268 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1097728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:33.685 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013593+0000", 2024-09-14T22:38:33.685 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.621890217000001, 2024-09-14T22:38:33.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057751793000000003, 2024-09-14T22:38:33.685 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:33.685 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:33.685 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:33.686 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:33.686 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:33.686 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:33.686 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123268 2024-09-14T22:38:33.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.686 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:33.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:33.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013593+0000", 2024-09-14T22:38:33.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:33.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013593+0000", 2024-09-14T22:38:33.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:33.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013595+0000", 2024-09-14T22:38:33.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.59e-06 2024-09-14T22:38:33.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:33.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013602+0000", 2024-09-14T22:38:33.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3560000000000004e-06 2024-09-14T22:38:33.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:33.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013625+0000", 2024-09-14T22:38:33.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2419e-05 2024-09-14T22:38:33.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.688 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:33.688 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013650+0000", 2024-09-14T22:38:33.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5151999999999999e-05 2024-09-14T22:38:33.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.688 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:33.688 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071096+0000", 2024-09-14T22:38:33.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057445877999999999 2024-09-14T22:38:33.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.688 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:33.688 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071212+0000", 2024-09-14T22:38:33.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116215 2024-09-14T22:38:33.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:33.689 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071345+0000", 2024-09-14T22:38:33.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013318299999999999 2024-09-14T22:38:33.689 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.689 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.689 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.689 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123269 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1060864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:33.689 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013676+0000", 2024-09-14T22:38:33.689 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.621807457999999, 2024-09-14T22:38:33.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058027982999999998, 2024-09-14T22:38:33.689 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:33.689 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:33.690 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:33.690 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:33.690 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:33.690 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:33.690 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123269 2024-09-14T22:38:33.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.690 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:33.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:33.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013676+0000", 2024-09-14T22:38:33.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:33.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013676+0000", 2024-09-14T22:38:33.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:33.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013677+0000", 2024-09-14T22:38:33.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.387e-06 2024-09-14T22:38:33.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:33.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013684+0000", 2024-09-14T22:38:33.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0530000000000001e-06 2024-09-14T22:38:33.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:33.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013706+0000", 2024-09-14T22:38:33.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1121999999999999e-05 2024-09-14T22:38:33.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.692 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:33.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013722+0000", 2024-09-14T22:38:33.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6249999999999999e-05 2024-09-14T22:38:33.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:33.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071402+0000", 2024-09-14T22:38:33.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057679945000000003 2024-09-14T22:38:33.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:33.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071547+0000", 2024-09-14T22:38:33.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000145632 2024-09-14T22:38:33.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:33.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071704+0000", 2024-09-14T22:38:33.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000156594 2024-09-14T22:38:33.693 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.693 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.694 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.694 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123270 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1064960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:33.694 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013744+0000", 2024-09-14T22:38:33.694 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.621739507000001, 2024-09-14T22:38:33.694 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058355872000000003, 2024-09-14T22:38:33.694 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:33.694 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:33.694 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:33.694 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:33.694 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:33.694 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:33.695 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123270 2024-09-14T22:38:33.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.695 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:33.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:33.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013744+0000", 2024-09-14T22:38:33.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:33.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013744+0000", 2024-09-14T22:38:33.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:33.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013745+0000", 2024-09-14T22:38:33.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2589999999999999e-06 2024-09-14T22:38:33.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:33.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013751+0000", 2024-09-14T22:38:33.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1929999999999998e-06 2024-09-14T22:38:33.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:33.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013778+0000", 2024-09-14T22:38:33.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6472e-05 2024-09-14T22:38:33.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:33.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013795+0000", 2024-09-14T22:38:33.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6626e-05 2024-09-14T22:38:33.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:33.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071775+0000", 2024-09-14T22:38:33.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057980232 2024-09-14T22:38:33.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:33.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071911+0000", 2024-09-14T22:38:33.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000136129 2024-09-14T22:38:33.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:33.698 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072100+0000", 2024-09-14T22:38:33.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000188961 2024-09-14T22:38:33.698 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.698 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.698 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.698 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123271 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1081344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:33.698 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013818+0000", 2024-09-14T22:38:33.698 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.621665311000001, 2024-09-14T22:38:33.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058549565999999997, 2024-09-14T22:38:33.698 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:33.698 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:33.698 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:33.698 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:33.699 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:33.699 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:33.699 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123271 2024-09-14T22:38:33.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.699 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:33.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:33.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013818+0000", 2024-09-14T22:38:33.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:33.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013818+0000", 2024-09-14T22:38:33.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:33.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013958+0000", 2024-09-14T22:38:33.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013934 2024-09-14T22:38:33.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:33.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013968+0000", 2024-09-14T22:38:33.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0329000000000001e-05 2024-09-14T22:38:33.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:33.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013995+0000", 2024-09-14T22:38:33.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7613000000000001e-05 2024-09-14T22:38:33.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:33.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014014+0000", 2024-09-14T22:38:33.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8338999999999998e-05 2024-09-14T22:38:33.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:33.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072155+0000", 2024-09-14T22:38:33.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058140989999999997 2024-09-14T22:38:33.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:33.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072247+0000", 2024-09-14T22:38:33.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1774999999999995e-05 2024-09-14T22:38:33.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:33.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072368+0000", 2024-09-14T22:38:33.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012118 2024-09-14T22:38:33.702 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.702 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.702 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.702 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123272 3.1 3.4b235a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:33.702 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.014046+0000", 2024-09-14T22:38:33.702 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.621437634999999, 2024-09-14T22:38:33.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058364077, 2024-09-14T22:38:33.702 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:33.702 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:33.702 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-14T22:38:33.702 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:33.702 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:33.703 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:33.703 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123272 2024-09-14T22:38:33.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.703 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:33.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:33.703 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014046+0000", 2024-09-14T22:38:33.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:33.703 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014046+0000", 2024-09-14T22:38:33.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:33.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014047+0000", 2024-09-14T22:38:33.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.533e-06 2024-09-14T22:38:33.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:33.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014085+0000", 2024-09-14T22:38:33.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7734999999999997e-05 2024-09-14T22:38:33.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:33.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014111+0000", 2024-09-14T22:38:33.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6055999999999999e-05 2024-09-14T22:38:33.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:33.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014136+0000", 2024-09-14T22:38:33.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4592000000000001e-05 2024-09-14T22:38:33.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:33.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072402+0000", 2024-09-14T22:38:33.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058266251999999998 2024-09-14T22:38:33.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:33.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072410+0000", 2024-09-14T22:38:33.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.909e-06 2024-09-14T22:38:33.705 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.705 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.705 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.706 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123273 3.1 3.4b235a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:33.706 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.014161+0000", 2024-09-14T22:38:33.706 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.621322607000003, 2024-09-14T22:38:33.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058284415999999999, 2024-09-14T22:38:33.706 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:33.706 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:33.706 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-14T22:38:33.706 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:33.706 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:33.706 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:33.706 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123273 2024-09-14T22:38:33.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.706 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:33.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:33.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014161+0000", 2024-09-14T22:38:33.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:33.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014161+0000", 2024-09-14T22:38:33.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:33.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.017996+0000", 2024-09-14T22:38:33.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0038350670000000002 2024-09-14T22:38:33.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:33.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.018009+0000", 2024-09-14T22:38:33.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3305999999999999e-05 2024-09-14T22:38:33.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:33.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.018044+0000", 2024-09-14T22:38:33.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4792000000000002e-05 2024-09-14T22:38:33.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:33.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.018067+0000", 2024-09-14T22:38:33.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2595e-05 2024-09-14T22:38:33.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:33.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072439+0000", 2024-09-14T22:38:33.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054372192 2024-09-14T22:38:33.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:33.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072445+0000", 2024-09-14T22:38:33.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4640000000000003e-06 2024-09-14T22:38:33.709 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.709 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.709 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.709 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:148469 3.18 3:1f6f7244:::rbd_header.113c827b523f:head [watch reconnect cookie 140250011614800 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-14T22:38:33.709 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:55.746449+0000", 2024-09-14T22:38:33.709 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.889033994999998, 2024-09-14T22:38:33.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.996484173, 2024-09-14T22:38:33.710 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:33.710 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:33.710 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:33.710 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:33.710 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:33.710 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:33.710 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 148469 2024-09-14T22:38:33.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.710 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:33.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:33.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746449+0000", 2024-09-14T22:38:33.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:33.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746449+0000", 2024-09-14T22:38:33.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:33.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746452+0000", 2024-09-14T22:38:33.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0019999999999998e-06 2024-09-14T22:38:33.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:33.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746461+0000", 2024-09-14T22:38:33.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0919999999999998e-06 2024-09-14T22:38:33.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:33.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746475+0000", 2024-09-14T22:38:33.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4853e-05 2024-09-14T22:38:33.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:33.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746484+0000", 2024-09-14T22:38:33.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3780000000000001e-06 2024-09-14T22:38:33.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:33.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746559+0000", 2024-09-14T22:38:33.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5259000000000006e-05 2024-09-14T22:38:33.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:33.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746572+0000", 2024-09-14T22:38:33.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.271e-05 2024-09-14T22:38:33.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:33.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727807+0000", 2024-09-14T22:38:33.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98123555900000003 2024-09-14T22:38:33.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:33.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727826+0000", 2024-09-14T22:38:33.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8828e-05 2024-09-14T22:38:33.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:33.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.739033+0000", 2024-09-14T22:38:33.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.011207062 2024-09-14T22:38:33.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:33.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.739479+0000", 2024-09-14T22:38:33.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044563200000000001 2024-09-14T22:38:33.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:33.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.742934+0000", 2024-09-14T22:38:33.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0034547979999999998 2024-09-14T22:38:33.714 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.714 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.714 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.715 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:148473 3.18 3:1f6f7244:::rbd_header.113c827b523f:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-14T22:38:33.715 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:55.755969+0000", 2024-09-14T22:38:33.715 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.879514229000002, 2024-09-14T22:38:33.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9872532889999999, 2024-09-14T22:38:33.715 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:33.715 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:33.715 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:33.715 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:33.715 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:33.715 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:33.715 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 148473 2024-09-14T22:38:33.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.715 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:33.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:33.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755969+0000", 2024-09-14T22:38:33.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:33.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755969+0000", 2024-09-14T22:38:33.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:33.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755971+0000", 2024-09-14T22:38:33.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.674e-06 2024-09-14T22:38:33.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.717 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:33.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755979+0000", 2024-09-14T22:38:33.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8599999999999993e-06 2024-09-14T22:38:33.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.717 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:33.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755992+0000", 2024-09-14T22:38:33.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.361e-05 2024-09-14T22:38:33.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.717 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:33.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.756001+0000", 2024-09-14T22:38:33.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.422e-06 2024-09-14T22:38:33.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:33.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.756069+0000", 2024-09-14T22:38:33.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8209999999999999e-05 2024-09-14T22:38:33.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:33.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.756077+0000", 2024-09-14T22:38:33.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.994e-06 2024-09-14T22:38:33.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:33.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727872+0000", 2024-09-14T22:38:33.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.97179484999999999 2024-09-14T22:38:33.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:33.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727885+0000", 2024-09-14T22:38:33.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3101e-05 2024-09-14T22:38:33.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:33.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.740559+0000", 2024-09-14T22:38:33.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0126736300000001 2024-09-14T22:38:33.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:33.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.740777+0000", 2024-09-14T22:38:33.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021822000000000001 2024-09-14T22:38:33.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 2", 2024-09-14T22:38:33.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.741114+0000", 2024-09-14T22:38:33.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000336793 2024-09-14T22:38:33.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:33.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.742668+0000", 2024-09-14T22:38:33.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001554061 2024-09-14T22:38:33.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:38:33.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.742947+0000", 2024-09-14T22:38:33.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027937200000000001 2024-09-14T22:38:33.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:38:33.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743030+0000", 2024-09-14T22:38:33.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3385000000000002e-05 2024-09-14T22:38:33.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:33.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743223+0000", 2024-09-14T22:38:33.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00019210700000000001 2024-09-14T22:38:33.721 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.721 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.721 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.722 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:148474 3.18 3:1f6f7244:::rbd_header.113c827b523f:head [watch reconnect cookie 140250011614800 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104)", 2024-09-14T22:38:33.722 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:56.728273+0000", 2024-09-14T22:38:33.722 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 36.907210982000002, 2024-09-14T22:38:33.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0180497770000001, 2024-09-14T22:38:33.722 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:33.722 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:33.722 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:33.722 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:33.722 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:33.722 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:33.722 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 148474 2024-09-14T22:38:33.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.723 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:33.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:33.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728273+0000", 2024-09-14T22:38:33.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:33.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728273+0000", 2024-09-14T22:38:33.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:33.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:33.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728274+0000", 2024-09-14T22:38:33.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7570000000000001e-06 2024-09-14T22:38:33.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:33.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728281+0000", 2024-09-14T22:38:33.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5089999999999999e-06 2024-09-14T22:38:33.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:33.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728298+0000", 2024-09-14T22:38:33.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6727000000000001e-05 2024-09-14T22:38:33.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:33.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728309+0000", 2024-09-14T22:38:33.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1021e-05 2024-09-14T22:38:33.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:33.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.730078+0000", 2024-09-14T22:38:33.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017694010000000001 2024-09-14T22:38:33.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:33.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.730096+0000", 2024-09-14T22:38:33.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8039000000000001e-05 2024-09-14T22:38:33.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:33.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.741672+0000", 2024-09-14T22:38:33.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.011575817 2024-09-14T22:38:33.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2024-09-14T22:38:33.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.741827+0000", 2024-09-14T22:38:33.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000155563 2024-09-14T22:38:33.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:33.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743299+0000", 2024-09-14T22:38:33.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0014712729999999999 2024-09-14T22:38:33.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:33.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743415+0000", 2024-09-14T22:38:33.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011594700000000001 2024-09-14T22:38:33.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:33.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:33.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:33.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.746322+0000", 2024-09-14T22:38:33.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0029077230000000001 2024-09-14T22:38:33.727 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.727 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.727 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.727 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:33.727 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:33.727 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:33.727 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:38:33.728 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:38:33.729 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:38:33.729 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:38:33.730 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:38:33.733 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:33.726+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:33.834 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:33.833+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:33.905 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:33.905 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:33.905 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:38:33.905 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:33.919 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:38:33.936 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:33.930+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:34.036 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:34.033+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:34.076 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:34.076 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:34.076 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:38:34.076 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:38:34.076 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:34.089 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:38:34.138 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:34.130+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:34.239 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:34.234+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:34.251 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:34.251 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:38:34.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:38:34.251 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:38:34.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.251 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:31370 3.0 3:02f48f58:::rbd_data.113cda0183a0.0000000000000000:6 [read 813056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:38:34.251 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:38:34.251 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 120.88686231299999, 2024-09-14T22:38:34.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047519796000000003, 2024-09-14T22:38:34.251 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.252 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.252 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.252 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.252 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.252 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.252 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 31370 2024-09-14T22:38:34.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.252 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.252 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.252 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:38:34.252 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:38:34.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:38:34.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0999999999999999e-07 2024-09-14T22:38:34.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362401+0000", 2024-09-14T22:38:34.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2900000000000001e-06 2024-09-14T22:38:34.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362406+0000", 2024-09-14T22:38:34.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6960000000000002e-06 2024-09-14T22:38:34.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362412+0000", 2024-09-14T22:38:34.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.823e-06 2024-09-14T22:38:34.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.400270+0000", 2024-09-14T22:38:34.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037857737000000002 2024-09-14T22:38:34.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.255 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.400342+0000", 2024-09-14T22:38:34.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1797000000000004e-05 2024-09-14T22:38:34.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.255 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.409918+0000", 2024-09-14T22:38:34.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0095760429999999994 2024-09-14T22:38:34.255 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.255 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.255 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.256 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:113947 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 5603328~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:34.256 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.923453+0000", 2024-09-14T22:38:34.256 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.325807181000002, 2024-09-14T22:38:34.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047310420999999998, 2024-09-14T22:38:34.256 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.256 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.256 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.256 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.256 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.256 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.256 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 113947 2024-09-14T22:38:34.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.256 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.923453+0000", 2024-09-14T22:38:34.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.923453+0000", 2024-09-14T22:38:34.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926004+0000", 2024-09-14T22:38:34.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002551382 2024-09-14T22:38:34.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926009+0000", 2024-09-14T22:38:34.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5639999999999998e-06 2024-09-14T22:38:34.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.259 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.259 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926022+0000", 2024-09-14T22:38:34.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2734999999999999e-05 2024-09-14T22:38:34.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.259 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.259 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926029+0000", 2024-09-14T22:38:34.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2520000000000002e-06 2024-09-14T22:38:34.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.259 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.259 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.970650+0000", 2024-09-14T22:38:34.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044621277000000001 2024-09-14T22:38:34.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.260 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.260 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.970700+0000", 2024-09-14T22:38:34.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9713000000000003e-05 2024-09-14T22:38:34.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.260 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.260 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.970763+0000", 2024-09-14T22:38:34.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3497999999999994e-05 2024-09-14T22:38:34.260 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.260 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.260 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.260 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114136 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3448832~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:34.261 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932179+0000", 2024-09-14T22:38:34.261 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.317080814999997, 2024-09-14T22:38:34.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047236017999999998, 2024-09-14T22:38:34.261 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.261 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.261 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.261 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.261 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.261 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.261 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114136 2024-09-14T22:38:34.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.261 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932179+0000", 2024-09-14T22:38:34.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932179+0000", 2024-09-14T22:38:34.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932324+0000", 2024-09-14T22:38:34.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014451000000000001 2024-09-14T22:38:34.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932328+0000", 2024-09-14T22:38:34.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5889999999999996e-06 2024-09-14T22:38:34.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932338+0000", 2024-09-14T22:38:34.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3000000000000007e-06 2024-09-14T22:38:34.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932344+0000", 2024-09-14T22:38:34.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7150000000000001e-06 2024-09-14T22:38:34.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979233+0000", 2024-09-14T22:38:34.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046888711 2024-09-14T22:38:34.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979313+0000", 2024-09-14T22:38:34.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9487000000000007e-05 2024-09-14T22:38:34.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979415+0000", 2024-09-14T22:38:34.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000102706 2024-09-14T22:38:34.264 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.264 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.264 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.265 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114138 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3497984~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:34.265 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932356+0000", 2024-09-14T22:38:34.265 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.316903929000006, 2024-09-14T22:38:34.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047285958000000003, 2024-09-14T22:38:34.265 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.265 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.265 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.265 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.265 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.265 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.265 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114138 2024-09-14T22:38:34.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.266 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.266 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.266 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932356+0000", 2024-09-14T22:38:34.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.266 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.266 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932356+0000", 2024-09-14T22:38:34.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.266 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.266 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932498+0000", 2024-09-14T22:38:34.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000142182 2024-09-14T22:38:34.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932503+0000", 2024-09-14T22:38:34.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.301e-06 2024-09-14T22:38:34.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932514+0000", 2024-09-14T22:38:34.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1484000000000001e-05 2024-09-14T22:38:34.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932521+0000", 2024-09-14T22:38:34.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0740000000000004e-06 2024-09-14T22:38:34.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979451+0000", 2024-09-14T22:38:34.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046929379 2024-09-14T22:38:34.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979535+0000", 2024-09-14T22:38:34.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4423000000000001e-05 2024-09-14T22:38:34.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979642+0000", 2024-09-14T22:38:34.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010711499999999999 2024-09-14T22:38:34.269 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.269 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.269 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.269 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114139 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3547136~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:34.269 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932554+0000", 2024-09-14T22:38:34.269 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.316706162000003, 2024-09-14T22:38:34.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047292230999999997, 2024-09-14T22:38:34.269 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.269 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.269 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.269 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.269 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.270 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.270 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114139 2024-09-14T22:38:34.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.270 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932554+0000", 2024-09-14T22:38:34.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932554+0000", 2024-09-14T22:38:34.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932819+0000", 2024-09-14T22:38:34.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026538400000000002 2024-09-14T22:38:34.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932825+0000", 2024-09-14T22:38:34.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3469999999999998e-06 2024-09-14T22:38:34.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932836+0000", 2024-09-14T22:38:34.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1e-05 2024-09-14T22:38:34.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932843+0000", 2024-09-14T22:38:34.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1790000000000002e-06 2024-09-14T22:38:34.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979673+0000", 2024-09-14T22:38:34.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046830389 2024-09-14T22:38:34.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979750+0000", 2024-09-14T22:38:34.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6312999999999999e-05 2024-09-14T22:38:34.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.273 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979846+0000", 2024-09-14T22:38:34.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6619000000000004e-05 2024-09-14T22:38:34.273 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.273 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.273 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.273 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114141 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3596288~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:34.273 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932855+0000", 2024-09-14T22:38:34.273 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.316405563999993, 2024-09-14T22:38:34.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047201870999999999, 2024-09-14T22:38:34.273 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.273 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.274 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.274 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.274 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.274 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.274 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114141 2024-09-14T22:38:34.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.274 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932855+0000", 2024-09-14T22:38:34.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932855+0000", 2024-09-14T22:38:34.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934033+0000", 2024-09-14T22:38:34.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001178264 2024-09-14T22:38:34.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934038+0000", 2024-09-14T22:38:34.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4820000000000002e-06 2024-09-14T22:38:34.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934048+0000", 2024-09-14T22:38:34.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9890000000000004e-06 2024-09-14T22:38:34.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934055+0000", 2024-09-14T22:38:34.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8329999999999996e-06 2024-09-14T22:38:34.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979878+0000", 2024-09-14T22:38:34.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045822773999999997 2024-09-14T22:38:34.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979958+0000", 2024-09-14T22:38:34.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0477999999999998e-05 2024-09-14T22:38:34.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.277 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.277 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.980056+0000", 2024-09-14T22:38:34.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8051e-05 2024-09-14T22:38:34.277 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.277 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.277 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.277 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114159 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3940352~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:34.277 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.934179+0000", 2024-09-14T22:38:34.278 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.315081527999993, 2024-09-14T22:38:34.278 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047341439999999999, 2024-09-14T22:38:34.278 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.278 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.278 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.278 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.278 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.278 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.278 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114159 2024-09-14T22:38:34.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.278 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.278 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.278 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934179+0000", 2024-09-14T22:38:34.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.279 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.279 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934179+0000", 2024-09-14T22:38:34.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.279 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.279 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935021+0000", 2024-09-14T22:38:34.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00084276800000000003 2024-09-14T22:38:34.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.279 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935027+0000", 2024-09-14T22:38:34.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2959999999999998e-06 2024-09-14T22:38:34.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935038+0000", 2024-09-14T22:38:34.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1198e-05 2024-09-14T22:38:34.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935054+0000", 2024-09-14T22:38:34.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5764e-05 2024-09-14T22:38:34.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.981341+0000", 2024-09-14T22:38:34.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.04628782 2024-09-14T22:38:34.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.981415+0000", 2024-09-14T22:38:34.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3349999999999994e-05 2024-09-14T22:38:34.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.981520+0000", 2024-09-14T22:38:34.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105244 2024-09-14T22:38:34.282 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.282 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.282 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.282 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114180 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4136960~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:34.282 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:38:34.282 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.314081991999998, 2024-09-14T22:38:34.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047190621000000002, 2024-09-14T22:38:34.282 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.282 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.282 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.282 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.283 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.283 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.283 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114180 2024-09-14T22:38:34.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.283 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.283 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.283 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:38:34.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:38:34.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:38:34.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6800000000000002e-07 2024-09-14T22:38:34.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935180+0000", 2024-09-14T22:38:34.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.42e-06 2024-09-14T22:38:34.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935185+0000", 2024-09-14T22:38:34.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1359999999999996e-06 2024-09-14T22:38:34.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935188+0000", 2024-09-14T22:38:34.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.711e-06 2024-09-14T22:38:34.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982201+0000", 2024-09-14T22:38:34.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047013309000000003 2024-09-14T22:38:34.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982278+0000", 2024-09-14T22:38:34.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6705000000000005e-05 2024-09-14T22:38:34.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982369+0000", 2024-09-14T22:38:34.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1071999999999998e-05 2024-09-14T22:38:34.286 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.286 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.287 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.287 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114182 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4186112~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:34.287 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:38:34.287 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.314053369999996, 2024-09-14T22:38:34.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047368990999999999, 2024-09-14T22:38:34.287 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.287 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.287 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.287 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.287 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.287 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.287 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114182 2024-09-14T22:38:34.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.288 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.288 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.288 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:38:34.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.288 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.288 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:38:34.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.288 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:38:34.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7099999999999998e-07 2024-09-14T22:38:34.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935209+0000", 2024-09-14T22:38:34.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0260000000000002e-06 2024-09-14T22:38:34.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935213+0000", 2024-09-14T22:38:34.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3429999999999998e-06 2024-09-14T22:38:34.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935217+0000", 2024-09-14T22:38:34.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2480000000000001e-06 2024-09-14T22:38:34.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982406+0000", 2024-09-14T22:38:34.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047188920000000002 2024-09-14T22:38:34.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982483+0000", 2024-09-14T22:38:34.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7008999999999994e-05 2024-09-14T22:38:34.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982576+0000", 2024-09-14T22:38:34.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3173999999999999e-05 2024-09-14T22:38:34.291 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.291 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.291 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.291 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114185 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4202496~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:34.291 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.935221+0000", 2024-09-14T22:38:34.291 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.314039205, 2024-09-14T22:38:34.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047601128999999999, 2024-09-14T22:38:34.291 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.291 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.291 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.291 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.291 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.291 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.292 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114185 2024-09-14T22:38:34.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.292 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.292 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.292 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935221+0000", 2024-09-14T22:38:34.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.292 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.292 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935221+0000", 2024-09-14T22:38:34.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.293 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.293 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935970+0000", 2024-09-14T22:38:34.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074930899999999998 2024-09-14T22:38:34.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.293 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.293 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.293 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935976+0000", 2024-09-14T22:38:34.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5439999999999998e-06 2024-09-14T22:38:34.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.293 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.293 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.293 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935988+0000", 2024-09-14T22:38:34.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1779e-05 2024-09-14T22:38:34.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935996+0000", 2024-09-14T22:38:34.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.106e-06 2024-09-14T22:38:34.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982607+0000", 2024-09-14T22:38:34.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046611504999999998 2024-09-14T22:38:34.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.295 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982686+0000", 2024-09-14T22:38:34.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8805000000000002e-05 2024-09-14T22:38:34.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.295 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982822+0000", 2024-09-14T22:38:34.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000136081 2024-09-14T22:38:34.295 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.295 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.295 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.295 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114189 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4300800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:34.295 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.936048+0000", 2024-09-14T22:38:34.295 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.313212073000003, 2024-09-14T22:38:34.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047231539000000003, 2024-09-14T22:38:34.296 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.296 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.296 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.296 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.296 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.296 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.296 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114189 2024-09-14T22:38:34.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.296 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.296 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936048+0000", 2024-09-14T22:38:34.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.297 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936048+0000", 2024-09-14T22:38:34.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.297 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936049+0000", 2024-09-14T22:38:34.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4099999999999999e-07 2024-09-14T22:38:34.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.297 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936052+0000", 2024-09-14T22:38:34.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0029999999999999e-06 2024-09-14T22:38:34.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936058+0000", 2024-09-14T22:38:34.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7880000000000001e-06 2024-09-14T22:38:34.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936064+0000", 2024-09-14T22:38:34.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2279999999999998e-06 2024-09-14T22:38:34.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983120+0000", 2024-09-14T22:38:34.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047056384999999999 2024-09-14T22:38:34.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983194+0000", 2024-09-14T22:38:34.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3811000000000002e-05 2024-09-14T22:38:34.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983280+0000", 2024-09-14T22:38:34.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5883000000000004e-05 2024-09-14T22:38:34.299 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.299 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.299 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.300 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114191 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4349952~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:34.300 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:38:34.300 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.313188332999999, 2024-09-14T22:38:34.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047365365, 2024-09-14T22:38:34.300 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.300 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.300 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.300 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.300 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.300 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.300 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114191 2024-09-14T22:38:34.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.300 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:38:34.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:38:34.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:38:34.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0499999999999999e-07 2024-09-14T22:38:34.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936075+0000", 2024-09-14T22:38:34.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.734e-06 2024-09-14T22:38:34.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936082+0000", 2024-09-14T22:38:34.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8360000000000003e-06 2024-09-14T22:38:34.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936087+0000", 2024-09-14T22:38:34.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3129999999999998e-06 2024-09-14T22:38:34.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983319+0000", 2024-09-14T22:38:34.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047232140999999998 2024-09-14T22:38:34.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983365+0000", 2024-09-14T22:38:34.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5776999999999999e-05 2024-09-14T22:38:34.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983437+0000", 2024-09-14T22:38:34.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2158999999999998e-05 2024-09-14T22:38:34.304 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.304 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.304 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.304 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114193 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4399104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:34.304 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.936095+0000", 2024-09-14T22:38:34.304 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.313165397000006, 2024-09-14T22:38:34.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047505915000000003, 2024-09-14T22:38:34.304 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.304 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.304 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.304 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.304 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.304 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.305 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114193 2024-09-14T22:38:34.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.305 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936095+0000", 2024-09-14T22:38:34.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936095+0000", 2024-09-14T22:38:34.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.306 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938021+0000", 2024-09-14T22:38:34.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001926178 2024-09-14T22:38:34.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.306 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938027+0000", 2024-09-14T22:38:34.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3110000000000002e-06 2024-09-14T22:38:34.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.306 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938041+0000", 2024-09-14T22:38:34.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3344e-05 2024-09-14T22:38:34.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938049+0000", 2024-09-14T22:38:34.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2279999999999998e-06 2024-09-14T22:38:34.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983468+0000", 2024-09-14T22:38:34.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045419494999999997 2024-09-14T22:38:34.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983524+0000", 2024-09-14T22:38:34.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5952000000000002e-05 2024-09-14T22:38:34.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983601+0000", 2024-09-14T22:38:34.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6407000000000002e-05 2024-09-14T22:38:34.308 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.308 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.308 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.308 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114710 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4857856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:34.308 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.962571+0000", 2024-09-14T22:38:34.308 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.286689124999995, 2024-09-14T22:38:34.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047253314999999997, 2024-09-14T22:38:34.308 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.308 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.308 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.309 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.309 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.309 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.309 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114710 2024-09-14T22:38:34.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.309 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962571+0000", 2024-09-14T22:38:34.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962571+0000", 2024-09-14T22:38:34.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962572+0000", 2024-09-14T22:38:34.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1399999999999997e-07 2024-09-14T22:38:34.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962574+0000", 2024-09-14T22:38:34.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8679999999999999e-06 2024-09-14T22:38:34.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.311 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.311 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962583+0000", 2024-09-14T22:38:34.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8310000000000007e-06 2024-09-14T22:38:34.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.311 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.311 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962589+0000", 2024-09-14T22:38:34.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9529999999999996e-06 2024-09-14T22:38:34.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.311 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.311 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009692+0000", 2024-09-14T22:38:34.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047103249999999999 2024-09-14T22:38:34.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.312 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.312 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009767+0000", 2024-09-14T22:38:34.312 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4672000000000006e-05 2024-09-14T22:38:34.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.312 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.312 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009824+0000", 2024-09-14T22:38:34.312 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7226999999999999e-05 2024-09-14T22:38:34.312 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.312 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.312 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.313 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114712 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4907008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:34.313 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:38:34.313 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.286663645000004, 2024-09-14T22:38:34.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047421087000000001, 2024-09-14T22:38:34.313 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.313 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.313 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.313 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.313 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.313 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.313 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114712 2024-09-14T22:38:34.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.313 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:38:34.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:38:34.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:38:34.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.03e-07 2024-09-14T22:38:34.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962599+0000", 2024-09-14T22:38:34.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3199999999999998e-06 2024-09-14T22:38:34.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.315 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962606+0000", 2024-09-14T22:38:34.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5289999999999997e-06 2024-09-14T22:38:34.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.315 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962612+0000", 2024-09-14T22:38:34.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.947e-06 2024-09-14T22:38:34.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.316 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.316 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009866+0000", 2024-09-14T22:38:34.316 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047254681999999999 2024-09-14T22:38:34.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.316 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.316 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009924+0000", 2024-09-14T22:38:34.316 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7556000000000003e-05 2024-09-14T22:38:34.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.316 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.316 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010018+0000", 2024-09-14T22:38:34.316 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.365e-05 2024-09-14T22:38:34.316 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.316 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.317 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.317 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114714 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4956160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:34.317 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.962618+0000", 2024-09-14T22:38:34.317 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.286642126999993, 2024-09-14T22:38:34.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.04759302, 2024-09-14T22:38:34.317 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.317 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.317 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.317 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.317 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.317 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.317 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114714 2024-09-14T22:38:34.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.318 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962618+0000", 2024-09-14T22:38:34.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962618+0000", 2024-09-14T22:38:34.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966011+0000", 2024-09-14T22:38:34.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003393067 2024-09-14T22:38:34.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966016+0000", 2024-09-14T22:38:34.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3580000000000002e-06 2024-09-14T22:38:34.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966032+0000", 2024-09-14T22:38:34.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5356e-05 2024-09-14T22:38:34.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966040+0000", 2024-09-14T22:38:34.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1980000000000001e-06 2024-09-14T22:38:34.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.320 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.320 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010055+0000", 2024-09-14T22:38:34.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044014676000000003 2024-09-14T22:38:34.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.320 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.320 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010127+0000", 2024-09-14T22:38:34.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2206000000000006e-05 2024-09-14T22:38:34.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.320 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.320 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010211+0000", 2024-09-14T22:38:34.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4159000000000005e-05 2024-09-14T22:38:34.321 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.321 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.321 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.321 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114754 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 6283264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:34.321 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:38:34.321 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.282467213999993, 2024-09-14T22:38:34.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047228150000000003, 2024-09-14T22:38:34.321 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.321 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.321 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.321 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.321 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.322 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.322 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114754 2024-09-14T22:38:34.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.322 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:38:34.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:38:34.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:38:34.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7399999999999999e-07 2024-09-14T22:38:34.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966796+0000", 2024-09-14T22:38:34.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3630000000000001e-06 2024-09-14T22:38:34.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966802+0000", 2024-09-14T22:38:34.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4679999999999997e-06 2024-09-14T22:38:34.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966807+0000", 2024-09-14T22:38:34.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1109999999999999e-06 2024-09-14T22:38:34.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.013929+0000", 2024-09-14T22:38:34.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047121720999999998 2024-09-14T22:38:34.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.013968+0000", 2024-09-14T22:38:34.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8877000000000001e-05 2024-09-14T22:38:34.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.325 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014021+0000", 2024-09-14T22:38:34.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3236000000000003e-05 2024-09-14T22:38:34.325 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.325 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.325 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.325 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114789 3.19 3:9836bb25:::rbd_object_map.113c14c67864.000000000000001b:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:34.325 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.966813+0000", 2024-09-14T22:38:34.325 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.282446751999998, 2024-09-14T22:38:34.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05001311, 2024-09-14T22:38:34.325 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.325 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.326 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:34.326 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.326 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.326 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.326 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114789 2024-09-14T22:38:34.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.326 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966813+0000", 2024-09-14T22:38:34.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966813+0000", 2024-09-14T22:38:34.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966814+0000", 2024-09-14T22:38:34.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9200000000000002e-07 2024-09-14T22:38:34.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966817+0000", 2024-09-14T22:38:34.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8380000000000002e-06 2024-09-14T22:38:34.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966831+0000", 2024-09-14T22:38:34.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4209999999999999e-05 2024-09-14T22:38:34.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966845+0000", 2024-09-14T22:38:34.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3799e-05 2024-09-14T22:38:34.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014046+0000", 2024-09-14T22:38:34.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047201604000000001 2024-09-14T22:38:34.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014116+0000", 2024-09-14T22:38:34.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9387000000000005e-05 2024-09-14T22:38:34.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-14T22:38:34.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014469+0000", 2024-09-14T22:38:34.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00035355800000000002 2024-09-14T22:38:34.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:34.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016719+0000", 2024-09-14T22:38:34.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0022498459999999998 2024-09-14T22:38:34.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:38:34.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016735+0000", 2024-09-14T22:38:34.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5792999999999999e-05 2024-09-14T22:38:34.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:38:34.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016790+0000", 2024-09-14T22:38:34.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4715e-05 2024-09-14T22:38:34.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016827+0000", 2024-09-14T22:38:34.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6968000000000002e-05 2024-09-14T22:38:34.330 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.330 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.330 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.331 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114803 3.19 3:9836bb25:::rbd_object_map.113c14c67864.000000000000001b:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:34.331 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.966853+0000", 2024-09-14T22:38:34.331 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.282407321999997, 2024-09-14T22:38:34.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050656093999999999, 2024-09-14T22:38:34.331 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.331 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.331 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:34.331 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.331 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.331 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.331 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114803 2024-09-14T22:38:34.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.331 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966853+0000", 2024-09-14T22:38:34.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966853+0000", 2024-09-14T22:38:34.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969930+0000", 2024-09-14T22:38:34.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030769759999999999 2024-09-14T22:38:34.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969936+0000", 2024-09-14T22:38:34.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7150000000000003e-06 2024-09-14T22:38:34.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969949+0000", 2024-09-14T22:38:34.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3166000000000001e-05 2024-09-14T22:38:34.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969957+0000", 2024-09-14T22:38:34.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8269999999999998e-06 2024-09-14T22:38:34.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014996+0000", 2024-09-14T22:38:34.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045039202 2024-09-14T22:38:34.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.015085+0000", 2024-09-14T22:38:34.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8987999999999996e-05 2024-09-14T22:38:34.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-14T22:38:34.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.015318+0000", 2024-09-14T22:38:34.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023284999999999999 2024-09-14T22:38:34.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:34.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016898+0000", 2024-09-14T22:38:34.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0015807379999999999 2024-09-14T22:38:34.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:38:34.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.017455+0000", 2024-09-14T22:38:34.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00055664600000000005 2024-09-14T22:38:34.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:38:34.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.017483+0000", 2024-09-14T22:38:34.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8238e-05 2024-09-14T22:38:34.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.017509+0000", 2024-09-14T22:38:34.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5748000000000001e-05 2024-09-14T22:38:34.336 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.336 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.336 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.336 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:150759 3.8 3:15dc94e5:::rbd_header.113c88f88d4a:head [create,call rbd.create in=9b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e113)", 2024-09-14T22:38:34.336 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:38:05.726302+0000", 2024-09-14T22:38:34.336 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.522958630000002, 2024-09-14T22:38:34.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0241160819999999, 2024-09-14T22:38:34.336 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.336 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.336 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:34.336 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.336 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.337 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.337 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 150759 2024-09-14T22:38:34.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.337 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.726302+0000", 2024-09-14T22:38:34.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.726302+0000", 2024-09-14T22:38:34.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.726304+0000", 2024-09-14T22:38:34.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.075e-06 2024-09-14T22:38:34.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.726312+0000", 2024-09-14T22:38:34.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2050000000000002e-06 2024-09-14T22:38:34.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.727969+0000", 2024-09-14T22:38:34.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001657499 2024-09-14T22:38:34.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.727985+0000", 2024-09-14T22:38:34.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6082999999999998e-05 2024-09-14T22:38:34.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.728193+0000", 2024-09-14T22:38:34.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000208082 2024-09-14T22:38:34.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:34.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.728205+0000", 2024-09-14T22:38:34.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1897e-05 2024-09-14T22:38:34.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.741926+0000", 2024-09-14T22:38:34.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.013720591 2024-09-14T22:38:34.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.742076+0000", 2024-09-14T22:38:34.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015024099999999999 2024-09-14T22:38:34.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-14T22:38:34.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.742415+0000", 2024-09-14T22:38:34.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00033887700000000002 2024-09-14T22:38:34.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:38:34.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.743436+0000", 2024-09-14T22:38:34.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010210639999999999 2024-09-14T22:38:34.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:34.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.750216+0000", 2024-09-14T22:38:34.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0067798950000000002 2024-09-14T22:38:34.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:38:34.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.750288+0000", 2024-09-14T22:38:34.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2192000000000003e-05 2024-09-14T22:38:34.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.750418+0000", 2024-09-14T22:38:34.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000129381 2024-09-14T22:38:34.341 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.342 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.342 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.342 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.342 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.342 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:34.343 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:38:34.344 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:34.341+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:34.445 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:34.438+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:34.513 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:34.514 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:34.514 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:38:34.514 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:34.528 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:38:34.545 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:34.538+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:34.646 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:34.642+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:34.652 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:34.652 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:34.653 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:38:34.653 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:38:34.653 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:34.667 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:38:34.747 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:34.745+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:34.795 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:34.795 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:38:34.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:38:34.795 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:38:34.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.795 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145249 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 15745024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:34.795 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.693850+0000", 2024-09-14T22:38:34.796 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.096895863, 2024-09-14T22:38:34.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.060850297999999997, 2024-09-14T22:38:34.796 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.796 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.796 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.796 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.796 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.796 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.796 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145249 2024-09-14T22:38:34.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.796 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693850+0000", 2024-09-14T22:38:34.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693850+0000", 2024-09-14T22:38:34.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693851+0000", 2024-09-14T22:38:34.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.46e-07 2024-09-14T22:38:34.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693853+0000", 2024-09-14T22:38:34.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0940000000000002e-06 2024-09-14T22:38:34.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693858+0000", 2024-09-14T22:38:34.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2100000000000001e-06 2024-09-14T22:38:34.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693863+0000", 2024-09-14T22:38:34.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5549999999999996e-06 2024-09-14T22:38:34.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754503+0000", 2024-09-14T22:38:34.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.060640558999999997 2024-09-14T22:38:34.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754613+0000", 2024-09-14T22:38:34.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109966 2024-09-14T22:38:34.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754701+0000", 2024-09-14T22:38:34.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7567999999999999e-05 2024-09-14T22:38:34.800 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.800 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.800 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.801 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145250 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16039936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:34.801 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.693868+0000", 2024-09-14T22:38:34.801 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.096878134999997, 2024-09-14T22:38:34.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061182667000000003, 2024-09-14T22:38:34.801 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.801 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.801 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.801 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.801 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.801 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.801 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145250 2024-09-14T22:38:34.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.802 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693868+0000", 2024-09-14T22:38:34.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693868+0000", 2024-09-14T22:38:34.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693869+0000", 2024-09-14T22:38:34.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4400000000000001e-07 2024-09-14T22:38:34.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693870+0000", 2024-09-14T22:38:34.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9130000000000001e-06 2024-09-14T22:38:34.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693876+0000", 2024-09-14T22:38:34.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1270000000000002e-06 2024-09-14T22:38:34.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693880+0000", 2024-09-14T22:38:34.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.673e-06 2024-09-14T22:38:34.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754780+0000", 2024-09-14T22:38:34.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.060899495999999997 2024-09-14T22:38:34.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754925+0000", 2024-09-14T22:38:34.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014511799999999999 2024-09-14T22:38:34.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755051+0000", 2024-09-14T22:38:34.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012599599999999999 2024-09-14T22:38:34.806 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.806 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.806 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.806 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145251 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16236544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:34.806 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.693886+0000", 2024-09-14T22:38:34.806 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.096860481999997, 2024-09-14T22:38:34.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061498816999999997, 2024-09-14T22:38:34.807 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.807 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.807 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.807 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.807 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.807 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.807 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145251 2024-09-14T22:38:34.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.807 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693886+0000", 2024-09-14T22:38:34.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693886+0000", 2024-09-14T22:38:34.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693886+0000", 2024-09-14T22:38:34.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1199999999999998e-07 2024-09-14T22:38:34.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.809 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693888+0000", 2024-09-14T22:38:34.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.877e-06 2024-09-14T22:38:34.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.809 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693894+0000", 2024-09-14T22:38:34.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5350000000000004e-06 2024-09-14T22:38:34.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693917+0000", 2024-09-14T22:38:34.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.296e-05 2024-09-14T22:38:34.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755127+0000", 2024-09-14T22:38:34.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061210286000000003 2024-09-14T22:38:34.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755266+0000", 2024-09-14T22:38:34.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013871599999999999 2024-09-14T22:38:34.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755385+0000", 2024-09-14T22:38:34.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011903100000000001 2024-09-14T22:38:34.811 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.811 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.811 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.812 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145252 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16302080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:34.812 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.693985+0000", 2024-09-14T22:38:34.812 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.096761800000003, 2024-09-14T22:38:34.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061743767999999997, 2024-09-14T22:38:34.812 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.812 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.812 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.812 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.812 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.812 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.812 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145252 2024-09-14T22:38:34.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.813 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693985+0000", 2024-09-14T22:38:34.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693985+0000", 2024-09-14T22:38:34.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693986+0000", 2024-09-14T22:38:34.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0640000000000001e-06 2024-09-14T22:38:34.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693991+0000", 2024-09-14T22:38:34.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7599999999999999e-06 2024-09-14T22:38:34.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694005+0000", 2024-09-14T22:38:34.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3271999999999999e-05 2024-09-14T22:38:34.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694015+0000", 2024-09-14T22:38:34.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0617e-05 2024-09-14T22:38:34.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755459+0000", 2024-09-14T22:38:34.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061443820000000003 2024-09-14T22:38:34.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755605+0000", 2024-09-14T22:38:34.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014569 2024-09-14T22:38:34.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755728+0000", 2024-09-14T22:38:34.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000123545 2024-09-14T22:38:34.817 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.817 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.817 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.817 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145253 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16138240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:34.817 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694031+0000", 2024-09-14T22:38:34.817 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.096715246000002, 2024-09-14T22:38:34.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062032836000000001, 2024-09-14T22:38:34.818 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.818 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.818 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.818 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.818 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.818 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.818 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145253 2024-09-14T22:38:34.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.818 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694031+0000", 2024-09-14T22:38:34.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694031+0000", 2024-09-14T22:38:34.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694032+0000", 2024-09-14T22:38:34.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7999999999999995e-07 2024-09-14T22:38:34.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694036+0000", 2024-09-14T22:38:34.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8800000000000001e-06 2024-09-14T22:38:34.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694047+0000", 2024-09-14T22:38:34.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0835999999999999e-05 2024-09-14T22:38:34.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694055+0000", 2024-09-14T22:38:34.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6619999999999994e-06 2024-09-14T22:38:34.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755804+0000", 2024-09-14T22:38:34.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061748448999999997 2024-09-14T22:38:34.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755941+0000", 2024-09-14T22:38:34.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013751599999999999 2024-09-14T22:38:34.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756064+0000", 2024-09-14T22:38:34.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012281299999999999 2024-09-14T22:38:34.822 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.822 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.822 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.823 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145254 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16400384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:34.823 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694067+0000", 2024-09-14T22:38:34.823 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.096679080999998, 2024-09-14T22:38:34.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062280549999999997, 2024-09-14T22:38:34.823 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.823 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.823 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.823 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.823 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.824 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.824 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145254 2024-09-14T22:38:34.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.824 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694067+0000", 2024-09-14T22:38:34.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694067+0000", 2024-09-14T22:38:34.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694068+0000", 2024-09-14T22:38:34.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2200000000000003e-07 2024-09-14T22:38:34.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694072+0000", 2024-09-14T22:38:34.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5729999999999999e-06 2024-09-14T22:38:34.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694082+0000", 2024-09-14T22:38:34.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0703999999999999e-05 2024-09-14T22:38:34.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694091+0000", 2024-09-14T22:38:34.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8030000000000003e-06 2024-09-14T22:38:34.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756155+0000", 2024-09-14T22:38:34.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062063937 2024-09-14T22:38:34.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756263+0000", 2024-09-14T22:38:34.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108229 2024-09-14T22:38:34.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756348+0000", 2024-09-14T22:38:34.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4581999999999998e-05 2024-09-14T22:38:34.828 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.828 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.828 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.828 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145255 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16596992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:34.828 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694103+0000", 2024-09-14T22:38:34.828 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.0966436, 2024-09-14T22:38:34.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062580620000000003, 2024-09-14T22:38:34.828 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.829 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.829 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.829 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.829 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.829 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.829 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145255 2024-09-14T22:38:34.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.829 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694103+0000", 2024-09-14T22:38:34.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694103+0000", 2024-09-14T22:38:34.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694103+0000", 2024-09-14T22:38:34.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7800000000000001e-07 2024-09-14T22:38:34.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694107+0000", 2024-09-14T22:38:34.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5939999999999998e-06 2024-09-14T22:38:34.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694121+0000", 2024-09-14T22:38:34.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3913999999999999e-05 2024-09-14T22:38:34.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694130+0000", 2024-09-14T22:38:34.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7690000000000002e-06 2024-09-14T22:38:34.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756431+0000", 2024-09-14T22:38:34.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062301593000000002 2024-09-14T22:38:34.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756564+0000", 2024-09-14T22:38:34.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000132693 2024-09-14T22:38:34.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756683+0000", 2024-09-14T22:38:34.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119479 2024-09-14T22:38:34.833 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.833 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.833 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.834 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145256 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16695296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:34.834 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694142+0000", 2024-09-14T22:38:34.834 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.096604526999997, 2024-09-14T22:38:34.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062806034999999996, 2024-09-14T22:38:34.834 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.834 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.834 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.834 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.834 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.834 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.834 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145256 2024-09-14T22:38:34.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.835 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694142+0000", 2024-09-14T22:38:34.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694142+0000", 2024-09-14T22:38:34.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694142+0000", 2024-09-14T22:38:34.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1699999999999998e-07 2024-09-14T22:38:34.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694146+0000", 2024-09-14T22:38:34.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7450000000000001e-06 2024-09-14T22:38:34.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694157+0000", 2024-09-14T22:38:34.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1161e-05 2024-09-14T22:38:34.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694166+0000", 2024-09-14T22:38:34.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0450000000000006e-06 2024-09-14T22:38:34.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756768+0000", 2024-09-14T22:38:34.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062601554000000004 2024-09-14T22:38:34.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756867+0000", 2024-09-14T22:38:34.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8741999999999998e-05 2024-09-14T22:38:34.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756948+0000", 2024-09-14T22:38:34.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1170999999999999e-05 2024-09-14T22:38:34.838 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.838 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.839 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.839 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145257 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16498688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:34.839 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694179+0000", 2024-09-14T22:38:34.839 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.096567227999998, 2024-09-14T22:38:34.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063118469999999996, 2024-09-14T22:38:34.839 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.839 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.839 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.839 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.839 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.840 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.840 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145257 2024-09-14T22:38:34.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.840 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.840 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.840 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694179+0000", 2024-09-14T22:38:34.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.840 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.840 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694179+0000", 2024-09-14T22:38:34.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694180+0000", 2024-09-14T22:38:34.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1099999999999995e-07 2024-09-14T22:38:34.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694196+0000", 2024-09-14T22:38:34.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5996000000000001e-05 2024-09-14T22:38:34.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694208+0000", 2024-09-14T22:38:34.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2584e-05 2024-09-14T22:38:34.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694219+0000", 2024-09-14T22:38:34.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0462000000000001e-05 2024-09-14T22:38:34.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757033+0000", 2024-09-14T22:38:34.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062814485000000003 2024-09-14T22:38:34.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757183+0000", 2024-09-14T22:38:34.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014914099999999999 2024-09-14T22:38:34.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757298+0000", 2024-09-14T22:38:34.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115091 2024-09-14T22:38:34.844 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.844 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.844 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.844 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145258 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16269312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:34.844 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694257+0000", 2024-09-14T22:38:34.844 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.096489034000001, 2024-09-14T22:38:34.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063297279999999997, 2024-09-14T22:38:34.844 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.844 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.845 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.845 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.845 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.845 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.845 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145258 2024-09-14T22:38:34.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.845 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694257+0000", 2024-09-14T22:38:34.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694257+0000", 2024-09-14T22:38:34.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694258+0000", 2024-09-14T22:38:34.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7700000000000004e-07 2024-09-14T22:38:34.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694263+0000", 2024-09-14T22:38:34.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6609999999999999e-06 2024-09-14T22:38:34.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694271+0000", 2024-09-14T22:38:34.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7809999999999994e-06 2024-09-14T22:38:34.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694277+0000", 2024-09-14T22:38:34.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.823e-06 2024-09-14T22:38:34.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757377+0000", 2024-09-14T22:38:34.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063100081000000002 2024-09-14T22:38:34.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757476+0000", 2024-09-14T22:38:34.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8507999999999999e-05 2024-09-14T22:38:34.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757555+0000", 2024-09-14T22:38:34.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8749000000000001e-05 2024-09-14T22:38:34.849 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.849 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.849 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.849 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145259 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16465920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:34.849 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694286+0000", 2024-09-14T22:38:34.850 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.096460837000002, 2024-09-14T22:38:34.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063608168000000007, 2024-09-14T22:38:34.850 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.850 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.850 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.850 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.850 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.850 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.850 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145259 2024-09-14T22:38:34.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.850 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694286+0000", 2024-09-14T22:38:34.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694286+0000", 2024-09-14T22:38:34.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694286+0000", 2024-09-14T22:38:34.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9999999999999998e-07 2024-09-14T22:38:34.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694288+0000", 2024-09-14T22:38:34.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3760000000000002e-06 2024-09-14T22:38:34.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694294+0000", 2024-09-14T22:38:34.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6779999999999997e-06 2024-09-14T22:38:34.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694298+0000", 2024-09-14T22:38:34.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3309999999999997e-06 2024-09-14T22:38:34.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757638+0000", 2024-09-14T22:38:34.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063339464999999998 2024-09-14T22:38:34.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757769+0000", 2024-09-14T22:38:34.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013148700000000001 2024-09-14T22:38:34.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757894+0000", 2024-09-14T22:38:34.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012443099999999999 2024-09-14T22:38:34.854 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.854 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.854 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.855 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145260 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16367616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:34.855 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694304+0000", 2024-09-14T22:38:34.855 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.096442043000003, 2024-09-14T22:38:34.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063838521999999995, 2024-09-14T22:38:34.855 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.855 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.855 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.855 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.855 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.855 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.855 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145260 2024-09-14T22:38:34.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.856 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694304+0000", 2024-09-14T22:38:34.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694304+0000", 2024-09-14T22:38:34.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694305+0000", 2024-09-14T22:38:34.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8299999999999997e-07 2024-09-14T22:38:34.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694307+0000", 2024-09-14T22:38:34.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9989999999999998e-06 2024-09-14T22:38:34.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694312+0000", 2024-09-14T22:38:34.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2689999999999999e-06 2024-09-14T22:38:34.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694317+0000", 2024-09-14T22:38:34.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9359999999999998e-06 2024-09-14T22:38:34.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757944+0000", 2024-09-14T22:38:34.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063626951000000001 2024-09-14T22:38:34.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758065+0000", 2024-09-14T22:38:34.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120945 2024-09-14T22:38:34.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758143+0000", 2024-09-14T22:38:34.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7838999999999998e-05 2024-09-14T22:38:34.860 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.860 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.860 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.860 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145261 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16564224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:34.860 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694323+0000", 2024-09-14T22:38:34.860 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.096423340000001, 2024-09-14T22:38:34.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064150180000000001, 2024-09-14T22:38:34.860 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.860 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.861 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.861 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.861 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.861 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.861 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145261 2024-09-14T22:38:34.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.861 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694323+0000", 2024-09-14T22:38:34.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694323+0000", 2024-09-14T22:38:34.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694323+0000", 2024-09-14T22:38:34.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4900000000000001e-07 2024-09-14T22:38:34.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.863 INFO:teuthology.orchestra.run.smithi001.stderr:set 3.9 pg_upmap_items mapping to [2->4,0->7] 2024-09-14T22:38:34.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694325+0000", 2024-09-14T22:38:34.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.04e-06 2024-09-14T22:38:34.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694331+0000", 2024-09-14T22:38:34.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3449999999999996e-06 2024-09-14T22:38:34.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694335+0000", 2024-09-14T22:38:34.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.566e-06 2024-09-14T22:38:34.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758225+0000", 2024-09-14T22:38:34.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063889535999999997 2024-09-14T22:38:34.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758357+0000", 2024-09-14T22:38:34.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013181 2024-09-14T22:38:34.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758473+0000", 2024-09-14T22:38:34.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011653400000000001 2024-09-14T22:38:34.866 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.866 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.867 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.867 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145262 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16760832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:34.867 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694341+0000", 2024-09-14T22:38:34.867 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.096405482000002, 2024-09-14T22:38:34.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064424631999999996, 2024-09-14T22:38:34.867 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.867 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.867 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.867 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.867 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.868 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.868 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145262 2024-09-14T22:38:34.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.868 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694341+0000", 2024-09-14T22:38:34.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694341+0000", 2024-09-14T22:38:34.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694341+0000", 2024-09-14T22:38:34.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5499999999999999e-07 2024-09-14T22:38:34.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694343+0000", 2024-09-14T22:38:34.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9630000000000001e-06 2024-09-14T22:38:34.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694349+0000", 2024-09-14T22:38:34.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3120000000000001e-06 2024-09-14T22:38:34.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694353+0000", 2024-09-14T22:38:34.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6500000000000004e-06 2024-09-14T22:38:34.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758561+0000", 2024-09-14T22:38:34.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064207934999999994 2024-09-14T22:38:34.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758677+0000", 2024-09-14T22:38:34.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116241 2024-09-14T22:38:34.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758766+0000", 2024-09-14T22:38:34.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8176000000000004e-05 2024-09-14T22:38:34.871 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.872 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.872 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.872 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145263 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16662528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:34.872 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694359+0000", 2024-09-14T22:38:34.872 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.096387639, 2024-09-14T22:38:34.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064743189000000007, 2024-09-14T22:38:34.872 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.872 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.872 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.873 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.873 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.873 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.873 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145263 2024-09-14T22:38:34.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.873 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694359+0000", 2024-09-14T22:38:34.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694359+0000", 2024-09-14T22:38:34.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694359+0000", 2024-09-14T22:38:34.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5100000000000001e-07 2024-09-14T22:38:34.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694361+0000", 2024-09-14T22:38:34.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9429999999999999e-06 2024-09-14T22:38:34.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694366+0000", 2024-09-14T22:38:34.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3489999999999999e-06 2024-09-14T22:38:34.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694371+0000", 2024-09-14T22:38:34.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4399999999999998e-06 2024-09-14T22:38:34.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758848+0000", 2024-09-14T22:38:34.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064477396000000006 2024-09-14T22:38:34.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758980+0000", 2024-09-14T22:38:34.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013162199999999999 2024-09-14T22:38:34.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759102+0000", 2024-09-14T22:38:34.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012208799999999999 2024-09-14T22:38:34.877 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.877 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.877 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.877 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145264 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16334848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:34.877 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694377+0000", 2024-09-14T22:38:34.877 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.096369605, 2024-09-14T22:38:34.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065048054999999994, 2024-09-14T22:38:34.877 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.877 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.878 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.878 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.878 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.878 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.878 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145264 2024-09-14T22:38:34.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.878 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694377+0000", 2024-09-14T22:38:34.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694377+0000", 2024-09-14T22:38:34.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694377+0000", 2024-09-14T22:38:34.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.39e-07 2024-09-14T22:38:34.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694379+0000", 2024-09-14T22:38:34.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0190000000000001e-06 2024-09-14T22:38:34.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694385+0000", 2024-09-14T22:38:34.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4399999999999996e-06 2024-09-14T22:38:34.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694389+0000", 2024-09-14T22:38:34.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.639e-06 2024-09-14T22:38:34.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759175+0000", 2024-09-14T22:38:34.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064785899999999993 2024-09-14T22:38:34.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759309+0000", 2024-09-14T22:38:34.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013374400000000001 2024-09-14T22:38:34.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759425+0000", 2024-09-14T22:38:34.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115974 2024-09-14T22:38:34.882 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.882 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.882 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.882 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145265 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16531456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:34.882 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694395+0000", 2024-09-14T22:38:34.882 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.096351482999999, 2024-09-14T22:38:34.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065298325000000004, 2024-09-14T22:38:34.883 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.883 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.883 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.883 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.883 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.883 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.883 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145265 2024-09-14T22:38:34.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.883 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694395+0000", 2024-09-14T22:38:34.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694395+0000", 2024-09-14T22:38:34.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694395+0000", 2024-09-14T22:38:34.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6699999999999999e-07 2024-09-14T22:38:34.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694397+0000", 2024-09-14T22:38:34.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.984e-06 2024-09-14T22:38:34.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694403+0000", 2024-09-14T22:38:34.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3519999999999997e-06 2024-09-14T22:38:34.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694407+0000", 2024-09-14T22:38:34.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3239999999999996e-06 2024-09-14T22:38:34.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.886 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759516+0000", 2024-09-14T22:38:34.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065109279000000006 2024-09-14T22:38:34.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.886 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759610+0000", 2024-09-14T22:38:34.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3566000000000005e-05 2024-09-14T22:38:34.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759693+0000", 2024-09-14T22:38:34.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3453000000000002e-05 2024-09-14T22:38:34.887 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.887 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.887 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.887 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145266 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16433152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:34.887 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694413+0000", 2024-09-14T22:38:34.887 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.096333684000001, 2024-09-14T22:38:34.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065618308, 2024-09-14T22:38:34.888 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.888 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.888 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.888 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.888 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.888 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.888 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145266 2024-09-14T22:38:34.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.888 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694413+0000", 2024-09-14T22:38:34.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694413+0000", 2024-09-14T22:38:34.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694413+0000", 2024-09-14T22:38:34.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4799999999999999e-07 2024-09-14T22:38:34.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694415+0000", 2024-09-14T22:38:34.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.08e-06 2024-09-14T22:38:34.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694420+0000", 2024-09-14T22:38:34.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.203e-06 2024-09-14T22:38:34.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694425+0000", 2024-09-14T22:38:34.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4649999999999996e-06 2024-09-14T22:38:34.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759768+0000", 2024-09-14T22:38:34.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065343613999999994 2024-09-14T22:38:34.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759909+0000", 2024-09-14T22:38:34.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014054099999999999 2024-09-14T22:38:34.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760031+0000", 2024-09-14T22:38:34.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122057 2024-09-14T22:38:34.892 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.892 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.892 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.893 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145267 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16629760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:34.893 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694431+0000", 2024-09-14T22:38:34.893 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.096315769999997, 2024-09-14T22:38:34.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065850237000000006, 2024-09-14T22:38:34.893 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.893 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.893 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.893 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.893 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.893 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.893 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145267 2024-09-14T22:38:34.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.894 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694431+0000", 2024-09-14T22:38:34.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694431+0000", 2024-09-14T22:38:34.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694431+0000", 2024-09-14T22:38:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4799999999999999e-07 2024-09-14T22:38:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694433+0000", 2024-09-14T22:38:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8470000000000001e-06 2024-09-14T22:38:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694438+0000", 2024-09-14T22:38:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3249999999999998e-06 2024-09-14T22:38:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694442+0000", 2024-09-14T22:38:34.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3490000000000002e-06 2024-09-14T22:38:34.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760113+0000", 2024-09-14T22:38:34.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065670872000000005 2024-09-14T22:38:34.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760199+0000", 2024-09-14T22:38:34.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5982000000000004e-05 2024-09-14T22:38:34.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760281+0000", 2024-09-14T22:38:34.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1514000000000006e-05 2024-09-14T22:38:34.897 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.897 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.897 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.897 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145268 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16728064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:34.897 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694448+0000", 2024-09-14T22:38:34.898 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.096298185999999, 2024-09-14T22:38:34.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066157585000000005, 2024-09-14T22:38:34.898 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:34.898 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:34.898 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:34.898 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:34.898 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:34.898 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:34.898 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145268 2024-09-14T22:38:34.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.898 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:34.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:34.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694448+0000", 2024-09-14T22:38:34.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:34.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694448+0000", 2024-09-14T22:38:34.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:34.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:34.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694449+0000", 2024-09-14T22:38:34.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.39e-07 2024-09-14T22:38:34.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:34.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694450+0000", 2024-09-14T22:38:34.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.906e-06 2024-09-14T22:38:34.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:34.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694456+0000", 2024-09-14T22:38:34.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3820000000000003e-06 2024-09-14T22:38:34.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:34.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694460+0000", 2024-09-14T22:38:34.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6290000000000001e-06 2024-09-14T22:38:34.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:34.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760345+0000", 2024-09-14T22:38:34.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065884540000000005 2024-09-14T22:38:34.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:34.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760482+0000", 2024-09-14T22:38:34.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013731600000000001 2024-09-14T22:38:34.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:34.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:34.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:34.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760606+0000", 2024-09-14T22:38:34.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000123473 2024-09-14T22:38:34.902 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.902 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.903 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.903 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:34.903 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:34.903 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:34.903 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:38:34.904 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T22:38:34.910 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:34.909+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:35.006 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:35.005+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:35.042 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:35.042 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:35.042 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:38:35.042 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:35.056 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:38:35.107 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:35.105+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:35.178 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:35.179 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:35.179 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:38:35.179 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:38:35.179 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:35.194 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:38:35.208 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:35.205+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:35.309 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:35.302+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:35.322 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:35.322 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:38:35.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:38:35.322 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:38:35.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.322 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6395 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8386560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:38:35.322 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:38:35.323 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 131.38274775599999, 2024-09-14T22:38:35.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054534379000000001, 2024-09-14T22:38:35.323 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.323 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.323 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.323 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.323 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.323 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.323 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6395 2024-09-14T22:38:35.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.323 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:38:35.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:38:35.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933833+0000", 2024-09-14T22:38:35.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3859999999999999e-06 2024-09-14T22:38:35.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933838+0000", 2024-09-14T22:38:35.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8709999999999996e-06 2024-09-14T22:38:35.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933854+0000", 2024-09-14T22:38:35.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6189e-05 2024-09-14T22:38:35.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933871+0000", 2024-09-14T22:38:35.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.626e-05 2024-09-14T22:38:35.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988198+0000", 2024-09-14T22:38:35.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054327581999999999 2024-09-14T22:38:35.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988274+0000", 2024-09-14T22:38:35.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5759000000000004e-05 2024-09-14T22:38:35.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988366+0000", 2024-09-14T22:38:35.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2331999999999996e-05 2024-09-14T22:38:35.327 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.327 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.327 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.327 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104944 3.3 3:c3780f3b:::rbd_data.113ca561f7a9.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:35.327 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:38:35.327 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.283145883000003, 2024-09-14T22:38:35.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073041087000000005, 2024-09-14T22:38:35.328 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.328 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.328 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:35.328 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.328 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.328 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.328 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104944 2024-09-14T22:38:35.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.328 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:38:35.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:38:35.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033436+0000", 2024-09-14T22:38:35.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2060000000000001e-06 2024-09-14T22:38:35.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033448+0000", 2024-09-14T22:38:35.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.169e-05 2024-09-14T22:38:35.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033474+0000", 2024-09-14T22:38:35.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6400000000000001e-05 2024-09-14T22:38:35.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033489+0000", 2024-09-14T22:38:35.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5067e-05 2024-09-14T22:38:35.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033658+0000", 2024-09-14T22:38:35.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000168565 2024-09-14T22:38:35.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033793+0000", 2024-09-14T22:38:35.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013536900000000001 2024-09-14T22:38:35.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:38:35.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.047559+0000", 2024-09-14T22:38:35.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013766294 2024-09-14T22:38:35.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:38:35.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106157+0000", 2024-09-14T22:38:35.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058597110000000001 2024-09-14T22:38:35.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:35.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106290+0000", 2024-09-14T22:38:35.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000133244 2024-09-14T22:38:35.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:38:35.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106396+0000", 2024-09-14T22:38:35.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106246 2024-09-14T22:38:35.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106475+0000", 2024-09-14T22:38:35.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8895999999999998e-05 2024-09-14T22:38:35.334 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.334 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.334 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.334 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:111858 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 10403840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:35.334 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:38:35.334 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 67.492692762000004, 2024-09-14T22:38:35.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054398798999999998, 2024-09-14T22:38:35.335 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.335 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.335 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.335 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.335 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.335 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.335 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 111858 2024-09-14T22:38:35.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.335 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:38:35.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:38:35.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823888+0000", 2024-09-14T22:38:35.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0200000000000001e-07 2024-09-14T22:38:35.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823927+0000", 2024-09-14T22:38:35.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8940999999999999e-05 2024-09-14T22:38:35.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823940+0000", 2024-09-14T22:38:35.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3682e-05 2024-09-14T22:38:35.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823950+0000", 2024-09-14T22:38:35.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7110000000000007e-06 2024-09-14T22:38:35.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878202+0000", 2024-09-14T22:38:35.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054251969999999997 2024-09-14T22:38:35.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878249+0000", 2024-09-14T22:38:35.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6854999999999998e-05 2024-09-14T22:38:35.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878286+0000", 2024-09-14T22:38:35.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6838000000000002e-05 2024-09-14T22:38:35.339 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.339 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.339 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.339 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:111859 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 10452992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:35.339 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:38:35.339 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 67.492603642999995, 2024-09-14T22:38:35.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054435292000000003, 2024-09-14T22:38:35.339 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.339 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.339 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.339 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.339 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.340 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.340 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 111859 2024-09-14T22:38:35.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.340 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:38:35.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:38:35.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823977+0000", 2024-09-14T22:38:35.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3500000000000005e-07 2024-09-14T22:38:35.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823981+0000", 2024-09-14T22:38:35.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2050000000000004e-06 2024-09-14T22:38:35.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824007+0000", 2024-09-14T22:38:35.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.62e-05 2024-09-14T22:38:35.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824018+0000", 2024-09-14T22:38:35.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0389e-05 2024-09-14T22:38:35.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878312+0000", 2024-09-14T22:38:35.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054294754000000001 2024-09-14T22:38:35.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878350+0000", 2024-09-14T22:38:35.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7982000000000003e-05 2024-09-14T22:38:35.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878411+0000", 2024-09-14T22:38:35.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0927000000000001e-05 2024-09-14T22:38:35.344 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.344 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.344 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.344 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112030 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6316032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:35.344 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:38:35.344 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 67.492549300999997, 2024-09-14T22:38:35.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054597826000000002, 2024-09-14T22:38:35.344 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.345 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.345 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.345 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.345 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.345 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.345 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112030 2024-09-14T22:38:35.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.345 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:38:35.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:38:35.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824031+0000", 2024-09-14T22:38:35.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3600000000000003e-07 2024-09-14T22:38:35.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824036+0000", 2024-09-14T22:38:35.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7350000000000004e-06 2024-09-14T22:38:35.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.347 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824049+0000", 2024-09-14T22:38:35.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3329000000000001e-05 2024-09-14T22:38:35.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.347 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824057+0000", 2024-09-14T22:38:35.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2460000000000003e-06 2024-09-14T22:38:35.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878464+0000", 2024-09-14T22:38:35.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054406207999999998 2024-09-14T22:38:35.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878554+0000", 2024-09-14T22:38:35.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0199999999999997e-05 2024-09-14T22:38:35.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878628+0000", 2024-09-14T22:38:35.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4371999999999999e-05 2024-09-14T22:38:35.349 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.349 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.349 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.349 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112031 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6365184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:35.349 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:38:35.349 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 67.492510983000003, 2024-09-14T22:38:35.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054741868999999999, 2024-09-14T22:38:35.349 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.349 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.350 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.350 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.350 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.350 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.350 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112031 2024-09-14T22:38:35.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.350 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:38:35.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:38:35.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824070+0000", 2024-09-14T22:38:35.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5099999999999998e-07 2024-09-14T22:38:35.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824073+0000", 2024-09-14T22:38:35.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6940000000000001e-06 2024-09-14T22:38:35.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824084+0000", 2024-09-14T22:38:35.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0380000000000001e-05 2024-09-14T22:38:35.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824091+0000", 2024-09-14T22:38:35.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5730000000000002e-06 2024-09-14T22:38:35.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.353 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878670+0000", 2024-09-14T22:38:35.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054579176 2024-09-14T22:38:35.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.353 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878742+0000", 2024-09-14T22:38:35.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T22:38:35.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.353 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878811+0000", 2024-09-14T22:38:35.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8170999999999995e-05 2024-09-14T22:38:35.354 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.354 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.354 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.354 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112032 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6414336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:35.354 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:38:35.354 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 67.492477117000007, 2024-09-14T22:38:35.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054931916999999997, 2024-09-14T22:38:35.354 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.354 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.354 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.354 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.355 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.355 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.355 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112032 2024-09-14T22:38:35.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.355 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:38:35.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:38:35.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.356 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.356 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:38:35.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:38:35.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.356 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.356 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824107+0000", 2024-09-14T22:38:35.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5640000000000001e-06 2024-09-14T22:38:35.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.357 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824117+0000", 2024-09-14T22:38:35.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0366e-05 2024-09-14T22:38:35.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.357 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824158+0000", 2024-09-14T22:38:35.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0670000000000002e-05 2024-09-14T22:38:35.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.357 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878853+0000", 2024-09-14T22:38:35.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054694853000000002 2024-09-14T22:38:35.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878934+0000", 2024-09-14T22:38:35.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1754000000000004e-05 2024-09-14T22:38:35.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879034+0000", 2024-09-14T22:38:35.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000100115 2024-09-14T22:38:35.358 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.358 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.359 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.359 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112033 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6463488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:35.359 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:38:35.359 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 67.492408459000004, 2024-09-14T22:38:35.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054991076999999999, 2024-09-14T22:38:35.359 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.359 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.359 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.359 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.359 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.359 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.360 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112033 2024-09-14T22:38:35.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.360 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:38:35.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:38:35.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.361 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824172+0000", 2024-09-14T22:38:35.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9999999999999996e-07 2024-09-14T22:38:35.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.361 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824177+0000", 2024-09-14T22:38:35.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5310000000000003e-06 2024-09-14T22:38:35.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.361 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824188+0000", 2024-09-14T22:38:35.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1235e-05 2024-09-14T22:38:35.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824197+0000", 2024-09-14T22:38:35.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7069999999999998e-06 2024-09-14T22:38:35.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879075+0000", 2024-09-14T22:38:35.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054878749999999997 2024-09-14T22:38:35.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879118+0000", 2024-09-14T22:38:35.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2749e-05 2024-09-14T22:38:35.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879162+0000", 2024-09-14T22:38:35.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4205000000000001e-05 2024-09-14T22:38:35.363 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.363 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.363 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.364 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112034 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6512640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:35.364 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:38:35.364 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 67.492371508000005, 2024-09-14T22:38:35.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055181246000000003, 2024-09-14T22:38:35.364 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.364 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.364 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.364 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.364 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.364 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.364 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112034 2024-09-14T22:38:35.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.364 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:38:35.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:38:35.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824209+0000", 2024-09-14T22:38:35.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5499999999999998e-07 2024-09-14T22:38:35.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824212+0000", 2024-09-14T22:38:35.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6119999999999999e-06 2024-09-14T22:38:35.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824224+0000", 2024-09-14T22:38:35.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1127999999999999e-05 2024-09-14T22:38:35.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824232+0000", 2024-09-14T22:38:35.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4700000000000002e-06 2024-09-14T22:38:35.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879209+0000", 2024-09-14T22:38:35.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054977201000000003 2024-09-14T22:38:35.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879308+0000", 2024-09-14T22:38:35.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8419999999999996e-05 2024-09-14T22:38:35.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879389+0000", 2024-09-14T22:38:35.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1760000000000003e-05 2024-09-14T22:38:35.368 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.368 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.368 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.368 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112035 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6561792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:35.369 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:38:35.369 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 67.492277779999995, 2024-09-14T22:38:35.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055300522999999997, 2024-09-14T22:38:35.369 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.369 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.369 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.369 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.369 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.369 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.369 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112035 2024-09-14T22:38:35.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.369 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.370 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:38:35.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.370 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:38:35.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824303+0000", 2024-09-14T22:38:35.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5099999999999998e-07 2024-09-14T22:38:35.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824308+0000", 2024-09-14T22:38:35.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6689999999999997e-06 2024-09-14T22:38:35.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824744+0000", 2024-09-14T22:38:35.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00043601600000000001 2024-09-14T22:38:35.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824756+0000", 2024-09-14T22:38:35.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2476999999999999e-05 2024-09-14T22:38:35.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879434+0000", 2024-09-14T22:38:35.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054677630999999997 2024-09-14T22:38:35.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879518+0000", 2024-09-14T22:38:35.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4220000000000003e-05 2024-09-14T22:38:35.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879602+0000", 2024-09-14T22:38:35.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4559000000000001e-05 2024-09-14T22:38:35.373 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.373 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.373 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.374 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112036 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6610944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:35.374 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:38:35.374 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 67.491786042000001, 2024-09-14T22:38:35.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055036320999999999, 2024-09-14T22:38:35.374 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.374 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.374 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.374 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.374 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.374 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.374 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112036 2024-09-14T22:38:35.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.375 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:38:35.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:38:35.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824795+0000", 2024-09-14T22:38:35.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-06 2024-09-14T22:38:35.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824802+0000", 2024-09-14T22:38:35.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6680000000000004e-06 2024-09-14T22:38:35.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824814+0000", 2024-09-14T22:38:35.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2102e-05 2024-09-14T22:38:35.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824823+0000", 2024-09-14T22:38:35.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5550000000000008e-06 2024-09-14T22:38:35.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879648+0000", 2024-09-14T22:38:35.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054825167000000001 2024-09-14T22:38:35.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879747+0000", 2024-09-14T22:38:35.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8968999999999994e-05 2024-09-14T22:38:35.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879830+0000", 2024-09-14T22:38:35.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2633999999999995e-05 2024-09-14T22:38:35.378 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.378 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.378 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.379 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112037 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6660096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:35.379 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:38:35.379 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 67.491743333000002, 2024-09-14T22:38:35.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055122926000000003, 2024-09-14T22:38:35.379 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.379 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.379 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.379 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.379 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.379 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.379 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112037 2024-09-14T22:38:35.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.379 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:38:35.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:38:35.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824837+0000", 2024-09-14T22:38:35.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5499999999999998e-07 2024-09-14T22:38:35.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824840+0000", 2024-09-14T22:38:35.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0429999999999999e-06 2024-09-14T22:38:35.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824851+0000", 2024-09-14T22:38:35.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1046999999999999e-05 2024-09-14T22:38:35.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824859+0000", 2024-09-14T22:38:35.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1200000000000002e-06 2024-09-14T22:38:35.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879867+0000", 2024-09-14T22:38:35.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055007697000000001 2024-09-14T22:38:35.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879910+0000", 2024-09-14T22:38:35.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3402000000000002e-05 2024-09-14T22:38:35.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879959+0000", 2024-09-14T22:38:35.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9061999999999999e-05 2024-09-14T22:38:35.383 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.383 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.383 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.384 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112038 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6709248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:35.384 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:38:35.384 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 67.491708637000002, 2024-09-14T22:38:35.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055312828000000001, 2024-09-14T22:38:35.384 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.384 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.385 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.385 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.385 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.385 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.385 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112038 2024-09-14T22:38:35.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.386 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:38:35.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:38:35.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824872+0000", 2024-09-14T22:38:35.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T22:38:35.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824875+0000", 2024-09-14T22:38:35.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7519999999999998e-06 2024-09-14T22:38:35.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824917+0000", 2024-09-14T22:38:35.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1047999999999997e-05 2024-09-14T22:38:35.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824926+0000", 2024-09-14T22:38:35.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3749999999999992e-06 2024-09-14T22:38:35.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880006+0000", 2024-09-14T22:38:35.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055080525999999998 2024-09-14T22:38:35.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880102+0000", 2024-09-14T22:38:35.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5995999999999994e-05 2024-09-14T22:38:35.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880184+0000", 2024-09-14T22:38:35.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1478999999999997e-05 2024-09-14T22:38:35.389 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.389 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.389 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.389 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112039 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6758400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:35.390 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:38:35.390 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 67.491639966999998, 2024-09-14T22:38:35.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055449709999999999, 2024-09-14T22:38:35.390 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.390 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.390 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.390 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.390 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.390 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.390 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112039 2024-09-14T22:38:35.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.390 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:38:35.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:38:35.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824941+0000", 2024-09-14T22:38:35.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3699999999999999e-07 2024-09-14T22:38:35.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824946+0000", 2024-09-14T22:38:35.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0209999999999999e-06 2024-09-14T22:38:35.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824972+0000", 2024-09-14T22:38:35.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6750999999999998e-05 2024-09-14T22:38:35.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824982+0000", 2024-09-14T22:38:35.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5019999999999998e-06 2024-09-14T22:38:35.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880227+0000", 2024-09-14T22:38:35.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055245257999999998 2024-09-14T22:38:35.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880305+0000", 2024-09-14T22:38:35.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7424999999999998e-05 2024-09-14T22:38:35.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880389+0000", 2024-09-14T22:38:35.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4815999999999996e-05 2024-09-14T22:38:35.394 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.394 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.394 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.395 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112040 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6807552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:35.395 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:38:35.395 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 67.491583250999994, 2024-09-14T22:38:35.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055534615000000002, 2024-09-14T22:38:35.395 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.395 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.395 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.395 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.395 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.395 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.395 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112040 2024-09-14T22:38:35.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.396 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:38:35.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:38:35.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824997+0000", 2024-09-14T22:38:35.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7199999999999998e-07 2024-09-14T22:38:35.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825042+0000", 2024-09-14T22:38:35.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5065999999999998e-05 2024-09-14T22:38:35.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825060+0000", 2024-09-14T22:38:35.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7520000000000002e-05 2024-09-14T22:38:35.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825070+0000", 2024-09-14T22:38:35.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8230000000000006e-06 2024-09-14T22:38:35.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880427+0000", 2024-09-14T22:38:35.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055357244 2024-09-14T22:38:35.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880483+0000", 2024-09-14T22:38:35.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5807000000000003e-05 2024-09-14T22:38:35.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880531+0000", 2024-09-14T22:38:35.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8383000000000001e-05 2024-09-14T22:38:35.399 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.399 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.399 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.399 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112041 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6856704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:35.399 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:38:35.400 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 67.491494076999999, 2024-09-14T22:38:35.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055573887000000002, 2024-09-14T22:38:35.400 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.400 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.400 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.400 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.400 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.400 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.400 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112041 2024-09-14T22:38:35.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.400 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:38:35.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:38:35.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825087+0000", 2024-09-14T22:38:35.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2399999999999996e-07 2024-09-14T22:38:35.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825091+0000", 2024-09-14T22:38:35.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9729999999999999e-06 2024-09-14T22:38:35.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825101+0000", 2024-09-14T22:38:35.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.093e-05 2024-09-14T22:38:35.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825110+0000", 2024-09-14T22:38:35.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1450000000000008e-06 2024-09-14T22:38:35.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880559+0000", 2024-09-14T22:38:35.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055449684999999999 2024-09-14T22:38:35.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880601+0000", 2024-09-14T22:38:35.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1248000000000002e-05 2024-09-14T22:38:35.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880659+0000", 2024-09-14T22:38:35.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8981999999999999e-05 2024-09-14T22:38:35.404 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.404 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.405 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.405 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112042 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6905856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:35.405 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:38:35.405 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 67.491459343000002, 2024-09-14T22:38:35.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055781415000000001, 2024-09-14T22:38:35.405 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.405 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.405 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.405 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.405 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.406 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.406 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112042 2024-09-14T22:38:35.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.406 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:38:35.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:38:35.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825478+0000", 2024-09-14T22:38:35.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035775 2024-09-14T22:38:35.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825494+0000", 2024-09-14T22:38:35.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5994999999999998e-05 2024-09-14T22:38:35.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825520+0000", 2024-09-14T22:38:35.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5738999999999999e-05 2024-09-14T22:38:35.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825547+0000", 2024-09-14T22:38:35.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7667000000000001e-05 2024-09-14T22:38:35.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880725+0000", 2024-09-14T22:38:35.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055177962999999997 2024-09-14T22:38:35.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880816+0000", 2024-09-14T22:38:35.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0315000000000005e-05 2024-09-14T22:38:35.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880902+0000", 2024-09-14T22:38:35.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5986e-05 2024-09-14T22:38:35.409 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.410 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.410 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.410 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112043 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:35.410 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:38:35.410 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 67.491011338999996, 2024-09-14T22:38:35.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055537912000000002, 2024-09-14T22:38:35.410 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.410 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.410 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.410 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.411 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.411 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.411 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112043 2024-09-14T22:38:35.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.411 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:38:35.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:38:35.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825911+0000", 2024-09-14T22:38:35.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00034306899999999999 2024-09-14T22:38:35.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825918+0000", 2024-09-14T22:38:35.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.002e-06 2024-09-14T22:38:35.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825931+0000", 2024-09-14T22:38:35.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2401e-05 2024-09-14T22:38:35.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825942+0000", 2024-09-14T22:38:35.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0903e-05 2024-09-14T22:38:35.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880951+0000", 2024-09-14T22:38:35.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055009379999999997 2024-09-14T22:38:35.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881027+0000", 2024-09-14T22:38:35.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5944000000000003e-05 2024-09-14T22:38:35.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881106+0000", 2024-09-14T22:38:35.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9213000000000002e-05 2024-09-14T22:38:35.415 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.415 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.415 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.415 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112044 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 7004160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:35.415 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:38:35.415 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 67.490617670000006, 2024-09-14T22:38:35.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055265416999999997, 2024-09-14T22:38:35.415 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.416 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.416 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.416 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.416 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.416 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.416 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112044 2024-09-14T22:38:35.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.416 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:38:35.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:38:35.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826239+0000", 2024-09-14T22:38:35.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027676000000000003 2024-09-14T22:38:35.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826247+0000", 2024-09-14T22:38:35.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7440000000000007e-06 2024-09-14T22:38:35.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826275+0000", 2024-09-14T22:38:35.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8053000000000001e-05 2024-09-14T22:38:35.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826286+0000", 2024-09-14T22:38:35.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1780000000000001e-05 2024-09-14T22:38:35.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881137+0000", 2024-09-14T22:38:35.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054850500000000003 2024-09-14T22:38:35.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881181+0000", 2024-09-14T22:38:35.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4153000000000002e-05 2024-09-14T22:38:35.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881227+0000", 2024-09-14T22:38:35.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6427000000000001e-05 2024-09-14T22:38:35.420 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.420 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.420 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.420 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112045 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 7053312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:35.421 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:38:35.421 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 67.490255437000002, 2024-09-14T22:38:35.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055045050999999998, 2024-09-14T22:38:35.421 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.421 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.421 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.421 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.421 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.421 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.421 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112045 2024-09-14T22:38:35.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.422 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:38:35.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:38:35.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830250+0000", 2024-09-14T22:38:35.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0039258670000000004 2024-09-14T22:38:35.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830260+0000", 2024-09-14T22:38:35.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0088000000000001e-05 2024-09-14T22:38:35.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830281+0000", 2024-09-14T22:38:35.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0720000000000002e-05 2024-09-14T22:38:35.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.424 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830292+0000", 2024-09-14T22:38:35.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1455e-05 2024-09-14T22:38:35.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.424 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881255+0000", 2024-09-14T22:38:35.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.050962247000000002 2024-09-14T22:38:35.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881296+0000", 2024-09-14T22:38:35.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1557999999999998e-05 2024-09-14T22:38:35.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881369+0000", 2024-09-14T22:38:35.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3115999999999996e-05 2024-09-14T22:38:35.425 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.425 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.425 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.425 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.425 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.426 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:35.426 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:38:35.427 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:35.425+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:35.528 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:35.525+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:35.552 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:35.553 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:35.553 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:38:35.553 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:35.568 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:38:35.629 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:35.622+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:35.692 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:35.692 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:35.692 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:38:35.692 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:38:35.693 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:35.706 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:38:35.725 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2024-09-14T22:38:35.729 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:35.725+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:35.730 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:38:35.830 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:35.829+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:35.886 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:35.886 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:38:35.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:38:35.886 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:38:35.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.886 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97197 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 796672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:35.886 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:38:35.886 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 85.811780509000002, 2024-09-14T22:38:35.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12434473, 2024-09-14T22:38:35.887 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.887 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.887 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.887 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.887 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.887 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.887 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97197 2024-09-14T22:38:35.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.887 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.887 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.887 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:38:35.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.888 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.888 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.888 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.888 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:38:35.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.888 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.888 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.888 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.888 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070524+0000", 2024-09-14T22:38:35.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0299999999999998e-07 2024-09-14T22:38:35.888 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.888 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.889 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070527+0000", 2024-09-14T22:38:35.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9569999999999998e-06 2024-09-14T22:38:35.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.889 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070535+0000", 2024-09-14T22:38:35.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7630000000000006e-06 2024-09-14T22:38:35.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.889 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070543+0000", 2024-09-14T22:38:35.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4869999999999996e-06 2024-09-14T22:38:35.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194525+0000", 2024-09-14T22:38:35.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123982096 2024-09-14T22:38:35.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194699+0000", 2024-09-14T22:38:35.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000174105 2024-09-14T22:38:35.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194868+0000", 2024-09-14T22:38:35.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016861900000000001 2024-09-14T22:38:35.891 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.891 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.891 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.891 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97198 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 805888~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:35.891 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:38:35.891 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 85.811752132999999, 2024-09-14T22:38:35.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.1248088, 2024-09-14T22:38:35.891 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.891 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.891 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.892 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.892 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.892 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.892 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97198 2024-09-14T22:38:35.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.892 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.892 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.892 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:38:35.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:38:35.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070552+0000", 2024-09-14T22:38:35.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T22:38:35.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070555+0000", 2024-09-14T22:38:35.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.853e-06 2024-09-14T22:38:35.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.894 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.894 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070565+0000", 2024-09-14T22:38:35.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.048e-05 2024-09-14T22:38:35.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.894 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.894 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070573+0000", 2024-09-14T22:38:35.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7200000000000006e-06 2024-09-14T22:38:35.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.894 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194954+0000", 2024-09-14T22:38:35.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12438089400000001 2024-09-14T22:38:35.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.895 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195145+0000", 2024-09-14T22:38:35.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019091299999999999 2024-09-14T22:38:35.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.895 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195360+0000", 2024-09-14T22:38:35.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021527300000000001 2024-09-14T22:38:35.895 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.896 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.896 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.896 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97199 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 815104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:35.896 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:38:35.896 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 85.811721535999993, 2024-09-14T22:38:35.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125251901, 2024-09-14T22:38:35.896 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.896 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.896 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.896 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.897 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.897 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.897 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97199 2024-09-14T22:38:35.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.897 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:38:35.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:38:35.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070583+0000", 2024-09-14T22:38:35.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6300000000000005e-07 2024-09-14T22:38:35.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070586+0000", 2024-09-14T22:38:35.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8270000000000002e-06 2024-09-14T22:38:35.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070597+0000", 2024-09-14T22:38:35.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1960000000000001e-05 2024-09-14T22:38:35.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.899 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.899 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070605+0000", 2024-09-14T22:38:35.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6459999999999993e-06 2024-09-14T22:38:35.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.899 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.899 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195432+0000", 2024-09-14T22:38:35.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124826926 2024-09-14T22:38:35.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.899 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.900 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195626+0000", 2024-09-14T22:38:35.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019357099999999999 2024-09-14T22:38:35.900 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.900 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.900 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195834+0000", 2024-09-14T22:38:35.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020830800000000001 2024-09-14T22:38:35.900 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.900 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.900 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.900 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.900 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97200 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 824320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:35.901 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:38:35.901 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 85.811689490000006, 2024-09-14T22:38:35.901 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125693477, 2024-09-14T22:38:35.901 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.901 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.901 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.901 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.901 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.901 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.901 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97200 2024-09-14T22:38:35.901 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.901 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.901 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:38:35.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:38:35.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070615+0000", 2024-09-14T22:38:35.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T22:38:35.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070618+0000", 2024-09-14T22:38:35.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.943e-06 2024-09-14T22:38:35.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070627+0000", 2024-09-14T22:38:35.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1099999999999992e-06 2024-09-14T22:38:35.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070634+0000", 2024-09-14T22:38:35.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:38:35.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.904 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.904 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195908+0000", 2024-09-14T22:38:35.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125273893 2024-09-14T22:38:35.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.904 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.904 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196097+0000", 2024-09-14T22:38:35.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018953899999999999 2024-09-14T22:38:35.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.904 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.904 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196308+0000", 2024-09-14T22:38:35.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021027799999999999 2024-09-14T22:38:35.905 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.905 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.905 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.905 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97201 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 833536~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:35.905 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:38:35.905 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 85.811659302999999, 2024-09-14T22:38:35.905 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126133826, 2024-09-14T22:38:35.905 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.905 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.905 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.905 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.906 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.906 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.906 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97201 2024-09-14T22:38:35.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.906 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.906 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.906 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:38:35.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.906 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.906 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:38:35.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074109+0000", 2024-09-14T22:38:35.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003464834 2024-09-14T22:38:35.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074134+0000", 2024-09-14T22:38:35.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5170000000000001e-05 2024-09-14T22:38:35.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074166+0000", 2024-09-14T22:38:35.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1347000000000003e-05 2024-09-14T22:38:35.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074179+0000", 2024-09-14T22:38:35.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.364e-05 2024-09-14T22:38:35.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196377+0000", 2024-09-14T22:38:35.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12219793900000001 2024-09-14T22:38:35.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196568+0000", 2024-09-14T22:38:35.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019097999999999999 2024-09-14T22:38:35.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.909 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.909 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196778+0000", 2024-09-14T22:38:35.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020991599999999999 2024-09-14T22:38:35.909 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.909 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.909 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.909 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97205 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 870400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:35.909 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:38:35.909 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 85.807989739999996, 2024-09-14T22:38:35.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12436148299999999, 2024-09-14T22:38:35.910 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.910 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.910 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.910 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.910 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.910 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.910 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97205 2024-09-14T22:38:35.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.910 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.910 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:38:35.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:38:35.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074315+0000", 2024-09-14T22:38:35.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9299999999999997e-07 2024-09-14T22:38:35.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074318+0000", 2024-09-14T22:38:35.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.19e-06 2024-09-14T22:38:35.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074327+0000", 2024-09-14T22:38:35.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8529999999999998e-06 2024-09-14T22:38:35.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074334+0000", 2024-09-14T22:38:35.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3869999999999997e-06 2024-09-14T22:38:35.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198365+0000", 2024-09-14T22:38:35.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124031222 2024-09-14T22:38:35.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198513+0000", 2024-09-14T22:38:35.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014772299999999999 2024-09-14T22:38:35.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198675+0000", 2024-09-14T22:38:35.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016241500000000001 2024-09-14T22:38:35.913 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.913 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.913 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.914 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97206 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 879616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:35.914 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:38:35.914 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 85.807958283999994, 2024-09-14T22:38:35.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124706284, 2024-09-14T22:38:35.914 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.914 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.914 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.914 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.914 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.914 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.914 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97206 2024-09-14T22:38:35.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.914 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:38:35.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:38:35.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074346+0000", 2024-09-14T22:38:35.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8999999999999996e-07 2024-09-14T22:38:35.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074349+0000", 2024-09-14T22:38:35.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.96e-06 2024-09-14T22:38:35.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074358+0000", 2024-09-14T22:38:35.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9169999999999995e-06 2024-09-14T22:38:35.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074365+0000", 2024-09-14T22:38:35.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3699999999999997e-06 2024-09-14T22:38:35.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198761+0000", 2024-09-14T22:38:35.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124395743 2024-09-14T22:38:35.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198899+0000", 2024-09-14T22:38:35.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013763499999999999 2024-09-14T22:38:35.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199052+0000", 2024-09-14T22:38:35.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015296900000000001 2024-09-14T22:38:35.918 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.918 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.918 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.918 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97207 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 888832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:35.918 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:38:35.918 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 85.807929071000004, 2024-09-14T22:38:35.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124988433, 2024-09-14T22:38:35.918 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.918 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.918 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.918 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.918 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.918 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.919 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97207 2024-09-14T22:38:35.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.919 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:38:35.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:38:35.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074375+0000", 2024-09-14T22:38:35.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:38:35.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074378+0000", 2024-09-14T22:38:35.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1829999999999998e-06 2024-09-14T22:38:35.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074389+0000", 2024-09-14T22:38:35.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0737000000000001e-05 2024-09-14T22:38:35.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074396+0000", 2024-09-14T22:38:35.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2559999999999996e-06 2024-09-14T22:38:35.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199139+0000", 2024-09-14T22:38:35.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12474253 2024-09-14T22:38:35.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199240+0000", 2024-09-14T22:38:35.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101221 2024-09-14T22:38:35.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199363+0000", 2024-09-14T22:38:35.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000122822 2024-09-14T22:38:35.922 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.922 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.922 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.922 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97208 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 898048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:35.922 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:38:35.922 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 85.807897767, 2024-09-14T22:38:35.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12524854899999999, 2024-09-14T22:38:35.923 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.923 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.923 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.923 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.923 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.923 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.923 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97208 2024-09-14T22:38:35.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.923 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.923 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.923 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:38:35.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:38:35.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:38:35.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:38:35.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074410+0000", 2024-09-14T22:38:35.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.18e-06 2024-09-14T22:38:35.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074435+0000", 2024-09-14T22:38:35.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5273e-05 2024-09-14T22:38:35.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074443+0000", 2024-09-14T22:38:35.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3000000000000002e-06 2024-09-14T22:38:35.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199433+0000", 2024-09-14T22:38:35.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124989502 2024-09-14T22:38:35.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199535+0000", 2024-09-14T22:38:35.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102557 2024-09-14T22:38:35.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199654+0000", 2024-09-14T22:38:35.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119061 2024-09-14T22:38:35.926 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.926 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.926 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.927 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97209 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 907264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:35.927 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:38:35.927 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 85.807849962000006, 2024-09-14T22:38:35.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125487354, 2024-09-14T22:38:35.927 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.927 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.927 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.927 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.927 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.927 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.927 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97209 2024-09-14T22:38:35.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.927 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:38:35.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:38:35.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:38:35.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9400000000000005e-07 2024-09-14T22:38:35.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074458+0000", 2024-09-14T22:38:35.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3299999999999999e-06 2024-09-14T22:38:35.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074467+0000", 2024-09-14T22:38:35.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.346e-06 2024-09-14T22:38:35.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074474+0000", 2024-09-14T22:38:35.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.374e-06 2024-09-14T22:38:35.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199720+0000", 2024-09-14T22:38:35.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12524561200000001 2024-09-14T22:38:35.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199820+0000", 2024-09-14T22:38:35.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010049599999999999 2024-09-14T22:38:35.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199941+0000", 2024-09-14T22:38:35.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012050200000000001 2024-09-14T22:38:35.930 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.931 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.931 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.931 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97210 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:35.931 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:38:35.931 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 85.807819968000004, 2024-09-14T22:38:35.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125744942, 2024-09-14T22:38:35.931 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.931 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.931 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.931 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.931 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.931 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.932 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97210 2024-09-14T22:38:35.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.932 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.932 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.932 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.932 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:38:35.932 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.932 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.932 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.932 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:38:35.932 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:38:35.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6599999999999996e-07 2024-09-14T22:38:35.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074487+0000", 2024-09-14T22:38:35.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8140000000000001e-06 2024-09-14T22:38:35.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074496+0000", 2024-09-14T22:38:35.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9330000000000007e-06 2024-09-14T22:38:35.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074503+0000", 2024-09-14T22:38:35.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4100000000000002e-06 2024-09-14T22:38:35.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200009+0000", 2024-09-14T22:38:35.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12550539899999999 2024-09-14T22:38:35.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200108+0000", 2024-09-14T22:38:35.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9182e-05 2024-09-14T22:38:35.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200229+0000", 2024-09-14T22:38:35.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120538 2024-09-14T22:38:35.935 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.935 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.935 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.935 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97211 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 925696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:35.935 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:38:35.935 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 85.807789008, 2024-09-14T22:38:35.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12599848299999999, 2024-09-14T22:38:35.936 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.936 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.936 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.936 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.936 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.936 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.936 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97211 2024-09-14T22:38:35.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.936 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.936 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:38:35.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.937 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.937 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.937 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:38:35.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.937 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.937 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.937 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074516+0000", 2024-09-14T22:38:35.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.42e-06 2024-09-14T22:38:35.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.937 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.937 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.937 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074521+0000", 2024-09-14T22:38:35.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7960000000000003e-06 2024-09-14T22:38:35.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.938 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.938 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074538+0000", 2024-09-14T22:38:35.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6900000000000001e-05 2024-09-14T22:38:35.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.938 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.938 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074555+0000", 2024-09-14T22:38:35.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.696e-05 2024-09-14T22:38:35.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.938 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.938 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200293+0000", 2024-09-14T22:38:35.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12573864100000001 2024-09-14T22:38:35.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200393+0000", 2024-09-14T22:38:35.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9264000000000004e-05 2024-09-14T22:38:35.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200513+0000", 2024-09-14T22:38:35.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012050200000000001 2024-09-14T22:38:35.939 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.939 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.939 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.939 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97212 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 934912~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:35.940 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:38:35.940 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 85.807714386000001, 2024-09-14T22:38:35.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12621127300000001, 2024-09-14T22:38:35.940 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.940 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.940 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.940 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.940 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.940 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.940 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97212 2024-09-14T22:38:35.940 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.940 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.940 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.941 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:38:35.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.941 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.941 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:38:35.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.941 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.941 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074590+0000", 2024-09-14T22:38:35.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1969999999999999e-06 2024-09-14T22:38:35.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.942 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.942 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.942 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074596+0000", 2024-09-14T22:38:35.942 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7629999999999997e-06 2024-09-14T22:38:35.942 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.942 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.942 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.942 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074614+0000", 2024-09-14T22:38:35.942 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7643e-05 2024-09-14T22:38:35.942 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.942 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.942 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.942 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074635+0000", 2024-09-14T22:38:35.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.075e-05 2024-09-14T22:38:35.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200580+0000", 2024-09-14T22:38:35.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125945482 2024-09-14T22:38:35.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200683+0000", 2024-09-14T22:38:35.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010267099999999999 2024-09-14T22:38:35.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200800+0000", 2024-09-14T22:38:35.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117767 2024-09-14T22:38:35.944 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.944 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.944 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.944 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97213 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 944128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:35.944 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:38:35.944 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 85.807649036000001, 2024-09-14T22:38:35.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126430709, 2024-09-14T22:38:35.944 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.944 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.944 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.944 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.945 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.945 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.945 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97213 2024-09-14T22:38:35.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.945 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:38:35.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:38:35.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.946 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.946 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078210+0000", 2024-09-14T22:38:35.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0035556839999999999 2024-09-14T22:38:35.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.946 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.946 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078231+0000", 2024-09-14T22:38:35.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0716e-05 2024-09-14T22:38:35.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.946 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.946 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078260+0000", 2024-09-14T22:38:35.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9473000000000001e-05 2024-09-14T22:38:35.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078279+0000", 2024-09-14T22:38:35.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8600000000000001e-05 2024-09-14T22:38:35.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200864+0000", 2024-09-14T22:38:35.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.122585295 2024-09-14T22:38:35.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200965+0000", 2024-09-14T22:38:35.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101004 2024-09-14T22:38:35.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.201085+0000", 2024-09-14T22:38:35.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119937 2024-09-14T22:38:35.948 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.948 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.948 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.948 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97219 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 999424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:35.948 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:38:35.948 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 85.803666758999995, 2024-09-14T22:38:35.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124138558, 2024-09-14T22:38:35.949 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.949 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.949 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.949 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.949 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.949 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.949 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97219 2024-09-14T22:38:35.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.949 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:38:35.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.950 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.950 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:38:35.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.950 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.950 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078638+0000", 2024-09-14T22:38:35.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.173e-06 2024-09-14T22:38:35.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.950 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.950 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078644+0000", 2024-09-14T22:38:35.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2570000000000004e-06 2024-09-14T22:38:35.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078672+0000", 2024-09-14T22:38:35.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7322999999999999e-05 2024-09-14T22:38:35.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078687+0000", 2024-09-14T22:38:35.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5639999999999999e-05 2024-09-14T22:38:35.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202567+0000", 2024-09-14T22:38:35.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123880134 2024-09-14T22:38:35.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202670+0000", 2024-09-14T22:38:35.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010273899999999999 2024-09-14T22:38:35.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202775+0000", 2024-09-14T22:38:35.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010529199999999999 2024-09-14T22:38:35.952 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.952 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.952 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.953 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97220 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1008640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:35.953 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:38:35.953 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 85.803597526000004, 2024-09-14T22:38:35.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12435937, 2024-09-14T22:38:35.953 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.953 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.953 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.953 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.953 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.953 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.953 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97220 2024-09-14T22:38:35.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.953 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:38:35.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:38:35.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078707+0000", 2024-09-14T22:38:35.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.266e-06 2024-09-14T22:38:35.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078713+0000", 2024-09-14T22:38:35.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7270000000000004e-06 2024-09-14T22:38:35.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078737+0000", 2024-09-14T22:38:35.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3640000000000001e-05 2024-09-14T22:38:35.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078751+0000", 2024-09-14T22:38:35.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.45e-05 2024-09-14T22:38:35.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202841+0000", 2024-09-14T22:38:35.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124089938 2024-09-14T22:38:35.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202944+0000", 2024-09-14T22:38:35.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010334 2024-09-14T22:38:35.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203065+0000", 2024-09-14T22:38:35.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012095900000000001 2024-09-14T22:38:35.956 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.956 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.956 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.957 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97221 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1017856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:35.957 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:38:35.957 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 85.803531762999995, 2024-09-14T22:38:35.957 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124574005, 2024-09-14T22:38:35.957 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.957 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.957 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.957 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.957 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.957 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.957 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97221 2024-09-14T22:38:35.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.957 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.958 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.958 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.958 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:38:35.958 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.958 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.958 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.958 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:38:35.958 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.958 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.958 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.958 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078773+0000", 2024-09-14T22:38:35.958 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.13e-06 2024-09-14T22:38:35.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.959 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.959 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.959 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078779+0000", 2024-09-14T22:38:35.959 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0260000000000004e-06 2024-09-14T22:38:35.959 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.959 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.959 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.959 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078797+0000", 2024-09-14T22:38:35.959 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.806e-05 2024-09-14T22:38:35.959 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.959 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.959 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.959 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078810+0000", 2024-09-14T22:38:35.959 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.308e-05 2024-09-14T22:38:35.959 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203131+0000", 2024-09-14T22:38:35.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12432077499999999 2024-09-14T22:38:35.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203227+0000", 2024-09-14T22:38:35.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6575000000000003e-05 2024-09-14T22:38:35.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203346+0000", 2024-09-14T22:38:35.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118359 2024-09-14T22:38:35.961 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.961 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.961 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.961 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97222 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:35.961 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:38:35.961 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 85.803475394000003, 2024-09-14T22:38:35.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12480110799999999, 2024-09-14T22:38:35.961 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.961 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.961 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.961 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.962 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.962 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.962 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97222 2024-09-14T22:38:35.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.962 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:38:35.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:38:35.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.963 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.963 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.963 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078829+0000", 2024-09-14T22:38:35.963 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.057e-06 2024-09-14T22:38:35.963 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.963 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.963 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.963 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078836+0000", 2024-09-14T22:38:35.963 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6159999999999998e-06 2024-09-14T22:38:35.963 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.963 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.963 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.963 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078856+0000", 2024-09-14T22:38:35.963 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0100000000000001e-05 2024-09-14T22:38:35.963 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.964 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078869+0000", 2024-09-14T22:38:35.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.272e-05 2024-09-14T22:38:35.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.964 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203412+0000", 2024-09-14T22:38:35.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12454283300000001 2024-09-14T22:38:35.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.964 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203511+0000", 2024-09-14T22:38:35.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9893000000000001e-05 2024-09-14T22:38:35.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203629+0000", 2024-09-14T22:38:35.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117889 2024-09-14T22:38:35.965 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.965 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.965 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.965 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97223 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1036288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:35.965 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:38:35.965 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 85.803103406999995, 2024-09-14T22:38:35.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124706279, 2024-09-14T22:38:35.966 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.966 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.966 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.966 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.966 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.966 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.966 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97223 2024-09-14T22:38:35.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.966 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:38:35.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.967 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.967 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:38:35.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.967 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.967 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079202+0000", 2024-09-14T22:38:35.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8330000000000001e-06 2024-09-14T22:38:35.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.967 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.967 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079212+0000", 2024-09-14T22:38:35.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0207000000000001e-05 2024-09-14T22:38:35.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.968 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.968 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079238+0000", 2024-09-14T22:38:35.968 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5626000000000002e-05 2024-09-14T22:38:35.968 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.968 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.968 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.968 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079256+0000", 2024-09-14T22:38:35.968 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8386999999999999e-05 2024-09-14T22:38:35.968 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.968 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.968 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.968 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203691+0000", 2024-09-14T22:38:35.968 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124434538 2024-09-14T22:38:35.968 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.968 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203786+0000", 2024-09-14T22:38:35.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5344000000000001e-05 2024-09-14T22:38:35.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203906+0000", 2024-09-14T22:38:35.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120344 2024-09-14T22:38:35.969 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.969 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.969 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.969 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97224 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1045504~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:35.970 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:38:35.970 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 85.803019586999994, 2024-09-14T22:38:35.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12513321199999999, 2024-09-14T22:38:35.970 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:35.970 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:35.970 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:35.970 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:35.970 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:35.970 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:35.970 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97224 2024-09-14T22:38:35.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.970 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:35.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:35.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:38:35.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:35.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:38:35.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:35.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:35.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082581+0000", 2024-09-14T22:38:35.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0032971530000000001 2024-09-14T22:38:35.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:35.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082592+0000", 2024-09-14T22:38:35.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1345999999999999e-05 2024-09-14T22:38:35.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:35.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082618+0000", 2024-09-14T22:38:35.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5873000000000001e-05 2024-09-14T22:38:35.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:35.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082638+0000", 2024-09-14T22:38:35.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9565999999999999e-05 2024-09-14T22:38:35.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:35.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204004+0000", 2024-09-14T22:38:35.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.121366038 2024-09-14T22:38:35.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:35.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204188+0000", 2024-09-14T22:38:35.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018427 2024-09-14T22:38:35.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:35.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:35.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:35.974 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204417+0000", 2024-09-14T22:38:35.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022896599999999999 2024-09-14T22:38:35.974 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.974 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.974 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.974 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:35.974 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:35.974 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:35.975 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:38:35.975 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:38:35.976 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:38:35.976 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:38:35.977 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:38:35.977 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:35.970+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:35.977 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:38:35.977 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:38:35.978 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:38:35.978 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:38:35.978 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:38:35.979 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:38:35.983 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:38:35.984 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:38:36.002 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:38:36.003 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:38:36.021 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:38:36.021 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:38:36.086 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:36.085+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:36.148 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:36.148 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:36.162 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:38:36.177 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:36.177+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:36.278 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:36.270+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:36.284 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:36.285 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:36.299 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:38:36.380 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:36.377+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:36.465 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:36.465 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:36.479 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:38:36.480 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:36.474+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:36.582 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:36.581+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:36.597 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:36.597 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:36.611 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:38:36.682 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:36.674+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:36.735 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:36.735 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:36.749 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:38:36.783 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:36.781+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:36.884 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:36.878+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:36.920 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:36.920 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:36.935 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:38:36.984 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:36.978+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:37.080 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:37.080 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:37.086 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:37.078+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:37.095 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:38:37.187 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:37.182+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:37.219 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:37.219 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:37.234 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:38:37.287 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:37.282+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:37.361 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:37.361 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:37.376 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:38:37.422 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:37.421+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:37.489 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:37.482+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:37.495 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:37.496 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:37.510 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:38:37.602 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:37.601+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:37.687 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:37.687 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:37.690 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:37.686+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:37.701 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:38:37.752 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T22:38:37.791 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:37.789+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:37.884 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:37.885 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:37.891 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:37.889+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:37.901 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:38:37.992 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:37.986+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:38.040 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:38:38.067 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:38.068 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:38.085 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:38:38.094 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:38.086+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:38.182 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:38:38.183 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:38:38.194 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:38.186+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:38.199 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:38:38.200 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:38:38.216 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:38:38.216 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:38:38.234 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:38:38.234 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:38:38.246 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:38.247 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:38.247 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:38:38.247 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:38:38.247 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:38.255 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:38:38.255 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:38:38.261 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:38:38.276 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:38:38.276 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:38:38.295 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:38.293+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:38.298 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:38:38.298 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:38:38.320 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:38:38.320 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:38:38.388 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:38.389 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:38:38.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:38:38.389 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:38:38.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.389 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:31370 3.0 3:02f48f58:::rbd_data.113cda0183a0.0000000000000000:6 [read 813056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:38:38.389 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:38:38.389 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 125.024329495, 2024-09-14T22:38:38.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047519796000000003, 2024-09-14T22:38:38.389 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:38.389 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:38.389 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:38.389 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:38.389 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:38.390 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:38.390 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 31370 2024-09-14T22:38:38.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.390 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:38.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:38.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:38:38.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:38.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:38:38.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:38.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:38:38.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0999999999999999e-07 2024-09-14T22:38:38.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:38.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362401+0000", 2024-09-14T22:38:38.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2900000000000001e-06 2024-09-14T22:38:38.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:38.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362406+0000", 2024-09-14T22:38:38.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6960000000000002e-06 2024-09-14T22:38:38.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:38.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362412+0000", 2024-09-14T22:38:38.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.823e-06 2024-09-14T22:38:38.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:38.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.400270+0000", 2024-09-14T22:38:38.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037857737000000002 2024-09-14T22:38:38.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:38.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.400342+0000", 2024-09-14T22:38:38.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1797000000000004e-05 2024-09-14T22:38:38.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:38.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.409918+0000", 2024-09-14T22:38:38.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0095760429999999994 2024-09-14T22:38:38.393 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.393 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:38.393 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.394 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:113947 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 5603328~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:38.394 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.923453+0000", 2024-09-14T22:38:38.394 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.463274362999996, 2024-09-14T22:38:38.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047310420999999998, 2024-09-14T22:38:38.394 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:38.394 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:38.394 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:38.394 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:38.394 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:38.394 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:38.394 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 113947 2024-09-14T22:38:38.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.395 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:38.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:38.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.923453+0000", 2024-09-14T22:38:38.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:38.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.923453+0000", 2024-09-14T22:38:38.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:38.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926004+0000", 2024-09-14T22:38:38.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002551382 2024-09-14T22:38:38.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:38.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926009+0000", 2024-09-14T22:38:38.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5639999999999998e-06 2024-09-14T22:38:38.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:38.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926022+0000", 2024-09-14T22:38:38.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2734999999999999e-05 2024-09-14T22:38:38.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:38.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926029+0000", 2024-09-14T22:38:38.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2520000000000002e-06 2024-09-14T22:38:38.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.397 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:38.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.970650+0000", 2024-09-14T22:38:38.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044621277000000001 2024-09-14T22:38:38.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.397 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:38.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.970700+0000", 2024-09-14T22:38:38.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9713000000000003e-05 2024-09-14T22:38:38.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:38.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.970763+0000", 2024-09-14T22:38:38.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3497999999999994e-05 2024-09-14T22:38:38.398 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.398 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:38.398 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.398 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114136 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3448832~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:38.398 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932179+0000", 2024-09-14T22:38:38.398 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.454547997000006, 2024-09-14T22:38:38.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047236017999999998, 2024-09-14T22:38:38.398 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:38.398 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:38.399 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:38.399 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:38.399 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:38.399 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:38.399 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114136 2024-09-14T22:38:38.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.399 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:38.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:38.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932179+0000", 2024-09-14T22:38:38.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:38.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932179+0000", 2024-09-14T22:38:38.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:38.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932324+0000", 2024-09-14T22:38:38.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014451000000000001 2024-09-14T22:38:38.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:38.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932328+0000", 2024-09-14T22:38:38.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5889999999999996e-06 2024-09-14T22:38:38.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:38.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932338+0000", 2024-09-14T22:38:38.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3000000000000007e-06 2024-09-14T22:38:38.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:38.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932344+0000", 2024-09-14T22:38:38.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7150000000000001e-06 2024-09-14T22:38:38.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:38.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979233+0000", 2024-09-14T22:38:38.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046888711 2024-09-14T22:38:38.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:38.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979313+0000", 2024-09-14T22:38:38.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9487000000000007e-05 2024-09-14T22:38:38.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:38.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979415+0000", 2024-09-14T22:38:38.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000102706 2024-09-14T22:38:38.402 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.402 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:38.402 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.402 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114138 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3497984~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:38.403 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932356+0000", 2024-09-14T22:38:38.403 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.454371111, 2024-09-14T22:38:38.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047285958000000003, 2024-09-14T22:38:38.403 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:38.403 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:38.403 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:38.403 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:38.403 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:38.403 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:38.403 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114138 2024-09-14T22:38:38.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.403 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:38.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:38.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932356+0000", 2024-09-14T22:38:38.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:38.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932356+0000", 2024-09-14T22:38:38.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:38.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932498+0000", 2024-09-14T22:38:38.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000142182 2024-09-14T22:38:38.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:38.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932503+0000", 2024-09-14T22:38:38.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.301e-06 2024-09-14T22:38:38.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:38.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932514+0000", 2024-09-14T22:38:38.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1484000000000001e-05 2024-09-14T22:38:38.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:38.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932521+0000", 2024-09-14T22:38:38.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0740000000000004e-06 2024-09-14T22:38:38.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:38.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979451+0000", 2024-09-14T22:38:38.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046929379 2024-09-14T22:38:38.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:38.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979535+0000", 2024-09-14T22:38:38.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4423000000000001e-05 2024-09-14T22:38:38.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:38.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979642+0000", 2024-09-14T22:38:38.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010711499999999999 2024-09-14T22:38:38.406 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.406 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:38.407 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.407 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114139 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3547136~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:38.407 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932554+0000", 2024-09-14T22:38:38.407 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.454173343999997, 2024-09-14T22:38:38.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047292230999999997, 2024-09-14T22:38:38.407 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:38.407 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:38.407 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:38.407 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:38.407 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:38.408 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:38.408 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114139 2024-09-14T22:38:38.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.408 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:38.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:38.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932554+0000", 2024-09-14T22:38:38.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:38.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932554+0000", 2024-09-14T22:38:38.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:38.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932819+0000", 2024-09-14T22:38:38.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026538400000000002 2024-09-14T22:38:38.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:38.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932825+0000", 2024-09-14T22:38:38.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3469999999999998e-06 2024-09-14T22:38:38.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:38.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932836+0000", 2024-09-14T22:38:38.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1e-05 2024-09-14T22:38:38.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:38.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932843+0000", 2024-09-14T22:38:38.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1790000000000002e-06 2024-09-14T22:38:38.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:38.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979673+0000", 2024-09-14T22:38:38.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046830389 2024-09-14T22:38:38.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:38.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979750+0000", 2024-09-14T22:38:38.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6312999999999999e-05 2024-09-14T22:38:38.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:38.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979846+0000", 2024-09-14T22:38:38.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6619000000000004e-05 2024-09-14T22:38:38.411 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.411 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:38.411 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.412 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114141 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3596288~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:38.412 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932855+0000", 2024-09-14T22:38:38.412 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.453872746000002, 2024-09-14T22:38:38.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047201870999999999, 2024-09-14T22:38:38.412 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:38.412 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:38.412 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:38.412 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:38.412 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:38.412 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:38.412 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114141 2024-09-14T22:38:38.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.413 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:38.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:38.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932855+0000", 2024-09-14T22:38:38.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:38.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932855+0000", 2024-09-14T22:38:38.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:38.414 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934033+0000", 2024-09-14T22:38:38.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001178264 2024-09-14T22:38:38.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:38.414 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934038+0000", 2024-09-14T22:38:38.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4820000000000002e-06 2024-09-14T22:38:38.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:38.414 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934048+0000", 2024-09-14T22:38:38.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9890000000000004e-06 2024-09-14T22:38:38.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:38.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934055+0000", 2024-09-14T22:38:38.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8329999999999996e-06 2024-09-14T22:38:38.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:38.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979878+0000", 2024-09-14T22:38:38.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045822773999999997 2024-09-14T22:38:38.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:38.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979958+0000", 2024-09-14T22:38:38.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0477999999999998e-05 2024-09-14T22:38:38.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:38.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.980056+0000", 2024-09-14T22:38:38.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8051e-05 2024-09-14T22:38:38.416 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.416 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:38.416 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.416 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114159 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3940352~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:38.416 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.934179+0000", 2024-09-14T22:38:38.416 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.452548710000002, 2024-09-14T22:38:38.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047341439999999999, 2024-09-14T22:38:38.416 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:38.416 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:38.417 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:38.417 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:38.417 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:38.417 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:38.417 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114159 2024-09-14T22:38:38.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.417 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:38.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:38.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934179+0000", 2024-09-14T22:38:38.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:38.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934179+0000", 2024-09-14T22:38:38.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:38.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935021+0000", 2024-09-14T22:38:38.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00084276800000000003 2024-09-14T22:38:38.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:38.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935027+0000", 2024-09-14T22:38:38.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2959999999999998e-06 2024-09-14T22:38:38.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:38.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935038+0000", 2024-09-14T22:38:38.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1198e-05 2024-09-14T22:38:38.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:38.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935054+0000", 2024-09-14T22:38:38.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5764e-05 2024-09-14T22:38:38.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:38.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.981341+0000", 2024-09-14T22:38:38.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.04628782 2024-09-14T22:38:38.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:38.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.981415+0000", 2024-09-14T22:38:38.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3349999999999994e-05 2024-09-14T22:38:38.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:38.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.981520+0000", 2024-09-14T22:38:38.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105244 2024-09-14T22:38:38.420 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.420 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:38.420 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.421 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114180 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4136960~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:38.421 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:38:38.421 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.451549173999993, 2024-09-14T22:38:38.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047190621000000002, 2024-09-14T22:38:38.421 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:38.421 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:38.421 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:38.421 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:38.421 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:38.421 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:38.421 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114180 2024-09-14T22:38:38.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.421 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:38.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:38.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:38:38.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:38.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:38:38.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:38.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:38:38.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6800000000000002e-07 2024-09-14T22:38:38.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:38.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935180+0000", 2024-09-14T22:38:38.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.42e-06 2024-09-14T22:38:38.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:38.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935185+0000", 2024-09-14T22:38:38.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1359999999999996e-06 2024-09-14T22:38:38.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:38.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935188+0000", 2024-09-14T22:38:38.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.711e-06 2024-09-14T22:38:38.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:38.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982201+0000", 2024-09-14T22:38:38.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047013309000000003 2024-09-14T22:38:38.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:38.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982278+0000", 2024-09-14T22:38:38.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6705000000000005e-05 2024-09-14T22:38:38.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:38.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982369+0000", 2024-09-14T22:38:38.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1071999999999998e-05 2024-09-14T22:38:38.425 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.425 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:38.425 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.425 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114182 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4186112~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:38.425 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:38:38.425 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.451520552000005, 2024-09-14T22:38:38.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047368990999999999, 2024-09-14T22:38:38.425 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:38.425 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:38.425 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:38.425 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:38.426 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:38.426 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:38.426 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114182 2024-09-14T22:38:38.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.426 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:38.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:38.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:38:38.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:38.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:38:38.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:38.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:38:38.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7099999999999998e-07 2024-09-14T22:38:38.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:38.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935209+0000", 2024-09-14T22:38:38.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0260000000000002e-06 2024-09-14T22:38:38.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:38.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935213+0000", 2024-09-14T22:38:38.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3429999999999998e-06 2024-09-14T22:38:38.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:38.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935217+0000", 2024-09-14T22:38:38.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2480000000000001e-06 2024-09-14T22:38:38.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:38.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982406+0000", 2024-09-14T22:38:38.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047188920000000002 2024-09-14T22:38:38.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:38.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982483+0000", 2024-09-14T22:38:38.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7008999999999994e-05 2024-09-14T22:38:38.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:38.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982576+0000", 2024-09-14T22:38:38.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3173999999999999e-05 2024-09-14T22:38:38.429 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.429 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:38.429 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.429 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114185 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4202496~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:38.429 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.935221+0000", 2024-09-14T22:38:38.429 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.451506386999995, 2024-09-14T22:38:38.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047601128999999999, 2024-09-14T22:38:38.430 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:38.430 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:38.430 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:38.430 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:38.430 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:38.430 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:38.430 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114185 2024-09-14T22:38:38.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.430 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:38.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:38.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935221+0000", 2024-09-14T22:38:38.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:38.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935221+0000", 2024-09-14T22:38:38.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:38.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935970+0000", 2024-09-14T22:38:38.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074930899999999998 2024-09-14T22:38:38.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:38.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935976+0000", 2024-09-14T22:38:38.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5439999999999998e-06 2024-09-14T22:38:38.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:38.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935988+0000", 2024-09-14T22:38:38.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1779e-05 2024-09-14T22:38:38.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:38.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935996+0000", 2024-09-14T22:38:38.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.106e-06 2024-09-14T22:38:38.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:38.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982607+0000", 2024-09-14T22:38:38.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046611504999999998 2024-09-14T22:38:38.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:38.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982686+0000", 2024-09-14T22:38:38.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8805000000000002e-05 2024-09-14T22:38:38.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:38.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982822+0000", 2024-09-14T22:38:38.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000136081 2024-09-14T22:38:38.433 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.433 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:38.433 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.434 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114189 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4300800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:38.434 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.936048+0000", 2024-09-14T22:38:38.434 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.450679254999997, 2024-09-14T22:38:38.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047231539000000003, 2024-09-14T22:38:38.434 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:38.434 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:38.434 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:38.434 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:38.434 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:38.434 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:38.434 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114189 2024-09-14T22:38:38.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.434 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:38.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:38.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936048+0000", 2024-09-14T22:38:38.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:38.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936048+0000", 2024-09-14T22:38:38.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:38.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936049+0000", 2024-09-14T22:38:38.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4099999999999999e-07 2024-09-14T22:38:38.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:38.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936052+0000", 2024-09-14T22:38:38.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0029999999999999e-06 2024-09-14T22:38:38.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:38.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936058+0000", 2024-09-14T22:38:38.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7880000000000001e-06 2024-09-14T22:38:38.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:38.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936064+0000", 2024-09-14T22:38:38.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2279999999999998e-06 2024-09-14T22:38:38.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:38.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983120+0000", 2024-09-14T22:38:38.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047056384999999999 2024-09-14T22:38:38.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:38.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983194+0000", 2024-09-14T22:38:38.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3811000000000002e-05 2024-09-14T22:38:38.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:38.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983280+0000", 2024-09-14T22:38:38.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5883000000000004e-05 2024-09-14T22:38:38.438 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.438 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:38.438 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.438 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114191 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4349952~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:38.438 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:38:38.438 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.450655514999994, 2024-09-14T22:38:38.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047365365, 2024-09-14T22:38:38.438 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:38.438 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:38.438 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:38.438 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:38.438 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:38.439 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:38.439 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114191 2024-09-14T22:38:38.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.439 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:38.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:38.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:38:38.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:38.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:38:38.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:38.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:38:38.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0499999999999999e-07 2024-09-14T22:38:38.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:38.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936075+0000", 2024-09-14T22:38:38.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.734e-06 2024-09-14T22:38:38.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:38.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936082+0000", 2024-09-14T22:38:38.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8360000000000003e-06 2024-09-14T22:38:38.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:38.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936087+0000", 2024-09-14T22:38:38.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3129999999999998e-06 2024-09-14T22:38:38.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:38.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983319+0000", 2024-09-14T22:38:38.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047232140999999998 2024-09-14T22:38:38.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:38.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983365+0000", 2024-09-14T22:38:38.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5776999999999999e-05 2024-09-14T22:38:38.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:38.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983437+0000", 2024-09-14T22:38:38.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2158999999999998e-05 2024-09-14T22:38:38.442 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.442 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:38.442 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.442 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114193 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4399104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:38.442 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.936095+0000", 2024-09-14T22:38:38.442 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.450632579000001, 2024-09-14T22:38:38.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047505915000000003, 2024-09-14T22:38:38.442 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:38.443 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:38.443 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:38.443 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:38.443 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:38.443 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:38.443 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114193 2024-09-14T22:38:38.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.443 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:38.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:38.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936095+0000", 2024-09-14T22:38:38.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:38.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936095+0000", 2024-09-14T22:38:38.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:38.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938021+0000", 2024-09-14T22:38:38.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001926178 2024-09-14T22:38:38.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:38.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938027+0000", 2024-09-14T22:38:38.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3110000000000002e-06 2024-09-14T22:38:38.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:38.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938041+0000", 2024-09-14T22:38:38.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3344e-05 2024-09-14T22:38:38.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:38.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938049+0000", 2024-09-14T22:38:38.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2279999999999998e-06 2024-09-14T22:38:38.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:38.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983468+0000", 2024-09-14T22:38:38.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045419494999999997 2024-09-14T22:38:38.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:38.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983524+0000", 2024-09-14T22:38:38.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5952000000000002e-05 2024-09-14T22:38:38.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:38.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983601+0000", 2024-09-14T22:38:38.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6407000000000002e-05 2024-09-14T22:38:38.446 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.446 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:38.446 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.446 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114710 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4857856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:38.447 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.962571+0000", 2024-09-14T22:38:38.447 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.424156307000004, 2024-09-14T22:38:38.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047253314999999997, 2024-09-14T22:38:38.447 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:38.447 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:38.447 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:38.447 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:38.447 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:38.447 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:38.447 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114710 2024-09-14T22:38:38.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.447 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:38.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:38.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962571+0000", 2024-09-14T22:38:38.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:38.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962571+0000", 2024-09-14T22:38:38.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:38.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962572+0000", 2024-09-14T22:38:38.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1399999999999997e-07 2024-09-14T22:38:38.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:38.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962574+0000", 2024-09-14T22:38:38.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8679999999999999e-06 2024-09-14T22:38:38.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:38.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962583+0000", 2024-09-14T22:38:38.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8310000000000007e-06 2024-09-14T22:38:38.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:38.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962589+0000", 2024-09-14T22:38:38.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9529999999999996e-06 2024-09-14T22:38:38.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:38.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009692+0000", 2024-09-14T22:38:38.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047103249999999999 2024-09-14T22:38:38.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.450 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:38.450 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009767+0000", 2024-09-14T22:38:38.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4672000000000006e-05 2024-09-14T22:38:38.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.450 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:38.450 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009824+0000", 2024-09-14T22:38:38.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7226999999999999e-05 2024-09-14T22:38:38.450 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.450 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:38.450 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.451 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114712 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4907008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:38.451 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:38:38.451 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.424130826999999, 2024-09-14T22:38:38.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047421087000000001, 2024-09-14T22:38:38.451 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:38.451 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:38.451 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:38.451 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:38.451 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:38.451 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:38.451 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114712 2024-09-14T22:38:38.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.451 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:38.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:38.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:38:38.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:38.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:38:38.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:38.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:38:38.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.03e-07 2024-09-14T22:38:38.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:38.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962599+0000", 2024-09-14T22:38:38.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3199999999999998e-06 2024-09-14T22:38:38.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:38.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962606+0000", 2024-09-14T22:38:38.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5289999999999997e-06 2024-09-14T22:38:38.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:38.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962612+0000", 2024-09-14T22:38:38.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.947e-06 2024-09-14T22:38:38.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:38.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009866+0000", 2024-09-14T22:38:38.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047254681999999999 2024-09-14T22:38:38.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:38.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009924+0000", 2024-09-14T22:38:38.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7556000000000003e-05 2024-09-14T22:38:38.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:38.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010018+0000", 2024-09-14T22:38:38.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.365e-05 2024-09-14T22:38:38.455 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.455 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:38.455 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.455 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114714 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4956160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:38.455 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.962618+0000", 2024-09-14T22:38:38.455 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.424109309000002, 2024-09-14T22:38:38.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.04759302, 2024-09-14T22:38:38.455 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:38.455 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:38.455 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:38.455 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:38.455 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:38.455 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:38.455 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114714 2024-09-14T22:38:38.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.456 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:38.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:38.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962618+0000", 2024-09-14T22:38:38.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:38.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962618+0000", 2024-09-14T22:38:38.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:38.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966011+0000", 2024-09-14T22:38:38.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003393067 2024-09-14T22:38:38.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:38.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966016+0000", 2024-09-14T22:38:38.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3580000000000002e-06 2024-09-14T22:38:38.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:38.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966032+0000", 2024-09-14T22:38:38.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5356e-05 2024-09-14T22:38:38.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:38.458 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966040+0000", 2024-09-14T22:38:38.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1980000000000001e-06 2024-09-14T22:38:38.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:38.458 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010055+0000", 2024-09-14T22:38:38.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044014676000000003 2024-09-14T22:38:38.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:38.458 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010127+0000", 2024-09-14T22:38:38.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2206000000000006e-05 2024-09-14T22:38:38.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:38.459 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010211+0000", 2024-09-14T22:38:38.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4159000000000005e-05 2024-09-14T22:38:38.459 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.459 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:38.459 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.459 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114754 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 6283264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:38.459 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:38:38.459 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.419934396000002, 2024-09-14T22:38:38.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047228150000000003, 2024-09-14T22:38:38.459 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:38.459 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:38.460 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:38.460 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:38.460 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:38.460 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:38.460 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114754 2024-09-14T22:38:38.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.460 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:38.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.460 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:38.460 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:38:38.460 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:38.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:38:38.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:38.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:38:38.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7399999999999999e-07 2024-09-14T22:38:38.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:38.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966796+0000", 2024-09-14T22:38:38.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3630000000000001e-06 2024-09-14T22:38:38.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:38.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966802+0000", 2024-09-14T22:38:38.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4679999999999997e-06 2024-09-14T22:38:38.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:38.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966807+0000", 2024-09-14T22:38:38.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1109999999999999e-06 2024-09-14T22:38:38.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:38.463 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.013929+0000", 2024-09-14T22:38:38.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047121720999999998 2024-09-14T22:38:38.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.463 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:38.463 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.013968+0000", 2024-09-14T22:38:38.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8877000000000001e-05 2024-09-14T22:38:38.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.463 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:38.463 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014021+0000", 2024-09-14T22:38:38.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3236000000000003e-05 2024-09-14T22:38:38.463 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.464 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:38.464 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.464 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114789 3.19 3:9836bb25:::rbd_object_map.113c14c67864.000000000000001b:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:38.464 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.966813+0000", 2024-09-14T22:38:38.464 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.419913933999993, 2024-09-14T22:38:38.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05001311, 2024-09-14T22:38:38.464 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:38.464 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:38.464 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:38.464 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:38.464 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:38.464 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:38.464 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114789 2024-09-14T22:38:38.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.465 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:38.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.465 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:38.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966813+0000", 2024-09-14T22:38:38.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.465 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:38.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966813+0000", 2024-09-14T22:38:38.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.465 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:38.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966814+0000", 2024-09-14T22:38:38.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9200000000000002e-07 2024-09-14T22:38:38.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:38.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966817+0000", 2024-09-14T22:38:38.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8380000000000002e-06 2024-09-14T22:38:38.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:38.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966831+0000", 2024-09-14T22:38:38.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4209999999999999e-05 2024-09-14T22:38:38.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:38.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966845+0000", 2024-09-14T22:38:38.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3799e-05 2024-09-14T22:38:38.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:38.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014046+0000", 2024-09-14T22:38:38.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047201604000000001 2024-09-14T22:38:38.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:38.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014116+0000", 2024-09-14T22:38:38.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9387000000000005e-05 2024-09-14T22:38:38.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-14T22:38:38.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014469+0000", 2024-09-14T22:38:38.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00035355800000000002 2024-09-14T22:38:38.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:38.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016719+0000", 2024-09-14T22:38:38.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0022498459999999998 2024-09-14T22:38:38.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:38:38.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016735+0000", 2024-09-14T22:38:38.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5792999999999999e-05 2024-09-14T22:38:38.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:38:38.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016790+0000", 2024-09-14T22:38:38.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4715e-05 2024-09-14T22:38:38.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:38.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016827+0000", 2024-09-14T22:38:38.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6968000000000002e-05 2024-09-14T22:38:38.469 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.469 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:38.469 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.469 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114803 3.19 3:9836bb25:::rbd_object_map.113c14c67864.000000000000001b:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:38:38.469 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.966853+0000", 2024-09-14T22:38:38.470 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.419874504000006, 2024-09-14T22:38:38.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050656093999999999, 2024-09-14T22:38:38.470 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:38.470 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:38.470 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:38.470 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:38.470 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:38.470 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:38.470 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114803 2024-09-14T22:38:38.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.470 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:38.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:38.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966853+0000", 2024-09-14T22:38:38.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:38.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966853+0000", 2024-09-14T22:38:38.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:38.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969930+0000", 2024-09-14T22:38:38.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030769759999999999 2024-09-14T22:38:38.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:38.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969936+0000", 2024-09-14T22:38:38.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7150000000000003e-06 2024-09-14T22:38:38.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:38.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969949+0000", 2024-09-14T22:38:38.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3166000000000001e-05 2024-09-14T22:38:38.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:38.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969957+0000", 2024-09-14T22:38:38.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8269999999999998e-06 2024-09-14T22:38:38.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:38.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014996+0000", 2024-09-14T22:38:38.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045039202 2024-09-14T22:38:38.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:38.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.015085+0000", 2024-09-14T22:38:38.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8987999999999996e-05 2024-09-14T22:38:38.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-14T22:38:38.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.015318+0000", 2024-09-14T22:38:38.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023284999999999999 2024-09-14T22:38:38.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:38.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016898+0000", 2024-09-14T22:38:38.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0015807379999999999 2024-09-14T22:38:38.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:38:38.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.017455+0000", 2024-09-14T22:38:38.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00055664600000000005 2024-09-14T22:38:38.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:38:38.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.017483+0000", 2024-09-14T22:38:38.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8238e-05 2024-09-14T22:38:38.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:38.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.017509+0000", 2024-09-14T22:38:38.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5748000000000001e-05 2024-09-14T22:38:38.475 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.475 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:38.475 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.475 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:150759 3.8 3:15dc94e5:::rbd_header.113c88f88d4a:head [create,call rbd.create in=9b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e113)", 2024-09-14T22:38:38.475 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:38:05.726302+0000", 2024-09-14T22:38:38.475 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.660425812, 2024-09-14T22:38:38.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0241160819999999, 2024-09-14T22:38:38.476 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:38.476 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:38.476 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:38.476 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:38.476 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:38.476 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:38.476 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 150759 2024-09-14T22:38:38.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.476 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:38.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:38.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.726302+0000", 2024-09-14T22:38:38.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:38.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.726302+0000", 2024-09-14T22:38:38.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:38.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.726304+0000", 2024-09-14T22:38:38.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.075e-06 2024-09-14T22:38:38.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:38.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.726312+0000", 2024-09-14T22:38:38.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2050000000000002e-06 2024-09-14T22:38:38.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:38.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.727969+0000", 2024-09-14T22:38:38.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001657499 2024-09-14T22:38:38.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:38.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.727985+0000", 2024-09-14T22:38:38.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6082999999999998e-05 2024-09-14T22:38:38.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:38.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.728193+0000", 2024-09-14T22:38:38.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000208082 2024-09-14T22:38:38.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:38.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.728205+0000", 2024-09-14T22:38:38.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1897e-05 2024-09-14T22:38:38.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:38.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.741926+0000", 2024-09-14T22:38:38.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.013720591 2024-09-14T22:38:38.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:38.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.742076+0000", 2024-09-14T22:38:38.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015024099999999999 2024-09-14T22:38:38.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-14T22:38:38.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.742415+0000", 2024-09-14T22:38:38.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00033887700000000002 2024-09-14T22:38:38.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:38:38.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.743436+0000", 2024-09-14T22:38:38.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010210639999999999 2024-09-14T22:38:38.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:38.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.750216+0000", 2024-09-14T22:38:38.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0067798950000000002 2024-09-14T22:38:38.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:38:38.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.750288+0000", 2024-09-14T22:38:38.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2192000000000003e-05 2024-09-14T22:38:38.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:38.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.750418+0000", 2024-09-14T22:38:38.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000129381 2024-09-14T22:38:38.481 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.481 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:38.481 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.481 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.481 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:38.481 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:38.483 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:38:38.483 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:38.478+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:38.584 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:38.581+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:38.604 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:38.604 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:38.604 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:38:38.604 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:38.618 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:38:38.710 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:38.709+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:38.785 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:38.785 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:38.785 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:38:38.785 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:38:38.785 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:38.786 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:38.778+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:38.800 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:38:38.828 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2024-09-14T22:38:38.887 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:38.882+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:38.958 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:38.958 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:38:38.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:38:38.958 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:38:38.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.958 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145249 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 15745024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:38.958 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.693850+0000", 2024-09-14T22:38:38.958 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.262367107999999, 2024-09-14T22:38:38.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.060850297999999997, 2024-09-14T22:38:38.959 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:38.959 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:38.959 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:38.959 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:38.959 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:38.959 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:38.959 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145249 2024-09-14T22:38:38.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.959 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:38.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:38.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693850+0000", 2024-09-14T22:38:38.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:38.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693850+0000", 2024-09-14T22:38:38.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:38.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693851+0000", 2024-09-14T22:38:38.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.46e-07 2024-09-14T22:38:38.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:38.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693853+0000", 2024-09-14T22:38:38.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0940000000000002e-06 2024-09-14T22:38:38.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:38.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693858+0000", 2024-09-14T22:38:38.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2100000000000001e-06 2024-09-14T22:38:38.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:38.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693863+0000", 2024-09-14T22:38:38.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5549999999999996e-06 2024-09-14T22:38:38.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:38.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754503+0000", 2024-09-14T22:38:38.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.060640558999999997 2024-09-14T22:38:38.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:38.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754613+0000", 2024-09-14T22:38:38.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109966 2024-09-14T22:38:38.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:38.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754701+0000", 2024-09-14T22:38:38.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7567999999999999e-05 2024-09-14T22:38:38.963 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.963 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:38.963 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.963 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145250 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16039936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:38.963 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.693868+0000", 2024-09-14T22:38:38.963 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.262349380000003, 2024-09-14T22:38:38.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061182667000000003, 2024-09-14T22:38:38.963 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:38.963 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:38.963 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:38.963 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:38.963 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:38.964 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:38.964 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145250 2024-09-14T22:38:38.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.964 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:38.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:38.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693868+0000", 2024-09-14T22:38:38.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:38.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693868+0000", 2024-09-14T22:38:38.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:38.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693869+0000", 2024-09-14T22:38:38.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4400000000000001e-07 2024-09-14T22:38:38.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:38.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693870+0000", 2024-09-14T22:38:38.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9130000000000001e-06 2024-09-14T22:38:38.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:38.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693876+0000", 2024-09-14T22:38:38.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1270000000000002e-06 2024-09-14T22:38:38.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:38.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693880+0000", 2024-09-14T22:38:38.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.673e-06 2024-09-14T22:38:38.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:38.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754780+0000", 2024-09-14T22:38:38.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.060899495999999997 2024-09-14T22:38:38.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:38.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754925+0000", 2024-09-14T22:38:38.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014511799999999999 2024-09-14T22:38:38.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:38.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755051+0000", 2024-09-14T22:38:38.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012599599999999999 2024-09-14T22:38:38.967 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.967 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:38.967 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.967 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145251 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16236544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:38.967 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.693886+0000", 2024-09-14T22:38:38.968 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.262331727000003, 2024-09-14T22:38:38.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061498816999999997, 2024-09-14T22:38:38.968 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:38.968 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:38.968 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:38.968 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:38.968 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:38.968 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:38.968 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145251 2024-09-14T22:38:38.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.968 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:38.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:38.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693886+0000", 2024-09-14T22:38:38.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:38.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693886+0000", 2024-09-14T22:38:38.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:38.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693886+0000", 2024-09-14T22:38:38.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1199999999999998e-07 2024-09-14T22:38:38.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:38.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693888+0000", 2024-09-14T22:38:38.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.877e-06 2024-09-14T22:38:38.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:38.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693894+0000", 2024-09-14T22:38:38.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5350000000000004e-06 2024-09-14T22:38:38.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:38.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693917+0000", 2024-09-14T22:38:38.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.296e-05 2024-09-14T22:38:38.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:38.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755127+0000", 2024-09-14T22:38:38.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061210286000000003 2024-09-14T22:38:38.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:38.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755266+0000", 2024-09-14T22:38:38.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013871599999999999 2024-09-14T22:38:38.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:38.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755385+0000", 2024-09-14T22:38:38.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011903100000000001 2024-09-14T22:38:38.972 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.972 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:38.972 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.973 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145252 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16302080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:38.973 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.693985+0000", 2024-09-14T22:38:38.973 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.262233045000002, 2024-09-14T22:38:38.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061743767999999997, 2024-09-14T22:38:38.973 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:38.973 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:38.973 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:38.973 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:38.973 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:38.973 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:38.973 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145252 2024-09-14T22:38:38.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.974 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:38.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:38.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693985+0000", 2024-09-14T22:38:38.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:38.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693985+0000", 2024-09-14T22:38:38.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:38.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693986+0000", 2024-09-14T22:38:38.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0640000000000001e-06 2024-09-14T22:38:38.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:38.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693991+0000", 2024-09-14T22:38:38.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7599999999999999e-06 2024-09-14T22:38:38.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:38.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694005+0000", 2024-09-14T22:38:38.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3271999999999999e-05 2024-09-14T22:38:38.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:38.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694015+0000", 2024-09-14T22:38:38.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0617e-05 2024-09-14T22:38:38.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:38.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755459+0000", 2024-09-14T22:38:38.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061443820000000003 2024-09-14T22:38:38.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:38.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755605+0000", 2024-09-14T22:38:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014569 2024-09-14T22:38:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755728+0000", 2024-09-14T22:38:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000123545 2024-09-14T22:38:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145253 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16138240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694031+0000", 2024-09-14T22:38:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.262186491000001, 2024-09-14T22:38:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062032836000000001, 2024-09-14T22:38:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145253 2024-09-14T22:38:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694031+0000", 2024-09-14T22:38:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:38.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694031+0000", 2024-09-14T22:38:38.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:38.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694032+0000", 2024-09-14T22:38:38.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7999999999999995e-07 2024-09-14T22:38:38.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:38.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694036+0000", 2024-09-14T22:38:38.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8800000000000001e-06 2024-09-14T22:38:38.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:38.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694047+0000", 2024-09-14T22:38:38.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0835999999999999e-05 2024-09-14T22:38:38.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:38.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694055+0000", 2024-09-14T22:38:38.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6619999999999994e-06 2024-09-14T22:38:38.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:38.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755804+0000", 2024-09-14T22:38:38.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061748448999999997 2024-09-14T22:38:38.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:38.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755941+0000", 2024-09-14T22:38:38.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013751599999999999 2024-09-14T22:38:38.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:38.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756064+0000", 2024-09-14T22:38:38.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012281299999999999 2024-09-14T22:38:38.981 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.982 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:38.982 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.982 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145254 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16400384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:38.982 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694067+0000", 2024-09-14T22:38:38.982 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.262150325999997, 2024-09-14T22:38:38.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062280549999999997, 2024-09-14T22:38:38.982 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:38.982 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:38.982 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:38.982 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:38.982 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:38.982 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:38.983 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145254 2024-09-14T22:38:38.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.983 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:38.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:38.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694067+0000", 2024-09-14T22:38:38.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:38.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694067+0000", 2024-09-14T22:38:38.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:38.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694068+0000", 2024-09-14T22:38:38.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2200000000000003e-07 2024-09-14T22:38:38.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:38.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694072+0000", 2024-09-14T22:38:38.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5729999999999999e-06 2024-09-14T22:38:38.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:38.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694082+0000", 2024-09-14T22:38:38.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0703999999999999e-05 2024-09-14T22:38:38.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:38.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694091+0000", 2024-09-14T22:38:38.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8030000000000003e-06 2024-09-14T22:38:38.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:38.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756155+0000", 2024-09-14T22:38:38.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062063937 2024-09-14T22:38:38.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756263+0000", 2024-09-14T22:38:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108229 2024-09-14T22:38:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756348+0000", 2024-09-14T22:38:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4581999999999998e-05 2024-09-14T22:38:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145255 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16596992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694103+0000", 2024-09-14T22:38:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.262114844999999, 2024-09-14T22:38:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062580620000000003, 2024-09-14T22:38:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145255 2024-09-14T22:38:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694103+0000", 2024-09-14T22:38:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694103+0000", 2024-09-14T22:38:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694103+0000", 2024-09-14T22:38:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7800000000000001e-07 2024-09-14T22:38:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694107+0000", 2024-09-14T22:38:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5939999999999998e-06 2024-09-14T22:38:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694121+0000", 2024-09-14T22:38:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3913999999999999e-05 2024-09-14T22:38:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694130+0000", 2024-09-14T22:38:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7690000000000002e-06 2024-09-14T22:38:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:38.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756431+0000", 2024-09-14T22:38:38.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062301593000000002 2024-09-14T22:38:38.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:38.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756564+0000", 2024-09-14T22:38:38.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000132693 2024-09-14T22:38:38.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:38.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756683+0000", 2024-09-14T22:38:38.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119479 2024-09-14T22:38:38.991 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.991 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:38.991 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.991 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145256 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16695296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:38.991 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694142+0000", 2024-09-14T22:38:38.991 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.262075772000003, 2024-09-14T22:38:38.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062806034999999996, 2024-09-14T22:38:38.991 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:38.991 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:38.991 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:38.992 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:38.992 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:38.992 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:38.992 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145256 2024-09-14T22:38:38.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.992 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:38.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694142+0000", 2024-09-14T22:38:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694142+0000", 2024-09-14T22:38:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694142+0000", 2024-09-14T22:38:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1699999999999998e-07 2024-09-14T22:38:38.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:38.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694146+0000", 2024-09-14T22:38:38.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7450000000000001e-06 2024-09-14T22:38:38.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:38.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694157+0000", 2024-09-14T22:38:38.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1161e-05 2024-09-14T22:38:38.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694166+0000", 2024-09-14T22:38:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0450000000000006e-06 2024-09-14T22:38:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756768+0000", 2024-09-14T22:38:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062601554000000004 2024-09-14T22:38:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756867+0000", 2024-09-14T22:38:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8741999999999998e-05 2024-09-14T22:38:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:38.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756948+0000", 2024-09-14T22:38:38.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1170999999999999e-05 2024-09-14T22:38:38.996 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.996 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:38.996 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:38.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.996 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145257 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16498688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:38.996 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694179+0000", 2024-09-14T22:38:38.996 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.262038472999997, 2024-09-14T22:38:38.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063118469999999996, 2024-09-14T22:38:38.996 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:38.997 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:38.997 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:38.997 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:38.997 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:38.997 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:38.997 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145257 2024-09-14T22:38:38.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.997 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:38.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:38.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694179+0000", 2024-09-14T22:38:38.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694179+0000", 2024-09-14T22:38:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694180+0000", 2024-09-14T22:38:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1099999999999995e-07 2024-09-14T22:38:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694196+0000", 2024-09-14T22:38:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5996000000000001e-05 2024-09-14T22:38:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694208+0000", 2024-09-14T22:38:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2584e-05 2024-09-14T22:38:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694219+0000", 2024-09-14T22:38:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0462000000000001e-05 2024-09-14T22:38:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:39.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757033+0000", 2024-09-14T22:38:39.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062814485000000003 2024-09-14T22:38:39.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:39.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757183+0000", 2024-09-14T22:38:39.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014914099999999999 2024-09-14T22:38:39.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:39.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757298+0000", 2024-09-14T22:38:39.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115091 2024-09-14T22:38:39.001 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:39.001 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:39.001 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:39.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.001 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145258 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16269312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:39.001 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694257+0000", 2024-09-14T22:38:39.001 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.261960279, 2024-09-14T22:38:39.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063297279999999997, 2024-09-14T22:38:39.001 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:39.001 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:39.001 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:39.001 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:39.001 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:39.002 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:39.002 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145258 2024-09-14T22:38:39.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.002 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:39.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.002 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:39.002 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694257+0000", 2024-09-14T22:38:39.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:39.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.002 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:39.002 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694257+0000", 2024-09-14T22:38:39.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:39.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:39.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694258+0000", 2024-09-14T22:38:39.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7700000000000004e-07 2024-09-14T22:38:39.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:39.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694263+0000", 2024-09-14T22:38:39.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6609999999999999e-06 2024-09-14T22:38:39.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:39.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694271+0000", 2024-09-14T22:38:39.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7809999999999994e-06 2024-09-14T22:38:39.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:39.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694277+0000", 2024-09-14T22:38:39.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.823e-06 2024-09-14T22:38:39.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:39.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757377+0000", 2024-09-14T22:38:39.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063100081000000002 2024-09-14T22:38:39.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:39.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757476+0000", 2024-09-14T22:38:39.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8507999999999999e-05 2024-09-14T22:38:39.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:39.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757555+0000", 2024-09-14T22:38:39.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8749000000000001e-05 2024-09-14T22:38:39.005 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:39.005 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:39.005 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:39.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.006 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145259 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16465920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:39.006 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694286+0000", 2024-09-14T22:38:39.006 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.261932082000001, 2024-09-14T22:38:39.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063608168000000007, 2024-09-14T22:38:39.006 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:39.006 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:39.006 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:39.006 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:39.006 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:39.006 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:39.006 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145259 2024-09-14T22:38:39.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.006 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:39.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694286+0000", 2024-09-14T22:38:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694286+0000", 2024-09-14T22:38:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694286+0000", 2024-09-14T22:38:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9999999999999998e-07 2024-09-14T22:38:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:39.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694288+0000", 2024-09-14T22:38:39.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3760000000000002e-06 2024-09-14T22:38:39.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:39.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694294+0000", 2024-09-14T22:38:39.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6779999999999997e-06 2024-09-14T22:38:39.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:39.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694298+0000", 2024-09-14T22:38:39.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3309999999999997e-06 2024-09-14T22:38:39.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:39.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757638+0000", 2024-09-14T22:38:39.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063339464999999998 2024-09-14T22:38:39.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:39.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757769+0000", 2024-09-14T22:38:39.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013148700000000001 2024-09-14T22:38:39.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:39.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757894+0000", 2024-09-14T22:38:39.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012443099999999999 2024-09-14T22:38:39.010 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:39.010 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:39.010 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:39.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.010 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145260 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16367616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:39.010 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694304+0000", 2024-09-14T22:38:39.010 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.261913288000002, 2024-09-14T22:38:39.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063838521999999995, 2024-09-14T22:38:39.011 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:39.011 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:39.011 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:39.011 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:39.011 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:39.011 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:39.011 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145260 2024-09-14T22:38:39.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.011 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:39.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:39.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694304+0000", 2024-09-14T22:38:39.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:39.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:39.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694304+0000", 2024-09-14T22:38:39.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:39.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:39.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694305+0000", 2024-09-14T22:38:39.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8299999999999997e-07 2024-09-14T22:38:39.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:39.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694307+0000", 2024-09-14T22:38:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9989999999999998e-06 2024-09-14T22:38:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694312+0000", 2024-09-14T22:38:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2689999999999999e-06 2024-09-14T22:38:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694317+0000", 2024-09-14T22:38:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9359999999999998e-06 2024-09-14T22:38:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757944+0000", 2024-09-14T22:38:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063626951000000001 2024-09-14T22:38:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758065+0000", 2024-09-14T22:38:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120945 2024-09-14T22:38:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758143+0000", 2024-09-14T22:38:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7838999999999998e-05 2024-09-14T22:38:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:39.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.015 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145261 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16564224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:39.015 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694323+0000", 2024-09-14T22:38:39.015 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.261894585, 2024-09-14T22:38:39.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064150180000000001, 2024-09-14T22:38:39.015 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:39.015 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:39.015 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:39.015 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:39.015 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:39.015 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:39.015 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145261 2024-09-14T22:38:39.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694323+0000", 2024-09-14T22:38:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694323+0000", 2024-09-14T22:38:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694323+0000", 2024-09-14T22:38:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4900000000000001e-07 2024-09-14T22:38:39.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:39.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694325+0000", 2024-09-14T22:38:39.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.04e-06 2024-09-14T22:38:39.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:39.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694331+0000", 2024-09-14T22:38:39.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3449999999999996e-06 2024-09-14T22:38:39.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694335+0000", 2024-09-14T22:38:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.566e-06 2024-09-14T22:38:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758225+0000", 2024-09-14T22:38:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063889535999999997 2024-09-14T22:38:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758357+0000", 2024-09-14T22:38:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013181 2024-09-14T22:38:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758473+0000", 2024-09-14T22:38:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011653400000000001 2024-09-14T22:38:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145262 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16760832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694341+0000", 2024-09-14T22:38:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.261876727000001, 2024-09-14T22:38:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064424631999999996, 2024-09-14T22:38:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145262 2024-09-14T22:38:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694341+0000", 2024-09-14T22:38:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694341+0000", 2024-09-14T22:38:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694341+0000", 2024-09-14T22:38:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5499999999999999e-07 2024-09-14T22:38:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694343+0000", 2024-09-14T22:38:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9630000000000001e-06 2024-09-14T22:38:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694349+0000", 2024-09-14T22:38:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3120000000000001e-06 2024-09-14T22:38:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694353+0000", 2024-09-14T22:38:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6500000000000004e-06 2024-09-14T22:38:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758561+0000", 2024-09-14T22:38:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064207934999999994 2024-09-14T22:38:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758677+0000", 2024-09-14T22:38:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116241 2024-09-14T22:38:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758766+0000", 2024-09-14T22:38:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8176000000000004e-05 2024-09-14T22:38:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145263 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16662528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694359+0000", 2024-09-14T22:38:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.261858883999999, 2024-09-14T22:38:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064743189000000007, 2024-09-14T22:38:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145263 2024-09-14T22:38:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694359+0000", 2024-09-14T22:38:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:39.025 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694359+0000", 2024-09-14T22:38:39.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:39.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:39.025 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694359+0000", 2024-09-14T22:38:39.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5100000000000001e-07 2024-09-14T22:38:39.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:39.025 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694361+0000", 2024-09-14T22:38:39.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9429999999999999e-06 2024-09-14T22:38:39.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694366+0000", 2024-09-14T22:38:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3489999999999999e-06 2024-09-14T22:38:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694371+0000", 2024-09-14T22:38:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4399999999999998e-06 2024-09-14T22:38:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758848+0000", 2024-09-14T22:38:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064477396000000006 2024-09-14T22:38:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:39.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758980+0000", 2024-09-14T22:38:39.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013162199999999999 2024-09-14T22:38:39.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:39.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759102+0000", 2024-09-14T22:38:39.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012208799999999999 2024-09-14T22:38:39.027 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:39.027 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:39.027 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:39.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.027 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145264 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16334848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:39.027 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694377+0000", 2024-09-14T22:38:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.261840849999999, 2024-09-14T22:38:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065048054999999994, 2024-09-14T22:38:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145264 2024-09-14T22:38:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694377+0000", 2024-09-14T22:38:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694377+0000", 2024-09-14T22:38:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694377+0000", 2024-09-14T22:38:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.39e-07 2024-09-14T22:38:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694379+0000", 2024-09-14T22:38:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0190000000000001e-06 2024-09-14T22:38:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694385+0000", 2024-09-14T22:38:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4399999999999996e-06 2024-09-14T22:38:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694389+0000", 2024-09-14T22:38:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.639e-06 2024-09-14T22:38:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:39.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759175+0000", 2024-09-14T22:38:39.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064785899999999993 2024-09-14T22:38:39.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:39.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759309+0000", 2024-09-14T22:38:39.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013374400000000001 2024-09-14T22:38:39.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:39.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759425+0000", 2024-09-14T22:38:39.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115974 2024-09-14T22:38:39.031 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:39.032 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:39.032 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:39.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.032 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145265 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16531456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:39.032 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694395+0000", 2024-09-14T22:38:39.032 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.261822727999999, 2024-09-14T22:38:39.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065298325000000004, 2024-09-14T22:38:39.032 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:39.032 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:39.032 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145265 2024-09-14T22:38:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694395+0000", 2024-09-14T22:38:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694395+0000", 2024-09-14T22:38:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694395+0000", 2024-09-14T22:38:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6699999999999999e-07 2024-09-14T22:38:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694397+0000", 2024-09-14T22:38:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.984e-06 2024-09-14T22:38:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:39.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694403+0000", 2024-09-14T22:38:39.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3519999999999997e-06 2024-09-14T22:38:39.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:39.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694407+0000", 2024-09-14T22:38:39.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3239999999999996e-06 2024-09-14T22:38:39.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:39.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759516+0000", 2024-09-14T22:38:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065109279000000006 2024-09-14T22:38:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759610+0000", 2024-09-14T22:38:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3566000000000005e-05 2024-09-14T22:38:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759693+0000", 2024-09-14T22:38:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3453000000000002e-05 2024-09-14T22:38:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145266 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16433152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694413+0000", 2024-09-14T22:38:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.261804929, 2024-09-14T22:38:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065618308, 2024-09-14T22:38:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145266 2024-09-14T22:38:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694413+0000", 2024-09-14T22:38:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694413+0000", 2024-09-14T22:38:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694413+0000", 2024-09-14T22:38:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4799999999999999e-07 2024-09-14T22:38:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694415+0000", 2024-09-14T22:38:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.08e-06 2024-09-14T22:38:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694420+0000", 2024-09-14T22:38:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.203e-06 2024-09-14T22:38:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694425+0000", 2024-09-14T22:38:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4649999999999996e-06 2024-09-14T22:38:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759768+0000", 2024-09-14T22:38:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065343613999999994 2024-09-14T22:38:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759909+0000", 2024-09-14T22:38:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014054099999999999 2024-09-14T22:38:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.041 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:39.041 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760031+0000", 2024-09-14T22:38:39.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122057 2024-09-14T22:38:39.041 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:39.041 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:39.041 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:39.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.041 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145267 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16629760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:39.041 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694431+0000", 2024-09-14T22:38:39.041 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.261787015000003, 2024-09-14T22:38:39.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065850237000000006, 2024-09-14T22:38:39.041 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145267 2024-09-14T22:38:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694431+0000", 2024-09-14T22:38:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694431+0000", 2024-09-14T22:38:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694431+0000", 2024-09-14T22:38:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4799999999999999e-07 2024-09-14T22:38:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694433+0000", 2024-09-14T22:38:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8470000000000001e-06 2024-09-14T22:38:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694438+0000", 2024-09-14T22:38:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3249999999999998e-06 2024-09-14T22:38:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694442+0000", 2024-09-14T22:38:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3490000000000002e-06 2024-09-14T22:38:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760113+0000", 2024-09-14T22:38:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065670872000000005 2024-09-14T22:38:39.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.045 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:39.045 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760199+0000", 2024-09-14T22:38:39.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5982000000000004e-05 2024-09-14T22:38:39.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.045 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:39.045 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760281+0000", 2024-09-14T22:38:39.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1514000000000006e-05 2024-09-14T22:38:39.045 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:39.045 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:39.045 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:39.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.046 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145268 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16728064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:39.046 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694448+0000", 2024-09-14T22:38:39.046 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.261769430999998, 2024-09-14T22:38:39.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066157585000000005, 2024-09-14T22:38:39.046 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:39.046 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:39.046 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:39.046 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:39.046 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:39.046 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:39.046 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145268 2024-09-14T22:38:39.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.046 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:39.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694448+0000", 2024-09-14T22:38:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694448+0000", 2024-09-14T22:38:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694449+0000", 2024-09-14T22:38:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.39e-07 2024-09-14T22:38:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694450+0000", 2024-09-14T22:38:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.906e-06 2024-09-14T22:38:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694456+0000", 2024-09-14T22:38:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3820000000000003e-06 2024-09-14T22:38:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694460+0000", 2024-09-14T22:38:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6290000000000001e-06 2024-09-14T22:38:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:39.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760345+0000", 2024-09-14T22:38:39.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065884540000000005 2024-09-14T22:38:39.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:39.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760482+0000", 2024-09-14T22:38:39.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013731600000000001 2024-09-14T22:38:39.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:39.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:39.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:39.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760606+0000", 2024-09-14T22:38:39.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000123473 2024-09-14T22:38:39.049 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:39.050 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:39.050 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:39.050 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:39.050 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:39.050 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:39.051 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:38:39.052 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:39.049+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:39.152 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:39.149+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:39.182 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:39.182 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:39.182 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:38:39.182 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:39.196 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:38:39.252 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:39.246+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:39.318 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:39.318 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:39.318 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:38:39.318 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:38:39.318 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:39.332 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:38:39.354 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:39.353+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:39.455 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:39.450+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:39.503 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:39.503 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:38:39.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:38:39.503 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:38:39.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.503 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6395 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8386560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:38:39.503 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:38:39.503 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 135.56486255300001, 2024-09-14T22:38:39.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054534379000000001, 2024-09-14T22:38:39.503 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:39.504 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:39.504 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:39.504 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:39.504 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:39.504 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:39.504 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6395 2024-09-14T22:38:39.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.504 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:39.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:39.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:38:39.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:39.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:38:39.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:39.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933833+0000", 2024-09-14T22:38:39.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3859999999999999e-06 2024-09-14T22:38:39.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:39.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933838+0000", 2024-09-14T22:38:39.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8709999999999996e-06 2024-09-14T22:38:39.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.506 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:39.506 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933854+0000", 2024-09-14T22:38:39.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6189e-05 2024-09-14T22:38:39.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.506 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:39.506 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933871+0000", 2024-09-14T22:38:39.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.626e-05 2024-09-14T22:38:39.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.506 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:39.506 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988198+0000", 2024-09-14T22:38:39.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054327581999999999 2024-09-14T22:38:39.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.507 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:39.507 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988274+0000", 2024-09-14T22:38:39.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5759000000000004e-05 2024-09-14T22:38:39.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.507 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:39.507 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988366+0000", 2024-09-14T22:38:39.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2331999999999996e-05 2024-09-14T22:38:39.507 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.507 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:39.507 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.508 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104944 3.3 3:c3780f3b:::rbd_data.113ca561f7a9.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:39.508 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:38:39.508 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.46526068, 2024-09-14T22:38:39.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073041087000000005, 2024-09-14T22:38:39.508 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:39.508 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:39.508 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:39.508 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:39.508 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:39.508 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:39.508 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104944 2024-09-14T22:38:39.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.508 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:39.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:39.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:38:39.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:39.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:38:39.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:39.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033436+0000", 2024-09-14T22:38:39.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2060000000000001e-06 2024-09-14T22:38:39.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:39.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033448+0000", 2024-09-14T22:38:39.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.169e-05 2024-09-14T22:38:39.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:39.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033474+0000", 2024-09-14T22:38:39.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6400000000000001e-05 2024-09-14T22:38:39.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:39.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033489+0000", 2024-09-14T22:38:39.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5067e-05 2024-09-14T22:38:39.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.511 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:39.511 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033658+0000", 2024-09-14T22:38:39.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000168565 2024-09-14T22:38:39.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.511 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:39.511 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033793+0000", 2024-09-14T22:38:39.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013536900000000001 2024-09-14T22:38:39.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:38:39.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.047559+0000", 2024-09-14T22:38:39.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013766294 2024-09-14T22:38:39.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:38:39.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106157+0000", 2024-09-14T22:38:39.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058597110000000001 2024-09-14T22:38:39.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:39.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106290+0000", 2024-09-14T22:38:39.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000133244 2024-09-14T22:38:39.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:38:39.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106396+0000", 2024-09-14T22:38:39.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106246 2024-09-14T22:38:39.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:39.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106475+0000", 2024-09-14T22:38:39.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8895999999999998e-05 2024-09-14T22:38:39.513 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.513 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:39.513 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.514 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:111858 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 10403840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:39.514 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:38:39.514 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.674807559000001, 2024-09-14T22:38:39.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054398798999999998, 2024-09-14T22:38:39.514 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:39.514 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:39.514 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:39.514 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:39.514 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:39.514 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:39.514 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 111858 2024-09-14T22:38:39.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.514 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:39.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:39.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:38:39.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:39.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:38:39.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:39.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823888+0000", 2024-09-14T22:38:39.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0200000000000001e-07 2024-09-14T22:38:39.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:39.516 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823927+0000", 2024-09-14T22:38:39.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8940999999999999e-05 2024-09-14T22:38:39.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.516 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:39.516 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823940+0000", 2024-09-14T22:38:39.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3682e-05 2024-09-14T22:38:39.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.516 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:39.516 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823950+0000", 2024-09-14T22:38:39.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7110000000000007e-06 2024-09-14T22:38:39.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.516 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:39.517 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878202+0000", 2024-09-14T22:38:39.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054251969999999997 2024-09-14T22:38:39.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.517 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:39.517 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878249+0000", 2024-09-14T22:38:39.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6854999999999998e-05 2024-09-14T22:38:39.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.517 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:39.517 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878286+0000", 2024-09-14T22:38:39.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6838000000000002e-05 2024-09-14T22:38:39.517 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.517 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:39.517 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.518 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:111859 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 10452992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:39.518 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:38:39.518 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.674718440000007, 2024-09-14T22:38:39.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054435292000000003, 2024-09-14T22:38:39.518 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:39.518 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:39.518 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:39.518 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:39.518 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:39.518 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:39.518 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 111859 2024-09-14T22:38:39.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.518 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:39.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:39.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:38:39.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:39.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:38:39.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:39.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823977+0000", 2024-09-14T22:38:39.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3500000000000005e-07 2024-09-14T22:38:39.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:39.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823981+0000", 2024-09-14T22:38:39.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2050000000000004e-06 2024-09-14T22:38:39.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:39.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824007+0000", 2024-09-14T22:38:39.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.62e-05 2024-09-14T22:38:39.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:39.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824018+0000", 2024-09-14T22:38:39.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0389e-05 2024-09-14T22:38:39.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:39.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878312+0000", 2024-09-14T22:38:39.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054294754000000001 2024-09-14T22:38:39.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.521 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:39.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878350+0000", 2024-09-14T22:38:39.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7982000000000003e-05 2024-09-14T22:38:39.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.521 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:39.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878411+0000", 2024-09-14T22:38:39.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0927000000000001e-05 2024-09-14T22:38:39.521 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.521 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:39.521 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.522 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112030 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6316032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:39.522 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:38:39.522 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.674664097999994, 2024-09-14T22:38:39.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054597826000000002, 2024-09-14T22:38:39.522 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:39.522 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:39.522 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:39.522 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:39.522 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:39.522 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:39.522 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112030 2024-09-14T22:38:39.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.522 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:39.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:39.523 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:38:39.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:39.523 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:38:39.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:39.523 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824031+0000", 2024-09-14T22:38:39.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3600000000000003e-07 2024-09-14T22:38:39.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:39.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824036+0000", 2024-09-14T22:38:39.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7350000000000004e-06 2024-09-14T22:38:39.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:39.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824049+0000", 2024-09-14T22:38:39.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3329000000000001e-05 2024-09-14T22:38:39.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:39.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824057+0000", 2024-09-14T22:38:39.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2460000000000003e-06 2024-09-14T22:38:39.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:39.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878464+0000", 2024-09-14T22:38:39.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054406207999999998 2024-09-14T22:38:39.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:39.525 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878554+0000", 2024-09-14T22:38:39.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0199999999999997e-05 2024-09-14T22:38:39.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:39.525 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878628+0000", 2024-09-14T22:38:39.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4371999999999999e-05 2024-09-14T22:38:39.525 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.525 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:39.525 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.526 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112031 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6365184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:39.526 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:38:39.526 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.67462578, 2024-09-14T22:38:39.526 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054741868999999999, 2024-09-14T22:38:39.526 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:39.526 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:39.526 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:39.526 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:39.526 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:39.526 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:39.526 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112031 2024-09-14T22:38:39.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.526 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:39.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:39.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:38:39.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:39.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:38:39.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:39.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824070+0000", 2024-09-14T22:38:39.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5099999999999998e-07 2024-09-14T22:38:39.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:39.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824073+0000", 2024-09-14T22:38:39.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6940000000000001e-06 2024-09-14T22:38:39.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.528 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:39.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824084+0000", 2024-09-14T22:38:39.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0380000000000001e-05 2024-09-14T22:38:39.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.528 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:39.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824091+0000", 2024-09-14T22:38:39.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5730000000000002e-06 2024-09-14T22:38:39.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:39.529 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878670+0000", 2024-09-14T22:38:39.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054579176 2024-09-14T22:38:39.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:39.529 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878742+0000", 2024-09-14T22:38:39.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T22:38:39.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:39.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878811+0000", 2024-09-14T22:38:39.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8170999999999995e-05 2024-09-14T22:38:39.530 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.530 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:39.530 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.530 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112032 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6414336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:39.530 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:38:39.530 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.674591914000004, 2024-09-14T22:38:39.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054931916999999997, 2024-09-14T22:38:39.530 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:39.530 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:39.532 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:39.532 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:39.532 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:39.532 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:39.532 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112032 2024-09-14T22:38:39.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.532 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:39.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.532 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:39.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:38:39.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:39.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:38:39.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:39.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:38:39.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:38:39.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.534 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:39.534 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824107+0000", 2024-09-14T22:38:39.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5640000000000001e-06 2024-09-14T22:38:39.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.534 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:39.534 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824117+0000", 2024-09-14T22:38:39.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0366e-05 2024-09-14T22:38:39.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.534 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:39.534 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824158+0000", 2024-09-14T22:38:39.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0670000000000002e-05 2024-09-14T22:38:39.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:39.535 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878853+0000", 2024-09-14T22:38:39.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054694853000000002 2024-09-14T22:38:39.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:39.535 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878934+0000", 2024-09-14T22:38:39.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1754000000000004e-05 2024-09-14T22:38:39.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:39.535 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879034+0000", 2024-09-14T22:38:39.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000100115 2024-09-14T22:38:39.535 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.535 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:39.536 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.536 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112033 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6463488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:39.536 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:38:39.536 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.674523256000001, 2024-09-14T22:38:39.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054991076999999999, 2024-09-14T22:38:39.536 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:39.536 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:39.536 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:39.536 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:39.536 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:39.536 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:39.536 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112033 2024-09-14T22:38:39.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.537 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:39.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:39.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:38:39.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:39.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:38:39.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:39.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824172+0000", 2024-09-14T22:38:39.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9999999999999996e-07 2024-09-14T22:38:39.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.538 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:39.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824177+0000", 2024-09-14T22:38:39.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5310000000000003e-06 2024-09-14T22:38:39.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.538 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:39.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824188+0000", 2024-09-14T22:38:39.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1235e-05 2024-09-14T22:38:39.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.538 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:39.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824197+0000", 2024-09-14T22:38:39.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7069999999999998e-06 2024-09-14T22:38:39.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:39.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879075+0000", 2024-09-14T22:38:39.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054878749999999997 2024-09-14T22:38:39.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:39.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879118+0000", 2024-09-14T22:38:39.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2749e-05 2024-09-14T22:38:39.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:39.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879162+0000", 2024-09-14T22:38:39.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4205000000000001e-05 2024-09-14T22:38:39.540 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.540 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:39.540 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.540 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112034 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6512640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:39.540 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:38:39.540 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.674486305000002, 2024-09-14T22:38:39.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055181246000000003, 2024-09-14T22:38:39.540 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:39.540 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:39.540 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:39.540 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:39.540 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:39.540 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:39.541 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112034 2024-09-14T22:38:39.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.541 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:39.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:39.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:38:39.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:39.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:38:39.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:39.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824209+0000", 2024-09-14T22:38:39.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5499999999999998e-07 2024-09-14T22:38:39.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:39.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824212+0000", 2024-09-14T22:38:39.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6119999999999999e-06 2024-09-14T22:38:39.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:39.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824224+0000", 2024-09-14T22:38:39.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1127999999999999e-05 2024-09-14T22:38:39.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:39.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824232+0000", 2024-09-14T22:38:39.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4700000000000002e-06 2024-09-14T22:38:39.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:39.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879209+0000", 2024-09-14T22:38:39.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054977201000000003 2024-09-14T22:38:39.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:39.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879308+0000", 2024-09-14T22:38:39.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8419999999999996e-05 2024-09-14T22:38:39.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.544 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:39.544 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879389+0000", 2024-09-14T22:38:39.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1760000000000003e-05 2024-09-14T22:38:39.544 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.544 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:39.544 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.544 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112035 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6561792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:39.544 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:38:39.544 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.674392577000006, 2024-09-14T22:38:39.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055300522999999997, 2024-09-14T22:38:39.544 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:39.545 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:39.545 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:39.545 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:39.545 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:39.545 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:39.545 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112035 2024-09-14T22:38:39.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.545 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:39.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:39.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:38:39.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:39.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:38:39.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:39.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824303+0000", 2024-09-14T22:38:39.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5099999999999998e-07 2024-09-14T22:38:39.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:39.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824308+0000", 2024-09-14T22:38:39.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6689999999999997e-06 2024-09-14T22:38:39.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.547 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:39.547 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824744+0000", 2024-09-14T22:38:39.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00043601600000000001 2024-09-14T22:38:39.547 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.547 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:39.547 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824756+0000", 2024-09-14T22:38:39.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2476999999999999e-05 2024-09-14T22:38:39.547 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.547 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:39.547 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879434+0000", 2024-09-14T22:38:39.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054677630999999997 2024-09-14T22:38:39.547 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.548 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:39.548 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879518+0000", 2024-09-14T22:38:39.548 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4220000000000003e-05 2024-09-14T22:38:39.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.548 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.548 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:39.548 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879602+0000", 2024-09-14T22:38:39.548 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4559000000000001e-05 2024-09-14T22:38:39.548 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.548 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:39.548 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.548 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.548 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112036 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6610944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:39.549 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:38:39.549 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.673900838999998, 2024-09-14T22:38:39.549 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055036320999999999, 2024-09-14T22:38:39.549 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:39.549 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:39.549 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:39.549 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:39.549 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:39.549 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:39.549 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112036 2024-09-14T22:38:39.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.549 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:39.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.549 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:39.549 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:38:39.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.550 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.550 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:39.550 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:38:39.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.550 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.550 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:39.550 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824795+0000", 2024-09-14T22:38:39.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-06 2024-09-14T22:38:39.550 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.550 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:39.551 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824802+0000", 2024-09-14T22:38:39.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6680000000000004e-06 2024-09-14T22:38:39.551 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.551 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:39.551 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824814+0000", 2024-09-14T22:38:39.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2102e-05 2024-09-14T22:38:39.551 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.551 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:39.551 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824823+0000", 2024-09-14T22:38:39.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5550000000000008e-06 2024-09-14T22:38:39.551 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.551 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:39.552 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879648+0000", 2024-09-14T22:38:39.552 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054825167000000001 2024-09-14T22:38:39.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.552 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:39.552 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879747+0000", 2024-09-14T22:38:39.552 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8968999999999994e-05 2024-09-14T22:38:39.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.552 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:39.552 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879830+0000", 2024-09-14T22:38:39.552 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2633999999999995e-05 2024-09-14T22:38:39.552 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.552 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:39.552 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.553 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.553 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112037 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6660096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:39.553 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:38:39.553 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.673858129999999, 2024-09-14T22:38:39.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055122926000000003, 2024-09-14T22:38:39.553 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:39.553 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:39.553 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:39.553 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:39.553 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:39.553 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:39.553 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112037 2024-09-14T22:38:39.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.553 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:39.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.554 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:39.554 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:38:39.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.554 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.554 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:39.554 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:38:39.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.554 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.554 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:39.554 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824837+0000", 2024-09-14T22:38:39.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5499999999999998e-07 2024-09-14T22:38:39.554 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.555 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:39.555 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824840+0000", 2024-09-14T22:38:39.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0429999999999999e-06 2024-09-14T22:38:39.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.555 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.555 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:39.555 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824851+0000", 2024-09-14T22:38:39.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1046999999999999e-05 2024-09-14T22:38:39.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.555 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.555 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:39.555 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824859+0000", 2024-09-14T22:38:39.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1200000000000002e-06 2024-09-14T22:38:39.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.555 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.555 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:39.555 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879867+0000", 2024-09-14T22:38:39.556 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055007697000000001 2024-09-14T22:38:39.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.556 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.556 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:39.556 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879910+0000", 2024-09-14T22:38:39.556 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3402000000000002e-05 2024-09-14T22:38:39.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.556 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.556 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:39.556 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879959+0000", 2024-09-14T22:38:39.556 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9061999999999999e-05 2024-09-14T22:38:39.556 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.556 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:39.556 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.557 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112038 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6709248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:39.557 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:38:39.557 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.673823433999999, 2024-09-14T22:38:39.557 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055312828000000001, 2024-09-14T22:38:39.557 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:39.557 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:39.557 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:39.557 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:39.557 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:39.557 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:39.557 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112038 2024-09-14T22:38:39.557 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.557 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:39.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.558 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:39.558 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:38:39.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.558 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:39.558 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:38:39.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.558 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:39.558 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824872+0000", 2024-09-14T22:38:39.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T22:38:39.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.559 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.559 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:39.559 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824875+0000", 2024-09-14T22:38:39.559 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7519999999999998e-06 2024-09-14T22:38:39.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.559 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.559 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:39.559 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824917+0000", 2024-09-14T22:38:39.559 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1047999999999997e-05 2024-09-14T22:38:39.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.559 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.559 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:39.559 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824926+0000", 2024-09-14T22:38:39.559 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3749999999999992e-06 2024-09-14T22:38:39.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.559 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.560 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:39.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880006+0000", 2024-09-14T22:38:39.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055080525999999998 2024-09-14T22:38:39.560 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.560 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.560 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:39.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880102+0000", 2024-09-14T22:38:39.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5995999999999994e-05 2024-09-14T22:38:39.560 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.560 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.560 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:39.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880184+0000", 2024-09-14T22:38:39.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1478999999999997e-05 2024-09-14T22:38:39.560 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.561 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:39.561 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.561 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.561 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.561 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112039 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6758400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:39.561 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:38:39.561 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.673754763999995, 2024-09-14T22:38:39.561 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055449709999999999, 2024-09-14T22:38:39.561 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:39.561 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:39.561 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:39.561 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:39.561 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:39.561 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:39.562 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112039 2024-09-14T22:38:39.562 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.562 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:39.562 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.562 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:39.562 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:38:39.562 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.562 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.562 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.562 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:39.562 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:38:39.562 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.562 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.562 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.562 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:39.562 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824941+0000", 2024-09-14T22:38:39.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3699999999999999e-07 2024-09-14T22:38:39.563 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.563 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:39.563 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824946+0000", 2024-09-14T22:38:39.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0209999999999999e-06 2024-09-14T22:38:39.563 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.563 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:39.563 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824972+0000", 2024-09-14T22:38:39.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6750999999999998e-05 2024-09-14T22:38:39.563 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.563 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:39.563 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824982+0000", 2024-09-14T22:38:39.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5019999999999998e-06 2024-09-14T22:38:39.564 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.564 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.564 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:39.564 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880227+0000", 2024-09-14T22:38:39.564 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055245257999999998 2024-09-14T22:38:39.564 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.564 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.564 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:39.564 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880305+0000", 2024-09-14T22:38:39.564 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7424999999999998e-05 2024-09-14T22:38:39.564 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.564 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.564 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:39.564 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880389+0000", 2024-09-14T22:38:39.564 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4815999999999996e-05 2024-09-14T22:38:39.565 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.565 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:39.565 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.565 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.565 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.565 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112040 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6807552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:39.565 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:38:39.565 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.673698048000006, 2024-09-14T22:38:39.565 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055534615000000002, 2024-09-14T22:38:39.565 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:39.565 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:39.565 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:39.565 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:39.565 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:39.565 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:39.565 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112040 2024-09-14T22:38:39.566 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.566 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:39.566 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.566 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:39.566 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:38:39.566 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.566 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.566 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.566 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:39.566 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:38:39.566 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.566 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.566 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.566 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:39.567 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824997+0000", 2024-09-14T22:38:39.567 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7199999999999998e-07 2024-09-14T22:38:39.567 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.567 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.567 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:39.567 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825042+0000", 2024-09-14T22:38:39.567 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5065999999999998e-05 2024-09-14T22:38:39.567 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.567 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.567 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:39.567 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825060+0000", 2024-09-14T22:38:39.567 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7520000000000002e-05 2024-09-14T22:38:39.567 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.567 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.568 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:39.568 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825070+0000", 2024-09-14T22:38:39.568 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8230000000000006e-06 2024-09-14T22:38:39.568 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.568 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.568 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:39.568 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880427+0000", 2024-09-14T22:38:39.568 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055357244 2024-09-14T22:38:39.568 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.568 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.568 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:39.568 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880483+0000", 2024-09-14T22:38:39.568 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5807000000000003e-05 2024-09-14T22:38:39.568 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.569 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.569 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:39.569 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880531+0000", 2024-09-14T22:38:39.569 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8383000000000001e-05 2024-09-14T22:38:39.569 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.569 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:39.569 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.569 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.569 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.569 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112041 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6856704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:39.569 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:38:39.569 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.673608873999996, 2024-09-14T22:38:39.569 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055573887000000002, 2024-09-14T22:38:39.569 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:39.570 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:39.570 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:39.570 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:39.570 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:39.570 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:39.570 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112041 2024-09-14T22:38:39.570 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.570 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:39.570 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.570 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:39.570 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:38:39.570 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.570 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.570 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.571 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:39.571 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:38:39.571 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.571 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.571 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.571 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:39.571 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825087+0000", 2024-09-14T22:38:39.571 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2399999999999996e-07 2024-09-14T22:38:39.571 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.571 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.571 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:39.571 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825091+0000", 2024-09-14T22:38:39.571 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9729999999999999e-06 2024-09-14T22:38:39.571 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.572 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.572 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:39.572 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825101+0000", 2024-09-14T22:38:39.572 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.093e-05 2024-09-14T22:38:39.572 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.572 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.572 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:39.572 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825110+0000", 2024-09-14T22:38:39.572 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1450000000000008e-06 2024-09-14T22:38:39.572 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.572 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.572 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:39.572 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880559+0000", 2024-09-14T22:38:39.572 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055449684999999999 2024-09-14T22:38:39.573 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.573 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.573 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:39.573 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880601+0000", 2024-09-14T22:38:39.573 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1248000000000002e-05 2024-09-14T22:38:39.573 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.573 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.573 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:39.573 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880659+0000", 2024-09-14T22:38:39.573 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8981999999999999e-05 2024-09-14T22:38:39.573 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.573 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:39.573 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.573 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.574 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.574 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112042 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6905856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:39.574 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:38:39.574 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.673574139999999, 2024-09-14T22:38:39.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055781415000000001, 2024-09-14T22:38:39.574 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:39.574 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:39.574 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:39.574 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:39.574 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:39.574 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:39.574 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112042 2024-09-14T22:38:39.574 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.574 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:39.575 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.575 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:39.575 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:38:39.575 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.575 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.575 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.575 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:39.575 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:38:39.575 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.575 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.575 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.575 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:39.575 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825478+0000", 2024-09-14T22:38:39.575 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035775 2024-09-14T22:38:39.576 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.576 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.576 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:39.576 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825494+0000", 2024-09-14T22:38:39.576 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5994999999999998e-05 2024-09-14T22:38:39.576 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.576 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.576 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:39.576 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825520+0000", 2024-09-14T22:38:39.576 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5738999999999999e-05 2024-09-14T22:38:39.576 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.576 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.576 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:39.577 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825547+0000", 2024-09-14T22:38:39.577 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7667000000000001e-05 2024-09-14T22:38:39.577 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.577 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.577 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:39.577 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880725+0000", 2024-09-14T22:38:39.577 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055177962999999997 2024-09-14T22:38:39.577 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.577 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.577 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:39.577 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880816+0000", 2024-09-14T22:38:39.577 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0315000000000005e-05 2024-09-14T22:38:39.577 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.577 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.577 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:39.578 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880902+0000", 2024-09-14T22:38:39.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5986e-05 2024-09-14T22:38:39.578 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.578 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:39.578 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.578 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.578 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112043 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:39.578 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:38:39.578 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.673126135999993, 2024-09-14T22:38:39.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055537912000000002, 2024-09-14T22:38:39.578 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:39.578 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:39.579 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:39.579 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:39.579 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:39.579 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:39.579 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112043 2024-09-14T22:38:39.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.579 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:39.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.579 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:39.579 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:38:39.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.579 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:39.579 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:38:39.580 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.580 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:39.580 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825911+0000", 2024-09-14T22:38:39.580 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00034306899999999999 2024-09-14T22:38:39.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.580 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:39.580 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825918+0000", 2024-09-14T22:38:39.580 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.002e-06 2024-09-14T22:38:39.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.580 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:39.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825931+0000", 2024-09-14T22:38:39.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2401e-05 2024-09-14T22:38:39.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.581 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:39.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825942+0000", 2024-09-14T22:38:39.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0903e-05 2024-09-14T22:38:39.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.581 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:39.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880951+0000", 2024-09-14T22:38:39.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055009379999999997 2024-09-14T22:38:39.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.582 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:39.582 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881027+0000", 2024-09-14T22:38:39.582 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5944000000000003e-05 2024-09-14T22:38:39.582 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.582 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:39.582 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881106+0000", 2024-09-14T22:38:39.582 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9213000000000002e-05 2024-09-14T22:38:39.582 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.582 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:39.582 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.582 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.582 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112044 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 7004160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:39.583 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:38:39.583 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.672732467000003, 2024-09-14T22:38:39.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055265416999999997, 2024-09-14T22:38:39.583 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:39.583 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:39.583 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:39.583 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:39.583 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:39.583 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:39.583 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112044 2024-09-14T22:38:39.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.583 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:39.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.583 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:39.583 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:38:39.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:39.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:38:39.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:39.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826239+0000", 2024-09-14T22:38:39.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027676000000000003 2024-09-14T22:38:39.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:39.585 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826247+0000", 2024-09-14T22:38:39.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7440000000000007e-06 2024-09-14T22:38:39.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:39.585 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826275+0000", 2024-09-14T22:38:39.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8053000000000001e-05 2024-09-14T22:38:39.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:39.585 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826286+0000", 2024-09-14T22:38:39.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1780000000000001e-05 2024-09-14T22:38:39.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.586 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:39.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881137+0000", 2024-09-14T22:38:39.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054850500000000003 2024-09-14T22:38:39.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.586 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:39.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881181+0000", 2024-09-14T22:38:39.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4153000000000002e-05 2024-09-14T22:38:39.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.586 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:39.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881227+0000", 2024-09-14T22:38:39.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6427000000000001e-05 2024-09-14T22:38:39.587 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.587 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:39.587 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.587 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112045 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 7053312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:39.587 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:38:39.587 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.672370233999999, 2024-09-14T22:38:39.587 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055045050999999998, 2024-09-14T22:38:39.587 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:39.587 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:39.587 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:39.587 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:39.587 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:39.588 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:39.588 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112045 2024-09-14T22:38:39.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.588 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:39.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:39.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:38:39.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:39.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:38:39.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:39.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:39.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830250+0000", 2024-09-14T22:38:39.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0039258670000000004 2024-09-14T22:38:39.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:39.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830260+0000", 2024-09-14T22:38:39.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0088000000000001e-05 2024-09-14T22:38:39.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:39.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830281+0000", 2024-09-14T22:38:39.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0720000000000002e-05 2024-09-14T22:38:39.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:39.590 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830292+0000", 2024-09-14T22:38:39.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1455e-05 2024-09-14T22:38:39.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:39.590 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881255+0000", 2024-09-14T22:38:39.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.050962247000000002 2024-09-14T22:38:39.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:39.590 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881296+0000", 2024-09-14T22:38:39.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1557999999999998e-05 2024-09-14T22:38:39.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:39.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:39.591 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:39.591 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881369+0000", 2024-09-14T22:38:39.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3115999999999996e-05 2024-09-14T22:38:39.591 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.591 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:39.591 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.591 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:39.591 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:39.591 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:39.593 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:38:39.593 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:39.590+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:39.694 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:39.686+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:39.765 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:39.765 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:39.765 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:38:39.765 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:39.779 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:38:39.795 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:39.793+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:39.896 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:39.890+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:39.905 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-14T22:38:39.905 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:38:39.905 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T22:38:39.905 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T22:38:39.967 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:39.967 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:39.967 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:38:39.967 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:38:39.967 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:39.985 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:38:40.027 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:40.022+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:40.097 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:40.097+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:40.153 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:40.154 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:38:40.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:38:40.154 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:38:40.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.154 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97197 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 796672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:40.154 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:38:40.154 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 90.078761197999995, 2024-09-14T22:38:40.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12434473, 2024-09-14T22:38:40.154 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:40.154 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:40.154 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:40.155 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:40.155 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:40.155 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:40.155 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97197 2024-09-14T22:38:40.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.155 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:40.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:40.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:38:40.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:40.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:38:40.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:40.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070524+0000", 2024-09-14T22:38:40.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0299999999999998e-07 2024-09-14T22:38:40.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:40.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070527+0000", 2024-09-14T22:38:40.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9569999999999998e-06 2024-09-14T22:38:40.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:40.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070535+0000", 2024-09-14T22:38:40.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7630000000000006e-06 2024-09-14T22:38:40.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:40.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070543+0000", 2024-09-14T22:38:40.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4869999999999996e-06 2024-09-14T22:38:40.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:40.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194525+0000", 2024-09-14T22:38:40.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123982096 2024-09-14T22:38:40.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:40.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194699+0000", 2024-09-14T22:38:40.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000174105 2024-09-14T22:38:40.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:40.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194868+0000", 2024-09-14T22:38:40.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016861900000000001 2024-09-14T22:38:40.158 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.158 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:40.158 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.159 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97198 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 805888~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:40.159 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:38:40.159 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 90.078732822000006, 2024-09-14T22:38:40.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.1248088, 2024-09-14T22:38:40.159 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:40.159 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:40.159 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:40.159 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:40.159 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:40.159 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:40.159 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97198 2024-09-14T22:38:40.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.159 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:40.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:40.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:38:40.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:40.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:38:40.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:40.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070552+0000", 2024-09-14T22:38:40.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T22:38:40.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.161 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:40.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070555+0000", 2024-09-14T22:38:40.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.853e-06 2024-09-14T22:38:40.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.161 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:40.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070565+0000", 2024-09-14T22:38:40.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.048e-05 2024-09-14T22:38:40.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.161 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:40.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070573+0000", 2024-09-14T22:38:40.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7200000000000006e-06 2024-09-14T22:38:40.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:40.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194954+0000", 2024-09-14T22:38:40.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12438089400000001 2024-09-14T22:38:40.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:40.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195145+0000", 2024-09-14T22:38:40.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019091299999999999 2024-09-14T22:38:40.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:40.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195360+0000", 2024-09-14T22:38:40.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021527300000000001 2024-09-14T22:38:40.163 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.163 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:40.163 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.163 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97199 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 815104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:40.163 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:38:40.163 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 90.078702225000001, 2024-09-14T22:38:40.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125251901, 2024-09-14T22:38:40.163 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:40.163 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:40.163 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:40.164 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:40.164 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:40.164 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:40.164 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97199 2024-09-14T22:38:40.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.164 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:40.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:40.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:38:40.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:40.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:38:40.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:40.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070583+0000", 2024-09-14T22:38:40.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6300000000000005e-07 2024-09-14T22:38:40.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:40.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070586+0000", 2024-09-14T22:38:40.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8270000000000002e-06 2024-09-14T22:38:40.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:40.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070597+0000", 2024-09-14T22:38:40.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1960000000000001e-05 2024-09-14T22:38:40.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:40.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070605+0000", 2024-09-14T22:38:40.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6459999999999993e-06 2024-09-14T22:38:40.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:40.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195432+0000", 2024-09-14T22:38:40.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124826926 2024-09-14T22:38:40.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:40.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195626+0000", 2024-09-14T22:38:40.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019357099999999999 2024-09-14T22:38:40.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:40.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195834+0000", 2024-09-14T22:38:40.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020830800000000001 2024-09-14T22:38:40.167 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.167 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:40.167 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.167 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97200 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 824320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:40.167 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:38:40.168 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 90.078670179, 2024-09-14T22:38:40.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125693477, 2024-09-14T22:38:40.168 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:40.168 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:40.168 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:40.168 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:40.168 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:40.168 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:40.168 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97200 2024-09-14T22:38:40.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.168 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:40.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:40.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:38:40.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:40.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:38:40.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:40.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070615+0000", 2024-09-14T22:38:40.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T22:38:40.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:40.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070618+0000", 2024-09-14T22:38:40.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.943e-06 2024-09-14T22:38:40.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:40.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070627+0000", 2024-09-14T22:38:40.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1099999999999992e-06 2024-09-14T22:38:40.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:40.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070634+0000", 2024-09-14T22:38:40.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:38:40.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:40.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195908+0000", 2024-09-14T22:38:40.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125273893 2024-09-14T22:38:40.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:40.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196097+0000", 2024-09-14T22:38:40.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018953899999999999 2024-09-14T22:38:40.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:40.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196308+0000", 2024-09-14T22:38:40.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021027799999999999 2024-09-14T22:38:40.171 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.171 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:40.172 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.172 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97201 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 833536~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:40.172 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:38:40.172 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 90.078639992000006, 2024-09-14T22:38:40.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126133826, 2024-09-14T22:38:40.172 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:40.172 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:40.172 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:40.172 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:40.172 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:40.173 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:40.173 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97201 2024-09-14T22:38:40.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.173 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:40.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:40.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:38:40.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:40.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:38:40.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:40.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074109+0000", 2024-09-14T22:38:40.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003464834 2024-09-14T22:38:40.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:40.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074134+0000", 2024-09-14T22:38:40.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5170000000000001e-05 2024-09-14T22:38:40.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:40.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074166+0000", 2024-09-14T22:38:40.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1347000000000003e-05 2024-09-14T22:38:40.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:40.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074179+0000", 2024-09-14T22:38:40.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.364e-05 2024-09-14T22:38:40.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:40.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196377+0000", 2024-09-14T22:38:40.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12219793900000001 2024-09-14T22:38:40.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:40.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196568+0000", 2024-09-14T22:38:40.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019097999999999999 2024-09-14T22:38:40.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:40.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196778+0000", 2024-09-14T22:38:40.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020991599999999999 2024-09-14T22:38:40.176 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.176 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:40.176 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.177 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97205 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 870400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:40.177 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:38:40.177 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 90.074970429000004, 2024-09-14T22:38:40.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12436148299999999, 2024-09-14T22:38:40.177 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:40.177 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:40.177 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:40.177 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:40.177 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:40.177 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:40.177 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97205 2024-09-14T22:38:40.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.178 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:40.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:40.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:38:40.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:40.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:38:40.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:40.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074315+0000", 2024-09-14T22:38:40.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9299999999999997e-07 2024-09-14T22:38:40.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:40.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074318+0000", 2024-09-14T22:38:40.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.19e-06 2024-09-14T22:38:40.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:40.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074327+0000", 2024-09-14T22:38:40.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8529999999999998e-06 2024-09-14T22:38:40.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:40.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074334+0000", 2024-09-14T22:38:40.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3869999999999997e-06 2024-09-14T22:38:40.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:40.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198365+0000", 2024-09-14T22:38:40.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124031222 2024-09-14T22:38:40.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:40.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198513+0000", 2024-09-14T22:38:40.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014772299999999999 2024-09-14T22:38:40.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:40.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198675+0000", 2024-09-14T22:38:40.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016241500000000001 2024-09-14T22:38:40.181 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.181 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:40.181 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.181 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97206 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 879616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:40.181 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:38:40.181 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 90.074938973000002, 2024-09-14T22:38:40.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124706284, 2024-09-14T22:38:40.182 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:40.182 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:40.182 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:40.182 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:40.182 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:40.182 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:40.182 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97206 2024-09-14T22:38:40.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.182 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:40.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:40.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:38:40.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:40.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:38:40.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:40.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074346+0000", 2024-09-14T22:38:40.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8999999999999996e-07 2024-09-14T22:38:40.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:40.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074349+0000", 2024-09-14T22:38:40.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.96e-06 2024-09-14T22:38:40.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:40.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074358+0000", 2024-09-14T22:38:40.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9169999999999995e-06 2024-09-14T22:38:40.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:40.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074365+0000", 2024-09-14T22:38:40.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3699999999999997e-06 2024-09-14T22:38:40.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:40.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198761+0000", 2024-09-14T22:38:40.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124395743 2024-09-14T22:38:40.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:40.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198899+0000", 2024-09-14T22:38:40.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013763499999999999 2024-09-14T22:38:40.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:40.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199052+0000", 2024-09-14T22:38:40.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015296900000000001 2024-09-14T22:38:40.185 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.185 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:40.185 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.186 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97207 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 888832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:40.186 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:38:40.186 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 90.074909759999997, 2024-09-14T22:38:40.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124988433, 2024-09-14T22:38:40.186 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:40.186 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:40.186 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:40.186 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:40.186 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:40.186 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:40.186 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97207 2024-09-14T22:38:40.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.186 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:40.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:40.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:38:40.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:40.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:38:40.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:40.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074375+0000", 2024-09-14T22:38:40.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:38:40.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:40.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074378+0000", 2024-09-14T22:38:40.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1829999999999998e-06 2024-09-14T22:38:40.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:40.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074389+0000", 2024-09-14T22:38:40.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0737000000000001e-05 2024-09-14T22:38:40.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:40.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074396+0000", 2024-09-14T22:38:40.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2559999999999996e-06 2024-09-14T22:38:40.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:40.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199139+0000", 2024-09-14T22:38:40.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12474253 2024-09-14T22:38:40.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:40.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199240+0000", 2024-09-14T22:38:40.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101221 2024-09-14T22:38:40.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:40.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199363+0000", 2024-09-14T22:38:40.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000122822 2024-09-14T22:38:40.190 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.190 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:40.190 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.190 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97208 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 898048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:40.190 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:38:40.190 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 90.074878455999993, 2024-09-14T22:38:40.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12524854899999999, 2024-09-14T22:38:40.190 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:40.191 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:40.191 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:40.191 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:40.191 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:40.191 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:40.191 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97208 2024-09-14T22:38:40.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.191 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:40.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:40.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:38:40.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:40.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:38:40.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:40.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:38:40.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:38:40.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:40.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074410+0000", 2024-09-14T22:38:40.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.18e-06 2024-09-14T22:38:40.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:40.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074435+0000", 2024-09-14T22:38:40.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5273e-05 2024-09-14T22:38:40.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:40.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074443+0000", 2024-09-14T22:38:40.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3000000000000002e-06 2024-09-14T22:38:40.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:40.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199433+0000", 2024-09-14T22:38:40.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124989502 2024-09-14T22:38:40.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:40.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199535+0000", 2024-09-14T22:38:40.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102557 2024-09-14T22:38:40.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:40.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199654+0000", 2024-09-14T22:38:40.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119061 2024-09-14T22:38:40.194 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.194 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:40.194 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.195 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97209 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 907264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:40.195 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:38:40.195 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 90.074830650999999, 2024-09-14T22:38:40.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125487354, 2024-09-14T22:38:40.195 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:40.195 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:40.195 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:40.195 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:40.195 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:40.195 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:40.195 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97209 2024-09-14T22:38:40.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.195 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:40.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:40.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:38:40.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:40.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:38:40.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:40.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:38:40.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9400000000000005e-07 2024-09-14T22:38:40.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:40.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074458+0000", 2024-09-14T22:38:40.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3299999999999999e-06 2024-09-14T22:38:40.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:40.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074467+0000", 2024-09-14T22:38:40.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.346e-06 2024-09-14T22:38:40.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:40.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074474+0000", 2024-09-14T22:38:40.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.374e-06 2024-09-14T22:38:40.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:40.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199720+0000", 2024-09-14T22:38:40.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12524561200000001 2024-09-14T22:38:40.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:40.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199820+0000", 2024-09-14T22:38:40.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010049599999999999 2024-09-14T22:38:40.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:40.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199941+0000", 2024-09-14T22:38:40.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012050200000000001 2024-09-14T22:38:40.198 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.199 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:40.199 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.199 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97210 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:40.199 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:38:40.199 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 90.074800656999997, 2024-09-14T22:38:40.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125744942, 2024-09-14T22:38:40.199 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:40.199 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:40.199 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:40.199 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:40.199 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:40.199 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:40.199 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97210 2024-09-14T22:38:40.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.200 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:40.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:40.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:38:40.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:40.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:38:40.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:40.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:38:40.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6599999999999996e-07 2024-09-14T22:38:40.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:40.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074487+0000", 2024-09-14T22:38:40.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8140000000000001e-06 2024-09-14T22:38:40.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:40.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074496+0000", 2024-09-14T22:38:40.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9330000000000007e-06 2024-09-14T22:38:40.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:40.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074503+0000", 2024-09-14T22:38:40.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4100000000000002e-06 2024-09-14T22:38:40.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:40.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200009+0000", 2024-09-14T22:38:40.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12550539899999999 2024-09-14T22:38:40.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:40.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200108+0000", 2024-09-14T22:38:40.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9182e-05 2024-09-14T22:38:40.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:40.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200229+0000", 2024-09-14T22:38:40.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120538 2024-09-14T22:38:40.203 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.203 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:40.203 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.203 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97211 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 925696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:40.203 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:38:40.203 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 90.074769696999994, 2024-09-14T22:38:40.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12599848299999999, 2024-09-14T22:38:40.203 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:40.203 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:40.203 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:40.204 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:40.204 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:40.204 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:40.204 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97211 2024-09-14T22:38:40.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.204 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:40.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:40.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:38:40.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:40.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:38:40.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:40.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074516+0000", 2024-09-14T22:38:40.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.42e-06 2024-09-14T22:38:40.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:40.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074521+0000", 2024-09-14T22:38:40.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7960000000000003e-06 2024-09-14T22:38:40.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:40.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074538+0000", 2024-09-14T22:38:40.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6900000000000001e-05 2024-09-14T22:38:40.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:40.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074555+0000", 2024-09-14T22:38:40.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.696e-05 2024-09-14T22:38:40.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:40.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200293+0000", 2024-09-14T22:38:40.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12573864100000001 2024-09-14T22:38:40.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:40.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200393+0000", 2024-09-14T22:38:40.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9264000000000004e-05 2024-09-14T22:38:40.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:40.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200513+0000", 2024-09-14T22:38:40.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012050200000000001 2024-09-14T22:38:40.207 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.207 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:40.207 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.207 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97212 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 934912~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:40.207 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:38:40.208 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 90.074695074999994, 2024-09-14T22:38:40.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12621127300000001, 2024-09-14T22:38:40.208 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:40.208 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:40.208 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:40.208 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:40.208 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:40.208 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:40.208 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97212 2024-09-14T22:38:40.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.208 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:40.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:40.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:38:40.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:40.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:38:40.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:40.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074590+0000", 2024-09-14T22:38:40.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1969999999999999e-06 2024-09-14T22:38:40.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:40.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074596+0000", 2024-09-14T22:38:40.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7629999999999997e-06 2024-09-14T22:38:40.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:40.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074614+0000", 2024-09-14T22:38:40.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7643e-05 2024-09-14T22:38:40.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:40.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074635+0000", 2024-09-14T22:38:40.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.075e-05 2024-09-14T22:38:40.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:40.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200580+0000", 2024-09-14T22:38:40.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125945482 2024-09-14T22:38:40.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:40.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200683+0000", 2024-09-14T22:38:40.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010267099999999999 2024-09-14T22:38:40.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:40.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200800+0000", 2024-09-14T22:38:40.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117767 2024-09-14T22:38:40.211 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.211 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:40.211 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.212 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97213 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 944128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:40.212 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:38:40.212 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 90.074629724999994, 2024-09-14T22:38:40.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126430709, 2024-09-14T22:38:40.212 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:40.212 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:40.212 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:40.212 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:40.212 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:40.212 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:40.213 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97213 2024-09-14T22:38:40.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.213 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:40.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:40.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:38:40.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:40.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:38:40.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:40.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078210+0000", 2024-09-14T22:38:40.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0035556839999999999 2024-09-14T22:38:40.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:40.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078231+0000", 2024-09-14T22:38:40.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0716e-05 2024-09-14T22:38:40.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:40.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078260+0000", 2024-09-14T22:38:40.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9473000000000001e-05 2024-09-14T22:38:40.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:40.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078279+0000", 2024-09-14T22:38:40.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8600000000000001e-05 2024-09-14T22:38:40.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:40.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200864+0000", 2024-09-14T22:38:40.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.122585295 2024-09-14T22:38:40.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:40.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200965+0000", 2024-09-14T22:38:40.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101004 2024-09-14T22:38:40.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:40.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.201085+0000", 2024-09-14T22:38:40.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119937 2024-09-14T22:38:40.216 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.216 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:40.216 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.216 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97219 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 999424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:40.216 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:38:40.217 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 90.070647448000003, 2024-09-14T22:38:40.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124138558, 2024-09-14T22:38:40.217 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:40.217 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:40.217 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:40.217 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:40.217 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:40.217 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:40.217 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97219 2024-09-14T22:38:40.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.217 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:40.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:40.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:38:40.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:40.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:38:40.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:40.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078638+0000", 2024-09-14T22:38:40.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.173e-06 2024-09-14T22:38:40.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:40.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078644+0000", 2024-09-14T22:38:40.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2570000000000004e-06 2024-09-14T22:38:40.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:40.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078672+0000", 2024-09-14T22:38:40.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7322999999999999e-05 2024-09-14T22:38:40.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:40.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078687+0000", 2024-09-14T22:38:40.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5639999999999999e-05 2024-09-14T22:38:40.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:40.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202567+0000", 2024-09-14T22:38:40.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123880134 2024-09-14T22:38:40.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:40.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202670+0000", 2024-09-14T22:38:40.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010273899999999999 2024-09-14T22:38:40.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:40.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202775+0000", 2024-09-14T22:38:40.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010529199999999999 2024-09-14T22:38:40.220 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.220 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:40.220 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.221 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97220 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1008640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:40.221 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:38:40.221 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 90.070578214999998, 2024-09-14T22:38:40.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12435937, 2024-09-14T22:38:40.221 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:40.221 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:40.221 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:40.221 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:40.221 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:40.221 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:40.221 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97220 2024-09-14T22:38:40.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.222 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:40.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:40.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:38:40.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:40.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:38:40.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:40.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078707+0000", 2024-09-14T22:38:40.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.266e-06 2024-09-14T22:38:40.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:40.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078713+0000", 2024-09-14T22:38:40.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7270000000000004e-06 2024-09-14T22:38:40.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:40.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078737+0000", 2024-09-14T22:38:40.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3640000000000001e-05 2024-09-14T22:38:40.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:40.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078751+0000", 2024-09-14T22:38:40.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.45e-05 2024-09-14T22:38:40.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:40.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202841+0000", 2024-09-14T22:38:40.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124089938 2024-09-14T22:38:40.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:40.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202944+0000", 2024-09-14T22:38:40.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010334 2024-09-14T22:38:40.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:40.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203065+0000", 2024-09-14T22:38:40.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012095900000000001 2024-09-14T22:38:40.224 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.225 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:40.225 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.225 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97221 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1017856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:40.225 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:38:40.225 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 90.070512452000003, 2024-09-14T22:38:40.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124574005, 2024-09-14T22:38:40.225 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:40.225 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:40.225 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:40.225 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:40.225 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:40.225 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:40.226 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97221 2024-09-14T22:38:40.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.226 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:40.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:40.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:38:40.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:40.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:38:40.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:40.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078773+0000", 2024-09-14T22:38:40.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.13e-06 2024-09-14T22:38:40.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:40.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078779+0000", 2024-09-14T22:38:40.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0260000000000004e-06 2024-09-14T22:38:40.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:40.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078797+0000", 2024-09-14T22:38:40.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.806e-05 2024-09-14T22:38:40.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:40.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078810+0000", 2024-09-14T22:38:40.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.308e-05 2024-09-14T22:38:40.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:40.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203131+0000", 2024-09-14T22:38:40.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12432077499999999 2024-09-14T22:38:40.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:40.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203227+0000", 2024-09-14T22:38:40.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6575000000000003e-05 2024-09-14T22:38:40.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:40.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203346+0000", 2024-09-14T22:38:40.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118359 2024-09-14T22:38:40.229 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.229 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:40.229 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.229 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97222 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:40.229 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:38:40.229 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 90.070456082999996, 2024-09-14T22:38:40.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12480110799999999, 2024-09-14T22:38:40.229 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:40.229 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:40.229 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:40.230 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:40.230 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:40.230 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:40.230 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97222 2024-09-14T22:38:40.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.230 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:40.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:40.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:38:40.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:40.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:38:40.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:40.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078829+0000", 2024-09-14T22:38:40.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.057e-06 2024-09-14T22:38:40.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:40.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078836+0000", 2024-09-14T22:38:40.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6159999999999998e-06 2024-09-14T22:38:40.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:40.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078856+0000", 2024-09-14T22:38:40.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0100000000000001e-05 2024-09-14T22:38:40.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:40.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078869+0000", 2024-09-14T22:38:40.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.272e-05 2024-09-14T22:38:40.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:40.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203412+0000", 2024-09-14T22:38:40.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12454283300000001 2024-09-14T22:38:40.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:40.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203511+0000", 2024-09-14T22:38:40.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9893000000000001e-05 2024-09-14T22:38:40.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:40.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203629+0000", 2024-09-14T22:38:40.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117889 2024-09-14T22:38:40.233 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.233 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:40.233 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.233 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97223 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1036288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:40.233 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:38:40.233 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 90.070084096000002, 2024-09-14T22:38:40.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124706279, 2024-09-14T22:38:40.234 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:40.234 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:40.234 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:40.234 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:40.234 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:40.234 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:40.234 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97223 2024-09-14T22:38:40.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.234 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:40.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:40.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:38:40.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:40.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:38:40.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:40.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079202+0000", 2024-09-14T22:38:40.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8330000000000001e-06 2024-09-14T22:38:40.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:40.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079212+0000", 2024-09-14T22:38:40.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0207000000000001e-05 2024-09-14T22:38:40.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:40.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079238+0000", 2024-09-14T22:38:40.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5626000000000002e-05 2024-09-14T22:38:40.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:40.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079256+0000", 2024-09-14T22:38:40.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8386999999999999e-05 2024-09-14T22:38:40.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:40.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203691+0000", 2024-09-14T22:38:40.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124434538 2024-09-14T22:38:40.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:40.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203786+0000", 2024-09-14T22:38:40.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5344000000000001e-05 2024-09-14T22:38:40.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:40.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203906+0000", 2024-09-14T22:38:40.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120344 2024-09-14T22:38:40.237 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.237 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:40.238 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.238 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97224 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1045504~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:40.238 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:38:40.238 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 90.070000276000002, 2024-09-14T22:38:40.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12513321199999999, 2024-09-14T22:38:40.238 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:40.238 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:40.238 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:40.238 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:40.238 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:40.238 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:40.239 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97224 2024-09-14T22:38:40.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.239 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:40.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:40.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:38:40.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:40.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:38:40.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:40.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:40.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082581+0000", 2024-09-14T22:38:40.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0032971530000000001 2024-09-14T22:38:40.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:40.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082592+0000", 2024-09-14T22:38:40.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1345999999999999e-05 2024-09-14T22:38:40.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:40.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082618+0000", 2024-09-14T22:38:40.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5873000000000001e-05 2024-09-14T22:38:40.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:40.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082638+0000", 2024-09-14T22:38:40.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9565999999999999e-05 2024-09-14T22:38:40.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:40.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204004+0000", 2024-09-14T22:38:40.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.121366038 2024-09-14T22:38:40.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:40.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204188+0000", 2024-09-14T22:38:40.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018427 2024-09-14T22:38:40.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:40.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:40.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:40.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204417+0000", 2024-09-14T22:38:40.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022896599999999999 2024-09-14T22:38:40.242 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.242 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:40.242 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.242 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:40.242 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:40.242 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:40.244 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:38:40.244 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:40.244 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 147, 2024-09-14T22:38:40.244 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "f07f407a-b17d-42c4-bf73-92dd80114a4f", 2024-09-14T22:38:40.244 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-09-14T22:35:45.983055+0000", 2024-09-14T22:38:40.244 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-09-14T22:38:39.830496+0000", 2024-09-14T22:38:40.244 INFO:teuthology.orchestra.run.smithi001.stdout: "last_up_change": "2024-09-14T22:37:12.616042+0000", 2024-09-14T22:38:40.244 INFO:teuthology.orchestra.run.smithi001.stdout: "last_in_change": "2024-09-14T22:35:48.283160+0000", 2024-09-14T22:38:40.244 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T22:38:40.244 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_num": 5801984, 2024-09-14T22:38:40.244 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_set": [ 2024-09-14T22:38:40.244 INFO:teuthology.orchestra.run.smithi001.stdout: "noscrub", 2024-09-14T22:38:40.245 INFO:teuthology.orchestra.run.smithi001.stdout: "pglog_hardlimit", 2024-09-14T22:38:40.245 INFO:teuthology.orchestra.run.smithi001.stdout: "purged_snapdirs", 2024-09-14T22:38:40.245 INFO:teuthology.orchestra.run.smithi001.stdout: "recovery_deletes", 2024-09-14T22:38:40.245 INFO:teuthology.orchestra.run.smithi001.stdout: "sortbitwise" 2024-09-14T22:38:40.245 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:38:40.245 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_version": 6, 2024-09-14T22:38:40.245 INFO:teuthology.orchestra.run.smithi001.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T22:38:40.245 INFO:teuthology.orchestra.run.smithi001.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T22:38:40.245 INFO:teuthology.orchestra.run.smithi001.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T22:38:40.245 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_snapshot": "", 2024-09-14T22:38:40.245 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_max": 3, 2024-09-14T22:38:40.245 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd": 8, 2024-09-14T22:38:40.245 INFO:teuthology.orchestra.run.smithi001.stdout: "require_min_compat_client": "luminous", 2024-09-14T22:38:40.246 INFO:teuthology.orchestra.run.smithi001.stdout: "min_compat_client": "luminous", 2024-09-14T22:38:40.246 INFO:teuthology.orchestra.run.smithi001.stdout: "require_osd_release": "squid", 2024-09-14T22:38:40.246 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_crimson": false, 2024-09-14T22:38:40.246 INFO:teuthology.orchestra.run.smithi001.stdout: "pools": [ 2024-09-14T22:38:40.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.246 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 1, 2024-09-14T22:38:40.246 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": ".mgr", 2024-09-14T22:38:40.246 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-14T22:35:58.938784+0000", 2024-09-14T22:38:40.246 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-14T22:38:40.246 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool", 2024-09-14T22:38:40.246 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-14T22:38:40.246 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-14T22:38:40.246 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-14T22:38:40.246 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-14T22:38:40.247 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:38:40.247 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:38:40.247 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:38:40.247 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:38:40.247 INFO:teuthology.orchestra.run.smithi001.stdout: "is_stretch_pool": false, 2024-09-14T22:38:40.247 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-14T22:38:40.247 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:38:40.247 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 1, 2024-09-14T22:38:40.247 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 1, 2024-09-14T22:38:40.247 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 1, 2024-09-14T22:38:40.247 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 1, 2024-09-14T22:38:40.247 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 1, 2024-09-14T22:38:40.248 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-14T22:38:40.248 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-14T22:38:40.248 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-14T22:38:40.248 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-14T22:38:40.248 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-14T22:38:40.248 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-14T22:38:40.248 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-14T22:38:40.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.248 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "18", 2024-09-14T22:38:40.248 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-14T22:38:40.248 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:38:40.249 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:38:40.249 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-14T22:38:40.249 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:38:40.249 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 0, 2024-09-14T22:38:40.249 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 0, 2024-09-14T22:38:40.249 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-14T22:38:40.249 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-14T22:38:40.249 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-14T22:38:40.249 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-14T22:38:40.249 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-14T22:38:40.249 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-14T22:38:40.249 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-14T22:38:40.249 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-14T22:38:40.250 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-14T22:38:40.250 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-14T22:38:40.250 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-14T22:38:40.250 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:38:40.250 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:38:40.250 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:38:40.250 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-14T22:38:40.250 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-14T22:38:40.250 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-14T22:38:40.250 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-14T22:38:40.250 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-14T22:38:40.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.250 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-14T22:38:40.251 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-14T22:38:40.251 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-14T22:38:40.251 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:38:40.251 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:38:40.251 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:38:40.251 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:38:40.251 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-14T22:38:40.251 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-14T22:38:40.251 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-14T22:38:40.251 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-14T22:38:40.251 INFO:teuthology.orchestra.run.smithi001.stdout: "options": { 2024-09-14T22:38:40.251 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_max": 32, 2024-09-14T22:38:40.252 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_min": 1 2024-09-14T22:38:40.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.252 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-14T22:38:40.252 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": {} 2024-09-14T22:38:40.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.252 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-14T22:38:40.252 INFO:teuthology.orchestra.run.smithi001.stdout: "score_type": "Fair distribution", 2024-09-14T22:38:40.252 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 8, 2024-09-14T22:38:40.252 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 8, 2024-09-14T22:38:40.252 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 0.25, 2024-09-14T22:38:40.252 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 2, 2024-09-14T22:38:40.252 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 2, 2024-09-14T22:38:40.252 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 0.78881263732910156, 2024-09-14T22:38:40.253 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 0.79000002145767212, 2024-09-14T22:38:40.253 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 0.79000002145767212 2024-09-14T22:38:40.253 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.253 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 2, 2024-09-14T22:38:40.253 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "rbd", 2024-09-14T22:38:40.253 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-14T22:36:04.304242+0000", 2024-09-14T22:38:40.253 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 8193, 2024-09-14T22:38:40.253 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T22:38:40.253 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-14T22:38:40.253 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-14T22:38:40.253 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-14T22:38:40.254 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-14T22:38:40.254 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:38:40.254 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:38:40.254 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:38:40.254 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:38:40.254 INFO:teuthology.orchestra.run.smithi001.stdout: "is_stretch_pool": false, 2024-09-14T22:38:40.254 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-14T22:38:40.254 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:38:40.254 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 8, 2024-09-14T22:38:40.254 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 8, 2024-09-14T22:38:40.254 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 8, 2024-09-14T22:38:40.254 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 8, 2024-09-14T22:38:40.254 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 8, 2024-09-14T22:38:40.255 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-14T22:38:40.255 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-14T22:38:40.255 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-14T22:38:40.255 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-14T22:38:40.255 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-14T22:38:40.255 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-14T22:38:40.255 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-14T22:38:40.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.255 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "22", 2024-09-14T22:38:40.255 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-14T22:38:40.255 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:38:40.255 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:38:40.255 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-14T22:38:40.255 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:38:40.255 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 2, 2024-09-14T22:38:40.256 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 22, 2024-09-14T22:38:40.256 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-14T22:38:40.256 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-14T22:38:40.256 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-14T22:38:40.256 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-14T22:38:40.256 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-14T22:38:40.256 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-14T22:38:40.256 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-14T22:38:40.256 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-14T22:38:40.256 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-14T22:38:40.256 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-14T22:38:40.256 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-14T22:38:40.256 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:38:40.256 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:38:40.256 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:38:40.257 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-14T22:38:40.257 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-14T22:38:40.257 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-14T22:38:40.257 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-14T22:38:40.257 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-14T22:38:40.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.257 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-14T22:38:40.257 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-14T22:38:40.257 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-14T22:38:40.257 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:38:40.257 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:38:40.257 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:38:40.257 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:38:40.257 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-14T22:38:40.258 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-14T22:38:40.258 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-14T22:38:40.258 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-14T22:38:40.258 INFO:teuthology.orchestra.run.smithi001.stdout: "options": {}, 2024-09-14T22:38:40.258 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-14T22:38:40.258 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd": {} 2024-09-14T22:38:40.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.258 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-14T22:38:40.258 INFO:teuthology.orchestra.run.smithi001.stdout: "score_type": "Fair distribution", 2024-09-14T22:38:40.258 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 2, 2024-09-14T22:38:40.258 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 2, 2024-09-14T22:38:40.258 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 1, 2024-09-14T22:38:40.258 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 2, 2024-09-14T22:38:40.258 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 2, 2024-09-14T22:38:40.259 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 0.78881263732910156, 2024-09-14T22:38:40.259 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 0.79000002145767212, 2024-09-14T22:38:40.259 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 0.79000002145767212 2024-09-14T22:38:40.259 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.259 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 3, 2024-09-14T22:38:40.259 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "pool_client.0", 2024-09-14T22:38:40.259 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-14T22:36:14.374249+0000", 2024-09-14T22:38:40.259 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 8193, 2024-09-14T22:38:40.259 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T22:38:40.259 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-14T22:38:40.259 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-14T22:38:40.259 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-14T22:38:40.260 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-14T22:38:40.260 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:38:40.260 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:38:40.260 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:38:40.260 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:38:40.260 INFO:teuthology.orchestra.run.smithi001.stdout: "is_stretch_pool": false, 2024-09-14T22:38:40.260 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-14T22:38:40.260 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:38:40.260 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 32, 2024-09-14T22:38:40.260 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 32, 2024-09-14T22:38:40.260 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 32, 2024-09-14T22:38:40.260 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 32, 2024-09-14T22:38:40.260 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 32, 2024-09-14T22:38:40.261 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-14T22:38:40.261 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-14T22:38:40.261 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-14T22:38:40.261 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-14T22:38:40.261 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-14T22:38:40.261 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-14T22:38:40.261 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-14T22:38:40.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.261 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "147", 2024-09-14T22:38:40.261 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-14T22:38:40.261 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:38:40.261 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:38:40.261 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-14T22:38:40.262 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:38:40.262 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 44, 2024-09-14T22:38:40.262 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 147, 2024-09-14T22:38:40.262 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-14T22:38:40.262 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-14T22:38:40.262 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-14T22:38:40.262 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-14T22:38:40.262 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-14T22:38:40.262 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-14T22:38:40.262 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-14T22:38:40.262 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-14T22:38:40.262 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-14T22:38:40.262 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-14T22:38:40.262 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-14T22:38:40.263 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:38:40.263 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:38:40.263 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:38:40.263 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-14T22:38:40.263 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-14T22:38:40.263 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-14T22:38:40.263 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-14T22:38:40.263 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-14T22:38:40.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.263 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-14T22:38:40.263 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-14T22:38:40.263 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-14T22:38:40.263 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:38:40.263 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:38:40.263 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:38:40.264 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:38:40.264 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-14T22:38:40.264 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-14T22:38:40.264 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-14T22:38:40.264 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-14T22:38:40.264 INFO:teuthology.orchestra.run.smithi001.stdout: "options": {}, 2024-09-14T22:38:40.264 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-14T22:38:40.264 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd": {} 2024-09-14T22:38:40.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.264 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-14T22:38:40.264 INFO:teuthology.orchestra.run.smithi001.stdout: "score_type": "Fair distribution", 2024-09-14T22:38:40.264 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 1.75, 2024-09-14T22:38:40.264 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 1.75, 2024-09-14T22:38:40.264 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 1, 2024-09-14T22:38:40.264 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 1.75, 2024-09-14T22:38:40.265 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 1.75, 2024-09-14T22:38:40.265 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 0.78881263732910156, 2024-09-14T22:38:40.265 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 0.79000002145767212, 2024-09-14T22:38:40.265 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 0.79000002145767212 2024-09-14T22:38:40.265 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.265 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.265 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:38:40.265 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2024-09-14T22:38:40.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.265 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 0, 2024-09-14T22:38:40.265 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "9831fcc5-9040-4a80-99d4-838225e65139", 2024-09-14T22:38:40.265 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:38:40.265 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:38:40.265 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:38:40.265 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:38:40.266 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:38:40.266 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:38:40.266 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:38:40.266 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 115, 2024-09-14T22:38:40.266 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:38:40.266 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:38:40.266 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:38:40.266 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:40.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.266 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:40.266 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6824", 2024-09-14T22:38:40.266 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:38:40.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.266 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:40.267 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6825", 2024-09-14T22:38:40.267 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:38:40.267 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.267 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.267 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:38:40.267 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:40.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.267 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:40.267 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6826", 2024-09-14T22:38:40.267 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:38:40.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.267 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:40.267 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6827", 2024-09-14T22:38:40.268 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:38:40.268 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.268 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.268 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:38:40.268 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:40.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.268 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:40.268 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6830", 2024-09-14T22:38:40.268 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:38:40.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.268 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:40.268 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6831", 2024-09-14T22:38:40.268 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:38:40.269 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.269 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.269 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:38:40.269 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:40.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.269 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:40.269 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6828", 2024-09-14T22:38:40.269 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:38:40.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.269 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:40.269 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6829", 2024-09-14T22:38:40.269 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1702929600 2024-09-14T22:38:40.269 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.269 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.270 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6825/1702929600", 2024-09-14T22:38:40.270 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6827/1702929600", 2024-09-14T22:38:40.270 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6831/1702929600", 2024-09-14T22:38:40.270 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6829/1702929600", 2024-09-14T22:38:40.270 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:38:40.270 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:38:40.270 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:38:40.270 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.270 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 1, 2024-09-14T22:38:40.270 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "7f79abe8-115d-46b2-8677-72fcb0c8ab7d", 2024-09-14T22:38:40.271 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:38:40.271 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:38:40.271 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:38:40.271 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 0.1606292724609375, 2024-09-14T22:38:40.271 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:38:40.271 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:38:40.271 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2024-09-14T22:38:40.271 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 128, 2024-09-14T22:38:40.271 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:38:40.271 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:38:40.271 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:38:40.271 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:40.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.271 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:40.272 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6808", 2024-09-14T22:38:40.272 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:38:40.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.272 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:40.272 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6809", 2024-09-14T22:38:40.272 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:38:40.272 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.272 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.272 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:38:40.272 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:40.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.272 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:40.272 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6810", 2024-09-14T22:38:40.273 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:38:40.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.273 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:40.273 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6811", 2024-09-14T22:38:40.273 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:38:40.273 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.273 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.273 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:38:40.273 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:40.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.273 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:40.274 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6814", 2024-09-14T22:38:40.274 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:38:40.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.274 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:40.274 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6815", 2024-09-14T22:38:40.274 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:38:40.274 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.274 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.274 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:38:40.274 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:40.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.274 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:40.274 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6812", 2024-09-14T22:38:40.275 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:38:40.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.275 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:40.275 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6813", 2024-09-14T22:38:40.275 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 719990653 2024-09-14T22:38:40.275 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.275 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.275 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6809/719990653", 2024-09-14T22:38:40.275 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6811/719990653", 2024-09-14T22:38:40.275 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6815/719990653", 2024-09-14T22:38:40.275 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6813/719990653", 2024-09-14T22:38:40.275 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:38:40.275 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:38:40.275 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:38:40.275 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.276 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 2, 2024-09-14T22:38:40.276 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "1f5768b2-4db4-4b2e-8d5e-5206f8bde28d", 2024-09-14T22:38:40.276 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:38:40.276 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:38:40.276 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:38:40.276 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:38:40.276 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 14, 2024-09-14T22:38:40.276 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 30, 2024-09-14T22:38:40.276 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 66, 2024-09-14T22:38:40.276 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 128, 2024-09-14T22:38:40.276 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 31, 2024-09-14T22:38:40.276 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:38:40.276 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:38:40.277 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:40.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.277 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:40.277 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6816", 2024-09-14T22:38:40.277 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:38:40.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.277 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:40.277 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6817", 2024-09-14T22:38:40.277 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:38:40.277 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.277 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.277 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:38:40.278 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:40.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.278 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:40.278 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6818", 2024-09-14T22:38:40.278 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:38:40.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.278 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:40.278 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6819", 2024-09-14T22:38:40.278 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:38:40.278 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.278 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.279 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:38:40.279 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:40.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.279 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:40.279 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6822", 2024-09-14T22:38:40.279 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:38:40.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.279 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:40.279 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6823", 2024-09-14T22:38:40.279 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:38:40.279 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.279 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.279 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:38:40.280 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:40.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.280 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:40.280 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6820", 2024-09-14T22:38:40.280 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:38:40.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.280 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:40.280 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6821", 2024-09-14T22:38:40.280 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1006730601 2024-09-14T22:38:40.280 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.280 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.280 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6817/1006730601", 2024-09-14T22:38:40.281 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6819/1006730601", 2024-09-14T22:38:40.281 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6823/1006730601", 2024-09-14T22:38:40.281 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6821/1006730601", 2024-09-14T22:38:40.281 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:38:40.281 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:38:40.281 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:38:40.281 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.281 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 3, 2024-09-14T22:38:40.281 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "8103f3c7-7899-4fc3-bb02-dd2273120138", 2024-09-14T22:38:40.281 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:38:40.281 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:38:40.281 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:38:40.282 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 0.852752685546875, 2024-09-14T22:38:40.282 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:38:40.282 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:38:40.282 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:38:40.282 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 128, 2024-09-14T22:38:40.282 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:38:40.282 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:38:40.282 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:38:40.282 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:40.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.282 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:40.282 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6800", 2024-09-14T22:38:40.282 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:38:40.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.283 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:40.283 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6801", 2024-09-14T22:38:40.283 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:38:40.283 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.283 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.283 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:38:40.283 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:40.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.283 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:40.283 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6802", 2024-09-14T22:38:40.283 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:38:40.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.284 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:40.284 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6803", 2024-09-14T22:38:40.284 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:38:40.284 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.284 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.284 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:38:40.284 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:40.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.284 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:40.284 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6806", 2024-09-14T22:38:40.284 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:38:40.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.285 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:40.285 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6807", 2024-09-14T22:38:40.285 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:38:40.285 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.285 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.285 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:38:40.285 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:40.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.285 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:40.285 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6804", 2024-09-14T22:38:40.285 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:38:40.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.286 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:40.286 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6805", 2024-09-14T22:38:40.286 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 750403671 2024-09-14T22:38:40.286 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.286 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.286 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6801/750403671", 2024-09-14T22:38:40.286 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6803/750403671", 2024-09-14T22:38:40.286 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6807/750403671", 2024-09-14T22:38:40.286 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6805/750403671", 2024-09-14T22:38:40.286 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:38:40.286 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:38:40.286 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:38:40.286 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.287 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 4, 2024-09-14T22:38:40.287 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "23cce5cb-726e-44c2-8edb-c2567a89dfbc", 2024-09-14T22:38:40.287 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:38:40.287 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:38:40.287 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:38:40.287 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:38:40.287 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:38:40.287 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:38:40.287 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:38:40.287 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 128, 2024-09-14T22:38:40.287 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:38:40.287 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:38:40.287 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:38:40.287 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:40.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.287 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:40.288 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6816", 2024-09-14T22:38:40.288 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:38:40.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.288 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:40.288 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6817", 2024-09-14T22:38:40.288 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:38:40.288 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.288 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.288 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:38:40.288 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:40.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.288 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:40.288 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6818", 2024-09-14T22:38:40.289 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:38:40.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.289 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:40.289 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6819", 2024-09-14T22:38:40.289 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:38:40.289 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.289 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.289 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:38:40.289 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:40.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.289 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:40.289 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6822", 2024-09-14T22:38:40.290 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:38:40.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.290 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:40.290 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6823", 2024-09-14T22:38:40.290 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:38:40.290 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.290 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.290 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:38:40.290 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:40.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.290 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:40.290 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6820", 2024-09-14T22:38:40.291 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:38:40.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.291 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:40.291 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6821", 2024-09-14T22:38:40.291 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1408471829 2024-09-14T22:38:40.291 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.291 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.291 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.176:6817/1408471829", 2024-09-14T22:38:40.291 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.176:6819/1408471829", 2024-09-14T22:38:40.291 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.176:6823/1408471829", 2024-09-14T22:38:40.291 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.176:6821/1408471829", 2024-09-14T22:38:40.291 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:38:40.292 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:38:40.292 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:38:40.292 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.292 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 5, 2024-09-14T22:38:40.292 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "eb2f187b-d2f1-49ab-ad5e-ea6dc441a28e", 2024-09-14T22:38:40.292 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:38:40.292 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:38:40.292 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:38:40.292 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 0.297119140625, 2024-09-14T22:38:40.292 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:38:40.292 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:38:40.292 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:38:40.292 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 116, 2024-09-14T22:38:40.293 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:38:40.293 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:38:40.293 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:38:40.293 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:40.293 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.293 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:40.293 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6805", 2024-09-14T22:38:40.293 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:38:40.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.293 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.293 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:40.293 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6808", 2024-09-14T22:38:40.293 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:38:40.293 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.293 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.294 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:38:40.294 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:40.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.294 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:40.294 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6810", 2024-09-14T22:38:40.294 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:38:40.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.294 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:40.294 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6811", 2024-09-14T22:38:40.294 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:38:40.294 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.294 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.294 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:38:40.295 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:40.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.295 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:40.295 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6814", 2024-09-14T22:38:40.295 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:38:40.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.295 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:40.295 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6815", 2024-09-14T22:38:40.295 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:38:40.295 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.295 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.295 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:38:40.295 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:40.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.296 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:40.296 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6812", 2024-09-14T22:38:40.296 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:38:40.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.296 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:40.296 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6813", 2024-09-14T22:38:40.296 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1945902326 2024-09-14T22:38:40.296 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.296 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.296 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.176:6808/1945902326", 2024-09-14T22:38:40.296 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.176:6811/1945902326", 2024-09-14T22:38:40.297 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.176:6815/1945902326", 2024-09-14T22:38:40.297 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.176:6813/1945902326", 2024-09-14T22:38:40.297 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:38:40.297 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:38:40.297 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:38:40.297 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.297 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 6, 2024-09-14T22:38:40.297 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "d371aed8-2ad2-49a5-8ece-4b7fd37b4822", 2024-09-14T22:38:40.297 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:38:40.297 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:38:40.297 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:38:40.297 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:38:40.297 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:38:40.298 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:38:40.298 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:38:40.298 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 116, 2024-09-14T22:38:40.298 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:38:40.298 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:38:40.298 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:38:40.298 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:40.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.298 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:40.298 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6800", 2024-09-14T22:38:40.298 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:38:40.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.298 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:40.298 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6801", 2024-09-14T22:38:40.298 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:38:40.299 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.299 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.299 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:38:40.299 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:40.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.299 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:40.299 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6802", 2024-09-14T22:38:40.299 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:38:40.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.299 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:40.299 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6803", 2024-09-14T22:38:40.299 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:38:40.300 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.300 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.300 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:38:40.300 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:40.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.300 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:40.300 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6807", 2024-09-14T22:38:40.300 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:38:40.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.300 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:40.300 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6809", 2024-09-14T22:38:40.300 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:38:40.300 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.301 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.301 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:38:40.301 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:40.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.301 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:40.301 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6804", 2024-09-14T22:38:40.301 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:38:40.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.301 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:40.301 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6806", 2024-09-14T22:38:40.301 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4116085586 2024-09-14T22:38:40.302 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.302 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.302 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.176:6801/4116085586", 2024-09-14T22:38:40.302 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.176:6803/4116085586", 2024-09-14T22:38:40.302 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.176:6809/4116085586", 2024-09-14T22:38:40.302 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.176:6806/4116085586", 2024-09-14T22:38:40.302 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:38:40.302 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:38:40.302 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:38:40.302 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.303 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 7, 2024-09-14T22:38:40.303 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "a012c819-8d49-4310-a840-c2752d9314ab", 2024-09-14T22:38:40.303 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-14T22:38:40.303 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-14T22:38:40.303 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-14T22:38:40.303 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-14T22:38:40.303 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-14T22:38:40.303 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-14T22:38:40.303 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-14T22:38:40.303 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 117, 2024-09-14T22:38:40.303 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-14T22:38:40.303 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-14T22:38:40.304 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-14T22:38:40.304 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:40.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.304 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:40.304 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6824", 2024-09-14T22:38:40.304 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:38:40.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.304 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:40.304 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6825", 2024-09-14T22:38:40.304 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:38:40.304 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.305 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.305 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-14T22:38:40.305 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:40.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.305 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:40.305 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6826", 2024-09-14T22:38:40.305 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:38:40.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.305 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:40.305 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6827", 2024-09-14T22:38:40.305 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:38:40.306 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.306 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.306 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-14T22:38:40.306 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:40.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.306 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:40.306 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6830", 2024-09-14T22:38:40.306 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:38:40.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.306 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:40.306 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6831", 2024-09-14T22:38:40.306 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:38:40.307 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.307 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.307 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-14T22:38:40.307 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-14T22:38:40.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.307 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-14T22:38:40.307 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6828", 2024-09-14T22:38:40.307 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:38:40.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.307 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-14T22:38:40.307 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.176:6829", 2024-09-14T22:38:40.308 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 877760244 2024-09-14T22:38:40.308 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.308 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.308 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.176:6825/877760244", 2024-09-14T22:38:40.308 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.176:6827/877760244", 2024-09-14T22:38:40.308 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.176:6831/877760244", 2024-09-14T22:38:40.308 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.176:6829/877760244", 2024-09-14T22:38:40.308 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-14T22:38:40.308 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-14T22:38:40.308 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-14T22:38:40.308 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.308 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.309 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:38:40.309 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_xinfo": [ 2024-09-14T22:38:40.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.309 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 0, 2024-09-14T22:38:40.309 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:38:40.309 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:38:40.309 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:38:40.309 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:38:40.309 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:38:40.309 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:55.511688+0000", 2024-09-14T22:38:40.309 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:38:40.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.310 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 1, 2024-09-14T22:38:40.310 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:38:40.310 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:38:40.310 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:38:40.310 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:38:40.310 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:38:40.310 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.742414+0000", 2024-09-14T22:38:40.310 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:38:40.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.310 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 2, 2024-09-14T22:38:40.310 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:38:40.310 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:38:40.310 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:38:40.311 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:38:40.311 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:38:40.311 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.871355+0000", 2024-09-14T22:38:40.311 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 31 2024-09-14T22:38:40.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.311 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 3, 2024-09-14T22:38:40.311 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:38:40.311 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:38:40.311 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:38:40.311 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:38:40.311 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:38:40.311 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.504553+0000", 2024-09-14T22:38:40.311 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:38:40.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.312 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 4, 2024-09-14T22:38:40.312 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:38:40.312 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:38:40.312 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:38:40.312 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:38:40.312 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:38:40.312 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.589640+0000", 2024-09-14T22:38:40.312 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:38:40.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.312 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 5, 2024-09-14T22:38:40.312 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:38:40.312 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:38:40.313 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:38:40.313 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:38:40.313 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:38:40.313 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.518845+0000", 2024-09-14T22:38:40.313 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:38:40.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.313 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 6, 2024-09-14T22:38:40.313 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:38:40.313 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:38:40.313 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:38:40.313 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:38:40.313 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:38:40.313 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.731632+0000", 2024-09-14T22:38:40.313 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:38:40.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.314 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 7, 2024-09-14T22:38:40.314 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-14T22:38:40.314 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-14T22:38:40.314 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-14T22:38:40.314 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540701547738038271, 2024-09-14T22:38:40.314 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-14T22:38:40.314 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-14T22:35:54.952634+0000", 2024-09-14T22:38:40.314 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-14T22:38:40.314 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.314 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:38:40.314 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap": [ 2024-09-14T22:38:40.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.315 INFO:teuthology.orchestra.run.smithi001.stdout: "pgid": "2.0", 2024-09-14T22:38:40.315 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2024-09-14T22:38:40.315 INFO:teuthology.orchestra.run.smithi001.stdout: 4, 2024-09-14T22:38:40.315 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-09-14T22:38:40.315 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.315 INFO:teuthology.orchestra.run.smithi001.stdout: "pgid": "3.1d", 2024-09-14T22:38:40.315 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2024-09-14T22:38:40.315 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-09-14T22:38:40.315 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-09-14T22:38:40.315 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.315 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.315 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:38:40.315 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap_items": [ 2024-09-14T22:38:40.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.316 INFO:teuthology.orchestra.run.smithi001.stdout: "pgid": "3.9", 2024-09-14T22:38:40.316 INFO:teuthology.orchestra.run.smithi001.stdout: "mappings": [ 2024-09-14T22:38:40.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.316 INFO:teuthology.orchestra.run.smithi001.stdout: "from": 2, 2024-09-14T22:38:40.316 INFO:teuthology.orchestra.run.smithi001.stdout: "to": 4 2024-09-14T22:38:40.316 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.316 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.316 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.316 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:38:40.316 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap_primaries": [], 2024-09-14T22:38:40.316 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_temp": [], 2024-09-14T22:38:40.316 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_temp": [], 2024-09-14T22:38:40.316 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklist": {}, 2024-09-14T22:38:40.316 INFO:teuthology.orchestra.run.smithi001.stdout: "range_blocklist": {}, 2024-09-14T22:38:40.317 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profiles": { 2024-09-14T22:38:40.317 INFO:teuthology.orchestra.run.smithi001.stdout: "default": { 2024-09-14T22:38:40.317 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-failure-domain": "osd", 2024-09-14T22:38:40.317 INFO:teuthology.orchestra.run.smithi001.stdout: "k": "2", 2024-09-14T22:38:40.317 INFO:teuthology.orchestra.run.smithi001.stdout: "m": "1", 2024-09-14T22:38:40.317 INFO:teuthology.orchestra.run.smithi001.stdout: "plugin": "jerasure", 2024-09-14T22:38:40.317 INFO:teuthology.orchestra.run.smithi001.stdout: "technique": "reed_sol_van" 2024-09-14T22:38:40.317 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:40.317 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps_queue": [ 2024-09-14T22:38:40.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.317 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 3, 2024-09-14T22:38:40.317 INFO:teuthology.orchestra.run.smithi001.stdout: "snaps": [ 2024-09-14T22:38:40.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.317 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 24, 2024-09-14T22:38:40.318 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 13 2024-09-14T22:38:40.318 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.318 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.318 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.318 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:38:40.318 INFO:teuthology.orchestra.run.smithi001.stdout: "new_removed_snaps": [ 2024-09-14T22:38:40.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.318 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 3, 2024-09-14T22:38:40.318 INFO:teuthology.orchestra.run.smithi001.stdout: "snaps": [ 2024-09-14T22:38:40.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:40.318 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 24, 2024-09-14T22:38:40.318 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2024-09-14T22:38:40.318 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.318 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:40.318 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.319 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:38:40.319 INFO:teuthology.orchestra.run.smithi001.stdout: "new_purged_snaps": [], 2024-09-14T22:38:40.319 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_node_flags": {}, 2024-09-14T22:38:40.319 INFO:teuthology.orchestra.run.smithi001.stdout: "device_class_flags": {}, 2024-09-14T22:38:40.319 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": { 2024-09-14T22:38:40.319 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode_enabled": false, 2024-09-14T22:38:40.319 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_bucket_count": 0, 2024-09-14T22:38:40.319 INFO:teuthology.orchestra.run.smithi001.stdout: "degraded_stretch_mode": 0, 2024-09-14T22:38:40.319 INFO:teuthology.orchestra.run.smithi001.stdout: "recovering_stretch_mode": 0, 2024-09-14T22:38:40.319 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode_bucket": 0 2024-09-14T22:38:40.319 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:40.319 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:40.320 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:38:40.321 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:40.238+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:40.322 INFO:tasks.thrashosds.thrasher:j is {'epoch': 147, 'fsid': 'f07f407a-b17d-42c4-bf73-92dd80114a4f', 'created': '2024-09-14T22:35:45.983055+0000', 'modified': '2024-09-14T22:38:39.830496+0000', 'last_up_change': '2024-09-14T22:37:12.616042+0000', 'last_in_change': '2024-09-14T22:35:48.283160+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T22:35:58.938784+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7888126373291016, 'average_primary_affinity': 0.7900000214576721, 'average_primary_affinity_weighted': 0.7900000214576721}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T22:36:04.304242+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7888126373291016, 'average_primary_affinity': 0.7900000214576721, 'average_primary_affinity_weighted': 0.7900000214576721}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T22:36:14.374249+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '147', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 44, 'snap_epoch': 147, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.7888126373291016, 'average_primary_affinity': 0.7900000214576721, 'average_primary_affinity_weighted': 0.7900000214576721}}], 'osds': [{'osd': 0, 'uuid': '9831fcc5-9040-4a80-99d4-838225e65139', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 115, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6824', 'nonce': 1702929600}, {'type': 'v1', 'addr': '172.21.15.1:6825', 'nonce': 1702929600}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6826', 'nonce': 1702929600}, {'type': 'v1', 'addr': '172.21.15.1:6827', 'nonce': 1702929600}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6830', 'nonce': 1702929600}, {'type': 'v1', 'addr': '172.21.15.1:6831', 'nonce': 1702929600}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6828', 'nonce': 1702929600}, {'type': 'v1', 'addr': '172.21.15.1:6829', 'nonce': 1702929600}]}, 'public_addr': '172.21.15.1:6825/1702929600', 'cluster_addr': '172.21.15.1:6827/1702929600', 'heartbeat_back_addr': '172.21.15.1:6831/1702929600', 'heartbeat_front_addr': '172.21.15.1:6829/1702929600', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '7f79abe8-115d-46b2-8677-72fcb0c8ab7d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.1606292724609375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 128, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6808', 'nonce': 719990653}, {'type': 'v1', 'addr': '172.21.15.1:6809', 'nonce': 719990653}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6810', 'nonce': 719990653}, {'type': 'v1', 'addr': '172.21.15.1:6811', 'nonce': 719990653}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6814', 'nonce': 719990653}, {'type': 'v1', 'addr': '172.21.15.1:6815', 'nonce': 719990653}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6812', 'nonce': 719990653}, {'type': 'v1', 'addr': '172.21.15.1:6813', 'nonce': 719990653}]}, 'public_addr': '172.21.15.1:6809/719990653', 'cluster_addr': '172.21.15.1:6811/719990653', 'heartbeat_back_addr': '172.21.15.1:6815/719990653', 'heartbeat_front_addr': '172.21.15.1:6813/719990653', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '1f5768b2-4db4-4b2e-8d5e-5206f8bde28d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 14, 'last_clean_end': 30, 'up_from': 66, 'up_thru': 128, 'down_at': 31, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6816', 'nonce': 1006730601}, {'type': 'v1', 'addr': '172.21.15.1:6817', 'nonce': 1006730601}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6818', 'nonce': 1006730601}, {'type': 'v1', 'addr': '172.21.15.1:6819', 'nonce': 1006730601}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6822', 'nonce': 1006730601}, {'type': 'v1', 'addr': '172.21.15.1:6823', 'nonce': 1006730601}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6820', 'nonce': 1006730601}, {'type': 'v1', 'addr': '172.21.15.1:6821', 'nonce': 1006730601}]}, 'public_addr': '172.21.15.1:6817/1006730601', 'cluster_addr': '172.21.15.1:6819/1006730601', 'heartbeat_back_addr': '172.21.15.1:6823/1006730601', 'heartbeat_front_addr': '172.21.15.1:6821/1006730601', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '8103f3c7-7899-4fc3-bb02-dd2273120138', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.852752685546875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 128, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6800', 'nonce': 750403671}, {'type': 'v1', 'addr': '172.21.15.1:6801', 'nonce': 750403671}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6802', 'nonce': 750403671}, {'type': 'v1', 'addr': '172.21.15.1:6803', 'nonce': 750403671}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6806', 'nonce': 750403671}, {'type': 'v1', 'addr': '172.21.15.1:6807', 'nonce': 750403671}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6804', 'nonce': 750403671}, {'type': 'v1', 'addr': '172.21.15.1:6805', 'nonce': 750403671}]}, 'public_addr': '172.21.15.1:6801/750403671', 'cluster_addr': '172.21.15.1:6803/750403671', 'heartbeat_back_addr': '172.21.15.1:6807/750403671', 'heartbeat_front_addr': '172.21.15.1:6805/750403671', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '23cce5cb-726e-44c2-8edb-c2567a89dfbc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 128, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6816', 'nonce': 1408471829}, {'type': 'v1', 'addr': '172.21.15.176:6817', 'nonce': 1408471829}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6818', 'nonce': 1408471829}, {'type': 'v1', 'addr': '172.21.15.176:6819', 'nonce': 1408471829}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6822', 'nonce': 1408471829}, {'type': 'v1', 'addr': '172.21.15.176:6823', 'nonce': 1408471829}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6820', 'nonce': 1408471829}, {'type': 'v1', 'addr': '172.21.15.176:6821', 'nonce': 1408471829}]}, 'public_addr': '172.21.15.176:6817/1408471829', 'cluster_addr': '172.21.15.176:6819/1408471829', 'heartbeat_back_addr': '172.21.15.176:6823/1408471829', 'heartbeat_front_addr': '172.21.15.176:6821/1408471829', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'eb2f187b-d2f1-49ab-ad5e-ea6dc441a28e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.297119140625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 116, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6805', 'nonce': 1945902326}, {'type': 'v1', 'addr': '172.21.15.176:6808', 'nonce': 1945902326}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6810', 'nonce': 1945902326}, {'type': 'v1', 'addr': '172.21.15.176:6811', 'nonce': 1945902326}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6814', 'nonce': 1945902326}, {'type': 'v1', 'addr': '172.21.15.176:6815', 'nonce': 1945902326}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6812', 'nonce': 1945902326}, {'type': 'v1', 'addr': '172.21.15.176:6813', 'nonce': 1945902326}]}, 'public_addr': '172.21.15.176:6808/1945902326', 'cluster_addr': '172.21.15.176:6811/1945902326', 'heartbeat_back_addr': '172.21.15.176:6815/1945902326', 'heartbeat_front_addr': '172.21.15.176:6813/1945902326', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'd371aed8-2ad2-49a5-8ece-4b7fd37b4822', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 116, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6800', 'nonce': 4116085586}, {'type': 'v1', 'addr': '172.21.15.176:6801', 'nonce': 4116085586}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6802', 'nonce': 4116085586}, {'type': 'v1', 'addr': '172.21.15.176:6803', 'nonce': 4116085586}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6807', 'nonce': 4116085586}, {'type': 'v1', 'addr': '172.21.15.176:6809', 'nonce': 4116085586}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6804', 'nonce': 4116085586}, {'type': 'v1', 'addr': '172.21.15.176:6806', 'nonce': 4116085586}]}, 'public_addr': '172.21.15.176:6801/4116085586', 'cluster_addr': '172.21.15.176:6803/4116085586', 'heartbeat_back_addr': '172.21.15.176:6809/4116085586', 'heartbeat_front_addr': '172.21.15.176:6806/4116085586', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'a012c819-8d49-4310-a840-c2752d9314ab', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 117, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6824', 'nonce': 877760244}, {'type': 'v1', 'addr': '172.21.15.176:6825', 'nonce': 877760244}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6826', 'nonce': 877760244}, {'type': 'v1', 'addr': '172.21.15.176:6827', 'nonce': 877760244}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6830', 'nonce': 877760244}, {'type': 'v1', 'addr': '172.21.15.176:6831', 'nonce': 877760244}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6828', 'nonce': 877760244}, {'type': 'v1', 'addr': '172.21.15.176:6829', 'nonce': 877760244}]}, 'public_addr': '172.21.15.176:6825/877760244', 'cluster_addr': '172.21.15.176:6827/877760244', 'heartbeat_back_addr': '172.21.15.176:6831/877760244', 'heartbeat_front_addr': '172.21.15.176:6829/877760244', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:55.511688+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.742414+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.871355+0000', 'dead_epoch': 31}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.504553+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.589640+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.518845+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.731632+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:35:54.952634+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.0', 'osds': [4, 2]}, {'pgid': '3.1d', 'osds': [1, 2]}], 'pg_upmap_items': [{'pgid': '3.9', 'mappings': [{'from': 2, 'to': 4}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 24, 'length': 13}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 24, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T22:38:40.322 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.9 2024-09-14T22:38:40.322 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap-items 3.9 2024-09-14T22:38:40.341 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:38:40.366 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:40.365+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:40.445 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:40.441+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:40.493 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:40.493 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:40.493 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:38:40.493 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:40.507 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:38:40.509 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:38:40.509 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:38:40.526 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:38:40.526 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:38:40.543 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:38:40.543 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:38:40.546 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:40.545+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:40.560 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:38:40.560 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:38:40.578 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:38:40.578 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:38:40.596 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:38:40.596 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:38:40.616 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:38:40.616 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:38:40.625 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:40.625 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:40.636 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:38:40.636 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:38:40.639 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:38:40.647 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:40.645+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:40.726 INFO:teuthology.orchestra.run.smithi001.stderr:clear 3.9 pg_upmap_items mapping 2024-09-14T22:38:40.748 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:40.745+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:40.767 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:40.767 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:40.782 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:38:40.849 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:40.845+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:40.855 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T22:38:40.906 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:40.906 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:40.920 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:38:40.949 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:40.945+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:41.045 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:41.046 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:41.050 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:41.042+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:41.060 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:38:41.151 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:41.149+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:41.226 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:41.226 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:41.240 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:38:41.252 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:41.246+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:41.352 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:41.349+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:41.371 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:41.371 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:41.385 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:38:41.474 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:41.473+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:41.555 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:41.550+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:41.558 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:41.558 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:41.573 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:38:41.656 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:41.650+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:41.742 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:41.742 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:41.756 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:38:41.757 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:41.753+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:41.854 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2024-09-14T22:38:41.858 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:41.857+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:41.903 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:41.903 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:41.920 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:38:41.962 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:41.961+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:42.060 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:42.054+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:42.101 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:42.101 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:42.116 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:38:42.160 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:42.154+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:42.241 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:42.241 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:42.255 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:38:42.261 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:42.254+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:42.362 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:42.358+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:42.374 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:42.374 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:42.388 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:38:42.463 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:42.461+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:42.516 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:42.516 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:42.532 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:38:42.564 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:42.558+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:42.655 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:38:42.665 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:42.658+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:42.702 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:42.702 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:42.718 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:38:42.766 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:42.765+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:42.803 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:38:42.803 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:38:42.820 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:38:42.820 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:38:42.837 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:38:42.838 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:38:42.852 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:42.852 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:42.856 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:38:42.856 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:38:42.867 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:42.865+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:42.874 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:38:42.885 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:38:42.885 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:38:42.908 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:38:42.908 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:38:42.934 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:38:42.934 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:38:42.959 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:38:42.959 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:38:42.968 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:42.962+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:43.020 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:43.020 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:43.020 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:38:43.020 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:38:43.020 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:43.036 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:38:43.068 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:43.062+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:43.169 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:43.165+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:43.196 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:43.196 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:38:43.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:38:43.197 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:38:43.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.197 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145249 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 15745024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:43.197 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.693850+0000", 2024-09-14T22:38:43.197 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.499608696000003, 2024-09-14T22:38:43.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.060850297999999997, 2024-09-14T22:38:43.197 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:43.197 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:43.197 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:43.197 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:43.197 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:43.197 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.197 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145249 2024-09-14T22:38:43.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.198 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:43.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:43.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693850+0000", 2024-09-14T22:38:43.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:43.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693850+0000", 2024-09-14T22:38:43.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:43.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693851+0000", 2024-09-14T22:38:43.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.46e-07 2024-09-14T22:38:43.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:43.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693853+0000", 2024-09-14T22:38:43.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0940000000000002e-06 2024-09-14T22:38:43.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:43.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693858+0000", 2024-09-14T22:38:43.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2100000000000001e-06 2024-09-14T22:38:43.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693863+0000", 2024-09-14T22:38:43.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5549999999999996e-06 2024-09-14T22:38:43.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:43.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754503+0000", 2024-09-14T22:38:43.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.060640558999999997 2024-09-14T22:38:43.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:43.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754613+0000", 2024-09-14T22:38:43.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109966 2024-09-14T22:38:43.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:43.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754701+0000", 2024-09-14T22:38:43.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7567999999999999e-05 2024-09-14T22:38:43.201 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.201 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:43.201 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.201 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145250 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16039936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:43.201 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.693868+0000", 2024-09-14T22:38:43.201 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.499590968, 2024-09-14T22:38:43.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061182667000000003, 2024-09-14T22:38:43.201 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:43.201 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:43.201 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:43.202 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:43.202 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:43.202 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.202 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145250 2024-09-14T22:38:43.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.202 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:43.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:43.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693868+0000", 2024-09-14T22:38:43.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:43.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693868+0000", 2024-09-14T22:38:43.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:43.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693869+0000", 2024-09-14T22:38:43.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4400000000000001e-07 2024-09-14T22:38:43.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:43.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693870+0000", 2024-09-14T22:38:43.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9130000000000001e-06 2024-09-14T22:38:43.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:43.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693876+0000", 2024-09-14T22:38:43.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1270000000000002e-06 2024-09-14T22:38:43.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693880+0000", 2024-09-14T22:38:43.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.673e-06 2024-09-14T22:38:43.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:43.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754780+0000", 2024-09-14T22:38:43.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.060899495999999997 2024-09-14T22:38:43.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:43.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754925+0000", 2024-09-14T22:38:43.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014511799999999999 2024-09-14T22:38:43.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:43.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755051+0000", 2024-09-14T22:38:43.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012599599999999999 2024-09-14T22:38:43.205 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.205 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:43.205 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.205 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145251 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16236544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:43.205 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.693886+0000", 2024-09-14T22:38:43.205 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.499573314999999, 2024-09-14T22:38:43.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061498816999999997, 2024-09-14T22:38:43.205 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:43.205 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:43.206 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:43.206 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:43.206 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:43.206 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.206 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145251 2024-09-14T22:38:43.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.206 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:43.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.206 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:43.206 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693886+0000", 2024-09-14T22:38:43.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.206 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:43.206 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693886+0000", 2024-09-14T22:38:43.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:43.207 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693886+0000", 2024-09-14T22:38:43.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1199999999999998e-07 2024-09-14T22:38:43.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:43.207 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693888+0000", 2024-09-14T22:38:43.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.877e-06 2024-09-14T22:38:43.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:43.207 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693894+0000", 2024-09-14T22:38:43.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5350000000000004e-06 2024-09-14T22:38:43.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693917+0000", 2024-09-14T22:38:43.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.296e-05 2024-09-14T22:38:43.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:43.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755127+0000", 2024-09-14T22:38:43.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061210286000000003 2024-09-14T22:38:43.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:43.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755266+0000", 2024-09-14T22:38:43.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013871599999999999 2024-09-14T22:38:43.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:43.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755385+0000", 2024-09-14T22:38:43.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011903100000000001 2024-09-14T22:38:43.209 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.209 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:43.209 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.209 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145252 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16302080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:43.209 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.693985+0000", 2024-09-14T22:38:43.209 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.499474632999998, 2024-09-14T22:38:43.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061743767999999997, 2024-09-14T22:38:43.209 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:43.209 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:43.210 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:43.210 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:43.210 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:43.210 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.210 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145252 2024-09-14T22:38:43.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.210 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:43.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:43.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693985+0000", 2024-09-14T22:38:43.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:43.211 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693985+0000", 2024-09-14T22:38:43.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.211 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.211 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:43.211 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693986+0000", 2024-09-14T22:38:43.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0640000000000001e-06 2024-09-14T22:38:43.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.211 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.211 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:43.212 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693991+0000", 2024-09-14T22:38:43.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7599999999999999e-06 2024-09-14T22:38:43.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.212 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:43.212 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694005+0000", 2024-09-14T22:38:43.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3271999999999999e-05 2024-09-14T22:38:43.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.212 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.212 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694015+0000", 2024-09-14T22:38:43.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0617e-05 2024-09-14T22:38:43.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.212 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:43.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755459+0000", 2024-09-14T22:38:43.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061443820000000003 2024-09-14T22:38:43.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:43.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755605+0000", 2024-09-14T22:38:43.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014569 2024-09-14T22:38:43.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:43.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755728+0000", 2024-09-14T22:38:43.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000123545 2024-09-14T22:38:43.216 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.216 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:43.216 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.216 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145253 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16138240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:43.216 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694031+0000", 2024-09-14T22:38:43.216 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.499428078999998, 2024-09-14T22:38:43.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062032836000000001, 2024-09-14T22:38:43.217 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:43.217 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:43.217 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:43.217 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:43.217 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:43.217 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.217 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145253 2024-09-14T22:38:43.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.217 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:43.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.217 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:43.217 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694031+0000", 2024-09-14T22:38:43.217 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:43.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694031+0000", 2024-09-14T22:38:43.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:43.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694032+0000", 2024-09-14T22:38:43.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7999999999999995e-07 2024-09-14T22:38:43.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:43.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694036+0000", 2024-09-14T22:38:43.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8800000000000001e-06 2024-09-14T22:38:43.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:43.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694047+0000", 2024-09-14T22:38:43.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0835999999999999e-05 2024-09-14T22:38:43.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694055+0000", 2024-09-14T22:38:43.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6619999999999994e-06 2024-09-14T22:38:43.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:43.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755804+0000", 2024-09-14T22:38:43.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061748448999999997 2024-09-14T22:38:43.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:43.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755941+0000", 2024-09-14T22:38:43.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013751599999999999 2024-09-14T22:38:43.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:43.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756064+0000", 2024-09-14T22:38:43.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012281299999999999 2024-09-14T22:38:43.220 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.220 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:43.220 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.221 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145254 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16400384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:43.221 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694067+0000", 2024-09-14T22:38:43.221 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.499391914, 2024-09-14T22:38:43.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062280549999999997, 2024-09-14T22:38:43.221 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:43.221 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:43.221 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:43.221 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:43.221 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:43.221 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.221 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145254 2024-09-14T22:38:43.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.221 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:43.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:43.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694067+0000", 2024-09-14T22:38:43.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:43.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694067+0000", 2024-09-14T22:38:43.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:43.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694068+0000", 2024-09-14T22:38:43.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2200000000000003e-07 2024-09-14T22:38:43.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:43.223 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694072+0000", 2024-09-14T22:38:43.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5729999999999999e-06 2024-09-14T22:38:43.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:43.223 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694082+0000", 2024-09-14T22:38:43.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0703999999999999e-05 2024-09-14T22:38:43.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.223 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694091+0000", 2024-09-14T22:38:43.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8030000000000003e-06 2024-09-14T22:38:43.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:43.224 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756155+0000", 2024-09-14T22:38:43.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062063937 2024-09-14T22:38:43.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.224 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:43.224 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756263+0000", 2024-09-14T22:38:43.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108229 2024-09-14T22:38:43.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.224 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:43.224 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756348+0000", 2024-09-14T22:38:43.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4581999999999998e-05 2024-09-14T22:38:43.224 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.224 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:43.225 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.225 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145255 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16596992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:43.225 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694103+0000", 2024-09-14T22:38:43.225 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.499356433000003, 2024-09-14T22:38:43.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062580620000000003, 2024-09-14T22:38:43.225 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:43.225 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:43.225 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:43.225 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:43.225 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:43.225 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.225 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145255 2024-09-14T22:38:43.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.225 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:43.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:43.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694103+0000", 2024-09-14T22:38:43.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:43.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694103+0000", 2024-09-14T22:38:43.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:43.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694103+0000", 2024-09-14T22:38:43.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7800000000000001e-07 2024-09-14T22:38:43.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:43.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694107+0000", 2024-09-14T22:38:43.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5939999999999998e-06 2024-09-14T22:38:43.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:43.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694121+0000", 2024-09-14T22:38:43.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3913999999999999e-05 2024-09-14T22:38:43.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694130+0000", 2024-09-14T22:38:43.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7690000000000002e-06 2024-09-14T22:38:43.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.228 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:43.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756431+0000", 2024-09-14T22:38:43.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062301593000000002 2024-09-14T22:38:43.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.228 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:43.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756564+0000", 2024-09-14T22:38:43.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000132693 2024-09-14T22:38:43.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.228 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:43.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756683+0000", 2024-09-14T22:38:43.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119479 2024-09-14T22:38:43.229 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.229 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:43.229 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.229 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145256 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16695296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:43.229 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694142+0000", 2024-09-14T22:38:43.229 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.499317359999999, 2024-09-14T22:38:43.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062806034999999996, 2024-09-14T22:38:43.229 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:43.229 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:43.229 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:43.229 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:43.229 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:43.229 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.230 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145256 2024-09-14T22:38:43.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.230 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:43.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:43.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694142+0000", 2024-09-14T22:38:43.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:43.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694142+0000", 2024-09-14T22:38:43.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:43.231 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694142+0000", 2024-09-14T22:38:43.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1699999999999998e-07 2024-09-14T22:38:43.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:43.231 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694146+0000", 2024-09-14T22:38:43.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7450000000000001e-06 2024-09-14T22:38:43.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:43.231 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694157+0000", 2024-09-14T22:38:43.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1161e-05 2024-09-14T22:38:43.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694166+0000", 2024-09-14T22:38:43.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0450000000000006e-06 2024-09-14T22:38:43.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:43.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756768+0000", 2024-09-14T22:38:43.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062601554000000004 2024-09-14T22:38:43.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:43.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756867+0000", 2024-09-14T22:38:43.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8741999999999998e-05 2024-09-14T22:38:43.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:43.233 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756948+0000", 2024-09-14T22:38:43.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1170999999999999e-05 2024-09-14T22:38:43.233 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.233 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:43.233 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.233 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145257 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16498688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:43.233 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694179+0000", 2024-09-14T22:38:43.233 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.499280061, 2024-09-14T22:38:43.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063118469999999996, 2024-09-14T22:38:43.233 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:43.233 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:43.234 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:43.234 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:43.234 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:43.234 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.234 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145257 2024-09-14T22:38:43.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.234 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:43.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:43.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694179+0000", 2024-09-14T22:38:43.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:43.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694179+0000", 2024-09-14T22:38:43.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:43.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694180+0000", 2024-09-14T22:38:43.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1099999999999995e-07 2024-09-14T22:38:43.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:43.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694196+0000", 2024-09-14T22:38:43.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5996000000000001e-05 2024-09-14T22:38:43.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:43.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694208+0000", 2024-09-14T22:38:43.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2584e-05 2024-09-14T22:38:43.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.236 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694219+0000", 2024-09-14T22:38:43.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0462000000000001e-05 2024-09-14T22:38:43.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:43.236 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757033+0000", 2024-09-14T22:38:43.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062814485000000003 2024-09-14T22:38:43.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:43.236 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757183+0000", 2024-09-14T22:38:43.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014914099999999999 2024-09-14T22:38:43.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:43.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757298+0000", 2024-09-14T22:38:43.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115091 2024-09-14T22:38:43.237 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.237 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:43.237 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.237 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145258 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16269312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:43.237 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694257+0000", 2024-09-14T22:38:43.237 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.499201866999996, 2024-09-14T22:38:43.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063297279999999997, 2024-09-14T22:38:43.237 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:43.238 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:43.238 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:43.238 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:43.238 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:43.238 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.238 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145258 2024-09-14T22:38:43.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.238 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:43.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:43.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694257+0000", 2024-09-14T22:38:43.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:43.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694257+0000", 2024-09-14T22:38:43.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:43.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694258+0000", 2024-09-14T22:38:43.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7700000000000004e-07 2024-09-14T22:38:43.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:43.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694263+0000", 2024-09-14T22:38:43.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6609999999999999e-06 2024-09-14T22:38:43.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:43.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694271+0000", 2024-09-14T22:38:43.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7809999999999994e-06 2024-09-14T22:38:43.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694277+0000", 2024-09-14T22:38:43.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.823e-06 2024-09-14T22:38:43.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:43.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757377+0000", 2024-09-14T22:38:43.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063100081000000002 2024-09-14T22:38:43.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:43.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757476+0000", 2024-09-14T22:38:43.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8507999999999999e-05 2024-09-14T22:38:43.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:43.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757555+0000", 2024-09-14T22:38:43.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8749000000000001e-05 2024-09-14T22:38:43.241 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.241 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:43.241 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.242 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145259 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16465920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:43.242 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694286+0000", 2024-09-14T22:38:43.242 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.499173669999998, 2024-09-14T22:38:43.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063608168000000007, 2024-09-14T22:38:43.242 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:43.242 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:43.242 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:43.242 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:43.242 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:43.242 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.242 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145259 2024-09-14T22:38:43.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.242 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:43.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.243 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:43.243 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694286+0000", 2024-09-14T22:38:43.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.243 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:43.243 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694286+0000", 2024-09-14T22:38:43.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.243 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:43.243 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694286+0000", 2024-09-14T22:38:43.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9999999999999998e-07 2024-09-14T22:38:43.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:43.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694288+0000", 2024-09-14T22:38:43.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3760000000000002e-06 2024-09-14T22:38:43.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:43.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694294+0000", 2024-09-14T22:38:43.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6779999999999997e-06 2024-09-14T22:38:43.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694298+0000", 2024-09-14T22:38:43.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3309999999999997e-06 2024-09-14T22:38:43.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:43.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757638+0000", 2024-09-14T22:38:43.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063339464999999998 2024-09-14T22:38:43.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:43.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757769+0000", 2024-09-14T22:38:43.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013148700000000001 2024-09-14T22:38:43.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:43.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757894+0000", 2024-09-14T22:38:43.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012443099999999999 2024-09-14T22:38:43.245 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.245 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:43.245 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.246 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145260 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16367616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:43.246 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694304+0000", 2024-09-14T22:38:43.246 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.499154875999999, 2024-09-14T22:38:43.246 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063838521999999995, 2024-09-14T22:38:43.246 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:43.246 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:43.246 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:43.246 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:43.246 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:43.246 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.246 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145260 2024-09-14T22:38:43.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.246 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:43.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:43.247 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694304+0000", 2024-09-14T22:38:43.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:43.247 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694304+0000", 2024-09-14T22:38:43.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:43.247 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694305+0000", 2024-09-14T22:38:43.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8299999999999997e-07 2024-09-14T22:38:43.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:43.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694307+0000", 2024-09-14T22:38:43.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9989999999999998e-06 2024-09-14T22:38:43.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:43.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694312+0000", 2024-09-14T22:38:43.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2689999999999999e-06 2024-09-14T22:38:43.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694317+0000", 2024-09-14T22:38:43.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9359999999999998e-06 2024-09-14T22:38:43.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:43.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757944+0000", 2024-09-14T22:38:43.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063626951000000001 2024-09-14T22:38:43.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:43.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758065+0000", 2024-09-14T22:38:43.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120945 2024-09-14T22:38:43.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:43.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758143+0000", 2024-09-14T22:38:43.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7838999999999998e-05 2024-09-14T22:38:43.249 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.249 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:43.250 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.250 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145261 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16564224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:43.250 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694323+0000", 2024-09-14T22:38:43.250 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.499136172999997, 2024-09-14T22:38:43.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064150180000000001, 2024-09-14T22:38:43.250 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:43.250 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:43.250 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:43.250 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:43.250 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:43.250 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.250 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145261 2024-09-14T22:38:43.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.251 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:43.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.251 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:43.251 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694323+0000", 2024-09-14T22:38:43.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.251 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:43.251 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694323+0000", 2024-09-14T22:38:43.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.251 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:43.251 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694323+0000", 2024-09-14T22:38:43.252 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4900000000000001e-07 2024-09-14T22:38:43.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.252 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:43.252 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694325+0000", 2024-09-14T22:38:43.252 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.04e-06 2024-09-14T22:38:43.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.252 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:43.252 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694331+0000", 2024-09-14T22:38:43.252 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3449999999999996e-06 2024-09-14T22:38:43.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694335+0000", 2024-09-14T22:38:43.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.566e-06 2024-09-14T22:38:43.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:43.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758225+0000", 2024-09-14T22:38:43.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063889535999999997 2024-09-14T22:38:43.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:43.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758357+0000", 2024-09-14T22:38:43.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013181 2024-09-14T22:38:43.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:43.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758473+0000", 2024-09-14T22:38:43.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011653400000000001 2024-09-14T22:38:43.254 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.254 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:43.254 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.254 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145262 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16760832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:43.254 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694341+0000", 2024-09-14T22:38:43.255 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.499118314999997, 2024-09-14T22:38:43.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064424631999999996, 2024-09-14T22:38:43.255 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:43.255 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:43.255 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:43.255 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:43.255 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:43.255 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.255 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145262 2024-09-14T22:38:43.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.255 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:43.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:43.255 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694341+0000", 2024-09-14T22:38:43.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:43.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694341+0000", 2024-09-14T22:38:43.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:43.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694341+0000", 2024-09-14T22:38:43.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5499999999999999e-07 2024-09-14T22:38:43.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:43.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694343+0000", 2024-09-14T22:38:43.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9630000000000001e-06 2024-09-14T22:38:43.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:43.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694349+0000", 2024-09-14T22:38:43.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3120000000000001e-06 2024-09-14T22:38:43.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694353+0000", 2024-09-14T22:38:43.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6500000000000004e-06 2024-09-14T22:38:43.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:43.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758561+0000", 2024-09-14T22:38:43.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064207934999999994 2024-09-14T22:38:43.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:43.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758677+0000", 2024-09-14T22:38:43.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116241 2024-09-14T22:38:43.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:43.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758766+0000", 2024-09-14T22:38:43.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8176000000000004e-05 2024-09-14T22:38:43.258 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.258 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:43.258 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.258 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145263 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16662528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:43.258 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694359+0000", 2024-09-14T22:38:43.259 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.499100472000002, 2024-09-14T22:38:43.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064743189000000007, 2024-09-14T22:38:43.259 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:43.259 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:43.259 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:43.259 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:43.259 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:43.259 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.259 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145263 2024-09-14T22:38:43.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.259 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:43.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.259 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:43.259 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694359+0000", 2024-09-14T22:38:43.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.260 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:43.260 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694359+0000", 2024-09-14T22:38:43.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.260 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:43.260 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694359+0000", 2024-09-14T22:38:43.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5100000000000001e-07 2024-09-14T22:38:43.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.260 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:43.260 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694361+0000", 2024-09-14T22:38:43.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9429999999999999e-06 2024-09-14T22:38:43.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.261 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:43.261 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694366+0000", 2024-09-14T22:38:43.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3489999999999999e-06 2024-09-14T22:38:43.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.261 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.261 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694371+0000", 2024-09-14T22:38:43.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4399999999999998e-06 2024-09-14T22:38:43.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:43.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758848+0000", 2024-09-14T22:38:43.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064477396000000006 2024-09-14T22:38:43.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:43.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758980+0000", 2024-09-14T22:38:43.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013162199999999999 2024-09-14T22:38:43.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:43.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759102+0000", 2024-09-14T22:38:43.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012208799999999999 2024-09-14T22:38:43.427 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.427 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:43.427 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.428 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145264 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16334848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:43.428 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694377+0000", 2024-09-14T22:38:43.428 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.499082438000002, 2024-09-14T22:38:43.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065048054999999994, 2024-09-14T22:38:43.428 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:43.428 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:43.428 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:43.428 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:43.428 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:43.428 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.428 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145264 2024-09-14T22:38:43.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.428 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:43.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:43.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694377+0000", 2024-09-14T22:38:43.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:43.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694377+0000", 2024-09-14T22:38:43.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:43.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694377+0000", 2024-09-14T22:38:43.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.39e-07 2024-09-14T22:38:43.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:43.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694379+0000", 2024-09-14T22:38:43.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0190000000000001e-06 2024-09-14T22:38:43.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:43.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694385+0000", 2024-09-14T22:38:43.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4399999999999996e-06 2024-09-14T22:38:43.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694389+0000", 2024-09-14T22:38:43.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.639e-06 2024-09-14T22:38:43.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:43.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759175+0000", 2024-09-14T22:38:43.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064785899999999993 2024-09-14T22:38:43.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:43.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759309+0000", 2024-09-14T22:38:43.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013374400000000001 2024-09-14T22:38:43.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:43.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759425+0000", 2024-09-14T22:38:43.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115974 2024-09-14T22:38:43.432 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.432 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:43.432 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.432 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145265 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16531456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:43.432 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694395+0000", 2024-09-14T22:38:43.433 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.499064316000002, 2024-09-14T22:38:43.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065298325000000004, 2024-09-14T22:38:43.433 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:43.433 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:43.433 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:43.433 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:43.433 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:43.433 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.433 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145265 2024-09-14T22:38:43.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.433 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:43.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:43.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694395+0000", 2024-09-14T22:38:43.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:43.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694395+0000", 2024-09-14T22:38:43.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:43.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694395+0000", 2024-09-14T22:38:43.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6699999999999999e-07 2024-09-14T22:38:43.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:43.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694397+0000", 2024-09-14T22:38:43.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.984e-06 2024-09-14T22:38:43.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:43.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694403+0000", 2024-09-14T22:38:43.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3519999999999997e-06 2024-09-14T22:38:43.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694407+0000", 2024-09-14T22:38:43.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3239999999999996e-06 2024-09-14T22:38:43.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:43.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759516+0000", 2024-09-14T22:38:43.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065109279000000006 2024-09-14T22:38:43.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:43.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759610+0000", 2024-09-14T22:38:43.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3566000000000005e-05 2024-09-14T22:38:43.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:43.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759693+0000", 2024-09-14T22:38:43.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3453000000000002e-05 2024-09-14T22:38:43.437 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.437 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:43.437 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.437 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145266 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16433152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:43.437 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694413+0000", 2024-09-14T22:38:43.437 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.499046516999996, 2024-09-14T22:38:43.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065618308, 2024-09-14T22:38:43.437 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:43.437 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:43.437 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:43.437 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:43.437 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:43.437 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.438 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145266 2024-09-14T22:38:43.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.438 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:43.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:43.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694413+0000", 2024-09-14T22:38:43.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:43.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694413+0000", 2024-09-14T22:38:43.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:43.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694413+0000", 2024-09-14T22:38:43.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4799999999999999e-07 2024-09-14T22:38:43.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:43.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694415+0000", 2024-09-14T22:38:43.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.08e-06 2024-09-14T22:38:43.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:43.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694420+0000", 2024-09-14T22:38:43.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.203e-06 2024-09-14T22:38:43.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694425+0000", 2024-09-14T22:38:43.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4649999999999996e-06 2024-09-14T22:38:43.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:43.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759768+0000", 2024-09-14T22:38:43.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065343613999999994 2024-09-14T22:38:43.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:43.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759909+0000", 2024-09-14T22:38:43.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014054099999999999 2024-09-14T22:38:43.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:43.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760031+0000", 2024-09-14T22:38:43.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122057 2024-09-14T22:38:43.441 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.441 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:43.441 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.441 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145267 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16629760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:43.442 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694431+0000", 2024-09-14T22:38:43.442 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.499028602999999, 2024-09-14T22:38:43.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065850237000000006, 2024-09-14T22:38:43.442 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:43.442 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:43.442 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:43.442 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:43.442 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:43.442 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.442 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145267 2024-09-14T22:38:43.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.442 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:43.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:43.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694431+0000", 2024-09-14T22:38:43.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:43.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694431+0000", 2024-09-14T22:38:43.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:43.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694431+0000", 2024-09-14T22:38:43.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4799999999999999e-07 2024-09-14T22:38:43.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:43.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694433+0000", 2024-09-14T22:38:43.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8470000000000001e-06 2024-09-14T22:38:43.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:43.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694438+0000", 2024-09-14T22:38:43.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3249999999999998e-06 2024-09-14T22:38:43.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694442+0000", 2024-09-14T22:38:43.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3490000000000002e-06 2024-09-14T22:38:43.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:43.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760113+0000", 2024-09-14T22:38:43.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065670872000000005 2024-09-14T22:38:43.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:43.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760199+0000", 2024-09-14T22:38:43.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5982000000000004e-05 2024-09-14T22:38:43.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:43.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760281+0000", 2024-09-14T22:38:43.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1514000000000006e-05 2024-09-14T22:38:43.445 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.446 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:43.446 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.446 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145268 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16728064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:43.446 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694448+0000", 2024-09-14T22:38:43.446 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.499011019000001, 2024-09-14T22:38:43.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066157585000000005, 2024-09-14T22:38:43.446 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:43.446 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:43.446 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:43.446 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:43.446 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:43.446 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.447 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145268 2024-09-14T22:38:43.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.447 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:43.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:43.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694448+0000", 2024-09-14T22:38:43.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:43.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694448+0000", 2024-09-14T22:38:43.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:43.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:43.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694449+0000", 2024-09-14T22:38:43.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.39e-07 2024-09-14T22:38:43.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:43.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694450+0000", 2024-09-14T22:38:43.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.906e-06 2024-09-14T22:38:43.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:43.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694456+0000", 2024-09-14T22:38:43.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3820000000000003e-06 2024-09-14T22:38:43.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694460+0000", 2024-09-14T22:38:43.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6290000000000001e-06 2024-09-14T22:38:43.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:43.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760345+0000", 2024-09-14T22:38:43.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065884540000000005 2024-09-14T22:38:43.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:43.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760482+0000", 2024-09-14T22:38:43.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013731600000000001 2024-09-14T22:38:43.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:43.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:43.450 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:43.450 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760606+0000", 2024-09-14T22:38:43.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000123473 2024-09-14T22:38:43.450 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.450 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:43.450 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.450 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:43.450 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:43.450 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:43.452 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:38:43.452 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:43.449+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:43.553 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:43.549+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:43.584 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:43.584 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:43.584 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:38:43.584 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:43.598 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:38:43.654 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:43.646+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:43.722 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:43.722 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:43.722 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:38:43.722 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:38:43.722 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:43.736 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:38:43.755 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:43.753+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:43.856 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:43.850+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:43.896 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T22:38:43.914 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:43.914 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:38:43.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:38:43.915 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:38:43.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.915 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6395 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8386560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:38:43.915 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:38:43.915 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 139.975250657, 2024-09-14T22:38:43.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054534379000000001, 2024-09-14T22:38:43.916 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:43.916 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:43.916 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:43.916 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:43.916 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:43.917 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.917 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6395 2024-09-14T22:38:43.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.917 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:43.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:43.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:38:43.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:43.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:43.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:38:43.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:43.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:43.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933833+0000", 2024-09-14T22:38:43.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3859999999999999e-06 2024-09-14T22:38:43.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:43.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933838+0000", 2024-09-14T22:38:43.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8709999999999996e-06 2024-09-14T22:38:43.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:43.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933854+0000", 2024-09-14T22:38:43.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6189e-05 2024-09-14T22:38:43.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933871+0000", 2024-09-14T22:38:43.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.626e-05 2024-09-14T22:38:43.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:43.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988198+0000", 2024-09-14T22:38:43.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054327581999999999 2024-09-14T22:38:43.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:43.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988274+0000", 2024-09-14T22:38:43.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5759000000000004e-05 2024-09-14T22:38:43.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.923 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:43.923 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988366+0000", 2024-09-14T22:38:43.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2331999999999996e-05 2024-09-14T22:38:43.924 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:43.924 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:43.924 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:43.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.924 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104944 3.3 3:c3780f3b:::rbd_data.113ca561f7a9.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:43.924 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:38:43.924 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 84.875648784000006, 2024-09-14T22:38:43.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073041087000000005, 2024-09-14T22:38:43.925 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:43.925 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:43.925 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:43.925 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:43.925 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:43.926 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.926 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104944 2024-09-14T22:38:43.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.926 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:43.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:43.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:38:43.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:43.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:43.927 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:38:43.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:43.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:43.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033436+0000", 2024-09-14T22:38:43.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2060000000000001e-06 2024-09-14T22:38:43.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:43.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033448+0000", 2024-09-14T22:38:43.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.169e-05 2024-09-14T22:38:43.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:43.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033474+0000", 2024-09-14T22:38:43.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6400000000000001e-05 2024-09-14T22:38:43.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033489+0000", 2024-09-14T22:38:43.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5067e-05 2024-09-14T22:38:43.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:43.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033658+0000", 2024-09-14T22:38:43.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000168565 2024-09-14T22:38:43.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:43.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033793+0000", 2024-09-14T22:38:43.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013536900000000001 2024-09-14T22:38:43.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.932 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:38:43.932 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.047559+0000", 2024-09-14T22:38:43.932 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013766294 2024-09-14T22:38:43.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.932 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.932 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:38:43.932 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106157+0000", 2024-09-14T22:38:43.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058597110000000001 2024-09-14T22:38:43.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:43.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106290+0000", 2024-09-14T22:38:43.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000133244 2024-09-14T22:38:43.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:38:43.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106396+0000", 2024-09-14T22:38:43.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106246 2024-09-14T22:38:43.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:43.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106475+0000", 2024-09-14T22:38:43.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8895999999999998e-05 2024-09-14T22:38:43.935 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:43.935 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:43.935 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:43.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.936 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:111858 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 10403840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:43.936 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:38:43.936 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.085195662999993, 2024-09-14T22:38:43.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054398798999999998, 2024-09-14T22:38:43.936 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:43.936 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:43.936 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:43.937 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:43.937 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:43.937 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.937 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 111858 2024-09-14T22:38:43.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.937 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:43.937 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.937 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:43.937 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:38:43.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:43.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.938 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:43.938 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:38:43.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:43.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.938 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:43.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823888+0000", 2024-09-14T22:38:43.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0200000000000001e-07 2024-09-14T22:38:43.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:43.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823927+0000", 2024-09-14T22:38:43.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8940999999999999e-05 2024-09-14T22:38:43.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.940 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:43.940 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823940+0000", 2024-09-14T22:38:43.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3682e-05 2024-09-14T22:38:43.940 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.940 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.940 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823950+0000", 2024-09-14T22:38:43.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7110000000000007e-06 2024-09-14T22:38:43.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.941 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:43.941 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878202+0000", 2024-09-14T22:38:43.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054251969999999997 2024-09-14T22:38:43.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.941 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:43.942 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878249+0000", 2024-09-14T22:38:43.942 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6854999999999998e-05 2024-09-14T22:38:43.942 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.942 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.942 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:43.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878286+0000", 2024-09-14T22:38:43.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6838000000000002e-05 2024-09-14T22:38:43.943 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:43.943 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:43.943 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:43.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.943 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:111859 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 10452992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:43.943 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:38:43.943 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.085106543999999, 2024-09-14T22:38:43.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054435292000000003, 2024-09-14T22:38:43.944 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:43.944 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:43.944 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:43.944 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:43.944 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:43.944 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.944 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 111859 2024-09-14T22:38:43.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.944 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:43.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:43.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:38:43.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:43.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:43.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:38:43.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:43.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:43.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823977+0000", 2024-09-14T22:38:43.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3500000000000005e-07 2024-09-14T22:38:43.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:43.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823981+0000", 2024-09-14T22:38:43.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2050000000000004e-06 2024-09-14T22:38:43.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.946 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:43.946 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824007+0000", 2024-09-14T22:38:43.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.62e-05 2024-09-14T22:38:43.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.946 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.946 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824018+0000", 2024-09-14T22:38:43.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0389e-05 2024-09-14T22:38:43.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:43.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878312+0000", 2024-09-14T22:38:43.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054294754000000001 2024-09-14T22:38:43.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:43.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878350+0000", 2024-09-14T22:38:43.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7982000000000003e-05 2024-09-14T22:38:43.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:43.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878411+0000", 2024-09-14T22:38:43.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0927000000000001e-05 2024-09-14T22:38:43.948 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:43.948 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:43.948 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:43.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.948 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112030 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6316032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:43.948 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:38:43.948 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.085052202, 2024-09-14T22:38:43.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054597826000000002, 2024-09-14T22:38:43.948 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:43.948 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:43.949 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:43.949 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:43.949 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:43.949 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.949 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112030 2024-09-14T22:38:43.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.949 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:43.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:43.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:38:43.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:43.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.950 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:43.950 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:38:43.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:43.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.950 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:43.950 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824031+0000", 2024-09-14T22:38:43.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3600000000000003e-07 2024-09-14T22:38:43.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:43.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824036+0000", 2024-09-14T22:38:43.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7350000000000004e-06 2024-09-14T22:38:43.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:43.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824049+0000", 2024-09-14T22:38:43.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3329000000000001e-05 2024-09-14T22:38:43.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824057+0000", 2024-09-14T22:38:43.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2460000000000003e-06 2024-09-14T22:38:43.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:43.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878464+0000", 2024-09-14T22:38:43.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054406207999999998 2024-09-14T22:38:43.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:43.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878554+0000", 2024-09-14T22:38:43.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0199999999999997e-05 2024-09-14T22:38:43.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:43.953 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878628+0000", 2024-09-14T22:38:43.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4371999999999999e-05 2024-09-14T22:38:43.953 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:43.953 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:43.953 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:43.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.953 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112031 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6365184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:43.953 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:38:43.954 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.085013884000006, 2024-09-14T22:38:43.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054741868999999999, 2024-09-14T22:38:43.954 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:43.954 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:43.954 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:43.954 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:43.954 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:43.954 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.954 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112031 2024-09-14T22:38:43.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.954 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:43.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:43.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:38:43.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:43.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:43.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:38:43.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:43.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:43.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824070+0000", 2024-09-14T22:38:43.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5099999999999998e-07 2024-09-14T22:38:43.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:43.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824073+0000", 2024-09-14T22:38:43.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6940000000000001e-06 2024-09-14T22:38:43.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:43.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824084+0000", 2024-09-14T22:38:43.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0380000000000001e-05 2024-09-14T22:38:43.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.957 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.957 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824091+0000", 2024-09-14T22:38:43.957 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5730000000000002e-06 2024-09-14T22:38:43.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.957 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:43.957 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878670+0000", 2024-09-14T22:38:43.957 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054579176 2024-09-14T22:38:43.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.957 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:43.958 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878742+0000", 2024-09-14T22:38:43.958 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T22:38:43.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.958 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.958 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:43.958 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878811+0000", 2024-09-14T22:38:43.958 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8170999999999995e-05 2024-09-14T22:38:43.958 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:43.958 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:43.958 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:43.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.958 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.958 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112032 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6414336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:43.959 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:38:43.959 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.084980017999996, 2024-09-14T22:38:43.959 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054931916999999997, 2024-09-14T22:38:43.959 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:43.959 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:43.959 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:43.959 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:43.959 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:43.959 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.959 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112032 2024-09-14T22:38:43.959 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.959 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:43.959 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:43.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:38:43.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:43.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:43.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:38:43.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:43.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.961 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:43.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:38:43.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:38:43.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.961 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:43.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824107+0000", 2024-09-14T22:38:43.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5640000000000001e-06 2024-09-14T22:38:43.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.961 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:43.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824117+0000", 2024-09-14T22:38:43.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0366e-05 2024-09-14T22:38:43.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824158+0000", 2024-09-14T22:38:43.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0670000000000002e-05 2024-09-14T22:38:43.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:43.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878853+0000", 2024-09-14T22:38:43.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054694853000000002 2024-09-14T22:38:43.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:43.963 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878934+0000", 2024-09-14T22:38:43.963 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1754000000000004e-05 2024-09-14T22:38:43.963 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.963 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.963 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:43.963 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879034+0000", 2024-09-14T22:38:43.963 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000100115 2024-09-14T22:38:43.963 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:43.963 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:43.964 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:43.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.964 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112033 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6463488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:43.964 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:38:43.964 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.084911360000007, 2024-09-14T22:38:43.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054991076999999999, 2024-09-14T22:38:43.964 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:43.964 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:43.964 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:43.964 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:43.964 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:43.965 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.965 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112033 2024-09-14T22:38:43.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.965 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:43.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:43.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:38:43.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:43.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:43.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:38:43.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:43.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:43.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824172+0000", 2024-09-14T22:38:43.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9999999999999996e-07 2024-09-14T22:38:43.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:43.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824177+0000", 2024-09-14T22:38:43.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5310000000000003e-06 2024-09-14T22:38:43.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:43.967 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824188+0000", 2024-09-14T22:38:43.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1235e-05 2024-09-14T22:38:43.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.967 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.967 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824197+0000", 2024-09-14T22:38:43.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7069999999999998e-06 2024-09-14T22:38:43.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.967 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:43.967 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879075+0000", 2024-09-14T22:38:43.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054878749999999997 2024-09-14T22:38:43.968 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.968 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.968 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:43.968 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879118+0000", 2024-09-14T22:38:43.968 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2749e-05 2024-09-14T22:38:43.968 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.968 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.968 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:43.968 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879162+0000", 2024-09-14T22:38:43.968 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4205000000000001e-05 2024-09-14T22:38:43.968 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:43.968 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:43.968 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:43.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.969 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112034 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6512640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:43.969 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:38:43.969 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.084874408999994, 2024-09-14T22:38:43.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055181246000000003, 2024-09-14T22:38:43.969 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:43.969 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:43.969 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:43.969 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:43.969 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:43.969 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.969 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112034 2024-09-14T22:38:43.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.970 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:43.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:43.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:38:43.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:43.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:43.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:38:43.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:43.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:43.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824209+0000", 2024-09-14T22:38:43.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5499999999999998e-07 2024-09-14T22:38:43.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:43.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824212+0000", 2024-09-14T22:38:43.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6119999999999999e-06 2024-09-14T22:38:43.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:43.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824224+0000", 2024-09-14T22:38:43.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1127999999999999e-05 2024-09-14T22:38:43.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824232+0000", 2024-09-14T22:38:43.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4700000000000002e-06 2024-09-14T22:38:43.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:43.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879209+0000", 2024-09-14T22:38:43.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054977201000000003 2024-09-14T22:38:43.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:43.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879308+0000", 2024-09-14T22:38:43.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8419999999999996e-05 2024-09-14T22:38:43.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:43.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879389+0000", 2024-09-14T22:38:43.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1760000000000003e-05 2024-09-14T22:38:43.973 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:43.974 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:43.974 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:43.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.974 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112035 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6561792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:43.974 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:38:43.974 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.084780680999998, 2024-09-14T22:38:43.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055300522999999997, 2024-09-14T22:38:43.974 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:43.974 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:43.974 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:43.974 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:43.974 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:43.974 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.974 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112035 2024-09-14T22:38:43.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.975 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:43.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:43.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:38:43.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:43.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:43.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:38:43.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:43.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.976 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:43.976 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824303+0000", 2024-09-14T22:38:43.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5099999999999998e-07 2024-09-14T22:38:43.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.976 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:43.976 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824308+0000", 2024-09-14T22:38:43.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6689999999999997e-06 2024-09-14T22:38:43.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.976 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:43.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824744+0000", 2024-09-14T22:38:43.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00043601600000000001 2024-09-14T22:38:43.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824756+0000", 2024-09-14T22:38:43.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2476999999999999e-05 2024-09-14T22:38:43.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:43.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879434+0000", 2024-09-14T22:38:43.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054677630999999997 2024-09-14T22:38:43.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:43.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879518+0000", 2024-09-14T22:38:43.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4220000000000003e-05 2024-09-14T22:38:43.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:43.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879602+0000", 2024-09-14T22:38:43.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4559000000000001e-05 2024-09-14T22:38:43.979 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:43.979 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:43.979 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:43.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.979 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112036 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6610944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:43.979 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:38:43.979 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.084288943000004, 2024-09-14T22:38:43.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055036320999999999, 2024-09-14T22:38:43.979 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:43.979 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:43.979 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:43.980 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:43.980 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:43.980 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.980 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112036 2024-09-14T22:38:43.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.980 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:43.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.980 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:43.980 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:38:43.980 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:43.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.981 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:43.981 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:38:43.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:43.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.981 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:43.981 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824795+0000", 2024-09-14T22:38:43.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-06 2024-09-14T22:38:43.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.981 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:43.981 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824802+0000", 2024-09-14T22:38:43.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6680000000000004e-06 2024-09-14T22:38:43.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:43.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824814+0000", 2024-09-14T22:38:43.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2102e-05 2024-09-14T22:38:43.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824823+0000", 2024-09-14T22:38:43.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5550000000000008e-06 2024-09-14T22:38:43.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:43.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879648+0000", 2024-09-14T22:38:43.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054825167000000001 2024-09-14T22:38:43.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:43.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879747+0000", 2024-09-14T22:38:43.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8968999999999994e-05 2024-09-14T22:38:43.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:43.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879830+0000", 2024-09-14T22:38:43.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2633999999999995e-05 2024-09-14T22:38:43.984 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:43.984 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:43.984 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:43.984 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.984 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112037 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6660096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:43.984 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:38:43.984 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.084246234000005, 2024-09-14T22:38:43.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055122926000000003, 2024-09-14T22:38:43.984 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:43.984 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:43.984 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:43.985 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:43.985 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:43.985 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.985 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112037 2024-09-14T22:38:43.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.985 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:43.985 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.985 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:43.985 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:38:43.985 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:43.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.985 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.985 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:43.985 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:38:43.985 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:43.986 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.986 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:43.986 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824837+0000", 2024-09-14T22:38:43.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5499999999999998e-07 2024-09-14T22:38:43.986 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.986 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:43.986 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824840+0000", 2024-09-14T22:38:43.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0429999999999999e-06 2024-09-14T22:38:43.986 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:43.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824851+0000", 2024-09-14T22:38:43.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1046999999999999e-05 2024-09-14T22:38:43.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824859+0000", 2024-09-14T22:38:43.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1200000000000002e-06 2024-09-14T22:38:43.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:43.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879867+0000", 2024-09-14T22:38:43.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055007697000000001 2024-09-14T22:38:43.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.988 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:43.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879910+0000", 2024-09-14T22:38:43.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3402000000000002e-05 2024-09-14T22:38:43.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.988 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:43.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879959+0000", 2024-09-14T22:38:43.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9061999999999999e-05 2024-09-14T22:38:43.988 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:43.988 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:43.988 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:43.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.989 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112038 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6709248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:43.989 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:38:43.989 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.084211538000005, 2024-09-14T22:38:43.989 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055312828000000001, 2024-09-14T22:38:43.989 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:43.989 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:43.989 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:43.989 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:43.989 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:43.989 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.989 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112038 2024-09-14T22:38:43.989 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.989 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:43.989 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.990 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:43.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:38:43.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:43.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.990 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:43.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:38:43.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:43.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.990 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:43.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824872+0000", 2024-09-14T22:38:43.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T22:38:43.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.991 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:43.991 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824875+0000", 2024-09-14T22:38:43.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7519999999999998e-06 2024-09-14T22:38:43.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.991 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.991 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:43.991 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824917+0000", 2024-09-14T22:38:43.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1047999999999997e-05 2024-09-14T22:38:43.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.991 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.991 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.991 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824926+0000", 2024-09-14T22:38:43.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3749999999999992e-06 2024-09-14T22:38:43.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.991 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:43.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880006+0000", 2024-09-14T22:38:43.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055080525999999998 2024-09-14T22:38:43.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:43.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880102+0000", 2024-09-14T22:38:43.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5995999999999994e-05 2024-09-14T22:38:43.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:43.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880184+0000", 2024-09-14T22:38:43.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1478999999999997e-05 2024-09-14T22:38:43.993 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:43.993 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:43.993 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:43.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.993 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.993 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112039 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6758400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:43.993 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:38:43.993 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.084142868000001, 2024-09-14T22:38:43.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055449709999999999, 2024-09-14T22:38:43.993 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:43.993 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:43.993 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:43.993 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:43.994 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:43.994 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.994 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112039 2024-09-14T22:38:43.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.994 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:43.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:43.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:38:43.994 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:43.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:43.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:38:43.994 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:43.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.995 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:43.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824941+0000", 2024-09-14T22:38:43.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3699999999999999e-07 2024-09-14T22:38:43.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.995 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:43.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824946+0000", 2024-09-14T22:38:43.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0209999999999999e-06 2024-09-14T22:38:43.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.995 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:43.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824972+0000", 2024-09-14T22:38:43.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6750999999999998e-05 2024-09-14T22:38:43.996 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.996 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:43.996 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824982+0000", 2024-09-14T22:38:43.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5019999999999998e-06 2024-09-14T22:38:43.996 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.996 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:43.996 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880227+0000", 2024-09-14T22:38:43.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055245257999999998 2024-09-14T22:38:43.996 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.996 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:43.996 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880305+0000", 2024-09-14T22:38:43.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7424999999999998e-05 2024-09-14T22:38:43.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.997 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:43.997 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880389+0000", 2024-09-14T22:38:43.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4815999999999996e-05 2024-09-14T22:38:43.997 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:43.997 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:43.997 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:43.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.997 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112040 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6807552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:43.997 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:38:43.997 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.084086151999998, 2024-09-14T22:38:43.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055534615000000002, 2024-09-14T22:38:43.997 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:43.998 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:43.998 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:43.998 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:43.998 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:43.998 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:43.998 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112040 2024-09-14T22:38:43.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.998 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:43.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:43.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:38:43.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:43.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:43.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:38:43.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:43.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:43.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824997+0000", 2024-09-14T22:38:43.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7199999999999998e-07 2024-09-14T22:38:43.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:43.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825042+0000", 2024-09-14T22:38:43.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5065999999999998e-05 2024-09-14T22:38:43.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:43.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:43.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:44.000 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825060+0000", 2024-09-14T22:38:44.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7520000000000002e-05 2024-09-14T22:38:44.000 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.000 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:44.000 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825070+0000", 2024-09-14T22:38:44.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8230000000000006e-06 2024-09-14T22:38:44.000 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.000 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:44.000 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880427+0000", 2024-09-14T22:38:44.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055357244 2024-09-14T22:38:44.000 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:44.001 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880483+0000", 2024-09-14T22:38:44.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5807000000000003e-05 2024-09-14T22:38:44.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:44.001 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880531+0000", 2024-09-14T22:38:44.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8383000000000001e-05 2024-09-14T22:38:44.001 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.001 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:44.001 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.002 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112041 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6856704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:44.002 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:38:44.002 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.083996978000002, 2024-09-14T22:38:44.002 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055573887000000002, 2024-09-14T22:38:44.002 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:44.002 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:44.002 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:44.003 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:44.003 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:44.003 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:44.003 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112041 2024-09-14T22:38:44.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.003 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:44.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:44.004 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:38:44.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:44.005 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:38:44.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.005 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.005 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.005 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:44.005 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825087+0000", 2024-09-14T22:38:44.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2399999999999996e-07 2024-09-14T22:38:44.005 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.006 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.006 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:44.006 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825091+0000", 2024-09-14T22:38:44.006 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9729999999999999e-06 2024-09-14T22:38:44.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.006 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.006 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:44.006 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825101+0000", 2024-09-14T22:38:44.006 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.093e-05 2024-09-14T22:38:44.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.006 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.006 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:44.006 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825110+0000", 2024-09-14T22:38:44.006 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1450000000000008e-06 2024-09-14T22:38:44.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.007 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:44.007 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880559+0000", 2024-09-14T22:38:44.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055449684999999999 2024-09-14T22:38:44.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.007 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:44.007 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880601+0000", 2024-09-14T22:38:44.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1248000000000002e-05 2024-09-14T22:38:44.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.007 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:44.007 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880659+0000", 2024-09-14T22:38:44.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8981999999999999e-05 2024-09-14T22:38:44.008 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.008 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:44.008 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.008 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112042 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6905856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:44.008 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:38:44.008 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.083962244000006, 2024-09-14T22:38:44.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055781415000000001, 2024-09-14T22:38:44.008 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:44.008 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:44.008 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:44.008 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:44.008 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:44.009 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:44.009 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112042 2024-09-14T22:38:44.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.009 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:44.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:44.009 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:38:44.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:44.009 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:38:44.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.010 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:44.010 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825478+0000", 2024-09-14T22:38:44.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035775 2024-09-14T22:38:44.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.010 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:44.010 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825494+0000", 2024-09-14T22:38:44.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5994999999999998e-05 2024-09-14T22:38:44.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.010 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:44.010 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825520+0000", 2024-09-14T22:38:44.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5738999999999999e-05 2024-09-14T22:38:44.011 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.011 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.011 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:44.011 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825547+0000", 2024-09-14T22:38:44.011 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7667000000000001e-05 2024-09-14T22:38:44.011 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.011 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.011 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:44.011 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880725+0000", 2024-09-14T22:38:44.011 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055177962999999997 2024-09-14T22:38:44.011 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.011 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.011 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:44.011 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880816+0000", 2024-09-14T22:38:44.011 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0315000000000005e-05 2024-09-14T22:38:44.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:44.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880902+0000", 2024-09-14T22:38:44.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5986e-05 2024-09-14T22:38:44.012 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.012 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:44.012 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.012 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112043 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:44.012 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:38:44.012 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.08351424, 2024-09-14T22:38:44.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055537912000000002, 2024-09-14T22:38:44.013 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:44.013 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:44.013 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:44.013 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:44.013 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:44.013 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:44.013 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112043 2024-09-14T22:38:44.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.013 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:44.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:44.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:38:44.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:44.014 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:38:44.014 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:44.014 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825911+0000", 2024-09-14T22:38:44.014 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00034306899999999999 2024-09-14T22:38:44.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:44.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825918+0000", 2024-09-14T22:38:44.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.002e-06 2024-09-14T22:38:44.015 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.015 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.015 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:44.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825931+0000", 2024-09-14T22:38:44.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2401e-05 2024-09-14T22:38:44.015 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.015 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.015 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:44.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825942+0000", 2024-09-14T22:38:44.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0903e-05 2024-09-14T22:38:44.015 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.015 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.015 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:44.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880951+0000", 2024-09-14T22:38:44.016 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055009379999999997 2024-09-14T22:38:44.016 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.016 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.016 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:44.016 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881027+0000", 2024-09-14T22:38:44.016 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5944000000000003e-05 2024-09-14T22:38:44.016 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.016 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.016 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:44.016 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881106+0000", 2024-09-14T22:38:44.016 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9213000000000002e-05 2024-09-14T22:38:44.016 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.016 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:44.016 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.016 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.017 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112044 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 7004160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:44.017 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:38:44.017 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.083120570999995, 2024-09-14T22:38:44.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055265416999999997, 2024-09-14T22:38:44.017 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:44.017 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:44.017 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:44.017 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:44.017 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:44.017 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:44.017 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112044 2024-09-14T22:38:44.017 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.018 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:44.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.018 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:44.018 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:38:44.018 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.018 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.018 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:44.018 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:38:44.018 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.018 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.018 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:44.018 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826239+0000", 2024-09-14T22:38:44.018 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027676000000000003 2024-09-14T22:38:44.019 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.019 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.019 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:44.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826247+0000", 2024-09-14T22:38:44.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7440000000000007e-06 2024-09-14T22:38:44.019 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.019 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.019 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:44.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826275+0000", 2024-09-14T22:38:44.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8053000000000001e-05 2024-09-14T22:38:44.019 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.019 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.019 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:44.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826286+0000", 2024-09-14T22:38:44.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1780000000000001e-05 2024-09-14T22:38:44.020 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.020 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.020 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:44.020 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881137+0000", 2024-09-14T22:38:44.020 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054850500000000003 2024-09-14T22:38:44.020 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.020 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.020 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:44.020 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881181+0000", 2024-09-14T22:38:44.020 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4153000000000002e-05 2024-09-14T22:38:44.020 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.020 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.020 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:44.020 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881227+0000", 2024-09-14T22:38:44.021 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6427000000000001e-05 2024-09-14T22:38:44.021 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.021 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:44.021 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.021 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.021 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.021 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112045 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 7053312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:44.021 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:38:44.021 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.082758338000005, 2024-09-14T22:38:44.021 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055045050999999998, 2024-09-14T22:38:44.021 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:44.021 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:44.022 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:44.022 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:44.022 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:44.022 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:44.022 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112045 2024-09-14T22:38:44.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.022 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:44.022 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:44.022 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:38:44.022 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:44.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:38:44.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:44.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830250+0000", 2024-09-14T22:38:44.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0039258670000000004 2024-09-14T22:38:44.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:44.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830260+0000", 2024-09-14T22:38:44.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0088000000000001e-05 2024-09-14T22:38:44.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.024 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:44.024 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830281+0000", 2024-09-14T22:38:44.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0720000000000002e-05 2024-09-14T22:38:44.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.024 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:44.024 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830292+0000", 2024-09-14T22:38:44.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1455e-05 2024-09-14T22:38:44.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.025 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:44.025 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881255+0000", 2024-09-14T22:38:44.025 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.050962247000000002 2024-09-14T22:38:44.025 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.025 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.025 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:44.025 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881296+0000", 2024-09-14T22:38:44.025 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1557999999999998e-05 2024-09-14T22:38:44.025 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.025 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.026 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:44.026 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881369+0000", 2024-09-14T22:38:44.026 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3115999999999996e-05 2024-09-14T22:38:44.026 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.026 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:44.026 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.026 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.026 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:44.026 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:44.027 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:38:44.028 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:44.025+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:44.129 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:44.125+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:44.201 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:44.201 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:44.201 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:38:44.201 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:44.214 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:38:44.230 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:44.229+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:44.331 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:44.326+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:44.338 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:44.338 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:44.338 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:38:44.338 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:38:44.338 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:44.352 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:38:44.439 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:44.434+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:44.517 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:44.517 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:38:44.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:38:44.517 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:38:44.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.518 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97197 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 796672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:44.518 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:38:44.518 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 94.442512819000001, 2024-09-14T22:38:44.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12434473, 2024-09-14T22:38:44.518 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:44.518 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:44.518 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:44.518 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:44.518 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:44.518 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:44.518 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97197 2024-09-14T22:38:44.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.518 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:44.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:44.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:38:44.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:44.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:38:44.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:44.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070524+0000", 2024-09-14T22:38:44.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0299999999999998e-07 2024-09-14T22:38:44.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:44.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070527+0000", 2024-09-14T22:38:44.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9569999999999998e-06 2024-09-14T22:38:44.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:44.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070535+0000", 2024-09-14T22:38:44.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7630000000000006e-06 2024-09-14T22:38:44.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:44.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070543+0000", 2024-09-14T22:38:44.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4869999999999996e-06 2024-09-14T22:38:44.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:44.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194525+0000", 2024-09-14T22:38:44.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123982096 2024-09-14T22:38:44.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.521 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:44.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194699+0000", 2024-09-14T22:38:44.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000174105 2024-09-14T22:38:44.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.521 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:44.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194868+0000", 2024-09-14T22:38:44.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016861900000000001 2024-09-14T22:38:44.521 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.521 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:44.521 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.522 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97198 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 805888~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:44.522 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:38:44.522 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 94.442484442999998, 2024-09-14T22:38:44.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.1248088, 2024-09-14T22:38:44.522 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:44.522 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:44.522 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:44.522 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:44.522 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:44.522 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:44.522 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97198 2024-09-14T22:38:44.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.522 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:44.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:44.523 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:38:44.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:44.523 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:38:44.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:44.523 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070552+0000", 2024-09-14T22:38:44.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T22:38:44.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:44.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070555+0000", 2024-09-14T22:38:44.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.853e-06 2024-09-14T22:38:44.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:44.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070565+0000", 2024-09-14T22:38:44.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.048e-05 2024-09-14T22:38:44.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:44.525 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070573+0000", 2024-09-14T22:38:44.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7200000000000006e-06 2024-09-14T22:38:44.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:44.525 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194954+0000", 2024-09-14T22:38:44.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12438089400000001 2024-09-14T22:38:44.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:44.525 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195145+0000", 2024-09-14T22:38:44.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019091299999999999 2024-09-14T22:38:44.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.526 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:44.526 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195360+0000", 2024-09-14T22:38:44.526 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021527300000000001 2024-09-14T22:38:44.526 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.526 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:44.526 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.526 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97199 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 815104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:44.526 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:38:44.526 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 94.442453846000006, 2024-09-14T22:38:44.526 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125251901, 2024-09-14T22:38:44.526 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:44.527 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:44.527 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:44.527 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:44.527 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:44.527 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:44.527 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97199 2024-09-14T22:38:44.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.527 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:44.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:44.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:38:44.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.528 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:44.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:38:44.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.528 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:44.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070583+0000", 2024-09-14T22:38:44.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6300000000000005e-07 2024-09-14T22:38:44.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.528 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:44.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070586+0000", 2024-09-14T22:38:44.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8270000000000002e-06 2024-09-14T22:38:44.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:44.529 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070597+0000", 2024-09-14T22:38:44.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1960000000000001e-05 2024-09-14T22:38:44.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:44.529 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070605+0000", 2024-09-14T22:38:44.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6459999999999993e-06 2024-09-14T22:38:44.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:44.529 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195432+0000", 2024-09-14T22:38:44.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124826926 2024-09-14T22:38:44.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:44.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195626+0000", 2024-09-14T22:38:44.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019357099999999999 2024-09-14T22:38:44.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:44.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195834+0000", 2024-09-14T22:38:44.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020830800000000001 2024-09-14T22:38:44.530 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.530 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:44.530 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.531 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97200 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 824320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:44.531 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:38:44.531 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 94.442421800000005, 2024-09-14T22:38:44.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125693477, 2024-09-14T22:38:44.531 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:44.531 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:44.531 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:44.531 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:44.531 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:44.531 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:44.531 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97200 2024-09-14T22:38:44.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.531 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:44.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:44.532 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:38:44.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.532 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:44.532 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:38:44.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.532 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:44.532 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070615+0000", 2024-09-14T22:38:44.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T22:38:44.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:44.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070618+0000", 2024-09-14T22:38:44.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.943e-06 2024-09-14T22:38:44.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:44.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070627+0000", 2024-09-14T22:38:44.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1099999999999992e-06 2024-09-14T22:38:44.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:44.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070634+0000", 2024-09-14T22:38:44.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:38:44.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.534 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:44.534 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195908+0000", 2024-09-14T22:38:44.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125273893 2024-09-14T22:38:44.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.534 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:44.534 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196097+0000", 2024-09-14T22:38:44.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018953899999999999 2024-09-14T22:38:44.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.534 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:44.534 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196308+0000", 2024-09-14T22:38:44.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021027799999999999 2024-09-14T22:38:44.535 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.535 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:44.535 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.535 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97201 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 833536~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:44.535 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:38:44.535 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 94.442391612999998, 2024-09-14T22:38:44.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126133826, 2024-09-14T22:38:44.535 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:44.535 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:44.535 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:44.535 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:44.535 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:44.535 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:44.536 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97201 2024-09-14T22:38:44.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.536 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:44.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:44.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:38:44.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:44.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:38:44.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:44.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074109+0000", 2024-09-14T22:38:44.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003464834 2024-09-14T22:38:44.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:44.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074134+0000", 2024-09-14T22:38:44.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5170000000000001e-05 2024-09-14T22:38:44.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:44.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074166+0000", 2024-09-14T22:38:44.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1347000000000003e-05 2024-09-14T22:38:44.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:44.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074179+0000", 2024-09-14T22:38:44.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.364e-05 2024-09-14T22:38:44.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.538 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:44.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196377+0000", 2024-09-14T22:38:44.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12219793900000001 2024-09-14T22:38:44.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.538 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:44.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196568+0000", 2024-09-14T22:38:44.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019097999999999999 2024-09-14T22:38:44.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.538 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:44.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196778+0000", 2024-09-14T22:38:44.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020991599999999999 2024-09-14T22:38:44.539 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.539 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:44.539 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.539 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97205 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 870400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:44.539 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:38:44.539 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 94.438722049999996, 2024-09-14T22:38:44.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12436148299999999, 2024-09-14T22:38:44.539 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:44.539 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:44.539 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:44.539 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:44.540 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:44.540 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:44.540 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97205 2024-09-14T22:38:44.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.540 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:44.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:44.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:38:44.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:44.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:38:44.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:44.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074315+0000", 2024-09-14T22:38:44.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9299999999999997e-07 2024-09-14T22:38:44.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:44.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074318+0000", 2024-09-14T22:38:44.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.19e-06 2024-09-14T22:38:44.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:44.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074327+0000", 2024-09-14T22:38:44.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8529999999999998e-06 2024-09-14T22:38:44.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:44.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074334+0000", 2024-09-14T22:38:44.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3869999999999997e-06 2024-09-14T22:38:44.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:44.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198365+0000", 2024-09-14T22:38:44.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124031222 2024-09-14T22:38:44.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:44.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198513+0000", 2024-09-14T22:38:44.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014772299999999999 2024-09-14T22:38:44.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:44.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198675+0000", 2024-09-14T22:38:44.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016241500000000001 2024-09-14T22:38:44.543 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.543 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:44.543 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.543 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97206 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 879616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:44.543 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:38:44.543 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 94.438690593999993, 2024-09-14T22:38:44.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124706284, 2024-09-14T22:38:44.544 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:44.544 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:44.544 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:44.544 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:44.544 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:44.544 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:44.544 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97206 2024-09-14T22:38:44.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.544 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:44.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.544 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:44.544 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:38:44.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:44.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:38:44.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:44.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074346+0000", 2024-09-14T22:38:44.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8999999999999996e-07 2024-09-14T22:38:44.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:44.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074349+0000", 2024-09-14T22:38:44.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.96e-06 2024-09-14T22:38:44.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:44.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074358+0000", 2024-09-14T22:38:44.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9169999999999995e-06 2024-09-14T22:38:44.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:44.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074365+0000", 2024-09-14T22:38:44.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3699999999999997e-06 2024-09-14T22:38:44.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.547 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:44.547 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198761+0000", 2024-09-14T22:38:44.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124395743 2024-09-14T22:38:44.547 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.547 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:44.547 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198899+0000", 2024-09-14T22:38:44.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013763499999999999 2024-09-14T22:38:44.547 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.547 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:44.547 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199052+0000", 2024-09-14T22:38:44.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015296900000000001 2024-09-14T22:38:44.548 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.548 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:44.548 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.548 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.548 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97207 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 888832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:44.548 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:38:44.548 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 94.438661381000003, 2024-09-14T22:38:44.548 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124988433, 2024-09-14T22:38:44.548 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:44.548 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:44.548 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:44.548 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:44.548 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:44.549 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:44.549 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97207 2024-09-14T22:38:44.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.549 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:44.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.549 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:44.549 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:38:44.549 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.549 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:44.549 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:38:44.549 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.550 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:44.550 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074375+0000", 2024-09-14T22:38:44.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:38:44.550 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.550 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:44.550 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074378+0000", 2024-09-14T22:38:44.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1829999999999998e-06 2024-09-14T22:38:44.550 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.550 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:44.550 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074389+0000", 2024-09-14T22:38:44.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0737000000000001e-05 2024-09-14T22:38:44.551 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.551 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:44.551 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074396+0000", 2024-09-14T22:38:44.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2559999999999996e-06 2024-09-14T22:38:44.551 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.551 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:44.551 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199139+0000", 2024-09-14T22:38:44.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12474253 2024-09-14T22:38:44.551 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.551 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:44.551 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199240+0000", 2024-09-14T22:38:44.552 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101221 2024-09-14T22:38:44.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.552 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:44.552 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199363+0000", 2024-09-14T22:38:44.552 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000122822 2024-09-14T22:38:44.552 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.552 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:44.552 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.553 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97208 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 898048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:44.553 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:38:44.553 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 94.438630076999999, 2024-09-14T22:38:44.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12524854899999999, 2024-09-14T22:38:44.553 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:44.553 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:44.553 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:44.554 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:44.554 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:44.554 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:44.554 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97208 2024-09-14T22:38:44.554 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.554 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:44.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.554 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:44.554 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:38:44.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.555 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.555 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:44.555 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:38:44.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.555 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.555 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:44.555 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:38:44.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:38:44.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.555 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.555 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:44.556 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074410+0000", 2024-09-14T22:38:44.556 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.18e-06 2024-09-14T22:38:44.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.556 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.556 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:44.556 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074435+0000", 2024-09-14T22:38:44.556 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5273e-05 2024-09-14T22:38:44.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.556 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.556 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:44.556 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074443+0000", 2024-09-14T22:38:44.557 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3000000000000002e-06 2024-09-14T22:38:44.557 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.557 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:44.557 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199433+0000", 2024-09-14T22:38:44.557 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124989502 2024-09-14T22:38:44.557 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.557 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:44.557 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199535+0000", 2024-09-14T22:38:44.557 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102557 2024-09-14T22:38:44.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.558 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:44.558 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199654+0000", 2024-09-14T22:38:44.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119061 2024-09-14T22:38:44.558 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.558 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:44.558 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.558 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97209 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 907264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:44.558 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:38:44.558 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 94.438582272000005, 2024-09-14T22:38:44.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125487354, 2024-09-14T22:38:44.559 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:44.559 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:44.559 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:44.559 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:44.559 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:44.559 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:44.559 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97209 2024-09-14T22:38:44.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.559 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:44.559 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.559 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:44.559 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:38:44.559 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.559 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.559 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:44.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:38:44.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.560 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.560 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.560 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:44.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:38:44.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9400000000000005e-07 2024-09-14T22:38:44.560 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.560 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.560 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:44.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074458+0000", 2024-09-14T22:38:44.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3299999999999999e-06 2024-09-14T22:38:44.560 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.561 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.561 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:44.561 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074467+0000", 2024-09-14T22:38:44.561 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.346e-06 2024-09-14T22:38:44.561 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.561 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.561 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:44.561 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074474+0000", 2024-09-14T22:38:44.561 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.374e-06 2024-09-14T22:38:44.561 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.561 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.561 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:44.561 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199720+0000", 2024-09-14T22:38:44.562 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12524561200000001 2024-09-14T22:38:44.562 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.562 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.562 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:44.562 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199820+0000", 2024-09-14T22:38:44.562 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010049599999999999 2024-09-14T22:38:44.562 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.562 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.562 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:44.562 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199941+0000", 2024-09-14T22:38:44.562 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012050200000000001 2024-09-14T22:38:44.562 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.562 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:44.562 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.563 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.563 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97210 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:44.563 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:38:44.563 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 94.438552278000003, 2024-09-14T22:38:44.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125744942, 2024-09-14T22:38:44.563 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:44.563 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:44.563 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:44.563 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:44.563 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:44.563 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:44.563 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97210 2024-09-14T22:38:44.563 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.563 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:44.564 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.564 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:44.564 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:38:44.564 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.564 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.564 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.564 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:44.564 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:38:44.564 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.564 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.564 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.564 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:44.564 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:38:44.565 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6599999999999996e-07 2024-09-14T22:38:44.565 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.565 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.565 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:44.565 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074487+0000", 2024-09-14T22:38:44.565 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8140000000000001e-06 2024-09-14T22:38:44.565 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.565 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.565 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:44.565 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074496+0000", 2024-09-14T22:38:44.565 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9330000000000007e-06 2024-09-14T22:38:44.565 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.565 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.566 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:44.566 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074503+0000", 2024-09-14T22:38:44.566 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4100000000000002e-06 2024-09-14T22:38:44.566 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.566 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.566 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:44.566 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200009+0000", 2024-09-14T22:38:44.566 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12550539899999999 2024-09-14T22:38:44.566 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.566 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.566 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:44.567 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200108+0000", 2024-09-14T22:38:44.567 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9182e-05 2024-09-14T22:38:44.567 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.567 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.567 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:44.567 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200229+0000", 2024-09-14T22:38:44.567 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120538 2024-09-14T22:38:44.567 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.567 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:44.567 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.567 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.567 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.567 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97211 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 925696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:44.567 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:38:44.568 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 94.438521317999999, 2024-09-14T22:38:44.568 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12599848299999999, 2024-09-14T22:38:44.568 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:44.568 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:44.568 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:44.568 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:44.568 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:44.568 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:44.568 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97211 2024-09-14T22:38:44.568 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.568 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:44.568 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.568 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:44.569 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:38:44.569 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.569 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.569 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.569 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:44.569 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:38:44.569 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.569 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.569 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.569 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:44.569 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074516+0000", 2024-09-14T22:38:44.569 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.42e-06 2024-09-14T22:38:44.570 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.570 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.570 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:44.570 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074521+0000", 2024-09-14T22:38:44.570 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7960000000000003e-06 2024-09-14T22:38:44.570 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.570 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.570 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:44.570 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074538+0000", 2024-09-14T22:38:44.570 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6900000000000001e-05 2024-09-14T22:38:44.570 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.571 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.571 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:44.571 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074555+0000", 2024-09-14T22:38:44.571 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.696e-05 2024-09-14T22:38:44.571 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.571 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.571 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:44.571 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200293+0000", 2024-09-14T22:38:44.571 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12573864100000001 2024-09-14T22:38:44.571 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.571 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.571 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:44.571 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200393+0000", 2024-09-14T22:38:44.572 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9264000000000004e-05 2024-09-14T22:38:44.572 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.572 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.572 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:44.572 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200513+0000", 2024-09-14T22:38:44.572 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012050200000000001 2024-09-14T22:38:44.572 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.572 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:44.572 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.572 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.572 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.572 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97212 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 934912~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:44.572 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:38:44.573 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 94.438446696, 2024-09-14T22:38:44.573 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12621127300000001, 2024-09-14T22:38:44.573 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:44.573 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:44.573 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:44.573 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:44.573 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:44.573 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:44.573 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97212 2024-09-14T22:38:44.574 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.574 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:44.574 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.574 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:44.574 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:38:44.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.574 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.574 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.574 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:44.574 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:38:44.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.574 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.574 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.575 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:44.575 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074590+0000", 2024-09-14T22:38:44.575 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1969999999999999e-06 2024-09-14T22:38:44.575 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.575 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.575 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:44.575 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074596+0000", 2024-09-14T22:38:44.575 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7629999999999997e-06 2024-09-14T22:38:44.575 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.575 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.575 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:44.576 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074614+0000", 2024-09-14T22:38:44.576 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7643e-05 2024-09-14T22:38:44.576 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.576 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.576 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:44.576 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074635+0000", 2024-09-14T22:38:44.576 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.075e-05 2024-09-14T22:38:44.576 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.576 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.576 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:44.576 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200580+0000", 2024-09-14T22:38:44.576 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125945482 2024-09-14T22:38:44.577 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.577 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.577 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:44.577 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200683+0000", 2024-09-14T22:38:44.577 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010267099999999999 2024-09-14T22:38:44.577 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.577 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.577 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:44.577 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200800+0000", 2024-09-14T22:38:44.577 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117767 2024-09-14T22:38:44.577 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.577 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:44.577 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.578 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.578 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97213 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 944128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:44.578 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:38:44.578 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 94.438381346, 2024-09-14T22:38:44.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126430709, 2024-09-14T22:38:44.578 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:44.578 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:44.578 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:44.578 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:44.578 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:44.578 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:44.578 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97213 2024-09-14T22:38:44.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.579 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:44.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.579 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:44.579 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:38:44.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.579 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:44.579 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:38:44.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.580 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:44.580 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078210+0000", 2024-09-14T22:38:44.580 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0035556839999999999 2024-09-14T22:38:44.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.580 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:44.580 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078231+0000", 2024-09-14T22:38:44.580 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0716e-05 2024-09-14T22:38:44.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.581 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:44.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078260+0000", 2024-09-14T22:38:44.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9473000000000001e-05 2024-09-14T22:38:44.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.581 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:44.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078279+0000", 2024-09-14T22:38:44.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8600000000000001e-05 2024-09-14T22:38:44.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.581 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:44.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200864+0000", 2024-09-14T22:38:44.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.122585295 2024-09-14T22:38:44.582 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.582 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:44.582 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200965+0000", 2024-09-14T22:38:44.582 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101004 2024-09-14T22:38:44.582 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.582 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:44.582 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.201085+0000", 2024-09-14T22:38:44.582 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119937 2024-09-14T22:38:44.582 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.582 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:44.582 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.582 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.583 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97219 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 999424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:44.583 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:38:44.583 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 94.434399068999994, 2024-09-14T22:38:44.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124138558, 2024-09-14T22:38:44.583 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:44.583 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:44.583 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:44.583 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:44.583 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:44.583 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:44.583 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97219 2024-09-14T22:38:44.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.583 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:44.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:44.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:38:44.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:44.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:38:44.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:44.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078638+0000", 2024-09-14T22:38:44.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.173e-06 2024-09-14T22:38:44.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:44.585 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078644+0000", 2024-09-14T22:38:44.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2570000000000004e-06 2024-09-14T22:38:44.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:44.585 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078672+0000", 2024-09-14T22:38:44.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7322999999999999e-05 2024-09-14T22:38:44.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.586 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:44.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078687+0000", 2024-09-14T22:38:44.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5639999999999999e-05 2024-09-14T22:38:44.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.586 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:44.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202567+0000", 2024-09-14T22:38:44.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123880134 2024-09-14T22:38:44.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.586 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:44.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202670+0000", 2024-09-14T22:38:44.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010273899999999999 2024-09-14T22:38:44.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.587 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:44.587 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202775+0000", 2024-09-14T22:38:44.587 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010529199999999999 2024-09-14T22:38:44.587 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.587 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:44.587 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.587 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97220 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1008640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:44.587 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:38:44.588 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 94.434329836000003, 2024-09-14T22:38:44.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12435937, 2024-09-14T22:38:44.588 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:44.588 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:44.588 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:44.588 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:44.588 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:44.588 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:44.588 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97220 2024-09-14T22:38:44.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.588 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:44.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:44.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:38:44.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:44.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:38:44.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:44.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078707+0000", 2024-09-14T22:38:44.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.266e-06 2024-09-14T22:38:44.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:44.590 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078713+0000", 2024-09-14T22:38:44.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7270000000000004e-06 2024-09-14T22:38:44.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:44.590 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078737+0000", 2024-09-14T22:38:44.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3640000000000001e-05 2024-09-14T22:38:44.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:44.591 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078751+0000", 2024-09-14T22:38:44.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.45e-05 2024-09-14T22:38:44.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.591 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:44.591 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202841+0000", 2024-09-14T22:38:44.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124089938 2024-09-14T22:38:44.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.591 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:44.591 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202944+0000", 2024-09-14T22:38:44.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010334 2024-09-14T22:38:44.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.592 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:44.592 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203065+0000", 2024-09-14T22:38:44.592 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012095900000000001 2024-09-14T22:38:44.592 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.592 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:44.592 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.592 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97221 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1017856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:44.593 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:38:44.593 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 94.434264072999994, 2024-09-14T22:38:44.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124574005, 2024-09-14T22:38:44.593 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:44.593 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:44.593 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:44.593 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:44.593 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:44.593 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:44.593 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97221 2024-09-14T22:38:44.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.593 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:44.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.593 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:44.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:38:44.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:44.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:38:44.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:44.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078773+0000", 2024-09-14T22:38:44.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.13e-06 2024-09-14T22:38:44.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.595 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:44.595 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078779+0000", 2024-09-14T22:38:44.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0260000000000004e-06 2024-09-14T22:38:44.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.595 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:44.595 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078797+0000", 2024-09-14T22:38:44.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.806e-05 2024-09-14T22:38:44.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.595 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:44.595 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078810+0000", 2024-09-14T22:38:44.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.308e-05 2024-09-14T22:38:44.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.596 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.596 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:44.596 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203131+0000", 2024-09-14T22:38:44.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12432077499999999 2024-09-14T22:38:44.596 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.596 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.596 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:44.596 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203227+0000", 2024-09-14T22:38:44.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6575000000000003e-05 2024-09-14T22:38:44.596 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.596 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.596 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:44.596 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203346+0000", 2024-09-14T22:38:44.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118359 2024-09-14T22:38:44.597 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.597 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:44.597 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.597 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97222 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:44.597 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:38:44.597 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 94.434207704000002, 2024-09-14T22:38:44.597 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12480110799999999, 2024-09-14T22:38:44.597 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:44.597 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:44.597 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:44.597 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:44.597 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:44.597 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:44.598 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97222 2024-09-14T22:38:44.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.598 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:44.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:44.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:38:44.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:44.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:38:44.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:44.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078829+0000", 2024-09-14T22:38:44.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.057e-06 2024-09-14T22:38:44.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:44.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078836+0000", 2024-09-14T22:38:44.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6159999999999998e-06 2024-09-14T22:38:44.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:44.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078856+0000", 2024-09-14T22:38:44.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0100000000000001e-05 2024-09-14T22:38:44.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:44.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078869+0000", 2024-09-14T22:38:44.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.272e-05 2024-09-14T22:38:44.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.600 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:44.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203412+0000", 2024-09-14T22:38:44.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12454283300000001 2024-09-14T22:38:44.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.600 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:44.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203511+0000", 2024-09-14T22:38:44.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9893000000000001e-05 2024-09-14T22:38:44.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:44.601 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203629+0000", 2024-09-14T22:38:44.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117889 2024-09-14T22:38:44.601 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.601 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:44.601 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.601 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97223 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1036288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:44.601 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:38:44.601 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 94.433835716999994, 2024-09-14T22:38:44.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124706279, 2024-09-14T22:38:44.601 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:44.601 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:44.601 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:44.602 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:44.602 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:44.602 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:44.602 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97223 2024-09-14T22:38:44.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.602 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:44.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.602 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:44.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:38:44.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.602 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:44.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:38:44.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:44.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079202+0000", 2024-09-14T22:38:44.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8330000000000001e-06 2024-09-14T22:38:44.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:44.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079212+0000", 2024-09-14T22:38:44.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0207000000000001e-05 2024-09-14T22:38:44.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:44.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079238+0000", 2024-09-14T22:38:44.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5626000000000002e-05 2024-09-14T22:38:44.604 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.604 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:44.604 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079256+0000", 2024-09-14T22:38:44.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8386999999999999e-05 2024-09-14T22:38:44.604 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.604 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:44.604 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203691+0000", 2024-09-14T22:38:44.605 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124434538 2024-09-14T22:38:44.605 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.605 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.605 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:44.605 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203786+0000", 2024-09-14T22:38:44.605 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5344000000000001e-05 2024-09-14T22:38:44.605 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.605 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.605 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:44.605 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203906+0000", 2024-09-14T22:38:44.605 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120344 2024-09-14T22:38:44.605 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.605 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:44.605 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.606 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97224 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1045504~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:44.606 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:38:44.606 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 94.433751896999993, 2024-09-14T22:38:44.606 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12513321199999999, 2024-09-14T22:38:44.606 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:44.606 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:44.606 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:44.606 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:44.606 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:44.606 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:44.607 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97224 2024-09-14T22:38:44.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.607 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:44.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:44.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:38:44.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:44.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:38:44.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:44.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:44.608 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082581+0000", 2024-09-14T22:38:44.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0032971530000000001 2024-09-14T22:38:44.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:44.608 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082592+0000", 2024-09-14T22:38:44.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1345999999999999e-05 2024-09-14T22:38:44.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:44.609 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082618+0000", 2024-09-14T22:38:44.609 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5873000000000001e-05 2024-09-14T22:38:44.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.609 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.609 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:44.609 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082638+0000", 2024-09-14T22:38:44.609 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9565999999999999e-05 2024-09-14T22:38:44.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.609 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.609 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:44.609 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204004+0000", 2024-09-14T22:38:44.609 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.121366038 2024-09-14T22:38:44.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.609 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.609 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:44.610 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204188+0000", 2024-09-14T22:38:44.610 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018427 2024-09-14T22:38:44.610 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:44.610 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:44.610 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:44.610 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204417+0000", 2024-09-14T22:38:44.610 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022896599999999999 2024-09-14T22:38:44.610 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.610 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:44.610 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.610 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:44.610 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:44.610 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:44.612 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:38:44.612 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:44.609+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:44.713 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:44.709+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:44.781 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:44.781 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:44.781 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:38:44.781 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:44.795 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:38:44.839 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:44.834+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:44.869 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2024-09-14T22:38:44.920 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:44.920 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:44.921 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:38:44.921 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:38:44.921 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:44.936 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:38:44.946 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:44.945+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:44.980 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:38:45.023 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:45.018+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:45.111 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:45.111 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:38:45.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:38:45.111 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:38:45.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.112 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:72252 3.2 3:442c6d72:::rbd_data.113cb4c83968.0000000000000015:head [zero 4456448~393216] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-14T22:38:45.112 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:38:45.112 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 114.40749995900001, 2024-09-14T22:38:45.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0176591500000001, 2024-09-14T22:38:45.112 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:45.112 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:45.112 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T22:38:45.112 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:45.112 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:45.112 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:45.112 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 72252 2024-09-14T22:38:45.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.113 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:45.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:45.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:38:45.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:45.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:38:45.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:45.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700101+0000", 2024-09-14T22:38:45.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4570000000000001e-06 2024-09-14T22:38:45.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:45.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700114+0000", 2024-09-14T22:38:45.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2799999999999999e-05 2024-09-14T22:38:45.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:45.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700146+0000", 2024-09-14T22:38:45.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2116000000000002e-05 2024-09-14T22:38:45.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:45.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700170+0000", 2024-09-14T22:38:45.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3770000000000001e-05 2024-09-14T22:38:45.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:45.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700285+0000", 2024-09-14T22:38:45.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000115228 2024-09-14T22:38:45.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op must wait for map", 2024-09-14T22:38:45.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700297+0000", 2024-09-14T22:38:45.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1681999999999999e-05 2024-09-14T22:38:45.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:45.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.703312+0000", 2024-09-14T22:38:45.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003015046 2024-09-14T22:38:45.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:38:45.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.703327+0000", 2024-09-14T22:38:45.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5481999999999998e-05 2024-09-14T22:38:45.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:45.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:51.717566+0000", 2024-09-14T22:38:45.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0142383509999999 2024-09-14T22:38:45.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:45.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:51.717757+0000", 2024-09-14T22:38:45.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000191218 2024-09-14T22:38:45.116 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.117 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:45.117 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.117 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:74156 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T22:38:45.117 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:38:45.117 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 109.59303124900001, 2024-09-14T22:38:45.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062073521, 2024-09-14T22:38:45.117 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:45.117 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:45.117 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:45.117 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:45.117 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:45.117 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:45.118 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74156 2024-09-14T22:38:45.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.118 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:45.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:45.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:38:45.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:45.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:38:45.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:45.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514569+0000", 2024-09-14T22:38:45.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0229999999999999e-06 2024-09-14T22:38:45.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:45.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514596+0000", 2024-09-14T22:38:45.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6903e-05 2024-09-14T22:38:45.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:45.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514623+0000", 2024-09-14T22:38:45.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7206e-05 2024-09-14T22:38:45.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:45.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514637+0000", 2024-09-14T22:38:45.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:38:45.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:45.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514781+0000", 2024-09-14T22:38:45.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014394899999999999 2024-09-14T22:38:45.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:45.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.515005+0000", 2024-09-14T22:38:45.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022458499999999999 2024-09-14T22:38:45.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:38:45.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.527785+0000", 2024-09-14T22:38:45.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012780112 2024-09-14T22:38:45.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.121 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:38:45.121 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576209+0000", 2024-09-14T22:38:45.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.048423992999999999 2024-09-14T22:38:45.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.121 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:45.121 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576346+0000", 2024-09-14T22:38:45.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000136909 2024-09-14T22:38:45.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:38:45.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576517+0000", 2024-09-14T22:38:45.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000170848 2024-09-14T22:38:45.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:45.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576640+0000", 2024-09-14T22:38:45.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000122953 2024-09-14T22:38:45.122 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.122 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:45.122 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.122 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82513 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 557056~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:45.122 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:38:45.123 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 105.68839943, 2024-09-14T22:38:45.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059345486000000003, 2024-09-14T22:38:45.123 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:45.123 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:45.123 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:45.123 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:45.123 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:45.123 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:45.123 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82513 2024-09-14T22:38:45.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.123 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:45.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:45.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:38:45.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:45.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:38:45.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:45.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419200+0000", 2024-09-14T22:38:45.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.184e-06 2024-09-14T22:38:45.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:45.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419207+0000", 2024-09-14T22:38:45.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2690000000000002e-06 2024-09-14T22:38:45.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:45.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419224+0000", 2024-09-14T22:38:45.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6623e-05 2024-09-14T22:38:45.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:45.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419236+0000", 2024-09-14T22:38:45.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2367e-05 2024-09-14T22:38:45.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.126 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:45.126 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478294+0000", 2024-09-14T22:38:45.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059057927000000003 2024-09-14T22:38:45.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.126 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:45.126 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478402+0000", 2024-09-14T22:38:45.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107921 2024-09-14T22:38:45.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.126 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:45.126 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478544+0000", 2024-09-14T22:38:45.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014219499999999999 2024-09-14T22:38:45.127 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.127 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:45.127 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.127 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82514 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 704512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:45.127 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:38:45.127 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 105.68834032700001, 2024-09-14T22:38:45.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059580793, 2024-09-14T22:38:45.127 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:45.127 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:45.127 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:45.127 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:45.127 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:45.128 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:45.128 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82514 2024-09-14T22:38:45.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.128 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:45.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:45.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:38:45.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:45.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:38:45.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:45.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419259+0000", 2024-09-14T22:38:45.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.353e-06 2024-09-14T22:38:45.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:45.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419276+0000", 2024-09-14T22:38:45.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7567e-05 2024-09-14T22:38:45.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:45.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419294+0000", 2024-09-14T22:38:45.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.713e-05 2024-09-14T22:38:45.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:45.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419306+0000", 2024-09-14T22:38:45.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.279e-05 2024-09-14T22:38:45.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:45.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478589+0000", 2024-09-14T22:38:45.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059282978 2024-09-14T22:38:45.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:45.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478694+0000", 2024-09-14T22:38:45.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000104561 2024-09-14T22:38:45.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:45.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478838+0000", 2024-09-14T22:38:45.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014441399999999999 2024-09-14T22:38:45.131 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.131 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:45.131 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.131 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82515 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:45.131 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:38:45.131 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 105.68827517699999, 2024-09-14T22:38:45.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059815123999999997, 2024-09-14T22:38:45.132 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:45.132 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:45.132 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:45.132 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:45.132 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:45.132 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:45.132 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82515 2024-09-14T22:38:45.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.132 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:45.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:45.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:38:45.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:45.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:38:45.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:45.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419324+0000", 2024-09-14T22:38:45.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2130000000000001e-06 2024-09-14T22:38:45.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:45.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419331+0000", 2024-09-14T22:38:45.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4730000000000003e-06 2024-09-14T22:38:45.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:45.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419348+0000", 2024-09-14T22:38:45.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6269999999999998e-05 2024-09-14T22:38:45.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:45.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419360+0000", 2024-09-14T22:38:45.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2243e-05 2024-09-14T22:38:45.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:45.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478884+0000", 2024-09-14T22:38:45.135 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059524369000000001 2024-09-14T22:38:45.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.135 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.135 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:45.135 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478990+0000", 2024-09-14T22:38:45.135 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010569 2024-09-14T22:38:45.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.135 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.135 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:45.135 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479138+0000", 2024-09-14T22:38:45.135 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000147866 2024-09-14T22:38:45.135 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.135 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:45.135 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.136 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82516 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 999424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:45.136 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:38:45.136 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 105.68822267199999, 2024-09-14T22:38:45.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061979595999999998, 2024-09-14T22:38:45.136 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:45.136 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:45.136 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:45.136 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:45.136 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:45.136 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:45.136 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82516 2024-09-14T22:38:45.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.137 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:45.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:45.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:38:45.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:45.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:38:45.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:45.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419376+0000", 2024-09-14T22:38:45.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.237e-06 2024-09-14T22:38:45.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:45.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419383+0000", 2024-09-14T22:38:45.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8170000000000001e-06 2024-09-14T22:38:45.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:45.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419400+0000", 2024-09-14T22:38:45.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6636000000000001e-05 2024-09-14T22:38:45.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:45.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419412+0000", 2024-09-14T22:38:45.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2357e-05 2024-09-14T22:38:45.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.139 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:45.139 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481080+0000", 2024-09-14T22:38:45.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061668032999999997 2024-09-14T22:38:45.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.139 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:45.139 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481193+0000", 2024-09-14T22:38:45.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112335 2024-09-14T22:38:45.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.139 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:45.139 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481355+0000", 2024-09-14T22:38:45.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016218100000000001 2024-09-14T22:38:45.140 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.140 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:45.140 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.140 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82538 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1097728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:45.140 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:38:45.140 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 105.688170602, 2024-09-14T22:38:45.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059971705, 2024-09-14T22:38:45.140 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:45.140 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:45.140 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:45.140 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:45.140 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:45.141 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:45.141 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82538 2024-09-14T22:38:45.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.141 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:45.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:45.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:38:45.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:45.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:38:45.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:45.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419429+0000", 2024-09-14T22:38:45.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.497e-06 2024-09-14T22:38:45.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:45.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419436+0000", 2024-09-14T22:38:45.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2099999999999996e-06 2024-09-14T22:38:45.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:45.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419453+0000", 2024-09-14T22:38:45.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6840000000000001e-05 2024-09-14T22:38:45.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:45.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419465+0000", 2024-09-14T22:38:45.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2283000000000001e-05 2024-09-14T22:38:45.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.143 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:45.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479182+0000", 2024-09-14T22:38:45.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059717339000000001 2024-09-14T22:38:45.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.143 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:45.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479297+0000", 2024-09-14T22:38:45.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000114259 2024-09-14T22:38:45.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:45.144 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479399+0000", 2024-09-14T22:38:45.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102277 2024-09-14T22:38:45.144 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.144 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:45.144 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.144 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82539 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:45.144 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:38:45.144 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 105.688117752, 2024-09-14T22:38:45.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060166935999999997, 2024-09-14T22:38:45.144 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:45.144 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:45.144 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:45.144 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:45.145 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:45.145 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:45.145 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82539 2024-09-14T22:38:45.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.145 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:45.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:45.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:38:45.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:45.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:38:45.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:45.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419482+0000", 2024-09-14T22:38:45.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4130000000000001e-06 2024-09-14T22:38:45.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:45.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419488+0000", 2024-09-14T22:38:45.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8460000000000002e-06 2024-09-14T22:38:45.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:45.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419505+0000", 2024-09-14T22:38:45.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6477000000000002e-05 2024-09-14T22:38:45.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:45.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419518+0000", 2024-09-14T22:38:45.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2797e-05 2024-09-14T22:38:45.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:45.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479438+0000", 2024-09-14T22:38:45.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059919897 2024-09-14T22:38:45.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:45.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479547+0000", 2024-09-14T22:38:45.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109807 2024-09-14T22:38:45.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.148 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:45.148 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479647+0000", 2024-09-14T22:38:45.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9698999999999995e-05 2024-09-14T22:38:45.148 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.148 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:45.148 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.148 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82542 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:45.148 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:38:45.148 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 105.688065309, 2024-09-14T22:38:45.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060367430999999999, 2024-09-14T22:38:45.148 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:45.148 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:45.148 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:45.149 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:45.149 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:45.149 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:45.149 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82542 2024-09-14T22:38:45.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.149 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:45.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.149 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:45.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:38:45.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.149 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:45.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:38:45.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:45.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419534+0000", 2024-09-14T22:38:45.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T22:38:45.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:45.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419542+0000", 2024-09-14T22:38:45.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8199999999999997e-06 2024-09-14T22:38:45.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:45.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419559+0000", 2024-09-14T22:38:45.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6852999999999999e-05 2024-09-14T22:38:45.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:45.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419571+0000", 2024-09-14T22:38:45.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2364000000000001e-05 2024-09-14T22:38:45.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:45.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479686+0000", 2024-09-14T22:38:45.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060115522999999997 2024-09-14T22:38:45.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:45.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479795+0000", 2024-09-14T22:38:45.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010831399999999999 2024-09-14T22:38:45.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.152 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:45.152 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479900+0000", 2024-09-14T22:38:45.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105284 2024-09-14T22:38:45.152 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.152 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:45.152 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.152 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82544 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:45.152 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:38:45.152 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 105.688011573, 2024-09-14T22:38:45.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060590515999999997, 2024-09-14T22:38:45.152 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:45.153 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:45.153 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:45.153 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:45.153 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:45.153 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:45.153 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82544 2024-09-14T22:38:45.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.153 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:45.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:45.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:38:45.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:45.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:38:45.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:45.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419588+0000", 2024-09-14T22:38:45.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.35e-06 2024-09-14T22:38:45.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:45.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419595+0000", 2024-09-14T22:38:45.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4030000000000001e-06 2024-09-14T22:38:45.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:45.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419612+0000", 2024-09-14T22:38:45.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6670000000000001e-05 2024-09-14T22:38:45.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:45.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419624+0000", 2024-09-14T22:38:45.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-05 2024-09-14T22:38:45.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:45.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479957+0000", 2024-09-14T22:38:45.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060333287999999999 2024-09-14T22:38:45.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:45.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480074+0000", 2024-09-14T22:38:45.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116443 2024-09-14T22:38:45.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:45.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480177+0000", 2024-09-14T22:38:45.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000103102 2024-09-14T22:38:45.156 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.156 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:45.156 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.156 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82546 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:45.157 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:38:45.157 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 105.687958654, 2024-09-14T22:38:45.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060795428999999998, 2024-09-14T22:38:45.157 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:45.157 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:45.157 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:45.157 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:45.157 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:45.157 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:45.157 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82546 2024-09-14T22:38:45.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.157 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:45.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:45.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:38:45.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:45.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:38:45.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:45.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419640+0000", 2024-09-14T22:38:45.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.254e-06 2024-09-14T22:38:45.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:45.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419647+0000", 2024-09-14T22:38:45.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9e-06 2024-09-14T22:38:45.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:45.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419664+0000", 2024-09-14T22:38:45.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6345999999999999e-05 2024-09-14T22:38:45.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:45.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419676+0000", 2024-09-14T22:38:45.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.224e-05 2024-09-14T22:38:45.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:45.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480218+0000", 2024-09-14T22:38:45.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060542133999999997 2024-09-14T22:38:45.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:45.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480336+0000", 2024-09-14T22:38:45.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117472 2024-09-14T22:38:45.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:45.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480435+0000", 2024-09-14T22:38:45.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9083e-05 2024-09-14T22:38:45.161 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.161 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:45.161 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.161 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82548 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:45.161 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:38:45.161 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 105.68790664399999, 2024-09-14T22:38:45.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061011636000000001, 2024-09-14T22:38:45.161 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:45.161 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:45.161 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:45.161 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:45.161 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:45.162 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:45.162 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82548 2024-09-14T22:38:45.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.162 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:45.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:45.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:38:45.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:45.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:38:45.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:45.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419693+0000", 2024-09-14T22:38:45.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.387e-06 2024-09-14T22:38:45.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:45.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419699+0000", 2024-09-14T22:38:45.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8730000000000001e-06 2024-09-14T22:38:45.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:45.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419716+0000", 2024-09-14T22:38:45.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6416e-05 2024-09-14T22:38:45.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:45.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419728+0000", 2024-09-14T22:38:45.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2597e-05 2024-09-14T22:38:45.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:45.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480476+0000", 2024-09-14T22:38:45.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060747989000000002 2024-09-14T22:38:45.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:45.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480583+0000", 2024-09-14T22:38:45.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010638500000000001 2024-09-14T22:38:45.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:45.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480703+0000", 2024-09-14T22:38:45.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011998900000000001 2024-09-14T22:38:45.165 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.165 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:45.165 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.165 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82550 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1982464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:45.165 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:38:45.165 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 105.687853891, 2024-09-14T22:38:45.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061295483999999997, 2024-09-14T22:38:45.166 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:45.166 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:45.166 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:45.166 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:45.166 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:45.166 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:45.166 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82550 2024-09-14T22:38:45.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.166 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:45.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:45.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:38:45.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:45.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:38:45.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:45.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419745+0000", 2024-09-14T22:38:45.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.406e-06 2024-09-14T22:38:45.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:45.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419753+0000", 2024-09-14T22:38:45.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8140000000000001e-06 2024-09-14T22:38:45.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:45.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419770+0000", 2024-09-14T22:38:45.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6719999999999999e-05 2024-09-14T22:38:45.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:45.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419782+0000", 2024-09-14T22:38:45.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2252999999999999e-05 2024-09-14T22:38:45.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:45.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480745+0000", 2024-09-14T22:38:45.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060962385000000001 2024-09-14T22:38:45.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:45.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480900+0000", 2024-09-14T22:38:45.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015513899999999999 2024-09-14T22:38:45.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:45.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481039+0000", 2024-09-14T22:38:45.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013976700000000001 2024-09-14T22:38:45.169 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.169 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:45.170 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.170 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82588 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1064960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:45.170 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:38:45.170 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 105.687800001, 2024-09-14T22:38:45.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061863732999999997, 2024-09-14T22:38:45.170 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:45.170 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:45.170 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:45.170 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:45.170 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:45.170 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:45.170 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82588 2024-09-14T22:38:45.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.171 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:45.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:45.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:38:45.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:45.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:38:45.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:45.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419799+0000", 2024-09-14T22:38:45.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4160000000000001e-06 2024-09-14T22:38:45.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:45.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419807+0000", 2024-09-14T22:38:45.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2169999999999994e-06 2024-09-14T22:38:45.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:45.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419824+0000", 2024-09-14T22:38:45.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6776e-05 2024-09-14T22:38:45.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:45.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419837+0000", 2024-09-14T22:38:45.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2680000000000001e-05 2024-09-14T22:38:45.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:45.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481395+0000", 2024-09-14T22:38:45.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061558416999999997 2024-09-14T22:38:45.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:45.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481532+0000", 2024-09-14T22:38:45.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000136861 2024-09-14T22:38:45.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:45.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481662+0000", 2024-09-14T22:38:45.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012936600000000001 2024-09-14T22:38:45.174 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.174 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:45.174 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.174 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82589 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:45.174 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:38:45.174 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 105.687745406, 2024-09-14T22:38:45.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062064592000000002, 2024-09-14T22:38:45.174 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:45.174 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:45.174 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:45.174 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:45.175 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:45.175 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:45.175 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82589 2024-09-14T22:38:45.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.175 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:45.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:45.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:38:45.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:45.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:38:45.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:45.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419854+0000", 2024-09-14T22:38:45.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.367e-06 2024-09-14T22:38:45.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:45.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419862+0000", 2024-09-14T22:38:45.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8930000000000005e-06 2024-09-14T22:38:45.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:45.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419896+0000", 2024-09-14T22:38:45.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4134000000000002e-05 2024-09-14T22:38:45.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:45.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419909+0000", 2024-09-14T22:38:45.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3332999999999999e-05 2024-09-14T22:38:45.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:45.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481702+0000", 2024-09-14T22:38:45.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061792947000000001 2024-09-14T22:38:45.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:45.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481808+0000", 2024-09-14T22:38:45.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105655 2024-09-14T22:38:45.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:45.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481917+0000", 2024-09-14T22:38:45.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109263 2024-09-14T22:38:45.178 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.178 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:45.178 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.178 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82590 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1359872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:45.178 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:38:45.178 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 105.687671799, 2024-09-14T22:38:45.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062362895000000002, 2024-09-14T22:38:45.179 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:45.179 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:45.179 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:45.179 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:45.179 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:45.179 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:45.179 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82590 2024-09-14T22:38:45.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.179 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:45.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:45.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:38:45.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:45.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:38:45.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:45.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419928+0000", 2024-09-14T22:38:45.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4470000000000001e-06 2024-09-14T22:38:45.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:45.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419942+0000", 2024-09-14T22:38:45.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4229e-05 2024-09-14T22:38:45.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:45.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419972+0000", 2024-09-14T22:38:45.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.065e-05 2024-09-14T22:38:45.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:45.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419985+0000", 2024-09-14T22:38:45.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2903e-05 2024-09-14T22:38:45.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:45.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481960+0000", 2024-09-14T22:38:45.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061974435000000001 2024-09-14T22:38:45.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:45.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482106+0000", 2024-09-14T22:38:45.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014643800000000001 2024-09-14T22:38:45.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:45.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482289+0000", 2024-09-14T22:38:45.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000182793 2024-09-14T22:38:45.182 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.182 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:45.182 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.183 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82592 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:45.183 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:38:45.183 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 105.687595577, 2024-09-14T22:38:45.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062631507000000003, 2024-09-14T22:38:45.183 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:45.183 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:45.183 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:45.183 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:45.183 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:45.183 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:45.183 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82592 2024-09-14T22:38:45.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.183 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:45.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:45.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:38:45.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:45.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:38:45.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:45.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420017+0000", 2024-09-14T22:38:45.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4744e-05 2024-09-14T22:38:45.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:45.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420036+0000", 2024-09-14T22:38:45.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8680000000000001e-05 2024-09-14T22:38:45.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:45.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420054+0000", 2024-09-14T22:38:45.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7799999999999999e-05 2024-09-14T22:38:45.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:45.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420067+0000", 2024-09-14T22:38:45.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3013000000000001e-05 2024-09-14T22:38:45.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:45.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482358+0000", 2024-09-14T22:38:45.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062291445000000001 2024-09-14T22:38:45.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:45.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482469+0000", 2024-09-14T22:38:45.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110608 2024-09-14T22:38:45.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:45.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482634+0000", 2024-09-14T22:38:45.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000165217 2024-09-14T22:38:45.187 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.187 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:45.187 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.187 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82594 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1654784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:45.187 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:38:45.187 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 105.687497601, 2024-09-14T22:38:45.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062893849000000002, 2024-09-14T22:38:45.187 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:45.187 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:45.187 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:45.187 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:45.187 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:45.187 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:45.188 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82594 2024-09-14T22:38:45.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.188 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:45.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:45.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:38:45.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:45.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:38:45.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:45.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420102+0000", 2024-09-14T22:38:45.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5e-06 2024-09-14T22:38:45.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:45.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420115+0000", 2024-09-14T22:38:45.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.324e-05 2024-09-14T22:38:45.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:45.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420133+0000", 2024-09-14T22:38:45.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8213e-05 2024-09-14T22:38:45.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:45.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420147+0000", 2024-09-14T22:38:45.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3284e-05 2024-09-14T22:38:45.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:45.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482688+0000", 2024-09-14T22:38:45.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062541425999999997 2024-09-14T22:38:45.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:45.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482794+0000", 2024-09-14T22:38:45.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010575500000000001 2024-09-14T22:38:45.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:45.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482994+0000", 2024-09-14T22:38:45.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020043099999999999 2024-09-14T22:38:45.191 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.191 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:45.191 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.191 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82596 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:45.191 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:38:45.192 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 105.687434301, 2024-09-14T22:38:45.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063187238000000007, 2024-09-14T22:38:45.192 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:45.192 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:45.192 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:45.192 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:45.192 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:45.192 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:45.192 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82596 2024-09-14T22:38:45.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.192 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:45.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:45.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:38:45.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:45.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:38:45.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:45.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420165+0000", 2024-09-14T22:38:45.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4100000000000001e-06 2024-09-14T22:38:45.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:45.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420173+0000", 2024-09-14T22:38:45.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8129999999999996e-06 2024-09-14T22:38:45.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:45.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420190+0000", 2024-09-14T22:38:45.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7076999999999999e-05 2024-09-14T22:38:45.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:45.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420203+0000", 2024-09-14T22:38:45.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2683e-05 2024-09-14T22:38:45.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:45.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483060+0000", 2024-09-14T22:38:45.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062857114000000006 2024-09-14T22:38:45.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:45.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483167+0000", 2024-09-14T22:38:45.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010713999999999999 2024-09-14T22:38:45.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:45.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483351+0000", 2024-09-14T22:38:45.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018400099999999999 2024-09-14T22:38:45.196 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.196 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:45.196 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.196 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82597 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1949696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:45.196 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:38:45.196 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 105.68737747500001, 2024-09-14T22:38:45.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063466513000000002, 2024-09-14T22:38:45.196 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:45.196 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:45.196 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:45.196 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:45.196 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:45.197 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:45.197 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82597 2024-09-14T22:38:45.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.197 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:45.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:45.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:38:45.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:45.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:38:45.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:45.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:45.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420221+0000", 2024-09-14T22:38:45.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.043e-06 2024-09-14T22:38:45.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:45.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420229+0000", 2024-09-14T22:38:45.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0469999999999994e-06 2024-09-14T22:38:45.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:45.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420246+0000", 2024-09-14T22:38:45.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6339999999999999e-05 2024-09-14T22:38:45.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:45.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420258+0000", 2024-09-14T22:38:45.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2296000000000001e-05 2024-09-14T22:38:45.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:45.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483402+0000", 2024-09-14T22:38:45.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063144318000000005 2024-09-14T22:38:45.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:45.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483530+0000", 2024-09-14T22:38:45.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012713199999999999 2024-09-14T22:38:45.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:45.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:45.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:45.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483687+0000", 2024-09-14T22:38:45.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015733699999999999 2024-09-14T22:38:45.200 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.200 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:45.200 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.200 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:45.200 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:45.200 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:45.201 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:38:45.202 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:38:45.202 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:38:45.203 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:38:45.203 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:38:45.203 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:45.194+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:45.303 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:45.301+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:45.374 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:45.374 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:45.374 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:38:45.374 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:45.388 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:38:45.388 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:38:45.389 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:38:45.404 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:45.401+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:45.422 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:38:45.422 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:38:45.452 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:38:45.452 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:38:45.479 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:38:45.479 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:38:45.504 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:45.501+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:45.507 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:45.507 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:45.507 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:38:45.507 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:38:45.508 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:45.510 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:38:45.511 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:38:45.522 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:38:45.540 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:38:45.540 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:38:45.605 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:45.598+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:45.644 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:45.644 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:45.659 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:38:45.706 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:45.698+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:45.757 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-14T22:38:45.757 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:38:45.757 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-14T22:38:45.757 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T22:38:45.757 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T22:38:45.757 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-14T22:38:45.806 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:45.805+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:45.809 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:45.809 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:45.827 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:38:45.908 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:45.908 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "" 2024-09-14T22:38:45.908 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:45.908 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:45.905+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:46.008 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:46.008 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:46.008 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:46.005+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:46.022 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:38:46.109 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:46.102+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:46.191 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:46.191 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:46.205 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:38:46.250 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:46.249+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:46.311 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:46.309+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:46.324 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:46.324 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:46.338 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:38:46.411 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:46.406+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:46.509 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:46.510 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:46.512 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:46.509+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:46.524 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:38:46.612 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:46.609+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:46.685 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:46.685 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:46.700 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:38:46.713 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:46.710+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:46.814 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:46.806+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:46.870 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:46.870 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:46.885 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:38:46.903 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T22:38:46.930 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:46.929+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:47.014 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:47.014 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:47.015 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:47.010+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:47.030 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:38:47.116 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:47.113+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:47.154 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:47.154 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:47.168 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:38:47.217 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:47.210+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:47.318 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:47.317+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:47.341 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:47.341 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:47.356 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:38:47.442 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:47.441+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:47.520 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:47.514+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:47.523 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:47.523 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:47.537 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:38:47.567 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:38:47.621 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:47.618+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:47.719 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:47.719 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:47.724 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:47.714+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:47.724 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:38:47.725 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:38:47.734 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:38:47.741 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:38:47.741 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:38:47.758 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:38:47.758 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:38:47.774 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:38:47.775 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:38:47.795 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:38:47.795 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:38:47.814 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:38:47.814 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:38:47.822 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:47.821+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:47.833 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:38:47.833 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:38:47.854 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:38:47.854 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:38:47.867 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:47.867 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:47.867 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:38:47.867 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:47.882 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:38:47.907 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2024-09-14T22:38:47.923 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:47.921+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:48.014 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:48.014 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:48.015 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:38:48.015 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:38:48.015 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:48.024 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:48.021+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:48.030 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:38:48.128 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:48.121+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:48.165 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:48.165 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:38:48.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:38:48.165 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:38:48.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.165 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:6395 3.9 3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:4 [read 8386560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:38:48.165 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:38:48.165 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 144.226356885, 2024-09-14T22:38:48.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054534379000000001, 2024-09-14T22:38:48.165 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.166 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.166 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.166 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.166 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.166 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.166 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 6395 2024-09-14T22:38:48.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.166 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:38:48.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933832+0000", 2024-09-14T22:38:48.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933833+0000", 2024-09-14T22:38:48.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3859999999999999e-06 2024-09-14T22:38:48.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933838+0000", 2024-09-14T22:38:48.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8709999999999996e-06 2024-09-14T22:38:48.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933854+0000", 2024-09-14T22:38:48.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6189e-05 2024-09-14T22:38:48.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.933871+0000", 2024-09-14T22:38:48.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.626e-05 2024-09-14T22:38:48.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988198+0000", 2024-09-14T22:38:48.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054327581999999999 2024-09-14T22:38:48.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988274+0000", 2024-09-14T22:38:48.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5759000000000004e-05 2024-09-14T22:38:48.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:23.988366+0000", 2024-09-14T22:38:48.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2331999999999996e-05 2024-09-14T22:38:48.169 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.169 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.170 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.170 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104944 3.3 3:c3780f3b:::rbd_data.113ca561f7a9.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:48.170 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:38:48.170 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 89.126755012000004, 2024-09-14T22:38:48.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073041087000000005, 2024-09-14T22:38:48.170 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.170 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.170 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:48.170 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.170 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.170 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.170 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104944 2024-09-14T22:38:48.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.171 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:38:48.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033434+0000", 2024-09-14T22:38:48.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033436+0000", 2024-09-14T22:38:48.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2060000000000001e-06 2024-09-14T22:38:48.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033448+0000", 2024-09-14T22:38:48.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.169e-05 2024-09-14T22:38:48.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033474+0000", 2024-09-14T22:38:48.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6400000000000001e-05 2024-09-14T22:38:48.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033489+0000", 2024-09-14T22:38:48.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5067e-05 2024-09-14T22:38:48.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033658+0000", 2024-09-14T22:38:48.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000168565 2024-09-14T22:38:48.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.033793+0000", 2024-09-14T22:38:48.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013536900000000001 2024-09-14T22:38:48.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:38:48.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.047559+0000", 2024-09-14T22:38:48.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013766294 2024-09-14T22:38:48.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:38:48.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106157+0000", 2024-09-14T22:38:48.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.058597110000000001 2024-09-14T22:38:48.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:48.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106290+0000", 2024-09-14T22:38:48.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000133244 2024-09-14T22:38:48.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:38:48.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106396+0000", 2024-09-14T22:38:48.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106246 2024-09-14T22:38:48.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:19.106475+0000", 2024-09-14T22:38:48.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8895999999999998e-05 2024-09-14T22:38:48.175 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.175 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.175 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.176 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:111858 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 10403840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:48.176 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:38:48.176 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.336301891000005, 2024-09-14T22:38:48.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054398798999999998, 2024-09-14T22:38:48.176 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.176 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.176 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.176 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.176 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.176 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.176 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 111858 2024-09-14T22:38:48.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.176 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:38:48.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823887+0000", 2024-09-14T22:38:48.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823888+0000", 2024-09-14T22:38:48.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0200000000000001e-07 2024-09-14T22:38:48.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823927+0000", 2024-09-14T22:38:48.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8940999999999999e-05 2024-09-14T22:38:48.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823940+0000", 2024-09-14T22:38:48.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3682e-05 2024-09-14T22:38:48.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823950+0000", 2024-09-14T22:38:48.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7110000000000007e-06 2024-09-14T22:38:48.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878202+0000", 2024-09-14T22:38:48.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054251969999999997 2024-09-14T22:38:48.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878249+0000", 2024-09-14T22:38:48.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6854999999999998e-05 2024-09-14T22:38:48.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878286+0000", 2024-09-14T22:38:48.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6838000000000002e-05 2024-09-14T22:38:48.179 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.179 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.180 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.180 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:111859 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 10452992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:48.180 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:38:48.180 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.336212771999996, 2024-09-14T22:38:48.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054435292000000003, 2024-09-14T22:38:48.180 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.180 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.180 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.180 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.180 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.180 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.180 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 111859 2024-09-14T22:38:48.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.181 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:38:48.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823976+0000", 2024-09-14T22:38:48.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823977+0000", 2024-09-14T22:38:48.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3500000000000005e-07 2024-09-14T22:38:48.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.823981+0000", 2024-09-14T22:38:48.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2050000000000004e-06 2024-09-14T22:38:48.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824007+0000", 2024-09-14T22:38:48.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.62e-05 2024-09-14T22:38:48.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824018+0000", 2024-09-14T22:38:48.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0389e-05 2024-09-14T22:38:48.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878312+0000", 2024-09-14T22:38:48.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054294754000000001 2024-09-14T22:38:48.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878350+0000", 2024-09-14T22:38:48.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7982000000000003e-05 2024-09-14T22:38:48.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878411+0000", 2024-09-14T22:38:48.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0927000000000001e-05 2024-09-14T22:38:48.184 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.184 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.184 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.184 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112030 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6316032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:48.184 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:38:48.184 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.336158429999998, 2024-09-14T22:38:48.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054597826000000002, 2024-09-14T22:38:48.184 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.184 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.184 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.184 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.185 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.185 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.185 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112030 2024-09-14T22:38:48.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.185 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:38:48.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824030+0000", 2024-09-14T22:38:48.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824031+0000", 2024-09-14T22:38:48.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3600000000000003e-07 2024-09-14T22:38:48.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824036+0000", 2024-09-14T22:38:48.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7350000000000004e-06 2024-09-14T22:38:48.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824049+0000", 2024-09-14T22:38:48.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3329000000000001e-05 2024-09-14T22:38:48.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824057+0000", 2024-09-14T22:38:48.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2460000000000003e-06 2024-09-14T22:38:48.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878464+0000", 2024-09-14T22:38:48.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054406207999999998 2024-09-14T22:38:48.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878554+0000", 2024-09-14T22:38:48.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0199999999999997e-05 2024-09-14T22:38:48.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878628+0000", 2024-09-14T22:38:48.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4371999999999999e-05 2024-09-14T22:38:48.188 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.188 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.188 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.188 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112031 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6365184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:48.189 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:38:48.189 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.336120112000003, 2024-09-14T22:38:48.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054741868999999999, 2024-09-14T22:38:48.189 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.189 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.189 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.189 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.189 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.189 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.189 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112031 2024-09-14T22:38:48.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.189 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:38:48.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824069+0000", 2024-09-14T22:38:48.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824070+0000", 2024-09-14T22:38:48.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5099999999999998e-07 2024-09-14T22:38:48.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824073+0000", 2024-09-14T22:38:48.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6940000000000001e-06 2024-09-14T22:38:48.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824084+0000", 2024-09-14T22:38:48.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0380000000000001e-05 2024-09-14T22:38:48.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824091+0000", 2024-09-14T22:38:48.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5730000000000002e-06 2024-09-14T22:38:48.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878670+0000", 2024-09-14T22:38:48.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054579176 2024-09-14T22:38:48.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878742+0000", 2024-09-14T22:38:48.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T22:38:48.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878811+0000", 2024-09-14T22:38:48.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8170999999999995e-05 2024-09-14T22:38:48.193 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.193 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.193 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.193 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112032 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6414336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:48.193 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:38:48.193 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.336086245999994, 2024-09-14T22:38:48.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054931916999999997, 2024-09-14T22:38:48.193 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.194 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.194 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.194 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.194 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.194 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.194 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112032 2024-09-14T22:38:48.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.194 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:38:48.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:38:48.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824103+0000", 2024-09-14T22:38:48.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:38:48.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824107+0000", 2024-09-14T22:38:48.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5640000000000001e-06 2024-09-14T22:38:48.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824117+0000", 2024-09-14T22:38:48.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0366e-05 2024-09-14T22:38:48.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824158+0000", 2024-09-14T22:38:48.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0670000000000002e-05 2024-09-14T22:38:48.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878853+0000", 2024-09-14T22:38:48.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054694853000000002 2024-09-14T22:38:48.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.878934+0000", 2024-09-14T22:38:48.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1754000000000004e-05 2024-09-14T22:38:48.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879034+0000", 2024-09-14T22:38:48.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000100115 2024-09-14T22:38:48.197 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.197 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.197 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.198 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112033 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6463488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:48.198 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:38:48.198 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.336017588000004, 2024-09-14T22:38:48.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054991076999999999, 2024-09-14T22:38:48.198 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.198 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.198 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.198 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.198 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.198 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.198 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112033 2024-09-14T22:38:48.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.199 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:38:48.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824171+0000", 2024-09-14T22:38:48.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824172+0000", 2024-09-14T22:38:48.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9999999999999996e-07 2024-09-14T22:38:48.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824177+0000", 2024-09-14T22:38:48.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5310000000000003e-06 2024-09-14T22:38:48.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824188+0000", 2024-09-14T22:38:48.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1235e-05 2024-09-14T22:38:48.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824197+0000", 2024-09-14T22:38:48.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7069999999999998e-06 2024-09-14T22:38:48.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879075+0000", 2024-09-14T22:38:48.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054878749999999997 2024-09-14T22:38:48.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879118+0000", 2024-09-14T22:38:48.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2749e-05 2024-09-14T22:38:48.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879162+0000", 2024-09-14T22:38:48.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4205000000000001e-05 2024-09-14T22:38:48.202 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.202 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.202 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.202 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112034 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6512640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:48.202 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:38:48.202 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.335980637000006, 2024-09-14T22:38:48.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055181246000000003, 2024-09-14T22:38:48.203 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.203 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.203 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.203 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.203 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.203 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.203 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112034 2024-09-14T22:38:48.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.203 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:38:48.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824208+0000", 2024-09-14T22:38:48.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824209+0000", 2024-09-14T22:38:48.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5499999999999998e-07 2024-09-14T22:38:48.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824212+0000", 2024-09-14T22:38:48.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6119999999999999e-06 2024-09-14T22:38:48.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824224+0000", 2024-09-14T22:38:48.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1127999999999999e-05 2024-09-14T22:38:48.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824232+0000", 2024-09-14T22:38:48.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4700000000000002e-06 2024-09-14T22:38:48.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879209+0000", 2024-09-14T22:38:48.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054977201000000003 2024-09-14T22:38:48.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879308+0000", 2024-09-14T22:38:48.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8419999999999996e-05 2024-09-14T22:38:48.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879389+0000", 2024-09-14T22:38:48.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1760000000000003e-05 2024-09-14T22:38:48.207 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.207 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.207 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.207 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112035 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6561792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:48.207 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:38:48.207 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.335886908999996, 2024-09-14T22:38:48.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055300522999999997, 2024-09-14T22:38:48.208 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.208 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.208 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.208 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.208 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.208 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.208 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112035 2024-09-14T22:38:48.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.208 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.208 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:38:48.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824302+0000", 2024-09-14T22:38:48.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824303+0000", 2024-09-14T22:38:48.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5099999999999998e-07 2024-09-14T22:38:48.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824308+0000", 2024-09-14T22:38:48.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6689999999999997e-06 2024-09-14T22:38:48.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824744+0000", 2024-09-14T22:38:48.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00043601600000000001 2024-09-14T22:38:48.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824756+0000", 2024-09-14T22:38:48.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2476999999999999e-05 2024-09-14T22:38:48.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879434+0000", 2024-09-14T22:38:48.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054677630999999997 2024-09-14T22:38:48.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879518+0000", 2024-09-14T22:38:48.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4220000000000003e-05 2024-09-14T22:38:48.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879602+0000", 2024-09-14T22:38:48.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4559000000000001e-05 2024-09-14T22:38:48.212 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.212 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.212 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.212 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112036 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6610944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:48.212 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:38:48.212 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.335395171000002, 2024-09-14T22:38:48.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055036320999999999, 2024-09-14T22:38:48.212 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.212 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.212 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.212 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.213 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.213 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.213 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112036 2024-09-14T22:38:48.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.213 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:38:48.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824794+0000", 2024-09-14T22:38:48.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824795+0000", 2024-09-14T22:38:48.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-06 2024-09-14T22:38:48.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824802+0000", 2024-09-14T22:38:48.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6680000000000004e-06 2024-09-14T22:38:48.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824814+0000", 2024-09-14T22:38:48.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2102e-05 2024-09-14T22:38:48.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824823+0000", 2024-09-14T22:38:48.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5550000000000008e-06 2024-09-14T22:38:48.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879648+0000", 2024-09-14T22:38:48.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054825167000000001 2024-09-14T22:38:48.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879747+0000", 2024-09-14T22:38:48.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8968999999999994e-05 2024-09-14T22:38:48.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879830+0000", 2024-09-14T22:38:48.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2633999999999995e-05 2024-09-14T22:38:48.216 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.216 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.216 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.216 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112037 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6660096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:48.217 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:38:48.217 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.335352462000003, 2024-09-14T22:38:48.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055122926000000003, 2024-09-14T22:38:48.217 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.217 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.217 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.217 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.217 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.217 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.217 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112037 2024-09-14T22:38:48.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.217 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:38:48.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824836+0000", 2024-09-14T22:38:48.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824837+0000", 2024-09-14T22:38:48.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5499999999999998e-07 2024-09-14T22:38:48.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824840+0000", 2024-09-14T22:38:48.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0429999999999999e-06 2024-09-14T22:38:48.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824851+0000", 2024-09-14T22:38:48.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1046999999999999e-05 2024-09-14T22:38:48.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824859+0000", 2024-09-14T22:38:48.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1200000000000002e-06 2024-09-14T22:38:48.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879867+0000", 2024-09-14T22:38:48.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055007697000000001 2024-09-14T22:38:48.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879910+0000", 2024-09-14T22:38:48.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3402000000000002e-05 2024-09-14T22:38:48.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.879959+0000", 2024-09-14T22:38:48.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9061999999999999e-05 2024-09-14T22:38:48.221 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.221 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.221 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.221 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112038 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6709248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:48.221 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:38:48.221 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.335317766000003, 2024-09-14T22:38:48.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055312828000000001, 2024-09-14T22:38:48.221 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.222 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.222 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.222 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.222 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.222 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.222 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112038 2024-09-14T22:38:48.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.222 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:38:48.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824871+0000", 2024-09-14T22:38:48.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824872+0000", 2024-09-14T22:38:48.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T22:38:48.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824875+0000", 2024-09-14T22:38:48.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7519999999999998e-06 2024-09-14T22:38:48.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824917+0000", 2024-09-14T22:38:48.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1047999999999997e-05 2024-09-14T22:38:48.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824926+0000", 2024-09-14T22:38:48.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3749999999999992e-06 2024-09-14T22:38:48.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880006+0000", 2024-09-14T22:38:48.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055080525999999998 2024-09-14T22:38:48.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880102+0000", 2024-09-14T22:38:48.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5995999999999994e-05 2024-09-14T22:38:48.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880184+0000", 2024-09-14T22:38:48.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1478999999999997e-05 2024-09-14T22:38:48.225 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.225 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.226 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.226 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112039 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6758400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:48.226 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:38:48.226 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.335249095999998, 2024-09-14T22:38:48.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055449709999999999, 2024-09-14T22:38:48.226 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.226 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.226 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.226 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.226 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.226 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.226 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112039 2024-09-14T22:38:48.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.227 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:38:48.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824940+0000", 2024-09-14T22:38:48.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824941+0000", 2024-09-14T22:38:48.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3699999999999999e-07 2024-09-14T22:38:48.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824946+0000", 2024-09-14T22:38:48.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0209999999999999e-06 2024-09-14T22:38:48.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824972+0000", 2024-09-14T22:38:48.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6750999999999998e-05 2024-09-14T22:38:48.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824982+0000", 2024-09-14T22:38:48.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5019999999999998e-06 2024-09-14T22:38:48.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880227+0000", 2024-09-14T22:38:48.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055245257999999998 2024-09-14T22:38:48.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880305+0000", 2024-09-14T22:38:48.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7424999999999998e-05 2024-09-14T22:38:48.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880389+0000", 2024-09-14T22:38:48.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4815999999999996e-05 2024-09-14T22:38:48.230 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.230 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.230 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.230 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112040 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6807552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:48.230 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:38:48.230 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.335192379999995, 2024-09-14T22:38:48.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055534615000000002, 2024-09-14T22:38:48.231 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.231 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.231 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.231 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.231 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.231 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.231 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112040 2024-09-14T22:38:48.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.231 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:38:48.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824996+0000", 2024-09-14T22:38:48.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.824997+0000", 2024-09-14T22:38:48.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7199999999999998e-07 2024-09-14T22:38:48.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825042+0000", 2024-09-14T22:38:48.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5065999999999998e-05 2024-09-14T22:38:48.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825060+0000", 2024-09-14T22:38:48.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7520000000000002e-05 2024-09-14T22:38:48.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825070+0000", 2024-09-14T22:38:48.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8230000000000006e-06 2024-09-14T22:38:48.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880427+0000", 2024-09-14T22:38:48.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055357244 2024-09-14T22:38:48.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880483+0000", 2024-09-14T22:38:48.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5807000000000003e-05 2024-09-14T22:38:48.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880531+0000", 2024-09-14T22:38:48.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8383000000000001e-05 2024-09-14T22:38:48.235 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.235 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.235 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.235 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112041 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6856704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:48.235 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:38:48.235 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.335103205999999, 2024-09-14T22:38:48.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055573887000000002, 2024-09-14T22:38:48.235 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.235 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.235 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.235 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.235 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.236 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.236 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112041 2024-09-14T22:38:48.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.236 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:38:48.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825086+0000", 2024-09-14T22:38:48.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825087+0000", 2024-09-14T22:38:48.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2399999999999996e-07 2024-09-14T22:38:48.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825091+0000", 2024-09-14T22:38:48.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9729999999999999e-06 2024-09-14T22:38:48.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825101+0000", 2024-09-14T22:38:48.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.093e-05 2024-09-14T22:38:48.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825110+0000", 2024-09-14T22:38:48.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1450000000000008e-06 2024-09-14T22:38:48.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880559+0000", 2024-09-14T22:38:48.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055449684999999999 2024-09-14T22:38:48.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880601+0000", 2024-09-14T22:38:48.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1248000000000002e-05 2024-09-14T22:38:48.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880659+0000", 2024-09-14T22:38:48.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8981999999999999e-05 2024-09-14T22:38:48.239 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.239 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.239 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.239 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112042 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6905856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:48.240 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:38:48.240 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.335068472000003, 2024-09-14T22:38:48.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055781415000000001, 2024-09-14T22:38:48.240 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.240 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.240 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.240 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.240 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.240 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.240 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112042 2024-09-14T22:38:48.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.240 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:38:48.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825120+0000", 2024-09-14T22:38:48.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825478+0000", 2024-09-14T22:38:48.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035775 2024-09-14T22:38:48.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825494+0000", 2024-09-14T22:38:48.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5994999999999998e-05 2024-09-14T22:38:48.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825520+0000", 2024-09-14T22:38:48.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5738999999999999e-05 2024-09-14T22:38:48.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825547+0000", 2024-09-14T22:38:48.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7667000000000001e-05 2024-09-14T22:38:48.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880725+0000", 2024-09-14T22:38:48.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055177962999999997 2024-09-14T22:38:48.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880816+0000", 2024-09-14T22:38:48.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0315000000000005e-05 2024-09-14T22:38:48.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880902+0000", 2024-09-14T22:38:48.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5986e-05 2024-09-14T22:38:48.244 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.244 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.244 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.244 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112043 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:48.244 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:38:48.244 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.334620467999997, 2024-09-14T22:38:48.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055537912000000002, 2024-09-14T22:38:48.244 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.244 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.245 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.245 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.245 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.245 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.245 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112043 2024-09-14T22:38:48.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.245 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:38:48.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825568+0000", 2024-09-14T22:38:48.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825911+0000", 2024-09-14T22:38:48.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00034306899999999999 2024-09-14T22:38:48.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825918+0000", 2024-09-14T22:38:48.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.002e-06 2024-09-14T22:38:48.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825931+0000", 2024-09-14T22:38:48.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2401e-05 2024-09-14T22:38:48.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825942+0000", 2024-09-14T22:38:48.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0903e-05 2024-09-14T22:38:48.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.880951+0000", 2024-09-14T22:38:48.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055009379999999997 2024-09-14T22:38:48.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.248 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.248 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881027+0000", 2024-09-14T22:38:48.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5944000000000003e-05 2024-09-14T22:38:48.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.248 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.248 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881106+0000", 2024-09-14T22:38:48.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9213000000000002e-05 2024-09-14T22:38:48.248 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.248 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.248 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.249 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112044 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 7004160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:48.249 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:38:48.249 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.334226799000007, 2024-09-14T22:38:48.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055265416999999997, 2024-09-14T22:38:48.249 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.249 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.249 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.249 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.249 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.249 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.249 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112044 2024-09-14T22:38:48.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.249 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:38:48.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.825962+0000", 2024-09-14T22:38:48.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826239+0000", 2024-09-14T22:38:48.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027676000000000003 2024-09-14T22:38:48.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826247+0000", 2024-09-14T22:38:48.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7440000000000007e-06 2024-09-14T22:38:48.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826275+0000", 2024-09-14T22:38:48.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8053000000000001e-05 2024-09-14T22:38:48.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826286+0000", 2024-09-14T22:38:48.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1780000000000001e-05 2024-09-14T22:38:48.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881137+0000", 2024-09-14T22:38:48.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054850500000000003 2024-09-14T22:38:48.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881181+0000", 2024-09-14T22:38:48.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4153000000000002e-05 2024-09-14T22:38:48.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881227+0000", 2024-09-14T22:38:48.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6427000000000001e-05 2024-09-14T22:38:48.253 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.253 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.253 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.253 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:112045 3.3 3:c7a12632:::rbd_data.113cf633433b.0000000000000003:1a [read 7053312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-14T22:38:48.253 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:38:48.253 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.333864566000003, 2024-09-14T22:38:48.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055045050999999998, 2024-09-14T22:38:48.254 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.254 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.254 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.254 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.254 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.254 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.254 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 112045 2024-09-14T22:38:48.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.254 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.254 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.254 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:38:48.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.826324+0000", 2024-09-14T22:38:48.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830250+0000", 2024-09-14T22:38:48.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0039258670000000004 2024-09-14T22:38:48.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830260+0000", 2024-09-14T22:38:48.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0088000000000001e-05 2024-09-14T22:38:48.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830281+0000", 2024-09-14T22:38:48.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0720000000000002e-05 2024-09-14T22:38:48.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.830292+0000", 2024-09-14T22:38:48.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1455e-05 2024-09-14T22:38:48.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881255+0000", 2024-09-14T22:38:48.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.050962247000000002 2024-09-14T22:38:48.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881296+0000", 2024-09-14T22:38:48.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1557999999999998e-05 2024-09-14T22:38:48.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:27.881369+0000", 2024-09-14T22:38:48.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3115999999999996e-05 2024-09-14T22:38:48.257 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.257 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.257 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.257 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.257 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.258 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:48.259 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:38:48.259 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:48.254+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:48.360 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:48.354+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:48.432 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:48.432 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:48.432 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:38:48.432 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:48.448 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:38:48.460 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:48.457+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:48.561 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:48.554+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:48.613 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:48.613 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:48.613 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:38:48.613 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:38:48.613 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:48.627 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:38:48.671 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:48.666+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:48.763 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:48.761+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:48.797 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:48.797 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:38:48.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:38:48.797 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:38:48.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.797 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97197 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 796672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:48.797 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:38:48.797 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.722162752000003, 2024-09-14T22:38:48.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12434473, 2024-09-14T22:38:48.798 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.798 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.798 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.798 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.798 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.798 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.798 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97197 2024-09-14T22:38:48.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.798 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:38:48.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:38:48.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070524+0000", 2024-09-14T22:38:48.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0299999999999998e-07 2024-09-14T22:38:48.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070527+0000", 2024-09-14T22:38:48.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9569999999999998e-06 2024-09-14T22:38:48.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070535+0000", 2024-09-14T22:38:48.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7630000000000006e-06 2024-09-14T22:38:48.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070543+0000", 2024-09-14T22:38:48.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4869999999999996e-06 2024-09-14T22:38:48.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194525+0000", 2024-09-14T22:38:48.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123982096 2024-09-14T22:38:48.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194699+0000", 2024-09-14T22:38:48.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000174105 2024-09-14T22:38:48.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194868+0000", 2024-09-14T22:38:48.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016861900000000001 2024-09-14T22:38:48.802 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.802 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.802 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.802 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97198 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 805888~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:48.802 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:38:48.802 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.722134376, 2024-09-14T22:38:48.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.1248088, 2024-09-14T22:38:48.802 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.803 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.803 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.803 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.803 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.803 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.803 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97198 2024-09-14T22:38:48.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.803 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:38:48.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:38:48.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070552+0000", 2024-09-14T22:38:48.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T22:38:48.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070555+0000", 2024-09-14T22:38:48.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.853e-06 2024-09-14T22:38:48.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070565+0000", 2024-09-14T22:38:48.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.048e-05 2024-09-14T22:38:48.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070573+0000", 2024-09-14T22:38:48.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7200000000000006e-06 2024-09-14T22:38:48.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194954+0000", 2024-09-14T22:38:48.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12438089400000001 2024-09-14T22:38:48.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195145+0000", 2024-09-14T22:38:48.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019091299999999999 2024-09-14T22:38:48.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195360+0000", 2024-09-14T22:38:48.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021527300000000001 2024-09-14T22:38:48.806 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.806 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.806 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.807 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97199 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 815104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:48.807 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:38:48.807 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.722103778999994, 2024-09-14T22:38:48.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125251901, 2024-09-14T22:38:48.807 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.807 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.807 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.807 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.807 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.807 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.807 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97199 2024-09-14T22:38:48.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.807 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:38:48.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:38:48.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070583+0000", 2024-09-14T22:38:48.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6300000000000005e-07 2024-09-14T22:38:48.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070586+0000", 2024-09-14T22:38:48.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8270000000000002e-06 2024-09-14T22:38:48.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070597+0000", 2024-09-14T22:38:48.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1960000000000001e-05 2024-09-14T22:38:48.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070605+0000", 2024-09-14T22:38:48.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6459999999999993e-06 2024-09-14T22:38:48.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195432+0000", 2024-09-14T22:38:48.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124826926 2024-09-14T22:38:48.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195626+0000", 2024-09-14T22:38:48.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019357099999999999 2024-09-14T22:38:48.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195834+0000", 2024-09-14T22:38:48.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020830800000000001 2024-09-14T22:38:48.810 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.811 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.811 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.811 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97200 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 824320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:48.811 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:38:48.811 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.722071733000007, 2024-09-14T22:38:48.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125693477, 2024-09-14T22:38:48.811 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.811 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.811 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.811 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.811 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.811 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.812 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97200 2024-09-14T22:38:48.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.812 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:38:48.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:38:48.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070615+0000", 2024-09-14T22:38:48.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T22:38:48.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070618+0000", 2024-09-14T22:38:48.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.943e-06 2024-09-14T22:38:48.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070627+0000", 2024-09-14T22:38:48.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1099999999999992e-06 2024-09-14T22:38:48.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070634+0000", 2024-09-14T22:38:48.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:38:48.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195908+0000", 2024-09-14T22:38:48.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125273893 2024-09-14T22:38:48.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196097+0000", 2024-09-14T22:38:48.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018953899999999999 2024-09-14T22:38:48.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196308+0000", 2024-09-14T22:38:48.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021027799999999999 2024-09-14T22:38:48.815 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.815 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.815 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.815 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97201 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 833536~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:48.815 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:38:48.815 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.722041546, 2024-09-14T22:38:48.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126133826, 2024-09-14T22:38:48.815 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.815 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.816 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.816 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.816 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.816 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.816 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97201 2024-09-14T22:38:48.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.816 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:38:48.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:38:48.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074109+0000", 2024-09-14T22:38:48.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003464834 2024-09-14T22:38:48.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074134+0000", 2024-09-14T22:38:48.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5170000000000001e-05 2024-09-14T22:38:48.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074166+0000", 2024-09-14T22:38:48.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1347000000000003e-05 2024-09-14T22:38:48.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074179+0000", 2024-09-14T22:38:48.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.364e-05 2024-09-14T22:38:48.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.819 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.819 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196377+0000", 2024-09-14T22:38:48.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12219793900000001 2024-09-14T22:38:48.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.819 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.819 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196568+0000", 2024-09-14T22:38:48.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019097999999999999 2024-09-14T22:38:48.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.819 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.819 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196778+0000", 2024-09-14T22:38:48.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020991599999999999 2024-09-14T22:38:48.819 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.820 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.820 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.820 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97205 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 870400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:48.820 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:38:48.820 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.718371982999997, 2024-09-14T22:38:48.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12436148299999999, 2024-09-14T22:38:48.820 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.820 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.820 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.820 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.820 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.820 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.821 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97205 2024-09-14T22:38:48.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.821 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:38:48.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:38:48.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074315+0000", 2024-09-14T22:38:48.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9299999999999997e-07 2024-09-14T22:38:48.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074318+0000", 2024-09-14T22:38:48.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.19e-06 2024-09-14T22:38:48.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074327+0000", 2024-09-14T22:38:48.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8529999999999998e-06 2024-09-14T22:38:48.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074334+0000", 2024-09-14T22:38:48.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3869999999999997e-06 2024-09-14T22:38:48.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198365+0000", 2024-09-14T22:38:48.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124031222 2024-09-14T22:38:48.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198513+0000", 2024-09-14T22:38:48.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014772299999999999 2024-09-14T22:38:48.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.824 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198675+0000", 2024-09-14T22:38:48.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016241500000000001 2024-09-14T22:38:48.824 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.824 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.824 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.824 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97206 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 879616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:48.824 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:38:48.824 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.718340526999995, 2024-09-14T22:38:48.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124706284, 2024-09-14T22:38:48.824 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.824 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.824 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.825 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.825 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.825 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.825 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97206 2024-09-14T22:38:48.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.825 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.825 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.825 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:38:48.825 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.825 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.826 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:38:48.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.826 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.826 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074346+0000", 2024-09-14T22:38:48.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8999999999999996e-07 2024-09-14T22:38:48.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.826 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.826 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074349+0000", 2024-09-14T22:38:48.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.96e-06 2024-09-14T22:38:48.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.826 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074358+0000", 2024-09-14T22:38:48.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9169999999999995e-06 2024-09-14T22:38:48.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074365+0000", 2024-09-14T22:38:48.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3699999999999997e-06 2024-09-14T22:38:48.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198761+0000", 2024-09-14T22:38:48.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124395743 2024-09-14T22:38:48.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198899+0000", 2024-09-14T22:38:48.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013763499999999999 2024-09-14T22:38:48.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199052+0000", 2024-09-14T22:38:48.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015296900000000001 2024-09-14T22:38:48.828 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.828 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.828 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.828 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97207 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 888832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:48.828 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:38:48.829 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.718311314000005, 2024-09-14T22:38:48.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124988433, 2024-09-14T22:38:48.829 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.829 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.829 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.829 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.829 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.829 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.829 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97207 2024-09-14T22:38:48.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.829 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:38:48.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:38:48.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074375+0000", 2024-09-14T22:38:48.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:38:48.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074378+0000", 2024-09-14T22:38:48.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1829999999999998e-06 2024-09-14T22:38:48.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074389+0000", 2024-09-14T22:38:48.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0737000000000001e-05 2024-09-14T22:38:48.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074396+0000", 2024-09-14T22:38:48.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2559999999999996e-06 2024-09-14T22:38:48.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.832 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199139+0000", 2024-09-14T22:38:48.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12474253 2024-09-14T22:38:48.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.832 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.832 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199240+0000", 2024-09-14T22:38:48.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101221 2024-09-14T22:38:48.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.832 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.832 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199363+0000", 2024-09-14T22:38:48.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000122822 2024-09-14T22:38:48.832 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.832 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.833 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.833 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97208 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 898048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:48.833 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:38:48.833 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.718280010000001, 2024-09-14T22:38:48.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12524854899999999, 2024-09-14T22:38:48.833 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.833 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.833 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.833 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.833 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.833 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.834 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97208 2024-09-14T22:38:48.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.834 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.834 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:38:48.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.834 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:38:48.835 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.835 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.835 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:38:48.835 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:38:48.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.835 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.835 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074410+0000", 2024-09-14T22:38:48.835 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.18e-06 2024-09-14T22:38:48.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.835 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.835 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074435+0000", 2024-09-14T22:38:48.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5273e-05 2024-09-14T22:38:48.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074443+0000", 2024-09-14T22:38:48.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3000000000000002e-06 2024-09-14T22:38:48.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199433+0000", 2024-09-14T22:38:48.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124989502 2024-09-14T22:38:48.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.837 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199535+0000", 2024-09-14T22:38:48.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102557 2024-09-14T22:38:48.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.837 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.837 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199654+0000", 2024-09-14T22:38:48.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119061 2024-09-14T22:38:48.837 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.837 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.837 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.837 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97209 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 907264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:48.837 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:38:48.837 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.718232205000007, 2024-09-14T22:38:48.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125487354, 2024-09-14T22:38:48.838 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.838 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.838 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.838 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.838 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.838 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.838 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97209 2024-09-14T22:38:48.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.838 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:38:48.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:38:48.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:38:48.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9400000000000005e-07 2024-09-14T22:38:48.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074458+0000", 2024-09-14T22:38:48.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3299999999999999e-06 2024-09-14T22:38:48.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074467+0000", 2024-09-14T22:38:48.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.346e-06 2024-09-14T22:38:48.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074474+0000", 2024-09-14T22:38:48.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.374e-06 2024-09-14T22:38:48.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199720+0000", 2024-09-14T22:38:48.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12524561200000001 2024-09-14T22:38:48.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199820+0000", 2024-09-14T22:38:48.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010049599999999999 2024-09-14T22:38:48.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199941+0000", 2024-09-14T22:38:48.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012050200000000001 2024-09-14T22:38:48.841 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.841 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.841 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.842 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97210 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:48.842 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:38:48.842 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.718202211000005, 2024-09-14T22:38:48.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125744942, 2024-09-14T22:38:48.842 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.842 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.842 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.842 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.842 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.842 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.842 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97210 2024-09-14T22:38:48.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.842 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:38:48.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:38:48.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:38:48.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6599999999999996e-07 2024-09-14T22:38:48.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.844 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.844 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074487+0000", 2024-09-14T22:38:48.844 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8140000000000001e-06 2024-09-14T22:38:48.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.844 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.844 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074496+0000", 2024-09-14T22:38:48.844 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9330000000000007e-06 2024-09-14T22:38:48.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.844 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.844 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074503+0000", 2024-09-14T22:38:48.844 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4100000000000002e-06 2024-09-14T22:38:48.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200009+0000", 2024-09-14T22:38:48.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12550539899999999 2024-09-14T22:38:48.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200108+0000", 2024-09-14T22:38:48.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9182e-05 2024-09-14T22:38:48.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200229+0000", 2024-09-14T22:38:48.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120538 2024-09-14T22:38:48.846 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.846 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.846 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.846 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97211 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 925696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:48.846 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:38:48.846 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.718171251000001, 2024-09-14T22:38:48.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12599848299999999, 2024-09-14T22:38:48.846 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.846 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.846 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.847 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.847 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.847 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.847 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97211 2024-09-14T22:38:48.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.847 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:38:48.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:38:48.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074516+0000", 2024-09-14T22:38:48.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.42e-06 2024-09-14T22:38:48.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074521+0000", 2024-09-14T22:38:48.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7960000000000003e-06 2024-09-14T22:38:48.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074538+0000", 2024-09-14T22:38:48.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6900000000000001e-05 2024-09-14T22:38:48.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074555+0000", 2024-09-14T22:38:48.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.696e-05 2024-09-14T22:38:48.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200293+0000", 2024-09-14T22:38:48.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12573864100000001 2024-09-14T22:38:48.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200393+0000", 2024-09-14T22:38:48.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9264000000000004e-05 2024-09-14T22:38:48.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.850 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200513+0000", 2024-09-14T22:38:48.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012050200000000001 2024-09-14T22:38:48.850 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.850 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.850 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.850 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97212 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 934912~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:48.850 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:38:48.850 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.718096629000001, 2024-09-14T22:38:48.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12621127300000001, 2024-09-14T22:38:48.851 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.851 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.851 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.851 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.851 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.851 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.851 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97212 2024-09-14T22:38:48.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.851 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:38:48.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.852 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:38:48.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.852 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074590+0000", 2024-09-14T22:38:48.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1969999999999999e-06 2024-09-14T22:38:48.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074596+0000", 2024-09-14T22:38:48.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7629999999999997e-06 2024-09-14T22:38:48.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074614+0000", 2024-09-14T22:38:48.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7643e-05 2024-09-14T22:38:48.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074635+0000", 2024-09-14T22:38:48.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.075e-05 2024-09-14T22:38:48.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200580+0000", 2024-09-14T22:38:48.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125945482 2024-09-14T22:38:48.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200683+0000", 2024-09-14T22:38:48.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010267099999999999 2024-09-14T22:38:48.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200800+0000", 2024-09-14T22:38:48.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117767 2024-09-14T22:38:48.855 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.855 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.855 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.855 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97213 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 944128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:48.855 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:38:48.855 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.718031279000002, 2024-09-14T22:38:48.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126430709, 2024-09-14T22:38:48.855 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.855 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.855 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.855 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.856 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.856 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.856 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97213 2024-09-14T22:38:48.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.856 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.856 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.856 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:38:48.856 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.856 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.856 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:38:48.856 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.857 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078210+0000", 2024-09-14T22:38:48.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0035556839999999999 2024-09-14T22:38:48.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.857 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078231+0000", 2024-09-14T22:38:48.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0716e-05 2024-09-14T22:38:48.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.857 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078260+0000", 2024-09-14T22:38:48.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9473000000000001e-05 2024-09-14T22:38:48.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078279+0000", 2024-09-14T22:38:48.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8600000000000001e-05 2024-09-14T22:38:48.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200864+0000", 2024-09-14T22:38:48.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.122585295 2024-09-14T22:38:48.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200965+0000", 2024-09-14T22:38:48.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101004 2024-09-14T22:38:48.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.859 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.201085+0000", 2024-09-14T22:38:48.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119937 2024-09-14T22:38:48.859 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.859 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.859 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.859 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97219 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 999424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:48.859 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:38:48.859 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.714049001999996, 2024-09-14T22:38:48.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124138558, 2024-09-14T22:38:48.860 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.860 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.860 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.860 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.860 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.860 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.860 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97219 2024-09-14T22:38:48.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.860 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:38:48.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:38:48.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078638+0000", 2024-09-14T22:38:48.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.173e-06 2024-09-14T22:38:48.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078644+0000", 2024-09-14T22:38:48.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2570000000000004e-06 2024-09-14T22:38:48.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078672+0000", 2024-09-14T22:38:48.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7322999999999999e-05 2024-09-14T22:38:48.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078687+0000", 2024-09-14T22:38:48.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5639999999999999e-05 2024-09-14T22:38:48.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202567+0000", 2024-09-14T22:38:48.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123880134 2024-09-14T22:38:48.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.864 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202670+0000", 2024-09-14T22:38:48.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010273899999999999 2024-09-14T22:38:48.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.864 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202775+0000", 2024-09-14T22:38:48.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010529199999999999 2024-09-14T22:38:48.864 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.864 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.865 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.865 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.865 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97220 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1008640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:48.865 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:38:48.865 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.713979769000005, 2024-09-14T22:38:48.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12435937, 2024-09-14T22:38:48.865 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.865 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.865 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.865 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.865 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.865 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.865 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97220 2024-09-14T22:38:48.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.866 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:38:48.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:38:48.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078707+0000", 2024-09-14T22:38:48.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.266e-06 2024-09-14T22:38:48.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078713+0000", 2024-09-14T22:38:48.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7270000000000004e-06 2024-09-14T22:38:48.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078737+0000", 2024-09-14T22:38:48.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3640000000000001e-05 2024-09-14T22:38:48.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078751+0000", 2024-09-14T22:38:48.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.45e-05 2024-09-14T22:38:48.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202841+0000", 2024-09-14T22:38:48.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124089938 2024-09-14T22:38:48.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202944+0000", 2024-09-14T22:38:48.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010334 2024-09-14T22:38:48.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.869 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.869 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203065+0000", 2024-09-14T22:38:48.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012095900000000001 2024-09-14T22:38:48.869 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.869 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.869 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.869 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97221 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1017856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:48.869 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:38:48.869 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.713914005999996, 2024-09-14T22:38:48.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124574005, 2024-09-14T22:38:48.869 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.869 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.870 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.870 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.870 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.870 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.870 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97221 2024-09-14T22:38:48.870 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.870 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.870 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.870 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:38:48.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.870 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:38:48.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078773+0000", 2024-09-14T22:38:48.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.13e-06 2024-09-14T22:38:48.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078779+0000", 2024-09-14T22:38:48.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0260000000000004e-06 2024-09-14T22:38:48.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078797+0000", 2024-09-14T22:38:48.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.806e-05 2024-09-14T22:38:48.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078810+0000", 2024-09-14T22:38:48.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.308e-05 2024-09-14T22:38:48.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203131+0000", 2024-09-14T22:38:48.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12432077499999999 2024-09-14T22:38:48.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203227+0000", 2024-09-14T22:38:48.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6575000000000003e-05 2024-09-14T22:38:48.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203346+0000", 2024-09-14T22:38:48.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118359 2024-09-14T22:38:48.874 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.874 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.874 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.874 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97222 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:48.874 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:38:48.874 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.713857637000004, 2024-09-14T22:38:48.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12480110799999999, 2024-09-14T22:38:48.874 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.874 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.874 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.874 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.874 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.875 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.875 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97222 2024-09-14T22:38:48.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.875 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.875 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.875 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:38:48.875 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.875 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.875 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:38:48.875 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078829+0000", 2024-09-14T22:38:48.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.057e-06 2024-09-14T22:38:48.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078836+0000", 2024-09-14T22:38:48.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6159999999999998e-06 2024-09-14T22:38:48.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078856+0000", 2024-09-14T22:38:48.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0100000000000001e-05 2024-09-14T22:38:48.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078869+0000", 2024-09-14T22:38:48.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.272e-05 2024-09-14T22:38:48.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203412+0000", 2024-09-14T22:38:48.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12454283300000001 2024-09-14T22:38:48.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203511+0000", 2024-09-14T22:38:48.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9893000000000001e-05 2024-09-14T22:38:48.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203629+0000", 2024-09-14T22:38:48.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117889 2024-09-14T22:38:48.878 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.878 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.878 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.878 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97223 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1036288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:48.878 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:38:48.879 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.713485649999996, 2024-09-14T22:38:48.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124706279, 2024-09-14T22:38:48.879 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.879 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.879 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.879 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.879 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.879 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.879 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97223 2024-09-14T22:38:48.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.879 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.879 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.879 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:38:48.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:38:48.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079202+0000", 2024-09-14T22:38:48.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8330000000000001e-06 2024-09-14T22:38:48.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079212+0000", 2024-09-14T22:38:48.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0207000000000001e-05 2024-09-14T22:38:48.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.881 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079238+0000", 2024-09-14T22:38:48.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5626000000000002e-05 2024-09-14T22:38:48.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.881 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079256+0000", 2024-09-14T22:38:48.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8386999999999999e-05 2024-09-14T22:38:48.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203691+0000", 2024-09-14T22:38:48.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124434538 2024-09-14T22:38:48.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203786+0000", 2024-09-14T22:38:48.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5344000000000001e-05 2024-09-14T22:38:48.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203906+0000", 2024-09-14T22:38:48.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120344 2024-09-14T22:38:48.882 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.882 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.882 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.883 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97224 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1045504~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:48.883 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:38:48.883 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.713401829999995, 2024-09-14T22:38:48.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12513321199999999, 2024-09-14T22:38:48.883 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:48.883 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:48.883 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:48.883 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:48.883 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:48.883 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:48.883 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97224 2024-09-14T22:38:48.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.884 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:48.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.884 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:48.884 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:38:48.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.884 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:48.884 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:38:48.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:48.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.884 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:48.884 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082581+0000", 2024-09-14T22:38:48.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0032971530000000001 2024-09-14T22:38:48.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:48.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082592+0000", 2024-09-14T22:38:48.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1345999999999999e-05 2024-09-14T22:38:48.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:48.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082618+0000", 2024-09-14T22:38:48.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5873000000000001e-05 2024-09-14T22:38:48.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:48.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082638+0000", 2024-09-14T22:38:48.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9565999999999999e-05 2024-09-14T22:38:48.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:48.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204004+0000", 2024-09-14T22:38:48.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.121366038 2024-09-14T22:38:48.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:48.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204188+0000", 2024-09-14T22:38:48.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018427 2024-09-14T22:38:48.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:48.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:48.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:48.887 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204417+0000", 2024-09-14T22:38:48.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022896599999999999 2024-09-14T22:38:48.887 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.887 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.887 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.887 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:48.887 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:48.887 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:48.889 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:38:48.889 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:48.882+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:48.990 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:48.989+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:49.069 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:49.070 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:49.070 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:38:49.070 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:49.085 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:38:49.131 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:49.126+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:49.191 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:49.189+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:49.215 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:49.215 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:49.215 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:38:49.215 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:38:49.215 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:49.229 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:38:49.315 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:49.310+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:49.393 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:49.386+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:49.396 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:49.397 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:38:49.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:38:49.397 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:38:49.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.397 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:72252 3.2 3:442c6d72:::rbd_data.113cb4c83968.0000000000000015:head [zero 4456448~393216] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-14T22:38:49.397 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:38:49.397 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 118.692297423, 2024-09-14T22:38:49.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0176591500000001, 2024-09-14T22:38:49.397 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:49.397 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:49.397 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T22:38:49.397 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:49.397 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:49.398 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:49.398 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 72252 2024-09-14T22:38:49.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.398 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:49.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:49.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:38:49.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:49.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:38:49.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:49.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700101+0000", 2024-09-14T22:38:49.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4570000000000001e-06 2024-09-14T22:38:49.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:49.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700114+0000", 2024-09-14T22:38:49.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2799999999999999e-05 2024-09-14T22:38:49.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:49.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700146+0000", 2024-09-14T22:38:49.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2116000000000002e-05 2024-09-14T22:38:49.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:49.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700170+0000", 2024-09-14T22:38:49.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3770000000000001e-05 2024-09-14T22:38:49.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:49.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700285+0000", 2024-09-14T22:38:49.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000115228 2024-09-14T22:38:49.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op must wait for map", 2024-09-14T22:38:49.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700297+0000", 2024-09-14T22:38:49.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1681999999999999e-05 2024-09-14T22:38:49.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:49.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.703312+0000", 2024-09-14T22:38:49.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003015046 2024-09-14T22:38:49.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:38:49.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.703327+0000", 2024-09-14T22:38:49.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5481999999999998e-05 2024-09-14T22:38:49.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:49.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:51.717566+0000", 2024-09-14T22:38:49.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0142383509999999 2024-09-14T22:38:49.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:49.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:51.717757+0000", 2024-09-14T22:38:49.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000191218 2024-09-14T22:38:49.402 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.402 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:49.402 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.403 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:74156 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T22:38:49.403 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:38:49.403 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.877828713, 2024-09-14T22:38:49.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062073521, 2024-09-14T22:38:49.403 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:49.403 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:49.403 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:49.403 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:49.403 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:49.403 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:49.403 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74156 2024-09-14T22:38:49.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.404 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:49.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:49.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:38:49.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:49.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:38:49.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:49.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514569+0000", 2024-09-14T22:38:49.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0229999999999999e-06 2024-09-14T22:38:49.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:49.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514596+0000", 2024-09-14T22:38:49.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6903e-05 2024-09-14T22:38:49.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:49.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514623+0000", 2024-09-14T22:38:49.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7206e-05 2024-09-14T22:38:49.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:49.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514637+0000", 2024-09-14T22:38:49.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:38:49.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:49.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514781+0000", 2024-09-14T22:38:49.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014394899999999999 2024-09-14T22:38:49.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:49.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.515005+0000", 2024-09-14T22:38:49.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022458499999999999 2024-09-14T22:38:49.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:38:49.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.527785+0000", 2024-09-14T22:38:49.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012780112 2024-09-14T22:38:49.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:38:49.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576209+0000", 2024-09-14T22:38:49.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.048423992999999999 2024-09-14T22:38:49.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:49.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576346+0000", 2024-09-14T22:38:49.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000136909 2024-09-14T22:38:49.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:38:49.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576517+0000", 2024-09-14T22:38:49.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000170848 2024-09-14T22:38:49.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:49.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576640+0000", 2024-09-14T22:38:49.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000122953 2024-09-14T22:38:49.409 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.409 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:49.409 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.409 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82513 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 557056~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:49.409 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:38:49.409 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 109.973196894, 2024-09-14T22:38:49.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059345486000000003, 2024-09-14T22:38:49.409 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:49.409 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:49.409 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:49.410 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:49.410 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:49.410 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:49.410 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82513 2024-09-14T22:38:49.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.410 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:49.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:49.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:38:49.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:49.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:38:49.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:49.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419200+0000", 2024-09-14T22:38:49.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.184e-06 2024-09-14T22:38:49.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:49.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419207+0000", 2024-09-14T22:38:49.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2690000000000002e-06 2024-09-14T22:38:49.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:49.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419224+0000", 2024-09-14T22:38:49.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6623e-05 2024-09-14T22:38:49.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:49.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419236+0000", 2024-09-14T22:38:49.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2367e-05 2024-09-14T22:38:49.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:49.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478294+0000", 2024-09-14T22:38:49.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059057927000000003 2024-09-14T22:38:49.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:49.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478402+0000", 2024-09-14T22:38:49.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107921 2024-09-14T22:38:49.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:49.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478544+0000", 2024-09-14T22:38:49.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014219499999999999 2024-09-14T22:38:49.413 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.413 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:49.413 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.414 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82514 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 704512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:49.414 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:38:49.414 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 109.973137791, 2024-09-14T22:38:49.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059580793, 2024-09-14T22:38:49.414 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:49.414 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:49.414 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:49.414 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:49.414 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:49.414 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:49.414 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82514 2024-09-14T22:38:49.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.415 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:49.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:49.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:38:49.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:49.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:38:49.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:49.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419259+0000", 2024-09-14T22:38:49.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.353e-06 2024-09-14T22:38:49.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:49.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419276+0000", 2024-09-14T22:38:49.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7567e-05 2024-09-14T22:38:49.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:49.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419294+0000", 2024-09-14T22:38:49.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.713e-05 2024-09-14T22:38:49.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:49.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419306+0000", 2024-09-14T22:38:49.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.279e-05 2024-09-14T22:38:49.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:49.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478589+0000", 2024-09-14T22:38:49.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059282978 2024-09-14T22:38:49.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:49.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478694+0000", 2024-09-14T22:38:49.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000104561 2024-09-14T22:38:49.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:49.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478838+0000", 2024-09-14T22:38:49.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014441399999999999 2024-09-14T22:38:49.418 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.418 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:49.418 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.418 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82515 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:49.418 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:38:49.419 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 109.973072641, 2024-09-14T22:38:49.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059815123999999997, 2024-09-14T22:38:49.419 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:49.419 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:49.419 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:49.419 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:49.419 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:49.419 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:49.419 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82515 2024-09-14T22:38:49.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.419 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:49.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:49.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:38:49.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:49.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:38:49.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:49.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419324+0000", 2024-09-14T22:38:49.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2130000000000001e-06 2024-09-14T22:38:49.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:49.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419331+0000", 2024-09-14T22:38:49.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4730000000000003e-06 2024-09-14T22:38:49.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:49.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419348+0000", 2024-09-14T22:38:49.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6269999999999998e-05 2024-09-14T22:38:49.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:49.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419360+0000", 2024-09-14T22:38:49.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2243e-05 2024-09-14T22:38:49.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:49.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478884+0000", 2024-09-14T22:38:49.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059524369000000001 2024-09-14T22:38:49.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:49.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478990+0000", 2024-09-14T22:38:49.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010569 2024-09-14T22:38:49.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:49.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479138+0000", 2024-09-14T22:38:49.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000147866 2024-09-14T22:38:49.422 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.423 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:49.423 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.423 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82516 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 999424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:49.423 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:38:49.423 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 109.973020136, 2024-09-14T22:38:49.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061979595999999998, 2024-09-14T22:38:49.423 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:49.423 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:49.423 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:49.423 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:49.423 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:49.423 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:49.424 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82516 2024-09-14T22:38:49.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.424 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:49.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:49.424 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:38:49.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:49.424 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:38:49.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:49.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419376+0000", 2024-09-14T22:38:49.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.237e-06 2024-09-14T22:38:49.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:49.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419383+0000", 2024-09-14T22:38:49.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8170000000000001e-06 2024-09-14T22:38:49.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:49.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419400+0000", 2024-09-14T22:38:49.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6636000000000001e-05 2024-09-14T22:38:49.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:49.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419412+0000", 2024-09-14T22:38:49.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2357e-05 2024-09-14T22:38:49.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:49.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481080+0000", 2024-09-14T22:38:49.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061668032999999997 2024-09-14T22:38:49.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:49.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481193+0000", 2024-09-14T22:38:49.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112335 2024-09-14T22:38:49.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:49.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481355+0000", 2024-09-14T22:38:49.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016218100000000001 2024-09-14T22:38:49.427 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.427 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:49.427 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.427 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82538 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1097728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:49.428 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:38:49.428 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 109.97296806599999, 2024-09-14T22:38:49.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059971705, 2024-09-14T22:38:49.428 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:49.428 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:49.428 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:49.428 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:49.428 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:49.428 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:49.428 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82538 2024-09-14T22:38:49.428 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.428 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:49.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.428 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:49.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:38:49.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:49.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:38:49.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:49.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419429+0000", 2024-09-14T22:38:49.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.497e-06 2024-09-14T22:38:49.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:49.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419436+0000", 2024-09-14T22:38:49.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2099999999999996e-06 2024-09-14T22:38:49.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:49.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419453+0000", 2024-09-14T22:38:49.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6840000000000001e-05 2024-09-14T22:38:49.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:49.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419465+0000", 2024-09-14T22:38:49.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2283000000000001e-05 2024-09-14T22:38:49.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:49.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479182+0000", 2024-09-14T22:38:49.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059717339000000001 2024-09-14T22:38:49.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:49.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479297+0000", 2024-09-14T22:38:49.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000114259 2024-09-14T22:38:49.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:49.432 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479399+0000", 2024-09-14T22:38:49.432 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102277 2024-09-14T22:38:49.432 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.432 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:49.432 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.432 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.432 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.432 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82539 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:49.432 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:38:49.432 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 109.972915216, 2024-09-14T22:38:49.432 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060166935999999997, 2024-09-14T22:38:49.432 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:49.432 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:49.433 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:49.433 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:49.433 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:49.433 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:49.433 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82539 2024-09-14T22:38:49.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.433 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:49.433 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.433 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:49.433 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:38:49.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.433 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.433 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:49.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:38:49.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:49.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419482+0000", 2024-09-14T22:38:49.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4130000000000001e-06 2024-09-14T22:38:49.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:49.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419488+0000", 2024-09-14T22:38:49.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8460000000000002e-06 2024-09-14T22:38:49.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.435 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:49.435 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419505+0000", 2024-09-14T22:38:49.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6477000000000002e-05 2024-09-14T22:38:49.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.435 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:49.435 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419518+0000", 2024-09-14T22:38:49.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2797e-05 2024-09-14T22:38:49.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.435 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:49.435 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479438+0000", 2024-09-14T22:38:49.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059919897 2024-09-14T22:38:49.436 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.436 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.436 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:49.436 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479547+0000", 2024-09-14T22:38:49.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109807 2024-09-14T22:38:49.436 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.436 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.436 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:49.436 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479647+0000", 2024-09-14T22:38:49.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9698999999999995e-05 2024-09-14T22:38:49.436 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.436 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:49.436 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.437 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.437 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82542 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:49.437 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:38:49.437 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 109.972862773, 2024-09-14T22:38:49.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060367430999999999, 2024-09-14T22:38:49.437 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:49.437 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:49.437 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:49.437 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:49.437 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:49.437 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:49.437 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82542 2024-09-14T22:38:49.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.438 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:49.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:49.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:38:49.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:49.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:38:49.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:49.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419534+0000", 2024-09-14T22:38:49.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T22:38:49.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:49.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419542+0000", 2024-09-14T22:38:49.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8199999999999997e-06 2024-09-14T22:38:49.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:49.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419559+0000", 2024-09-14T22:38:49.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6852999999999999e-05 2024-09-14T22:38:49.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:49.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419571+0000", 2024-09-14T22:38:49.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2364000000000001e-05 2024-09-14T22:38:49.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:49.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479686+0000", 2024-09-14T22:38:49.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060115522999999997 2024-09-14T22:38:49.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:49.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479795+0000", 2024-09-14T22:38:49.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010831399999999999 2024-09-14T22:38:49.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:49.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479900+0000", 2024-09-14T22:38:49.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105284 2024-09-14T22:38:49.441 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.441 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:49.441 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.441 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82544 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:49.441 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:38:49.442 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 109.972809037, 2024-09-14T22:38:49.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060590515999999997, 2024-09-14T22:38:49.442 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:49.442 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:49.442 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:49.442 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:49.442 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:49.442 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:49.442 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82544 2024-09-14T22:38:49.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.442 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:49.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:49.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:38:49.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.443 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:49.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:38:49.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.443 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:49.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419588+0000", 2024-09-14T22:38:49.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.35e-06 2024-09-14T22:38:49.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.444 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:49.444 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419595+0000", 2024-09-14T22:38:49.444 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4030000000000001e-06 2024-09-14T22:38:49.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.444 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:49.444 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419612+0000", 2024-09-14T22:38:49.444 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6670000000000001e-05 2024-09-14T22:38:49.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.444 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:49.444 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419624+0000", 2024-09-14T22:38:49.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-05 2024-09-14T22:38:49.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:49.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479957+0000", 2024-09-14T22:38:49.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060333287999999999 2024-09-14T22:38:49.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:49.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480074+0000", 2024-09-14T22:38:49.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116443 2024-09-14T22:38:49.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:49.446 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480177+0000", 2024-09-14T22:38:49.446 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000103102 2024-09-14T22:38:49.446 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.446 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:49.446 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.446 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.446 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82546 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:49.446 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:38:49.446 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 109.97275611800001, 2024-09-14T22:38:49.446 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060795428999999998, 2024-09-14T22:38:49.446 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:49.446 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:49.447 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:49.447 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:49.447 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:49.447 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:49.447 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82546 2024-09-14T22:38:49.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.447 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:49.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.447 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:49.447 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:38:49.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.447 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:49.448 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:38:49.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.448 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:49.448 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419640+0000", 2024-09-14T22:38:49.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.254e-06 2024-09-14T22:38:49.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.448 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:49.448 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419647+0000", 2024-09-14T22:38:49.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9e-06 2024-09-14T22:38:49.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.449 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:49.449 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419664+0000", 2024-09-14T22:38:49.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6345999999999999e-05 2024-09-14T22:38:49.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.449 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:49.449 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419676+0000", 2024-09-14T22:38:49.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.224e-05 2024-09-14T22:38:49.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.449 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:49.449 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480218+0000", 2024-09-14T22:38:49.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060542133999999997 2024-09-14T22:38:49.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.450 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.450 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:49.450 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480336+0000", 2024-09-14T22:38:49.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117472 2024-09-14T22:38:49.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.450 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.450 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:49.450 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480435+0000", 2024-09-14T22:38:49.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9083e-05 2024-09-14T22:38:49.450 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.450 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:49.450 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.451 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82548 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:49.451 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:38:49.451 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 109.972704108, 2024-09-14T22:38:49.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061011636000000001, 2024-09-14T22:38:49.451 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:49.451 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:49.451 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:49.451 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:49.451 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:49.451 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:49.451 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82548 2024-09-14T22:38:49.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.452 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:49.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:49.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:38:49.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:49.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:38:49.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:49.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419693+0000", 2024-09-14T22:38:49.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.387e-06 2024-09-14T22:38:49.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:49.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419699+0000", 2024-09-14T22:38:49.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8730000000000001e-06 2024-09-14T22:38:49.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:49.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419716+0000", 2024-09-14T22:38:49.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6416e-05 2024-09-14T22:38:49.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.454 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:49.454 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419728+0000", 2024-09-14T22:38:49.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2597e-05 2024-09-14T22:38:49.454 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.454 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.454 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:49.454 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480476+0000", 2024-09-14T22:38:49.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060747989000000002 2024-09-14T22:38:49.454 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.454 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.454 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:49.454 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480583+0000", 2024-09-14T22:38:49.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010638500000000001 2024-09-14T22:38:49.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.455 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.455 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:49.455 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480703+0000", 2024-09-14T22:38:49.455 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011998900000000001 2024-09-14T22:38:49.455 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.455 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:49.455 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.455 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.455 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82550 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1982464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:49.455 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:38:49.455 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 109.972651355, 2024-09-14T22:38:49.455 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061295483999999997, 2024-09-14T22:38:49.456 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:49.456 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:49.456 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:49.456 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:49.456 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:49.456 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:49.456 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82550 2024-09-14T22:38:49.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.456 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:49.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:49.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:38:49.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:49.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:38:49.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:49.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419745+0000", 2024-09-14T22:38:49.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.406e-06 2024-09-14T22:38:49.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:49.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419753+0000", 2024-09-14T22:38:49.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8140000000000001e-06 2024-09-14T22:38:49.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:49.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419770+0000", 2024-09-14T22:38:49.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6719999999999999e-05 2024-09-14T22:38:49.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:49.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419782+0000", 2024-09-14T22:38:49.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2252999999999999e-05 2024-09-14T22:38:49.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.459 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:49.459 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480745+0000", 2024-09-14T22:38:49.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060962385000000001 2024-09-14T22:38:49.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.459 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:49.459 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480900+0000", 2024-09-14T22:38:49.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015513899999999999 2024-09-14T22:38:49.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.459 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:49.459 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481039+0000", 2024-09-14T22:38:49.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013976700000000001 2024-09-14T22:38:49.460 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.460 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:49.460 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.460 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.460 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82588 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1064960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:49.460 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:38:49.460 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 109.97259746500001, 2024-09-14T22:38:49.460 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061863732999999997, 2024-09-14T22:38:49.460 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:49.460 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:49.460 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:49.460 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:49.461 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:49.461 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:49.461 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82588 2024-09-14T22:38:49.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.461 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:49.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:49.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:38:49.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:49.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:38:49.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:49.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419799+0000", 2024-09-14T22:38:49.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4160000000000001e-06 2024-09-14T22:38:49.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:49.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419807+0000", 2024-09-14T22:38:49.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2169999999999994e-06 2024-09-14T22:38:49.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:49.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419824+0000", 2024-09-14T22:38:49.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6776e-05 2024-09-14T22:38:49.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:49.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419837+0000", 2024-09-14T22:38:49.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2680000000000001e-05 2024-09-14T22:38:49.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:49.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481395+0000", 2024-09-14T22:38:49.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061558416999999997 2024-09-14T22:38:49.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:49.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481532+0000", 2024-09-14T22:38:49.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000136861 2024-09-14T22:38:49.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:49.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481662+0000", 2024-09-14T22:38:49.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012936600000000001 2024-09-14T22:38:49.464 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.464 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:49.464 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.464 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82589 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:49.465 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:38:49.465 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 109.97254287, 2024-09-14T22:38:49.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062064592000000002, 2024-09-14T22:38:49.465 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:49.465 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:49.465 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:49.465 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:49.465 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:49.465 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:49.465 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82589 2024-09-14T22:38:49.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.465 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:49.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:49.466 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:38:49.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:49.466 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:38:49.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:49.466 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419854+0000", 2024-09-14T22:38:49.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.367e-06 2024-09-14T22:38:49.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:49.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419862+0000", 2024-09-14T22:38:49.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8930000000000005e-06 2024-09-14T22:38:49.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:49.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419896+0000", 2024-09-14T22:38:49.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4134000000000002e-05 2024-09-14T22:38:49.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:49.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419909+0000", 2024-09-14T22:38:49.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3332999999999999e-05 2024-09-14T22:38:49.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:49.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481702+0000", 2024-09-14T22:38:49.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061792947000000001 2024-09-14T22:38:49.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:49.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481808+0000", 2024-09-14T22:38:49.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105655 2024-09-14T22:38:49.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:49.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481917+0000", 2024-09-14T22:38:49.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109263 2024-09-14T22:38:49.469 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.469 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:49.469 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.469 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82590 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1359872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:49.469 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:38:49.469 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 109.97246926299999, 2024-09-14T22:38:49.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062362895000000002, 2024-09-14T22:38:49.469 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:49.470 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:49.470 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:49.470 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:49.470 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:49.470 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:49.470 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82590 2024-09-14T22:38:49.470 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.470 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:49.470 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.470 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:49.470 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:38:49.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.470 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:49.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:38:49.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:49.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419928+0000", 2024-09-14T22:38:49.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4470000000000001e-06 2024-09-14T22:38:49.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:49.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419942+0000", 2024-09-14T22:38:49.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4229e-05 2024-09-14T22:38:49.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:49.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419972+0000", 2024-09-14T22:38:49.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.065e-05 2024-09-14T22:38:49.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:49.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419985+0000", 2024-09-14T22:38:49.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2903e-05 2024-09-14T22:38:49.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:49.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481960+0000", 2024-09-14T22:38:49.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061974435000000001 2024-09-14T22:38:49.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:49.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482106+0000", 2024-09-14T22:38:49.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014643800000000001 2024-09-14T22:38:49.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:49.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482289+0000", 2024-09-14T22:38:49.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000182793 2024-09-14T22:38:49.473 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.474 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:49.474 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.474 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82592 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:49.474 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:38:49.474 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 109.972393041, 2024-09-14T22:38:49.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062631507000000003, 2024-09-14T22:38:49.474 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:49.474 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:49.474 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:49.474 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:49.474 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:49.475 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:49.475 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82592 2024-09-14T22:38:49.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.475 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:49.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.475 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:49.475 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:38:49.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.475 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:49.475 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:38:49.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:49.476 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420017+0000", 2024-09-14T22:38:49.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4744e-05 2024-09-14T22:38:49.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:49.476 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420036+0000", 2024-09-14T22:38:49.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8680000000000001e-05 2024-09-14T22:38:49.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:49.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420054+0000", 2024-09-14T22:38:49.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7799999999999999e-05 2024-09-14T22:38:49.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:49.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420067+0000", 2024-09-14T22:38:49.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3013000000000001e-05 2024-09-14T22:38:49.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:49.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482358+0000", 2024-09-14T22:38:49.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062291445000000001 2024-09-14T22:38:49.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:49.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482469+0000", 2024-09-14T22:38:49.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110608 2024-09-14T22:38:49.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:49.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482634+0000", 2024-09-14T22:38:49.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000165217 2024-09-14T22:38:49.478 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.478 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:49.478 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.479 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82594 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1654784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:49.479 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:38:49.479 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 109.972295065, 2024-09-14T22:38:49.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062893849000000002, 2024-09-14T22:38:49.479 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:49.479 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:49.479 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:49.479 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:49.479 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:49.479 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:49.479 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82594 2024-09-14T22:38:49.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.480 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:49.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.480 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:49.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:38:49.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.480 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:49.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:38:49.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.480 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:49.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420102+0000", 2024-09-14T22:38:49.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5e-06 2024-09-14T22:38:49.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:49.481 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420115+0000", 2024-09-14T22:38:49.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.324e-05 2024-09-14T22:38:49.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:49.481 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420133+0000", 2024-09-14T22:38:49.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8213e-05 2024-09-14T22:38:49.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:49.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420147+0000", 2024-09-14T22:38:49.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3284e-05 2024-09-14T22:38:49.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:49.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482688+0000", 2024-09-14T22:38:49.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062541425999999997 2024-09-14T22:38:49.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:49.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482794+0000", 2024-09-14T22:38:49.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010575500000000001 2024-09-14T22:38:49.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:49.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482994+0000", 2024-09-14T22:38:49.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020043099999999999 2024-09-14T22:38:49.483 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.483 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:49.483 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.483 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82596 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:49.483 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:38:49.483 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 109.972231765, 2024-09-14T22:38:49.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063187238000000007, 2024-09-14T22:38:49.484 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:49.484 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:49.484 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:49.484 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:49.484 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:49.484 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:49.484 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82596 2024-09-14T22:38:49.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.484 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:49.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.484 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:49.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:38:49.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.485 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:49.485 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:38:49.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.485 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:49.485 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420165+0000", 2024-09-14T22:38:49.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4100000000000001e-06 2024-09-14T22:38:49.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.485 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:49.485 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420173+0000", 2024-09-14T22:38:49.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8129999999999996e-06 2024-09-14T22:38:49.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.486 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:49.486 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420190+0000", 2024-09-14T22:38:49.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7076999999999999e-05 2024-09-14T22:38:49.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.486 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:49.486 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420203+0000", 2024-09-14T22:38:49.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2683e-05 2024-09-14T22:38:49.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.486 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:49.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483060+0000", 2024-09-14T22:38:49.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062857114000000006 2024-09-14T22:38:49.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:49.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483167+0000", 2024-09-14T22:38:49.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010713999999999999 2024-09-14T22:38:49.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:49.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483351+0000", 2024-09-14T22:38:49.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018400099999999999 2024-09-14T22:38:49.487 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.488 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:49.488 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.488 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82597 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1949696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:49.488 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:38:49.488 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 109.972174939, 2024-09-14T22:38:49.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063466513000000002, 2024-09-14T22:38:49.488 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:49.488 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:49.488 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:49.488 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:49.488 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:49.488 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:49.489 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82597 2024-09-14T22:38:49.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.489 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:49.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:49.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:38:49.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:49.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:38:49.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:49.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:49.490 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420221+0000", 2024-09-14T22:38:49.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.043e-06 2024-09-14T22:38:49.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:49.490 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420229+0000", 2024-09-14T22:38:49.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0469999999999994e-06 2024-09-14T22:38:49.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:49.490 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420246+0000", 2024-09-14T22:38:49.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6339999999999999e-05 2024-09-14T22:38:49.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:49.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420258+0000", 2024-09-14T22:38:49.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2296000000000001e-05 2024-09-14T22:38:49.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:49.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483402+0000", 2024-09-14T22:38:49.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063144318000000005 2024-09-14T22:38:49.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:49.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483530+0000", 2024-09-14T22:38:49.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012713199999999999 2024-09-14T22:38:49.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:49.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:49.492 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:49.492 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483687+0000", 2024-09-14T22:38:49.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015733699999999999 2024-09-14T22:38:49.492 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.492 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:49.492 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.492 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:49.492 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:49.492 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:49.494 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:38:49.494 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:49.493+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:49.594 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:49.586+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:49.661 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:49.661 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:49.661 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:38:49.661 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:49.675 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:38:49.719 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:49.714+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:49.795 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:49.790+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:49.840 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:49.841 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:49.841 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:38:49.841 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:38:49.841 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:49.854 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:38:49.872 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:38:49.899 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:49.894+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:49.933 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T22:38:49.997 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:49.990+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:50.041 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:50.041 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:38:50.041 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:38:50.041 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:38:50.042 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.042 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:38:50.042 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:38:50.042 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 170.02899608999999, 2024-09-14T22:38:50.042 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.398029152, 2024-09-14T22:38:50.042 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:50.042 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:50.042 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:50.042 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:50.042 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:38:50.042 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:38:50.042 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 1 2024-09-14T22:38:50.042 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.042 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:50.043 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.043 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:50.043 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:38:50.043 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.043 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.043 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.043 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:50.043 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:38:50.043 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.043 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.043 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.043 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:50.043 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006183+0000", 2024-09-14T22:38:50.044 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9059999999999998e-06 2024-09-14T22:38:50.044 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.044 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.044 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:50.044 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006198+0000", 2024-09-14T22:38:50.044 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.523e-05 2024-09-14T22:38:50.044 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.044 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.044 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:50.044 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006219+0000", 2024-09-14T22:38:50.044 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1342999999999999e-05 2024-09-14T22:38:50.044 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.045 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.045 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:50.045 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006240+0000", 2024-09-14T22:38:50.045 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1186999999999999e-05 2024-09-14T22:38:50.045 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.045 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.045 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:50.045 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006363+0000", 2024-09-14T22:38:50.045 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012258200000000001 2024-09-14T22:38:50.045 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.045 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.045 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:38:50.045 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006392+0000", 2024-09-14T22:38:50.045 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9607e-05 2024-09-14T22:38:50.046 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.046 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.046 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:50.046 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401084+0000", 2024-09-14T22:38:50.046 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.39469142699999998 2024-09-14T22:38:50.046 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.046 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.046 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:50.046 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401394+0000", 2024-09-14T22:38:50.046 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030987799999999998 2024-09-14T22:38:50.046 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.046 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.047 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:38:50.047 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401860+0000", 2024-09-14T22:38:50.047 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00046654299999999998 2024-09-14T22:38:50.047 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.047 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.047 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:38:50.047 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.403386+0000", 2024-09-14T22:38:50.047 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001525322 2024-09-14T22:38:50.047 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.048 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.048 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:50.048 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404042+0000", 2024-09-14T22:38:50.048 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065676099999999996 2024-09-14T22:38:50.048 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.048 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.048 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:38:50.048 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404136+0000", 2024-09-14T22:38:50.048 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4032999999999999e-05 2024-09-14T22:38:50.048 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.048 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.048 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:50.048 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404209+0000", 2024-09-14T22:38:50.049 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2333e-05 2024-09-14T22:38:50.049 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.049 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:50.049 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.049 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.049 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.049 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104182 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14352384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:50.049 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:38:50.049 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 91.546142103999998, 2024-09-14T22:38:50.049 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071642496999999999, 2024-09-14T22:38:50.049 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:50.049 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:50.049 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:50.050 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:50.050 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:50.050 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:50.050 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104182 2024-09-14T22:38:50.050 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.050 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:50.050 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.050 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:50.050 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:38:50.050 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.050 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.050 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:50.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:38:50.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:50.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489035+0000", 2024-09-14T22:38:50.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3430000000000001e-06 2024-09-14T22:38:50.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:50.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489041+0000", 2024-09-14T22:38:50.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8139999999999997e-06 2024-09-14T22:38:50.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.052 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:50.052 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489059+0000", 2024-09-14T22:38:50.052 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8193e-05 2024-09-14T22:38:50.052 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.052 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:50.052 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489073+0000", 2024-09-14T22:38:50.052 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:38:50.052 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.052 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:50.052 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560434+0000", 2024-09-14T22:38:50.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071361463999999999 2024-09-14T22:38:50.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:50.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560545+0000", 2024-09-14T22:38:50.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110263 2024-09-14T22:38:50.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:50.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560676+0000", 2024-09-14T22:38:50.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013138000000000001 2024-09-14T22:38:50.053 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.054 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:50.054 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.054 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104183 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14483456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:50.054 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:38:50.054 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 91.546018785000001, 2024-09-14T22:38:50.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071781943000000001, 2024-09-14T22:38:50.054 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:50.054 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:50.054 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:50.054 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:50.054 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:50.054 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:50.054 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104183 2024-09-14T22:38:50.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.055 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:50.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:50.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:38:50.055 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:50.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:38:50.055 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:50.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489159+0000", 2024-09-14T22:38:50.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5629999999999999e-06 2024-09-14T22:38:50.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.056 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:50.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489165+0000", 2024-09-14T22:38:50.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7070000000000003e-06 2024-09-14T22:38:50.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.056 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:50.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489186+0000", 2024-09-14T22:38:50.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.088e-05 2024-09-14T22:38:50.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.057 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:50.057 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489201+0000", 2024-09-14T22:38:50.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5203e-05 2024-09-14T22:38:50.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.057 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:50.057 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560713+0000", 2024-09-14T22:38:50.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071511447000000006 2024-09-14T22:38:50.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.058 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:50.058 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560819+0000", 2024-09-14T22:38:50.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106579 2024-09-14T22:38:50.058 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.059 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:50.059 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560939+0000", 2024-09-14T22:38:50.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119564 2024-09-14T22:38:50.059 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.059 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:50.059 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.059 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104184 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:50.059 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:38:50.059 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 91.545954342000002, 2024-09-14T22:38:50.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071970107000000005, 2024-09-14T22:38:50.060 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:50.060 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:50.060 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:50.060 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:50.060 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:50.060 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:50.060 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104184 2024-09-14T22:38:50.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.060 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:50.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:50.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:38:50.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:50.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:38:50.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:50.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489223+0000", 2024-09-14T22:38:50.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4470000000000001e-06 2024-09-14T22:38:50.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:50.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489229+0000", 2024-09-14T22:38:50.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8200000000000002e-06 2024-09-14T22:38:50.063 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.063 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.063 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:50.063 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489247+0000", 2024-09-14T22:38:50.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8150000000000001e-05 2024-09-14T22:38:50.063 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.063 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.063 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:50.063 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489260+0000", 2024-09-14T22:38:50.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3570000000000001e-05 2024-09-14T22:38:50.063 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.063 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.064 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:50.064 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560979+0000", 2024-09-14T22:38:50.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071718831999999996 2024-09-14T22:38:50.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.064 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:50.064 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561075+0000", 2024-09-14T22:38:50.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5848999999999997e-05 2024-09-14T22:38:50.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.064 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:50.064 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561191+0000", 2024-09-14T22:38:50.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116439 2024-09-14T22:38:50.065 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.065 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:50.065 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.065 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104185 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14745600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:50.065 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:38:50.065 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 91.545894759000007, 2024-09-14T22:38:50.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072173750999999994, 2024-09-14T22:38:50.065 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:50.065 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:50.065 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:50.065 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:50.066 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:50.066 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:50.066 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104185 2024-09-14T22:38:50.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.066 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:50.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:50.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:38:50.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:50.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:38:50.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.067 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:50.067 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489282+0000", 2024-09-14T22:38:50.067 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2500000000000001e-06 2024-09-14T22:38:50.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.067 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:50.067 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489287+0000", 2024-09-14T22:38:50.067 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1359999999999996e-06 2024-09-14T22:38:50.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.067 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:50.067 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489304+0000", 2024-09-14T22:38:50.068 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7033000000000001e-05 2024-09-14T22:38:50.068 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.068 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.068 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:50.068 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489318+0000", 2024-09-14T22:38:50.068 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3934000000000001e-05 2024-09-14T22:38:50.068 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.068 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.068 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:50.068 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561232+0000", 2024-09-14T22:38:50.068 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071913759999999993 2024-09-14T22:38:50.068 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.068 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:50.069 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561340+0000", 2024-09-14T22:38:50.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107921 2024-09-14T22:38:50.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:50.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561455+0000", 2024-09-14T22:38:50.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011471700000000001 2024-09-14T22:38:50.070 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.070 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:50.070 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.071 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104186 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14876672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:50.071 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:38:50.071 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 91.545838849999996, 2024-09-14T22:38:50.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072393374999999996, 2024-09-14T22:38:50.071 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:50.072 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:50.072 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:50.072 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:50.072 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:50.072 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:50.072 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104186 2024-09-14T22:38:50.073 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.073 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:50.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.073 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:50.073 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:38:50.073 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.074 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:50.074 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:38:50.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.075 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:50.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489338+0000", 2024-09-14T22:38:50.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.48e-06 2024-09-14T22:38:50.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.075 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.075 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:50.076 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489366+0000", 2024-09-14T22:38:50.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7307000000000001e-05 2024-09-14T22:38:50.076 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.076 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.076 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:50.076 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489385+0000", 2024-09-14T22:38:50.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9283000000000002e-05 2024-09-14T22:38:50.077 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.077 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:50.077 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489399+0000", 2024-09-14T22:38:50.077 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.452e-05 2024-09-14T22:38:50.077 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.078 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:50.078 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561494+0000", 2024-09-14T22:38:50.078 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072094885999999997 2024-09-14T22:38:50.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.078 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.078 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:50.079 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561604+0000", 2024-09-14T22:38:50.079 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010925900000000001 2024-09-14T22:38:50.079 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.079 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.079 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:50.079 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561730+0000", 2024-09-14T22:38:50.080 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012663999999999999 2024-09-14T22:38:50.080 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.080 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:50.080 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.080 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.081 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.081 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104187 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15007744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:50.081 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:38:50.081 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 91.545755481, 2024-09-14T22:38:50.081 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072599143000000005, 2024-09-14T22:38:50.081 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:50.082 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:50.082 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:50.082 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:50.082 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:50.082 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:50.082 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104187 2024-09-14T22:38:50.082 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.083 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:50.083 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.083 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:50.083 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:38:50.083 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.083 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.083 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.083 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:50.083 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:38:50.084 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.084 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.084 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.084 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:50.084 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489422+0000", 2024-09-14T22:38:50.084 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5340000000000001e-06 2024-09-14T22:38:50.084 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.084 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.085 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:50.085 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489428+0000", 2024-09-14T22:38:50.085 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1430000000000003e-06 2024-09-14T22:38:50.085 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.085 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.085 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:50.085 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489445+0000", 2024-09-14T22:38:50.085 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7042999999999999e-05 2024-09-14T22:38:50.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:50.086 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489458+0000", 2024-09-14T22:38:50.086 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3474e-05 2024-09-14T22:38:50.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:50.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561771+0000", 2024-09-14T22:38:50.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072312600000000005 2024-09-14T22:38:50.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.088 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:50.088 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561885+0000", 2024-09-14T22:38:50.088 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011411799999999999 2024-09-14T22:38:50.088 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.088 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.088 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:50.089 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562019+0000", 2024-09-14T22:38:50.089 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013423100000000001 2024-09-14T22:38:50.089 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.089 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:50.089 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.089 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.089 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.089 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104188 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15138816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:50.089 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:38:50.089 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 91.545695557000002, 2024-09-14T22:38:50.090 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072824624000000004, 2024-09-14T22:38:50.090 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:50.090 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:50.090 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:50.090 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:50.090 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:50.090 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:50.090 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104188 2024-09-14T22:38:50.090 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.090 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:50.090 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.090 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:50.090 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:38:50.090 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.091 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:50.091 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:38:50.091 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.091 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:50.091 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489481+0000", 2024-09-14T22:38:50.091 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.22e-06 2024-09-14T22:38:50.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.091 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:50.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489487+0000", 2024-09-14T22:38:50.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6930000000000004e-06 2024-09-14T22:38:50.092 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.092 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.092 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:50.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489508+0000", 2024-09-14T22:38:50.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1135999999999999e-05 2024-09-14T22:38:50.092 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.092 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.092 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:50.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489522+0000", 2024-09-14T22:38:50.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3817e-05 2024-09-14T22:38:50.092 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.092 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.093 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:50.093 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562066+0000", 2024-09-14T22:38:50.093 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072543813999999998 2024-09-14T22:38:50.093 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.093 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.093 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:50.093 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562177+0000", 2024-09-14T22:38:50.093 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000111264 2024-09-14T22:38:50.093 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.093 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.093 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:50.093 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562305+0000", 2024-09-14T22:38:50.093 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012768 2024-09-14T22:38:50.094 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.094 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:50.094 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.094 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.094 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.094 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104189 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15269888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:50.094 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:38:50.094 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 91.545635107999999, 2024-09-14T22:38:50.094 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073061300999999995, 2024-09-14T22:38:50.094 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:50.094 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:50.094 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:50.094 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:50.094 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:50.095 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:50.095 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104189 2024-09-14T22:38:50.095 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.095 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:50.095 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.095 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:50.095 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:38:50.095 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.095 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.095 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.095 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:50.095 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:38:50.095 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.095 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.096 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:50.096 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489542+0000", 2024-09-14T22:38:50.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1659999999999999e-06 2024-09-14T22:38:50.096 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.096 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:50.096 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489561+0000", 2024-09-14T22:38:50.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.908e-05 2024-09-14T22:38:50.096 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.096 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:50.096 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489579+0000", 2024-09-14T22:38:50.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8536999999999999e-05 2024-09-14T22:38:50.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.097 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:50.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489593+0000", 2024-09-14T22:38:50.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3977e-05 2024-09-14T22:38:50.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.097 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:50.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562347+0000", 2024-09-14T22:38:50.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072754092000000006 2024-09-14T22:38:50.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.097 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:50.098 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562458+0000", 2024-09-14T22:38:50.098 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110722 2024-09-14T22:38:50.098 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.098 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.098 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:50.098 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562602+0000", 2024-09-14T22:38:50.098 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014372699999999999 2024-09-14T22:38:50.098 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.098 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:50.098 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.098 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.098 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.098 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104190 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:50.098 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:38:50.099 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 91.545562812, 2024-09-14T22:38:50.099 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.07326133, 2024-09-14T22:38:50.099 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:50.099 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:50.099 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:50.099 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:50.099 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:50.099 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:50.099 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104190 2024-09-14T22:38:50.099 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.100 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:50.100 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.100 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:50.100 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:38:50.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.100 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.100 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:50.100 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:38:50.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:50.101 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489614+0000", 2024-09-14T22:38:50.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1999999999999999e-06 2024-09-14T22:38:50.101 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:50.101 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489620+0000", 2024-09-14T22:38:50.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.524e-06 2024-09-14T22:38:50.101 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:50.101 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489637+0000", 2024-09-14T22:38:50.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7825999999999998e-05 2024-09-14T22:38:50.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:50.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489671+0000", 2024-09-14T22:38:50.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3726000000000002e-05 2024-09-14T22:38:50.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:50.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562644+0000", 2024-09-14T22:38:50.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072973089000000005 2024-09-14T22:38:50.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:50.103 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562762+0000", 2024-09-14T22:38:50.103 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011743600000000001 2024-09-14T22:38:50.103 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.103 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.103 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:50.103 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562874+0000", 2024-09-14T22:38:50.103 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011252900000000001 2024-09-14T22:38:50.103 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.103 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:50.103 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.103 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.103 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.103 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104191 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15532032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:50.104 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:38:50.104 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 91.545454389, 2024-09-14T22:38:50.104 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073311166999999997, 2024-09-14T22:38:50.104 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:50.104 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:50.104 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:50.104 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:50.104 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:50.104 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:50.104 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104191 2024-09-14T22:38:50.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.105 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:50.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:50.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:38:50.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:50.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:38:50.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:50.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489723+0000", 2024-09-14T22:38:50.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6330000000000001e-06 2024-09-14T22:38:50.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:50.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489730+0000", 2024-09-14T22:38:50.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2300000000000002e-06 2024-09-14T22:38:50.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:50.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489751+0000", 2024-09-14T22:38:50.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0849999999999999e-05 2024-09-14T22:38:50.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.107 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:50.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489766+0000", 2024-09-14T22:38:50.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.524e-05 2024-09-14T22:38:50.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.107 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:50.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562896+0000", 2024-09-14T22:38:50.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073129662999999998 2024-09-14T22:38:50.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.107 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:50.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562960+0000", 2024-09-14T22:38:50.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3689999999999995e-05 2024-09-14T22:38:50.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:50.108 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563033+0000", 2024-09-14T22:38:50.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2861000000000006e-05 2024-09-14T22:38:50.108 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.108 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:50.108 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.108 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104192 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:50.108 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:38:50.109 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 91.545358583999999, 2024-09-14T22:38:50.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073463632000000001, 2024-09-14T22:38:50.109 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:50.109 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:50.109 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:50.109 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:50.109 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:50.109 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:50.109 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104192 2024-09-14T22:38:50.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.109 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:50.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.109 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:50.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:38:50.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:50.110 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:38:50.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:50.110 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489819+0000", 2024-09-14T22:38:50.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.424e-06 2024-09-14T22:38:50.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:50.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489825+0000", 2024-09-14T22:38:50.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.02e-06 2024-09-14T22:38:50.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:50.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489844+0000", 2024-09-14T22:38:50.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9760000000000001e-05 2024-09-14T22:38:50.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:50.112 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489860+0000", 2024-09-14T22:38:50.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5563e-05 2024-09-14T22:38:50.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.112 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:50.112 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563070+0000", 2024-09-14T22:38:50.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073209724000000004 2024-09-14T22:38:50.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.112 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:50.112 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563175+0000", 2024-09-14T22:38:50.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105183 2024-09-14T22:38:50.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:50.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563281+0000", 2024-09-14T22:38:50.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105958 2024-09-14T22:38:50.113 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.113 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:50.113 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.113 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104193 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15794176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:50.113 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:38:50.113 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 91.545296417000003, 2024-09-14T22:38:50.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073600040000000005, 2024-09-14T22:38:50.113 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:50.114 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:50.114 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:50.114 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:50.114 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:50.114 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:50.114 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104193 2024-09-14T22:38:50.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.114 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:50.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:50.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:38:50.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:50.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:38:50.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:50.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489881+0000", 2024-09-14T22:38:50.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5460000000000001e-06 2024-09-14T22:38:50.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:50.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489886+0000", 2024-09-14T22:38:50.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1739999999999999e-06 2024-09-14T22:38:50.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:50.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489912+0000", 2024-09-14T22:38:50.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6461999999999998e-05 2024-09-14T22:38:50.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:50.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489927+0000", 2024-09-14T22:38:50.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:38:50.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:50.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563310+0000", 2024-09-14T22:38:50.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073382977000000002 2024-09-14T22:38:50.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.117 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:50.117 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563385+0000", 2024-09-14T22:38:50.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5218999999999999e-05 2024-09-14T22:38:50.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.117 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:50.117 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563479+0000", 2024-09-14T22:38:50.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4622000000000003e-05 2024-09-14T22:38:50.118 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.118 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:50.118 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.118 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104194 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15925248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:50.118 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:38:50.118 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 91.545229825000007, 2024-09-14T22:38:50.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073745434999999998, 2024-09-14T22:38:50.118 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:50.118 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:50.118 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:50.119 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:50.119 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:50.119 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:50.119 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104194 2024-09-14T22:38:50.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.119 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:50.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:50.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:38:50.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:50.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:38:50.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:50.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489947+0000", 2024-09-14T22:38:50.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.327e-06 2024-09-14T22:38:50.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.121 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:50.121 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489953+0000", 2024-09-14T22:38:50.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9329999999999998e-06 2024-09-14T22:38:50.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:50.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489971+0000", 2024-09-14T22:38:50.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7994000000000001e-05 2024-09-14T22:38:50.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:50.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489985+0000", 2024-09-14T22:38:50.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4226000000000001e-05 2024-09-14T22:38:50.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:50.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563510+0000", 2024-09-14T22:38:50.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073524667000000002 2024-09-14T22:38:50.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:50.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563597+0000", 2024-09-14T22:38:50.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6550000000000003e-05 2024-09-14T22:38:50.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:50.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563691+0000", 2024-09-14T22:38:50.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4738e-05 2024-09-14T22:38:50.124 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.124 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:50.124 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.125 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104195 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:50.125 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:38:50.125 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 91.545171628000006, 2024-09-14T22:38:50.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073876598000000002, 2024-09-14T22:38:50.125 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:50.125 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:50.125 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:50.125 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:50.125 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:50.125 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:50.125 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104195 2024-09-14T22:38:50.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.126 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:50.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.126 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:50.126 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:38:50.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.126 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:50.126 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:38:50.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.126 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:50.126 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490006+0000", 2024-09-14T22:38:50.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4699999999999999e-06 2024-09-14T22:38:50.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:50.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490011+0000", 2024-09-14T22:38:50.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.366e-06 2024-09-14T22:38:50.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:50.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490046+0000", 2024-09-14T22:38:50.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5496e-05 2024-09-14T22:38:50.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:50.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490061+0000", 2024-09-14T22:38:50.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5054e-05 2024-09-14T22:38:50.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:50.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563719+0000", 2024-09-14T22:38:50.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073657412000000005 2024-09-14T22:38:50.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:50.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563792+0000", 2024-09-14T22:38:50.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3319999999999996e-05 2024-09-14T22:38:50.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:50.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563881+0000", 2024-09-14T22:38:50.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8480000000000007e-05 2024-09-14T22:38:50.129 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.129 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:50.129 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.129 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104196 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16187392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:50.129 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:38:50.130 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 91.545094478999999, 2024-09-14T22:38:50.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073997496999999995, 2024-09-14T22:38:50.130 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:50.130 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:50.130 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:50.130 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:50.130 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:50.130 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:50.130 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104196 2024-09-14T22:38:50.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.130 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:50.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:50.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:38:50.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:50.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:38:50.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:50.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490083+0000", 2024-09-14T22:38:50.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.347e-06 2024-09-14T22:38:50.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:50.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490089+0000", 2024-09-14T22:38:50.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9630000000000003e-06 2024-09-14T22:38:50.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:50.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490107+0000", 2024-09-14T22:38:50.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8859999999999999e-05 2024-09-14T22:38:50.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:50.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490122+0000", 2024-09-14T22:38:50.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.432e-05 2024-09-14T22:38:50.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:50.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563910+0000", 2024-09-14T22:38:50.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073788753999999998 2024-09-14T22:38:50.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:50.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563988+0000", 2024-09-14T22:38:50.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7862999999999996e-05 2024-09-14T22:38:50.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.135 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:50.135 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564079+0000", 2024-09-14T22:38:50.135 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0389999999999993e-05 2024-09-14T22:38:50.135 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.135 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:50.135 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.136 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104197 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16318464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:50.136 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:38:50.136 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 91.545002850000003, 2024-09-14T22:38:50.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074103547000000006, 2024-09-14T22:38:50.137 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:50.137 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:50.137 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:50.137 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:50.137 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:50.137 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:50.137 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104197 2024-09-14T22:38:50.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.138 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:50.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:50.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:38:50.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.139 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:50.139 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:38:50.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.140 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:50.140 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490174+0000", 2024-09-14T22:38:50.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.37e-06 2024-09-14T22:38:50.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:50.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490197+0000", 2024-09-14T22:38:50.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3017e-05 2024-09-14T22:38:50.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:50.144 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490219+0000", 2024-09-14T22:38:50.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.181e-05 2024-09-14T22:38:50.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:50.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490240+0000", 2024-09-14T22:38:50.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0786000000000001e-05 2024-09-14T22:38:50.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:50.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564109+0000", 2024-09-14T22:38:50.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073869031000000002 2024-09-14T22:38:50.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:50.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564186+0000", 2024-09-14T22:38:50.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6823999999999995e-05 2024-09-14T22:38:50.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:50.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564276+0000", 2024-09-14T22:38:50.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0709000000000002e-05 2024-09-14T22:38:50.146 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.146 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:50.146 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.146 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104198 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16449536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:50.146 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:38:50.146 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 91.544910614000003, 2024-09-14T22:38:50.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074167693000000007, 2024-09-14T22:38:50.147 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:50.147 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:50.147 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:50.147 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:50.147 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:50.147 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:50.147 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104198 2024-09-14T22:38:50.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.147 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:50.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:50.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:38:50.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.148 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:50.148 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:38:50.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.148 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:50.148 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490266+0000", 2024-09-14T22:38:50.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3599999999999999e-06 2024-09-14T22:38:50.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.148 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:50.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490272+0000", 2024-09-14T22:38:50.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9599999999999997e-06 2024-09-14T22:38:50.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.149 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:50.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490291+0000", 2024-09-14T22:38:50.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8485999999999999e-05 2024-09-14T22:38:50.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.149 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:50.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490305+0000", 2024-09-14T22:38:50.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4327e-05 2024-09-14T22:38:50.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:50.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564306+0000", 2024-09-14T22:38:50.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074000745000000007 2024-09-14T22:38:50.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:50.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564367+0000", 2024-09-14T22:38:50.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1501999999999995e-05 2024-09-14T22:38:50.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:50.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564433+0000", 2024-09-14T22:38:50.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5313000000000003e-05 2024-09-14T22:38:50.150 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.151 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:50.151 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.151 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104199 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16580608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:50.151 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:38:50.151 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 91.544829061000002, 2024-09-14T22:38:50.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074231631000000006, 2024-09-14T22:38:50.151 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:50.151 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:50.151 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:50.151 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:50.151 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:50.151 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:50.152 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104199 2024-09-14T22:38:50.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.152 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:50.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.152 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:50.152 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:38:50.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.152 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:50.152 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:38:50.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:50.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490349+0000", 2024-09-14T22:38:50.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9429999999999999e-06 2024-09-14T22:38:50.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:50.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490355+0000", 2024-09-14T22:38:50.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1670000000000004e-06 2024-09-14T22:38:50.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:50.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490374+0000", 2024-09-14T22:38:50.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9613e-05 2024-09-14T22:38:50.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:50.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490405+0000", 2024-09-14T22:38:50.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0936000000000003e-05 2024-09-14T22:38:50.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:50.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564451+0000", 2024-09-14T22:38:50.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074045266999999998 2024-09-14T22:38:50.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:50.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564494+0000", 2024-09-14T22:38:50.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3133000000000002e-05 2024-09-14T22:38:50.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:50.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564578+0000", 2024-09-14T22:38:50.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4572000000000003e-05 2024-09-14T22:38:50.155 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.155 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:50.155 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.155 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104200 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:50.155 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:38:50.155 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 91.543033832999996, 2024-09-14T22:38:50.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072582435000000001, 2024-09-14T22:38:50.155 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:50.156 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:50.156 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:50.156 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:50.156 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:50.156 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:50.156 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104200 2024-09-14T22:38:50.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.156 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:50.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:50.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:38:50.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:50.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:38:50.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:50.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:50.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492144+0000", 2024-09-14T22:38:50.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3070000000000001e-06 2024-09-14T22:38:50.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:50.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492155+0000", 2024-09-14T22:38:50.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0573e-05 2024-09-14T22:38:50.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:50.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492185+0000", 2024-09-14T22:38:50.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.008e-05 2024-09-14T22:38:50.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:50.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492205+0000", 2024-09-14T22:38:50.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9882999999999999e-05 2024-09-14T22:38:50.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:50.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564609+0000", 2024-09-14T22:38:50.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072404181999999997 2024-09-14T22:38:50.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:50.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564667+0000", 2024-09-14T22:38:50.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8102e-05 2024-09-14T22:38:50.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:50.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:50.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:50.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564724+0000", 2024-09-14T22:38:50.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7308000000000001e-05 2024-09-14T22:38:50.159 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.159 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:50.159 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.159 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:50.160 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:50.160 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:50.160 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:38:50.161 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:38:50.161 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:38:50.162 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:38:50.162 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:38:50.163 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:38:50.163 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:38:50.163 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:38:50.163 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:38:50.164 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-14T22:38:50.164 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:38:50.164 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:38:50.164 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:38:50.165 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:38:50.165 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:38:50.165 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:50.157+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:50.176 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:38:50.177 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:38:50.263 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:38:50.258+0000 7f36371f3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26504) UID: 0 2024-09-14T22:38:50.283 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:50.283 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:50.297 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:38:50.363 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:50.358+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:50.417 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:50.417 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:50.432 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:38:50.478 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:50.477+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:50.556 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:50.556 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:50.565 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:50.561+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:50.571 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:38:50.666 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:50.665+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:50.733 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2024-09-14T22:38:50.741 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:50.741 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:50.760 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:38:50.767 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:50.762+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:50.868 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:50.862+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:50.930 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-14T22:38:50.930 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:38:50.930 INFO:tasks.thrashosds.thrasher:Killing osd 4, live_osds are [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-14T22:38:50.931 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-14T22:38:50.931 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T22:38:50.945 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:50.945 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:50.961 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:38:51.006 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:51.005+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:51.019 INFO:tasks.ceph.osd.4:Stopped 2024-09-14T22:38:51.019 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.4 2024-09-14T22:38:51.019 INFO:tasks.thrashosds.thrasher:remote for osd 4 is ubuntu@smithi176.front.sepia.ceph.com 2024-09-14T22:38:51.019 DEBUG:teuthology.orchestra.run.smithi176:> type ceph-objectstore-tool 2024-09-14T22:38:51.024 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-14T22:38:51.024 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --op list-pgs 2024-09-14T22:38:51.070 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:51.069+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:51.088 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:51.088 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:51.102 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:38:51.190 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:51.189+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:51.269 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:51.269 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:51.273 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:51.269+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:51.283 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:38:51.374 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:51.373+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:51.455 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:51.455 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:51.469 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:38:51.514 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:51.513+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:51.575 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:51.573+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:51.637 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:51.637 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:51.652 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:38:51.694 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:51.693+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:51.776 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:51.770+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:51.816 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:51.816 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:51.831 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:38:51.878 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:51.877+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:51.950 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:51.950 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:51.964 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:38:51.978 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:51.970+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:52.079 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:52.074+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:52.131 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:52.131 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:52.146 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:38:52.180 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:52.177+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:52.196 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:38:52.268 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:52.268 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:52.280 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:38:52.281 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:52.274+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:52.346 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:38:52.347 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:38:52.363 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:38:52.364 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:38:52.380 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:38:52.380 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:38:52.382 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:52.376+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:52.396 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:38:52.396 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:38:52.396 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T22:38:52.416 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:38:52.417 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:38:52.438 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:38:52.438 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:38:52.444 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:52.444 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:52.445 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:38:52.445 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:38:52.445 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:52.455 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:38:52.456 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:38:52.458 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:38:52.459 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:38:52.459 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:38:52.503 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:52.498+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:52.580 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:38:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:38:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:38:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97197 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 796672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:38:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 102.505800953, 2024-09-14T22:38:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12434473, 2024-09-14T22:38:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:52.582 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:52.582 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:52.582 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97197 2024-09-14T22:38:52.582 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.582 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:52.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.582 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:52.582 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:38:52.582 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.582 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.582 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:52.582 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:38:52.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.583 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:52.583 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070524+0000", 2024-09-14T22:38:52.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0299999999999998e-07 2024-09-14T22:38:52.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.583 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:52.583 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070527+0000", 2024-09-14T22:38:52.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9569999999999998e-06 2024-09-14T22:38:52.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.583 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:52.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070535+0000", 2024-09-14T22:38:52.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7630000000000006e-06 2024-09-14T22:38:52.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:52.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070543+0000", 2024-09-14T22:38:52.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4869999999999996e-06 2024-09-14T22:38:52.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:52.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194525+0000", 2024-09-14T22:38:52.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123982096 2024-09-14T22:38:52.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:52.585 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194699+0000", 2024-09-14T22:38:52.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000174105 2024-09-14T22:38:52.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:52.585 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194868+0000", 2024-09-14T22:38:52.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016861900000000001 2024-09-14T22:38:52.585 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.585 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:52.585 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.586 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97198 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 805888~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:52.586 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:38:52.586 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 102.505772577, 2024-09-14T22:38:52.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.1248088, 2024-09-14T22:38:52.586 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:52.586 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:52.586 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:52.586 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:52.586 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:52.586 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:52.586 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97198 2024-09-14T22:38:52.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.587 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:52.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.587 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:52.587 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:38:52.587 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.587 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:52.587 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:38:52.587 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.587 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:52.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070552+0000", 2024-09-14T22:38:52.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T22:38:52.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:52.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070555+0000", 2024-09-14T22:38:52.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.853e-06 2024-09-14T22:38:52.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:52.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070565+0000", 2024-09-14T22:38:52.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.048e-05 2024-09-14T22:38:52.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:52.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070573+0000", 2024-09-14T22:38:52.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7200000000000006e-06 2024-09-14T22:38:52.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:52.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194954+0000", 2024-09-14T22:38:52.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12438089400000001 2024-09-14T22:38:52.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:52.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195145+0000", 2024-09-14T22:38:52.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019091299999999999 2024-09-14T22:38:52.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:52.590 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195360+0000", 2024-09-14T22:38:52.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021527300000000001 2024-09-14T22:38:52.590 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.590 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:52.590 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.590 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97199 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 815104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:52.590 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:38:52.590 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 102.50574198, 2024-09-14T22:38:52.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125251901, 2024-09-14T22:38:52.591 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:52.591 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:52.591 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:52.591 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:52.591 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:52.591 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:52.591 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97199 2024-09-14T22:38:52.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.591 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:52.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.591 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:52.591 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:38:52.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.592 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:52.592 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:38:52.592 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.592 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:52.592 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070583+0000", 2024-09-14T22:38:52.592 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6300000000000005e-07 2024-09-14T22:38:52.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.592 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:52.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070586+0000", 2024-09-14T22:38:52.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8270000000000002e-06 2024-09-14T22:38:52.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.593 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:52.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070597+0000", 2024-09-14T22:38:52.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1960000000000001e-05 2024-09-14T22:38:52.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.593 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:52.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070605+0000", 2024-09-14T22:38:52.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6459999999999993e-06 2024-09-14T22:38:52.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:52.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195432+0000", 2024-09-14T22:38:52.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124826926 2024-09-14T22:38:52.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:52.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195626+0000", 2024-09-14T22:38:52.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019357099999999999 2024-09-14T22:38:52.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:52.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195834+0000", 2024-09-14T22:38:52.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020830800000000001 2024-09-14T22:38:52.595 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.595 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:52.595 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.595 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97200 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 824320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:52.595 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:38:52.595 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 102.505709934, 2024-09-14T22:38:52.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125693477, 2024-09-14T22:38:52.595 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:52.596 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:52.596 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:52.596 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:52.596 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:52.596 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:52.596 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97200 2024-09-14T22:38:52.596 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.596 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:52.596 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.596 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:52.596 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:38:52.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.596 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.596 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.597 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:52.597 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:38:52.597 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.597 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:52.597 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070615+0000", 2024-09-14T22:38:52.597 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T22:38:52.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.597 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:52.597 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070618+0000", 2024-09-14T22:38:52.597 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.943e-06 2024-09-14T22:38:52.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:52.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070627+0000", 2024-09-14T22:38:52.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1099999999999992e-06 2024-09-14T22:38:52.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:52.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070634+0000", 2024-09-14T22:38:52.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:38:52.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:52.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195908+0000", 2024-09-14T22:38:52.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125273893 2024-09-14T22:38:52.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:52.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196097+0000", 2024-09-14T22:38:52.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018953899999999999 2024-09-14T22:38:52.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:52.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196308+0000", 2024-09-14T22:38:52.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021027799999999999 2024-09-14T22:38:52.599 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.599 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:52.600 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.600 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97201 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 833536~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:52.600 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:38:52.600 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 102.505679747, 2024-09-14T22:38:52.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126133826, 2024-09-14T22:38:52.600 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:52.600 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:52.600 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:52.600 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:52.600 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:52.600 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:52.600 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97201 2024-09-14T22:38:52.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.601 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:52.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:52.601 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:38:52.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:52.601 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:38:52.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:52.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074109+0000", 2024-09-14T22:38:52.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003464834 2024-09-14T22:38:52.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.602 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:52.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074134+0000", 2024-09-14T22:38:52.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5170000000000001e-05 2024-09-14T22:38:52.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.602 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:52.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074166+0000", 2024-09-14T22:38:52.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1347000000000003e-05 2024-09-14T22:38:52.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:52.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074179+0000", 2024-09-14T22:38:52.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.364e-05 2024-09-14T22:38:52.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:52.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196377+0000", 2024-09-14T22:38:52.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12219793900000001 2024-09-14T22:38:52.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:52.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196568+0000", 2024-09-14T22:38:52.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019097999999999999 2024-09-14T22:38:52.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:52.604 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196778+0000", 2024-09-14T22:38:52.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020991599999999999 2024-09-14T22:38:52.604 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.604 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:52.604 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.604 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.604 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97205 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 870400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:52.604 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:38:52.604 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 102.502010184, 2024-09-14T22:38:52.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12436148299999999, 2024-09-14T22:38:52.604 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:52.604 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:52.604 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:52.605 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:52.605 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:52.605 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:52.605 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97205 2024-09-14T22:38:52.605 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.605 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:52.605 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.605 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:52.605 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:38:52.605 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.605 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.605 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.605 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:52.606 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:38:52.606 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.606 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:52.606 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074315+0000", 2024-09-14T22:38:52.606 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9299999999999997e-07 2024-09-14T22:38:52.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.606 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:52.606 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074318+0000", 2024-09-14T22:38:52.606 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.19e-06 2024-09-14T22:38:52.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:52.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074327+0000", 2024-09-14T22:38:52.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8529999999999998e-06 2024-09-14T22:38:52.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:52.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074334+0000", 2024-09-14T22:38:52.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3869999999999997e-06 2024-09-14T22:38:52.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:52.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198365+0000", 2024-09-14T22:38:52.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124031222 2024-09-14T22:38:52.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:52.608 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198513+0000", 2024-09-14T22:38:52.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014772299999999999 2024-09-14T22:38:52.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:52.608 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198675+0000", 2024-09-14T22:38:52.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016241500000000001 2024-09-14T22:38:52.608 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.608 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:52.608 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.609 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.609 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97206 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 879616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:52.609 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:38:52.609 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 102.501978728, 2024-09-14T22:38:52.609 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124706284, 2024-09-14T22:38:52.609 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:52.609 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:52.609 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:52.609 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:52.609 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:52.609 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:52.609 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97206 2024-09-14T22:38:52.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.609 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:52.609 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.610 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:52.610 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:38:52.610 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.610 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.610 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.610 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:52.610 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:38:52.610 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.610 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.610 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.610 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:52.610 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074346+0000", 2024-09-14T22:38:52.611 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8999999999999996e-07 2024-09-14T22:38:52.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.611 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:52.611 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074349+0000", 2024-09-14T22:38:52.611 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.96e-06 2024-09-14T22:38:52.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.611 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:52.611 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074358+0000", 2024-09-14T22:38:52.611 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9169999999999995e-06 2024-09-14T22:38:52.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.611 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:52.611 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074365+0000", 2024-09-14T22:38:52.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3699999999999997e-06 2024-09-14T22:38:52.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:52.612 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198761+0000", 2024-09-14T22:38:52.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124395743 2024-09-14T22:38:52.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:52.612 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198899+0000", 2024-09-14T22:38:52.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013763499999999999 2024-09-14T22:38:52.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:52.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199052+0000", 2024-09-14T22:38:52.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015296900000000001 2024-09-14T22:38:52.613 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.613 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:52.613 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.613 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97207 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 888832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:52.613 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:38:52.613 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 102.50194951500001, 2024-09-14T22:38:52.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124988433, 2024-09-14T22:38:52.613 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:52.613 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:52.613 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:52.614 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:52.614 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:52.614 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:52.614 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97207 2024-09-14T22:38:52.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.614 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:52.614 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:52.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:38:52.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.614 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:52.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:38:52.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.615 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.615 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:52.615 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074375+0000", 2024-09-14T22:38:52.615 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:38:52.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.615 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.615 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:52.615 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074378+0000", 2024-09-14T22:38:52.615 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1829999999999998e-06 2024-09-14T22:38:52.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.615 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.615 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:52.616 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074389+0000", 2024-09-14T22:38:52.616 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0737000000000001e-05 2024-09-14T22:38:52.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.616 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.616 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:52.616 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074396+0000", 2024-09-14T22:38:52.616 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2559999999999996e-06 2024-09-14T22:38:52.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.616 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.616 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:52.616 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199139+0000", 2024-09-14T22:38:52.616 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12474253 2024-09-14T22:38:52.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.617 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:52.617 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199240+0000", 2024-09-14T22:38:52.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101221 2024-09-14T22:38:52.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.617 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:52.617 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199363+0000", 2024-09-14T22:38:52.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000122822 2024-09-14T22:38:52.617 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.617 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:52.617 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.618 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97208 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 898048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:52.618 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:38:52.618 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 102.501918211, 2024-09-14T22:38:52.618 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12524854899999999, 2024-09-14T22:38:52.618 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:52.618 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:52.618 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:52.618 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:52.618 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:52.618 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:52.618 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97208 2024-09-14T22:38:52.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.618 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:52.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:52.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:38:52.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:52.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:38:52.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:52.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:38:52.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:38:52.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:52.620 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074410+0000", 2024-09-14T22:38:52.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.18e-06 2024-09-14T22:38:52.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:52.620 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074435+0000", 2024-09-14T22:38:52.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5273e-05 2024-09-14T22:38:52.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:52.620 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074443+0000", 2024-09-14T22:38:52.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3000000000000002e-06 2024-09-14T22:38:52.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.621 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:52.621 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199433+0000", 2024-09-14T22:38:52.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124989502 2024-09-14T22:38:52.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.621 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:52.621 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199535+0000", 2024-09-14T22:38:52.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102557 2024-09-14T22:38:52.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:52.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199654+0000", 2024-09-14T22:38:52.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119061 2024-09-14T22:38:52.622 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.622 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:52.622 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.622 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97209 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 907264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:52.622 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:38:52.622 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 102.50187040599999, 2024-09-14T22:38:52.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125487354, 2024-09-14T22:38:52.622 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:52.622 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:52.623 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:52.623 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:52.623 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:52.623 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:52.623 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97209 2024-09-14T22:38:52.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.623 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:52.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.623 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:52.623 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:38:52.623 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.623 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:52.623 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:38:52.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.624 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.624 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:52.624 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:38:52.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9400000000000005e-07 2024-09-14T22:38:52.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.624 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.624 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:52.624 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074458+0000", 2024-09-14T22:38:52.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3299999999999999e-06 2024-09-14T22:38:52.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.625 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.625 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:52.625 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074467+0000", 2024-09-14T22:38:52.625 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.346e-06 2024-09-14T22:38:52.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.625 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.625 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:52.625 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074474+0000", 2024-09-14T22:38:52.625 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.374e-06 2024-09-14T22:38:52.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.625 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.625 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:52.625 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199720+0000", 2024-09-14T22:38:52.625 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12524561200000001 2024-09-14T22:38:52.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:52.626 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199820+0000", 2024-09-14T22:38:52.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010049599999999999 2024-09-14T22:38:52.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:52.626 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199941+0000", 2024-09-14T22:38:52.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012050200000000001 2024-09-14T22:38:52.626 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.626 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:52.626 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.627 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97210 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:52.627 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:38:52.627 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 102.50184041200001, 2024-09-14T22:38:52.627 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125744942, 2024-09-14T22:38:52.627 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:52.627 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:52.627 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:52.627 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:52.627 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:52.627 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:52.627 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97210 2024-09-14T22:38:52.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.628 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:52.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:52.628 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:38:52.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:52.628 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:38:52.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:52.629 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:38:52.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6599999999999996e-07 2024-09-14T22:38:52.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.629 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.629 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:52.629 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074487+0000", 2024-09-14T22:38:52.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8140000000000001e-06 2024-09-14T22:38:52.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.629 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.629 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:52.629 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074496+0000", 2024-09-14T22:38:52.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9330000000000007e-06 2024-09-14T22:38:52.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.629 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.630 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:52.630 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074503+0000", 2024-09-14T22:38:52.630 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4100000000000002e-06 2024-09-14T22:38:52.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.630 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.630 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:52.630 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200009+0000", 2024-09-14T22:38:52.630 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12550539899999999 2024-09-14T22:38:52.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.630 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.630 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:52.630 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200108+0000", 2024-09-14T22:38:52.630 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9182e-05 2024-09-14T22:38:52.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:52.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200229+0000", 2024-09-14T22:38:52.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120538 2024-09-14T22:38:52.631 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.631 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:52.631 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.631 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97211 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 925696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:52.631 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:38:52.631 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 102.501809452, 2024-09-14T22:38:52.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12599848299999999, 2024-09-14T22:38:52.631 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:52.631 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:52.632 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:52.632 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:52.632 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:52.632 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:52.632 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97211 2024-09-14T22:38:52.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.632 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:52.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.632 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:52.632 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:38:52.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.633 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:52.633 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:38:52.633 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.633 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:52.633 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074516+0000", 2024-09-14T22:38:52.633 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.42e-06 2024-09-14T22:38:52.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.633 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:52.633 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074521+0000", 2024-09-14T22:38:52.633 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7960000000000003e-06 2024-09-14T22:38:52.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.634 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:52.634 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074538+0000", 2024-09-14T22:38:52.634 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6900000000000001e-05 2024-09-14T22:38:52.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.634 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.634 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:52.634 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074555+0000", 2024-09-14T22:38:52.634 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.696e-05 2024-09-14T22:38:52.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.634 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.634 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:52.634 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200293+0000", 2024-09-14T22:38:52.634 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12573864100000001 2024-09-14T22:38:52.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.635 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.635 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:52.635 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200393+0000", 2024-09-14T22:38:52.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9264000000000004e-05 2024-09-14T22:38:52.635 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.635 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.635 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:52.635 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200513+0000", 2024-09-14T22:38:52.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012050200000000001 2024-09-14T22:38:52.635 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.635 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:52.635 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.636 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.636 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97212 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 934912~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:52.636 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:38:52.636 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 102.50173483, 2024-09-14T22:38:52.636 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12621127300000001, 2024-09-14T22:38:52.636 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:52.636 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:52.636 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:52.636 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:52.636 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:52.636 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:52.636 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97212 2024-09-14T22:38:52.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.636 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:52.637 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.637 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:52.637 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:38:52.637 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.637 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.637 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:52.637 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:38:52.637 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.637 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.637 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:52.637 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074590+0000", 2024-09-14T22:38:52.637 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1969999999999999e-06 2024-09-14T22:38:52.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:52.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074596+0000", 2024-09-14T22:38:52.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7629999999999997e-06 2024-09-14T22:38:52.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:52.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074614+0000", 2024-09-14T22:38:52.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7643e-05 2024-09-14T22:38:52.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.639 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:52.639 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074635+0000", 2024-09-14T22:38:52.639 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.075e-05 2024-09-14T22:38:52.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.639 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.639 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:52.639 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200580+0000", 2024-09-14T22:38:52.639 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125945482 2024-09-14T22:38:52.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.639 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.639 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:52.639 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200683+0000", 2024-09-14T22:38:52.639 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010267099999999999 2024-09-14T22:38:52.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.640 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.640 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:52.640 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200800+0000", 2024-09-14T22:38:52.640 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117767 2024-09-14T22:38:52.640 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.640 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:52.640 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.640 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.640 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.640 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97213 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 944128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:52.640 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:38:52.640 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 102.50166948, 2024-09-14T22:38:52.640 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126430709, 2024-09-14T22:38:52.640 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:52.641 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:52.641 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:52.641 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:52.641 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:52.641 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:52.641 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97213 2024-09-14T22:38:52.641 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.641 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:52.641 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.641 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:52.641 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:38:52.641 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.642 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.642 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:52.642 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:38:52.642 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.642 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.642 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:52.642 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078210+0000", 2024-09-14T22:38:52.642 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0035556839999999999 2024-09-14T22:38:52.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.642 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.642 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:52.642 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078231+0000", 2024-09-14T22:38:52.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0716e-05 2024-09-14T22:38:52.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.643 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:52.643 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078260+0000", 2024-09-14T22:38:52.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9473000000000001e-05 2024-09-14T22:38:52.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.643 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:52.643 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078279+0000", 2024-09-14T22:38:52.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8600000000000001e-05 2024-09-14T22:38:52.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.643 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:52.644 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200864+0000", 2024-09-14T22:38:52.644 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.122585295 2024-09-14T22:38:52.644 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.644 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.644 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:52.644 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200965+0000", 2024-09-14T22:38:52.644 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101004 2024-09-14T22:38:52.644 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.644 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.644 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:52.644 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.201085+0000", 2024-09-14T22:38:52.644 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119937 2024-09-14T22:38:52.645 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.645 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:52.645 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.645 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.645 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.645 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97219 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 999424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:52.645 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:38:52.645 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 102.497687203, 2024-09-14T22:38:52.645 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124138558, 2024-09-14T22:38:52.645 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:52.645 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:52.645 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:52.645 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:52.645 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:52.646 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:52.646 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97219 2024-09-14T22:38:52.646 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.646 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:52.646 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.646 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:52.646 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:38:52.646 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.646 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.646 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.646 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:52.646 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:38:52.646 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.647 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:52.647 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078638+0000", 2024-09-14T22:38:52.647 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.173e-06 2024-09-14T22:38:52.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.647 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:52.647 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078644+0000", 2024-09-14T22:38:52.647 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2570000000000004e-06 2024-09-14T22:38:52.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.647 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:52.648 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078672+0000", 2024-09-14T22:38:52.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7322999999999999e-05 2024-09-14T22:38:52.648 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.648 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.648 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:52.648 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078687+0000", 2024-09-14T22:38:52.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5639999999999999e-05 2024-09-14T22:38:52.648 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.648 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.648 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:52.648 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202567+0000", 2024-09-14T22:38:52.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123880134 2024-09-14T22:38:52.648 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.648 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.649 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:52.649 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202670+0000", 2024-09-14T22:38:52.649 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010273899999999999 2024-09-14T22:38:52.649 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.649 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.649 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:52.649 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202775+0000", 2024-09-14T22:38:52.649 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010529199999999999 2024-09-14T22:38:52.649 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.649 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:52.649 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.649 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.649 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.650 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97220 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1008640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:52.650 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:38:52.650 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 102.49761796999999, 2024-09-14T22:38:52.650 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12435937, 2024-09-14T22:38:52.650 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:52.650 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:52.650 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:52.650 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:52.650 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:52.650 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:52.650 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97220 2024-09-14T22:38:52.650 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.650 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:52.650 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.651 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:52.651 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:38:52.651 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.651 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.651 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.651 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:52.651 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:38:52.651 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.651 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.651 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.651 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:52.651 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078707+0000", 2024-09-14T22:38:52.651 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.266e-06 2024-09-14T22:38:52.651 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.652 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:52.652 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078713+0000", 2024-09-14T22:38:52.652 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7270000000000004e-06 2024-09-14T22:38:52.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.652 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:52.652 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078737+0000", 2024-09-14T22:38:52.652 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3640000000000001e-05 2024-09-14T22:38:52.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.652 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:52.652 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078751+0000", 2024-09-14T22:38:52.652 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.45e-05 2024-09-14T22:38:52.653 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.653 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.653 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:52.653 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202841+0000", 2024-09-14T22:38:52.653 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124089938 2024-09-14T22:38:52.653 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.653 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.653 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:52.653 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202944+0000", 2024-09-14T22:38:52.653 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010334 2024-09-14T22:38:52.653 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.653 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.653 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:52.654 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203065+0000", 2024-09-14T22:38:52.654 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012095900000000001 2024-09-14T22:38:52.654 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.654 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:52.654 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.654 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.654 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.654 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97221 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1017856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:52.654 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:38:52.654 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 102.497552207, 2024-09-14T22:38:52.654 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124574005, 2024-09-14T22:38:52.654 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:52.654 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:52.654 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:52.655 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:52.655 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:52.655 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:52.655 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97221 2024-09-14T22:38:52.655 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.655 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:52.655 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.655 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:52.655 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:38:52.655 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.655 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.655 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.655 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:52.656 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:38:52.656 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.656 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.656 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.656 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:52.656 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078773+0000", 2024-09-14T22:38:52.656 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.13e-06 2024-09-14T22:38:52.656 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.656 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.656 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:52.656 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078779+0000", 2024-09-14T22:38:52.656 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0260000000000004e-06 2024-09-14T22:38:52.656 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.657 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.657 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:52.657 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078797+0000", 2024-09-14T22:38:52.657 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.806e-05 2024-09-14T22:38:52.657 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.657 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.657 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:52.657 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078810+0000", 2024-09-14T22:38:52.657 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.308e-05 2024-09-14T22:38:52.657 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.657 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.657 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:52.657 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203131+0000", 2024-09-14T22:38:52.657 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12432077499999999 2024-09-14T22:38:52.658 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.658 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.658 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:52.658 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203227+0000", 2024-09-14T22:38:52.658 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6575000000000003e-05 2024-09-14T22:38:52.658 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.658 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.658 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:52.658 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203346+0000", 2024-09-14T22:38:52.658 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118359 2024-09-14T22:38:52.658 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.658 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:52.658 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.658 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.658 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.659 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97222 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:52.659 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:38:52.659 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 102.49749583800001, 2024-09-14T22:38:52.659 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12480110799999999, 2024-09-14T22:38:52.659 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:52.659 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:52.659 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:52.659 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:52.659 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:52.659 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:52.659 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97222 2024-09-14T22:38:52.659 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.659 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:52.659 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.659 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:52.660 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:38:52.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.660 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:52.660 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:38:52.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.660 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:52.660 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078829+0000", 2024-09-14T22:38:52.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.057e-06 2024-09-14T22:38:52.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.660 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:52.661 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078836+0000", 2024-09-14T22:38:52.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6159999999999998e-06 2024-09-14T22:38:52.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.661 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:52.661 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078856+0000", 2024-09-14T22:38:52.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0100000000000001e-05 2024-09-14T22:38:52.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.661 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:52.661 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078869+0000", 2024-09-14T22:38:52.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.272e-05 2024-09-14T22:38:52.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:52.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203412+0000", 2024-09-14T22:38:52.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12454283300000001 2024-09-14T22:38:52.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:52.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203511+0000", 2024-09-14T22:38:52.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9893000000000001e-05 2024-09-14T22:38:52.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:52.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203629+0000", 2024-09-14T22:38:52.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117889 2024-09-14T22:38:52.663 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.663 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:52.663 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.663 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97223 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1036288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:52.663 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:38:52.663 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 102.497123851, 2024-09-14T22:38:52.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124706279, 2024-09-14T22:38:52.663 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:52.663 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:52.663 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:52.663 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:52.664 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:52.664 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:52.664 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97223 2024-09-14T22:38:52.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.664 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:52.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.664 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:52.664 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:38:52.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.664 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:52.664 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:38:52.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:52.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079202+0000", 2024-09-14T22:38:52.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8330000000000001e-06 2024-09-14T22:38:52.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:52.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079212+0000", 2024-09-14T22:38:52.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0207000000000001e-05 2024-09-14T22:38:52.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:52.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079238+0000", 2024-09-14T22:38:52.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5626000000000002e-05 2024-09-14T22:38:52.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:52.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079256+0000", 2024-09-14T22:38:52.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8386999999999999e-05 2024-09-14T22:38:52.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:52.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203691+0000", 2024-09-14T22:38:52.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124434538 2024-09-14T22:38:52.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:52.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203786+0000", 2024-09-14T22:38:52.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5344000000000001e-05 2024-09-14T22:38:52.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:52.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203906+0000", 2024-09-14T22:38:52.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120344 2024-09-14T22:38:52.667 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.667 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:52.667 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.667 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97224 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1045504~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:38:52.667 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:38:52.667 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 102.497040031, 2024-09-14T22:38:52.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12513321199999999, 2024-09-14T22:38:52.668 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:52.668 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:52.668 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:52.668 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:52.668 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:52.668 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:52.668 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97224 2024-09-14T22:38:52.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.668 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:52.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:52.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:38:52.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:52.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:38:52.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:52.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:52.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082581+0000", 2024-09-14T22:38:52.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0032971530000000001 2024-09-14T22:38:52.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:52.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082592+0000", 2024-09-14T22:38:52.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1345999999999999e-05 2024-09-14T22:38:52.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:52.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082618+0000", 2024-09-14T22:38:52.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5873000000000001e-05 2024-09-14T22:38:52.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:52.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082638+0000", 2024-09-14T22:38:52.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9565999999999999e-05 2024-09-14T22:38:52.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:52.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204004+0000", 2024-09-14T22:38:52.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.121366038 2024-09-14T22:38:52.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:52.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204188+0000", 2024-09-14T22:38:52.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018427 2024-09-14T22:38:52.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:52.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:52.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:52.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204417+0000", 2024-09-14T22:38:52.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022896599999999999 2024-09-14T22:38:52.671 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.671 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:52.671 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.671 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:52.671 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:52.671 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:52.673 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:38:52.673 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:52.666+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:52.760 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T22:38:52.773 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:52.766+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:52.832 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:52.833 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:52.833 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:38:52.833 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:52.846 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:38:52.874 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:52.872+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:52.976 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:52.970+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:53.018 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:53.018 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:53.018 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:38:53.018 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:38:53.018 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:53.033 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:38:53.034 INFO:teuthology.orchestra.run.smithi176.stdout:3.f 2024-09-14T22:38:53.034 INFO:teuthology.orchestra.run.smithi176.stdout:3.e 2024-09-14T22:38:53.034 INFO:teuthology.orchestra.run.smithi176.stdout:3.9 2024-09-14T22:38:53.034 INFO:teuthology.orchestra.run.smithi176.stdout:3.3 2024-09-14T22:38:53.034 INFO:teuthology.orchestra.run.smithi176.stdout:3.1c 2024-09-14T22:38:53.034 INFO:teuthology.orchestra.run.smithi176.stdout:3.1a 2024-09-14T22:38:53.034 INFO:teuthology.orchestra.run.smithi176.stdout:3.1 2024-09-14T22:38:53.034 INFO:teuthology.orchestra.run.smithi176.stdout:3.13 2024-09-14T22:38:53.034 INFO:teuthology.orchestra.run.smithi176.stdout:3.11 2024-09-14T22:38:53.035 INFO:teuthology.orchestra.run.smithi176.stdout:3.a 2024-09-14T22:38:53.035 INFO:teuthology.orchestra.run.smithi176.stdout:3.1b 2024-09-14T22:38:53.035 INFO:teuthology.orchestra.run.smithi176.stdout:3.14 2024-09-14T22:38:53.035 INFO:teuthology.orchestra.run.smithi176.stdout:2.0 2024-09-14T22:38:53.076 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:53.072+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:53.177 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:53.170+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:53.199 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:53.199 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:38:53.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:38:53.200 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:38:53.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.200 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:72252 3.2 3:442c6d72:::rbd_data.113cb4c83968.0000000000000015:head [zero 4456448~393216] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-14T22:38:53.200 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:38:53.200 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 122.495277212, 2024-09-14T22:38:53.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0176591500000001, 2024-09-14T22:38:53.200 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.200 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.200 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T22:38:53.200 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.200 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.200 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.201 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 72252 2024-09-14T22:38:53.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.201 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:38:53.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:38:53.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700101+0000", 2024-09-14T22:38:53.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4570000000000001e-06 2024-09-14T22:38:53.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700114+0000", 2024-09-14T22:38:53.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2799999999999999e-05 2024-09-14T22:38:53.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700146+0000", 2024-09-14T22:38:53.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2116000000000002e-05 2024-09-14T22:38:53.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700170+0000", 2024-09-14T22:38:53.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3770000000000001e-05 2024-09-14T22:38:53.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700285+0000", 2024-09-14T22:38:53.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000115228 2024-09-14T22:38:53.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op must wait for map", 2024-09-14T22:38:53.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700297+0000", 2024-09-14T22:38:53.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1681999999999999e-05 2024-09-14T22:38:53.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.703312+0000", 2024-09-14T22:38:53.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003015046 2024-09-14T22:38:53.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:38:53.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.703327+0000", 2024-09-14T22:38:53.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5481999999999998e-05 2024-09-14T22:38:53.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:51.717566+0000", 2024-09-14T22:38:53.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0142383509999999 2024-09-14T22:38:53.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:51.717757+0000", 2024-09-14T22:38:53.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000191218 2024-09-14T22:38:53.205 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.205 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.205 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.205 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:74156 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T22:38:53.205 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:38:53.205 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 117.680808502, 2024-09-14T22:38:53.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062073521, 2024-09-14T22:38:53.206 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.206 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.206 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:53.206 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.206 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.206 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.206 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74156 2024-09-14T22:38:53.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.206 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:38:53.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:38:53.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514569+0000", 2024-09-14T22:38:53.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0229999999999999e-06 2024-09-14T22:38:53.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.208 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514596+0000", 2024-09-14T22:38:53.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6903e-05 2024-09-14T22:38:53.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.208 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514623+0000", 2024-09-14T22:38:53.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7206e-05 2024-09-14T22:38:53.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.208 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514637+0000", 2024-09-14T22:38:53.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:38:53.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514781+0000", 2024-09-14T22:38:53.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014394899999999999 2024-09-14T22:38:53.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.515005+0000", 2024-09-14T22:38:53.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022458499999999999 2024-09-14T22:38:53.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:38:53.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.527785+0000", 2024-09-14T22:38:53.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012780112 2024-09-14T22:38:53.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:38:53.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576209+0000", 2024-09-14T22:38:53.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.048423992999999999 2024-09-14T22:38:53.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:53.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576346+0000", 2024-09-14T22:38:53.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000136909 2024-09-14T22:38:53.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:38:53.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576517+0000", 2024-09-14T22:38:53.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000170848 2024-09-14T22:38:53.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576640+0000", 2024-09-14T22:38:53.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000122953 2024-09-14T22:38:53.211 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.211 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.211 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.211 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82513 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 557056~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:53.211 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:38:53.211 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.776176683, 2024-09-14T22:38:53.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059345486000000003, 2024-09-14T22:38:53.212 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.212 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.212 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.212 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.212 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.212 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.212 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82513 2024-09-14T22:38:53.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.212 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.212 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.212 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:38:53.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:38:53.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419200+0000", 2024-09-14T22:38:53.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.184e-06 2024-09-14T22:38:53.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419207+0000", 2024-09-14T22:38:53.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2690000000000002e-06 2024-09-14T22:38:53.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419224+0000", 2024-09-14T22:38:53.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6623e-05 2024-09-14T22:38:53.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419236+0000", 2024-09-14T22:38:53.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2367e-05 2024-09-14T22:38:53.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478294+0000", 2024-09-14T22:38:53.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059057927000000003 2024-09-14T22:38:53.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478402+0000", 2024-09-14T22:38:53.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107921 2024-09-14T22:38:53.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478544+0000", 2024-09-14T22:38:53.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014219499999999999 2024-09-14T22:38:53.215 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.215 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.215 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.216 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82514 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 704512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:53.216 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:38:53.216 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.77611758, 2024-09-14T22:38:53.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059580793, 2024-09-14T22:38:53.216 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.216 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.216 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.216 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.216 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.216 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.216 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82514 2024-09-14T22:38:53.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.216 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:38:53.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:38:53.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419259+0000", 2024-09-14T22:38:53.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.353e-06 2024-09-14T22:38:53.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419276+0000", 2024-09-14T22:38:53.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7567e-05 2024-09-14T22:38:53.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419294+0000", 2024-09-14T22:38:53.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.713e-05 2024-09-14T22:38:53.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419306+0000", 2024-09-14T22:38:53.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.279e-05 2024-09-14T22:38:53.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478589+0000", 2024-09-14T22:38:53.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059282978 2024-09-14T22:38:53.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478694+0000", 2024-09-14T22:38:53.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000104561 2024-09-14T22:38:53.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478838+0000", 2024-09-14T22:38:53.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014441399999999999 2024-09-14T22:38:53.220 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.220 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.220 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.220 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82515 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:53.220 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:38:53.220 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.77605242999999, 2024-09-14T22:38:53.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059815123999999997, 2024-09-14T22:38:53.220 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.221 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.221 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.221 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.221 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.221 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.221 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82515 2024-09-14T22:38:53.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.221 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:38:53.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:38:53.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419324+0000", 2024-09-14T22:38:53.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2130000000000001e-06 2024-09-14T22:38:53.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419331+0000", 2024-09-14T22:38:53.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4730000000000003e-06 2024-09-14T22:38:53.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419348+0000", 2024-09-14T22:38:53.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6269999999999998e-05 2024-09-14T22:38:53.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419360+0000", 2024-09-14T22:38:53.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2243e-05 2024-09-14T22:38:53.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478884+0000", 2024-09-14T22:38:53.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059524369000000001 2024-09-14T22:38:53.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478990+0000", 2024-09-14T22:38:53.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010569 2024-09-14T22:38:53.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479138+0000", 2024-09-14T22:38:53.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000147866 2024-09-14T22:38:53.224 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.224 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.224 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.225 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82516 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 999424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:53.225 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:38:53.225 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.77599992499999, 2024-09-14T22:38:53.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061979595999999998, 2024-09-14T22:38:53.225 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.225 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.225 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.225 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.225 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.225 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.226 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82516 2024-09-14T22:38:53.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.226 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:38:53.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:38:53.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419376+0000", 2024-09-14T22:38:53.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.237e-06 2024-09-14T22:38:53.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419383+0000", 2024-09-14T22:38:53.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8170000000000001e-06 2024-09-14T22:38:53.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419400+0000", 2024-09-14T22:38:53.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6636000000000001e-05 2024-09-14T22:38:53.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419412+0000", 2024-09-14T22:38:53.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2357e-05 2024-09-14T22:38:53.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481080+0000", 2024-09-14T22:38:53.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061668032999999997 2024-09-14T22:38:53.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481193+0000", 2024-09-14T22:38:53.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112335 2024-09-14T22:38:53.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481355+0000", 2024-09-14T22:38:53.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016218100000000001 2024-09-14T22:38:53.229 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.229 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.229 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.229 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82538 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1097728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:53.229 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:38:53.229 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.775947855, 2024-09-14T22:38:53.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059971705, 2024-09-14T22:38:53.230 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.230 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.230 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.230 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.230 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.230 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.230 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82538 2024-09-14T22:38:53.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.230 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:38:53.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:38:53.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419429+0000", 2024-09-14T22:38:53.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.497e-06 2024-09-14T22:38:53.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419436+0000", 2024-09-14T22:38:53.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2099999999999996e-06 2024-09-14T22:38:53.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419453+0000", 2024-09-14T22:38:53.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6840000000000001e-05 2024-09-14T22:38:53.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419465+0000", 2024-09-14T22:38:53.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2283000000000001e-05 2024-09-14T22:38:53.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479182+0000", 2024-09-14T22:38:53.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059717339000000001 2024-09-14T22:38:53.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479297+0000", 2024-09-14T22:38:53.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000114259 2024-09-14T22:38:53.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479399+0000", 2024-09-14T22:38:53.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102277 2024-09-14T22:38:53.233 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.233 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.234 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.234 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82539 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:53.234 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:38:53.234 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.775895005, 2024-09-14T22:38:53.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060166935999999997, 2024-09-14T22:38:53.234 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.234 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.234 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.234 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.234 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.234 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.234 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82539 2024-09-14T22:38:53.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.235 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:38:53.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:38:53.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419482+0000", 2024-09-14T22:38:53.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4130000000000001e-06 2024-09-14T22:38:53.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419488+0000", 2024-09-14T22:38:53.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8460000000000002e-06 2024-09-14T22:38:53.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419505+0000", 2024-09-14T22:38:53.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6477000000000002e-05 2024-09-14T22:38:53.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419518+0000", 2024-09-14T22:38:53.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2797e-05 2024-09-14T22:38:53.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479438+0000", 2024-09-14T22:38:53.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059919897 2024-09-14T22:38:53.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479547+0000", 2024-09-14T22:38:53.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109807 2024-09-14T22:38:53.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479647+0000", 2024-09-14T22:38:53.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9698999999999995e-05 2024-09-14T22:38:53.238 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.238 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.238 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.238 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82542 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:53.238 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:38:53.239 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.77584256199999, 2024-09-14T22:38:53.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060367430999999999, 2024-09-14T22:38:53.239 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.239 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.239 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.239 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.239 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.239 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.239 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82542 2024-09-14T22:38:53.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.239 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:38:53.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:38:53.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419534+0000", 2024-09-14T22:38:53.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T22:38:53.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419542+0000", 2024-09-14T22:38:53.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8199999999999997e-06 2024-09-14T22:38:53.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419559+0000", 2024-09-14T22:38:53.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6852999999999999e-05 2024-09-14T22:38:53.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419571+0000", 2024-09-14T22:38:53.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2364000000000001e-05 2024-09-14T22:38:53.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479686+0000", 2024-09-14T22:38:53.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060115522999999997 2024-09-14T22:38:53.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479795+0000", 2024-09-14T22:38:53.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010831399999999999 2024-09-14T22:38:53.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479900+0000", 2024-09-14T22:38:53.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105284 2024-09-14T22:38:53.243 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.243 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.243 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.243 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82544 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:53.243 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:38:53.243 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.775788826, 2024-09-14T22:38:53.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060590515999999997, 2024-09-14T22:38:53.243 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.243 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.243 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.244 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.244 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.244 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.244 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82544 2024-09-14T22:38:53.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.244 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.244 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:38:53.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:38:53.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419588+0000", 2024-09-14T22:38:53.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.35e-06 2024-09-14T22:38:53.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419595+0000", 2024-09-14T22:38:53.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4030000000000001e-06 2024-09-14T22:38:53.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419612+0000", 2024-09-14T22:38:53.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6670000000000001e-05 2024-09-14T22:38:53.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419624+0000", 2024-09-14T22:38:53.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-05 2024-09-14T22:38:53.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479957+0000", 2024-09-14T22:38:53.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060333287999999999 2024-09-14T22:38:53.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480074+0000", 2024-09-14T22:38:53.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116443 2024-09-14T22:38:53.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480177+0000", 2024-09-14T22:38:53.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000103102 2024-09-14T22:38:53.247 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.247 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.247 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.248 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82546 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:53.248 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:38:53.248 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.775735907, 2024-09-14T22:38:53.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060795428999999998, 2024-09-14T22:38:53.248 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.248 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.248 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.248 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.248 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.248 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.248 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82546 2024-09-14T22:38:53.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.248 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.249 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.249 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:38:53.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.249 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.249 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:38:53.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.249 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.249 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419640+0000", 2024-09-14T22:38:53.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.254e-06 2024-09-14T22:38:53.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419647+0000", 2024-09-14T22:38:53.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9e-06 2024-09-14T22:38:53.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419664+0000", 2024-09-14T22:38:53.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6345999999999999e-05 2024-09-14T22:38:53.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419676+0000", 2024-09-14T22:38:53.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.224e-05 2024-09-14T22:38:53.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480218+0000", 2024-09-14T22:38:53.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060542133999999997 2024-09-14T22:38:53.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480336+0000", 2024-09-14T22:38:53.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117472 2024-09-14T22:38:53.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480435+0000", 2024-09-14T22:38:53.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9083e-05 2024-09-14T22:38:53.252 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.252 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.252 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.252 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82548 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:53.252 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:38:53.252 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.77568389699999, 2024-09-14T22:38:53.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061011636000000001, 2024-09-14T22:38:53.252 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.252 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.252 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.252 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.253 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.253 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.253 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82548 2024-09-14T22:38:53.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.253 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:38:53.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:38:53.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.254 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.254 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419693+0000", 2024-09-14T22:38:53.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.387e-06 2024-09-14T22:38:53.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.254 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.254 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419699+0000", 2024-09-14T22:38:53.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8730000000000001e-06 2024-09-14T22:38:53.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.254 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.254 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419716+0000", 2024-09-14T22:38:53.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6416e-05 2024-09-14T22:38:53.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419728+0000", 2024-09-14T22:38:53.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2597e-05 2024-09-14T22:38:53.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480476+0000", 2024-09-14T22:38:53.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060747989000000002 2024-09-14T22:38:53.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480583+0000", 2024-09-14T22:38:53.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010638500000000001 2024-09-14T22:38:53.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480703+0000", 2024-09-14T22:38:53.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011998900000000001 2024-09-14T22:38:53.256 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.256 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.256 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.256 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82550 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1982464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:53.256 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:38:53.256 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.775631144, 2024-09-14T22:38:53.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061295483999999997, 2024-09-14T22:38:53.256 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.256 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.256 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.256 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.257 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.257 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.257 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82550 2024-09-14T22:38:53.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.257 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:38:53.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:38:53.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419745+0000", 2024-09-14T22:38:53.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.406e-06 2024-09-14T22:38:53.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419753+0000", 2024-09-14T22:38:53.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8140000000000001e-06 2024-09-14T22:38:53.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419770+0000", 2024-09-14T22:38:53.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6719999999999999e-05 2024-09-14T22:38:53.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419782+0000", 2024-09-14T22:38:53.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2252999999999999e-05 2024-09-14T22:38:53.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480745+0000", 2024-09-14T22:38:53.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060962385000000001 2024-09-14T22:38:53.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480900+0000", 2024-09-14T22:38:53.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015513899999999999 2024-09-14T22:38:53.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481039+0000", 2024-09-14T22:38:53.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013976700000000001 2024-09-14T22:38:53.260 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.260 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.260 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.260 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82588 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1064960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:53.260 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:38:53.260 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.775577254, 2024-09-14T22:38:53.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061863732999999997, 2024-09-14T22:38:53.261 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.261 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.261 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.261 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.261 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.261 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.261 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82588 2024-09-14T22:38:53.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.261 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.261 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:38:53.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:38:53.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419799+0000", 2024-09-14T22:38:53.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4160000000000001e-06 2024-09-14T22:38:53.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419807+0000", 2024-09-14T22:38:53.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2169999999999994e-06 2024-09-14T22:38:53.263 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.263 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.263 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.263 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419824+0000", 2024-09-14T22:38:53.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6776e-05 2024-09-14T22:38:53.263 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.263 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.263 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.263 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419837+0000", 2024-09-14T22:38:53.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2680000000000001e-05 2024-09-14T22:38:53.263 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.263 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.263 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481395+0000", 2024-09-14T22:38:53.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061558416999999997 2024-09-14T22:38:53.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481532+0000", 2024-09-14T22:38:53.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000136861 2024-09-14T22:38:53.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481662+0000", 2024-09-14T22:38:53.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012936600000000001 2024-09-14T22:38:53.264 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.265 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.265 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.265 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82589 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:53.265 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:38:53.265 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.775522659, 2024-09-14T22:38:53.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062064592000000002, 2024-09-14T22:38:53.265 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.265 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.265 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.265 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.265 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.266 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.266 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82589 2024-09-14T22:38:53.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.266 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:38:53.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:38:53.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419854+0000", 2024-09-14T22:38:53.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.367e-06 2024-09-14T22:38:53.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419862+0000", 2024-09-14T22:38:53.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8930000000000005e-06 2024-09-14T22:38:53.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419896+0000", 2024-09-14T22:38:53.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4134000000000002e-05 2024-09-14T22:38:53.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.268 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.268 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419909+0000", 2024-09-14T22:38:53.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3332999999999999e-05 2024-09-14T22:38:53.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.268 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.268 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481702+0000", 2024-09-14T22:38:53.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061792947000000001 2024-09-14T22:38:53.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.268 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.268 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481808+0000", 2024-09-14T22:38:53.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105655 2024-09-14T22:38:53.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481917+0000", 2024-09-14T22:38:53.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109263 2024-09-14T22:38:53.269 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.269 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.269 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.269 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82590 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1359872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:53.269 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:38:53.269 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.775449052, 2024-09-14T22:38:53.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062362895000000002, 2024-09-14T22:38:53.270 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.270 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.270 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.270 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.270 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.270 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.270 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82590 2024-09-14T22:38:53.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.270 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:38:53.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:38:53.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419928+0000", 2024-09-14T22:38:53.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4470000000000001e-06 2024-09-14T22:38:53.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.272 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419942+0000", 2024-09-14T22:38:53.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4229e-05 2024-09-14T22:38:53.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.272 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.272 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.272 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419972+0000", 2024-09-14T22:38:53.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.065e-05 2024-09-14T22:38:53.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.272 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.272 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.272 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419985+0000", 2024-09-14T22:38:53.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2903e-05 2024-09-14T22:38:53.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481960+0000", 2024-09-14T22:38:53.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061974435000000001 2024-09-14T22:38:53.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482106+0000", 2024-09-14T22:38:53.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014643800000000001 2024-09-14T22:38:53.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482289+0000", 2024-09-14T22:38:53.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000182793 2024-09-14T22:38:53.274 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.274 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.274 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.274 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82592 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:53.274 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:38:53.274 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.77537282999999, 2024-09-14T22:38:53.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062631507000000003, 2024-09-14T22:38:53.274 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.274 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.274 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.274 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.274 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.275 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.275 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82592 2024-09-14T22:38:53.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.275 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:38:53.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:38:53.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420017+0000", 2024-09-14T22:38:53.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4744e-05 2024-09-14T22:38:53.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420036+0000", 2024-09-14T22:38:53.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8680000000000001e-05 2024-09-14T22:38:53.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420054+0000", 2024-09-14T22:38:53.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7799999999999999e-05 2024-09-14T22:38:53.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420067+0000", 2024-09-14T22:38:53.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3013000000000001e-05 2024-09-14T22:38:53.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482358+0000", 2024-09-14T22:38:53.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062291445000000001 2024-09-14T22:38:53.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482469+0000", 2024-09-14T22:38:53.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110608 2024-09-14T22:38:53.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482634+0000", 2024-09-14T22:38:53.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000165217 2024-09-14T22:38:53.278 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.278 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.278 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.278 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82594 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1654784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:53.279 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:38:53.279 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.775274854, 2024-09-14T22:38:53.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062893849000000002, 2024-09-14T22:38:53.279 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.279 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.279 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.279 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.279 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.279 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.279 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82594 2024-09-14T22:38:53.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.279 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.280 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:38:53.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.280 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.280 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:38:53.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.280 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.280 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420102+0000", 2024-09-14T22:38:53.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5e-06 2024-09-14T22:38:53.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420115+0000", 2024-09-14T22:38:53.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.324e-05 2024-09-14T22:38:53.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420133+0000", 2024-09-14T22:38:53.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8213e-05 2024-09-14T22:38:53.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420147+0000", 2024-09-14T22:38:53.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3284e-05 2024-09-14T22:38:53.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482688+0000", 2024-09-14T22:38:53.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062541425999999997 2024-09-14T22:38:53.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482794+0000", 2024-09-14T22:38:53.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010575500000000001 2024-09-14T22:38:53.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482994+0000", 2024-09-14T22:38:53.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020043099999999999 2024-09-14T22:38:53.282 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.283 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.283 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.283 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82596 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:53.283 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:38:53.283 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.77521155399999, 2024-09-14T22:38:53.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063187238000000007, 2024-09-14T22:38:53.283 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.283 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.283 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.283 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.283 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.283 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.283 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82596 2024-09-14T22:38:53.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.284 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:38:53.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:38:53.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420165+0000", 2024-09-14T22:38:53.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4100000000000001e-06 2024-09-14T22:38:53.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.285 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.285 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420173+0000", 2024-09-14T22:38:53.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8129999999999996e-06 2024-09-14T22:38:53.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.285 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.285 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420190+0000", 2024-09-14T22:38:53.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7076999999999999e-05 2024-09-14T22:38:53.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.285 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420203+0000", 2024-09-14T22:38:53.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2683e-05 2024-09-14T22:38:53.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483060+0000", 2024-09-14T22:38:53.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062857114000000006 2024-09-14T22:38:53.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483167+0000", 2024-09-14T22:38:53.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010713999999999999 2024-09-14T22:38:53.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483351+0000", 2024-09-14T22:38:53.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018400099999999999 2024-09-14T22:38:53.287 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.287 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.287 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.287 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82597 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1949696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:38:53.287 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:38:53.287 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.775154728, 2024-09-14T22:38:53.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063466513000000002, 2024-09-14T22:38:53.287 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.287 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.287 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.287 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.287 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.287 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.288 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82597 2024-09-14T22:38:53.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.288 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:38:53.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:38:53.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420221+0000", 2024-09-14T22:38:53.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.043e-06 2024-09-14T22:38:53.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420229+0000", 2024-09-14T22:38:53.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0469999999999994e-06 2024-09-14T22:38:53.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420246+0000", 2024-09-14T22:38:53.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6339999999999999e-05 2024-09-14T22:38:53.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420258+0000", 2024-09-14T22:38:53.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2296000000000001e-05 2024-09-14T22:38:53.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483402+0000", 2024-09-14T22:38:53.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063144318000000005 2024-09-14T22:38:53.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483530+0000", 2024-09-14T22:38:53.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012713199999999999 2024-09-14T22:38:53.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483687+0000", 2024-09-14T22:38:53.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015733699999999999 2024-09-14T22:38:53.291 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.291 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.291 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.291 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.291 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.291 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:53.293 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:38:53.293 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:53.288+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:53.393 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:53.386+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:53.461 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:53.461 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:53.461 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:38:53.461 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:53.475 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:38:53.494 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:53.492+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:53.595 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:53.590+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:53.601 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --op export --pgid 3.9 --file /var/log/ceph/exp.3.9.4 2024-09-14T22:38:53.646 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:53.646 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:53.647 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:38:53.647 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:38:53.647 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:53.661 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:38:53.696 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:53.692+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:53.796 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:53.790+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:53.837 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:53.837 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:38:53.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:38:53.838 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:38:53.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.838 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:38:53.838 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:38:53.838 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 173.82706006199999, 2024-09-14T22:38:53.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.398029152, 2024-09-14T22:38:53.838 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.838 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.838 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:53.838 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.838 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:38:53.838 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:38:53.838 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 1 2024-09-14T22:38:53.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.839 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:38:53.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:38:53.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006183+0000", 2024-09-14T22:38:53.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9059999999999998e-06 2024-09-14T22:38:53.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006198+0000", 2024-09-14T22:38:53.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.523e-05 2024-09-14T22:38:53.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006219+0000", 2024-09-14T22:38:53.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1342999999999999e-05 2024-09-14T22:38:53.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006240+0000", 2024-09-14T22:38:53.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1186999999999999e-05 2024-09-14T22:38:53.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006363+0000", 2024-09-14T22:38:53.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012258200000000001 2024-09-14T22:38:53.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:38:53.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006392+0000", 2024-09-14T22:38:53.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9607e-05 2024-09-14T22:38:53.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401084+0000", 2024-09-14T22:38:53.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.39469142699999998 2024-09-14T22:38:53.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401394+0000", 2024-09-14T22:38:53.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030987799999999998 2024-09-14T22:38:53.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:38:53.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401860+0000", 2024-09-14T22:38:53.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00046654299999999998 2024-09-14T22:38:53.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:38:53.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.403386+0000", 2024-09-14T22:38:53.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001525322 2024-09-14T22:38:53.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:53.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404042+0000", 2024-09-14T22:38:53.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065676099999999996 2024-09-14T22:38:53.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.844 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:38:53.844 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404136+0000", 2024-09-14T22:38:53.844 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4032999999999999e-05 2024-09-14T22:38:53.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.844 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.844 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404209+0000", 2024-09-14T22:38:53.844 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2333e-05 2024-09-14T22:38:53.844 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.844 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.844 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.844 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104182 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14352384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:53.845 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:38:53.845 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.344206076000006, 2024-09-14T22:38:53.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071642496999999999, 2024-09-14T22:38:53.845 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.845 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.845 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.845 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.845 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.845 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.845 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104182 2024-09-14T22:38:53.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.845 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:38:53.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:38:53.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489035+0000", 2024-09-14T22:38:53.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3430000000000001e-06 2024-09-14T22:38:53.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489041+0000", 2024-09-14T22:38:53.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8139999999999997e-06 2024-09-14T22:38:53.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489059+0000", 2024-09-14T22:38:53.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8193e-05 2024-09-14T22:38:53.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489073+0000", 2024-09-14T22:38:53.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:38:53.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560434+0000", 2024-09-14T22:38:53.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071361463999999999 2024-09-14T22:38:53.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560545+0000", 2024-09-14T22:38:53.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110263 2024-09-14T22:38:53.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.850 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560676+0000", 2024-09-14T22:38:53.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013138000000000001 2024-09-14T22:38:53.850 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.850 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.850 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.850 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104183 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14483456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:53.850 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:38:53.850 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.344082756999995, 2024-09-14T22:38:53.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071781943000000001, 2024-09-14T22:38:53.851 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.851 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.851 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.851 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.851 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.851 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.851 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104183 2024-09-14T22:38:53.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.852 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.852 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:38:53.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.852 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:38:53.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489159+0000", 2024-09-14T22:38:53.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5629999999999999e-06 2024-09-14T22:38:53.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489165+0000", 2024-09-14T22:38:53.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7070000000000003e-06 2024-09-14T22:38:53.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489186+0000", 2024-09-14T22:38:53.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.088e-05 2024-09-14T22:38:53.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489201+0000", 2024-09-14T22:38:53.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5203e-05 2024-09-14T22:38:53.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560713+0000", 2024-09-14T22:38:53.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071511447000000006 2024-09-14T22:38:53.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560819+0000", 2024-09-14T22:38:53.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106579 2024-09-14T22:38:53.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560939+0000", 2024-09-14T22:38:53.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119564 2024-09-14T22:38:53.855 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.855 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.855 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.856 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104184 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:53.856 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:38:53.856 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.344018313999996, 2024-09-14T22:38:53.856 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071970107000000005, 2024-09-14T22:38:53.856 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.856 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.856 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.856 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.856 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.856 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.856 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104184 2024-09-14T22:38:53.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.856 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.856 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.857 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:38:53.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.857 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:38:53.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.857 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489223+0000", 2024-09-14T22:38:53.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4470000000000001e-06 2024-09-14T22:38:53.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489229+0000", 2024-09-14T22:38:53.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8200000000000002e-06 2024-09-14T22:38:53.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489247+0000", 2024-09-14T22:38:53.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8150000000000001e-05 2024-09-14T22:38:53.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489260+0000", 2024-09-14T22:38:53.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3570000000000001e-05 2024-09-14T22:38:53.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.859 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560979+0000", 2024-09-14T22:38:53.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071718831999999996 2024-09-14T22:38:53.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.859 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561075+0000", 2024-09-14T22:38:53.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5848999999999997e-05 2024-09-14T22:38:53.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.859 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561191+0000", 2024-09-14T22:38:53.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116439 2024-09-14T22:38:53.859 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.860 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.860 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.860 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104185 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14745600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:53.860 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:38:53.860 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.343958731000001, 2024-09-14T22:38:53.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072173750999999994, 2024-09-14T22:38:53.860 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.860 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.860 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.860 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.860 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.860 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.861 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104185 2024-09-14T22:38:53.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.861 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:38:53.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:38:53.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489282+0000", 2024-09-14T22:38:53.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2500000000000001e-06 2024-09-14T22:38:53.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489287+0000", 2024-09-14T22:38:53.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1359999999999996e-06 2024-09-14T22:38:53.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489304+0000", 2024-09-14T22:38:53.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7033000000000001e-05 2024-09-14T22:38:53.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489318+0000", 2024-09-14T22:38:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3934000000000001e-05 2024-09-14T22:38:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561232+0000", 2024-09-14T22:38:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071913759999999993 2024-09-14T22:38:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561340+0000", 2024-09-14T22:38:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107921 2024-09-14T22:38:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561455+0000", 2024-09-14T22:38:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011471700000000001 2024-09-14T22:38:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104186 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14876672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:38:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.343902822000004, 2024-09-14T22:38:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072393374999999996, 2024-09-14T22:38:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104186 2024-09-14T22:38:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:38:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:38:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489338+0000", 2024-09-14T22:38:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.48e-06 2024-09-14T22:38:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489366+0000", 2024-09-14T22:38:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7307000000000001e-05 2024-09-14T22:38:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489385+0000", 2024-09-14T22:38:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9283000000000002e-05 2024-09-14T22:38:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489399+0000", 2024-09-14T22:38:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.452e-05 2024-09-14T22:38:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561494+0000", 2024-09-14T22:38:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072094885999999997 2024-09-14T22:38:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561604+0000", 2024-09-14T22:38:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010925900000000001 2024-09-14T22:38:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561730+0000", 2024-09-14T22:38:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012663999999999999 2024-09-14T22:38:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104187 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15007744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:38:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.343819452999995, 2024-09-14T22:38:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072599143000000005, 2024-09-14T22:38:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104187 2024-09-14T22:38:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:38:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:38:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489422+0000", 2024-09-14T22:38:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5340000000000001e-06 2024-09-14T22:38:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489428+0000", 2024-09-14T22:38:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1430000000000003e-06 2024-09-14T22:38:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489445+0000", 2024-09-14T22:38:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7042999999999999e-05 2024-09-14T22:38:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489458+0000", 2024-09-14T22:38:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3474e-05 2024-09-14T22:38:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561771+0000", 2024-09-14T22:38:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072312600000000005 2024-09-14T22:38:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561885+0000", 2024-09-14T22:38:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011411799999999999 2024-09-14T22:38:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562019+0000", 2024-09-14T22:38:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013423100000000001 2024-09-14T22:38:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104188 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15138816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:38:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.343759528999996, 2024-09-14T22:38:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072824624000000004, 2024-09-14T22:38:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104188 2024-09-14T22:38:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:38:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:38:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489481+0000", 2024-09-14T22:38:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.22e-06 2024-09-14T22:38:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489487+0000", 2024-09-14T22:38:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6930000000000004e-06 2024-09-14T22:38:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489508+0000", 2024-09-14T22:38:53.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1135999999999999e-05 2024-09-14T22:38:53.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489522+0000", 2024-09-14T22:38:53.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3817e-05 2024-09-14T22:38:53.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562066+0000", 2024-09-14T22:38:53.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072543813999999998 2024-09-14T22:38:53.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562177+0000", 2024-09-14T22:38:53.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000111264 2024-09-14T22:38:53.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562305+0000", 2024-09-14T22:38:53.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012768 2024-09-14T22:38:53.877 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.877 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.877 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.877 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104189 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15269888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:53.877 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:38:53.878 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.343699079999993, 2024-09-14T22:38:53.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073061300999999995, 2024-09-14T22:38:53.878 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.878 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.878 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.878 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.878 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.878 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.878 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104189 2024-09-14T22:38:53.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.878 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:38:53.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.879 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.879 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:38:53.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.879 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.879 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489542+0000", 2024-09-14T22:38:53.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1659999999999999e-06 2024-09-14T22:38:53.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.879 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489561+0000", 2024-09-14T22:38:53.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.908e-05 2024-09-14T22:38:53.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489579+0000", 2024-09-14T22:38:53.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8536999999999999e-05 2024-09-14T22:38:53.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489593+0000", 2024-09-14T22:38:53.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3977e-05 2024-09-14T22:38:53.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.881 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562347+0000", 2024-09-14T22:38:53.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072754092000000006 2024-09-14T22:38:53.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.881 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562458+0000", 2024-09-14T22:38:53.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110722 2024-09-14T22:38:53.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.881 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562602+0000", 2024-09-14T22:38:53.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014372699999999999 2024-09-14T22:38:53.882 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.882 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.882 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.882 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104190 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:53.882 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:38:53.882 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.343626783999994, 2024-09-14T22:38:53.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.07326133, 2024-09-14T22:38:53.882 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.882 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.882 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.882 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.882 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.883 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.883 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104190 2024-09-14T22:38:53.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.883 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.883 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.883 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:38:53.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.883 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.884 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:38:53.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.884 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.884 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489614+0000", 2024-09-14T22:38:53.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1999999999999999e-06 2024-09-14T22:38:53.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.884 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.884 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489620+0000", 2024-09-14T22:38:53.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.524e-06 2024-09-14T22:38:53.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489637+0000", 2024-09-14T22:38:53.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7825999999999998e-05 2024-09-14T22:38:53.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489671+0000", 2024-09-14T22:38:53.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3726000000000002e-05 2024-09-14T22:38:53.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562644+0000", 2024-09-14T22:38:53.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072973089000000005 2024-09-14T22:38:53.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562762+0000", 2024-09-14T22:38:53.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011743600000000001 2024-09-14T22:38:53.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562874+0000", 2024-09-14T22:38:53.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011252900000000001 2024-09-14T22:38:53.886 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.886 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.886 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.887 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104191 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15532032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:53.887 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:38:53.887 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.343518360999994, 2024-09-14T22:38:53.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073311166999999997, 2024-09-14T22:38:53.887 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.887 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.887 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.887 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.887 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.887 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.887 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104191 2024-09-14T22:38:53.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.887 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.887 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.888 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:38:53.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.888 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.888 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.888 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.888 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:38:53.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.888 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.888 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.888 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.888 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489723+0000", 2024-09-14T22:38:53.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6330000000000001e-06 2024-09-14T22:38:53.888 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.889 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489730+0000", 2024-09-14T22:38:53.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2300000000000002e-06 2024-09-14T22:38:53.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.889 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489751+0000", 2024-09-14T22:38:53.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0849999999999999e-05 2024-09-14T22:38:53.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.889 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489766+0000", 2024-09-14T22:38:53.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.524e-05 2024-09-14T22:38:53.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562896+0000", 2024-09-14T22:38:53.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073129662999999998 2024-09-14T22:38:53.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562960+0000", 2024-09-14T22:38:53.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3689999999999995e-05 2024-09-14T22:38:53.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563033+0000", 2024-09-14T22:38:53.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2861000000000006e-05 2024-09-14T22:38:53.891 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.891 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.891 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.891 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104192 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:53.891 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:38:53.891 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.343422555999993, 2024-09-14T22:38:53.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073463632000000001, 2024-09-14T22:38:53.891 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.891 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.891 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.891 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.891 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.891 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.892 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104192 2024-09-14T22:38:53.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.892 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.892 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.892 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:38:53.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.892 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.892 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:38:53.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489819+0000", 2024-09-14T22:38:53.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.424e-06 2024-09-14T22:38:53.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489825+0000", 2024-09-14T22:38:53.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.02e-06 2024-09-14T22:38:53.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489844+0000", 2024-09-14T22:38:53.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9760000000000001e-05 2024-09-14T22:38:53.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.894 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.894 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489860+0000", 2024-09-14T22:38:53.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5563e-05 2024-09-14T22:38:53.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.894 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.894 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563070+0000", 2024-09-14T22:38:53.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073209724000000004 2024-09-14T22:38:53.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.894 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.894 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563175+0000", 2024-09-14T22:38:53.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105183 2024-09-14T22:38:53.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.895 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563281+0000", 2024-09-14T22:38:53.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105958 2024-09-14T22:38:53.895 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.895 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.895 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.895 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104193 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15794176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:53.895 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:38:53.895 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.343360388999997, 2024-09-14T22:38:53.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073600040000000005, 2024-09-14T22:38:53.896 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.896 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.896 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.896 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.896 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.896 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.896 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104193 2024-09-14T22:38:53.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.896 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.896 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.896 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:38:53.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:38:53.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489881+0000", 2024-09-14T22:38:53.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5460000000000001e-06 2024-09-14T22:38:53.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489886+0000", 2024-09-14T22:38:53.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1739999999999999e-06 2024-09-14T22:38:53.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489912+0000", 2024-09-14T22:38:53.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6461999999999998e-05 2024-09-14T22:38:53.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489927+0000", 2024-09-14T22:38:53.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:38:53.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563310+0000", 2024-09-14T22:38:53.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073382977000000002 2024-09-14T22:38:53.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.899 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.899 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563385+0000", 2024-09-14T22:38:53.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5218999999999999e-05 2024-09-14T22:38:53.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.899 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.899 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563479+0000", 2024-09-14T22:38:53.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4622000000000003e-05 2024-09-14T22:38:53.899 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.899 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.900 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.900 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.900 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104194 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15925248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:53.900 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:38:53.900 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.343293797000001, 2024-09-14T22:38:53.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073745434999999998, 2024-09-14T22:38:53.900 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.900 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.900 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.900 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.900 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.900 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.900 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104194 2024-09-14T22:38:53.901 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.901 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.901 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.901 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.901 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:38:53.901 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.901 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.901 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.901 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.901 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:38:53.901 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.901 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.901 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.901 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489947+0000", 2024-09-14T22:38:53.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.327e-06 2024-09-14T22:38:53.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489953+0000", 2024-09-14T22:38:53.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9329999999999998e-06 2024-09-14T22:38:53.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489971+0000", 2024-09-14T22:38:53.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7994000000000001e-05 2024-09-14T22:38:53.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489985+0000", 2024-09-14T22:38:53.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4226000000000001e-05 2024-09-14T22:38:53.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563510+0000", 2024-09-14T22:38:53.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073524667000000002 2024-09-14T22:38:53.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563597+0000", 2024-09-14T22:38:53.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6550000000000003e-05 2024-09-14T22:38:53.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.904 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.904 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563691+0000", 2024-09-14T22:38:53.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4738e-05 2024-09-14T22:38:53.904 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.904 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.904 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.904 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104195 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:53.904 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:38:53.904 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.3432356, 2024-09-14T22:38:53.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073876598000000002, 2024-09-14T22:38:53.904 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.905 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.905 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.905 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.905 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.905 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.905 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104195 2024-09-14T22:38:53.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.905 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.905 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.905 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:38:53.905 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.905 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.906 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:38:53.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.906 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.906 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490006+0000", 2024-09-14T22:38:53.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4699999999999999e-06 2024-09-14T22:38:53.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.906 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.906 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490011+0000", 2024-09-14T22:38:53.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.366e-06 2024-09-14T22:38:53.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490046+0000", 2024-09-14T22:38:53.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5496e-05 2024-09-14T22:38:53.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490061+0000", 2024-09-14T22:38:53.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5054e-05 2024-09-14T22:38:53.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563719+0000", 2024-09-14T22:38:53.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073657412000000005 2024-09-14T22:38:53.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563792+0000", 2024-09-14T22:38:53.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3319999999999996e-05 2024-09-14T22:38:53.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563881+0000", 2024-09-14T22:38:53.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8480000000000007e-05 2024-09-14T22:38:53.908 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.908 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.908 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.909 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104196 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16187392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:53.909 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:38:53.909 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.343158450999994, 2024-09-14T22:38:53.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073997496999999995, 2024-09-14T22:38:53.909 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.909 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.909 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.909 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.909 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.909 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.909 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104196 2024-09-14T22:38:53.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.910 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.910 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:38:53.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.910 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:38:53.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.910 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490083+0000", 2024-09-14T22:38:53.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.347e-06 2024-09-14T22:38:53.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490089+0000", 2024-09-14T22:38:53.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9630000000000003e-06 2024-09-14T22:38:53.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490107+0000", 2024-09-14T22:38:53.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8859999999999999e-05 2024-09-14T22:38:53.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490122+0000", 2024-09-14T22:38:53.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.432e-05 2024-09-14T22:38:53.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563910+0000", 2024-09-14T22:38:53.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073788753999999998 2024-09-14T22:38:53.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563988+0000", 2024-09-14T22:38:53.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7862999999999996e-05 2024-09-14T22:38:53.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564079+0000", 2024-09-14T22:38:53.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0389999999999993e-05 2024-09-14T22:38:53.913 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.913 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.913 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.913 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104197 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16318464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:53.913 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:38:53.913 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.343066821999997, 2024-09-14T22:38:53.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074103547000000006, 2024-09-14T22:38:53.913 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.913 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.914 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.914 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.914 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.914 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.914 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104197 2024-09-14T22:38:53.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.914 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.914 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:38:53.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.914 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:38:53.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490174+0000", 2024-09-14T22:38:53.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.37e-06 2024-09-14T22:38:53.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490197+0000", 2024-09-14T22:38:53.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3017e-05 2024-09-14T22:38:53.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490219+0000", 2024-09-14T22:38:53.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.181e-05 2024-09-14T22:38:53.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490240+0000", 2024-09-14T22:38:53.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0786000000000001e-05 2024-09-14T22:38:53.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564109+0000", 2024-09-14T22:38:53.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073869031000000002 2024-09-14T22:38:53.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564186+0000", 2024-09-14T22:38:53.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6823999999999995e-05 2024-09-14T22:38:53.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564276+0000", 2024-09-14T22:38:53.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0709000000000002e-05 2024-09-14T22:38:53.917 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.917 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.917 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.917 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104198 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16449536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:53.918 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:38:53.918 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.342974585999997, 2024-09-14T22:38:53.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074167693000000007, 2024-09-14T22:38:53.918 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.918 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.918 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.918 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.918 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.918 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.918 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104198 2024-09-14T22:38:53.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.918 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:38:53.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:38:53.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490266+0000", 2024-09-14T22:38:53.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3599999999999999e-06 2024-09-14T22:38:53.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490272+0000", 2024-09-14T22:38:53.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9599999999999997e-06 2024-09-14T22:38:53.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490291+0000", 2024-09-14T22:38:53.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8485999999999999e-05 2024-09-14T22:38:53.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490305+0000", 2024-09-14T22:38:53.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4327e-05 2024-09-14T22:38:53.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564306+0000", 2024-09-14T22:38:53.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074000745000000007 2024-09-14T22:38:53.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564367+0000", 2024-09-14T22:38:53.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1501999999999995e-05 2024-09-14T22:38:53.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564433+0000", 2024-09-14T22:38:53.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5313000000000003e-05 2024-09-14T22:38:53.921 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.921 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.921 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.922 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104199 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16580608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:53.922 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:38:53.922 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.342893032999996, 2024-09-14T22:38:53.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074231631000000006, 2024-09-14T22:38:53.922 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.922 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.922 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.922 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.922 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.922 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.922 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104199 2024-09-14T22:38:53.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.923 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.923 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.923 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:38:53.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.923 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.923 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:38:53.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.923 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.923 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490349+0000", 2024-09-14T22:38:53.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9429999999999999e-06 2024-09-14T22:38:53.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490355+0000", 2024-09-14T22:38:53.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1670000000000004e-06 2024-09-14T22:38:53.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490374+0000", 2024-09-14T22:38:53.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9613e-05 2024-09-14T22:38:53.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490405+0000", 2024-09-14T22:38:53.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0936000000000003e-05 2024-09-14T22:38:53.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564451+0000", 2024-09-14T22:38:53.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074045266999999998 2024-09-14T22:38:53.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564494+0000", 2024-09-14T22:38:53.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3133000000000002e-05 2024-09-14T22:38:53.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564578+0000", 2024-09-14T22:38:53.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4572000000000003e-05 2024-09-14T22:38:53.926 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.926 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.926 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.926 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104200 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:53.926 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:38:53.926 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.341097805000004, 2024-09-14T22:38:53.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072582435000000001, 2024-09-14T22:38:53.926 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:53.926 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:53.926 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:53.927 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:53.927 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:53.927 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:53.927 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104200 2024-09-14T22:38:53.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.927 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:53.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:53.927 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:38:53.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:53.927 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:38:53.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:53.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:53.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492144+0000", 2024-09-14T22:38:53.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3070000000000001e-06 2024-09-14T22:38:53.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:53.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492155+0000", 2024-09-14T22:38:53.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0573e-05 2024-09-14T22:38:53.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:53.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492185+0000", 2024-09-14T22:38:53.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.008e-05 2024-09-14T22:38:53.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:53.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492205+0000", 2024-09-14T22:38:53.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9882999999999999e-05 2024-09-14T22:38:53.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:53.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564609+0000", 2024-09-14T22:38:53.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072404181999999997 2024-09-14T22:38:53.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:53.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564667+0000", 2024-09-14T22:38:53.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8102e-05 2024-09-14T22:38:53.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:53.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:53.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:53.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564724+0000", 2024-09-14T22:38:53.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7308000000000001e-05 2024-09-14T22:38:53.930 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.930 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.930 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.930 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:53.930 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:53.930 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:53.932 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:38:53.932 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:53.926+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:53.992 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2024-09-14T22:38:54.032 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:54.026+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:54.085 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:54.085 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:38:54.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.085 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:162520 3.9 3:91856005:::rbd_header.113c609e90d7:head [call rbd.image_group_get in=18b out=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e161)", 2024-09-14T22:38:54.085 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:38:54.084223+0000", 2024-09-14T22:38:54.086 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.000366385, 2024-09-14T22:38:54.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00038690399999999998, 2024-09-14T22:38:54.086 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:54.086 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:54.086 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:54.086 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:54.086 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:54.086 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:54.086 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 162520 2024-09-14T22:38:54.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.086 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:54.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.086 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:54.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:54.084223+0000", 2024-09-14T22:38:54.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:54.087 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:54.084223+0000", 2024-09-14T22:38:54.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:54.087 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:54.084223+0000", 2024-09-14T22:38:54.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6700000000000003e-07 2024-09-14T22:38:54.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:54.088 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:54.084227+0000", 2024-09-14T22:38:54.088 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8360000000000002e-06 2024-09-14T22:38:54.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.088 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:54.088 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:54.084238+0000", 2024-09-14T22:38:54.088 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.079e-05 2024-09-14T22:38:54.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.088 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:54.088 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:54.084246+0000", 2024-09-14T22:38:54.088 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7989999999999994e-06 2024-09-14T22:38:54.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:54.089 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:54.084315+0000", 2024-09-14T22:38:54.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9039000000000001e-05 2024-09-14T22:38:54.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:54.089 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:54.084412+0000", 2024-09-14T22:38:54.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6908000000000001e-05 2024-09-14T22:38:54.089 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.089 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:54.089 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.089 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.090 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-14T22:38:54.090 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 1 2024-09-14T22:38:54.090 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:54.103 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:38:54.150 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:54.148+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:54.233 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:54.233 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:54.233 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:38:54.233 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:38:54.233 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:54.234 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:54.228+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:54.247 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:38:54.334 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:54.332+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:54.420 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:54.421 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:38:54.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:38:54.421 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:38:54.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.421 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144987 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3424256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:54.421 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549297+0000", 2024-09-14T22:38:54.421 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.867387348999998, 2024-09-14T22:38:54.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.079931556000000001, 2024-09-14T22:38:54.421 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:54.421 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:54.421 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:54.422 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:54.422 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:54.422 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:54.422 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144987 2024-09-14T22:38:54.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.422 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:54.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:54.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549297+0000", 2024-09-14T22:38:54.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:54.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549297+0000", 2024-09-14T22:38:54.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:54.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549298+0000", 2024-09-14T22:38:54.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2699999999999999e-07 2024-09-14T22:38:54.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:54.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549301+0000", 2024-09-14T22:38:54.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2959999999999999e-06 2024-09-14T22:38:54.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:54.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549314+0000", 2024-09-14T22:38:54.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2588999999999999e-05 2024-09-14T22:38:54.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:54.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549323+0000", 2024-09-14T22:38:54.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.927e-06 2024-09-14T22:38:54.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:54.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629038+0000", 2024-09-14T22:38:54.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.079714220000000002 2024-09-14T22:38:54.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:54.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629116+0000", 2024-09-14T22:38:54.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7966000000000005e-05 2024-09-14T22:38:54.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:54.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629228+0000", 2024-09-14T22:38:54.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112831 2024-09-14T22:38:54.425 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.425 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:54.425 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.426 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144988 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3325952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:54.426 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549335+0000", 2024-09-14T22:38:54.426 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.867349124, 2024-09-14T22:38:54.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080179116999999994, 2024-09-14T22:38:54.426 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:54.426 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:54.426 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:54.426 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:54.426 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:54.426 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:54.426 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144988 2024-09-14T22:38:54.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.427 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:54.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:54.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549335+0000", 2024-09-14T22:38:54.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:54.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549335+0000", 2024-09-14T22:38:54.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:54.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549336+0000", 2024-09-14T22:38:54.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7999999999999995e-07 2024-09-14T22:38:54.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:54.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549339+0000", 2024-09-14T22:38:54.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.836e-06 2024-09-14T22:38:54.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:54.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549352+0000", 2024-09-14T22:38:54.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3069e-05 2024-09-14T22:38:54.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:54.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549359+0000", 2024-09-14T22:38:54.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.357e-06 2024-09-14T22:38:54.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:54.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629288+0000", 2024-09-14T22:38:54.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.079928492000000004 2024-09-14T22:38:54.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:54.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629412+0000", 2024-09-14T22:38:54.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012413699999999999 2024-09-14T22:38:54.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:54.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629514+0000", 2024-09-14T22:38:54.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000102546 2024-09-14T22:38:54.430 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.430 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:54.430 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.430 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144989 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3522560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:54.430 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:38:54.430 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.867314716000003, 2024-09-14T22:38:54.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080393122999999997, 2024-09-14T22:38:54.430 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:54.430 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:54.431 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:54.431 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:54.431 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:54.431 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:54.431 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144989 2024-09-14T22:38:54.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.431 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:54.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:54.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:38:54.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:54.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:38:54.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:54.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:38:54.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3600000000000003e-07 2024-09-14T22:38:54.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:54.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549373+0000", 2024-09-14T22:38:54.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8169999999999999e-06 2024-09-14T22:38:54.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:54.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549385+0000", 2024-09-14T22:38:54.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1803e-05 2024-09-14T22:38:54.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:54.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549394+0000", 2024-09-14T22:38:54.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4290000000000007e-06 2024-09-14T22:38:54.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:54.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629576+0000", 2024-09-14T22:38:54.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080181551000000004 2024-09-14T22:38:54.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:54.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629654+0000", 2024-09-14T22:38:54.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7633999999999996e-05 2024-09-14T22:38:54.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:54.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629763+0000", 2024-09-14T22:38:54.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109153 2024-09-14T22:38:54.434 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.434 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:54.434 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.435 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144990 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:54.435 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:38:54.435 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.867271692000003, 2024-09-14T22:38:54.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080614496999999993, 2024-09-14T22:38:54.435 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:54.435 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:54.435 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:54.435 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:54.435 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:54.435 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:54.435 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144990 2024-09-14T22:38:54.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.435 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:54.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:54.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:38:54.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:54.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:38:54.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:54.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:38:54.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0999999999999997e-07 2024-09-14T22:38:54.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:54.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549419+0000", 2024-09-14T22:38:54.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5339999999999999e-06 2024-09-14T22:38:54.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:54.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549439+0000", 2024-09-14T22:38:54.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0035e-05 2024-09-14T22:38:54.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:54.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549449+0000", 2024-09-14T22:38:54.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0429e-05 2024-09-14T22:38:54.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:54.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629815+0000", 2024-09-14T22:38:54.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080365467999999995 2024-09-14T22:38:54.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:54.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629922+0000", 2024-09-14T22:38:54.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106914 2024-09-14T22:38:54.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:54.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630027+0000", 2024-09-14T22:38:54.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105207 2024-09-14T22:38:54.439 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.439 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:54.439 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.439 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144991 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 2998272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:54.439 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549463+0000", 2024-09-14T22:38:54.439 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.867221387999997, 2024-09-14T22:38:54.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080847637, 2024-09-14T22:38:54.439 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:54.439 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:54.439 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:54.439 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:54.439 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:54.439 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:54.440 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144991 2024-09-14T22:38:54.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.440 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:54.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:54.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549463+0000", 2024-09-14T22:38:54.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:54.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549463+0000", 2024-09-14T22:38:54.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:54.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549464+0000", 2024-09-14T22:38:54.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9999999999999997e-07 2024-09-14T22:38:54.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:54.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549467+0000", 2024-09-14T22:38:54.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0800000000000002e-06 2024-09-14T22:38:54.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:54.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549478+0000", 2024-09-14T22:38:54.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1813e-05 2024-09-14T22:38:54.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:54.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549488+0000", 2024-09-14T22:38:54.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7030000000000001e-06 2024-09-14T22:38:54.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:54.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630090+0000", 2024-09-14T22:38:54.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080601962999999999 2024-09-14T22:38:54.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:54.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630187+0000", 2024-09-14T22:38:54.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6552000000000007e-05 2024-09-14T22:38:54.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:54.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630311+0000", 2024-09-14T22:38:54.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000123826 2024-09-14T22:38:54.443 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.443 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:54.443 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.443 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144992 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3620864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:54.443 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:38:54.443 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.867185579999997, 2024-09-14T22:38:54.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081097875999999999, 2024-09-14T22:38:54.443 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:54.443 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:54.443 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:54.444 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:54.444 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:54.444 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:54.444 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144992 2024-09-14T22:38:54.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.444 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:54.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:54.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:38:54.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:54.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:38:54.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:54.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:38:54.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5400000000000001e-07 2024-09-14T22:38:54.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:54.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549502+0000", 2024-09-14T22:38:54.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7889999999999999e-06 2024-09-14T22:38:54.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:54.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549513+0000", 2024-09-14T22:38:54.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1204e-05 2024-09-14T22:38:54.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:54.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549523+0000", 2024-09-14T22:38:54.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5990000000000008e-06 2024-09-14T22:38:54.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:54.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630362+0000", 2024-09-14T22:38:54.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080839272000000004 2024-09-14T22:38:54.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:54.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630469+0000", 2024-09-14T22:38:54.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107285 2024-09-14T22:38:54.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:54.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630597+0000", 2024-09-14T22:38:54.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012707300000000001 2024-09-14T22:38:54.447 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.447 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:54.448 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.448 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144993 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:54.448 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:38:54.448 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.867152504000003, 2024-09-14T22:38:54.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081338411999999999, 2024-09-14T22:38:54.448 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:54.448 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:54.448 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:54.448 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:54.448 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:54.448 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:54.448 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144993 2024-09-14T22:38:54.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.449 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:54.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:54.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:38:54.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:54.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:38:54.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.450 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:54.450 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:38:54.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5300000000000004e-07 2024-09-14T22:38:54.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.450 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:54.450 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549544+0000", 2024-09-14T22:38:54.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1912e-05 2024-09-14T22:38:54.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.450 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:54.450 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549557+0000", 2024-09-14T22:38:54.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2413e-05 2024-09-14T22:38:54.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:54.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549567+0000", 2024-09-14T22:38:54.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8660000000000001e-06 2024-09-14T22:38:54.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:54.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630651+0000", 2024-09-14T22:38:54.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081084229999999993 2024-09-14T22:38:54.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:54.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630729+0000", 2024-09-14T22:38:54.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8265999999999999e-05 2024-09-14T22:38:54.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:54.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630870+0000", 2024-09-14T22:38:54.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014107200000000001 2024-09-14T22:38:54.452 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.452 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:54.452 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.452 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144994 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:54.452 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:38:54.452 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.867105537, 2024-09-14T22:38:54.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081547958000000004, 2024-09-14T22:38:54.453 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:54.453 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:54.453 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:54.453 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:54.453 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:54.453 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:54.453 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144994 2024-09-14T22:38:54.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.453 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:54.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:54.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:38:54.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:54.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:38:54.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:54.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:38:54.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6599999999999996e-07 2024-09-14T22:38:54.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:54.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549582+0000", 2024-09-14T22:38:54.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.004e-06 2024-09-14T22:38:54.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:54.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549594+0000", 2024-09-14T22:38:54.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1459e-05 2024-09-14T22:38:54.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:54.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549603+0000", 2024-09-14T22:38:54.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6129999999999993e-06 2024-09-14T22:38:54.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:54.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630932+0000", 2024-09-14T22:38:54.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081328558999999995 2024-09-14T22:38:54.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:54.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631027+0000", 2024-09-14T22:38:54.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4742999999999997e-05 2024-09-14T22:38:54.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:54.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631127+0000", 2024-09-14T22:38:54.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9914000000000006e-05 2024-09-14T22:38:54.457 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.457 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:54.457 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.457 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144995 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3391488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:54.457 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549615+0000", 2024-09-14T22:38:54.457 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.867069399000002, 2024-09-14T22:38:54.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081734251999999993, 2024-09-14T22:38:54.457 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:54.457 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:54.457 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:54.458 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:54.458 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:54.458 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:54.458 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144995 2024-09-14T22:38:54.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.458 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:54.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.460 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:54.460 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549615+0000", 2024-09-14T22:38:54.460 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.460 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:54.460 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549615+0000", 2024-09-14T22:38:54.460 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:54.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549616+0000", 2024-09-14T22:38:54.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9299999999999997e-07 2024-09-14T22:38:54.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:54.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549618+0000", 2024-09-14T22:38:54.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7269999999999999e-06 2024-09-14T22:38:54.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:54.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549630+0000", 2024-09-14T22:38:54.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1313e-05 2024-09-14T22:38:54.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:54.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549639+0000", 2024-09-14T22:38:54.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4159999999999993e-06 2024-09-14T22:38:54.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:54.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631183+0000", 2024-09-14T22:38:54.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081543633000000004 2024-09-14T22:38:54.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:54.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631252+0000", 2024-09-14T22:38:54.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9567999999999995e-05 2024-09-14T22:38:54.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.463 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:54.463 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631349+0000", 2024-09-14T22:38:54.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6801999999999999e-05 2024-09-14T22:38:54.463 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.463 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:54.463 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.463 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144996 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3194880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:54.463 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:38:54.463 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.867033720999999, 2024-09-14T22:38:54.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081947213000000005, 2024-09-14T22:38:54.464 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:54.464 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:54.464 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:54.464 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:54.464 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:54.464 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:54.464 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144996 2024-09-14T22:38:54.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.464 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:54.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.464 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:54.464 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:38:54.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.465 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:54.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:38:54.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.465 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:54.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:38:54.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3e-07 2024-09-14T22:38:54.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.465 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:54.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549654+0000", 2024-09-14T22:38:54.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6900000000000001e-06 2024-09-14T22:38:54.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:54.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549665+0000", 2024-09-14T22:38:54.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1392999999999999e-05 2024-09-14T22:38:54.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:54.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549675+0000", 2024-09-14T22:38:54.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5459999999999997e-06 2024-09-14T22:38:54.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:54.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631401+0000", 2024-09-14T22:38:54.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081726036000000002 2024-09-14T22:38:54.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:54.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631492+0000", 2024-09-14T22:38:54.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0886999999999999e-05 2024-09-14T22:38:54.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:54.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631598+0000", 2024-09-14T22:38:54.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106031 2024-09-14T22:38:54.467 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.467 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:54.467 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.468 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144997 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:54.468 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549692+0000", 2024-09-14T22:38:54.468 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.866992056999997, 2024-09-14T22:38:54.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082149925999999998, 2024-09-14T22:38:54.468 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:54.468 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:54.468 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:54.468 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:54.468 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:54.468 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:54.468 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144997 2024-09-14T22:38:54.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.469 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:54.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:54.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549692+0000", 2024-09-14T22:38:54.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:54.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549692+0000", 2024-09-14T22:38:54.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:54.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549693+0000", 2024-09-14T22:38:54.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.540000000000001e-07 2024-09-14T22:38:54.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:54.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549698+0000", 2024-09-14T22:38:54.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1460000000000004e-06 2024-09-14T22:38:54.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:54.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549715+0000", 2024-09-14T22:38:54.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6983e-05 2024-09-14T22:38:54.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:54.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549728+0000", 2024-09-14T22:38:54.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2326e-05 2024-09-14T22:38:54.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:54.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631649+0000", 2024-09-14T22:38:54.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081921830000000001 2024-09-14T22:38:54.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:54.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631726+0000", 2024-09-14T22:38:54.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6759000000000001e-05 2024-09-14T22:38:54.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:54.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631842+0000", 2024-09-14T22:38:54.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011592800000000001 2024-09-14T22:38:54.472 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.472 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:54.472 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.472 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144998 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 2965504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:54.472 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549741+0000", 2024-09-14T22:38:54.472 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.866942764999997, 2024-09-14T22:38:54.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082394832000000001, 2024-09-14T22:38:54.472 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:54.473 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:54.473 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:54.473 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:54.473 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:54.473 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:54.473 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144998 2024-09-14T22:38:54.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.473 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:54.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:54.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549741+0000", 2024-09-14T22:38:54.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:54.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549741+0000", 2024-09-14T22:38:54.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:54.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549742+0000", 2024-09-14T22:38:54.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.06e-07 2024-09-14T22:38:54.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:54.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549757+0000", 2024-09-14T22:38:54.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4839e-05 2024-09-14T22:38:54.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:54.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549770+0000", 2024-09-14T22:38:54.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2879e-05 2024-09-14T22:38:54.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:54.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549780+0000", 2024-09-14T22:38:54.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0304e-05 2024-09-14T22:38:54.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:54.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631904+0000", 2024-09-14T22:38:54.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082124168999999997 2024-09-14T22:38:54.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:54.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631995+0000", 2024-09-14T22:38:54.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0993000000000001e-05 2024-09-14T22:38:54.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:54.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632136+0000", 2024-09-14T22:38:54.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014094199999999999 2024-09-14T22:38:54.476 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.476 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:54.476 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.477 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144999 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3588096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:54.477 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:38:54.477 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.866892555, 2024-09-14T22:38:54.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082593742999999997, 2024-09-14T22:38:54.477 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:54.477 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:54.477 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:54.477 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:54.477 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:54.477 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:54.477 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144999 2024-09-14T22:38:54.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.477 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:54.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:54.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:38:54.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:54.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:38:54.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:54.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:38:54.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1999999999999999e-07 2024-09-14T22:38:54.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:54.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549795+0000", 2024-09-14T22:38:54.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9569999999999998e-06 2024-09-14T22:38:54.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:54.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549807+0000", 2024-09-14T22:38:54.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1691999999999999e-05 2024-09-14T22:38:54.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:54.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549817+0000", 2024-09-14T22:38:54.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5100000000000004e-06 2024-09-14T22:38:54.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:54.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632190+0000", 2024-09-14T22:38:54.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082373374999999999 2024-09-14T22:38:54.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:54.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632264+0000", 2024-09-14T22:38:54.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3972000000000003e-05 2024-09-14T22:38:54.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:54.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632385+0000", 2024-09-14T22:38:54.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121517 2024-09-14T22:38:54.481 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.481 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:54.481 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.481 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145000 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3063808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:54.481 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549828+0000", 2024-09-14T22:38:54.481 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.866856276999997, 2024-09-14T22:38:54.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082860235000000004, 2024-09-14T22:38:54.482 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:54.482 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:54.482 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:54.482 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:54.482 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:54.482 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:54.482 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145000 2024-09-14T22:38:54.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.482 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:54.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:54.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549828+0000", 2024-09-14T22:38:54.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:54.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549828+0000", 2024-09-14T22:38:54.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:54.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549829+0000", 2024-09-14T22:38:54.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3300000000000002e-07 2024-09-14T22:38:54.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:54.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549831+0000", 2024-09-14T22:38:54.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6740000000000001e-06 2024-09-14T22:38:54.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.484 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.484 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:54.484 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549842+0000", 2024-09-14T22:38:54.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0316000000000001e-05 2024-09-14T22:38:54.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.484 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.484 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:54.484 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549851+0000", 2024-09-14T22:38:54.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7229999999999999e-06 2024-09-14T22:38:54.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.484 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.484 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:54.484 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632448+0000", 2024-09-14T22:38:54.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082596648999999994 2024-09-14T22:38:54.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:54.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632568+0000", 2024-09-14T22:38:54.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119592 2024-09-14T22:38:54.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:54.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632688+0000", 2024-09-14T22:38:54.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120648 2024-09-14T22:38:54.485 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.486 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:54.486 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.486 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145001 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3162112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:54.486 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549863+0000", 2024-09-14T22:38:54.486 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.866821049000002, 2024-09-14T22:38:54.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083105534999999994, 2024-09-14T22:38:54.486 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:54.486 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:54.486 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:54.486 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:54.486 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:54.486 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:54.486 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145001 2024-09-14T22:38:54.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.487 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:54.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:54.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549863+0000", 2024-09-14T22:38:54.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:54.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549863+0000", 2024-09-14T22:38:54.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:54.488 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549864+0000", 2024-09-14T22:38:54.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4300000000000003e-07 2024-09-14T22:38:54.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.488 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:54.488 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549867+0000", 2024-09-14T22:38:54.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.757e-06 2024-09-14T22:38:54.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.488 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:54.488 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549878+0000", 2024-09-14T22:38:54.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1345999999999999e-05 2024-09-14T22:38:54.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:54.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549888+0000", 2024-09-14T22:38:54.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6460000000000005e-06 2024-09-14T22:38:54.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:54.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632754+0000", 2024-09-14T22:38:54.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082866258999999998 2024-09-14T22:38:54.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:54.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632841+0000", 2024-09-14T22:38:54.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6971000000000005e-05 2024-09-14T22:38:54.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:54.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632969+0000", 2024-09-14T22:38:54.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012791300000000001 2024-09-14T22:38:54.490 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.490 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:54.490 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.490 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145002 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3260416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:54.490 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549933+0000", 2024-09-14T22:38:54.490 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.866751546000003, 2024-09-14T22:38:54.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083291027000000004, 2024-09-14T22:38:54.491 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:54.491 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:54.491 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:54.491 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:54.491 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:54.491 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:54.491 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145002 2024-09-14T22:38:54.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.491 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:54.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:54.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549933+0000", 2024-09-14T22:38:54.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.492 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.492 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:54.492 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549933+0000", 2024-09-14T22:38:54.492 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.492 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.492 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:54.492 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549934+0000", 2024-09-14T22:38:54.492 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8700000000000004e-07 2024-09-14T22:38:54.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.492 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.492 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:54.492 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549938+0000", 2024-09-14T22:38:54.492 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5029999999999999e-06 2024-09-14T22:38:54.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.493 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:54.493 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549952+0000", 2024-09-14T22:38:54.493 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4382999999999999e-05 2024-09-14T22:38:54.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.493 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:54.493 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549963+0000", 2024-09-14T22:38:54.493 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0665999999999999e-05 2024-09-14T22:38:54.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.493 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:54.493 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633025+0000", 2024-09-14T22:38:54.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083061727000000002 2024-09-14T22:38:54.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.494 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:54.494 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633117+0000", 2024-09-14T22:38:54.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1812999999999997e-05 2024-09-14T22:38:54.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.494 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:54.494 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633224+0000", 2024-09-14T22:38:54.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107048 2024-09-14T22:38:54.494 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.494 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:54.494 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.495 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145003 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3457024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:54.495 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549985+0000", 2024-09-14T22:38:54.495 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.866699128, 2024-09-14T22:38:54.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083465871999999997, 2024-09-14T22:38:54.495 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:54.495 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:54.495 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:54.495 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:54.495 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:54.495 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:54.496 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145003 2024-09-14T22:38:54.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.496 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:54.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.496 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:54.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549985+0000", 2024-09-14T22:38:54.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.496 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:54.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549985+0000", 2024-09-14T22:38:54.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:54.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549986+0000", 2024-09-14T22:38:54.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7299999999999995e-07 2024-09-14T22:38:54.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:54.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549989+0000", 2024-09-14T22:38:54.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8169999999999999e-06 2024-09-14T22:38:54.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:54.498 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549998+0000", 2024-09-14T22:38:54.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4290000000000007e-06 2024-09-14T22:38:54.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.498 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:54.498 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550009+0000", 2024-09-14T22:38:54.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0633e-05 2024-09-14T22:38:54.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.498 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:54.498 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633275+0000", 2024-09-14T22:38:54.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083266397000000006 2024-09-14T22:38:54.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:54.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633353+0000", 2024-09-14T22:38:54.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8129999999999999e-05 2024-09-14T22:38:54.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:54.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633451+0000", 2024-09-14T22:38:54.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7793000000000004e-05 2024-09-14T22:38:54.499 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.499 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:54.499 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.499 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145004 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3358720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:54.499 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.550029+0000", 2024-09-14T22:38:54.500 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.866655557000001, 2024-09-14T22:38:54.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083662089999999995, 2024-09-14T22:38:54.500 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:54.500 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:54.500 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:54.500 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:54.500 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:54.500 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:54.500 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145004 2024-09-14T22:38:54.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.500 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:54.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:54.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550029+0000", 2024-09-14T22:38:54.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:54.501 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550029+0000", 2024-09-14T22:38:54.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:54.501 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550030+0000", 2024-09-14T22:38:54.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.33e-07 2024-09-14T22:38:54.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:54.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550035+0000", 2024-09-14T22:38:54.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3430000000000003e-06 2024-09-14T22:38:54.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.502 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:54.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550053+0000", 2024-09-14T22:38:54.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8485e-05 2024-09-14T22:38:54.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.502 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:54.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550064+0000", 2024-09-14T22:38:54.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0264e-05 2024-09-14T22:38:54.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:54.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633502+0000", 2024-09-14T22:38:54.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083438498999999999 2024-09-14T22:38:54.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:54.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633592+0000", 2024-09-14T22:38:54.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.022e-05 2024-09-14T22:38:54.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:54.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633691+0000", 2024-09-14T22:38:54.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8345999999999997e-05 2024-09-14T22:38:54.503 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.504 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:54.504 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.504 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145005 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3555328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:54.504 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.550077+0000", 2024-09-14T22:38:54.504 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.866607096999999, 2024-09-14T22:38:54.504 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083854340999999999, 2024-09-14T22:38:54.504 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:54.504 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:54.504 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:54.504 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:54.504 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:54.504 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:54.505 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145005 2024-09-14T22:38:54.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.505 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:54.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:54.505 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550077+0000", 2024-09-14T22:38:54.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:54.505 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550077+0000", 2024-09-14T22:38:54.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:54.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550078+0000", 2024-09-14T22:38:54.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8700000000000005e-07 2024-09-14T22:38:54.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:54.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550101+0000", 2024-09-14T22:38:54.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3632e-05 2024-09-14T22:38:54.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:54.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550115+0000", 2024-09-14T22:38:54.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3893000000000001e-05 2024-09-14T22:38:54.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:54.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550126+0000", 2024-09-14T22:38:54.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0426e-05 2024-09-14T22:38:54.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:54.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633742+0000", 2024-09-14T22:38:54.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083616633999999995 2024-09-14T22:38:54.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:54.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633816+0000", 2024-09-14T22:38:54.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3690999999999996e-05 2024-09-14T22:38:54.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:54.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633932+0000", 2024-09-14T22:38:54.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011537800000000001 2024-09-14T22:38:54.508 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.508 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:54.508 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.508 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145006 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3653632~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:54.508 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.550236+0000", 2024-09-14T22:38:54.508 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.866448591000001, 2024-09-14T22:38:54.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083924027999999998, 2024-09-14T22:38:54.508 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:54.509 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:54.509 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:54.509 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:54.509 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:54.509 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:54.509 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145006 2024-09-14T22:38:54.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.509 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:54.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:54.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550236+0000", 2024-09-14T22:38:54.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:54.510 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550236+0000", 2024-09-14T22:38:54.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:54.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:54.510 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550237+0000", 2024-09-14T22:38:54.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2929999999999999e-06 2024-09-14T22:38:54.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:54.510 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550243+0000", 2024-09-14T22:38:54.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0390000000000001e-06 2024-09-14T22:38:54.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:54.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550256+0000", 2024-09-14T22:38:54.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3393e-05 2024-09-14T22:38:54.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:54.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550271+0000", 2024-09-14T22:38:54.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4166e-05 2024-09-14T22:38:54.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:54.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633974+0000", 2024-09-14T22:38:54.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083703825999999995 2024-09-14T22:38:54.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:54.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.634047+0000", 2024-09-14T22:38:54.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2935999999999994e-05 2024-09-14T22:38:54.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:54.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:54.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:54.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.634160+0000", 2024-09-14T22:38:54.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011237499999999999 2024-09-14T22:38:54.512 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.512 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:54.512 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.512 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:54.512 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:54.512 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:54.513 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:38:54.514 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:54.512+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:54.515 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:38:54.615 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:54.612+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:54.696 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:54.697 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:54.697 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:38:54.697 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:54.706 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:38:54.706 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:38:54.711 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:38:54.723 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:38:54.724 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:38:54.724 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:54.720+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:54.740 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:38:54.741 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:38:54.758 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:38:54.758 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:38:54.758 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T22:38:54.779 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:38:54.779 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:38:54.800 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:38:54.800 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:38:54.817 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:54.810+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:54.821 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:38:54.821 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:38:54.827 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:38:54.827 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:38:54.841 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:54.841 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:54.856 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:38:54.918 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:54.910+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:55.019 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:55.016+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:55.021 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:55.021 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:55.038 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:38:55.120 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:55.114+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:55.192 INFO:teuthology.orchestra.run.smithi176.stderr:Exporting 3.9 info 3.9( v 158'425 (0'0,158'425] local-lis/les=116/117 n=47 ec=23/23 lis/c=116/116 les/c/f=117/117/0 sis=116) 2024-09-14T22:38:55.208 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:9063cc7e:::rbd_data.113cda0183a0.0000000000000007:head# 2024-09-14T22:38:55.211 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:9094ac4b:::rbd_data.113cf9fcaf37.0000000000000035:head# 2024-09-14T22:38:55.211 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:90a1593c:::rbd_data.113c179ecb6a.0000000000000260:head# 2024-09-14T22:38:55.211 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:90d69314:::rbd_data.113cf9fcaf37.0000000000000026:head# 2024-09-14T22:38:55.212 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:90db9e2a:::rbd_data.113c179ecb6a.00000000000001f7:head# 2024-09-14T22:38:55.212 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:91459c76:::rbd_data.113c179ecb6a.00000000000000ad:head# 2024-09-14T22:38:55.212 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:914b4970:::rbd_data.113c179ecb6a.00000000000000bc:head# 2024-09-14T22:38:55.212 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:91856005:::rbd_header.113c609e90d7:head# 2024-09-14T22:38:55.212 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:91b049a8:::rbd_data.113c179ecb6a.00000000000001c1:head# 2024-09-14T22:38:55.212 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:91b0d498:::rbd_data.113cf9fcaf37.000000000000009b:head# 2024-09-14T22:38:55.214 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:91f6bb73:::rbd_data.113c179ecb6a.00000000000002ac:head# 2024-09-14T22:38:55.215 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:9206cc21:::rbd_data.113c179ecb6a.00000000000000d9:head# 2024-09-14T22:38:55.215 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:928bbed7:::rbd_data.113c237f38ab.000000000000027b:head# 2024-09-14T22:38:55.215 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:92974bdd:::rbd_data.113c179ecb6a.00000000000001f1:head# 2024-09-14T22:38:55.215 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:92abd5f0:::rbd_data.113c4558bb1.00000000000000a8:7# 2024-09-14T22:38:55.215 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:92abd5f0:::rbd_data.113c4558bb1.00000000000000a8:head# 2024-09-14T22:38:55.215 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:93045a07:::rbd_data.113cf9fcaf37.00000000000000f7:head# 2024-09-14T22:38:55.215 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:931332d1:::rbd_data.113cf9fcaf37.00000000000000b2:head# 2024-09-14T22:38:55.215 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:93a23de8:::rbd_data.113c179ecb6a.0000000000000200:head# 2024-09-14T22:38:55.215 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:93a2dd18:::rbd_data.113c179ecb6a.0000000000000277:head# 2024-09-14T22:38:55.215 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:93d6b168:::rbd_data.113cec2505f0.000000000000001a:head# 2024-09-14T22:38:55.215 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:head# 2024-09-14T22:38:55.225 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:94389354:::rbd_data.113cf9fcaf37.000000000000004e:head# 2024-09-14T22:38:55.225 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:949c0cf8:::rbd_data.113cf9fcaf37.0000000000000069:head# 2024-09-14T22:38:55.225 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:94ce85d8:::rbd_data.113c4558bb1.0000000000000060:7# 2024-09-14T22:38:55.225 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:94ce85d8:::rbd_data.113c4558bb1.0000000000000060:head# 2024-09-14T22:38:55.225 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:9513a271:::rbd_data.113c179ecb6a.000000000000011b:head# 2024-09-14T22:38:55.226 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:952d3e01:::rbd_data.113c3c739f88.000000000000005a:head# 2024-09-14T22:38:55.226 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:953746ba:::rbd_data.113c179ecb6a.0000000000000113:head# 2024-09-14T22:38:55.226 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:9542a272:::rbd_data.113c179ecb6a.0000000000000112:head# 2024-09-14T22:38:55.226 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:954ce16b:::rbd_data.113c4558bb1.0000000000000074:7# 2024-09-14T22:38:55.226 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:954ce16b:::rbd_data.113c4558bb1.0000000000000074:head# 2024-09-14T22:38:55.227 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:955cce30:::rbd_data.113c179ecb6a.000000000000008d:head# 2024-09-14T22:38:55.227 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:95a325d3:::rbd_data.113cf9fcaf37.00000000000000fc:head# 2024-09-14T22:38:55.227 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:95ee2fb4:::rbd_data.113cf9fcaf37.00000000000000ed:head# 2024-09-14T22:38:55.227 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:9617419a:::rbd_data.113cf9fcaf37.000000000000006e:head# 2024-09-14T22:38:55.227 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:965ec7d5:::rbd_data.113c179ecb6a.00000000000002a7:head# 2024-09-14T22:38:55.227 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:96676ff5:::rbd_data.113cce440cee.0000000000000006:a# 2024-09-14T22:38:55.228 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:55.216+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:55.229 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:55.229 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:55.231 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:96676ff5:::rbd_data.113cce440cee.0000000000000006:head# 2024-09-14T22:38:55.231 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:96b055ea:::rbd_data.113cf9fcaf37.00000000000000e1:head# 2024-09-14T22:38:55.231 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:972a090b:::rbd_data.113c179ecb6a.00000000000001e2:head# 2024-09-14T22:38:55.231 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:9747b897:::rbd_data.113cf9fcaf37.00000000000000f0:head# 2024-09-14T22:38:55.231 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:97536c31:::rbd_data.113cf9fcaf37.00000000000000c6:head# 2024-09-14T22:38:55.231 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:975a890d:::rbd_data.113c179ecb6a.000000000000016b:head# 2024-09-14T22:38:55.231 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:97624012:::rbd_data.113c4558bb1.00000000000000b2:7# 2024-09-14T22:38:55.231 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:97624012:::rbd_data.113c4558bb1.00000000000000b2:head# 2024-09-14T22:38:55.231 INFO:teuthology.orchestra.run.smithi176.stderr:Read #3:978d708a:::rbd_data.113cf9fcaf37.00000000000000cd:head# 2024-09-14T22:38:55.231 INFO:teuthology.orchestra.run.smithi176.stderr:Export successful 2024-09-14T22:38:55.237 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:38:55.322 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:55.318+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:55.405 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:55.405 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:55.419 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:38:55.422 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:55.414+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:55.523 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:55.518+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:55.539 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:55.539 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:55.554 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:38:55.624 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:55.620+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:55.688 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:55.688 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:55.699 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:38:55.725 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:55.718+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:55.795 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --force --op remove --pgid 3.9 2024-09-14T22:38:55.839 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:55.834+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:55.878 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:55.878 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:55.894 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:38:55.942 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:55.940+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:56.022 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T22:38:56.023 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:56.023 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:56.038 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:38:56.039 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:56.036+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:56.129 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:56.124+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:56.169 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:56.169 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:56.183 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:38:56.229 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:56.222+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:56.313 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:56.313 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:56.327 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:38:56.371 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:56.366+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:56.432 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:56.426+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:56.489 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:56.489 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:56.503 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:38:56.533 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:56.528+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:56.634 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:56.626+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:56.672 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:56.672 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:56.691 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:38:56.735 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:56.732+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:56.741 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2024-09-14T22:38:56.828 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:38:56.858 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:56.859 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:56.874 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:38:56.875 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:56.872+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:56.937 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:56.932+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:56.991 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:56.991 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:56.993 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:38:56.993 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:38:57.005 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:38:57.011 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:38:57.011 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:38:57.028 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:38:57.028 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:38:57.044 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:38:57.044 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:38:57.045 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T22:38:57.051 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:57.046+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:57.061 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:38:57.062 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:38:57.081 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:38:57.081 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:38:57.099 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:38:57.099 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:38:57.105 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:38:57.106 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:38:57.136 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:57.136 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:57.137 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:38:57.137 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:57.150 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:38:57.151 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:57.148+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:57.240 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:57.236+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:57.310 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:57.311 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:38:57.311 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:38:57.311 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:38:57.311 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:57.325 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:38:57.341 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:57.336+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:57.442 INFO:teuthology.orchestra.run.smithi176.stdout: marking collection for removal 2024-09-14T22:38:57.442 INFO:teuthology.orchestra.run.smithi176.stdout:setting '_remove' omap key 2024-09-14T22:38:57.442 INFO:teuthology.orchestra.run.smithi176.stdout:finish_remove_pgs 3.9_head removing 3.9 2024-09-14T22:38:57.442 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:38:57.440+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:38:57.447 INFO:teuthology.orchestra.run.smithi176.stdout:Remove successful 2024-09-14T22:38:57.454 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:38:57.454 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:38:57.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:38:57.454 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:38:57.454 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.454 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:38:57.454 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:38:57.454 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 177.44518825899999, 2024-09-14T22:38:57.455 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.398029152, 2024-09-14T22:38:57.455 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:57.455 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:57.455 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:57.455 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:57.455 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:38:57.455 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:38:57.455 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 1 2024-09-14T22:38:57.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.455 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:57.455 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.455 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:57.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:38:57.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:57.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:38:57.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:57.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006183+0000", 2024-09-14T22:38:57.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9059999999999998e-06 2024-09-14T22:38:57.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:57.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006198+0000", 2024-09-14T22:38:57.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.523e-05 2024-09-14T22:38:57.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:57.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006219+0000", 2024-09-14T22:38:57.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1342999999999999e-05 2024-09-14T22:38:57.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:57.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006240+0000", 2024-09-14T22:38:57.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1186999999999999e-05 2024-09-14T22:38:57.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:57.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006363+0000", 2024-09-14T22:38:57.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012258200000000001 2024-09-14T22:38:57.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:38:57.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006392+0000", 2024-09-14T22:38:57.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9607e-05 2024-09-14T22:38:57.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:57.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401084+0000", 2024-09-14T22:38:57.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.39469142699999998 2024-09-14T22:38:57.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.459 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:57.459 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401394+0000", 2024-09-14T22:38:57.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030987799999999998 2024-09-14T22:38:57.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.459 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:38:57.459 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401860+0000", 2024-09-14T22:38:57.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00046654299999999998 2024-09-14T22:38:57.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.459 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:38:57.459 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.403386+0000", 2024-09-14T22:38:57.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001525322 2024-09-14T22:38:57.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.460 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.460 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:57.460 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404042+0000", 2024-09-14T22:38:57.460 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065676099999999996 2024-09-14T22:38:57.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.460 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.460 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:38:57.460 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404136+0000", 2024-09-14T22:38:57.460 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4032999999999999e-05 2024-09-14T22:38:57.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.460 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.460 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:57.460 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404209+0000", 2024-09-14T22:38:57.460 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2333e-05 2024-09-14T22:38:57.461 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.461 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:57.461 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.461 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104182 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14352384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:57.461 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:38:57.461 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.962334272999996, 2024-09-14T22:38:57.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071642496999999999, 2024-09-14T22:38:57.461 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:57.461 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:57.461 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:57.461 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:57.462 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:57.462 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:57.462 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104182 2024-09-14T22:38:57.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.462 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:57.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:57.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:38:57.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:57.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:38:57.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:57.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489035+0000", 2024-09-14T22:38:57.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3430000000000001e-06 2024-09-14T22:38:57.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:57.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489041+0000", 2024-09-14T22:38:57.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8139999999999997e-06 2024-09-14T22:38:57.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:57.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489059+0000", 2024-09-14T22:38:57.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8193e-05 2024-09-14T22:38:57.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:57.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489073+0000", 2024-09-14T22:38:57.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:38:57.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:57.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560434+0000", 2024-09-14T22:38:57.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071361463999999999 2024-09-14T22:38:57.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.465 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:57.465 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560545+0000", 2024-09-14T22:38:57.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110263 2024-09-14T22:38:57.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.465 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:57.465 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560676+0000", 2024-09-14T22:38:57.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013138000000000001 2024-09-14T22:38:57.465 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.465 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:57.465 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.466 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104183 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14483456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:57.466 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:38:57.466 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.962210954, 2024-09-14T22:38:57.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071781943000000001, 2024-09-14T22:38:57.466 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:57.466 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:57.466 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:57.466 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:57.466 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:57.466 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:57.466 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104183 2024-09-14T22:38:57.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.466 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:57.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:57.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:38:57.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:57.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:38:57.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:57.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489159+0000", 2024-09-14T22:38:57.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5629999999999999e-06 2024-09-14T22:38:57.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:57.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489165+0000", 2024-09-14T22:38:57.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7070000000000003e-06 2024-09-14T22:38:57.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:57.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489186+0000", 2024-09-14T22:38:57.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.088e-05 2024-09-14T22:38:57.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:57.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489201+0000", 2024-09-14T22:38:57.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5203e-05 2024-09-14T22:38:57.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:57.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560713+0000", 2024-09-14T22:38:57.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071511447000000006 2024-09-14T22:38:57.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:57.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560819+0000", 2024-09-14T22:38:57.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106579 2024-09-14T22:38:57.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:57.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560939+0000", 2024-09-14T22:38:57.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119564 2024-09-14T22:38:57.469 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.470 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:57.470 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.470 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.470 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.470 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104184 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:57.470 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:38:57.470 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.962146511, 2024-09-14T22:38:57.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071970107000000005, 2024-09-14T22:38:57.470 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:57.470 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:57.470 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:57.470 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:57.470 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:57.470 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:57.470 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104184 2024-09-14T22:38:57.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.471 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:57.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:57.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:38:57.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:57.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:38:57.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:57.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489223+0000", 2024-09-14T22:38:57.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4470000000000001e-06 2024-09-14T22:38:57.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:57.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489229+0000", 2024-09-14T22:38:57.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8200000000000002e-06 2024-09-14T22:38:57.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:57.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489247+0000", 2024-09-14T22:38:57.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8150000000000001e-05 2024-09-14T22:38:57.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:57.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489260+0000", 2024-09-14T22:38:57.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3570000000000001e-05 2024-09-14T22:38:57.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:57.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560979+0000", 2024-09-14T22:38:57.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071718831999999996 2024-09-14T22:38:57.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:57.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561075+0000", 2024-09-14T22:38:57.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5848999999999997e-05 2024-09-14T22:38:57.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:57.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561191+0000", 2024-09-14T22:38:57.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116439 2024-09-14T22:38:57.474 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.474 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:57.474 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.474 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104185 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14745600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:57.474 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:38:57.475 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.962086928000005, 2024-09-14T22:38:57.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072173750999999994, 2024-09-14T22:38:57.475 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:57.475 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:57.475 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:57.475 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:57.475 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:57.475 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:57.475 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104185 2024-09-14T22:38:57.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.475 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:57.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.475 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:57.475 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:38:57.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:57.476 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:38:57.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:57.476 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489282+0000", 2024-09-14T22:38:57.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2500000000000001e-06 2024-09-14T22:38:57.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:57.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489287+0000", 2024-09-14T22:38:57.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1359999999999996e-06 2024-09-14T22:38:57.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:57.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489304+0000", 2024-09-14T22:38:57.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7033000000000001e-05 2024-09-14T22:38:57.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:57.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489318+0000", 2024-09-14T22:38:57.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3934000000000001e-05 2024-09-14T22:38:57.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:57.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561232+0000", 2024-09-14T22:38:57.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071913759999999993 2024-09-14T22:38:57.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:57.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561340+0000", 2024-09-14T22:38:57.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107921 2024-09-14T22:38:57.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:57.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561455+0000", 2024-09-14T22:38:57.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011471700000000001 2024-09-14T22:38:57.478 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.478 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:57.478 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.479 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104186 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14876672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:57.479 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:38:57.479 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.962031018999994, 2024-09-14T22:38:57.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072393374999999996, 2024-09-14T22:38:57.479 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:57.479 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:57.479 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:57.479 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:57.479 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:57.479 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:57.479 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104186 2024-09-14T22:38:57.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.479 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:57.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.480 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:57.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:38:57.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.480 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:57.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:38:57.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:57.481 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489338+0000", 2024-09-14T22:38:57.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.48e-06 2024-09-14T22:38:57.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:57.481 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489366+0000", 2024-09-14T22:38:57.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7307000000000001e-05 2024-09-14T22:38:57.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:57.481 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489385+0000", 2024-09-14T22:38:57.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9283000000000002e-05 2024-09-14T22:38:57.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:57.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489399+0000", 2024-09-14T22:38:57.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.452e-05 2024-09-14T22:38:57.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:57.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561494+0000", 2024-09-14T22:38:57.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072094885999999997 2024-09-14T22:38:57.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:57.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561604+0000", 2024-09-14T22:38:57.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010925900000000001 2024-09-14T22:38:57.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:57.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561730+0000", 2024-09-14T22:38:57.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012663999999999999 2024-09-14T22:38:57.483 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.483 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:57.483 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.484 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104187 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15007744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:57.484 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:38:57.484 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.961947649999999, 2024-09-14T22:38:57.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072599143000000005, 2024-09-14T22:38:57.484 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:57.484 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:57.484 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:57.484 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:57.484 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:57.484 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:57.484 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104187 2024-09-14T22:38:57.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.485 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:57.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.485 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:57.485 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:38:57.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.485 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:57.485 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:38:57.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.485 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:57.485 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489422+0000", 2024-09-14T22:38:57.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5340000000000001e-06 2024-09-14T22:38:57.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.486 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:57.486 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489428+0000", 2024-09-14T22:38:57.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1430000000000003e-06 2024-09-14T22:38:57.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.486 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:57.486 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489445+0000", 2024-09-14T22:38:57.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7042999999999999e-05 2024-09-14T22:38:57.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.486 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:57.486 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489458+0000", 2024-09-14T22:38:57.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3474e-05 2024-09-14T22:38:57.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:57.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561771+0000", 2024-09-14T22:38:57.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072312600000000005 2024-09-14T22:38:57.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:57.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561885+0000", 2024-09-14T22:38:57.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011411799999999999 2024-09-14T22:38:57.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:57.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562019+0000", 2024-09-14T22:38:57.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013423100000000001 2024-09-14T22:38:57.488 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.488 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:57.488 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.488 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104188 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15138816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:57.488 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:38:57.488 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.961887726, 2024-09-14T22:38:57.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072824624000000004, 2024-09-14T22:38:57.488 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:57.488 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:57.488 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:57.488 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:57.488 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:57.489 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:57.489 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104188 2024-09-14T22:38:57.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.489 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:57.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:57.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:38:57.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:57.490 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:38:57.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:57.490 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489481+0000", 2024-09-14T22:38:57.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.22e-06 2024-09-14T22:38:57.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:57.490 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489487+0000", 2024-09-14T22:38:57.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6930000000000004e-06 2024-09-14T22:38:57.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:57.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489508+0000", 2024-09-14T22:38:57.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1135999999999999e-05 2024-09-14T22:38:57.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:57.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489522+0000", 2024-09-14T22:38:57.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3817e-05 2024-09-14T22:38:57.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:57.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562066+0000", 2024-09-14T22:38:57.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072543813999999998 2024-09-14T22:38:57.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:57.492 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562177+0000", 2024-09-14T22:38:57.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000111264 2024-09-14T22:38:57.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.492 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:57.492 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562305+0000", 2024-09-14T22:38:57.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012768 2024-09-14T22:38:57.492 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.492 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:57.492 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.492 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104189 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15269888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:57.492 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:38:57.492 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.961827276999998, 2024-09-14T22:38:57.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073061300999999995, 2024-09-14T22:38:57.493 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:57.493 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:57.493 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:57.493 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:57.493 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:57.493 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:57.493 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104189 2024-09-14T22:38:57.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.493 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:57.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.493 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:57.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:38:57.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.494 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:57.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:38:57.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.494 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:57.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489542+0000", 2024-09-14T22:38:57.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1659999999999999e-06 2024-09-14T22:38:57.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.495 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:57.495 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489561+0000", 2024-09-14T22:38:57.495 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.908e-05 2024-09-14T22:38:57.495 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.495 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:57.495 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489579+0000", 2024-09-14T22:38:57.495 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8536999999999999e-05 2024-09-14T22:38:57.495 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.496 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:57.496 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489593+0000", 2024-09-14T22:38:57.496 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3977e-05 2024-09-14T22:38:57.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.496 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:57.496 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562347+0000", 2024-09-14T22:38:57.496 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072754092000000006 2024-09-14T22:38:57.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.496 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:57.496 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562458+0000", 2024-09-14T22:38:57.496 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110722 2024-09-14T22:38:57.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.497 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:57.497 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562602+0000", 2024-09-14T22:38:57.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014372699999999999 2024-09-14T22:38:57.497 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.497 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:57.497 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.497 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104190 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:57.497 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:38:57.497 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.961754980999999, 2024-09-14T22:38:57.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.07326133, 2024-09-14T22:38:57.497 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:57.497 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:57.498 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:57.498 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:57.498 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:57.498 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:57.498 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104190 2024-09-14T22:38:57.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.498 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:57.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:57.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:38:57.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:57.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:38:57.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:57.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489614+0000", 2024-09-14T22:38:57.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1999999999999999e-06 2024-09-14T22:38:57.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:57.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489620+0000", 2024-09-14T22:38:57.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.524e-06 2024-09-14T22:38:57.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.500 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:57.500 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489637+0000", 2024-09-14T22:38:57.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7825999999999998e-05 2024-09-14T22:38:57.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.500 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:57.500 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489671+0000", 2024-09-14T22:38:57.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3726000000000002e-05 2024-09-14T22:38:57.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.500 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:57.500 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562644+0000", 2024-09-14T22:38:57.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072973089000000005 2024-09-14T22:38:57.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.501 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:57.501 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562762+0000", 2024-09-14T22:38:57.501 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011743600000000001 2024-09-14T22:38:57.501 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.501 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:57.501 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562874+0000", 2024-09-14T22:38:57.501 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011252900000000001 2024-09-14T22:38:57.501 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.501 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:57.501 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.501 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.502 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104191 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15532032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:57.502 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:38:57.502 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.961646557999998, 2024-09-14T22:38:57.502 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073311166999999997, 2024-09-14T22:38:57.502 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:57.502 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:57.502 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:57.502 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:57.502 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:57.502 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:57.502 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104191 2024-09-14T22:38:57.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.502 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:57.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.502 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:57.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:38:57.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.503 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:57.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:38:57.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.503 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:57.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489723+0000", 2024-09-14T22:38:57.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6330000000000001e-06 2024-09-14T22:38:57.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.503 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:57.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489730+0000", 2024-09-14T22:38:57.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2300000000000002e-06 2024-09-14T22:38:57.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:57.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489751+0000", 2024-09-14T22:38:57.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0849999999999999e-05 2024-09-14T22:38:57.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:57.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489766+0000", 2024-09-14T22:38:57.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.524e-05 2024-09-14T22:38:57.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:57.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562896+0000", 2024-09-14T22:38:57.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073129662999999998 2024-09-14T22:38:57.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:57.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562960+0000", 2024-09-14T22:38:57.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3689999999999995e-05 2024-09-14T22:38:57.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.506 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:57.506 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563033+0000", 2024-09-14T22:38:57.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2861000000000006e-05 2024-09-14T22:38:57.506 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.506 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:57.506 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.506 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104192 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:57.506 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:38:57.506 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.961550752999997, 2024-09-14T22:38:57.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073463632000000001, 2024-09-14T22:38:57.506 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:57.506 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:57.507 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:57.507 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:57.507 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:57.507 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:57.507 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104192 2024-09-14T22:38:57.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.507 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:57.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.507 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:57.507 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:38:57.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.507 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:57.507 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:38:57.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:57.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489819+0000", 2024-09-14T22:38:57.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.424e-06 2024-09-14T22:38:57.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:57.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489825+0000", 2024-09-14T22:38:57.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.02e-06 2024-09-14T22:38:57.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:57.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489844+0000", 2024-09-14T22:38:57.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9760000000000001e-05 2024-09-14T22:38:57.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:57.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489860+0000", 2024-09-14T22:38:57.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5563e-05 2024-09-14T22:38:57.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:57.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563070+0000", 2024-09-14T22:38:57.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073209724000000004 2024-09-14T22:38:57.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:57.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563175+0000", 2024-09-14T22:38:57.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105183 2024-09-14T22:38:57.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:57.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563281+0000", 2024-09-14T22:38:57.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105958 2024-09-14T22:38:57.510 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.510 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:57.510 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.510 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104193 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15794176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:57.510 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:38:57.510 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.961488586000002, 2024-09-14T22:38:57.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073600040000000005, 2024-09-14T22:38:57.511 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:57.511 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:57.511 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:57.511 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:57.511 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:57.511 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:57.511 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104193 2024-09-14T22:38:57.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.511 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:57.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.511 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:57.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:38:57.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:57.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:38:57.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:57.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489881+0000", 2024-09-14T22:38:57.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5460000000000001e-06 2024-09-14T22:38:57.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:57.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489886+0000", 2024-09-14T22:38:57.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1739999999999999e-06 2024-09-14T22:38:57.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:57.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489912+0000", 2024-09-14T22:38:57.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6461999999999998e-05 2024-09-14T22:38:57.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:57.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489927+0000", 2024-09-14T22:38:57.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:38:57.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:57.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563310+0000", 2024-09-14T22:38:57.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073382977000000002 2024-09-14T22:38:57.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:57.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563385+0000", 2024-09-14T22:38:57.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5218999999999999e-05 2024-09-14T22:38:57.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:57.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563479+0000", 2024-09-14T22:38:57.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4622000000000003e-05 2024-09-14T22:38:57.514 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.515 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:57.515 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.515 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104194 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15925248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:57.515 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:38:57.515 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.961421994000005, 2024-09-14T22:38:57.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073745434999999998, 2024-09-14T22:38:57.515 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:57.515 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:57.515 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:57.515 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:57.516 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:57.516 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:57.516 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104194 2024-09-14T22:38:57.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.516 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:57.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.516 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:57.516 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:38:57.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.516 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:57.516 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:38:57.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.517 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:57.517 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489947+0000", 2024-09-14T22:38:57.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.327e-06 2024-09-14T22:38:57.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.517 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:57.517 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489953+0000", 2024-09-14T22:38:57.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9329999999999998e-06 2024-09-14T22:38:57.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.517 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:57.517 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489971+0000", 2024-09-14T22:38:57.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7994000000000001e-05 2024-09-14T22:38:57.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:57.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489985+0000", 2024-09-14T22:38:57.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4226000000000001e-05 2024-09-14T22:38:57.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:57.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563510+0000", 2024-09-14T22:38:57.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073524667000000002 2024-09-14T22:38:57.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:57.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563597+0000", 2024-09-14T22:38:57.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6550000000000003e-05 2024-09-14T22:38:57.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:57.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563691+0000", 2024-09-14T22:38:57.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4738e-05 2024-09-14T22:38:57.519 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.519 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:57.519 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.519 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104195 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:57.519 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:38:57.519 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.961363797000004, 2024-09-14T22:38:57.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073876598000000002, 2024-09-14T22:38:57.519 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:57.519 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:57.520 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:57.520 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:57.520 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:57.520 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:57.520 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104195 2024-09-14T22:38:57.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.520 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:57.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:57.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:38:57.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:57.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:38:57.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.521 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:57.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490006+0000", 2024-09-14T22:38:57.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4699999999999999e-06 2024-09-14T22:38:57.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.521 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:57.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490011+0000", 2024-09-14T22:38:57.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.366e-06 2024-09-14T22:38:57.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.521 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:57.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490046+0000", 2024-09-14T22:38:57.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5496e-05 2024-09-14T22:38:57.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:57.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490061+0000", 2024-09-14T22:38:57.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5054e-05 2024-09-14T22:38:57.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:57.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563719+0000", 2024-09-14T22:38:57.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073657412000000005 2024-09-14T22:38:57.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:57.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563792+0000", 2024-09-14T22:38:57.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3319999999999996e-05 2024-09-14T22:38:57.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:57.523 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563881+0000", 2024-09-14T22:38:57.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8480000000000007e-05 2024-09-14T22:38:57.523 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.523 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:57.523 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.523 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104196 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16187392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:57.523 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:38:57.523 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.961286647999998, 2024-09-14T22:38:57.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073997496999999995, 2024-09-14T22:38:57.524 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:57.524 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:57.524 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:57.524 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:57.524 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:57.524 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:57.524 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104196 2024-09-14T22:38:57.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.524 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:57.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:57.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:38:57.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:57.525 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:38:57.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:57.525 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490083+0000", 2024-09-14T22:38:57.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.347e-06 2024-09-14T22:38:57.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:57.525 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490089+0000", 2024-09-14T22:38:57.526 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9630000000000003e-06 2024-09-14T22:38:57.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.526 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:57.526 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490107+0000", 2024-09-14T22:38:57.526 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8859999999999999e-05 2024-09-14T22:38:57.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.526 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:57.526 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490122+0000", 2024-09-14T22:38:57.526 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.432e-05 2024-09-14T22:38:57.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.526 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:57.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563910+0000", 2024-09-14T22:38:57.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073788753999999998 2024-09-14T22:38:57.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:57.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563988+0000", 2024-09-14T22:38:57.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7862999999999996e-05 2024-09-14T22:38:57.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:57.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564079+0000", 2024-09-14T22:38:57.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0389999999999993e-05 2024-09-14T22:38:57.527 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.527 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:57.527 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.528 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104197 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16318464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:57.528 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:38:57.528 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.961195019000002, 2024-09-14T22:38:57.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074103547000000006, 2024-09-14T22:38:57.528 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:57.528 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:57.528 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:57.528 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:57.528 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:57.528 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:57.528 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104197 2024-09-14T22:38:57.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.529 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:57.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:57.529 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:38:57.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:57.529 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:38:57.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:57.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490174+0000", 2024-09-14T22:38:57.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.37e-06 2024-09-14T22:38:57.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:57.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490197+0000", 2024-09-14T22:38:57.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3017e-05 2024-09-14T22:38:57.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:57.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490219+0000", 2024-09-14T22:38:57.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.181e-05 2024-09-14T22:38:57.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:57.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490240+0000", 2024-09-14T22:38:57.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0786000000000001e-05 2024-09-14T22:38:57.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:57.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564109+0000", 2024-09-14T22:38:57.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073869031000000002 2024-09-14T22:38:57.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:57.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564186+0000", 2024-09-14T22:38:57.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6823999999999995e-05 2024-09-14T22:38:57.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.532 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:57.532 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564276+0000", 2024-09-14T22:38:57.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0709000000000002e-05 2024-09-14T22:38:57.532 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.532 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:57.532 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.532 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104198 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16449536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:57.532 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:38:57.532 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.961102783000001, 2024-09-14T22:38:57.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074167693000000007, 2024-09-14T22:38:57.532 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:57.532 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:57.532 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:57.533 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:57.533 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:57.533 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:57.533 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104198 2024-09-14T22:38:57.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.533 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:57.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:57.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:38:57.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:57.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:38:57.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.534 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:57.534 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490266+0000", 2024-09-14T22:38:57.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3599999999999999e-06 2024-09-14T22:38:57.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.534 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:57.534 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490272+0000", 2024-09-14T22:38:57.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9599999999999997e-06 2024-09-14T22:38:57.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.534 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:57.534 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490291+0000", 2024-09-14T22:38:57.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8485999999999999e-05 2024-09-14T22:38:57.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:57.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490305+0000", 2024-09-14T22:38:57.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4327e-05 2024-09-14T22:38:57.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:57.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564306+0000", 2024-09-14T22:38:57.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074000745000000007 2024-09-14T22:38:57.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:57.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564367+0000", 2024-09-14T22:38:57.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1501999999999995e-05 2024-09-14T22:38:57.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:57.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564433+0000", 2024-09-14T22:38:57.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5313000000000003e-05 2024-09-14T22:38:57.538 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.538 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:57.538 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.538 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104199 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16580608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:57.538 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:38:57.538 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.96102123, 2024-09-14T22:38:57.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074231631000000006, 2024-09-14T22:38:57.538 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:57.538 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:57.538 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:57.538 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:57.538 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:57.538 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:57.539 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104199 2024-09-14T22:38:57.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.539 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:57.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:57.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:38:57.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:57.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:38:57.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:57.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490349+0000", 2024-09-14T22:38:57.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9429999999999999e-06 2024-09-14T22:38:57.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:57.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490355+0000", 2024-09-14T22:38:57.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1670000000000004e-06 2024-09-14T22:38:57.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:57.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490374+0000", 2024-09-14T22:38:57.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9613e-05 2024-09-14T22:38:57.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:57.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490405+0000", 2024-09-14T22:38:57.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0936000000000003e-05 2024-09-14T22:38:57.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:57.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564451+0000", 2024-09-14T22:38:57.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074045266999999998 2024-09-14T22:38:57.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:57.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564494+0000", 2024-09-14T22:38:57.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3133000000000002e-05 2024-09-14T22:38:57.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:57.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564578+0000", 2024-09-14T22:38:57.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4572000000000003e-05 2024-09-14T22:38:57.542 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.542 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:57.542 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.542 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104200 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:38:57.542 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:38:57.542 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 98.959226001999994, 2024-09-14T22:38:57.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072582435000000001, 2024-09-14T22:38:57.542 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:38:57.542 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:38:57.542 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:38:57.542 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:38:57.542 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:38:57.542 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:57.543 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104200 2024-09-14T22:38:57.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.543 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:38:57.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:38:57.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:38:57.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:38:57.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:38:57.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:38:57.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:38:57.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492144+0000", 2024-09-14T22:38:57.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3070000000000001e-06 2024-09-14T22:38:57.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.544 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:38:57.544 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492155+0000", 2024-09-14T22:38:57.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0573e-05 2024-09-14T22:38:57.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.544 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:38:57.544 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492185+0000", 2024-09-14T22:38:57.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.008e-05 2024-09-14T22:38:57.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.544 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:38:57.544 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492205+0000", 2024-09-14T22:38:57.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9882999999999999e-05 2024-09-14T22:38:57.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:38:57.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564609+0000", 2024-09-14T22:38:57.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072404181999999997 2024-09-14T22:38:57.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:38:57.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564667+0000", 2024-09-14T22:38:57.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8102e-05 2024-09-14T22:38:57.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:38:57.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:38:57.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:38:57.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564724+0000", 2024-09-14T22:38:57.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7308000000000001e-05 2024-09-14T22:38:57.545 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.546 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:57.546 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.546 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:38:57.546 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:38:57.546 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:38:57.547 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:38:57.548 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:57.542+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:57.648 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:57.644+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:57.683 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:57.684 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:57.684 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:38:57.684 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:57.698 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:38:57.749 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:57.742+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:38:57.850 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:57.848+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:57.889 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:57.889 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:57.889 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:38:57.889 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:38:57.889 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:57.904 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:38:57.979 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.9.4 2024-09-14T22:38:57.994 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:57.992+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:58.071 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:58.066+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:58.077 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:58.077 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:38:58.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:38:58.078 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:38:58.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.078 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144987 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3424256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:58.078 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549297+0000", 2024-09-14T22:38:58.078 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.524279653999997, 2024-09-14T22:38:58.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.079931556000000001, 2024-09-14T22:38:58.078 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.078 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.078 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.078 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.078 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.078 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.078 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144987 2024-09-14T22:38:58.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.079 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.079 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.079 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549297+0000", 2024-09-14T22:38:58.079 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.079 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.079 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549297+0000", 2024-09-14T22:38:58.079 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.079 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.079 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549298+0000", 2024-09-14T22:38:58.079 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2699999999999999e-07 2024-09-14T22:38:58.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549301+0000", 2024-09-14T22:38:58.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2959999999999999e-06 2024-09-14T22:38:58.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549314+0000", 2024-09-14T22:38:58.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2588999999999999e-05 2024-09-14T22:38:58.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549323+0000", 2024-09-14T22:38:58.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.927e-06 2024-09-14T22:38:58.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629038+0000", 2024-09-14T22:38:58.081 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.079714220000000002 2024-09-14T22:38:58.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629116+0000", 2024-09-14T22:38:58.081 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7966000000000005e-05 2024-09-14T22:38:58.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629228+0000", 2024-09-14T22:38:58.082 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112831 2024-09-14T22:38:58.082 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.082 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.082 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.082 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144988 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3325952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:58.082 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549335+0000", 2024-09-14T22:38:58.082 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.524241429, 2024-09-14T22:38:58.082 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080179116999999994, 2024-09-14T22:38:58.082 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.082 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.082 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.082 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.083 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.083 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.083 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144988 2024-09-14T22:38:58.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.083 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.083 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.083 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549335+0000", 2024-09-14T22:38:58.083 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.083 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.083 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549335+0000", 2024-09-14T22:38:58.083 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.084 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.084 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549336+0000", 2024-09-14T22:38:58.084 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7999999999999995e-07 2024-09-14T22:38:58.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.084 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.084 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549339+0000", 2024-09-14T22:38:58.084 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.836e-06 2024-09-14T22:38:58.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.084 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.084 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549352+0000", 2024-09-14T22:38:58.084 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3069e-05 2024-09-14T22:38:58.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.085 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.085 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549359+0000", 2024-09-14T22:38:58.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.357e-06 2024-09-14T22:38:58.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.085 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.085 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629288+0000", 2024-09-14T22:38:58.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.079928492000000004 2024-09-14T22:38:58.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.085 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.085 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629412+0000", 2024-09-14T22:38:58.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012413699999999999 2024-09-14T22:38:58.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.086 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629514+0000", 2024-09-14T22:38:58.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000102546 2024-09-14T22:38:58.086 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.086 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.086 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.086 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144989 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3522560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:58.086 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:38:58.086 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.524207020999995, 2024-09-14T22:38:58.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080393122999999997, 2024-09-14T22:38:58.087 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.087 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.087 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.087 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.087 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.087 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.087 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144989 2024-09-14T22:38:58.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.087 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.087 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:38:58.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.088 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:38:58.088 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.088 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.088 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:38:58.088 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3600000000000003e-07 2024-09-14T22:38:58.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.088 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.088 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549373+0000", 2024-09-14T22:38:58.088 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8169999999999999e-06 2024-09-14T22:38:58.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.088 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.089 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549385+0000", 2024-09-14T22:38:58.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1803e-05 2024-09-14T22:38:58.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.089 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549394+0000", 2024-09-14T22:38:58.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4290000000000007e-06 2024-09-14T22:38:58.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.089 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629576+0000", 2024-09-14T22:38:58.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080181551000000004 2024-09-14T22:38:58.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629654+0000", 2024-09-14T22:38:58.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7633999999999996e-05 2024-09-14T22:38:58.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629763+0000", 2024-09-14T22:38:58.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109153 2024-09-14T22:38:58.090 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.090 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.090 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.090 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144990 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:58.090 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:38:58.090 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.524163997000002, 2024-09-14T22:38:58.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080614496999999993, 2024-09-14T22:38:58.091 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.091 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.091 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.091 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.091 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.091 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.091 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144990 2024-09-14T22:38:58.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.091 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:38:58.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.092 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.092 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:38:58.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.092 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.092 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:38:58.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0999999999999997e-07 2024-09-14T22:38:58.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.092 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549419+0000", 2024-09-14T22:38:58.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5339999999999999e-06 2024-09-14T22:38:58.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.093 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549439+0000", 2024-09-14T22:38:58.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0035e-05 2024-09-14T22:38:58.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.093 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549449+0000", 2024-09-14T22:38:58.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0429e-05 2024-09-14T22:38:58.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629815+0000", 2024-09-14T22:38:58.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080365467999999995 2024-09-14T22:38:58.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629922+0000", 2024-09-14T22:38:58.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106914 2024-09-14T22:38:58.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630027+0000", 2024-09-14T22:38:58.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105207 2024-09-14T22:38:58.094 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.094 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.094 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.095 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144991 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 2998272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:58.095 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549463+0000", 2024-09-14T22:38:58.095 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.524113693000004, 2024-09-14T22:38:58.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080847637, 2024-09-14T22:38:58.095 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.095 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.095 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.095 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.095 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.095 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.095 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144991 2024-09-14T22:38:58.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.095 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549463+0000", 2024-09-14T22:38:58.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549463+0000", 2024-09-14T22:38:58.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549464+0000", 2024-09-14T22:38:58.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9999999999999997e-07 2024-09-14T22:38:58.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.097 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549467+0000", 2024-09-14T22:38:58.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0800000000000002e-06 2024-09-14T22:38:58.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.097 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549478+0000", 2024-09-14T22:38:58.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1813e-05 2024-09-14T22:38:58.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.097 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549488+0000", 2024-09-14T22:38:58.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7030000000000001e-06 2024-09-14T22:38:58.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.098 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630090+0000", 2024-09-14T22:38:58.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080601962999999999 2024-09-14T22:38:58.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.098 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630187+0000", 2024-09-14T22:38:58.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6552000000000007e-05 2024-09-14T22:38:58.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.098 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630311+0000", 2024-09-14T22:38:58.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000123826 2024-09-14T22:38:58.098 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.098 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.099 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.099 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144992 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3620864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:58.099 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:38:58.099 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.524077884999997, 2024-09-14T22:38:58.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081097875999999999, 2024-09-14T22:38:58.099 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.099 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.099 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.099 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.099 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.099 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.099 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144992 2024-09-14T22:38:58.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.099 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.100 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:38:58.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.100 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:38:58.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.100 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:38:58.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5400000000000001e-07 2024-09-14T22:38:58.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549502+0000", 2024-09-14T22:38:58.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7889999999999999e-06 2024-09-14T22:38:58.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549513+0000", 2024-09-14T22:38:58.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1204e-05 2024-09-14T22:38:58.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549523+0000", 2024-09-14T22:38:58.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5990000000000008e-06 2024-09-14T22:38:58.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.102 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630362+0000", 2024-09-14T22:38:58.102 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080839272000000004 2024-09-14T22:38:58.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.102 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630469+0000", 2024-09-14T22:38:58.102 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107285 2024-09-14T22:38:58.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.102 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630597+0000", 2024-09-14T22:38:58.102 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012707300000000001 2024-09-14T22:38:58.102 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.102 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.103 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.103 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144993 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:58.103 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:38:58.103 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.524044809000003, 2024-09-14T22:38:58.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081338411999999999, 2024-09-14T22:38:58.103 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.103 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.103 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.103 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.103 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.103 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.103 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144993 2024-09-14T22:38:58.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.104 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:38:58.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:38:58.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.105 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:38:58.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5300000000000004e-07 2024-09-14T22:38:58.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.105 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.105 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549544+0000", 2024-09-14T22:38:58.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1912e-05 2024-09-14T22:38:58.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.105 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.105 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549557+0000", 2024-09-14T22:38:58.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2413e-05 2024-09-14T22:38:58.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549567+0000", 2024-09-14T22:38:58.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8660000000000001e-06 2024-09-14T22:38:58.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630651+0000", 2024-09-14T22:38:58.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081084229999999993 2024-09-14T22:38:58.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630729+0000", 2024-09-14T22:38:58.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8265999999999999e-05 2024-09-14T22:38:58.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630870+0000", 2024-09-14T22:38:58.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014107200000000001 2024-09-14T22:38:58.107 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.107 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.107 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.107 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144994 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:58.107 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:38:58.107 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.523997842, 2024-09-14T22:38:58.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081547958000000004, 2024-09-14T22:38:58.107 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.107 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.107 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.107 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.108 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.108 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.108 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144994 2024-09-14T22:38:58.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.108 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:38:58.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:38:58.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:38:58.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6599999999999996e-07 2024-09-14T22:38:58.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549582+0000", 2024-09-14T22:38:58.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.004e-06 2024-09-14T22:38:58.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549594+0000", 2024-09-14T22:38:58.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1459e-05 2024-09-14T22:38:58.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549603+0000", 2024-09-14T22:38:58.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6129999999999993e-06 2024-09-14T22:38:58.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630932+0000", 2024-09-14T22:38:58.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081328558999999995 2024-09-14T22:38:58.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.111 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.111 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631027+0000", 2024-09-14T22:38:58.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4742999999999997e-05 2024-09-14T22:38:58.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.111 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.111 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631127+0000", 2024-09-14T22:38:58.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9914000000000006e-05 2024-09-14T22:38:58.111 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.111 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.111 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.111 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144995 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3391488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:58.112 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549615+0000", 2024-09-14T22:38:58.112 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.523961704000001, 2024-09-14T22:38:58.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081734251999999993, 2024-09-14T22:38:58.112 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.112 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.112 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.112 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.112 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.112 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.112 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144995 2024-09-14T22:38:58.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.112 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549615+0000", 2024-09-14T22:38:58.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549615+0000", 2024-09-14T22:38:58.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549616+0000", 2024-09-14T22:38:58.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9299999999999997e-07 2024-09-14T22:38:58.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549618+0000", 2024-09-14T22:38:58.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7269999999999999e-06 2024-09-14T22:38:58.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549630+0000", 2024-09-14T22:38:58.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1313e-05 2024-09-14T22:38:58.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549639+0000", 2024-09-14T22:38:58.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4159999999999993e-06 2024-09-14T22:38:58.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631183+0000", 2024-09-14T22:38:58.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081543633000000004 2024-09-14T22:38:58.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631252+0000", 2024-09-14T22:38:58.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9567999999999995e-05 2024-09-14T22:38:58.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631349+0000", 2024-09-14T22:38:58.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6801999999999999e-05 2024-09-14T22:38:58.115 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.115 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.115 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.116 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144996 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3194880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:58.116 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:38:58.116 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.523926025999998, 2024-09-14T22:38:58.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081947213000000005, 2024-09-14T22:38:58.116 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.116 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.116 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.116 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.116 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.116 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.116 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144996 2024-09-14T22:38:58.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.116 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:38:58.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:38:58.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:38:58.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3e-07 2024-09-14T22:38:58.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549654+0000", 2024-09-14T22:38:58.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6900000000000001e-06 2024-09-14T22:38:58.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549665+0000", 2024-09-14T22:38:58.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1392999999999999e-05 2024-09-14T22:38:58.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549675+0000", 2024-09-14T22:38:58.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5459999999999997e-06 2024-09-14T22:38:58.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631401+0000", 2024-09-14T22:38:58.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081726036000000002 2024-09-14T22:38:58.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631492+0000", 2024-09-14T22:38:58.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0886999999999999e-05 2024-09-14T22:38:58.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631598+0000", 2024-09-14T22:38:58.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106031 2024-09-14T22:38:58.120 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.120 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.120 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.120 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144997 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:58.120 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549692+0000", 2024-09-14T22:38:58.120 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.523884362000004, 2024-09-14T22:38:58.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082149925999999998, 2024-09-14T22:38:58.120 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.120 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.120 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.120 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.120 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.121 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.121 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144997 2024-09-14T22:38:58.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.121 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549692+0000", 2024-09-14T22:38:58.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549692+0000", 2024-09-14T22:38:58.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.122 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549693+0000", 2024-09-14T22:38:58.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.540000000000001e-07 2024-09-14T22:38:58.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.122 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549698+0000", 2024-09-14T22:38:58.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1460000000000004e-06 2024-09-14T22:38:58.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.122 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549715+0000", 2024-09-14T22:38:58.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6983e-05 2024-09-14T22:38:58.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549728+0000", 2024-09-14T22:38:58.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2326e-05 2024-09-14T22:38:58.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631649+0000", 2024-09-14T22:38:58.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081921830000000001 2024-09-14T22:38:58.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631726+0000", 2024-09-14T22:38:58.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6759000000000001e-05 2024-09-14T22:38:58.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631842+0000", 2024-09-14T22:38:58.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011592800000000001 2024-09-14T22:38:58.124 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.124 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.124 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.124 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144998 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 2965504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:58.124 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549741+0000", 2024-09-14T22:38:58.124 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.523835070000004, 2024-09-14T22:38:58.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082394832000000001, 2024-09-14T22:38:58.124 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.125 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.125 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.125 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.125 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.125 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.125 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144998 2024-09-14T22:38:58.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.125 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.125 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549741+0000", 2024-09-14T22:38:58.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549741+0000", 2024-09-14T22:38:58.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549742+0000", 2024-09-14T22:38:58.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.06e-07 2024-09-14T22:38:58.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549757+0000", 2024-09-14T22:38:58.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4839e-05 2024-09-14T22:38:58.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549770+0000", 2024-09-14T22:38:58.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2879e-05 2024-09-14T22:38:58.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549780+0000", 2024-09-14T22:38:58.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0304e-05 2024-09-14T22:38:58.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631904+0000", 2024-09-14T22:38:58.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082124168999999997 2024-09-14T22:38:58.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631995+0000", 2024-09-14T22:38:58.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0993000000000001e-05 2024-09-14T22:38:58.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632136+0000", 2024-09-14T22:38:58.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014094199999999999 2024-09-14T22:38:58.128 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.128 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.128 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.128 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144999 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3588096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:58.129 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:38:58.129 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.523784860000006, 2024-09-14T22:38:58.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082593742999999997, 2024-09-14T22:38:58.129 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.129 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.129 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.129 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.129 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.129 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.129 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144999 2024-09-14T22:38:58.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.129 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:38:58.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:38:58.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:38:58.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1999999999999999e-07 2024-09-14T22:38:58.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549795+0000", 2024-09-14T22:38:58.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9569999999999998e-06 2024-09-14T22:38:58.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549807+0000", 2024-09-14T22:38:58.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1691999999999999e-05 2024-09-14T22:38:58.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549817+0000", 2024-09-14T22:38:58.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5100000000000004e-06 2024-09-14T22:38:58.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632190+0000", 2024-09-14T22:38:58.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082373374999999999 2024-09-14T22:38:58.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632264+0000", 2024-09-14T22:38:58.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3972000000000003e-05 2024-09-14T22:38:58.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632385+0000", 2024-09-14T22:38:58.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121517 2024-09-14T22:38:58.132 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.133 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.133 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.133 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145000 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3063808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:58.133 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549828+0000", 2024-09-14T22:38:58.133 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.523748581999996, 2024-09-14T22:38:58.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082860235000000004, 2024-09-14T22:38:58.133 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.133 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.133 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.133 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.133 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.133 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.133 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145000 2024-09-14T22:38:58.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.134 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.134 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549828+0000", 2024-09-14T22:38:58.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.134 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549828+0000", 2024-09-14T22:38:58.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.134 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549829+0000", 2024-09-14T22:38:58.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3300000000000002e-07 2024-09-14T22:38:58.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549831+0000", 2024-09-14T22:38:58.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6740000000000001e-06 2024-09-14T22:38:58.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549842+0000", 2024-09-14T22:38:58.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0316000000000001e-05 2024-09-14T22:38:58.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549851+0000", 2024-09-14T22:38:58.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7229999999999999e-06 2024-09-14T22:38:58.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632448+0000", 2024-09-14T22:38:58.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082596648999999994 2024-09-14T22:38:58.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632568+0000", 2024-09-14T22:38:58.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119592 2024-09-14T22:38:58.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632688+0000", 2024-09-14T22:38:58.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120648 2024-09-14T22:38:58.137 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.137 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.137 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.137 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145001 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3162112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:58.137 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549863+0000", 2024-09-14T22:38:58.137 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.523713353999995, 2024-09-14T22:38:58.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083105534999999994, 2024-09-14T22:38:58.137 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.137 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.137 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.137 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.137 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.137 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.138 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145001 2024-09-14T22:38:58.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.138 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.138 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.138 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549863+0000", 2024-09-14T22:38:58.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.138 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.138 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549863+0000", 2024-09-14T22:38:58.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.139 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.139 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549864+0000", 2024-09-14T22:38:58.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4300000000000003e-07 2024-09-14T22:38:58.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.139 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.139 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549867+0000", 2024-09-14T22:38:58.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.757e-06 2024-09-14T22:38:58.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.139 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.139 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549878+0000", 2024-09-14T22:38:58.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1345999999999999e-05 2024-09-14T22:38:58.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549888+0000", 2024-09-14T22:38:58.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6460000000000005e-06 2024-09-14T22:38:58.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632754+0000", 2024-09-14T22:38:58.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082866258999999998 2024-09-14T22:38:58.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632841+0000", 2024-09-14T22:38:58.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6971000000000005e-05 2024-09-14T22:38:58.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632969+0000", 2024-09-14T22:38:58.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012791300000000001 2024-09-14T22:38:58.141 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.141 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.141 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.141 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145002 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3260416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:58.141 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549933+0000", 2024-09-14T22:38:58.141 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.523643851000003, 2024-09-14T22:38:58.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083291027000000004, 2024-09-14T22:38:58.141 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.142 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.142 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.142 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.142 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.142 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.142 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145002 2024-09-14T22:38:58.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.142 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549933+0000", 2024-09-14T22:38:58.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549933+0000", 2024-09-14T22:38:58.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549934+0000", 2024-09-14T22:38:58.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8700000000000004e-07 2024-09-14T22:38:58.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549938+0000", 2024-09-14T22:38:58.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5029999999999999e-06 2024-09-14T22:38:58.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549952+0000", 2024-09-14T22:38:58.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4382999999999999e-05 2024-09-14T22:38:58.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549963+0000", 2024-09-14T22:38:58.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0665999999999999e-05 2024-09-14T22:38:58.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633025+0000", 2024-09-14T22:38:58.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083061727000000002 2024-09-14T22:38:58.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633117+0000", 2024-09-14T22:38:58.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1812999999999997e-05 2024-09-14T22:38:58.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633224+0000", 2024-09-14T22:38:58.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107048 2024-09-14T22:38:58.145 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.145 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.145 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.145 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145003 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3457024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:58.146 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549985+0000", 2024-09-14T22:38:58.146 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.523591433000007, 2024-09-14T22:38:58.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083465871999999997, 2024-09-14T22:38:58.146 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.146 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.146 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.146 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.146 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.146 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.146 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145003 2024-09-14T22:38:58.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.146 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.148 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549985+0000", 2024-09-14T22:38:58.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.148 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549985+0000", 2024-09-14T22:38:58.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549986+0000", 2024-09-14T22:38:58.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7299999999999995e-07 2024-09-14T22:38:58.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549989+0000", 2024-09-14T22:38:58.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8169999999999999e-06 2024-09-14T22:38:58.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549998+0000", 2024-09-14T22:38:58.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4290000000000007e-06 2024-09-14T22:38:58.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550009+0000", 2024-09-14T22:38:58.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0633e-05 2024-09-14T22:38:58.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633275+0000", 2024-09-14T22:38:58.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083266397000000006 2024-09-14T22:38:58.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633353+0000", 2024-09-14T22:38:58.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8129999999999999e-05 2024-09-14T22:38:58.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633451+0000", 2024-09-14T22:38:58.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7793000000000004e-05 2024-09-14T22:38:58.151 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.151 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.151 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.151 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145004 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3358720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:58.151 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.550029+0000", 2024-09-14T22:38:58.151 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.523547862000001, 2024-09-14T22:38:58.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083662089999999995, 2024-09-14T22:38:58.151 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.152 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.152 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.152 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.152 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.152 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.152 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145004 2024-09-14T22:38:58.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.152 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.152 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550029+0000", 2024-09-14T22:38:58.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550029+0000", 2024-09-14T22:38:58.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550030+0000", 2024-09-14T22:38:58.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.33e-07 2024-09-14T22:38:58.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550035+0000", 2024-09-14T22:38:58.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3430000000000003e-06 2024-09-14T22:38:58.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550053+0000", 2024-09-14T22:38:58.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8485e-05 2024-09-14T22:38:58.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550064+0000", 2024-09-14T22:38:58.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0264e-05 2024-09-14T22:38:58.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633502+0000", 2024-09-14T22:38:58.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083438498999999999 2024-09-14T22:38:58.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633592+0000", 2024-09-14T22:38:58.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.022e-05 2024-09-14T22:38:58.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633691+0000", 2024-09-14T22:38:58.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8345999999999997e-05 2024-09-14T22:38:58.155 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.155 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.155 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.155 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145005 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3555328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:58.155 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.550077+0000", 2024-09-14T22:38:58.156 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.523499401999999, 2024-09-14T22:38:58.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083854340999999999, 2024-09-14T22:38:58.156 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.156 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.156 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.156 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.156 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.156 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.156 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145005 2024-09-14T22:38:58.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.156 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.157 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550077+0000", 2024-09-14T22:38:58.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.157 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.157 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550077+0000", 2024-09-14T22:38:58.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.157 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.157 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550078+0000", 2024-09-14T22:38:58.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8700000000000005e-07 2024-09-14T22:38:58.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550101+0000", 2024-09-14T22:38:58.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3632e-05 2024-09-14T22:38:58.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550115+0000", 2024-09-14T22:38:58.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3893000000000001e-05 2024-09-14T22:38:58.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550126+0000", 2024-09-14T22:38:58.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0426e-05 2024-09-14T22:38:58.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633742+0000", 2024-09-14T22:38:58.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083616633999999995 2024-09-14T22:38:58.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633816+0000", 2024-09-14T22:38:58.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3690999999999996e-05 2024-09-14T22:38:58.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633932+0000", 2024-09-14T22:38:58.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011537800000000001 2024-09-14T22:38:58.160 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.160 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.160 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.160 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145006 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3653632~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:38:58.160 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.550236+0000", 2024-09-14T22:38:58.160 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.523340895999993, 2024-09-14T22:38:58.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083924027999999998, 2024-09-14T22:38:58.160 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.160 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.160 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.161 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.161 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.161 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.161 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145006 2024-09-14T22:38:58.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.161 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.161 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.161 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550236+0000", 2024-09-14T22:38:58.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.161 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.161 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550236+0000", 2024-09-14T22:38:58.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550237+0000", 2024-09-14T22:38:58.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2929999999999999e-06 2024-09-14T22:38:58.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550243+0000", 2024-09-14T22:38:58.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0390000000000001e-06 2024-09-14T22:38:58.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550256+0000", 2024-09-14T22:38:58.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3393e-05 2024-09-14T22:38:58.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550271+0000", 2024-09-14T22:38:58.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4166e-05 2024-09-14T22:38:58.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633974+0000", 2024-09-14T22:38:58.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083703825999999995 2024-09-14T22:38:58.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.634047+0000", 2024-09-14T22:38:58.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2935999999999994e-05 2024-09-14T22:38:58.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.634160+0000", 2024-09-14T22:38:58.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011237499999999999 2024-09-14T22:38:58.164 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.164 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.164 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.164 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.164 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.164 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:58.165 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:38:58.166 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:58.160+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:58.266 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:58.258+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:58.340 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:58.340 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:58.340 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:38:58.340 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:58.354 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:38:58.367 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:38:58.362+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:38:58.468 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:58.464+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:58.475 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:58.476 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:58.476 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:38:58.476 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:38:58.476 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:58.490 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:38:58.578 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:58.576+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:58.658 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:58.658 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:38:58.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:38:58.658 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:38:58.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.659 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:122960 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 2347008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:58.659 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:41.998325+0000", 2024-09-14T22:38:58.659 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 76.656182091000005, 2024-09-14T22:38:58.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056570400999999999, 2024-09-14T22:38:58.659 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.659 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.659 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.659 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.659 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.659 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.659 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 122960 2024-09-14T22:38:58.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.659 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998325+0000", 2024-09-14T22:38:58.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998325+0000", 2024-09-14T22:38:58.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998326+0000", 2024-09-14T22:38:58.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9299999999999998e-07 2024-09-14T22:38:58.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998329+0000", 2024-09-14T22:38:58.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1370000000000002e-06 2024-09-14T22:38:58.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998340+0000", 2024-09-14T22:38:58.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1039999999999999e-05 2024-09-14T22:38:58.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998348+0000", 2024-09-14T22:38:58.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1920000000000005e-06 2024-09-14T22:38:58.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054700+0000", 2024-09-14T22:38:58.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056351714999999997 2024-09-14T22:38:58.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054771+0000", 2024-09-14T22:38:58.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1009999999999999e-05 2024-09-14T22:38:58.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054896+0000", 2024-09-14T22:38:58.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000124714 2024-09-14T22:38:58.664 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.664 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.664 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.664 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:122961 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 2351104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:58.665 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:41.998357+0000", 2024-09-14T22:38:58.665 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 76.656150135999994, 2024-09-14T22:38:58.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056974912000000003, 2024-09-14T22:38:58.665 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.665 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.665 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.665 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.665 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.665 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.665 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 122961 2024-09-14T22:38:58.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.665 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998357+0000", 2024-09-14T22:38:58.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.666 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998357+0000", 2024-09-14T22:38:58.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.666 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998358+0000", 2024-09-14T22:38:58.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9699999999999996e-07 2024-09-14T22:38:58.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998361+0000", 2024-09-14T22:38:58.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1370000000000002e-06 2024-09-14T22:38:58.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998371+0000", 2024-09-14T22:38:58.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5789999999999993e-06 2024-09-14T22:38:58.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998379+0000", 2024-09-14T22:38:58.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0129999999999993e-06 2024-09-14T22:38:58.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054998+0000", 2024-09-14T22:38:58.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056618961000000002 2024-09-14T22:38:58.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.055126+0000", 2024-09-14T22:38:58.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012795100000000001 2024-09-14T22:38:58.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.055332+0000", 2024-09-14T22:38:58.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020667400000000001 2024-09-14T22:38:58.670 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.670 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.670 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.670 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123259 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 741376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:58.670 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:38:58.670 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 76.642424618999996, 2024-09-14T22:38:58.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056672581999999999, 2024-09-14T22:38:58.671 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.671 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.671 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.671 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.671 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.671 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.671 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123259 2024-09-14T22:38:58.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.671 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:38:58.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:38:58.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:38:58.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1600000000000001e-07 2024-09-14T22:38:58.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012086+0000", 2024-09-14T22:38:58.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8499999999999998e-06 2024-09-14T22:38:58.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012097+0000", 2024-09-14T22:38:58.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0943e-05 2024-09-14T22:38:58.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012105+0000", 2024-09-14T22:38:58.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0830000000000004e-06 2024-09-14T22:38:58.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068584+0000", 2024-09-14T22:38:58.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056478387999999997 2024-09-14T22:38:58.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068644+0000", 2024-09-14T22:38:58.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9984e-05 2024-09-14T22:38:58.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068755+0000", 2024-09-14T22:38:58.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111718 2024-09-14T22:38:58.674 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.674 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.674 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.675 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123260 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 778240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:58.675 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012118+0000", 2024-09-14T22:38:58.675 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 76.642389394000006, 2024-09-14T22:38:58.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056825887999999998, 2024-09-14T22:38:58.675 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.675 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.675 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.675 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.675 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.675 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.675 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123260 2024-09-14T22:38:58.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.676 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012118+0000", 2024-09-14T22:38:58.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012118+0000", 2024-09-14T22:38:58.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012119+0000", 2024-09-14T22:38:58.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.13e-07 2024-09-14T22:38:58.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012130+0000", 2024-09-14T22:38:58.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1552999999999999e-05 2024-09-14T22:38:58.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012141+0000", 2024-09-14T22:38:58.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.03e-05 2024-09-14T22:38:58.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012149+0000", 2024-09-14T22:38:58.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1419999999999993e-06 2024-09-14T22:38:58.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068800+0000", 2024-09-14T22:38:58.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056651574000000003 2024-09-14T22:38:58.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068858+0000", 2024-09-14T22:38:58.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8047000000000002e-05 2024-09-14T22:38:58.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068944+0000", 2024-09-14T22:38:58.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5659000000000001e-05 2024-09-14T22:38:58.679 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.679 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.679 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.679 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123261 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 761856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:58.679 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012161+0000", 2024-09-14T22:38:58.679 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 76.642345993000006, 2024-09-14T22:38:58.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057113753000000003, 2024-09-14T22:38:58.679 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.679 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.679 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.680 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.680 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.680 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.680 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123261 2024-09-14T22:38:58.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.680 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012161+0000", 2024-09-14T22:38:58.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012161+0000", 2024-09-14T22:38:58.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.681 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.681 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012162+0000", 2024-09-14T22:38:58.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4399999999999999e-07 2024-09-14T22:38:58.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.681 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.681 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012177+0000", 2024-09-14T22:38:58.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5099000000000001e-05 2024-09-14T22:38:58.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.681 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012191+0000", 2024-09-14T22:38:58.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3631999999999999e-05 2024-09-14T22:38:58.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012199+0000", 2024-09-14T22:38:58.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5769999999999994e-06 2024-09-14T22:38:58.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069004+0000", 2024-09-14T22:38:58.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056804504999999998 2024-09-14T22:38:58.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069137+0000", 2024-09-14T22:38:58.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013302600000000001 2024-09-14T22:38:58.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069275+0000", 2024-09-14T22:38:58.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013816999999999999 2024-09-14T22:38:58.683 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.683 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.683 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.683 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123262 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 724992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:58.684 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012224+0000", 2024-09-14T22:38:58.684 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 76.642283546000002, 2024-09-14T22:38:58.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057417959999999997, 2024-09-14T22:38:58.684 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.684 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.684 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.684 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.684 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.684 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.684 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123262 2024-09-14T22:38:58.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.684 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012224+0000", 2024-09-14T22:38:58.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012224+0000", 2024-09-14T22:38:58.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012225+0000", 2024-09-14T22:38:58.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4000000000000001e-07 2024-09-14T22:38:58.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012228+0000", 2024-09-14T22:38:58.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5729999999999999e-06 2024-09-14T22:38:58.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012241+0000", 2024-09-14T22:38:58.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3210000000000001e-05 2024-09-14T22:38:58.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012255+0000", 2024-09-14T22:38:58.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3852000000000001e-05 2024-09-14T22:38:58.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069341+0000", 2024-09-14T22:38:58.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057086139000000001 2024-09-14T22:38:58.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069477+0000", 2024-09-14T22:38:58.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013595 2024-09-14T22:38:58.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069642+0000", 2024-09-14T22:38:58.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016459600000000001 2024-09-14T22:38:58.687 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.688 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.688 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.688 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123263 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 782336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:58.688 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012265+0000", 2024-09-14T22:38:58.688 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 76.642242003999996, 2024-09-14T22:38:58.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057665095, 2024-09-14T22:38:58.688 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.688 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.688 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.688 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.688 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.689 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.689 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123263 2024-09-14T22:38:58.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.689 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.689 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012265+0000", 2024-09-14T22:38:58.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.689 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012265+0000", 2024-09-14T22:38:58.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012266+0000", 2024-09-14T22:38:58.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8999999999999996e-07 2024-09-14T22:38:58.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012269+0000", 2024-09-14T22:38:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.796e-06 2024-09-14T22:38:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012279+0000", 2024-09-14T22:38:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7389999999999994e-06 2024-09-14T22:38:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012286+0000", 2024-09-14T22:38:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8199999999999997e-06 2024-09-14T22:38:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069705+0000", 2024-09-14T22:38:58.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057418590999999998 2024-09-14T22:38:58.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.692 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069799+0000", 2024-09-14T22:38:58.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4419999999999994e-05 2024-09-14T22:38:58.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.692 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069931+0000", 2024-09-14T22:38:58.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013113900000000001 2024-09-14T22:38:58.692 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.692 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123264 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 745472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012295+0000", 2024-09-14T22:38:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 76.642212002999997, 2024-09-14T22:38:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057959498999999998, 2024-09-14T22:38:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123264 2024-09-14T22:38:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.694 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.694 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.694 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012295+0000", 2024-09-14T22:38:58.694 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.694 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.694 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012295+0000", 2024-09-14T22:38:58.694 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.694 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.694 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012296+0000", 2024-09-14T22:38:58.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9699999999999996e-07 2024-09-14T22:38:58.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012299+0000", 2024-09-14T22:38:58.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.79e-06 2024-09-14T22:38:58.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012320+0000", 2024-09-14T22:38:58.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1242000000000001e-05 2024-09-14T22:38:58.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012338+0000", 2024-09-14T22:38:58.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7768999999999999e-05 2024-09-14T22:38:58.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069995+0000", 2024-09-14T22:38:58.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057657 2024-09-14T22:38:58.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070132+0000", 2024-09-14T22:38:58.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013727800000000001 2024-09-14T22:38:58.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070255+0000", 2024-09-14T22:38:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012282300000000001 2024-09-14T22:38:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123265 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1056768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012348+0000", 2024-09-14T22:38:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 76.642159047999996, 2024-09-14T22:38:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058145087999999998, 2024-09-14T22:38:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.698 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.698 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.698 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.698 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.698 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123265 2024-09-14T22:38:58.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.698 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.698 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.698 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012348+0000", 2024-09-14T22:38:58.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.698 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012348+0000", 2024-09-14T22:38:58.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012349+0000", 2024-09-14T22:38:58.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6599999999999996e-07 2024-09-14T22:38:58.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012352+0000", 2024-09-14T22:38:58.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3139999999999999e-06 2024-09-14T22:38:58.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012365+0000", 2024-09-14T22:38:58.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2625e-05 2024-09-14T22:38:58.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012373+0000", 2024-09-14T22:38:58.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2470000000000008e-06 2024-09-14T22:38:58.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070286+0000", 2024-09-14T22:38:58.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057912547000000002 2024-09-14T22:38:58.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070366+0000", 2024-09-14T22:38:58.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0468000000000003e-05 2024-09-14T22:38:58.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070493+0000", 2024-09-14T22:38:58.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012722100000000001 2024-09-14T22:38:58.701 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.701 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.701 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.702 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123266 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 765952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:58.702 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:38:58.702 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 76.642124800000005, 2024-09-14T22:38:58.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058386949, 2024-09-14T22:38:58.702 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.702 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.702 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.702 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.702 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.702 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.702 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123266 2024-09-14T22:38:58.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.702 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.703 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:38:58.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.703 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:38:58.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:38:58.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9299999999999998e-07 2024-09-14T22:38:58.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012386+0000", 2024-09-14T22:38:58.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8729999999999999e-06 2024-09-14T22:38:58.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012406+0000", 2024-09-14T22:38:58.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0103e-05 2024-09-14T22:38:58.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012419+0000", 2024-09-14T22:38:58.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.311e-05 2024-09-14T22:38:58.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070550+0000", 2024-09-14T22:38:58.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058130902999999998 2024-09-14T22:38:58.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070650+0000", 2024-09-14T22:38:58.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010005000000000001 2024-09-14T22:38:58.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070770+0000", 2024-09-14T22:38:58.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119317 2024-09-14T22:38:58.706 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.706 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.706 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.706 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123267 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1077248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:58.706 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012429+0000", 2024-09-14T22:38:58.707 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 76.642078885999993, 2024-09-14T22:38:58.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058620343999999998, 2024-09-14T22:38:58.707 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.707 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.707 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.707 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.707 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.707 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.707 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123267 2024-09-14T22:38:58.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.707 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012429+0000", 2024-09-14T22:38:58.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012429+0000", 2024-09-14T22:38:58.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013488+0000", 2024-09-14T22:38:58.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001059061 2024-09-14T22:38:58.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013499+0000", 2024-09-14T22:38:58.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1348999999999999e-05 2024-09-14T22:38:58.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013534+0000", 2024-09-14T22:38:58.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5206000000000001e-05 2024-09-14T22:38:58.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013558+0000", 2024-09-14T22:38:58.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4324999999999998e-05 2024-09-14T22:38:58.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070834+0000", 2024-09-14T22:38:58.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057275110999999997 2024-09-14T22:38:58.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070937+0000", 2024-09-14T22:38:58.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010306099999999999 2024-09-14T22:38:58.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071049+0000", 2024-09-14T22:38:58.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112231 2024-09-14T22:38:58.712 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.712 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.712 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.713 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123268 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1097728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:58.713 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013593+0000", 2024-09-14T22:38:58.713 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 76.640914163000005, 2024-09-14T22:38:58.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057751793000000003, 2024-09-14T22:38:58.713 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.714 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.714 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.714 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.714 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.714 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.714 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123268 2024-09-14T22:38:58.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.714 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013593+0000", 2024-09-14T22:38:58.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013593+0000", 2024-09-14T22:38:58.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013595+0000", 2024-09-14T22:38:58.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.59e-06 2024-09-14T22:38:58.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.717 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013602+0000", 2024-09-14T22:38:58.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3560000000000004e-06 2024-09-14T22:38:58.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013625+0000", 2024-09-14T22:38:58.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2419e-05 2024-09-14T22:38:58.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013650+0000", 2024-09-14T22:38:58.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5151999999999999e-05 2024-09-14T22:38:58.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071096+0000", 2024-09-14T22:38:58.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057445877999999999 2024-09-14T22:38:58.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071212+0000", 2024-09-14T22:38:58.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116215 2024-09-14T22:38:58.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071345+0000", 2024-09-14T22:38:58.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013318299999999999 2024-09-14T22:38:58.719 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.719 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.720 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.720 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123269 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1060864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:58.720 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013676+0000", 2024-09-14T22:38:58.720 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 76.640831403999996, 2024-09-14T22:38:58.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058027982999999998, 2024-09-14T22:38:58.720 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.720 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.720 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.720 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.720 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.720 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.721 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123269 2024-09-14T22:38:58.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.721 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013676+0000", 2024-09-14T22:38:58.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013676+0000", 2024-09-14T22:38:58.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.722 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013677+0000", 2024-09-14T22:38:58.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.387e-06 2024-09-14T22:38:58.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.722 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013684+0000", 2024-09-14T22:38:58.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0530000000000001e-06 2024-09-14T22:38:58.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013706+0000", 2024-09-14T22:38:58.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1121999999999999e-05 2024-09-14T22:38:58.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013722+0000", 2024-09-14T22:38:58.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6249999999999999e-05 2024-09-14T22:38:58.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071402+0000", 2024-09-14T22:38:58.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057679945000000003 2024-09-14T22:38:58.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071547+0000", 2024-09-14T22:38:58.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000145632 2024-09-14T22:38:58.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071704+0000", 2024-09-14T22:38:58.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000156594 2024-09-14T22:38:58.724 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.724 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.725 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.725 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123270 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1064960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:58.725 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013744+0000", 2024-09-14T22:38:58.725 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 76.640763453000005, 2024-09-14T22:38:58.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058355872000000003, 2024-09-14T22:38:58.725 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.725 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.725 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.725 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.725 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.725 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.725 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123270 2024-09-14T22:38:58.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.726 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013744+0000", 2024-09-14T22:38:58.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013744+0000", 2024-09-14T22:38:58.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013745+0000", 2024-09-14T22:38:58.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2589999999999999e-06 2024-09-14T22:38:58.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013751+0000", 2024-09-14T22:38:58.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1929999999999998e-06 2024-09-14T22:38:58.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013778+0000", 2024-09-14T22:38:58.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6472e-05 2024-09-14T22:38:58.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013795+0000", 2024-09-14T22:38:58.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6626e-05 2024-09-14T22:38:58.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071775+0000", 2024-09-14T22:38:58.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057980232 2024-09-14T22:38:58.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071911+0000", 2024-09-14T22:38:58.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000136129 2024-09-14T22:38:58.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072100+0000", 2024-09-14T22:38:58.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000188961 2024-09-14T22:38:58.729 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.729 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.729 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.729 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123271 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1081344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:58.729 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013818+0000", 2024-09-14T22:38:58.730 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 76.640689257000005, 2024-09-14T22:38:58.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058549565999999997, 2024-09-14T22:38:58.730 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.730 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.730 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.730 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.730 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.730 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.730 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123271 2024-09-14T22:38:58.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.730 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.731 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013818+0000", 2024-09-14T22:38:58.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.731 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.731 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013818+0000", 2024-09-14T22:38:58.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.731 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.731 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013958+0000", 2024-09-14T22:38:58.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013934 2024-09-14T22:38:58.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013968+0000", 2024-09-14T22:38:58.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0329000000000001e-05 2024-09-14T22:38:58.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013995+0000", 2024-09-14T22:38:58.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7613000000000001e-05 2024-09-14T22:38:58.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014014+0000", 2024-09-14T22:38:58.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8338999999999998e-05 2024-09-14T22:38:58.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072155+0000", 2024-09-14T22:38:58.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058140989999999997 2024-09-14T22:38:58.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072247+0000", 2024-09-14T22:38:58.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1774999999999995e-05 2024-09-14T22:38:58.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072368+0000", 2024-09-14T22:38:58.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012118 2024-09-14T22:38:58.734 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.734 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.734 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.734 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123272 3.1 3.4b235a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:58.734 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.014046+0000", 2024-09-14T22:38:58.734 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 76.640461580999997, 2024-09-14T22:38:58.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058364077, 2024-09-14T22:38:58.734 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.734 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.734 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-14T22:38:58.734 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.734 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.735 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.735 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123272 2024-09-14T22:38:58.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.735 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014046+0000", 2024-09-14T22:38:58.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014046+0000", 2024-09-14T22:38:58.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.736 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.736 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014047+0000", 2024-09-14T22:38:58.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.533e-06 2024-09-14T22:38:58.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.736 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.736 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014085+0000", 2024-09-14T22:38:58.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7734999999999997e-05 2024-09-14T22:38:58.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.736 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.736 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014111+0000", 2024-09-14T22:38:58.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6055999999999999e-05 2024-09-14T22:38:58.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014136+0000", 2024-09-14T22:38:58.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4592000000000001e-05 2024-09-14T22:38:58.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072402+0000", 2024-09-14T22:38:58.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058266251999999998 2024-09-14T22:38:58.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072410+0000", 2024-09-14T22:38:58.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.909e-06 2024-09-14T22:38:58.738 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.738 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.738 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.738 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123273 3.1 3.4b235a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:38:58.738 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.014161+0000", 2024-09-14T22:38:58.738 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 76.640346553000001, 2024-09-14T22:38:58.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058284415999999999, 2024-09-14T22:38:58.738 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.738 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.738 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-14T22:38:58.738 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.738 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.738 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.739 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123273 2024-09-14T22:38:58.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.739 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014161+0000", 2024-09-14T22:38:58.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014161+0000", 2024-09-14T22:38:58.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.017996+0000", 2024-09-14T22:38:58.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0038350670000000002 2024-09-14T22:38:58.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.018009+0000", 2024-09-14T22:38:58.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3305999999999999e-05 2024-09-14T22:38:58.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.018044+0000", 2024-09-14T22:38:58.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4792000000000002e-05 2024-09-14T22:38:58.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.018067+0000", 2024-09-14T22:38:58.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2595e-05 2024-09-14T22:38:58.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072439+0000", 2024-09-14T22:38:58.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054372192 2024-09-14T22:38:58.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072445+0000", 2024-09-14T22:38:58.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4640000000000003e-06 2024-09-14T22:38:58.742 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.742 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.742 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.742 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:148469 3.18 3:1f6f7244:::rbd_header.113c827b523f:head [watch reconnect cookie 140250011614800 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-14T22:38:58.742 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:55.746449+0000", 2024-09-14T22:38:58.742 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 62.908057941000003, 2024-09-14T22:38:58.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.996484173, 2024-09-14T22:38:58.742 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.742 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.742 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.742 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.743 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.743 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.743 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 148469 2024-09-14T22:38:58.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.743 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746449+0000", 2024-09-14T22:38:58.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746449+0000", 2024-09-14T22:38:58.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746452+0000", 2024-09-14T22:38:58.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0019999999999998e-06 2024-09-14T22:38:58.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746461+0000", 2024-09-14T22:38:58.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0919999999999998e-06 2024-09-14T22:38:58.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746475+0000", 2024-09-14T22:38:58.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4853e-05 2024-09-14T22:38:58.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746484+0000", 2024-09-14T22:38:58.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3780000000000001e-06 2024-09-14T22:38:58.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746559+0000", 2024-09-14T22:38:58.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5259000000000006e-05 2024-09-14T22:38:58.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:58.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746572+0000", 2024-09-14T22:38:58.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.271e-05 2024-09-14T22:38:58.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727807+0000", 2024-09-14T22:38:58.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98123555900000003 2024-09-14T22:38:58.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:58.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727826+0000", 2024-09-14T22:38:58.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8828e-05 2024-09-14T22:38:58.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.739033+0000", 2024-09-14T22:38:58.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.011207062 2024-09-14T22:38:58.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.739479+0000", 2024-09-14T22:38:58.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044563200000000001 2024-09-14T22:38:58.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.742934+0000", 2024-09-14T22:38:58.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0034547979999999998 2024-09-14T22:38:58.747 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.747 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.747 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.748 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:148473 3.18 3:1f6f7244:::rbd_header.113c827b523f:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-14T22:38:58.748 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:55.755969+0000", 2024-09-14T22:38:58.748 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 62.898538174999999, 2024-09-14T22:38:58.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9872532889999999, 2024-09-14T22:38:58.748 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.748 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.748 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:38:58.748 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.748 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.748 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.748 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 148473 2024-09-14T22:38:58.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.749 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755969+0000", 2024-09-14T22:38:58.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755969+0000", 2024-09-14T22:38:58.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755971+0000", 2024-09-14T22:38:58.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.674e-06 2024-09-14T22:38:58.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755979+0000", 2024-09-14T22:38:58.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8599999999999993e-06 2024-09-14T22:38:58.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755992+0000", 2024-09-14T22:38:58.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.361e-05 2024-09-14T22:38:58.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.756001+0000", 2024-09-14T22:38:58.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.422e-06 2024-09-14T22:38:58.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.756069+0000", 2024-09-14T22:38:58.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8209999999999999e-05 2024-09-14T22:38:58.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:58.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.756077+0000", 2024-09-14T22:38:58.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.994e-06 2024-09-14T22:38:58.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727872+0000", 2024-09-14T22:38:58.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.97179484999999999 2024-09-14T22:38:58.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:58.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727885+0000", 2024-09-14T22:38:58.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3101e-05 2024-09-14T22:38:58.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.740559+0000", 2024-09-14T22:38:58.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0126736300000001 2024-09-14T22:38:58.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.740777+0000", 2024-09-14T22:38:58.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021822000000000001 2024-09-14T22:38:58.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 2", 2024-09-14T22:38:58.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.741114+0000", 2024-09-14T22:38:58.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000336793 2024-09-14T22:38:58.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:38:58.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.742668+0000", 2024-09-14T22:38:58.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001554061 2024-09-14T22:38:58.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:38:58.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.742947+0000", 2024-09-14T22:38:58.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027937200000000001 2024-09-14T22:38:58.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:38:58.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743030+0000", 2024-09-14T22:38:58.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3385000000000002e-05 2024-09-14T22:38:58.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743223+0000", 2024-09-14T22:38:58.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00019210700000000001 2024-09-14T22:38:58.755 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.755 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.755 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.755 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:148474 3.18 3:1f6f7244:::rbd_header.113c827b523f:head [watch reconnect cookie 140250011614800 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104)", 2024-09-14T22:38:58.755 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:56.728273+0000", 2024-09-14T22:38:58.756 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.926234928, 2024-09-14T22:38:58.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0180497770000001, 2024-09-14T22:38:58.756 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:38:58.756 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:38:58.756 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:38:58.756 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:38:58.756 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:38:58.756 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:38:58.756 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 148474 2024-09-14T22:38:58.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.756 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:38:58.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:38:58.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728273+0000", 2024-09-14T22:38:58.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:38:58.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728273+0000", 2024-09-14T22:38:58.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:38:58.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:38:58.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728274+0000", 2024-09-14T22:38:58.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7570000000000001e-06 2024-09-14T22:38:58.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:38:58.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728281+0000", 2024-09-14T22:38:58.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5089999999999999e-06 2024-09-14T22:38:58.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:38:58.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728298+0000", 2024-09-14T22:38:58.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6727000000000001e-05 2024-09-14T22:38:58.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:38:58.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728309+0000", 2024-09-14T22:38:58.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1021e-05 2024-09-14T22:38:58.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.759 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.730078+0000", 2024-09-14T22:38:58.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017694010000000001 2024-09-14T22:38:58.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.759 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:38:58.759 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.730096+0000", 2024-09-14T22:38:58.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8039000000000001e-05 2024-09-14T22:38:58.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.759 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.759 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.741672+0000", 2024-09-14T22:38:58.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.011575817 2024-09-14T22:38:58.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.759 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2024-09-14T22:38:58.759 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.741827+0000", 2024-09-14T22:38:58.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000155563 2024-09-14T22:38:58.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:38:58.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743299+0000", 2024-09-14T22:38:58.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0014712729999999999 2024-09-14T22:38:58.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:38:58.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743415+0000", 2024-09-14T22:38:58.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011594700000000001 2024-09-14T22:38:58.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:38:58.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:38:58.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:38:58.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.746322+0000", 2024-09-14T22:38:58.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0029077230000000001 2024-09-14T22:38:58.761 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.761 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.761 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.761 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:38:58.761 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:38:58.761 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:58.765 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:38:58.765 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:58.760+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:58.772 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T22:38:58.865 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:58.860+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:58.938 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:58.939 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:58.939 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:38:58.939 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:58.958 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:38:59.002 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:59.000+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:59.067 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:59.064+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:59.107 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:38:59.119 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:38:59.119 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:38:59.119 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:38:59.119 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:38:59.119 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:38:59.134 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:38:59.178 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:38:59.176+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:38:59.265 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:38:59.266 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:38:59.269 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:59.264+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:59.283 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:38:59.284 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:38:59.300 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:59.300 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:59.302 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:38:59.302 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:38:59.315 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:38:59.322 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:38:59.322 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:38:59.322 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T22:38:59.345 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:38:59.345 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:38:59.366 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:38:59.367 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:38:59.370 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:38:59.364+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:38:59.387 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:38:59.388 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:38:59.391 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:38:59.391 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:38:59.441 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:59.441 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:59.455 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:38:59.502 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:59.500+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:59.551 INFO:teuthology.orchestra.run.smithi176.stdout:get_pg_num_history pg_num_history pg_num_history(e158 pg_nums {1={16=1},2={19=8},3={23=32}} deleted_pools ) 2024-09-14T22:38:59.551 INFO:teuthology.orchestra.run.smithi176.stdout:Importing pgid 3.9 2024-09-14T22:38:59.552 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:9063cc7e:::rbd_data.113cda0183a0.0000000000000007:head# 2024-09-14T22:38:59.553 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 0=[]:{} 2024-09-14T22:38:59.555 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:9094ac4b:::rbd_data.113cf9fcaf37.0000000000000035:head# 2024-09-14T22:38:59.555 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 25=[]:{} 2024-09-14T22:38:59.557 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:90a1593c:::rbd_data.113c179ecb6a.0000000000000260:head# 2024-09-14T22:38:59.558 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 2a=[]:{} 2024-09-14T22:38:59.560 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:90d69314:::rbd_data.113cf9fcaf37.0000000000000026:head# 2024-09-14T22:38:59.560 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 25=[]:{} 2024-09-14T22:38:59.563 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:90db9e2a:::rbd_data.113c179ecb6a.00000000000001f7:head# 2024-09-14T22:38:59.563 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 2a=[]:{} 2024-09-14T22:38:59.566 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:91459c76:::rbd_data.113c179ecb6a.00000000000000ad:head# 2024-09-14T22:38:59.566 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 2a=[]:{} 2024-09-14T22:38:59.569 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:914b4970:::rbd_data.113c179ecb6a.00000000000000bc:head# 2024-09-14T22:38:59.569 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 2a=[]:{} 2024-09-14T22:38:59.571 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:91856005:::rbd_header.113c609e90d7:head# 2024-09-14T22:38:59.572 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 0=[]:{} 2024-09-14T22:38:59.572 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:59.568+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:59.574 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:91b049a8:::rbd_data.113c179ecb6a.00000000000001c1:head# 2024-09-14T22:38:59.574 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 2a=[]:{} 2024-09-14T22:38:59.577 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:91b0d498:::rbd_data.113cf9fcaf37.000000000000009b:head# 2024-09-14T22:38:59.577 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 25=[]:{} 2024-09-14T22:38:59.581 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:59.581 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:59.582 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:91f6bb73:::rbd_data.113c179ecb6a.00000000000002ac:head# 2024-09-14T22:38:59.582 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 2a=[]:{} 2024-09-14T22:38:59.582 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:9206cc21:::rbd_data.113c179ecb6a.00000000000000d9:head# 2024-09-14T22:38:59.582 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 2a=[]:{} 2024-09-14T22:38:59.582 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:928bbed7:::rbd_data.113c237f38ab.000000000000027b:head# 2024-09-14T22:38:59.582 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 11=[]:{} 2024-09-14T22:38:59.582 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:92974bdd:::rbd_data.113c179ecb6a.00000000000001f1:head# 2024-09-14T22:38:59.583 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 2a=[]:{} 2024-09-14T22:38:59.583 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:92abd5f0:::rbd_data.113c4558bb1.00000000000000a8:7# 2024-09-14T22:38:59.583 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:92abd5f0:::rbd_data.113c4558bb1.00000000000000a8:head# 2024-09-14T22:38:59.584 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 7=[]:{7=[7]} 2024-09-14T22:38:59.585 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:93045a07:::rbd_data.113cf9fcaf37.00000000000000f7:head# 2024-09-14T22:38:59.585 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 25=[]:{} 2024-09-14T22:38:59.585 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:931332d1:::rbd_data.113cf9fcaf37.00000000000000b2:head# 2024-09-14T22:38:59.585 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 25=[]:{} 2024-09-14T22:38:59.585 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:93a23de8:::rbd_data.113c179ecb6a.0000000000000200:head# 2024-09-14T22:38:59.585 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 2a=[]:{} 2024-09-14T22:38:59.586 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:93a2dd18:::rbd_data.113c179ecb6a.0000000000000277:head# 2024-09-14T22:38:59.586 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 2a=[]:{} 2024-09-14T22:38:59.586 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:93d6b168:::rbd_data.113cec2505f0.000000000000001a:head# 2024-09-14T22:38:59.588 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 0=[]:{} 2024-09-14T22:38:59.591 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:9423ac3c:::rbd_data.113c3b74a67f.0000000000000005:head# 2024-09-14T22:38:59.593 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:38:59.597 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 0=[]:{} 2024-09-14T22:38:59.607 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:94389354:::rbd_data.113cf9fcaf37.000000000000004e:head# 2024-09-14T22:38:59.607 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 25=[]:{} 2024-09-14T22:38:59.608 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:949c0cf8:::rbd_data.113cf9fcaf37.0000000000000069:head# 2024-09-14T22:38:59.608 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 25=[]:{} 2024-09-14T22:38:59.609 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:94ce85d8:::rbd_data.113c4558bb1.0000000000000060:7# 2024-09-14T22:38:59.610 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:94ce85d8:::rbd_data.113c4558bb1.0000000000000060:head# 2024-09-14T22:38:59.610 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 7=[]:{7=[7]} 2024-09-14T22:38:59.611 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:9513a271:::rbd_data.113c179ecb6a.000000000000011b:head# 2024-09-14T22:38:59.611 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 2a=[]:{} 2024-09-14T22:38:59.612 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:952d3e01:::rbd_data.113c3c739f88.000000000000005a:head# 2024-09-14T22:38:59.612 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 0=[]:{} 2024-09-14T22:38:59.613 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:953746ba:::rbd_data.113c179ecb6a.0000000000000113:head# 2024-09-14T22:38:59.613 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 2a=[]:{} 2024-09-14T22:38:59.614 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:9542a272:::rbd_data.113c179ecb6a.0000000000000112:head# 2024-09-14T22:38:59.614 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 2a=[]:{} 2024-09-14T22:38:59.615 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:954ce16b:::rbd_data.113c4558bb1.0000000000000074:7# 2024-09-14T22:38:59.617 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:954ce16b:::rbd_data.113c4558bb1.0000000000000074:head# 2024-09-14T22:38:59.617 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 7=[]:{7=[7]} 2024-09-14T22:38:59.618 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:955cce30:::rbd_data.113c179ecb6a.000000000000008d:head# 2024-09-14T22:38:59.618 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 2a=[]:{} 2024-09-14T22:38:59.618 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:95a325d3:::rbd_data.113cf9fcaf37.00000000000000fc:head# 2024-09-14T22:38:59.618 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 25=[]:{} 2024-09-14T22:38:59.619 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:95ee2fb4:::rbd_data.113cf9fcaf37.00000000000000ed:head# 2024-09-14T22:38:59.619 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 25=[]:{} 2024-09-14T22:38:59.620 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:9617419a:::rbd_data.113cf9fcaf37.000000000000006e:head# 2024-09-14T22:38:59.620 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 25=[]:{} 2024-09-14T22:38:59.621 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:965ec7d5:::rbd_data.113c179ecb6a.00000000000002a7:head# 2024-09-14T22:38:59.621 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 2a=[]:{} 2024-09-14T22:38:59.621 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:96676ff5:::rbd_data.113cce440cee.0000000000000006:a# 2024-09-14T22:38:59.625 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:96676ff5:::rbd_data.113cce440cee.0000000000000006:head# 2024-09-14T22:38:59.625 INFO:teuthology.orchestra.run.smithi176.stdout:snapset a=[]:{a=[a]} 2024-09-14T22:38:59.626 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:96b055ea:::rbd_data.113cf9fcaf37.00000000000000e1:head# 2024-09-14T22:38:59.627 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 25=[]:{} 2024-09-14T22:38:59.627 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:972a090b:::rbd_data.113c179ecb6a.00000000000001e2:head# 2024-09-14T22:38:59.627 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 2a=[]:{} 2024-09-14T22:38:59.628 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:9747b897:::rbd_data.113cf9fcaf37.00000000000000f0:head# 2024-09-14T22:38:59.628 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 25=[]:{} 2024-09-14T22:38:59.629 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:97536c31:::rbd_data.113cf9fcaf37.00000000000000c6:head# 2024-09-14T22:38:59.629 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 25=[]:{} 2024-09-14T22:38:59.629 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:975a890d:::rbd_data.113c179ecb6a.000000000000016b:head# 2024-09-14T22:38:59.630 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 2a=[]:{} 2024-09-14T22:38:59.630 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:97624012:::rbd_data.113c4558bb1.00000000000000b2:7# 2024-09-14T22:38:59.632 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:97624012:::rbd_data.113c4558bb1.00000000000000b2:head# 2024-09-14T22:38:59.632 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 7=[]:{7=[7]} 2024-09-14T22:38:59.633 INFO:teuthology.orchestra.run.smithi176.stdout:Write #3:978d708a:::rbd_data.113cf9fcaf37.00000000000000cd:head# 2024-09-14T22:38:59.633 INFO:teuthology.orchestra.run.smithi176.stdout:snapset 25=[]:{} 2024-09-14T22:38:59.634 INFO:teuthology.orchestra.run.smithi176.stdout:write_pg epoch 158 info 3.9( v 158'425 (0'0,158'425] local-lis/les=116/117 n=47 ec=23/23 lis/c=116/116 les/c/f=117/117/0 sis=116) 2024-09-14T22:38:59.640 INFO:teuthology.orchestra.run.smithi176.stdout:Import successful 2024-09-14T22:38:59.673 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:59.666+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:59.754 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2024-09-14T22:38:59.762 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:59.762 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:59.773 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:38:59.768+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:38:59.786 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:38:59.874 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:38:59.866+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:38:59.920 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:38:59.920 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:38:59.936 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:38:59.980 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:38:59.974+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:39:00.076 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:39:00.072+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:39:00.102 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:39:00.103 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:39:00.117 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:39:00.177 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:39:00.170+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:39:00.205 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f /var/log/ceph/exp.3.9.4 2024-09-14T22:39:00.240 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:39:00.240 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:39:00.255 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:39:00.278 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:39:00.272+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:39:00.378 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:39:00.376+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:39:00.420 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:39:00.420 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:39:00.435 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:39:00.479 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:39:00.474+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:39:00.551 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:39:00.551 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:39:00.566 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:39:00.607 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:39:00.602+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:39:00.681 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:39:00.676+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:39:00.727 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:39:00.727 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:39:00.742 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:39:00.782 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:39:00.780+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:39:00.868 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:39:00.868 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:39:00.882 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:39:00.883 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:39:00.880+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:39:00.984 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:39:00.980+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:39:01.008 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:39:01.009 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:39:01.020 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:39:01.085 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:39:01.080+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:39:01.142 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T22:39:01.142 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:39:01.156 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:39:01.202 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:39:01.200+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:39:01.286 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:39:01.284+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:39:01.291 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:39:01.291 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:39:01.306 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:39:01.388 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:39:01.382+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:39:01.392 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:39:01.471 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-14T22:39:01.471 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:39:01.486 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:39:01.488 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:39:01.482+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:39:01.538 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-14T22:39:01.538 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:39:01.555 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-14T22:39:01.556 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:39:01.574 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-14T22:39:01.574 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:39:01.590 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:39:01.582+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:39:01.591 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-14T22:39:01.591 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:39:01.592 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T22:39:01.613 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-14T22:39:01.613 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:39:01.616 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:39:01.616 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:39:01.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:39:01.617 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:39:01.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.617 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144987 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3424256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:01.617 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549297+0000", 2024-09-14T22:39:01.617 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.064500655000003, 2024-09-14T22:39:01.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.079931556000000001, 2024-09-14T22:39:01.618 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:01.618 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:01.618 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:01.619 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:01.619 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:01.619 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:01.619 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144987 2024-09-14T22:39:01.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.620 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:01.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:01.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549297+0000", 2024-09-14T22:39:01.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:01.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549297+0000", 2024-09-14T22:39:01.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:01.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549298+0000", 2024-09-14T22:39:01.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2699999999999999e-07 2024-09-14T22:39:01.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:01.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549301+0000", 2024-09-14T22:39:01.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2959999999999999e-06 2024-09-14T22:39:01.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:01.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549314+0000", 2024-09-14T22:39:01.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2588999999999999e-05 2024-09-14T22:39:01.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:01.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549323+0000", 2024-09-14T22:39:01.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.927e-06 2024-09-14T22:39:01.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.625 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:01.625 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629038+0000", 2024-09-14T22:39:01.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.079714220000000002 2024-09-14T22:39:01.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:01.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629116+0000", 2024-09-14T22:39:01.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7966000000000005e-05 2024-09-14T22:39:01.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:01.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629228+0000", 2024-09-14T22:39:01.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112831 2024-09-14T22:39:01.627 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.627 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:01.628 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.628 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144988 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3325952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:01.628 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549335+0000", 2024-09-14T22:39:01.629 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.064462430000006, 2024-09-14T22:39:01.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080179116999999994, 2024-09-14T22:39:01.629 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:01.629 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:01.629 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:01.629 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:01.630 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:01.630 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:01.630 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144988 2024-09-14T22:39:01.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.630 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:01.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:01.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549335+0000", 2024-09-14T22:39:01.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:01.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549335+0000", 2024-09-14T22:39:01.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:01.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549336+0000", 2024-09-14T22:39:01.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7999999999999995e-07 2024-09-14T22:39:01.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:01.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549339+0000", 2024-09-14T22:39:01.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.836e-06 2024-09-14T22:39:01.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:01.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549352+0000", 2024-09-14T22:39:01.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3069e-05 2024-09-14T22:39:01.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:01.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549359+0000", 2024-09-14T22:39:01.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.357e-06 2024-09-14T22:39:01.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:01.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629288+0000", 2024-09-14T22:39:01.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.079928492000000004 2024-09-14T22:39:01.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:01.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629412+0000", 2024-09-14T22:39:01.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012413699999999999 2024-09-14T22:39:01.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:01.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629514+0000", 2024-09-14T22:39:01.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000102546 2024-09-14T22:39:01.634 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.634 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:01.634 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.634 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144989 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3522560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:01.634 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:39:01.634 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.064428022000001, 2024-09-14T22:39:01.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080393122999999997, 2024-09-14T22:39:01.634 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:01.634 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:01.635 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:01.635 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:01.635 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:01.635 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:01.635 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144989 2024-09-14T22:39:01.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.635 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:01.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:01.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:39:01.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:01.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:39:01.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:01.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:39:01.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3600000000000003e-07 2024-09-14T22:39:01.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:01.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549373+0000", 2024-09-14T22:39:01.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8169999999999999e-06 2024-09-14T22:39:01.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:01.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549385+0000", 2024-09-14T22:39:01.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1803e-05 2024-09-14T22:39:01.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:01.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549394+0000", 2024-09-14T22:39:01.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4290000000000007e-06 2024-09-14T22:39:01.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:01.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629576+0000", 2024-09-14T22:39:01.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080181551000000004 2024-09-14T22:39:01.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:01.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629654+0000", 2024-09-14T22:39:01.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7633999999999996e-05 2024-09-14T22:39:01.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:01.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629763+0000", 2024-09-14T22:39:01.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109153 2024-09-14T22:39:01.638 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.638 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:01.638 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.639 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144990 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:01.639 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:39:01.639 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.064384997999994, 2024-09-14T22:39:01.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080614496999999993, 2024-09-14T22:39:01.639 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:01.639 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:01.639 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:01.639 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:01.639 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:01.639 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:01.639 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144990 2024-09-14T22:39:01.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.640 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:01.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:01.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:39:01.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:01.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:39:01.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:01.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:39:01.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0999999999999997e-07 2024-09-14T22:39:01.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:01.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549419+0000", 2024-09-14T22:39:01.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5339999999999999e-06 2024-09-14T22:39:01.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:01.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549439+0000", 2024-09-14T22:39:01.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0035e-05 2024-09-14T22:39:01.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:01.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549449+0000", 2024-09-14T22:39:01.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0429e-05 2024-09-14T22:39:01.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:01.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629815+0000", 2024-09-14T22:39:01.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080365467999999995 2024-09-14T22:39:01.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:01.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629922+0000", 2024-09-14T22:39:01.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106914 2024-09-14T22:39:01.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:01.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630027+0000", 2024-09-14T22:39:01.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105207 2024-09-14T22:39:01.643 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.643 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:01.643 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.643 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144991 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 2998272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:01.643 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549463+0000", 2024-09-14T22:39:01.643 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.064334693999996, 2024-09-14T22:39:01.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080847637, 2024-09-14T22:39:01.643 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:01.644 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:01.644 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:01.644 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:01.644 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:01.644 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:01.644 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144991 2024-09-14T22:39:01.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.644 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:01.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.644 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:01.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549463+0000", 2024-09-14T22:39:01.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:01.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549463+0000", 2024-09-14T22:39:01.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:01.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549464+0000", 2024-09-14T22:39:01.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9999999999999997e-07 2024-09-14T22:39:01.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:01.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549467+0000", 2024-09-14T22:39:01.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0800000000000002e-06 2024-09-14T22:39:01.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:01.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549478+0000", 2024-09-14T22:39:01.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1813e-05 2024-09-14T22:39:01.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:01.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549488+0000", 2024-09-14T22:39:01.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7030000000000001e-06 2024-09-14T22:39:01.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:01.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630090+0000", 2024-09-14T22:39:01.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080601962999999999 2024-09-14T22:39:01.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:01.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630187+0000", 2024-09-14T22:39:01.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6552000000000007e-05 2024-09-14T22:39:01.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:01.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630311+0000", 2024-09-14T22:39:01.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000123826 2024-09-14T22:39:01.647 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.647 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:01.647 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.648 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144992 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3620864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:01.648 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:39:01.648 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.064298886000003, 2024-09-14T22:39:01.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081097875999999999, 2024-09-14T22:39:01.648 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:01.648 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:01.648 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:01.648 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:01.648 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:01.648 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:01.648 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144992 2024-09-14T22:39:01.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.649 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:01.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:01.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:39:01.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:01.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:39:01.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:01.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:39:01.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5400000000000001e-07 2024-09-14T22:39:01.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:01.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549502+0000", 2024-09-14T22:39:01.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7889999999999999e-06 2024-09-14T22:39:01.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:01.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549513+0000", 2024-09-14T22:39:01.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1204e-05 2024-09-14T22:39:01.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:01.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549523+0000", 2024-09-14T22:39:01.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5990000000000008e-06 2024-09-14T22:39:01.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:01.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630362+0000", 2024-09-14T22:39:01.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080839272000000004 2024-09-14T22:39:01.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:01.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630469+0000", 2024-09-14T22:39:01.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107285 2024-09-14T22:39:01.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:01.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630597+0000", 2024-09-14T22:39:01.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012707300000000001 2024-09-14T22:39:01.652 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.652 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:01.652 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.652 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144993 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:01.652 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:39:01.652 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.064265809999995, 2024-09-14T22:39:01.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081338411999999999, 2024-09-14T22:39:01.652 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:01.652 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:01.653 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:01.653 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:01.653 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:01.653 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:01.653 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144993 2024-09-14T22:39:01.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.653 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:01.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:01.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:39:01.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:01.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:39:01.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:01.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:39:01.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5300000000000004e-07 2024-09-14T22:39:01.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:01.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549544+0000", 2024-09-14T22:39:01.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1912e-05 2024-09-14T22:39:01.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:01.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549557+0000", 2024-09-14T22:39:01.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2413e-05 2024-09-14T22:39:01.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:01.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549567+0000", 2024-09-14T22:39:01.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8660000000000001e-06 2024-09-14T22:39:01.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:01.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630651+0000", 2024-09-14T22:39:01.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081084229999999993 2024-09-14T22:39:01.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:01.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630729+0000", 2024-09-14T22:39:01.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8265999999999999e-05 2024-09-14T22:39:01.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:01.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630870+0000", 2024-09-14T22:39:01.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014107200000000001 2024-09-14T22:39:01.656 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.656 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:01.656 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.657 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144994 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:01.657 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:39:01.657 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.064218843000006, 2024-09-14T22:39:01.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081547958000000004, 2024-09-14T22:39:01.657 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:01.657 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:01.657 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:01.657 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:01.657 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:01.657 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:01.657 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144994 2024-09-14T22:39:01.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.658 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:01.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:01.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:39:01.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:01.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:39:01.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:01.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:39:01.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6599999999999996e-07 2024-09-14T22:39:01.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:01.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549582+0000", 2024-09-14T22:39:01.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.004e-06 2024-09-14T22:39:01.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:01.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549594+0000", 2024-09-14T22:39:01.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1459e-05 2024-09-14T22:39:01.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:01.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549603+0000", 2024-09-14T22:39:01.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6129999999999993e-06 2024-09-14T22:39:01.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:01.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630932+0000", 2024-09-14T22:39:01.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081328558999999995 2024-09-14T22:39:01.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:01.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631027+0000", 2024-09-14T22:39:01.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4742999999999997e-05 2024-09-14T22:39:01.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:01.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631127+0000", 2024-09-14T22:39:01.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9914000000000006e-05 2024-09-14T22:39:01.661 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.661 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:01.661 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.661 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144995 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3391488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:01.661 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549615+0000", 2024-09-14T22:39:01.661 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.064182704999993, 2024-09-14T22:39:01.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081734251999999993, 2024-09-14T22:39:01.661 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:01.662 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:01.662 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:01.662 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:01.662 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:01.662 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:01.662 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144995 2024-09-14T22:39:01.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.662 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:01.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:01.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549615+0000", 2024-09-14T22:39:01.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:01.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549615+0000", 2024-09-14T22:39:01.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:01.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549616+0000", 2024-09-14T22:39:01.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9299999999999997e-07 2024-09-14T22:39:01.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:01.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549618+0000", 2024-09-14T22:39:01.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7269999999999999e-06 2024-09-14T22:39:01.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:01.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549630+0000", 2024-09-14T22:39:01.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1313e-05 2024-09-14T22:39:01.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:01.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549639+0000", 2024-09-14T22:39:01.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4159999999999993e-06 2024-09-14T22:39:01.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:01.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631183+0000", 2024-09-14T22:39:01.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081543633000000004 2024-09-14T22:39:01.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:01.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631252+0000", 2024-09-14T22:39:01.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9567999999999995e-05 2024-09-14T22:39:01.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:01.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631349+0000", 2024-09-14T22:39:01.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6801999999999999e-05 2024-09-14T22:39:01.665 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.665 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:01.665 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.666 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144996 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3194880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:01.666 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:39:01.666 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.064147027000004, 2024-09-14T22:39:01.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081947213000000005, 2024-09-14T22:39:01.666 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:01.666 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:01.666 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:01.666 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:01.666 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:01.666 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:01.666 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144996 2024-09-14T22:39:01.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.667 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:01.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:01.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:39:01.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:01.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:39:01.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:01.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:39:01.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3e-07 2024-09-14T22:39:01.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:01.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549654+0000", 2024-09-14T22:39:01.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6900000000000001e-06 2024-09-14T22:39:01.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:01.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549665+0000", 2024-09-14T22:39:01.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1392999999999999e-05 2024-09-14T22:39:01.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:01.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549675+0000", 2024-09-14T22:39:01.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5459999999999997e-06 2024-09-14T22:39:01.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:01.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631401+0000", 2024-09-14T22:39:01.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081726036000000002 2024-09-14T22:39:01.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:01.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631492+0000", 2024-09-14T22:39:01.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0886999999999999e-05 2024-09-14T22:39:01.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:01.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631598+0000", 2024-09-14T22:39:01.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106031 2024-09-14T22:39:01.670 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.670 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:01.670 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.670 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144997 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:01.671 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549692+0000", 2024-09-14T22:39:01.671 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.064105362999996, 2024-09-14T22:39:01.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082149925999999998, 2024-09-14T22:39:01.671 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:01.671 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:01.671 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:01.671 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:01.671 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:01.671 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:01.671 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144997 2024-09-14T22:39:01.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.671 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:01.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:01.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549692+0000", 2024-09-14T22:39:01.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:01.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549692+0000", 2024-09-14T22:39:01.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:01.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549693+0000", 2024-09-14T22:39:01.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.540000000000001e-07 2024-09-14T22:39:01.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:01.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549698+0000", 2024-09-14T22:39:01.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1460000000000004e-06 2024-09-14T22:39:01.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:01.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549715+0000", 2024-09-14T22:39:01.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6983e-05 2024-09-14T22:39:01.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:01.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549728+0000", 2024-09-14T22:39:01.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2326e-05 2024-09-14T22:39:01.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:01.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631649+0000", 2024-09-14T22:39:01.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081921830000000001 2024-09-14T22:39:01.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:01.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631726+0000", 2024-09-14T22:39:01.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6759000000000001e-05 2024-09-14T22:39:01.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:01.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631842+0000", 2024-09-14T22:39:01.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011592800000000001 2024-09-14T22:39:01.675 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.675 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:01.675 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.675 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144998 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 2965504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:01.675 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549741+0000", 2024-09-14T22:39:01.675 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.064056070999996, 2024-09-14T22:39:01.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082394832000000001, 2024-09-14T22:39:01.675 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:01.675 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:01.676 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:01.676 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:01.676 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:01.676 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:01.676 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144998 2024-09-14T22:39:01.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.676 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:01.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:01.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549741+0000", 2024-09-14T22:39:01.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:01.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549741+0000", 2024-09-14T22:39:01.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:01.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549742+0000", 2024-09-14T22:39:01.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.06e-07 2024-09-14T22:39:01.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:01.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549757+0000", 2024-09-14T22:39:01.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4839e-05 2024-09-14T22:39:01.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:01.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549770+0000", 2024-09-14T22:39:01.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2879e-05 2024-09-14T22:39:01.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:01.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549780+0000", 2024-09-14T22:39:01.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0304e-05 2024-09-14T22:39:01.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:01.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631904+0000", 2024-09-14T22:39:01.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082124168999999997 2024-09-14T22:39:01.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:01.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631995+0000", 2024-09-14T22:39:01.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0993000000000001e-05 2024-09-14T22:39:01.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:01.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632136+0000", 2024-09-14T22:39:01.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014094199999999999 2024-09-14T22:39:01.679 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.679 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:01.679 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.680 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144999 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3588096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:01.680 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:39:01.680 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.064005860999998, 2024-09-14T22:39:01.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082593742999999997, 2024-09-14T22:39:01.680 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:01.680 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:01.680 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:01.680 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:01.680 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:01.680 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:01.680 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144999 2024-09-14T22:39:01.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.681 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:01.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.681 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:01.681 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:39:01.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.681 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:01.681 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:39:01.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.681 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:01.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:39:01.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1999999999999999e-07 2024-09-14T22:39:01.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:01.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549795+0000", 2024-09-14T22:39:01.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9569999999999998e-06 2024-09-14T22:39:01.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:01.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549807+0000", 2024-09-14T22:39:01.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1691999999999999e-05 2024-09-14T22:39:01.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:01.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549817+0000", 2024-09-14T22:39:01.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5100000000000004e-06 2024-09-14T22:39:01.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:01.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632190+0000", 2024-09-14T22:39:01.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082373374999999999 2024-09-14T22:39:01.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:01.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632264+0000", 2024-09-14T22:39:01.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3972000000000003e-05 2024-09-14T22:39:01.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:01.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632385+0000", 2024-09-14T22:39:01.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121517 2024-09-14T22:39:01.684 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.684 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:01.684 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.684 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145000 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3063808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:01.684 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549828+0000", 2024-09-14T22:39:01.684 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.063969583000002, 2024-09-14T22:39:01.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082860235000000004, 2024-09-14T22:39:01.685 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:01.685 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:01.685 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:01.685 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:01.685 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:01.685 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:01.685 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145000 2024-09-14T22:39:01.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.685 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:01.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:01.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549828+0000", 2024-09-14T22:39:01.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:01.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549828+0000", 2024-09-14T22:39:01.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:01.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549829+0000", 2024-09-14T22:39:01.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3300000000000002e-07 2024-09-14T22:39:01.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:01.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549831+0000", 2024-09-14T22:39:01.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6740000000000001e-06 2024-09-14T22:39:01.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:01.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549842+0000", 2024-09-14T22:39:01.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0316000000000001e-05 2024-09-14T22:39:01.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:01.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549851+0000", 2024-09-14T22:39:01.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7229999999999999e-06 2024-09-14T22:39:01.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:01.688 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632448+0000", 2024-09-14T22:39:01.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082596648999999994 2024-09-14T22:39:01.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.688 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:01.688 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632568+0000", 2024-09-14T22:39:01.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119592 2024-09-14T22:39:01.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.688 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:01.688 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632688+0000", 2024-09-14T22:39:01.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120648 2024-09-14T22:39:01.688 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.689 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:01.689 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.689 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145001 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3162112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:01.689 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549863+0000", 2024-09-14T22:39:01.689 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.063934355000001, 2024-09-14T22:39:01.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083105534999999994, 2024-09-14T22:39:01.689 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:01.689 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:01.689 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:01.689 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:01.689 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:01.689 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:01.690 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145001 2024-09-14T22:39:01.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.690 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:01.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:01.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549863+0000", 2024-09-14T22:39:01.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:01.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549863+0000", 2024-09-14T22:39:01.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:01.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549864+0000", 2024-09-14T22:39:01.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4300000000000003e-07 2024-09-14T22:39:01.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:01.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549867+0000", 2024-09-14T22:39:01.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.757e-06 2024-09-14T22:39:01.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:01.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549878+0000", 2024-09-14T22:39:01.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1345999999999999e-05 2024-09-14T22:39:01.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.692 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:01.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549888+0000", 2024-09-14T22:39:01.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6460000000000005e-06 2024-09-14T22:39:01.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.692 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:01.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632754+0000", 2024-09-14T22:39:01.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082866258999999998 2024-09-14T22:39:01.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.692 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:01.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632841+0000", 2024-09-14T22:39:01.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6971000000000005e-05 2024-09-14T22:39:01.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:01.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632969+0000", 2024-09-14T22:39:01.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012791300000000001 2024-09-14T22:39:01.693 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.693 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:01.693 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.693 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145002 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3260416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:01.693 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549933+0000", 2024-09-14T22:39:01.694 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.063864851999995, 2024-09-14T22:39:01.694 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083291027000000004, 2024-09-14T22:39:01.694 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:01.694 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:01.694 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:01.694 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:01.694 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:01.694 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:01.694 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145002 2024-09-14T22:39:01.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.694 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:01.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.694 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:01.694 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549933+0000", 2024-09-14T22:39:01.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:01.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549933+0000", 2024-09-14T22:39:01.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:01.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549934+0000", 2024-09-14T22:39:01.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8700000000000004e-07 2024-09-14T22:39:01.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:01.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549938+0000", 2024-09-14T22:39:01.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5029999999999999e-06 2024-09-14T22:39:01.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:01.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549952+0000", 2024-09-14T22:39:01.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4382999999999999e-05 2024-09-14T22:39:01.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:01.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549963+0000", 2024-09-14T22:39:01.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0665999999999999e-05 2024-09-14T22:39:01.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:01.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633025+0000", 2024-09-14T22:39:01.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083061727000000002 2024-09-14T22:39:01.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:01.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633117+0000", 2024-09-14T22:39:01.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1812999999999997e-05 2024-09-14T22:39:01.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:01.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633224+0000", 2024-09-14T22:39:01.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107048 2024-09-14T22:39:01.698 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.698 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:01.698 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.698 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145003 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3457024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:01.698 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549985+0000", 2024-09-14T22:39:01.698 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.063812433999999, 2024-09-14T22:39:01.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083465871999999997, 2024-09-14T22:39:01.698 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:01.698 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:01.698 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:01.698 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:01.699 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:01.699 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:01.699 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145003 2024-09-14T22:39:01.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.699 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:01.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:01.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549985+0000", 2024-09-14T22:39:01.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:01.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549985+0000", 2024-09-14T22:39:01.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:01.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549986+0000", 2024-09-14T22:39:01.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7299999999999995e-07 2024-09-14T22:39:01.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:01.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549989+0000", 2024-09-14T22:39:01.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8169999999999999e-06 2024-09-14T22:39:01.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:01.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549998+0000", 2024-09-14T22:39:01.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4290000000000007e-06 2024-09-14T22:39:01.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:01.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550009+0000", 2024-09-14T22:39:01.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0633e-05 2024-09-14T22:39:01.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:01.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633275+0000", 2024-09-14T22:39:01.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083266397000000006 2024-09-14T22:39:01.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:01.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633353+0000", 2024-09-14T22:39:01.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8129999999999999e-05 2024-09-14T22:39:01.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:01.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633451+0000", 2024-09-14T22:39:01.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7793000000000004e-05 2024-09-14T22:39:01.702 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.702 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:01.702 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.703 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145004 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3358720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:01.703 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.550029+0000", 2024-09-14T22:39:01.703 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.063768863000007, 2024-09-14T22:39:01.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083662089999999995, 2024-09-14T22:39:01.703 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:01.703 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:01.703 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:01.703 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:01.703 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:01.703 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:01.703 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145004 2024-09-14T22:39:01.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.703 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:01.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:01.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550029+0000", 2024-09-14T22:39:01.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:01.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550029+0000", 2024-09-14T22:39:01.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:01.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550030+0000", 2024-09-14T22:39:01.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.33e-07 2024-09-14T22:39:01.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:01.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550035+0000", 2024-09-14T22:39:01.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3430000000000003e-06 2024-09-14T22:39:01.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:01.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550053+0000", 2024-09-14T22:39:01.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8485e-05 2024-09-14T22:39:01.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:01.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550064+0000", 2024-09-14T22:39:01.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0264e-05 2024-09-14T22:39:01.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:01.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633502+0000", 2024-09-14T22:39:01.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083438498999999999 2024-09-14T22:39:01.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:01.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633592+0000", 2024-09-14T22:39:01.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.022e-05 2024-09-14T22:39:01.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:01.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633691+0000", 2024-09-14T22:39:01.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8345999999999997e-05 2024-09-14T22:39:01.707 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.707 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:01.707 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.707 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145005 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3555328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:01.707 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.550077+0000", 2024-09-14T22:39:01.707 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.063720403000005, 2024-09-14T22:39:01.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083854340999999999, 2024-09-14T22:39:01.707 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:01.707 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:01.707 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:01.708 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:01.708 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:01.708 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:01.708 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145005 2024-09-14T22:39:01.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.708 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:01.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:01.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550077+0000", 2024-09-14T22:39:01.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:01.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550077+0000", 2024-09-14T22:39:01.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:01.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550078+0000", 2024-09-14T22:39:01.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8700000000000005e-07 2024-09-14T22:39:01.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:01.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550101+0000", 2024-09-14T22:39:01.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3632e-05 2024-09-14T22:39:01.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:01.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550115+0000", 2024-09-14T22:39:01.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3893000000000001e-05 2024-09-14T22:39:01.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:01.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550126+0000", 2024-09-14T22:39:01.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0426e-05 2024-09-14T22:39:01.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:01.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633742+0000", 2024-09-14T22:39:01.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083616633999999995 2024-09-14T22:39:01.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:01.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633816+0000", 2024-09-14T22:39:01.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3690999999999996e-05 2024-09-14T22:39:01.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:01.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633932+0000", 2024-09-14T22:39:01.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011537800000000001 2024-09-14T22:39:01.711 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.711 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:01.711 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.711 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145006 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3653632~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:01.712 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.550236+0000", 2024-09-14T22:39:01.712 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 69.063561897, 2024-09-14T22:39:01.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083924027999999998, 2024-09-14T22:39:01.712 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:01.712 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:01.712 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:01.712 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:01.712 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:01.712 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:01.712 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145006 2024-09-14T22:39:01.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.712 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:01.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:01.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550236+0000", 2024-09-14T22:39:01.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:01.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550236+0000", 2024-09-14T22:39:01.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:01.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:01.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550237+0000", 2024-09-14T22:39:01.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2929999999999999e-06 2024-09-14T22:39:01.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:01.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550243+0000", 2024-09-14T22:39:01.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0390000000000001e-06 2024-09-14T22:39:01.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:01.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550256+0000", 2024-09-14T22:39:01.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3393e-05 2024-09-14T22:39:01.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:01.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550271+0000", 2024-09-14T22:39:01.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4166e-05 2024-09-14T22:39:01.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:01.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633974+0000", 2024-09-14T22:39:01.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083703825999999995 2024-09-14T22:39:01.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:01.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.634047+0000", 2024-09-14T22:39:01.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2935999999999994e-05 2024-09-14T22:39:01.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:01.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:01.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:01.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.634160+0000", 2024-09-14T22:39:01.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011237499999999999 2024-09-14T22:39:01.716 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.716 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:01.716 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.716 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:01.716 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:01.716 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:39:01.717 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-14T22:39:01.718 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:39:01.718 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:39:01.718 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-14T22:39:01.719 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:39:01.719 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:39:01.719 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:39:01.720 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-14T22:39:01.712+0000 7feacdd8c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26331) UID: 0 2024-09-14T22:39:01.786 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T22:39:01.818 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:39:01.810+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:39:01.908 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:39:01.908 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:39:01.908 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:39:01.909 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:39:01.919 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:39:01.916+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:39:01.928 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:39:02.020 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:39:02.016+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:39:02.087 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:39:02.087 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:39:02.087 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:39:02.087 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:39:02.087 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:39:02.101 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:39:02.122 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:39:02.114+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:39:02.223 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:39:02.220+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:39:02.268 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:39:02.268 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:39:02.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:39:02.268 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:39:02.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.269 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:122960 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 2347008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:39:02.269 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:41.998325+0000", 2024-09-14T22:39:02.269 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 80.265157208999995, 2024-09-14T22:39:02.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056570400999999999, 2024-09-14T22:39:02.269 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.269 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.269 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.270 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.270 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.270 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.270 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 122960 2024-09-14T22:39:02.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.270 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998325+0000", 2024-09-14T22:39:02.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998325+0000", 2024-09-14T22:39:02.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998326+0000", 2024-09-14T22:39:02.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9299999999999998e-07 2024-09-14T22:39:02.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.273 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998329+0000", 2024-09-14T22:39:02.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1370000000000002e-06 2024-09-14T22:39:02.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998340+0000", 2024-09-14T22:39:02.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1039999999999999e-05 2024-09-14T22:39:02.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998348+0000", 2024-09-14T22:39:02.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1920000000000005e-06 2024-09-14T22:39:02.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054700+0000", 2024-09-14T22:39:02.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056351714999999997 2024-09-14T22:39:02.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054771+0000", 2024-09-14T22:39:02.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1009999999999999e-05 2024-09-14T22:39:02.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.277 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.277 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054896+0000", 2024-09-14T22:39:02.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000124714 2024-09-14T22:39:02.277 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.277 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.277 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.278 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:122961 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 2351104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:39:02.278 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:41.998357+0000", 2024-09-14T22:39:02.278 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 80.265125253999997, 2024-09-14T22:39:02.278 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056974912000000003, 2024-09-14T22:39:02.278 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.279 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.279 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.279 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.279 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.279 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.279 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 122961 2024-09-14T22:39:02.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.280 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998357+0000", 2024-09-14T22:39:02.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998357+0000", 2024-09-14T22:39:02.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998358+0000", 2024-09-14T22:39:02.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9699999999999996e-07 2024-09-14T22:39:02.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998361+0000", 2024-09-14T22:39:02.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1370000000000002e-06 2024-09-14T22:39:02.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998371+0000", 2024-09-14T22:39:02.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5789999999999993e-06 2024-09-14T22:39:02.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:41.998379+0000", 2024-09-14T22:39:02.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0129999999999993e-06 2024-09-14T22:39:02.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.283 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.054998+0000", 2024-09-14T22:39:02.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056618961000000002 2024-09-14T22:39:02.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.283 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.283 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.055126+0000", 2024-09-14T22:39:02.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012795100000000001 2024-09-14T22:39:02.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.283 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.283 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.055332+0000", 2024-09-14T22:39:02.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020667400000000001 2024-09-14T22:39:02.283 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.283 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.284 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.284 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123259 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 741376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:39:02.284 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:39:02.284 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 80.251399737, 2024-09-14T22:39:02.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056672581999999999, 2024-09-14T22:39:02.284 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.284 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.284 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.284 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.284 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.284 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.284 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123259 2024-09-14T22:39:02.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.285 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:39:02.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:39:02.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012083+0000", 2024-09-14T22:39:02.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1600000000000001e-07 2024-09-14T22:39:02.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012086+0000", 2024-09-14T22:39:02.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8499999999999998e-06 2024-09-14T22:39:02.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012097+0000", 2024-09-14T22:39:02.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0943e-05 2024-09-14T22:39:02.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.287 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.287 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012105+0000", 2024-09-14T22:39:02.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0830000000000004e-06 2024-09-14T22:39:02.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.287 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.287 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068584+0000", 2024-09-14T22:39:02.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056478387999999997 2024-09-14T22:39:02.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.287 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.287 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068644+0000", 2024-09-14T22:39:02.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9984e-05 2024-09-14T22:39:02.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.288 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.288 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068755+0000", 2024-09-14T22:39:02.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111718 2024-09-14T22:39:02.288 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.288 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.288 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.288 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123260 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 778240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:39:02.288 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012118+0000", 2024-09-14T22:39:02.288 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 80.251364511999995, 2024-09-14T22:39:02.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056825887999999998, 2024-09-14T22:39:02.289 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.289 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.289 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.289 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.289 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.289 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.289 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123260 2024-09-14T22:39:02.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.289 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012118+0000", 2024-09-14T22:39:02.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012118+0000", 2024-09-14T22:39:02.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012119+0000", 2024-09-14T22:39:02.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.13e-07 2024-09-14T22:39:02.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012130+0000", 2024-09-14T22:39:02.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1552999999999999e-05 2024-09-14T22:39:02.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012141+0000", 2024-09-14T22:39:02.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.03e-05 2024-09-14T22:39:02.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012149+0000", 2024-09-14T22:39:02.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1419999999999993e-06 2024-09-14T22:39:02.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068800+0000", 2024-09-14T22:39:02.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056651574000000003 2024-09-14T22:39:02.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.292 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.292 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068858+0000", 2024-09-14T22:39:02.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8047000000000002e-05 2024-09-14T22:39:02.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.292 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.292 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.068944+0000", 2024-09-14T22:39:02.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5659000000000001e-05 2024-09-14T22:39:02.292 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.292 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.292 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.293 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.293 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123261 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 761856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:39:02.293 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012161+0000", 2024-09-14T22:39:02.293 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 80.251321110999996, 2024-09-14T22:39:02.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057113753000000003, 2024-09-14T22:39:02.293 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.293 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.293 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.293 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.293 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.293 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.293 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123261 2024-09-14T22:39:02.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.294 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012161+0000", 2024-09-14T22:39:02.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012161+0000", 2024-09-14T22:39:02.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012162+0000", 2024-09-14T22:39:02.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4399999999999999e-07 2024-09-14T22:39:02.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.295 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012177+0000", 2024-09-14T22:39:02.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5099000000000001e-05 2024-09-14T22:39:02.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.295 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012191+0000", 2024-09-14T22:39:02.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3631999999999999e-05 2024-09-14T22:39:02.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012199+0000", 2024-09-14T22:39:02.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5769999999999994e-06 2024-09-14T22:39:02.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.296 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069004+0000", 2024-09-14T22:39:02.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056804504999999998 2024-09-14T22:39:02.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.296 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069137+0000", 2024-09-14T22:39:02.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013302600000000001 2024-09-14T22:39:02.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.297 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069275+0000", 2024-09-14T22:39:02.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013816999999999999 2024-09-14T22:39:02.297 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.297 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.297 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.297 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123262 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 724992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:39:02.297 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012224+0000", 2024-09-14T22:39:02.297 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 80.251258664000005, 2024-09-14T22:39:02.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057417959999999997, 2024-09-14T22:39:02.297 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.298 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.298 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.298 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.298 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.298 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.298 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123262 2024-09-14T22:39:02.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.298 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012224+0000", 2024-09-14T22:39:02.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012224+0000", 2024-09-14T22:39:02.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012225+0000", 2024-09-14T22:39:02.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4000000000000001e-07 2024-09-14T22:39:02.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012228+0000", 2024-09-14T22:39:02.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5729999999999999e-06 2024-09-14T22:39:02.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012241+0000", 2024-09-14T22:39:02.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3210000000000001e-05 2024-09-14T22:39:02.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012255+0000", 2024-09-14T22:39:02.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3852000000000001e-05 2024-09-14T22:39:02.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069341+0000", 2024-09-14T22:39:02.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057086139000000001 2024-09-14T22:39:02.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069477+0000", 2024-09-14T22:39:02.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013595 2024-09-14T22:39:02.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069642+0000", 2024-09-14T22:39:02.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016459600000000001 2024-09-14T22:39:02.301 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.301 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.301 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.302 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123263 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 782336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:39:02.302 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012265+0000", 2024-09-14T22:39:02.302 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 80.251217122, 2024-09-14T22:39:02.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057665095, 2024-09-14T22:39:02.302 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.302 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.302 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.302 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.302 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.302 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.302 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123263 2024-09-14T22:39:02.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.302 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012265+0000", 2024-09-14T22:39:02.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012265+0000", 2024-09-14T22:39:02.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012266+0000", 2024-09-14T22:39:02.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8999999999999996e-07 2024-09-14T22:39:02.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012269+0000", 2024-09-14T22:39:02.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.796e-06 2024-09-14T22:39:02.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012279+0000", 2024-09-14T22:39:02.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7389999999999994e-06 2024-09-14T22:39:02.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012286+0000", 2024-09-14T22:39:02.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8199999999999997e-06 2024-09-14T22:39:02.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069705+0000", 2024-09-14T22:39:02.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057418590999999998 2024-09-14T22:39:02.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069799+0000", 2024-09-14T22:39:02.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4419999999999994e-05 2024-09-14T22:39:02.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.306 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069931+0000", 2024-09-14T22:39:02.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013113900000000001 2024-09-14T22:39:02.306 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.306 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.306 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.306 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123264 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 745472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:39:02.306 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012295+0000", 2024-09-14T22:39:02.306 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 80.251187121000001, 2024-09-14T22:39:02.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057959498999999998, 2024-09-14T22:39:02.306 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.307 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.307 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.307 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.307 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.307 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.307 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123264 2024-09-14T22:39:02.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.307 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012295+0000", 2024-09-14T22:39:02.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012295+0000", 2024-09-14T22:39:02.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012296+0000", 2024-09-14T22:39:02.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9699999999999996e-07 2024-09-14T22:39:02.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012299+0000", 2024-09-14T22:39:02.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.79e-06 2024-09-14T22:39:02.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012320+0000", 2024-09-14T22:39:02.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1242000000000001e-05 2024-09-14T22:39:02.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012338+0000", 2024-09-14T22:39:02.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7768999999999999e-05 2024-09-14T22:39:02.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.069995+0000", 2024-09-14T22:39:02.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057657 2024-09-14T22:39:02.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070132+0000", 2024-09-14T22:39:02.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013727800000000001 2024-09-14T22:39:02.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070255+0000", 2024-09-14T22:39:02.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012282300000000001 2024-09-14T22:39:02.310 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.310 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.310 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.311 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123265 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1056768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:39:02.311 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012348+0000", 2024-09-14T22:39:02.311 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 80.251134166, 2024-09-14T22:39:02.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058145087999999998, 2024-09-14T22:39:02.311 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.311 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.311 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.311 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.311 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.311 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.311 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123265 2024-09-14T22:39:02.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.311 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.312 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.312 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012348+0000", 2024-09-14T22:39:02.312 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.312 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.312 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012348+0000", 2024-09-14T22:39:02.312 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.312 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.312 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012349+0000", 2024-09-14T22:39:02.312 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6599999999999996e-07 2024-09-14T22:39:02.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012352+0000", 2024-09-14T22:39:02.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3139999999999999e-06 2024-09-14T22:39:02.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012365+0000", 2024-09-14T22:39:02.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2625e-05 2024-09-14T22:39:02.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012373+0000", 2024-09-14T22:39:02.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2470000000000008e-06 2024-09-14T22:39:02.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070286+0000", 2024-09-14T22:39:02.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057912547000000002 2024-09-14T22:39:02.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070366+0000", 2024-09-14T22:39:02.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0468000000000003e-05 2024-09-14T22:39:02.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070493+0000", 2024-09-14T22:39:02.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012722100000000001 2024-09-14T22:39:02.315 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.315 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.315 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.315 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123266 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 765952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:39:02.315 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:39:02.315 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 80.251099917999994, 2024-09-14T22:39:02.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058386949, 2024-09-14T22:39:02.315 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.316 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.316 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.316 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.316 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.316 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.316 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123266 2024-09-14T22:39:02.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.316 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.316 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.316 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:39:02.316 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:39:02.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012383+0000", 2024-09-14T22:39:02.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9299999999999998e-07 2024-09-14T22:39:02.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012386+0000", 2024-09-14T22:39:02.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8729999999999999e-06 2024-09-14T22:39:02.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012406+0000", 2024-09-14T22:39:02.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0103e-05 2024-09-14T22:39:02.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012419+0000", 2024-09-14T22:39:02.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.311e-05 2024-09-14T22:39:02.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070550+0000", 2024-09-14T22:39:02.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058130902999999998 2024-09-14T22:39:02.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070650+0000", 2024-09-14T22:39:02.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010005000000000001 2024-09-14T22:39:02.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070770+0000", 2024-09-14T22:39:02.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119317 2024-09-14T22:39:02.319 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.319 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.319 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.320 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123267 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1077248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:39:02.320 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.012429+0000", 2024-09-14T22:39:02.320 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 80.251054003999997, 2024-09-14T22:39:02.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058620343999999998, 2024-09-14T22:39:02.320 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.320 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.320 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.320 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.320 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.320 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.321 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123267 2024-09-14T22:39:02.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.321 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.321 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.321 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012429+0000", 2024-09-14T22:39:02.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.321 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.321 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.012429+0000", 2024-09-14T22:39:02.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013488+0000", 2024-09-14T22:39:02.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001059061 2024-09-14T22:39:02.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013499+0000", 2024-09-14T22:39:02.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1348999999999999e-05 2024-09-14T22:39:02.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013534+0000", 2024-09-14T22:39:02.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5206000000000001e-05 2024-09-14T22:39:02.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013558+0000", 2024-09-14T22:39:02.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4324999999999998e-05 2024-09-14T22:39:02.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070834+0000", 2024-09-14T22:39:02.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057275110999999997 2024-09-14T22:39:02.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.070937+0000", 2024-09-14T22:39:02.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010306099999999999 2024-09-14T22:39:02.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071049+0000", 2024-09-14T22:39:02.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112231 2024-09-14T22:39:02.324 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.324 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.324 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.324 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123268 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1097728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:39:02.324 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013593+0000", 2024-09-14T22:39:02.324 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 80.249889280999994, 2024-09-14T22:39:02.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057751793000000003, 2024-09-14T22:39:02.325 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.325 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.325 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.325 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.325 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.325 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.325 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123268 2024-09-14T22:39:02.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.325 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.325 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013593+0000", 2024-09-14T22:39:02.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013593+0000", 2024-09-14T22:39:02.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013595+0000", 2024-09-14T22:39:02.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.59e-06 2024-09-14T22:39:02.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013602+0000", 2024-09-14T22:39:02.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3560000000000004e-06 2024-09-14T22:39:02.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013625+0000", 2024-09-14T22:39:02.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2419e-05 2024-09-14T22:39:02.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013650+0000", 2024-09-14T22:39:02.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5151999999999999e-05 2024-09-14T22:39:02.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071096+0000", 2024-09-14T22:39:02.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057445877999999999 2024-09-14T22:39:02.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071212+0000", 2024-09-14T22:39:02.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116215 2024-09-14T22:39:02.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071345+0000", 2024-09-14T22:39:02.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013318299999999999 2024-09-14T22:39:02.328 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.328 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.328 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.329 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123269 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1060864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:39:02.329 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013676+0000", 2024-09-14T22:39:02.329 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 80.249806522, 2024-09-14T22:39:02.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058027982999999998, 2024-09-14T22:39:02.329 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.329 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.329 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.329 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.329 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.329 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.329 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123269 2024-09-14T22:39:02.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.330 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013676+0000", 2024-09-14T22:39:02.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013676+0000", 2024-09-14T22:39:02.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013677+0000", 2024-09-14T22:39:02.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.387e-06 2024-09-14T22:39:02.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013684+0000", 2024-09-14T22:39:02.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0530000000000001e-06 2024-09-14T22:39:02.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013706+0000", 2024-09-14T22:39:02.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1121999999999999e-05 2024-09-14T22:39:02.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013722+0000", 2024-09-14T22:39:02.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6249999999999999e-05 2024-09-14T22:39:02.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071402+0000", 2024-09-14T22:39:02.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057679945000000003 2024-09-14T22:39:02.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071547+0000", 2024-09-14T22:39:02.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000145632 2024-09-14T22:39:02.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071704+0000", 2024-09-14T22:39:02.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000156594 2024-09-14T22:39:02.333 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.333 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.333 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.333 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123270 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1064960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:39:02.333 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013744+0000", 2024-09-14T22:39:02.333 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 80.249738570999995, 2024-09-14T22:39:02.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058355872000000003, 2024-09-14T22:39:02.334 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.334 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.334 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.334 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.334 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.334 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.334 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123270 2024-09-14T22:39:02.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.334 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013744+0000", 2024-09-14T22:39:02.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013744+0000", 2024-09-14T22:39:02.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013745+0000", 2024-09-14T22:39:02.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2589999999999999e-06 2024-09-14T22:39:02.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013751+0000", 2024-09-14T22:39:02.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1929999999999998e-06 2024-09-14T22:39:02.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013778+0000", 2024-09-14T22:39:02.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6472e-05 2024-09-14T22:39:02.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013795+0000", 2024-09-14T22:39:02.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6626e-05 2024-09-14T22:39:02.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071775+0000", 2024-09-14T22:39:02.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057980232 2024-09-14T22:39:02.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.071911+0000", 2024-09-14T22:39:02.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000136129 2024-09-14T22:39:02.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072100+0000", 2024-09-14T22:39:02.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000188961 2024-09-14T22:39:02.337 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.337 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.338 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.338 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123271 3.1 3:815ac4d2:::rbd_data.113c9668137.0000000000000000:1d [read 1081344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:39:02.338 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.013818+0000", 2024-09-14T22:39:02.338 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 80.249664374999995, 2024-09-14T22:39:02.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058549565999999997, 2024-09-14T22:39:02.338 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.338 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.338 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.338 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.338 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.338 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.338 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123271 2024-09-14T22:39:02.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.339 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013818+0000", 2024-09-14T22:39:02.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013818+0000", 2024-09-14T22:39:02.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013958+0000", 2024-09-14T22:39:02.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013934 2024-09-14T22:39:02.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013968+0000", 2024-09-14T22:39:02.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0329000000000001e-05 2024-09-14T22:39:02.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.013995+0000", 2024-09-14T22:39:02.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7613000000000001e-05 2024-09-14T22:39:02.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014014+0000", 2024-09-14T22:39:02.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8338999999999998e-05 2024-09-14T22:39:02.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072155+0000", 2024-09-14T22:39:02.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058140989999999997 2024-09-14T22:39:02.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072247+0000", 2024-09-14T22:39:02.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1774999999999995e-05 2024-09-14T22:39:02.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072368+0000", 2024-09-14T22:39:02.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012118 2024-09-14T22:39:02.342 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.342 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.342 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.342 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123272 3.1 3.4b235a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:39:02.342 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.014046+0000", 2024-09-14T22:39:02.342 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 80.249436699, 2024-09-14T22:39:02.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058364077, 2024-09-14T22:39:02.342 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.342 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.342 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-14T22:39:02.343 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.343 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.343 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.343 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123272 2024-09-14T22:39:02.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.343 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.343 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.343 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014046+0000", 2024-09-14T22:39:02.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.343 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.343 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014046+0000", 2024-09-14T22:39:02.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014047+0000", 2024-09-14T22:39:02.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.533e-06 2024-09-14T22:39:02.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014085+0000", 2024-09-14T22:39:02.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7734999999999997e-05 2024-09-14T22:39:02.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014111+0000", 2024-09-14T22:39:02.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6055999999999999e-05 2024-09-14T22:39:02.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014136+0000", 2024-09-14T22:39:02.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4592000000000001e-05 2024-09-14T22:39:02.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072402+0000", 2024-09-14T22:39:02.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058266251999999998 2024-09-14T22:39:02.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072410+0000", 2024-09-14T22:39:02.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.909e-06 2024-09-14T22:39:02.346 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.346 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.346 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.346 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:123273 3.1 3.4b235a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T22:39:02.346 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:42.014161+0000", 2024-09-14T22:39:02.347 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 80.249321671000004, 2024-09-14T22:39:02.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058284415999999999, 2024-09-14T22:39:02.347 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.347 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.347 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-14T22:39:02.347 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.347 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.347 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.347 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 123273 2024-09-14T22:39:02.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.347 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014161+0000", 2024-09-14T22:39:02.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.014161+0000", 2024-09-14T22:39:02.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.017996+0000", 2024-09-14T22:39:02.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0038350670000000002 2024-09-14T22:39:02.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.018009+0000", 2024-09-14T22:39:02.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3305999999999999e-05 2024-09-14T22:39:02.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.018044+0000", 2024-09-14T22:39:02.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4792000000000002e-05 2024-09-14T22:39:02.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.018067+0000", 2024-09-14T22:39:02.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2595e-05 2024-09-14T22:39:02.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072439+0000", 2024-09-14T22:39:02.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054372192 2024-09-14T22:39:02.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:42.072445+0000", 2024-09-14T22:39:02.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4640000000000003e-06 2024-09-14T22:39:02.350 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.350 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.350 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.350 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:148469 3.18 3:1f6f7244:::rbd_header.113c827b523f:head [watch reconnect cookie 140250011614800 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-14T22:39:02.351 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:55.746449+0000", 2024-09-14T22:39:02.351 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 66.517033058999999, 2024-09-14T22:39:02.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.996484173, 2024-09-14T22:39:02.351 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.351 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.351 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.351 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.351 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.351 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.351 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 148469 2024-09-14T22:39:02.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.351 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746449+0000", 2024-09-14T22:39:02.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746449+0000", 2024-09-14T22:39:02.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746452+0000", 2024-09-14T22:39:02.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0019999999999998e-06 2024-09-14T22:39:02.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746461+0000", 2024-09-14T22:39:02.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0919999999999998e-06 2024-09-14T22:39:02.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746475+0000", 2024-09-14T22:39:02.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4853e-05 2024-09-14T22:39:02.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746484+0000", 2024-09-14T22:39:02.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3780000000000001e-06 2024-09-14T22:39:02.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746559+0000", 2024-09-14T22:39:02.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5259000000000006e-05 2024-09-14T22:39:02.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:39:02.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.746572+0000", 2024-09-14T22:39:02.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.271e-05 2024-09-14T22:39:02.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727807+0000", 2024-09-14T22:39:02.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98123555900000003 2024-09-14T22:39:02.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:39:02.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727826+0000", 2024-09-14T22:39:02.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8828e-05 2024-09-14T22:39:02.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.739033+0000", 2024-09-14T22:39:02.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.011207062 2024-09-14T22:39:02.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.739479+0000", 2024-09-14T22:39:02.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044563200000000001 2024-09-14T22:39:02.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.742934+0000", 2024-09-14T22:39:02.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0034547979999999998 2024-09-14T22:39:02.356 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.356 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.356 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.356 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:148473 3.18 3:1f6f7244:::rbd_header.113c827b523f:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-14T22:39:02.356 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:55.755969+0000", 2024-09-14T22:39:02.356 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 66.507513293000002, 2024-09-14T22:39:02.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9872532889999999, 2024-09-14T22:39:02.357 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.357 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.357 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:39:02.357 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.357 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.357 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.357 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 148473 2024-09-14T22:39:02.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.357 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755969+0000", 2024-09-14T22:39:02.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755969+0000", 2024-09-14T22:39:02.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755971+0000", 2024-09-14T22:39:02.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.674e-06 2024-09-14T22:39:02.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755979+0000", 2024-09-14T22:39:02.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8599999999999993e-06 2024-09-14T22:39:02.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.359 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.755992+0000", 2024-09-14T22:39:02.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.361e-05 2024-09-14T22:39:02.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.359 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.756001+0000", 2024-09-14T22:39:02.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.422e-06 2024-09-14T22:39:02.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.359 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.756069+0000", 2024-09-14T22:39:02.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8209999999999999e-05 2024-09-14T22:39:02.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:39:02.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:55.756077+0000", 2024-09-14T22:39:02.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.994e-06 2024-09-14T22:39:02.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727872+0000", 2024-09-14T22:39:02.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.97179484999999999 2024-09-14T22:39:02.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:39:02.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.727885+0000", 2024-09-14T22:39:02.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3101e-05 2024-09-14T22:39:02.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.740559+0000", 2024-09-14T22:39:02.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0126736300000001 2024-09-14T22:39:02.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.740777+0000", 2024-09-14T22:39:02.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021822000000000001 2024-09-14T22:39:02.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 2", 2024-09-14T22:39:02.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.741114+0000", 2024-09-14T22:39:02.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000336793 2024-09-14T22:39:02.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:39:02.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.742668+0000", 2024-09-14T22:39:02.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001554061 2024-09-14T22:39:02.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:39:02.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.742947+0000", 2024-09-14T22:39:02.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027937200000000001 2024-09-14T22:39:02.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:39:02.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743030+0000", 2024-09-14T22:39:02.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3385000000000002e-05 2024-09-14T22:39:02.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743223+0000", 2024-09-14T22:39:02.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00019210700000000001 2024-09-14T22:39:02.363 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.363 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.363 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.363 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:148474 3.18 3:1f6f7244:::rbd_header.113c827b523f:head [watch reconnect cookie 140250011614800 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104)", 2024-09-14T22:39:02.363 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:56.728273+0000", 2024-09-14T22:39:02.363 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 65.535210046000003, 2024-09-14T22:39:02.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0180497770000001, 2024-09-14T22:39:02.364 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.364 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.364 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.364 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.364 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.364 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.364 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 148474 2024-09-14T22:39:02.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.364 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728273+0000", 2024-09-14T22:39:02.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728273+0000", 2024-09-14T22:39:02.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728274+0000", 2024-09-14T22:39:02.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7570000000000001e-06 2024-09-14T22:39:02.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728281+0000", 2024-09-14T22:39:02.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5089999999999999e-06 2024-09-14T22:39:02.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728298+0000", 2024-09-14T22:39:02.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6727000000000001e-05 2024-09-14T22:39:02.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.728309+0000", 2024-09-14T22:39:02.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1021e-05 2024-09-14T22:39:02.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.730078+0000", 2024-09-14T22:39:02.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017694010000000001 2024-09-14T22:39:02.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:39:02.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:56.730096+0000", 2024-09-14T22:39:02.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8039000000000001e-05 2024-09-14T22:39:02.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.741672+0000", 2024-09-14T22:39:02.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.011575817 2024-09-14T22:39:02.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2024-09-14T22:39:02.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.741827+0000", 2024-09-14T22:39:02.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000155563 2024-09-14T22:39:02.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743299+0000", 2024-09-14T22:39:02.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0014712729999999999 2024-09-14T22:39:02.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.743415+0000", 2024-09-14T22:39:02.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011594700000000001 2024-09-14T22:39:02.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:57.746322+0000", 2024-09-14T22:39:02.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0029077230000000001 2024-09-14T22:39:02.368 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.369 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.369 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.369 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.369 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.369 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:39:02.370 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:39:02.371 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:39:02.362+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:39:02.471 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-14T22:39:02.468+0000 7fabe92d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26329) UID: 0 2024-09-14T22:39:02.545 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:39:02.545 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:39:02.546 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:39:02.546 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:39:02.560 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:39:02.602 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-14T22:39:02.600+0000 7f73a7308640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35913) UID: 0 2024-09-14T22:39:02.672 INFO:tasks.ceph.osd.6.smithi176.stderr:2024-09-14T22:39:02.666+0000 7f003d7ba640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26506) UID: 0 2024-09-14T22:39:02.730 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:39:02.730 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:39:02.730 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:39:02.730 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:39:02.730 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:39:02.744 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:39:02.766 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2024-09-14T22:39:02.774 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-14T22:39:02.768+0000 7feb67124640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26335) UID: 0 2024-09-14T22:39:02.875 INFO:tasks.ceph.osd.7.smithi176.stderr:2024-09-14T22:39:02.866+0000 7f8b294aa640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26510) UID: 0 2024-09-14T22:39:02.893 INFO:teuthology.orchestra.run.smithi001.stdout:All operations completed A-OK! 2024-09-14T22:39:02.895 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:39:02.895 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:39:02.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:39:02.895 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:39:02.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.895 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:31370 3.0 3:02f48f58:::rbd_data.113cda0183a0.0000000000000000:6 [read 813056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T22:39:02.895 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:39:02.895 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 149.529798279, 2024-09-14T22:39:02.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047519796000000003, 2024-09-14T22:39:02.895 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.896 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.896 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.896 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.896 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.896 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.896 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 31370 2024-09-14T22:39:02.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.896 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:39:02.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:39:02.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362398+0000", 2024-09-14T22:39:02.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0999999999999999e-07 2024-09-14T22:39:02.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362401+0000", 2024-09-14T22:39:02.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2900000000000001e-06 2024-09-14T22:39:02.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362406+0000", 2024-09-14T22:39:02.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6960000000000002e-06 2024-09-14T22:39:02.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.362412+0000", 2024-09-14T22:39:02.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.823e-06 2024-09-14T22:39:02.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.400270+0000", 2024-09-14T22:39:02.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037857737000000002 2024-09-14T22:39:02.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.400342+0000", 2024-09-14T22:39:02.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1797000000000004e-05 2024-09-14T22:39:02.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:36:33.409918+0000", 2024-09-14T22:39:02.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0095760429999999994 2024-09-14T22:39:02.899 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.899 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.899 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.900 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:113947 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 5603328~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:39:02.900 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.923453+0000", 2024-09-14T22:39:02.900 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 93.968743146999998, 2024-09-14T22:39:02.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047310420999999998, 2024-09-14T22:39:02.900 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.900 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.900 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.900 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.900 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.900 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.900 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 113947 2024-09-14T22:39:02.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.900 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.923453+0000", 2024-09-14T22:39:02.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.923453+0000", 2024-09-14T22:39:02.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926004+0000", 2024-09-14T22:39:02.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002551382 2024-09-14T22:39:02.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926009+0000", 2024-09-14T22:39:02.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5639999999999998e-06 2024-09-14T22:39:02.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926022+0000", 2024-09-14T22:39:02.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2734999999999999e-05 2024-09-14T22:39:02.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.926029+0000", 2024-09-14T22:39:02.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2520000000000002e-06 2024-09-14T22:39:02.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.970650+0000", 2024-09-14T22:39:02.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044621277000000001 2024-09-14T22:39:02.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.970700+0000", 2024-09-14T22:39:02.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9713000000000003e-05 2024-09-14T22:39:02.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.970763+0000", 2024-09-14T22:39:02.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3497999999999994e-05 2024-09-14T22:39:02.903 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.904 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.904 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.904 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114136 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3448832~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:39:02.904 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932179+0000", 2024-09-14T22:39:02.904 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 93.960016780999993, 2024-09-14T22:39:02.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047236017999999998, 2024-09-14T22:39:02.904 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.904 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.904 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.904 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.904 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.904 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.904 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114136 2024-09-14T22:39:02.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.905 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932179+0000", 2024-09-14T22:39:02.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932179+0000", 2024-09-14T22:39:02.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932324+0000", 2024-09-14T22:39:02.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014451000000000001 2024-09-14T22:39:02.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.906 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.906 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932328+0000", 2024-09-14T22:39:02.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5889999999999996e-06 2024-09-14T22:39:02.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.906 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.906 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932338+0000", 2024-09-14T22:39:02.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3000000000000007e-06 2024-09-14T22:39:02.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.906 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.906 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932344+0000", 2024-09-14T22:39:02.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7150000000000001e-06 2024-09-14T22:39:02.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979233+0000", 2024-09-14T22:39:02.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046888711 2024-09-14T22:39:02.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979313+0000", 2024-09-14T22:39:02.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9487000000000007e-05 2024-09-14T22:39:02.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979415+0000", 2024-09-14T22:39:02.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000102706 2024-09-14T22:39:02.908 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.908 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.908 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.908 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114138 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3497984~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:39:02.908 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932356+0000", 2024-09-14T22:39:02.908 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 93.959839895000002, 2024-09-14T22:39:02.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047285958000000003, 2024-09-14T22:39:02.908 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.908 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.908 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.908 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.908 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.908 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.909 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114138 2024-09-14T22:39:02.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.909 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932356+0000", 2024-09-14T22:39:02.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932356+0000", 2024-09-14T22:39:02.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932498+0000", 2024-09-14T22:39:02.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000142182 2024-09-14T22:39:02.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932503+0000", 2024-09-14T22:39:02.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.301e-06 2024-09-14T22:39:02.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932514+0000", 2024-09-14T22:39:02.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1484000000000001e-05 2024-09-14T22:39:02.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932521+0000", 2024-09-14T22:39:02.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0740000000000004e-06 2024-09-14T22:39:02.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979451+0000", 2024-09-14T22:39:02.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046929379 2024-09-14T22:39:02.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979535+0000", 2024-09-14T22:39:02.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4423000000000001e-05 2024-09-14T22:39:02.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979642+0000", 2024-09-14T22:39:02.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010711499999999999 2024-09-14T22:39:02.912 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.912 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.912 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.912 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114139 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3547136~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:39:02.912 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932554+0000", 2024-09-14T22:39:02.912 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 93.959642127999999, 2024-09-14T22:39:02.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047292230999999997, 2024-09-14T22:39:02.912 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.912 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.912 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.913 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.913 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.913 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.913 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114139 2024-09-14T22:39:02.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.913 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932554+0000", 2024-09-14T22:39:02.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932554+0000", 2024-09-14T22:39:02.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932819+0000", 2024-09-14T22:39:02.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026538400000000002 2024-09-14T22:39:02.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932825+0000", 2024-09-14T22:39:02.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3469999999999998e-06 2024-09-14T22:39:02.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932836+0000", 2024-09-14T22:39:02.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1e-05 2024-09-14T22:39:02.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932843+0000", 2024-09-14T22:39:02.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1790000000000002e-06 2024-09-14T22:39:02.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979673+0000", 2024-09-14T22:39:02.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046830389 2024-09-14T22:39:02.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979750+0000", 2024-09-14T22:39:02.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6312999999999999e-05 2024-09-14T22:39:02.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979846+0000", 2024-09-14T22:39:02.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6619000000000004e-05 2024-09-14T22:39:02.916 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.916 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.916 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.916 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114141 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3596288~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:39:02.916 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.932855+0000", 2024-09-14T22:39:02.916 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 93.959341530000003, 2024-09-14T22:39:02.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047201870999999999, 2024-09-14T22:39:02.917 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.917 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.917 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.917 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.917 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.917 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.917 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114141 2024-09-14T22:39:02.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.917 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932855+0000", 2024-09-14T22:39:02.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.932855+0000", 2024-09-14T22:39:02.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934033+0000", 2024-09-14T22:39:02.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001178264 2024-09-14T22:39:02.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934038+0000", 2024-09-14T22:39:02.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4820000000000002e-06 2024-09-14T22:39:02.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934048+0000", 2024-09-14T22:39:02.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9890000000000004e-06 2024-09-14T22:39:02.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934055+0000", 2024-09-14T22:39:02.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8329999999999996e-06 2024-09-14T22:39:02.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979878+0000", 2024-09-14T22:39:02.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045822773999999997 2024-09-14T22:39:02.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.920 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.979958+0000", 2024-09-14T22:39:02.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0477999999999998e-05 2024-09-14T22:39:02.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.920 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.980056+0000", 2024-09-14T22:39:02.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8051e-05 2024-09-14T22:39:02.920 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.920 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.920 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.920 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114159 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 3940352~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:39:02.920 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.934179+0000", 2024-09-14T22:39:02.921 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 93.958017494000003, 2024-09-14T22:39:02.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047341439999999999, 2024-09-14T22:39:02.921 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.921 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.921 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.921 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.921 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.921 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.921 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114159 2024-09-14T22:39:02.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.921 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934179+0000", 2024-09-14T22:39:02.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.934179+0000", 2024-09-14T22:39:02.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935021+0000", 2024-09-14T22:39:02.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00084276800000000003 2024-09-14T22:39:02.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935027+0000", 2024-09-14T22:39:02.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2959999999999998e-06 2024-09-14T22:39:02.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935038+0000", 2024-09-14T22:39:02.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1198e-05 2024-09-14T22:39:02.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935054+0000", 2024-09-14T22:39:02.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5764e-05 2024-09-14T22:39:02.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.981341+0000", 2024-09-14T22:39:02.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.04628782 2024-09-14T22:39:02.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.981415+0000", 2024-09-14T22:39:02.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3349999999999994e-05 2024-09-14T22:39:02.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.981520+0000", 2024-09-14T22:39:02.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105244 2024-09-14T22:39:02.925 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.925 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.925 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.925 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114180 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4136960~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:39:02.925 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:39:02.925 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 93.957017957999994, 2024-09-14T22:39:02.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047190621000000002, 2024-09-14T22:39:02.925 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.925 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.925 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.925 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.925 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.926 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.926 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114180 2024-09-14T22:39:02.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.926 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:39:02.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:39:02.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935178+0000", 2024-09-14T22:39:02.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6800000000000002e-07 2024-09-14T22:39:02.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935180+0000", 2024-09-14T22:39:02.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.42e-06 2024-09-14T22:39:02.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935185+0000", 2024-09-14T22:39:02.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1359999999999996e-06 2024-09-14T22:39:02.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935188+0000", 2024-09-14T22:39:02.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.711e-06 2024-09-14T22:39:02.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982201+0000", 2024-09-14T22:39:02.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047013309000000003 2024-09-14T22:39:02.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982278+0000", 2024-09-14T22:39:02.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6705000000000005e-05 2024-09-14T22:39:02.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982369+0000", 2024-09-14T22:39:02.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1071999999999998e-05 2024-09-14T22:39:02.929 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.929 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.929 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.929 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114182 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4186112~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:39:02.929 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:39:02.929 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 93.956989336000007, 2024-09-14T22:39:02.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047368990999999999, 2024-09-14T22:39:02.929 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.929 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.930 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.930 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.930 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.930 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.930 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114182 2024-09-14T22:39:02.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.930 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:39:02.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:39:02.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935207+0000", 2024-09-14T22:39:02.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7099999999999998e-07 2024-09-14T22:39:02.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935209+0000", 2024-09-14T22:39:02.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0260000000000002e-06 2024-09-14T22:39:02.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935213+0000", 2024-09-14T22:39:02.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3429999999999998e-06 2024-09-14T22:39:02.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935217+0000", 2024-09-14T22:39:02.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2480000000000001e-06 2024-09-14T22:39:02.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982406+0000", 2024-09-14T22:39:02.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047188920000000002 2024-09-14T22:39:02.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982483+0000", 2024-09-14T22:39:02.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7008999999999994e-05 2024-09-14T22:39:02.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982576+0000", 2024-09-14T22:39:02.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3173999999999999e-05 2024-09-14T22:39:02.933 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.933 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.933 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.933 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114185 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4202496~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:39:02.933 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.935221+0000", 2024-09-14T22:39:02.933 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 93.956975170999996, 2024-09-14T22:39:02.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047601128999999999, 2024-09-14T22:39:02.934 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.934 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.934 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.934 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.934 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.934 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.934 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114185 2024-09-14T22:39:02.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.934 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935221+0000", 2024-09-14T22:39:02.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935221+0000", 2024-09-14T22:39:02.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935970+0000", 2024-09-14T22:39:02.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074930899999999998 2024-09-14T22:39:02.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935976+0000", 2024-09-14T22:39:02.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5439999999999998e-06 2024-09-14T22:39:02.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935988+0000", 2024-09-14T22:39:02.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1779e-05 2024-09-14T22:39:02.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.935996+0000", 2024-09-14T22:39:02.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.106e-06 2024-09-14T22:39:02.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982607+0000", 2024-09-14T22:39:02.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046611504999999998 2024-09-14T22:39:02.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982686+0000", 2024-09-14T22:39:02.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8805000000000002e-05 2024-09-14T22:39:02.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.982822+0000", 2024-09-14T22:39:02.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000136081 2024-09-14T22:39:02.937 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.937 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.937 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.938 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114189 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4300800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:39:02.938 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.936048+0000", 2024-09-14T22:39:02.938 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 93.956148038999999, 2024-09-14T22:39:02.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047231539000000003, 2024-09-14T22:39:02.938 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.938 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.938 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.938 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.938 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.938 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.938 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114189 2024-09-14T22:39:02.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.938 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936048+0000", 2024-09-14T22:39:02.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936048+0000", 2024-09-14T22:39:02.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936049+0000", 2024-09-14T22:39:02.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4099999999999999e-07 2024-09-14T22:39:02.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936052+0000", 2024-09-14T22:39:02.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0029999999999999e-06 2024-09-14T22:39:02.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936058+0000", 2024-09-14T22:39:02.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7880000000000001e-06 2024-09-14T22:39:02.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936064+0000", 2024-09-14T22:39:02.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2279999999999998e-06 2024-09-14T22:39:02.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983120+0000", 2024-09-14T22:39:02.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047056384999999999 2024-09-14T22:39:02.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983194+0000", 2024-09-14T22:39:02.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3811000000000002e-05 2024-09-14T22:39:02.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983280+0000", 2024-09-14T22:39:02.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5883000000000004e-05 2024-09-14T22:39:02.941 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.941 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.942 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.942 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114191 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4349952~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:39:02.942 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:39:02.942 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 93.956124298999995, 2024-09-14T22:39:02.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047365365, 2024-09-14T22:39:02.942 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.942 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.942 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.942 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.942 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.942 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.942 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114191 2024-09-14T22:39:02.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.942 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:39:02.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:39:02.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936072+0000", 2024-09-14T22:39:02.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0499999999999999e-07 2024-09-14T22:39:02.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936075+0000", 2024-09-14T22:39:02.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.734e-06 2024-09-14T22:39:02.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936082+0000", 2024-09-14T22:39:02.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8360000000000003e-06 2024-09-14T22:39:02.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936087+0000", 2024-09-14T22:39:02.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3129999999999998e-06 2024-09-14T22:39:02.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.945 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983319+0000", 2024-09-14T22:39:02.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047232140999999998 2024-09-14T22:39:02.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.945 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983365+0000", 2024-09-14T22:39:02.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5776999999999999e-05 2024-09-14T22:39:02.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983437+0000", 2024-09-14T22:39:02.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2158999999999998e-05 2024-09-14T22:39:02.946 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.946 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.946 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.946 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114193 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4399104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:39:02.946 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.936095+0000", 2024-09-14T22:39:02.946 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 93.956101363000002, 2024-09-14T22:39:02.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047505915000000003, 2024-09-14T22:39:02.946 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.946 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.946 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.947 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.947 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.947 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.947 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114193 2024-09-14T22:39:02.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.947 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936095+0000", 2024-09-14T22:39:02.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.936095+0000", 2024-09-14T22:39:02.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938021+0000", 2024-09-14T22:39:02.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001926178 2024-09-14T22:39:02.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938027+0000", 2024-09-14T22:39:02.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3110000000000002e-06 2024-09-14T22:39:02.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938041+0000", 2024-09-14T22:39:02.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3344e-05 2024-09-14T22:39:02.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.938049+0000", 2024-09-14T22:39:02.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2279999999999998e-06 2024-09-14T22:39:02.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983468+0000", 2024-09-14T22:39:02.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045419494999999997 2024-09-14T22:39:02.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983524+0000", 2024-09-14T22:39:02.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5952000000000002e-05 2024-09-14T22:39:02.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.983601+0000", 2024-09-14T22:39:02.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6407000000000002e-05 2024-09-14T22:39:02.950 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.950 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.950 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.950 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114710 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4857856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:39:02.950 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.962571+0000", 2024-09-14T22:39:02.951 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 93.929625091000005, 2024-09-14T22:39:02.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047253314999999997, 2024-09-14T22:39:02.951 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.951 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.951 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.951 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.951 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.951 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.951 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114710 2024-09-14T22:39:02.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.951 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.951 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.951 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962571+0000", 2024-09-14T22:39:02.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962571+0000", 2024-09-14T22:39:02.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962572+0000", 2024-09-14T22:39:02.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1399999999999997e-07 2024-09-14T22:39:02.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962574+0000", 2024-09-14T22:39:02.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8679999999999999e-06 2024-09-14T22:39:02.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962583+0000", 2024-09-14T22:39:02.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8310000000000007e-06 2024-09-14T22:39:02.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962589+0000", 2024-09-14T22:39:02.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9529999999999996e-06 2024-09-14T22:39:02.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.954 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009692+0000", 2024-09-14T22:39:02.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047103249999999999 2024-09-14T22:39:02.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.954 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.954 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009767+0000", 2024-09-14T22:39:02.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4672000000000006e-05 2024-09-14T22:39:02.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.954 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.954 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009824+0000", 2024-09-14T22:39:02.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7226999999999999e-05 2024-09-14T22:39:02.954 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.954 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.954 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.955 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114712 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4907008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:39:02.955 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:39:02.955 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 93.929599611, 2024-09-14T22:39:02.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047421087000000001, 2024-09-14T22:39:02.955 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.955 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.955 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.955 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.955 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.955 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.955 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114712 2024-09-14T22:39:02.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.956 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:39:02.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:39:02.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962597+0000", 2024-09-14T22:39:02.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.03e-07 2024-09-14T22:39:02.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962599+0000", 2024-09-14T22:39:02.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3199999999999998e-06 2024-09-14T22:39:02.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962606+0000", 2024-09-14T22:39:02.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5289999999999997e-06 2024-09-14T22:39:02.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962612+0000", 2024-09-14T22:39:02.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.947e-06 2024-09-14T22:39:02.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009866+0000", 2024-09-14T22:39:02.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047254681999999999 2024-09-14T22:39:02.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.009924+0000", 2024-09-14T22:39:02.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7556000000000003e-05 2024-09-14T22:39:02.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010018+0000", 2024-09-14T22:39:02.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.365e-05 2024-09-14T22:39:02.959 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.959 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.959 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.959 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114714 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 4956160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:39:02.959 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.962618+0000", 2024-09-14T22:39:02.959 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 93.929578093000003, 2024-09-14T22:39:02.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.04759302, 2024-09-14T22:39:02.959 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.959 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.960 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.960 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.960 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.960 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.960 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114714 2024-09-14T22:39:02.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.960 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962618+0000", 2024-09-14T22:39:02.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.962618+0000", 2024-09-14T22:39:02.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966011+0000", 2024-09-14T22:39:02.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003393067 2024-09-14T22:39:02.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966016+0000", 2024-09-14T22:39:02.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3580000000000002e-06 2024-09-14T22:39:02.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966032+0000", 2024-09-14T22:39:02.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5356e-05 2024-09-14T22:39:02.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966040+0000", 2024-09-14T22:39:02.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1980000000000001e-06 2024-09-14T22:39:02.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010055+0000", 2024-09-14T22:39:02.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044014676000000003 2024-09-14T22:39:02.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010127+0000", 2024-09-14T22:39:02.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2206000000000006e-05 2024-09-14T22:39:02.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.010211+0000", 2024-09-14T22:39:02.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4159000000000005e-05 2024-09-14T22:39:02.963 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.963 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.963 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.963 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114754 3.19 3:9e2a0ff9:::rbd_data.113cf633433b.0000000000000001:1a [read 6283264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:39:02.964 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:39:02.964 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 93.925403180000004, 2024-09-14T22:39:02.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047228150000000003, 2024-09-14T22:39:02.964 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.964 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.964 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:02.964 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.964 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.964 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.964 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114754 2024-09-14T22:39:02.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.964 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:39:02.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:39:02.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966793+0000", 2024-09-14T22:39:02.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7399999999999999e-07 2024-09-14T22:39:02.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966796+0000", 2024-09-14T22:39:02.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3630000000000001e-06 2024-09-14T22:39:02.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966802+0000", 2024-09-14T22:39:02.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4679999999999997e-06 2024-09-14T22:39:02.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966807+0000", 2024-09-14T22:39:02.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1109999999999999e-06 2024-09-14T22:39:02.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.013929+0000", 2024-09-14T22:39:02.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047121720999999998 2024-09-14T22:39:02.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.013968+0000", 2024-09-14T22:39:02.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8877000000000001e-05 2024-09-14T22:39:02.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014021+0000", 2024-09-14T22:39:02.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3236000000000003e-05 2024-09-14T22:39:02.968 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.968 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.968 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.968 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114789 3.19 3:9836bb25:::rbd_object_map.113c14c67864.000000000000001b:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:39:02.968 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.966813+0000", 2024-09-14T22:39:02.968 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 93.925382717999994, 2024-09-14T22:39:02.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05001311, 2024-09-14T22:39:02.968 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.968 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.968 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:39:02.969 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.969 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.969 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.969 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114789 2024-09-14T22:39:02.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.969 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966813+0000", 2024-09-14T22:39:02.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966813+0000", 2024-09-14T22:39:02.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966814+0000", 2024-09-14T22:39:02.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9200000000000002e-07 2024-09-14T22:39:02.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966817+0000", 2024-09-14T22:39:02.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8380000000000002e-06 2024-09-14T22:39:02.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966831+0000", 2024-09-14T22:39:02.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4209999999999999e-05 2024-09-14T22:39:02.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966845+0000", 2024-09-14T22:39:02.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3799e-05 2024-09-14T22:39:02.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014046+0000", 2024-09-14T22:39:02.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047201604000000001 2024-09-14T22:39:02.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014116+0000", 2024-09-14T22:39:02.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9387000000000005e-05 2024-09-14T22:39:02.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-14T22:39:02.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014469+0000", 2024-09-14T22:39:02.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00035355800000000002 2024-09-14T22:39:02.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:39:02.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016719+0000", 2024-09-14T22:39:02.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0022498459999999998 2024-09-14T22:39:02.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:39:02.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016735+0000", 2024-09-14T22:39:02.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5792999999999999e-05 2024-09-14T22:39:02.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.973 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:39:02.973 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016790+0000", 2024-09-14T22:39:02.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4715e-05 2024-09-14T22:39:02.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.973 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.973 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016827+0000", 2024-09-14T22:39:02.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6968000000000002e-05 2024-09-14T22:39:02.973 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.973 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.973 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.974 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:114803 3.19 3:9836bb25:::rbd_object_map.113c14c67864.000000000000001b:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-14T22:39:02.974 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:28.966853+0000", 2024-09-14T22:39:02.974 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 93.925343287999993, 2024-09-14T22:39:02.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050656093999999999, 2024-09-14T22:39:02.974 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.974 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.974 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:39:02.974 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.974 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.974 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.974 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114803 2024-09-14T22:39:02.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.975 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966853+0000", 2024-09-14T22:39:02.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.966853+0000", 2024-09-14T22:39:02.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969930+0000", 2024-09-14T22:39:02.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030769759999999999 2024-09-14T22:39:02.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969936+0000", 2024-09-14T22:39:02.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7150000000000003e-06 2024-09-14T22:39:02.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969949+0000", 2024-09-14T22:39:02.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3166000000000001e-05 2024-09-14T22:39:02.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:28.969957+0000", 2024-09-14T22:39:02.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8269999999999998e-06 2024-09-14T22:39:02.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.014996+0000", 2024-09-14T22:39:02.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045039202 2024-09-14T22:39:02.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.015085+0000", 2024-09-14T22:39:02.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8987999999999996e-05 2024-09-14T22:39:02.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-14T22:39:02.978 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.015318+0000", 2024-09-14T22:39:02.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023284999999999999 2024-09-14T22:39:02.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:39:02.978 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.016898+0000", 2024-09-14T22:39:02.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0015807379999999999 2024-09-14T22:39:02.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:39:02.978 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.017455+0000", 2024-09-14T22:39:02.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00055664600000000005 2024-09-14T22:39:02.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:39:02.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.017483+0000", 2024-09-14T22:39:02.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8238e-05 2024-09-14T22:39:02.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:29.017509+0000", 2024-09-14T22:39:02.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5748000000000001e-05 2024-09-14T22:39:02.979 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.979 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.979 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.979 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:150759 3.8 3:15dc94e5:::rbd_header.113c88f88d4a:head [create,call rbd.create in=9b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e113)", 2024-09-14T22:39:02.980 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:38:05.726302+0000", 2024-09-14T22:39:02.980 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 57.165894596000001, 2024-09-14T22:39:02.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0241160819999999, 2024-09-14T22:39:02.980 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:02.980 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:02.980 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:39:02.980 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:02.980 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:02.980 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:02.980 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 150759 2024-09-14T22:39:02.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.980 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:02.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:02.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.726302+0000", 2024-09-14T22:39:02.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:02.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.726302+0000", 2024-09-14T22:39:02.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:02.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:02.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.726304+0000", 2024-09-14T22:39:02.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.075e-06 2024-09-14T22:39:02.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:02.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.726312+0000", 2024-09-14T22:39:02.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2050000000000002e-06 2024-09-14T22:39:02.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:02.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.727969+0000", 2024-09-14T22:39:02.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001657499 2024-09-14T22:39:02.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:02.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.727985+0000", 2024-09-14T22:39:02.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6082999999999998e-05 2024-09-14T22:39:02.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.728193+0000", 2024-09-14T22:39:02.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000208082 2024-09-14T22:39:02.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-14T22:39:02.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:05.728205+0000", 2024-09-14T22:39:02.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1897e-05 2024-09-14T22:39:02.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:02.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.741926+0000", 2024-09-14T22:39:02.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.013720591 2024-09-14T22:39:02.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:02.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.742076+0000", 2024-09-14T22:39:02.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015024099999999999 2024-09-14T22:39:02.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-14T22:39:02.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.742415+0000", 2024-09-14T22:39:02.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00033887700000000002 2024-09-14T22:39:02.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-14T22:39:02.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.743436+0000", 2024-09-14T22:39:02.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010210639999999999 2024-09-14T22:39:02.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:39:02.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.750216+0000", 2024-09-14T22:39:02.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0067798950000000002 2024-09-14T22:39:02.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-14T22:39:02.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.750288+0000", 2024-09-14T22:39:02.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2192000000000003e-05 2024-09-14T22:39:02.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:02.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:02.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:02.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:38:06.750418+0000", 2024-09-14T22:39:02.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000129381 2024-09-14T22:39:02.985 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.986 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.986 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.986 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:02.986 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:02.986 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:39:02.987 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:39:02.988 DEBUG:teuthology.parallel:result is None 2024-09-14T22:39:02.989 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-14T22:39:02.997 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-14T22:39:03.016 INFO:tasks.thrashosds:joining thrashosds 2024-09-14T22:39:03.016 INFO:tasks.ceph.osd.5.smithi176.stderr:2024-09-14T22:39:02.982+0000 7f17e3ab7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26508) UID: 0 2024-09-14T22:39:03.163 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:39:03.163 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:39:03.163 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:39:03.163 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:39:03.178 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:39:03.305 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:39:03.305 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:39:03.305 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:39:03.306 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:39:03.306 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:39:03.319 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:39:03.442 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:39:03.442 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:39:03.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:39:03.442 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:39:03.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.442 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145249 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 15745024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:03.442 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.693850+0000", 2024-09-14T22:39:03.442 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 70.744308871000001, 2024-09-14T22:39:03.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.060850297999999997, 2024-09-14T22:39:03.442 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:03.442 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:03.442 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:03.443 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:03.443 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:03.443 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.443 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145249 2024-09-14T22:39:03.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.443 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:03.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:03.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693850+0000", 2024-09-14T22:39:03.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:03.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693850+0000", 2024-09-14T22:39:03.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:03.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693851+0000", 2024-09-14T22:39:03.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.46e-07 2024-09-14T22:39:03.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:03.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693853+0000", 2024-09-14T22:39:03.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0940000000000002e-06 2024-09-14T22:39:03.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:03.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693858+0000", 2024-09-14T22:39:03.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2100000000000001e-06 2024-09-14T22:39:03.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693863+0000", 2024-09-14T22:39:03.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5549999999999996e-06 2024-09-14T22:39:03.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:03.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754503+0000", 2024-09-14T22:39:03.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.060640558999999997 2024-09-14T22:39:03.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:03.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754613+0000", 2024-09-14T22:39:03.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109966 2024-09-14T22:39:03.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:03.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754701+0000", 2024-09-14T22:39:03.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7567999999999999e-05 2024-09-14T22:39:03.446 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.446 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:03.446 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.446 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145250 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16039936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:03.446 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.693868+0000", 2024-09-14T22:39:03.446 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 70.744291142999998, 2024-09-14T22:39:03.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061182667000000003, 2024-09-14T22:39:03.446 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:03.447 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:03.447 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:03.447 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:03.447 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:03.447 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.447 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145250 2024-09-14T22:39:03.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.447 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:03.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:03.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693868+0000", 2024-09-14T22:39:03.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:03.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693868+0000", 2024-09-14T22:39:03.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:03.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693869+0000", 2024-09-14T22:39:03.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4400000000000001e-07 2024-09-14T22:39:03.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:03.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693870+0000", 2024-09-14T22:39:03.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9130000000000001e-06 2024-09-14T22:39:03.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:03.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693876+0000", 2024-09-14T22:39:03.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1270000000000002e-06 2024-09-14T22:39:03.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693880+0000", 2024-09-14T22:39:03.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.673e-06 2024-09-14T22:39:03.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:03.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754780+0000", 2024-09-14T22:39:03.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.060899495999999997 2024-09-14T22:39:03.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.450 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:03.450 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.754925+0000", 2024-09-14T22:39:03.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014511799999999999 2024-09-14T22:39:03.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.450 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:03.450 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755051+0000", 2024-09-14T22:39:03.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012599599999999999 2024-09-14T22:39:03.450 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.450 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:03.450 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.450 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145251 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16236544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:03.451 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.693886+0000", 2024-09-14T22:39:03.451 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 70.744273489999998, 2024-09-14T22:39:03.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061498816999999997, 2024-09-14T22:39:03.451 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:03.451 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:03.451 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:03.451 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:03.451 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:03.451 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.451 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145251 2024-09-14T22:39:03.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.451 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:03.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:03.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693886+0000", 2024-09-14T22:39:03.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:03.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693886+0000", 2024-09-14T22:39:03.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:03.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693886+0000", 2024-09-14T22:39:03.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1199999999999998e-07 2024-09-14T22:39:03.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:03.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693888+0000", 2024-09-14T22:39:03.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.877e-06 2024-09-14T22:39:03.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:03.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693894+0000", 2024-09-14T22:39:03.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5350000000000004e-06 2024-09-14T22:39:03.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693917+0000", 2024-09-14T22:39:03.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.296e-05 2024-09-14T22:39:03.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:03.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755127+0000", 2024-09-14T22:39:03.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061210286000000003 2024-09-14T22:39:03.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:03.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755266+0000", 2024-09-14T22:39:03.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013871599999999999 2024-09-14T22:39:03.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:03.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755385+0000", 2024-09-14T22:39:03.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011903100000000001 2024-09-14T22:39:03.454 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.454 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:03.454 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.455 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145252 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16302080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:03.455 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.693985+0000", 2024-09-14T22:39:03.455 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 70.744174807999997, 2024-09-14T22:39:03.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061743767999999997, 2024-09-14T22:39:03.455 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:03.455 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:03.455 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:03.455 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:03.455 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:03.455 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.456 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145252 2024-09-14T22:39:03.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.456 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:03.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:03.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693985+0000", 2024-09-14T22:39:03.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:03.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693985+0000", 2024-09-14T22:39:03.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:03.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693986+0000", 2024-09-14T22:39:03.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0640000000000001e-06 2024-09-14T22:39:03.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:03.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.693991+0000", 2024-09-14T22:39:03.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7599999999999999e-06 2024-09-14T22:39:03.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:03.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694005+0000", 2024-09-14T22:39:03.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3271999999999999e-05 2024-09-14T22:39:03.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.458 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694015+0000", 2024-09-14T22:39:03.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0617e-05 2024-09-14T22:39:03.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:03.458 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755459+0000", 2024-09-14T22:39:03.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061443820000000003 2024-09-14T22:39:03.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:03.459 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755605+0000", 2024-09-14T22:39:03.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014569 2024-09-14T22:39:03.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.459 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:03.459 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755728+0000", 2024-09-14T22:39:03.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000123545 2024-09-14T22:39:03.459 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.459 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:03.459 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.459 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145253 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16138240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:03.459 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694031+0000", 2024-09-14T22:39:03.459 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 70.744128254000003, 2024-09-14T22:39:03.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062032836000000001, 2024-09-14T22:39:03.460 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:03.460 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:03.460 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:03.460 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:03.460 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:03.460 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.460 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145253 2024-09-14T22:39:03.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.460 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:03.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.460 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:03.460 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694031+0000", 2024-09-14T22:39:03.460 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.460 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:03.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694031+0000", 2024-09-14T22:39:03.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:03.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694032+0000", 2024-09-14T22:39:03.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7999999999999995e-07 2024-09-14T22:39:03.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:03.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694036+0000", 2024-09-14T22:39:03.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8800000000000001e-06 2024-09-14T22:39:03.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:03.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694047+0000", 2024-09-14T22:39:03.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0835999999999999e-05 2024-09-14T22:39:03.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694055+0000", 2024-09-14T22:39:03.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6619999999999994e-06 2024-09-14T22:39:03.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:03.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755804+0000", 2024-09-14T22:39:03.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.061748448999999997 2024-09-14T22:39:03.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:03.463 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.755941+0000", 2024-09-14T22:39:03.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013751599999999999 2024-09-14T22:39:03.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.463 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:03.463 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756064+0000", 2024-09-14T22:39:03.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012281299999999999 2024-09-14T22:39:03.463 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.463 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:03.463 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.463 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145254 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16400384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:03.463 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694067+0000", 2024-09-14T22:39:03.463 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 70.744092089000006, 2024-09-14T22:39:03.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062280549999999997, 2024-09-14T22:39:03.464 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:03.464 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:03.464 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:03.464 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:03.464 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:03.464 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.464 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145254 2024-09-14T22:39:03.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.464 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:03.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.464 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:03.464 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694067+0000", 2024-09-14T22:39:03.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.465 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:03.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694067+0000", 2024-09-14T22:39:03.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.465 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:03.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694068+0000", 2024-09-14T22:39:03.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2200000000000003e-07 2024-09-14T22:39:03.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.465 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:03.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694072+0000", 2024-09-14T22:39:03.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5729999999999999e-06 2024-09-14T22:39:03.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:03.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694082+0000", 2024-09-14T22:39:03.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0703999999999999e-05 2024-09-14T22:39:03.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694091+0000", 2024-09-14T22:39:03.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8030000000000003e-06 2024-09-14T22:39:03.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:03.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756155+0000", 2024-09-14T22:39:03.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062063937 2024-09-14T22:39:03.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:03.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756263+0000", 2024-09-14T22:39:03.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108229 2024-09-14T22:39:03.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:03.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756348+0000", 2024-09-14T22:39:03.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4581999999999998e-05 2024-09-14T22:39:03.467 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.467 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:03.467 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.467 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145255 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16596992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:03.467 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694103+0000", 2024-09-14T22:39:03.468 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 70.744056607999994, 2024-09-14T22:39:03.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062580620000000003, 2024-09-14T22:39:03.468 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:03.468 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:03.468 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:03.468 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:03.468 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:03.468 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.468 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145255 2024-09-14T22:39:03.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.468 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:03.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:03.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694103+0000", 2024-09-14T22:39:03.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:03.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694103+0000", 2024-09-14T22:39:03.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:03.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694103+0000", 2024-09-14T22:39:03.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7800000000000001e-07 2024-09-14T22:39:03.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:03.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694107+0000", 2024-09-14T22:39:03.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5939999999999998e-06 2024-09-14T22:39:03.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:03.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694121+0000", 2024-09-14T22:39:03.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3913999999999999e-05 2024-09-14T22:39:03.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694130+0000", 2024-09-14T22:39:03.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7690000000000002e-06 2024-09-14T22:39:03.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:03.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756431+0000", 2024-09-14T22:39:03.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062301593000000002 2024-09-14T22:39:03.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:03.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756564+0000", 2024-09-14T22:39:03.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000132693 2024-09-14T22:39:03.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:03.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756683+0000", 2024-09-14T22:39:03.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119479 2024-09-14T22:39:03.471 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.471 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:03.471 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.472 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145256 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16695296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:03.472 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694142+0000", 2024-09-14T22:39:03.472 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 70.744017534999998, 2024-09-14T22:39:03.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062806034999999996, 2024-09-14T22:39:03.472 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:03.472 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:03.472 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:03.472 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:03.472 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:03.472 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.472 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145256 2024-09-14T22:39:03.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.472 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:03.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:03.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694142+0000", 2024-09-14T22:39:03.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:03.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694142+0000", 2024-09-14T22:39:03.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:03.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694142+0000", 2024-09-14T22:39:03.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1699999999999998e-07 2024-09-14T22:39:03.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:03.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694146+0000", 2024-09-14T22:39:03.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7450000000000001e-06 2024-09-14T22:39:03.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:03.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694157+0000", 2024-09-14T22:39:03.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1161e-05 2024-09-14T22:39:03.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694166+0000", 2024-09-14T22:39:03.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0450000000000006e-06 2024-09-14T22:39:03.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:03.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756768+0000", 2024-09-14T22:39:03.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062601554000000004 2024-09-14T22:39:03.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:03.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756867+0000", 2024-09-14T22:39:03.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8741999999999998e-05 2024-09-14T22:39:03.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:03.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.756948+0000", 2024-09-14T22:39:03.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1170999999999999e-05 2024-09-14T22:39:03.475 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.475 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:03.475 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.475 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145257 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16498688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:03.476 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694179+0000", 2024-09-14T22:39:03.476 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 70.743980235999999, 2024-09-14T22:39:03.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063118469999999996, 2024-09-14T22:39:03.476 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:03.476 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:03.476 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:03.476 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:03.476 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:03.476 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.476 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145257 2024-09-14T22:39:03.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.476 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:03.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:03.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694179+0000", 2024-09-14T22:39:03.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:03.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694179+0000", 2024-09-14T22:39:03.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:03.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694180+0000", 2024-09-14T22:39:03.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1099999999999995e-07 2024-09-14T22:39:03.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:03.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694196+0000", 2024-09-14T22:39:03.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5996000000000001e-05 2024-09-14T22:39:03.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:03.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694208+0000", 2024-09-14T22:39:03.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2584e-05 2024-09-14T22:39:03.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694219+0000", 2024-09-14T22:39:03.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0462000000000001e-05 2024-09-14T22:39:03.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:03.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757033+0000", 2024-09-14T22:39:03.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.062814485000000003 2024-09-14T22:39:03.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:03.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757183+0000", 2024-09-14T22:39:03.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014914099999999999 2024-09-14T22:39:03.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:03.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757298+0000", 2024-09-14T22:39:03.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115091 2024-09-14T22:39:03.479 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.479 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:03.479 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.479 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145258 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16269312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:03.480 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694257+0000", 2024-09-14T22:39:03.480 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 70.743902042000002, 2024-09-14T22:39:03.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063297279999999997, 2024-09-14T22:39:03.480 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:03.480 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:03.480 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:03.480 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:03.480 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:03.480 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.480 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145258 2024-09-14T22:39:03.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.480 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:03.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:03.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694257+0000", 2024-09-14T22:39:03.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:03.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694257+0000", 2024-09-14T22:39:03.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:03.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694258+0000", 2024-09-14T22:39:03.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7700000000000004e-07 2024-09-14T22:39:03.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:03.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694263+0000", 2024-09-14T22:39:03.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6609999999999999e-06 2024-09-14T22:39:03.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:03.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694271+0000", 2024-09-14T22:39:03.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7809999999999994e-06 2024-09-14T22:39:03.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694277+0000", 2024-09-14T22:39:03.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.823e-06 2024-09-14T22:39:03.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:03.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757377+0000", 2024-09-14T22:39:03.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063100081000000002 2024-09-14T22:39:03.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:03.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757476+0000", 2024-09-14T22:39:03.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8507999999999999e-05 2024-09-14T22:39:03.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:03.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757555+0000", 2024-09-14T22:39:03.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8749000000000001e-05 2024-09-14T22:39:03.484 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.484 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:03.484 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.484 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.484 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145259 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16465920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:03.484 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694286+0000", 2024-09-14T22:39:03.484 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 70.743873844999996, 2024-09-14T22:39:03.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063608168000000007, 2024-09-14T22:39:03.484 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:03.484 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:03.484 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:03.484 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:03.485 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:03.485 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.485 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145259 2024-09-14T22:39:03.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.485 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:03.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:03.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694286+0000", 2024-09-14T22:39:03.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:03.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694286+0000", 2024-09-14T22:39:03.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:03.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694286+0000", 2024-09-14T22:39:03.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9999999999999998e-07 2024-09-14T22:39:03.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:03.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694288+0000", 2024-09-14T22:39:03.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3760000000000002e-06 2024-09-14T22:39:03.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:03.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694294+0000", 2024-09-14T22:39:03.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6779999999999997e-06 2024-09-14T22:39:03.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694298+0000", 2024-09-14T22:39:03.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3309999999999997e-06 2024-09-14T22:39:03.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:03.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757638+0000", 2024-09-14T22:39:03.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063339464999999998 2024-09-14T22:39:03.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:03.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757769+0000", 2024-09-14T22:39:03.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013148700000000001 2024-09-14T22:39:03.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.488 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:03.488 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757894+0000", 2024-09-14T22:39:03.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012443099999999999 2024-09-14T22:39:03.488 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.488 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:03.488 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.488 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145260 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16367616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:03.488 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694304+0000", 2024-09-14T22:39:03.488 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 70.743855050999997, 2024-09-14T22:39:03.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063838521999999995, 2024-09-14T22:39:03.489 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:03.489 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:03.489 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:03.489 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:03.489 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:03.489 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.489 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145260 2024-09-14T22:39:03.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.489 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:03.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:03.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694304+0000", 2024-09-14T22:39:03.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:03.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694304+0000", 2024-09-14T22:39:03.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:03.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694305+0000", 2024-09-14T22:39:03.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8299999999999997e-07 2024-09-14T22:39:03.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:03.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694307+0000", 2024-09-14T22:39:03.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9989999999999998e-06 2024-09-14T22:39:03.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:03.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694312+0000", 2024-09-14T22:39:03.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2689999999999999e-06 2024-09-14T22:39:03.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694317+0000", 2024-09-14T22:39:03.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9359999999999998e-06 2024-09-14T22:39:03.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:03.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.757944+0000", 2024-09-14T22:39:03.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063626951000000001 2024-09-14T22:39:03.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.492 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:03.492 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758065+0000", 2024-09-14T22:39:03.492 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120945 2024-09-14T22:39:03.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.492 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.492 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:03.492 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758143+0000", 2024-09-14T22:39:03.492 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7838999999999998e-05 2024-09-14T22:39:03.492 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.492 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:03.492 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.492 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.492 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145261 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16564224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:03.493 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694323+0000", 2024-09-14T22:39:03.493 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 70.743836348000002, 2024-09-14T22:39:03.493 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064150180000000001, 2024-09-14T22:39:03.493 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:03.493 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:03.493 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:03.493 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:03.493 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:03.493 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.493 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145261 2024-09-14T22:39:03.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.493 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:03.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.493 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:03.494 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694323+0000", 2024-09-14T22:39:03.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.494 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:03.494 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694323+0000", 2024-09-14T22:39:03.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.494 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:03.494 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694323+0000", 2024-09-14T22:39:03.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4900000000000001e-07 2024-09-14T22:39:03.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.494 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:03.494 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694325+0000", 2024-09-14T22:39:03.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.04e-06 2024-09-14T22:39:03.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:03.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694331+0000", 2024-09-14T22:39:03.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3449999999999996e-06 2024-09-14T22:39:03.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694335+0000", 2024-09-14T22:39:03.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.566e-06 2024-09-14T22:39:03.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:03.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758225+0000", 2024-09-14T22:39:03.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063889535999999997 2024-09-14T22:39:03.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.496 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:03.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758357+0000", 2024-09-14T22:39:03.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013181 2024-09-14T22:39:03.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.496 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:03.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758473+0000", 2024-09-14T22:39:03.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011653400000000001 2024-09-14T22:39:03.496 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.496 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:03.496 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.497 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145262 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16760832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:03.497 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694341+0000", 2024-09-14T22:39:03.497 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 70.743818489999995, 2024-09-14T22:39:03.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064424631999999996, 2024-09-14T22:39:03.497 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:03.497 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:03.497 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:03.497 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:03.497 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:03.497 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.498 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145262 2024-09-14T22:39:03.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.498 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:03.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.498 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:03.498 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694341+0000", 2024-09-14T22:39:03.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.498 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:03.498 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694341+0000", 2024-09-14T22:39:03.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.498 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:03.498 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694341+0000", 2024-09-14T22:39:03.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5499999999999999e-07 2024-09-14T22:39:03.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:03.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694343+0000", 2024-09-14T22:39:03.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9630000000000001e-06 2024-09-14T22:39:03.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:03.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694349+0000", 2024-09-14T22:39:03.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3120000000000001e-06 2024-09-14T22:39:03.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694353+0000", 2024-09-14T22:39:03.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6500000000000004e-06 2024-09-14T22:39:03.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:03.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758561+0000", 2024-09-14T22:39:03.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064207934999999994 2024-09-14T22:39:03.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:03.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758677+0000", 2024-09-14T22:39:03.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116241 2024-09-14T22:39:03.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:03.501 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758766+0000", 2024-09-14T22:39:03.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8176000000000004e-05 2024-09-14T22:39:03.501 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.501 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:03.501 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.501 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145263 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16662528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:03.501 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694359+0000", 2024-09-14T22:39:03.501 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 70.743800647, 2024-09-14T22:39:03.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064743189000000007, 2024-09-14T22:39:03.501 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:03.501 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:03.501 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:03.502 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:03.502 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:03.502 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.502 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145263 2024-09-14T22:39:03.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.502 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:03.502 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:03.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694359+0000", 2024-09-14T22:39:03.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.502 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:03.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694359+0000", 2024-09-14T22:39:03.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:03.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694359+0000", 2024-09-14T22:39:03.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5100000000000001e-07 2024-09-14T22:39:03.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:03.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694361+0000", 2024-09-14T22:39:03.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9429999999999999e-06 2024-09-14T22:39:03.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.504 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:03.504 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694366+0000", 2024-09-14T22:39:03.504 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3489999999999999e-06 2024-09-14T22:39:03.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.504 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.504 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694371+0000", 2024-09-14T22:39:03.504 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4399999999999998e-06 2024-09-14T22:39:03.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.504 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:03.504 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758848+0000", 2024-09-14T22:39:03.504 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064477396000000006 2024-09-14T22:39:03.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:03.505 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.758980+0000", 2024-09-14T22:39:03.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013162199999999999 2024-09-14T22:39:03.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:03.505 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759102+0000", 2024-09-14T22:39:03.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012208799999999999 2024-09-14T22:39:03.505 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.505 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:03.505 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.506 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145264 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16334848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:03.506 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694377+0000", 2024-09-14T22:39:03.506 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 70.743782612999993, 2024-09-14T22:39:03.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065048054999999994, 2024-09-14T22:39:03.506 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:03.506 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:03.506 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:03.506 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:03.506 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:03.506 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.506 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145264 2024-09-14T22:39:03.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.506 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:03.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:03.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694377+0000", 2024-09-14T22:39:03.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:03.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694377+0000", 2024-09-14T22:39:03.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:03.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694377+0000", 2024-09-14T22:39:03.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.39e-07 2024-09-14T22:39:03.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:03.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694379+0000", 2024-09-14T22:39:03.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0190000000000001e-06 2024-09-14T22:39:03.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:03.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694385+0000", 2024-09-14T22:39:03.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4399999999999996e-06 2024-09-14T22:39:03.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694389+0000", 2024-09-14T22:39:03.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.639e-06 2024-09-14T22:39:03.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:03.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759175+0000", 2024-09-14T22:39:03.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064785899999999993 2024-09-14T22:39:03.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:03.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759309+0000", 2024-09-14T22:39:03.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013374400000000001 2024-09-14T22:39:03.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:03.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759425+0000", 2024-09-14T22:39:03.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115974 2024-09-14T22:39:03.510 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.510 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:03.510 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.510 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145265 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16531456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:03.510 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694395+0000", 2024-09-14T22:39:03.510 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 70.743764490999993, 2024-09-14T22:39:03.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065298325000000004, 2024-09-14T22:39:03.510 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:03.510 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:03.510 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:03.510 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:03.510 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:03.510 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.510 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145265 2024-09-14T22:39:03.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.511 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:03.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:03.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694395+0000", 2024-09-14T22:39:03.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:03.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694395+0000", 2024-09-14T22:39:03.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:03.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694395+0000", 2024-09-14T22:39:03.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6699999999999999e-07 2024-09-14T22:39:03.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:03.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694397+0000", 2024-09-14T22:39:03.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.984e-06 2024-09-14T22:39:03.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:03.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694403+0000", 2024-09-14T22:39:03.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3519999999999997e-06 2024-09-14T22:39:03.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694407+0000", 2024-09-14T22:39:03.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3239999999999996e-06 2024-09-14T22:39:03.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:03.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759516+0000", 2024-09-14T22:39:03.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065109279000000006 2024-09-14T22:39:03.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:03.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759610+0000", 2024-09-14T22:39:03.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3566000000000005e-05 2024-09-14T22:39:03.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:03.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759693+0000", 2024-09-14T22:39:03.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3453000000000002e-05 2024-09-14T22:39:03.514 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.514 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:03.514 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.514 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145266 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16433152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:03.514 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694413+0000", 2024-09-14T22:39:03.514 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 70.743746692000002, 2024-09-14T22:39:03.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065618308, 2024-09-14T22:39:03.514 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:03.514 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:03.514 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:03.514 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:03.514 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:03.514 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.515 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145266 2024-09-14T22:39:03.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.515 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:03.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:03.515 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694413+0000", 2024-09-14T22:39:03.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:03.515 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694413+0000", 2024-09-14T22:39:03.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:03.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694413+0000", 2024-09-14T22:39:03.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4799999999999999e-07 2024-09-14T22:39:03.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:03.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694415+0000", 2024-09-14T22:39:03.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.08e-06 2024-09-14T22:39:03.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:03.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694420+0000", 2024-09-14T22:39:03.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.203e-06 2024-09-14T22:39:03.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694425+0000", 2024-09-14T22:39:03.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4649999999999996e-06 2024-09-14T22:39:03.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:03.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759768+0000", 2024-09-14T22:39:03.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065343613999999994 2024-09-14T22:39:03.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:03.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.759909+0000", 2024-09-14T22:39:03.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014054099999999999 2024-09-14T22:39:03.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:03.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760031+0000", 2024-09-14T22:39:03.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122057 2024-09-14T22:39:03.518 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.518 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:03.518 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.518 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145267 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16629760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:03.518 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694431+0000", 2024-09-14T22:39:03.518 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 70.743728778000005, 2024-09-14T22:39:03.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065850237000000006, 2024-09-14T22:39:03.518 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:03.518 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:03.519 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:03.519 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:03.519 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:03.519 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.519 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145267 2024-09-14T22:39:03.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.519 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:03.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:03.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694431+0000", 2024-09-14T22:39:03.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:03.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694431+0000", 2024-09-14T22:39:03.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:03.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694431+0000", 2024-09-14T22:39:03.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4799999999999999e-07 2024-09-14T22:39:03.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:03.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694433+0000", 2024-09-14T22:39:03.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8470000000000001e-06 2024-09-14T22:39:03.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:03.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694438+0000", 2024-09-14T22:39:03.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3249999999999998e-06 2024-09-14T22:39:03.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694442+0000", 2024-09-14T22:39:03.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3490000000000002e-06 2024-09-14T22:39:03.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:03.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760113+0000", 2024-09-14T22:39:03.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065670872000000005 2024-09-14T22:39:03.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:03.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760199+0000", 2024-09-14T22:39:03.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5982000000000004e-05 2024-09-14T22:39:03.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.522 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:03.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760281+0000", 2024-09-14T22:39:03.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1514000000000006e-05 2024-09-14T22:39:03.522 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.522 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:03.522 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.522 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145268 3.8 3:118a9eb6:::rbd_data.113ce98dded2.0000000000000002:24 [read 16728064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:03.522 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.694448+0000", 2024-09-14T22:39:03.522 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 70.743711193999999, 2024-09-14T22:39:03.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066157585000000005, 2024-09-14T22:39:03.522 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:03.522 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:03.522 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:03.523 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:03.523 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:03.523 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.523 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145268 2024-09-14T22:39:03.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.523 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:03.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.523 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:03.523 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694448+0000", 2024-09-14T22:39:03.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.523 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:03.523 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694448+0000", 2024-09-14T22:39:03.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:03.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:03.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694449+0000", 2024-09-14T22:39:03.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.39e-07 2024-09-14T22:39:03.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:03.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694450+0000", 2024-09-14T22:39:03.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.906e-06 2024-09-14T22:39:03.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:03.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694456+0000", 2024-09-14T22:39:03.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3820000000000003e-06 2024-09-14T22:39:03.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.694460+0000", 2024-09-14T22:39:03.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6290000000000001e-06 2024-09-14T22:39:03.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:03.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760345+0000", 2024-09-14T22:39:03.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065884540000000005 2024-09-14T22:39:03.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:03.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760482+0000", 2024-09-14T22:39:03.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013731600000000001 2024-09-14T22:39:03.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:03.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:03.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:03.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.760606+0000", 2024-09-14T22:39:03.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000123473 2024-09-14T22:39:03.526 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.526 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:03.526 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.526 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:03.526 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:03.526 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:39:03.527 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:39:03.658 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:39:03.659 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:39:03.659 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:39:03.659 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:39:03.673 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:39:03.793 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:39:03.793 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:39:03.793 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:39:03.793 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:39:03.793 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:39:03.807 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:39:03.933 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:39:03.933 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:39:03.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:39:03.933 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:39:03.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.933 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97197 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 796672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:39:03.933 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:39:03.933 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.85805523000001, 2024-09-14T22:39:03.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12434473, 2024-09-14T22:39:03.933 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:03.933 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:03.933 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:03.934 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:03.934 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:03.934 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.934 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97197 2024-09-14T22:39:03.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.934 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:03.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:03.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:39:03.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:03.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:03.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070523+0000", 2024-09-14T22:39:03.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:03.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:03.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070524+0000", 2024-09-14T22:39:03.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0299999999999998e-07 2024-09-14T22:39:03.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:03.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070527+0000", 2024-09-14T22:39:03.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9569999999999998e-06 2024-09-14T22:39:03.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:03.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070535+0000", 2024-09-14T22:39:03.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7630000000000006e-06 2024-09-14T22:39:03.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.936 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070543+0000", 2024-09-14T22:39:03.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4869999999999996e-06 2024-09-14T22:39:03.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.936 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:03.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194525+0000", 2024-09-14T22:39:03.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123982096 2024-09-14T22:39:03.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.936 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:03.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194699+0000", 2024-09-14T22:39:03.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000174105 2024-09-14T22:39:03.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.937 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.937 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:03.937 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194868+0000", 2024-09-14T22:39:03.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016861900000000001 2024-09-14T22:39:03.937 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:03.937 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:03.937 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:03.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.937 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.937 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97198 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 805888~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:39:03.937 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:39:03.937 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.858026854, 2024-09-14T22:39:03.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.1248088, 2024-09-14T22:39:03.938 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:03.938 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:03.938 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:03.938 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:03.938 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:03.938 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.938 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97198 2024-09-14T22:39:03.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.938 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:03.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.938 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:03.938 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:39:03.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:03.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:03.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070551+0000", 2024-09-14T22:39:03.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:03.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:03.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070552+0000", 2024-09-14T22:39:03.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T22:39:03.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:03.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070555+0000", 2024-09-14T22:39:03.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.853e-06 2024-09-14T22:39:03.940 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.940 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:03.940 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070565+0000", 2024-09-14T22:39:03.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.048e-05 2024-09-14T22:39:03.940 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.940 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.940 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070573+0000", 2024-09-14T22:39:03.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7200000000000006e-06 2024-09-14T22:39:03.940 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.940 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:03.940 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.194954+0000", 2024-09-14T22:39:03.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12438089400000001 2024-09-14T22:39:03.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.941 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:03.941 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195145+0000", 2024-09-14T22:39:03.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019091299999999999 2024-09-14T22:39:03.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.941 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:03.941 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195360+0000", 2024-09-14T22:39:03.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021527300000000001 2024-09-14T22:39:03.941 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:03.941 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:03.941 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:03.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.941 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97199 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 815104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:39:03.942 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:39:03.942 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.857996257, 2024-09-14T22:39:03.942 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125251901, 2024-09-14T22:39:03.942 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:03.942 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:03.942 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:03.942 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:03.942 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:03.942 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.942 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97199 2024-09-14T22:39:03.942 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.942 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:03.942 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.942 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:03.942 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:39:03.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:03.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:03.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070582+0000", 2024-09-14T22:39:03.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:03.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:03.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070583+0000", 2024-09-14T22:39:03.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6300000000000005e-07 2024-09-14T22:39:03.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:03.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070586+0000", 2024-09-14T22:39:03.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8270000000000002e-06 2024-09-14T22:39:03.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:03.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070597+0000", 2024-09-14T22:39:03.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1960000000000001e-05 2024-09-14T22:39:03.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070605+0000", 2024-09-14T22:39:03.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6459999999999993e-06 2024-09-14T22:39:03.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:03.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195432+0000", 2024-09-14T22:39:03.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124826926 2024-09-14T22:39:03.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:03.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195626+0000", 2024-09-14T22:39:03.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019357099999999999 2024-09-14T22:39:03.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:03.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195834+0000", 2024-09-14T22:39:03.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020830800000000001 2024-09-14T22:39:03.945 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:03.945 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:03.945 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:03.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.946 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97200 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 824320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:39:03.946 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:39:03.946 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.857964211, 2024-09-14T22:39:03.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125693477, 2024-09-14T22:39:03.946 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:03.946 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:03.946 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:03.946 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:03.946 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:03.946 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.946 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97200 2024-09-14T22:39:03.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.946 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:03.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:03.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:39:03.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:03.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:03.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070614+0000", 2024-09-14T22:39:03.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:03.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:03.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070615+0000", 2024-09-14T22:39:03.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T22:39:03.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:03.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070618+0000", 2024-09-14T22:39:03.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.943e-06 2024-09-14T22:39:03.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:03.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070627+0000", 2024-09-14T22:39:03.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1099999999999992e-06 2024-09-14T22:39:03.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070634+0000", 2024-09-14T22:39:03.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:39:03.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:03.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.195908+0000", 2024-09-14T22:39:03.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125273893 2024-09-14T22:39:03.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:03.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196097+0000", 2024-09-14T22:39:03.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018953899999999999 2024-09-14T22:39:03.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:03.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196308+0000", 2024-09-14T22:39:03.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021027799999999999 2024-09-14T22:39:03.949 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:03.949 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:03.949 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:03.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.950 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97201 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 833536~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:39:03.950 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:39:03.950 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.857934024, 2024-09-14T22:39:03.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126133826, 2024-09-14T22:39:03.950 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:03.950 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:03.950 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:03.950 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:03.950 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:03.950 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.950 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97201 2024-09-14T22:39:03.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.950 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:03.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.950 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:03.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:39:03.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:03.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:03.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.070644+0000", 2024-09-14T22:39:03.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:03.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:03.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074109+0000", 2024-09-14T22:39:03.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003464834 2024-09-14T22:39:03.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:03.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074134+0000", 2024-09-14T22:39:03.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5170000000000001e-05 2024-09-14T22:39:03.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:03.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074166+0000", 2024-09-14T22:39:03.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1347000000000003e-05 2024-09-14T22:39:03.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074179+0000", 2024-09-14T22:39:03.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.364e-05 2024-09-14T22:39:03.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:03.953 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196377+0000", 2024-09-14T22:39:03.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12219793900000001 2024-09-14T22:39:03.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:03.953 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196568+0000", 2024-09-14T22:39:03.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019097999999999999 2024-09-14T22:39:03.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:03.953 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.196778+0000", 2024-09-14T22:39:03.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020991599999999999 2024-09-14T22:39:03.953 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:03.954 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:03.954 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:03.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.954 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97205 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 870400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:39:03.954 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:39:03.954 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.854264461, 2024-09-14T22:39:03.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12436148299999999, 2024-09-14T22:39:03.954 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:03.954 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:03.954 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:03.954 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:03.954 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:03.954 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.954 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97205 2024-09-14T22:39:03.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.955 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:03.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:03.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:39:03.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:03.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:03.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074314+0000", 2024-09-14T22:39:03.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:03.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:03.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074315+0000", 2024-09-14T22:39:03.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9299999999999997e-07 2024-09-14T22:39:03.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:03.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074318+0000", 2024-09-14T22:39:03.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.19e-06 2024-09-14T22:39:03.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:03.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074327+0000", 2024-09-14T22:39:03.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8529999999999998e-06 2024-09-14T22:39:03.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.957 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.957 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074334+0000", 2024-09-14T22:39:03.957 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3869999999999997e-06 2024-09-14T22:39:03.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.957 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:03.957 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198365+0000", 2024-09-14T22:39:03.957 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124031222 2024-09-14T22:39:03.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.957 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:03.958 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198513+0000", 2024-09-14T22:39:03.958 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014772299999999999 2024-09-14T22:39:03.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.958 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.958 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:03.958 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198675+0000", 2024-09-14T22:39:03.958 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016241500000000001 2024-09-14T22:39:03.958 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:03.958 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:03.958 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:03.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.958 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.958 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97206 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 879616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:39:03.958 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:39:03.958 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.854233005, 2024-09-14T22:39:03.959 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124706284, 2024-09-14T22:39:03.959 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:03.959 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:03.959 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:03.959 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:03.959 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:03.959 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.959 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97206 2024-09-14T22:39:03.959 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.959 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:03.959 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.959 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:03.959 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:39:03.959 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:03.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:03.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074345+0000", 2024-09-14T22:39:03.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:03.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:03.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074346+0000", 2024-09-14T22:39:03.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8999999999999996e-07 2024-09-14T22:39:03.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:03.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074349+0000", 2024-09-14T22:39:03.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.96e-06 2024-09-14T22:39:03.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.961 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:03.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074358+0000", 2024-09-14T22:39:03.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9169999999999995e-06 2024-09-14T22:39:03.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.961 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074365+0000", 2024-09-14T22:39:03.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3699999999999997e-06 2024-09-14T22:39:03.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:03.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198761+0000", 2024-09-14T22:39:03.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124395743 2024-09-14T22:39:03.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:03.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.198899+0000", 2024-09-14T22:39:03.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013763499999999999 2024-09-14T22:39:03.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:03.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199052+0000", 2024-09-14T22:39:03.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015296900000000001 2024-09-14T22:39:03.963 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:03.963 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:03.963 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:03.963 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.963 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.963 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97207 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 888832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:39:03.963 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:39:03.963 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.85420379200001, 2024-09-14T22:39:03.963 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124988433, 2024-09-14T22:39:03.963 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:03.963 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:03.963 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:03.963 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:03.963 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:03.963 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.963 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97207 2024-09-14T22:39:03.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.964 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:03.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:03.964 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:39:03.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:03.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:03.964 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074374+0000", 2024-09-14T22:39:03.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:03.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:03.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074375+0000", 2024-09-14T22:39:03.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:39:03.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:03.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074378+0000", 2024-09-14T22:39:03.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1829999999999998e-06 2024-09-14T22:39:03.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:03.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074389+0000", 2024-09-14T22:39:03.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0737000000000001e-05 2024-09-14T22:39:03.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074396+0000", 2024-09-14T22:39:03.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2559999999999996e-06 2024-09-14T22:39:03.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:03.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199139+0000", 2024-09-14T22:39:03.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12474253 2024-09-14T22:39:03.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:03.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199240+0000", 2024-09-14T22:39:03.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101221 2024-09-14T22:39:03.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.967 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:03.967 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199363+0000", 2024-09-14T22:39:03.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000122822 2024-09-14T22:39:03.967 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:03.967 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:03.967 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:03.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.967 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97208 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 898048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:39:03.967 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:39:03.967 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.854172488, 2024-09-14T22:39:03.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12524854899999999, 2024-09-14T22:39:03.968 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:03.968 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:03.968 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:03.968 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:03.968 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:03.968 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.968 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97208 2024-09-14T22:39:03.968 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.968 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:03.968 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.968 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:03.968 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:39:03.968 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:03.968 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:03.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:39:03.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:03.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:03.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074406+0000", 2024-09-14T22:39:03.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:39:03.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:03.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074410+0000", 2024-09-14T22:39:03.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.18e-06 2024-09-14T22:39:03.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:03.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074435+0000", 2024-09-14T22:39:03.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5273e-05 2024-09-14T22:39:03.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074443+0000", 2024-09-14T22:39:03.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3000000000000002e-06 2024-09-14T22:39:03.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:03.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199433+0000", 2024-09-14T22:39:03.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124989502 2024-09-14T22:39:03.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:03.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199535+0000", 2024-09-14T22:39:03.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102557 2024-09-14T22:39:03.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:03.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199654+0000", 2024-09-14T22:39:03.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119061 2024-09-14T22:39:03.971 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:03.971 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:03.971 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:03.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.972 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97209 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 907264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:39:03.972 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:39:03.972 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.85412468299999, 2024-09-14T22:39:03.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125487354, 2024-09-14T22:39:03.972 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:03.972 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:03.972 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:03.972 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:03.972 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:03.972 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.972 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97209 2024-09-14T22:39:03.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.973 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:03.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:03.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:39:03.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:03.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:03.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:39:03.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:03.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:03.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074454+0000", 2024-09-14T22:39:03.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9400000000000005e-07 2024-09-14T22:39:03.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:03.974 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074458+0000", 2024-09-14T22:39:03.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3299999999999999e-06 2024-09-14T22:39:03.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:03.974 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074467+0000", 2024-09-14T22:39:03.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.346e-06 2024-09-14T22:39:03.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074474+0000", 2024-09-14T22:39:03.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.374e-06 2024-09-14T22:39:03.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:03.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199720+0000", 2024-09-14T22:39:03.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12524561200000001 2024-09-14T22:39:03.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:03.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199820+0000", 2024-09-14T22:39:03.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010049599999999999 2024-09-14T22:39:03.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.976 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:03.976 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.199941+0000", 2024-09-14T22:39:03.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012050200000000001 2024-09-14T22:39:03.976 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:03.976 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:03.976 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:03.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.976 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97210 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:39:03.976 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:39:03.976 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.85409468899999, 2024-09-14T22:39:03.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125744942, 2024-09-14T22:39:03.976 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:03.976 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:03.977 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:03.977 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:03.977 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:03.977 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.977 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97210 2024-09-14T22:39:03.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.977 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:03.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:03.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:39:03.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:03.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:03.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:39:03.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:03.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:03.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074484+0000", 2024-09-14T22:39:03.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6599999999999996e-07 2024-09-14T22:39:03.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:03.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074487+0000", 2024-09-14T22:39:03.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8140000000000001e-06 2024-09-14T22:39:03.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.979 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:03.979 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074496+0000", 2024-09-14T22:39:03.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9330000000000007e-06 2024-09-14T22:39:03.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.979 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.979 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074503+0000", 2024-09-14T22:39:03.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4100000000000002e-06 2024-09-14T22:39:03.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.979 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:03.979 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200009+0000", 2024-09-14T22:39:03.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12550539899999999 2024-09-14T22:39:03.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.980 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:03.980 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200108+0000", 2024-09-14T22:39:03.980 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9182e-05 2024-09-14T22:39:03.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.980 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:03.980 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200229+0000", 2024-09-14T22:39:03.980 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120538 2024-09-14T22:39:03.980 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:03.980 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:03.980 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:03.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.980 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97211 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 925696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:39:03.981 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:39:03.981 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.854063729, 2024-09-14T22:39:03.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12599848299999999, 2024-09-14T22:39:03.981 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:03.981 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:03.981 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:03.981 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:03.981 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:03.981 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.981 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97211 2024-09-14T22:39:03.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.981 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:03.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.981 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:03.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:39:03.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:03.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:03.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074515+0000", 2024-09-14T22:39:03.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:03.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:03.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074516+0000", 2024-09-14T22:39:03.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.42e-06 2024-09-14T22:39:03.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:03.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074521+0000", 2024-09-14T22:39:03.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7960000000000003e-06 2024-09-14T22:39:03.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:03.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074538+0000", 2024-09-14T22:39:03.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6900000000000001e-05 2024-09-14T22:39:03.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074555+0000", 2024-09-14T22:39:03.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.696e-05 2024-09-14T22:39:03.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:03.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200293+0000", 2024-09-14T22:39:03.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12573864100000001 2024-09-14T22:39:03.984 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.984 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:03.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200393+0000", 2024-09-14T22:39:03.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9264000000000004e-05 2024-09-14T22:39:03.984 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.984 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:03.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200513+0000", 2024-09-14T22:39:03.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012050200000000001 2024-09-14T22:39:03.984 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:03.984 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:03.984 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:03.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.985 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.985 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97212 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 934912~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:39:03.985 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:39:03.985 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.853989107, 2024-09-14T22:39:03.985 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12621127300000001, 2024-09-14T22:39:03.985 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:03.985 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:03.985 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:03.985 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:03.985 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:03.985 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.985 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97212 2024-09-14T22:39:03.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.985 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:03.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.986 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:03.986 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:39:03.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:03.986 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.986 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:03.986 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074589+0000", 2024-09-14T22:39:03.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:03.986 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.986 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:03.986 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074590+0000", 2024-09-14T22:39:03.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1969999999999999e-06 2024-09-14T22:39:03.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:03.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074596+0000", 2024-09-14T22:39:03.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7629999999999997e-06 2024-09-14T22:39:03.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:03.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074614+0000", 2024-09-14T22:39:03.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7643e-05 2024-09-14T22:39:03.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074635+0000", 2024-09-14T22:39:03.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.075e-05 2024-09-14T22:39:03.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.988 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:03.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200580+0000", 2024-09-14T22:39:03.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.125945482 2024-09-14T22:39:03.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.988 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:03.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200683+0000", 2024-09-14T22:39:03.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010267099999999999 2024-09-14T22:39:03.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.988 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:03.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200800+0000", 2024-09-14T22:39:03.989 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117767 2024-09-14T22:39:03.989 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:03.989 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:03.989 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:03.989 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.989 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.989 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97213 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 944128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:39:03.989 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:39:03.989 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.853923757, 2024-09-14T22:39:03.989 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126430709, 2024-09-14T22:39:03.989 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:03.989 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:03.989 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:03.990 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:03.990 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:03.990 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.990 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97213 2024-09-14T22:39:03.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.990 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:03.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.990 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:03.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:39:03.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:03.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.990 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:03.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.074655+0000", 2024-09-14T22:39:03.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:03.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.991 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.991 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:03.991 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078210+0000", 2024-09-14T22:39:03.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0035556839999999999 2024-09-14T22:39:03.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.991 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.991 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:03.991 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078231+0000", 2024-09-14T22:39:03.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0716e-05 2024-09-14T22:39:03.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.991 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.991 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:03.991 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078260+0000", 2024-09-14T22:39:03.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9473000000000001e-05 2024-09-14T22:39:03.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078279+0000", 2024-09-14T22:39:03.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8600000000000001e-05 2024-09-14T22:39:03.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:03.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200864+0000", 2024-09-14T22:39:03.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.122585295 2024-09-14T22:39:03.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:03.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.200965+0000", 2024-09-14T22:39:03.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000101004 2024-09-14T22:39:03.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.993 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.993 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:03.993 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.201085+0000", 2024-09-14T22:39:03.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119937 2024-09-14T22:39:03.993 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:03.993 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:03.993 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:03.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.993 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.993 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97219 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 999424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:39:03.993 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:39:03.993 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.84994148, 2024-09-14T22:39:03.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124138558, 2024-09-14T22:39:03.993 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:03.994 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:03.994 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:03.994 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:03.994 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:03.994 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.994 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97219 2024-09-14T22:39:03.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.994 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:03.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:03.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:39:03.994 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:03.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:03.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078637+0000", 2024-09-14T22:39:03.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:03.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.995 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:03.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078638+0000", 2024-09-14T22:39:03.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.173e-06 2024-09-14T22:39:03.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.995 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:03.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078644+0000", 2024-09-14T22:39:03.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2570000000000004e-06 2024-09-14T22:39:03.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.995 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:03.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078672+0000", 2024-09-14T22:39:03.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7322999999999999e-05 2024-09-14T22:39:03.996 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.996 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:03.996 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078687+0000", 2024-09-14T22:39:03.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5639999999999999e-05 2024-09-14T22:39:03.996 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.996 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:03.996 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202567+0000", 2024-09-14T22:39:03.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123880134 2024-09-14T22:39:03.996 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.996 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:03.996 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202670+0000", 2024-09-14T22:39:03.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010273899999999999 2024-09-14T22:39:03.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.997 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:03.997 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202775+0000", 2024-09-14T22:39:03.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010529199999999999 2024-09-14T22:39:03.997 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:03.997 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:03.997 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:03.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.997 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97220 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1008640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:39:03.997 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:39:03.997 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.84987224699999, 2024-09-14T22:39:03.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12435937, 2024-09-14T22:39:03.998 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:03.998 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:03.998 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:03.998 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:03.998 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:03.998 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:03.998 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97220 2024-09-14T22:39:03.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.998 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:03.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:03.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:39:03.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:03.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:03.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078706+0000", 2024-09-14T22:39:03.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:03.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:03.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078707+0000", 2024-09-14T22:39:03.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.266e-06 2024-09-14T22:39:03.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:03.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:03.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:03.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078713+0000", 2024-09-14T22:39:03.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7270000000000004e-06 2024-09-14T22:39:03.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.000 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.000 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078737+0000", 2024-09-14T22:39:04.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3640000000000001e-05 2024-09-14T22:39:04.000 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.000 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.000 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078751+0000", 2024-09-14T22:39:04.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.45e-05 2024-09-14T22:39:04.000 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.000 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.000 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202841+0000", 2024-09-14T22:39:04.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124089938 2024-09-14T22:39:04.000 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.001 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.202944+0000", 2024-09-14T22:39:04.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010334 2024-09-14T22:39:04.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.001 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203065+0000", 2024-09-14T22:39:04.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012095900000000001 2024-09-14T22:39:04.001 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.001 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.001 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.001 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97221 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1017856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:39:04.002 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:39:04.002 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.849806484, 2024-09-14T22:39:04.002 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124574005, 2024-09-14T22:39:04.002 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.002 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.002 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.002 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.002 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.002 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.002 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97221 2024-09-14T22:39:04.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.002 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.002 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.002 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:39:04.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078772+0000", 2024-09-14T22:39:04.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078773+0000", 2024-09-14T22:39:04.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.13e-06 2024-09-14T22:39:04.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.004 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078779+0000", 2024-09-14T22:39:04.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0260000000000004e-06 2024-09-14T22:39:04.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.004 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078797+0000", 2024-09-14T22:39:04.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.806e-05 2024-09-14T22:39:04.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.004 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078810+0000", 2024-09-14T22:39:04.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.308e-05 2024-09-14T22:39:04.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.005 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.005 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203131+0000", 2024-09-14T22:39:04.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12432077499999999 2024-09-14T22:39:04.005 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.005 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.005 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.005 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203227+0000", 2024-09-14T22:39:04.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6575000000000003e-05 2024-09-14T22:39:04.005 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.005 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.005 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.005 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203346+0000", 2024-09-14T22:39:04.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118359 2024-09-14T22:39:04.005 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.005 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.006 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.006 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.006 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97222 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:39:04.006 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:39:04.006 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.84975011500001, 2024-09-14T22:39:04.006 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12480110799999999, 2024-09-14T22:39:04.006 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.006 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.006 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.006 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.006 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.006 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.006 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97222 2024-09-14T22:39:04.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.007 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.007 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.007 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:39:04.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.007 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.007 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078828+0000", 2024-09-14T22:39:04.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.007 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.007 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078829+0000", 2024-09-14T22:39:04.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.057e-06 2024-09-14T22:39:04.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078836+0000", 2024-09-14T22:39:04.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6159999999999998e-06 2024-09-14T22:39:04.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078856+0000", 2024-09-14T22:39:04.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0100000000000001e-05 2024-09-14T22:39:04.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.078869+0000", 2024-09-14T22:39:04.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.272e-05 2024-09-14T22:39:04.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.009 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203412+0000", 2024-09-14T22:39:04.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12454283300000001 2024-09-14T22:39:04.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.009 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203511+0000", 2024-09-14T22:39:04.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9893000000000001e-05 2024-09-14T22:39:04.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.009 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203629+0000", 2024-09-14T22:39:04.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117889 2024-09-14T22:39:04.009 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.010 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.010 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.010 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97223 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1036288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:39:04.010 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:39:04.010 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.849378128, 2024-09-14T22:39:04.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124706279, 2024-09-14T22:39:04.010 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.010 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.010 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.010 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.010 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.010 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.010 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97223 2024-09-14T22:39:04.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.011 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.011 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.011 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.011 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:39:04.011 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.011 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.011 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.011 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.011 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079200+0000", 2024-09-14T22:39:04.011 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.011 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.011 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.011 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.011 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079202+0000", 2024-09-14T22:39:04.011 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8330000000000001e-06 2024-09-14T22:39:04.011 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079212+0000", 2024-09-14T22:39:04.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0207000000000001e-05 2024-09-14T22:39:04.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079238+0000", 2024-09-14T22:39:04.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5626000000000002e-05 2024-09-14T22:39:04.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079256+0000", 2024-09-14T22:39:04.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8386999999999999e-05 2024-09-14T22:39:04.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203691+0000", 2024-09-14T22:39:04.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.124434538 2024-09-14T22:39:04.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203786+0000", 2024-09-14T22:39:04.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5344000000000001e-05 2024-09-14T22:39:04.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.203906+0000", 2024-09-14T22:39:04.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000120344 2024-09-14T22:39:04.013 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.013 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.014 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.014 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:97224 3.1d 3:bd1bd1dc:::rbd_data.113c5a3f285d.0000000000000001:15 [read 1045504~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-14T22:39:04.014 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:39:04.014 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.849294308, 2024-09-14T22:39:04.014 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12513321199999999, 2024-09-14T22:39:04.014 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.014 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.014 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.014 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.014 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.014 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.014 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97224 2024-09-14T22:39:04.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.014 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.015 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.015 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:39:04.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.015 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.015 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.015 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.079284+0000", 2024-09-14T22:39:04.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.015 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.015 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.015 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082581+0000", 2024-09-14T22:39:04.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0032971530000000001 2024-09-14T22:39:04.015 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.015 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.016 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.016 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082592+0000", 2024-09-14T22:39:04.016 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1345999999999999e-05 2024-09-14T22:39:04.016 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.016 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.016 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.016 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082618+0000", 2024-09-14T22:39:04.016 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5873000000000001e-05 2024-09-14T22:39:04.016 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.016 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.016 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.016 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.082638+0000", 2024-09-14T22:39:04.016 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9565999999999999e-05 2024-09-14T22:39:04.016 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.016 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204004+0000", 2024-09-14T22:39:04.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.121366038 2024-09-14T22:39:04.017 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204188+0000", 2024-09-14T22:39:04.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018427 2024-09-14T22:39:04.017 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:10.204417+0000", 2024-09-14T22:39:04.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022896599999999999 2024-09-14T22:39:04.017 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.017 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.018 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.018 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.018 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.018 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:39:04.019 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:39:04.147 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:39:04.147 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:39:04.147 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:39:04.147 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:39:04.162 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:39:04.281 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:39:04.281 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:39:04.281 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:39:04.282 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:39:04.282 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:39:04.296 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:39:04.415 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:39:04.416 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:39:04.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:39:04.416 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:39:04.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.416 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:72252 3.2 3:442c6d72:::rbd_data.113cb4c83968.0000000000000015:head [zero 4456448~393216] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-14T22:39:04.416 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:39:04.416 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 133.71148169, 2024-09-14T22:39:04.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0176591500000001, 2024-09-14T22:39:04.416 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.416 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.416 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T22:39:04.416 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.417 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.417 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.417 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 72252 2024-09-14T22:39:04.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.417 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:39:04.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700098+0000", 2024-09-14T22:39:04.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700101+0000", 2024-09-14T22:39:04.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4570000000000001e-06 2024-09-14T22:39:04.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700114+0000", 2024-09-14T22:39:04.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2799999999999999e-05 2024-09-14T22:39:04.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700146+0000", 2024-09-14T22:39:04.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2116000000000002e-05 2024-09-14T22:39:04.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700170+0000", 2024-09-14T22:39:04.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3770000000000001e-05 2024-09-14T22:39:04.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700285+0000", 2024-09-14T22:39:04.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000115228 2024-09-14T22:39:04.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op must wait for map", 2024-09-14T22:39:04.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.700297+0000", 2024-09-14T22:39:04.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1681999999999999e-05 2024-09-14T22:39:04.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.703312+0000", 2024-09-14T22:39:04.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003015046 2024-09-14T22:39:04.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:39:04.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:50.703327+0000", 2024-09-14T22:39:04.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5481999999999998e-05 2024-09-14T22:39:04.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:51.717566+0000", 2024-09-14T22:39:04.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0142383509999999 2024-09-14T22:39:04.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:51.717757+0000", 2024-09-14T22:39:04.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000191218 2024-09-14T22:39:04.421 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.422 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.422 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.422 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:74156 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T22:39:04.422 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:39:04.422 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 128.89701298, 2024-09-14T22:39:04.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062073521, 2024-09-14T22:39:04.422 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.422 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.422 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:39:04.422 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.422 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.422 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.422 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74156 2024-09-14T22:39:04.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.423 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:39:04.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514567+0000", 2024-09-14T22:39:04.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514569+0000", 2024-09-14T22:39:04.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0229999999999999e-06 2024-09-14T22:39:04.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.424 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514596+0000", 2024-09-14T22:39:04.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6903e-05 2024-09-14T22:39:04.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.424 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514623+0000", 2024-09-14T22:39:04.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7206e-05 2024-09-14T22:39:04.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.424 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514637+0000", 2024-09-14T22:39:04.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:39:04.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.514781+0000", 2024-09-14T22:39:04.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014394899999999999 2024-09-14T22:39:04.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.515005+0000", 2024-09-14T22:39:04.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022458499999999999 2024-09-14T22:39:04.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T22:39:04.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.527785+0000", 2024-09-14T22:39:04.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012780112 2024-09-14T22:39:04.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:39:04.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576209+0000", 2024-09-14T22:39:04.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.048423992999999999 2024-09-14T22:39:04.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:39:04.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576346+0000", 2024-09-14T22:39:04.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000136909 2024-09-14T22:39:04.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:39:04.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576517+0000", 2024-09-14T22:39:04.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000170848 2024-09-14T22:39:04.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:55.576640+0000", 2024-09-14T22:39:04.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000122953 2024-09-14T22:39:04.427 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.427 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.427 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.428 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82513 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 557056~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:39:04.428 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:39:04.428 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 124.992381161, 2024-09-14T22:39:04.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059345486000000003, 2024-09-14T22:39:04.428 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.428 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.428 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.428 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.428 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.428 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.428 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82513 2024-09-14T22:39:04.428 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.428 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:39:04.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419198+0000", 2024-09-14T22:39:04.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419200+0000", 2024-09-14T22:39:04.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.184e-06 2024-09-14T22:39:04.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419207+0000", 2024-09-14T22:39:04.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2690000000000002e-06 2024-09-14T22:39:04.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419224+0000", 2024-09-14T22:39:04.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6623e-05 2024-09-14T22:39:04.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419236+0000", 2024-09-14T22:39:04.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2367e-05 2024-09-14T22:39:04.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478294+0000", 2024-09-14T22:39:04.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059057927000000003 2024-09-14T22:39:04.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478402+0000", 2024-09-14T22:39:04.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107921 2024-09-14T22:39:04.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478544+0000", 2024-09-14T22:39:04.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014219499999999999 2024-09-14T22:39:04.432 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.432 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.432 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.432 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.432 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.432 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82514 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 704512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:39:04.432 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:39:04.432 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 124.992322058, 2024-09-14T22:39:04.432 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059580793, 2024-09-14T22:39:04.432 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.432 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.432 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.433 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.433 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.433 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.433 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82514 2024-09-14T22:39:04.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.433 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.433 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.433 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.433 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:39:04.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419258+0000", 2024-09-14T22:39:04.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419259+0000", 2024-09-14T22:39:04.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.353e-06 2024-09-14T22:39:04.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.435 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419276+0000", 2024-09-14T22:39:04.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7567e-05 2024-09-14T22:39:04.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.435 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.435 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419294+0000", 2024-09-14T22:39:04.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.713e-05 2024-09-14T22:39:04.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.435 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.435 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419306+0000", 2024-09-14T22:39:04.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.279e-05 2024-09-14T22:39:04.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.436 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.436 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.436 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478589+0000", 2024-09-14T22:39:04.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059282978 2024-09-14T22:39:04.436 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.436 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.436 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.436 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478694+0000", 2024-09-14T22:39:04.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000104561 2024-09-14T22:39:04.436 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.436 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.436 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.436 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478838+0000", 2024-09-14T22:39:04.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014441399999999999 2024-09-14T22:39:04.437 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.437 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.437 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.437 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.437 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82515 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 851968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:39:04.437 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:39:04.437 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 124.992256908, 2024-09-14T22:39:04.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059815123999999997, 2024-09-14T22:39:04.437 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.437 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.438 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.438 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.438 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.438 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.438 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82515 2024-09-14T22:39:04.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.438 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:39:04.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419323+0000", 2024-09-14T22:39:04.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419324+0000", 2024-09-14T22:39:04.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2130000000000001e-06 2024-09-14T22:39:04.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419331+0000", 2024-09-14T22:39:04.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4730000000000003e-06 2024-09-14T22:39:04.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419348+0000", 2024-09-14T22:39:04.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6269999999999998e-05 2024-09-14T22:39:04.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419360+0000", 2024-09-14T22:39:04.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2243e-05 2024-09-14T22:39:04.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478884+0000", 2024-09-14T22:39:04.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059524369000000001 2024-09-14T22:39:04.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.478990+0000", 2024-09-14T22:39:04.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010569 2024-09-14T22:39:04.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479138+0000", 2024-09-14T22:39:04.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000147866 2024-09-14T22:39:04.442 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.442 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.442 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.442 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82516 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 999424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:39:04.443 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:39:04.443 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 124.992204403, 2024-09-14T22:39:04.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061979595999999998, 2024-09-14T22:39:04.443 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.443 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.443 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.443 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.443 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.443 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.444 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82516 2024-09-14T22:39:04.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.444 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.444 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.444 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:39:04.444 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419375+0000", 2024-09-14T22:39:04.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.446 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419376+0000", 2024-09-14T22:39:04.446 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.237e-06 2024-09-14T22:39:04.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.446 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.446 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.446 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419383+0000", 2024-09-14T22:39:04.446 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8170000000000001e-06 2024-09-14T22:39:04.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.447 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.447 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419400+0000", 2024-09-14T22:39:04.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6636000000000001e-05 2024-09-14T22:39:04.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.447 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.448 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419412+0000", 2024-09-14T22:39:04.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2357e-05 2024-09-14T22:39:04.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.448 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.448 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481080+0000", 2024-09-14T22:39:04.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061668032999999997 2024-09-14T22:39:04.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.449 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.449 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481193+0000", 2024-09-14T22:39:04.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112335 2024-09-14T22:39:04.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.449 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.449 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481355+0000", 2024-09-14T22:39:04.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016218100000000001 2024-09-14T22:39:04.449 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.449 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.449 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.450 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82538 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1097728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:39:04.450 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:39:04.450 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 124.99215233300001, 2024-09-14T22:39:04.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059971705, 2024-09-14T22:39:04.450 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.450 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.450 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.450 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.450 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.450 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.450 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82538 2024-09-14T22:39:04.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.451 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:39:04.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419427+0000", 2024-09-14T22:39:04.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419429+0000", 2024-09-14T22:39:04.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.497e-06 2024-09-14T22:39:04.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419436+0000", 2024-09-14T22:39:04.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2099999999999996e-06 2024-09-14T22:39:04.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419453+0000", 2024-09-14T22:39:04.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6840000000000001e-05 2024-09-14T22:39:04.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419465+0000", 2024-09-14T22:39:04.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2283000000000001e-05 2024-09-14T22:39:04.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479182+0000", 2024-09-14T22:39:04.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059717339000000001 2024-09-14T22:39:04.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479297+0000", 2024-09-14T22:39:04.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000114259 2024-09-14T22:39:04.454 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.454 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.454 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.454 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479399+0000", 2024-09-14T22:39:04.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102277 2024-09-14T22:39:04.454 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.454 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.454 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.454 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.454 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.454 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82539 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:39:04.454 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:39:04.455 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 124.992099483, 2024-09-14T22:39:04.455 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060166935999999997, 2024-09-14T22:39:04.455 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.455 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.455 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.455 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.455 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.455 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.455 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82539 2024-09-14T22:39:04.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.455 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.455 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.455 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.455 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:39:04.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419480+0000", 2024-09-14T22:39:04.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419482+0000", 2024-09-14T22:39:04.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4130000000000001e-06 2024-09-14T22:39:04.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419488+0000", 2024-09-14T22:39:04.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8460000000000002e-06 2024-09-14T22:39:04.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419505+0000", 2024-09-14T22:39:04.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6477000000000002e-05 2024-09-14T22:39:04.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419518+0000", 2024-09-14T22:39:04.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2797e-05 2024-09-14T22:39:04.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479438+0000", 2024-09-14T22:39:04.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.059919897 2024-09-14T22:39:04.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.459 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479547+0000", 2024-09-14T22:39:04.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109807 2024-09-14T22:39:04.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.459 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.459 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479647+0000", 2024-09-14T22:39:04.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9698999999999995e-05 2024-09-14T22:39:04.459 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.459 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.459 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.459 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82542 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:39:04.459 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:39:04.459 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 124.99204704, 2024-09-14T22:39:04.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060367430999999999, 2024-09-14T22:39:04.460 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.460 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.460 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.460 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.460 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.460 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.460 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82542 2024-09-14T22:39:04.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.460 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.460 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.460 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.460 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:39:04.460 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419533+0000", 2024-09-14T22:39:04.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419534+0000", 2024-09-14T22:39:04.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T22:39:04.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419542+0000", 2024-09-14T22:39:04.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8199999999999997e-06 2024-09-14T22:39:04.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419559+0000", 2024-09-14T22:39:04.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6852999999999999e-05 2024-09-14T22:39:04.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419571+0000", 2024-09-14T22:39:04.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2364000000000001e-05 2024-09-14T22:39:04.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479686+0000", 2024-09-14T22:39:04.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060115522999999997 2024-09-14T22:39:04.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479795+0000", 2024-09-14T22:39:04.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010831399999999999 2024-09-14T22:39:04.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479900+0000", 2024-09-14T22:39:04.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105284 2024-09-14T22:39:04.464 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.464 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.464 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.464 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82544 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:39:04.464 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:39:04.464 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 124.991993304, 2024-09-14T22:39:04.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060590515999999997, 2024-09-14T22:39:04.464 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.464 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.464 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.464 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.465 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.465 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.465 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82544 2024-09-14T22:39:04.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.465 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.465 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.465 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:39:04.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.465 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.465 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419586+0000", 2024-09-14T22:39:04.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.466 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419588+0000", 2024-09-14T22:39:04.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.35e-06 2024-09-14T22:39:04.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.466 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419595+0000", 2024-09-14T22:39:04.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4030000000000001e-06 2024-09-14T22:39:04.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.466 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419612+0000", 2024-09-14T22:39:04.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6670000000000001e-05 2024-09-14T22:39:04.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419624+0000", 2024-09-14T22:39:04.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-05 2024-09-14T22:39:04.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.479957+0000", 2024-09-14T22:39:04.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060333287999999999 2024-09-14T22:39:04.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480074+0000", 2024-09-14T22:39:04.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116443 2024-09-14T22:39:04.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480177+0000", 2024-09-14T22:39:04.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000103102 2024-09-14T22:39:04.468 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.468 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.468 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.468 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82546 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1687552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:39:04.468 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:39:04.468 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 124.99194038500001, 2024-09-14T22:39:04.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060795428999999998, 2024-09-14T22:39:04.469 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.469 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.469 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.469 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.469 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.469 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.469 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82546 2024-09-14T22:39:04.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.469 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.470 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:39:04.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.470 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.470 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.470 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.470 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419639+0000", 2024-09-14T22:39:04.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.470 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.470 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.470 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.470 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419640+0000", 2024-09-14T22:39:04.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.254e-06 2024-09-14T22:39:04.470 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419647+0000", 2024-09-14T22:39:04.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9e-06 2024-09-14T22:39:04.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419664+0000", 2024-09-14T22:39:04.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6345999999999999e-05 2024-09-14T22:39:04.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419676+0000", 2024-09-14T22:39:04.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.224e-05 2024-09-14T22:39:04.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480218+0000", 2024-09-14T22:39:04.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060542133999999997 2024-09-14T22:39:04.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480336+0000", 2024-09-14T22:39:04.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117472 2024-09-14T22:39:04.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480435+0000", 2024-09-14T22:39:04.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9083e-05 2024-09-14T22:39:04.474 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.474 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.474 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.474 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82548 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:39:04.474 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:39:04.474 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 124.991888375, 2024-09-14T22:39:04.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061011636000000001, 2024-09-14T22:39:04.474 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.474 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.474 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.474 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.475 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.475 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.475 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82548 2024-09-14T22:39:04.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.475 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.475 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.475 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:39:04.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.475 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.475 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419691+0000", 2024-09-14T22:39:04.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.476 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419693+0000", 2024-09-14T22:39:04.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.387e-06 2024-09-14T22:39:04.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.476 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419699+0000", 2024-09-14T22:39:04.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8730000000000001e-06 2024-09-14T22:39:04.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.476 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419716+0000", 2024-09-14T22:39:04.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6416e-05 2024-09-14T22:39:04.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419728+0000", 2024-09-14T22:39:04.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2597e-05 2024-09-14T22:39:04.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480476+0000", 2024-09-14T22:39:04.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060747989000000002 2024-09-14T22:39:04.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480583+0000", 2024-09-14T22:39:04.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010638500000000001 2024-09-14T22:39:04.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480703+0000", 2024-09-14T22:39:04.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011998900000000001 2024-09-14T22:39:04.478 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.478 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.478 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.479 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82550 3.a 3:5401b312:::rbd_data.113cce28790.0000000000000005:10 [read 1982464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:39:04.479 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:39:04.479 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 124.991835622, 2024-09-14T22:39:04.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061295483999999997, 2024-09-14T22:39:04.479 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.479 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.479 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.479 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.479 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.479 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.479 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82550 2024-09-14T22:39:04.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.479 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.480 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:39:04.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.480 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419744+0000", 2024-09-14T22:39:04.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.481 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419745+0000", 2024-09-14T22:39:04.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.406e-06 2024-09-14T22:39:04.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.481 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419753+0000", 2024-09-14T22:39:04.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8140000000000001e-06 2024-09-14T22:39:04.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.481 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419770+0000", 2024-09-14T22:39:04.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6719999999999999e-05 2024-09-14T22:39:04.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419782+0000", 2024-09-14T22:39:04.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2252999999999999e-05 2024-09-14T22:39:04.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480745+0000", 2024-09-14T22:39:04.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060962385000000001 2024-09-14T22:39:04.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.480900+0000", 2024-09-14T22:39:04.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015513899999999999 2024-09-14T22:39:04.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481039+0000", 2024-09-14T22:39:04.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013976700000000001 2024-09-14T22:39:04.484 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.484 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.484 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.484 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82588 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1064960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:39:04.484 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:39:04.484 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 124.99178173200001, 2024-09-14T22:39:04.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061863732999999997, 2024-09-14T22:39:04.485 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.485 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.485 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.485 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.485 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.485 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.485 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82588 2024-09-14T22:39:04.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.486 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.486 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.486 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:39:04.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.486 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.486 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419798+0000", 2024-09-14T22:39:04.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419799+0000", 2024-09-14T22:39:04.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4160000000000001e-06 2024-09-14T22:39:04.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419807+0000", 2024-09-14T22:39:04.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2169999999999994e-06 2024-09-14T22:39:04.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.488 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.488 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419824+0000", 2024-09-14T22:39:04.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6776e-05 2024-09-14T22:39:04.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.488 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419837+0000", 2024-09-14T22:39:04.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2680000000000001e-05 2024-09-14T22:39:04.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481395+0000", 2024-09-14T22:39:04.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061558416999999997 2024-09-14T22:39:04.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.490 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481532+0000", 2024-09-14T22:39:04.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000136861 2024-09-14T22:39:04.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481662+0000", 2024-09-14T22:39:04.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012936600000000001 2024-09-14T22:39:04.491 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.491 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.491 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.492 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82589 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:39:04.492 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:39:04.492 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 124.991727137, 2024-09-14T22:39:04.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062064592000000002, 2024-09-14T22:39:04.492 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.492 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.492 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.492 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.492 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.492 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.493 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82589 2024-09-14T22:39:04.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.493 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.493 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:39:04.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.493 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419852+0000", 2024-09-14T22:39:04.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.494 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419854+0000", 2024-09-14T22:39:04.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.367e-06 2024-09-14T22:39:04.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.494 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419862+0000", 2024-09-14T22:39:04.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8930000000000005e-06 2024-09-14T22:39:04.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.494 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.495 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419896+0000", 2024-09-14T22:39:04.495 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4134000000000002e-05 2024-09-14T22:39:04.495 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.495 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.495 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419909+0000", 2024-09-14T22:39:04.495 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3332999999999999e-05 2024-09-14T22:39:04.495 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.495 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.495 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481702+0000", 2024-09-14T22:39:04.495 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061792947000000001 2024-09-14T22:39:04.495 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.495 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.495 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481808+0000", 2024-09-14T22:39:04.496 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105655 2024-09-14T22:39:04.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.496 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.496 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481917+0000", 2024-09-14T22:39:04.496 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109263 2024-09-14T22:39:04.496 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.496 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.496 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.496 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82590 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1359872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:39:04.496 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:39:04.496 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 124.99165352999999, 2024-09-14T22:39:04.496 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062362895000000002, 2024-09-14T22:39:04.496 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.497 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.497 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.497 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.497 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.497 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.497 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82590 2024-09-14T22:39:04.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.497 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:39:04.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419926+0000", 2024-09-14T22:39:04.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419928+0000", 2024-09-14T22:39:04.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4470000000000001e-06 2024-09-14T22:39:04.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419942+0000", 2024-09-14T22:39:04.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4229e-05 2024-09-14T22:39:04.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419972+0000", 2024-09-14T22:39:04.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.065e-05 2024-09-14T22:39:04.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.500 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.500 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.419985+0000", 2024-09-14T22:39:04.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2903e-05 2024-09-14T22:39:04.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.500 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.500 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.481960+0000", 2024-09-14T22:39:04.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.061974435000000001 2024-09-14T22:39:04.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.501 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.501 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482106+0000", 2024-09-14T22:39:04.501 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014643800000000001 2024-09-14T22:39:04.501 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.501 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.501 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482289+0000", 2024-09-14T22:39:04.501 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000182793 2024-09-14T22:39:04.502 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.502 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.502 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.502 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82592 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:39:04.502 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:39:04.502 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 124.991577308, 2024-09-14T22:39:04.502 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062631507000000003, 2024-09-14T22:39:04.502 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.502 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.502 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.502 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.503 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.503 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.503 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82592 2024-09-14T22:39:04.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.503 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.503 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:39:04.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.503 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420002+0000", 2024-09-14T22:39:04.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420017+0000", 2024-09-14T22:39:04.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4744e-05 2024-09-14T22:39:04.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420036+0000", 2024-09-14T22:39:04.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8680000000000001e-05 2024-09-14T22:39:04.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420054+0000", 2024-09-14T22:39:04.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7799999999999999e-05 2024-09-14T22:39:04.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420067+0000", 2024-09-14T22:39:04.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3013000000000001e-05 2024-09-14T22:39:04.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482358+0000", 2024-09-14T22:39:04.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062291445000000001 2024-09-14T22:39:04.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.506 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.506 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482469+0000", 2024-09-14T22:39:04.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110608 2024-09-14T22:39:04.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.506 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.506 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482634+0000", 2024-09-14T22:39:04.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000165217 2024-09-14T22:39:04.506 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.506 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.506 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.507 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82594 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1654784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:39:04.507 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:39:04.507 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 124.991479332, 2024-09-14T22:39:04.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062893849000000002, 2024-09-14T22:39:04.507 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.507 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.507 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.507 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.507 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.507 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.507 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82594 2024-09-14T22:39:04.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.507 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:39:04.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420100+0000", 2024-09-14T22:39:04.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420102+0000", 2024-09-14T22:39:04.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5e-06 2024-09-14T22:39:04.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420115+0000", 2024-09-14T22:39:04.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.324e-05 2024-09-14T22:39:04.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420133+0000", 2024-09-14T22:39:04.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8213e-05 2024-09-14T22:39:04.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420147+0000", 2024-09-14T22:39:04.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3284e-05 2024-09-14T22:39:04.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482688+0000", 2024-09-14T22:39:04.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062541425999999997 2024-09-14T22:39:04.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482794+0000", 2024-09-14T22:39:04.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010575500000000001 2024-09-14T22:39:04.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.482994+0000", 2024-09-14T22:39:04.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020043099999999999 2024-09-14T22:39:04.510 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.511 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.511 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.511 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82596 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:39:04.511 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:39:04.511 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 124.991416032, 2024-09-14T22:39:04.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063187238000000007, 2024-09-14T22:39:04.511 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.511 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.511 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.511 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.511 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.511 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.511 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82596 2024-09-14T22:39:04.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.512 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:39:04.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420164+0000", 2024-09-14T22:39:04.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420165+0000", 2024-09-14T22:39:04.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4100000000000001e-06 2024-09-14T22:39:04.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420173+0000", 2024-09-14T22:39:04.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8129999999999996e-06 2024-09-14T22:39:04.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420190+0000", 2024-09-14T22:39:04.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7076999999999999e-05 2024-09-14T22:39:04.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420203+0000", 2024-09-14T22:39:04.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2683e-05 2024-09-14T22:39:04.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483060+0000", 2024-09-14T22:39:04.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062857114000000006 2024-09-14T22:39:04.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483167+0000", 2024-09-14T22:39:04.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010713999999999999 2024-09-14T22:39:04.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483351+0000", 2024-09-14T22:39:04.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018400099999999999 2024-09-14T22:39:04.515 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.515 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.515 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.516 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:82597 3.2 3:403373e9:::rbd_data.113cce28790.0000000000000009:10 [read 1949696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T22:39:04.516 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:39:04.516 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 124.991359206, 2024-09-14T22:39:04.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063466513000000002, 2024-09-14T22:39:04.516 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.516 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.516 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.516 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.516 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.516 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.517 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 82597 2024-09-14T22:39:04.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.517 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.517 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.517 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:39:04.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.517 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420220+0000", 2024-09-14T22:39:04.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420221+0000", 2024-09-14T22:39:04.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.043e-06 2024-09-14T22:39:04.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420229+0000", 2024-09-14T22:39:04.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0469999999999994e-06 2024-09-14T22:39:04.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420246+0000", 2024-09-14T22:39:04.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6339999999999999e-05 2024-09-14T22:39:04.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.420258+0000", 2024-09-14T22:39:04.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2296000000000001e-05 2024-09-14T22:39:04.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483402+0000", 2024-09-14T22:39:04.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.063144318000000005 2024-09-14T22:39:04.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483530+0000", 2024-09-14T22:39:04.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012713199999999999 2024-09-14T22:39:04.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:59.483687+0000", 2024-09-14T22:39:04.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015733699999999999 2024-09-14T22:39:04.520 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.520 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.520 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.520 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.520 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.520 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:39:04.522 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:39:04.648 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:39:04.648 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:39:04.648 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:39:04.648 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:39:04.662 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:39:04.779 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:39:04.779 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:39:04.779 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T22:39:04.779 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T22:39:04.780 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:39:04.794 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:39:04.800 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T22:39:04.911 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:39:04.911 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T22:39:04.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T22:39:04.911 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T22:39:04.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.911 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4201.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:39:04.912 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:39:04.912 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 184.90270897799999, 2024-09-14T22:39:04.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.398029152, 2024-09-14T22:39:04.912 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.912 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.912 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:39:04.912 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.912 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4201", 2024-09-14T22:39:04.912 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/1473633451", 2024-09-14T22:39:04.912 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 1 2024-09-14T22:39:04.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.912 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:39:04.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006180+0000", 2024-09-14T22:39:04.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006183+0000", 2024-09-14T22:39:04.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9059999999999998e-06 2024-09-14T22:39:04.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.914 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006198+0000", 2024-09-14T22:39:04.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.523e-05 2024-09-14T22:39:04.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.914 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006219+0000", 2024-09-14T22:39:04.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1342999999999999e-05 2024-09-14T22:39:04.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.914 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006240+0000", 2024-09-14T22:39:04.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1186999999999999e-05 2024-09-14T22:39:04.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006363+0000", 2024-09-14T22:39:04.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012258200000000001 2024-09-14T22:39:04.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T22:39:04.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.006392+0000", 2024-09-14T22:39:04.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9607e-05 2024-09-14T22:39:04.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401084+0000", 2024-09-14T22:39:04.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.39469142699999998 2024-09-14T22:39:04.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401394+0000", 2024-09-14T22:39:04.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030987799999999998 2024-09-14T22:39:04.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T22:39:04.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.401860+0000", 2024-09-14T22:39:04.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00046654299999999998 2024-09-14T22:39:04.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T22:39:04.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.403386+0000", 2024-09-14T22:39:04.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001525322 2024-09-14T22:39:04.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:39:04.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404042+0000", 2024-09-14T22:39:04.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065676099999999996 2024-09-14T22:39:04.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T22:39:04.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404136+0000", 2024-09-14T22:39:04.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4032999999999999e-05 2024-09-14T22:39:04.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:36:00.404209+0000", 2024-09-14T22:39:04.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2333e-05 2024-09-14T22:39:04.917 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.918 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.918 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.918 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104182 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14352384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:39:04.918 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:39:04.918 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 106.419854992, 2024-09-14T22:39:04.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071642496999999999, 2024-09-14T22:39:04.918 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.918 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.918 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.918 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.918 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.918 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.918 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104182 2024-09-14T22:39:04.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.919 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:39:04.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489034+0000", 2024-09-14T22:39:04.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489035+0000", 2024-09-14T22:39:04.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3430000000000001e-06 2024-09-14T22:39:04.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489041+0000", 2024-09-14T22:39:04.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8139999999999997e-06 2024-09-14T22:39:04.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489059+0000", 2024-09-14T22:39:04.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8193e-05 2024-09-14T22:39:04.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489073+0000", 2024-09-14T22:39:04.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:39:04.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560434+0000", 2024-09-14T22:39:04.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071361463999999999 2024-09-14T22:39:04.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560545+0000", 2024-09-14T22:39:04.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110263 2024-09-14T22:39:04.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560676+0000", 2024-09-14T22:39:04.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013138000000000001 2024-09-14T22:39:04.922 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.922 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.922 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.923 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104183 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14483456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:39:04.923 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:39:04.923 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 106.419731673, 2024-09-14T22:39:04.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071781943000000001, 2024-09-14T22:39:04.923 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.923 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.923 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.924 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.924 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.924 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.924 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104183 2024-09-14T22:39:04.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.924 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:39:04.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489157+0000", 2024-09-14T22:39:04.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489159+0000", 2024-09-14T22:39:04.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5629999999999999e-06 2024-09-14T22:39:04.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489165+0000", 2024-09-14T22:39:04.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7070000000000003e-06 2024-09-14T22:39:04.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489186+0000", 2024-09-14T22:39:04.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.088e-05 2024-09-14T22:39:04.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489201+0000", 2024-09-14T22:39:04.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5203e-05 2024-09-14T22:39:04.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.927 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560713+0000", 2024-09-14T22:39:04.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071511447000000006 2024-09-14T22:39:04.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.927 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560819+0000", 2024-09-14T22:39:04.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000106579 2024-09-14T22:39:04.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560939+0000", 2024-09-14T22:39:04.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119564 2024-09-14T22:39:04.928 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.928 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.928 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.928 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104184 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:39:04.928 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:39:04.928 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 106.41966723, 2024-09-14T22:39:04.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071970107000000005, 2024-09-14T22:39:04.928 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.928 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.929 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.929 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.929 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.929 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.929 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104184 2024-09-14T22:39:04.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.929 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:39:04.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489221+0000", 2024-09-14T22:39:04.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489223+0000", 2024-09-14T22:39:04.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4470000000000001e-06 2024-09-14T22:39:04.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489229+0000", 2024-09-14T22:39:04.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8200000000000002e-06 2024-09-14T22:39:04.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489247+0000", 2024-09-14T22:39:04.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8150000000000001e-05 2024-09-14T22:39:04.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489260+0000", 2024-09-14T22:39:04.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3570000000000001e-05 2024-09-14T22:39:04.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.560979+0000", 2024-09-14T22:39:04.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071718831999999996 2024-09-14T22:39:04.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561075+0000", 2024-09-14T22:39:04.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5848999999999997e-05 2024-09-14T22:39:04.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.932 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.932 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.932 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561191+0000", 2024-09-14T22:39:04.932 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116439 2024-09-14T22:39:04.932 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.932 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.932 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.932 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.932 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104185 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14745600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:39:04.932 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:39:04.932 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 106.41960764700001, 2024-09-14T22:39:04.932 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072173750999999994, 2024-09-14T22:39:04.932 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.933 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.933 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.933 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.933 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.933 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.933 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104185 2024-09-14T22:39:04.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.933 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:39:04.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489281+0000", 2024-09-14T22:39:04.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489282+0000", 2024-09-14T22:39:04.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2500000000000001e-06 2024-09-14T22:39:04.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489287+0000", 2024-09-14T22:39:04.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1359999999999996e-06 2024-09-14T22:39:04.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489304+0000", 2024-09-14T22:39:04.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7033000000000001e-05 2024-09-14T22:39:04.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489318+0000", 2024-09-14T22:39:04.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3934000000000001e-05 2024-09-14T22:39:04.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.936 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561232+0000", 2024-09-14T22:39:04.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.071913759999999993 2024-09-14T22:39:04.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.936 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.937 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561340+0000", 2024-09-14T22:39:04.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107921 2024-09-14T22:39:04.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.937 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.937 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.938 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561455+0000", 2024-09-14T22:39:04.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011471700000000001 2024-09-14T22:39:04.938 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.938 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.938 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.939 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104186 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 14876672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:39:04.939 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:39:04.939 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 106.419551738, 2024-09-14T22:39:04.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072393374999999996, 2024-09-14T22:39:04.940 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.940 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.940 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.940 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.940 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.940 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.941 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104186 2024-09-14T22:39:04.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.941 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.941 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.941 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:39:04.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.942 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.942 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.942 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.942 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489337+0000", 2024-09-14T22:39:04.942 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.942 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.942 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.942 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.942 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489338+0000", 2024-09-14T22:39:04.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.48e-06 2024-09-14T22:39:04.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489366+0000", 2024-09-14T22:39:04.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7307000000000001e-05 2024-09-14T22:39:04.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489385+0000", 2024-09-14T22:39:04.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9283000000000002e-05 2024-09-14T22:39:04.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489399+0000", 2024-09-14T22:39:04.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.452e-05 2024-09-14T22:39:04.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561494+0000", 2024-09-14T22:39:04.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072094885999999997 2024-09-14T22:39:04.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561604+0000", 2024-09-14T22:39:04.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010925900000000001 2024-09-14T22:39:04.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561730+0000", 2024-09-14T22:39:04.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012663999999999999 2024-09-14T22:39:04.945 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.945 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.945 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.946 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104187 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15007744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:39:04.946 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:39:04.946 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 106.419468369, 2024-09-14T22:39:04.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072599143000000005, 2024-09-14T22:39:04.946 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.946 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.946 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.946 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.946 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.946 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.946 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104187 2024-09-14T22:39:04.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.946 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:39:04.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489420+0000", 2024-09-14T22:39:04.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489422+0000", 2024-09-14T22:39:04.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5340000000000001e-06 2024-09-14T22:39:04.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489428+0000", 2024-09-14T22:39:04.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1430000000000003e-06 2024-09-14T22:39:04.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489445+0000", 2024-09-14T22:39:04.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7042999999999999e-05 2024-09-14T22:39:04.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489458+0000", 2024-09-14T22:39:04.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3474e-05 2024-09-14T22:39:04.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561771+0000", 2024-09-14T22:39:04.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072312600000000005 2024-09-14T22:39:04.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.561885+0000", 2024-09-14T22:39:04.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011411799999999999 2024-09-14T22:39:04.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562019+0000", 2024-09-14T22:39:04.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013423100000000001 2024-09-14T22:39:04.949 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.949 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.950 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.950 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104188 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15138816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:39:04.950 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:39:04.950 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 106.419408445, 2024-09-14T22:39:04.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072824624000000004, 2024-09-14T22:39:04.950 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.950 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.950 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.950 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.950 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.950 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.950 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104188 2024-09-14T22:39:04.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.951 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:39:04.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489480+0000", 2024-09-14T22:39:04.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489481+0000", 2024-09-14T22:39:04.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.22e-06 2024-09-14T22:39:04.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489487+0000", 2024-09-14T22:39:04.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6930000000000004e-06 2024-09-14T22:39:04.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489508+0000", 2024-09-14T22:39:04.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1135999999999999e-05 2024-09-14T22:39:04.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489522+0000", 2024-09-14T22:39:04.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3817e-05 2024-09-14T22:39:04.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.953 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562066+0000", 2024-09-14T22:39:04.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072543813999999998 2024-09-14T22:39:04.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.953 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562177+0000", 2024-09-14T22:39:04.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000111264 2024-09-14T22:39:04.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562305+0000", 2024-09-14T22:39:04.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012768 2024-09-14T22:39:04.954 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.954 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.954 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.954 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104189 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15269888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:39:04.954 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:39:04.954 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 106.419347996, 2024-09-14T22:39:04.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073061300999999995, 2024-09-14T22:39:04.954 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.954 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.954 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.955 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.955 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.955 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.955 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104189 2024-09-14T22:39:04.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.955 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:39:04.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489541+0000", 2024-09-14T22:39:04.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489542+0000", 2024-09-14T22:39:04.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1659999999999999e-06 2024-09-14T22:39:04.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489561+0000", 2024-09-14T22:39:04.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.908e-05 2024-09-14T22:39:04.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.957 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.957 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489579+0000", 2024-09-14T22:39:04.957 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8536999999999999e-05 2024-09-14T22:39:04.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.957 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.957 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489593+0000", 2024-09-14T22:39:04.957 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3977e-05 2024-09-14T22:39:04.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.957 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.957 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562347+0000", 2024-09-14T22:39:04.958 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072754092000000006 2024-09-14T22:39:04.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.958 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.958 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.958 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562458+0000", 2024-09-14T22:39:04.958 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110722 2024-09-14T22:39:04.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.958 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.958 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.958 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562602+0000", 2024-09-14T22:39:04.958 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014372699999999999 2024-09-14T22:39:04.958 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.958 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.958 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.959 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.959 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104190 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:39:04.959 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:39:04.959 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 106.4192757, 2024-09-14T22:39:04.959 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.07326133, 2024-09-14T22:39:04.959 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.959 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.959 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.959 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.959 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.960 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.960 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104190 2024-09-14T22:39:04.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.960 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:39:04.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.961 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489613+0000", 2024-09-14T22:39:04.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489614+0000", 2024-09-14T22:39:04.963 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1999999999999999e-06 2024-09-14T22:39:04.963 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.963 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.963 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.963 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489620+0000", 2024-09-14T22:39:04.963 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.524e-06 2024-09-14T22:39:04.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.964 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489637+0000", 2024-09-14T22:39:04.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7825999999999998e-05 2024-09-14T22:39:04.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489671+0000", 2024-09-14T22:39:04.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3726000000000002e-05 2024-09-14T22:39:04.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562644+0000", 2024-09-14T22:39:04.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072973089000000005 2024-09-14T22:39:04.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562762+0000", 2024-09-14T22:39:04.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011743600000000001 2024-09-14T22:39:04.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.967 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562874+0000", 2024-09-14T22:39:04.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011252900000000001 2024-09-14T22:39:04.967 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.967 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.967 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.967 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104191 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15532032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:39:04.967 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:39:04.967 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 106.419167277, 2024-09-14T22:39:04.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073311166999999997, 2024-09-14T22:39:04.967 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.968 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.968 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.968 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.968 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.968 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.968 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104191 2024-09-14T22:39:04.968 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.968 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.968 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:39:04.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489721+0000", 2024-09-14T22:39:04.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489723+0000", 2024-09-14T22:39:04.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6330000000000001e-06 2024-09-14T22:39:04.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489730+0000", 2024-09-14T22:39:04.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2300000000000002e-06 2024-09-14T22:39:04.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489751+0000", 2024-09-14T22:39:04.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0849999999999999e-05 2024-09-14T22:39:04.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489766+0000", 2024-09-14T22:39:04.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.524e-05 2024-09-14T22:39:04.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562896+0000", 2024-09-14T22:39:04.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073129662999999998 2024-09-14T22:39:04.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.562960+0000", 2024-09-14T22:39:04.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3689999999999995e-05 2024-09-14T22:39:04.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563033+0000", 2024-09-14T22:39:04.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2861000000000006e-05 2024-09-14T22:39:04.973 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.973 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.973 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.973 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104192 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:39:04.973 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:39:04.974 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 106.419071472, 2024-09-14T22:39:04.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073463632000000001, 2024-09-14T22:39:04.974 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.974 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.974 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.974 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.974 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.974 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.974 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104192 2024-09-14T22:39:04.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.975 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:39:04.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489817+0000", 2024-09-14T22:39:04.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.976 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.976 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489819+0000", 2024-09-14T22:39:04.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.424e-06 2024-09-14T22:39:04.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.976 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.976 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489825+0000", 2024-09-14T22:39:04.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.02e-06 2024-09-14T22:39:04.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.976 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.976 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489844+0000", 2024-09-14T22:39:04.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9760000000000001e-05 2024-09-14T22:39:04.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489860+0000", 2024-09-14T22:39:04.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5563e-05 2024-09-14T22:39:04.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563070+0000", 2024-09-14T22:39:04.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073209724000000004 2024-09-14T22:39:04.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563175+0000", 2024-09-14T22:39:04.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105183 2024-09-14T22:39:04.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563281+0000", 2024-09-14T22:39:04.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105958 2024-09-14T22:39:04.978 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.978 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.978 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.978 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104193 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15794176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:39:04.978 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:39:04.978 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 106.419009305, 2024-09-14T22:39:04.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073600040000000005, 2024-09-14T22:39:04.979 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.979 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.979 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.979 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.979 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.979 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.979 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104193 2024-09-14T22:39:04.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.980 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.980 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.980 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:39:04.980 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.981 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.981 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489879+0000", 2024-09-14T22:39:04.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.981 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.981 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489881+0000", 2024-09-14T22:39:04.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5460000000000001e-06 2024-09-14T22:39:04.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489886+0000", 2024-09-14T22:39:04.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1739999999999999e-06 2024-09-14T22:39:04.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489912+0000", 2024-09-14T22:39:04.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6461999999999998e-05 2024-09-14T22:39:04.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489927+0000", 2024-09-14T22:39:04.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4039999999999999e-05 2024-09-14T22:39:04.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563310+0000", 2024-09-14T22:39:04.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073382977000000002 2024-09-14T22:39:04.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563385+0000", 2024-09-14T22:39:04.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5218999999999999e-05 2024-09-14T22:39:04.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.984 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563479+0000", 2024-09-14T22:39:04.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4622000000000003e-05 2024-09-14T22:39:04.984 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.984 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.984 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.984 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.984 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104194 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 15925248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:39:04.984 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:39:04.984 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 106.41894271300001, 2024-09-14T22:39:04.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073745434999999998, 2024-09-14T22:39:04.984 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.984 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.985 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.985 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.985 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.985 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.985 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104194 2024-09-14T22:39:04.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.985 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.985 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.985 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.985 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:39:04.985 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.985 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.985 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.986 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489946+0000", 2024-09-14T22:39:04.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.986 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.986 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.986 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489947+0000", 2024-09-14T22:39:04.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.327e-06 2024-09-14T22:39:04.986 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.986 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.986 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489953+0000", 2024-09-14T22:39:04.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9329999999999998e-06 2024-09-14T22:39:04.986 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489971+0000", 2024-09-14T22:39:04.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7994000000000001e-05 2024-09-14T22:39:04.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.489985+0000", 2024-09-14T22:39:04.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4226000000000001e-05 2024-09-14T22:39:04.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563510+0000", 2024-09-14T22:39:04.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073524667000000002 2024-09-14T22:39:04.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.988 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563597+0000", 2024-09-14T22:39:04.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6550000000000003e-05 2024-09-14T22:39:04.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.988 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563691+0000", 2024-09-14T22:39:04.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4738e-05 2024-09-14T22:39:04.988 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.988 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.988 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.988 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104195 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:39:04.989 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:39:04.989 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 106.41888451600001, 2024-09-14T22:39:04.989 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073876598000000002, 2024-09-14T22:39:04.989 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.989 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.989 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.989 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.989 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.989 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.989 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104195 2024-09-14T22:39:04.989 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.989 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.989 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.989 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:39:04.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.990 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490004+0000", 2024-09-14T22:39:04.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.990 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490006+0000", 2024-09-14T22:39:04.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4699999999999999e-06 2024-09-14T22:39:04.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.991 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.991 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490011+0000", 2024-09-14T22:39:04.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.366e-06 2024-09-14T22:39:04.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.991 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.991 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.991 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490046+0000", 2024-09-14T22:39:04.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5496e-05 2024-09-14T22:39:04.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.991 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.991 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.991 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490061+0000", 2024-09-14T22:39:04.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5054e-05 2024-09-14T22:39:04.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.991 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563719+0000", 2024-09-14T22:39:04.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073657412000000005 2024-09-14T22:39:04.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563792+0000", 2024-09-14T22:39:04.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3319999999999996e-05 2024-09-14T22:39:04.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563881+0000", 2024-09-14T22:39:04.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8480000000000007e-05 2024-09-14T22:39:04.992 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.993 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.993 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.993 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.993 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104196 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16187392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:39:04.993 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:39:04.993 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 106.418807367, 2024-09-14T22:39:04.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073997496999999995, 2024-09-14T22:39:04.993 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.993 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.993 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.993 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.993 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.993 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.994 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104196 2024-09-14T22:39:04.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.994 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:39:04.994 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490081+0000", 2024-09-14T22:39:04.994 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.995 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490083+0000", 2024-09-14T22:39:04.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.347e-06 2024-09-14T22:39:04.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.995 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490089+0000", 2024-09-14T22:39:04.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9630000000000003e-06 2024-09-14T22:39:04.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.995 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:04.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490107+0000", 2024-09-14T22:39:04.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8859999999999999e-05 2024-09-14T22:39:04.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.996 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:04.996 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490122+0000", 2024-09-14T22:39:04.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.432e-05 2024-09-14T22:39:04.996 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.996 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:04.996 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563910+0000", 2024-09-14T22:39:04.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073788753999999998 2024-09-14T22:39:04.996 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.996 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:04.996 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.563988+0000", 2024-09-14T22:39:04.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7862999999999996e-05 2024-09-14T22:39:04.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.997 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:04.997 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564079+0000", 2024-09-14T22:39:04.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0389999999999993e-05 2024-09-14T22:39:04.997 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.997 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:04.997 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:04.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.997 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104197 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16318464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:39:04.997 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:39:04.997 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 106.418715738, 2024-09-14T22:39:04.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074103547000000006, 2024-09-14T22:39:04.998 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:04.998 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:04.998 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:04.998 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:04.998 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:04.998 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:04.998 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104197 2024-09-14T22:39:04.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.998 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:04.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:04.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:39:04.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:04.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490173+0000", 2024-09-14T22:39:04.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:04.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:04.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490174+0000", 2024-09-14T22:39:04.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.37e-06 2024-09-14T22:39:04.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:04.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:04.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:04.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490197+0000", 2024-09-14T22:39:04.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3017e-05 2024-09-14T22:39:05.000 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.000 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:05.000 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490219+0000", 2024-09-14T22:39:05.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.181e-05 2024-09-14T22:39:05.000 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.000 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:05.000 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490240+0000", 2024-09-14T22:39:05.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0786000000000001e-05 2024-09-14T22:39:05.000 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.000 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:05.001 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564109+0000", 2024-09-14T22:39:05.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073869031000000002 2024-09-14T22:39:05.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:05.001 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564186+0000", 2024-09-14T22:39:05.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6823999999999995e-05 2024-09-14T22:39:05.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:05.001 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564276+0000", 2024-09-14T22:39:05.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0709000000000002e-05 2024-09-14T22:39:05.001 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:05.001 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:05.002 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:05.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.002 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.002 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104198 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16449536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:39:05.002 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:39:05.002 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 106.418623502, 2024-09-14T22:39:05.002 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074167693000000007, 2024-09-14T22:39:05.002 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:05.002 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:05.002 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:05.002 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:05.002 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:05.002 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:05.002 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104198 2024-09-14T22:39:05.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.003 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:05.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:05.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:39:05.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:05.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:05.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490265+0000", 2024-09-14T22:39:05.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:05.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:05.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490266+0000", 2024-09-14T22:39:05.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3599999999999999e-06 2024-09-14T22:39:05.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:05.004 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490272+0000", 2024-09-14T22:39:05.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9599999999999997e-06 2024-09-14T22:39:05.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:05.004 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490291+0000", 2024-09-14T22:39:05.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8485999999999999e-05 2024-09-14T22:39:05.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:05.005 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490305+0000", 2024-09-14T22:39:05.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4327e-05 2024-09-14T22:39:05.005 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.005 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.005 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:05.005 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564306+0000", 2024-09-14T22:39:05.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074000745000000007 2024-09-14T22:39:05.005 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.005 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.005 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:05.005 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564367+0000", 2024-09-14T22:39:05.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1501999999999995e-05 2024-09-14T22:39:05.005 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.005 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.006 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:05.006 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564433+0000", 2024-09-14T22:39:05.006 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5313000000000003e-05 2024-09-14T22:39:05.006 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:05.006 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:05.006 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:05.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.006 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.006 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104199 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16580608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:39:05.006 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:39:05.006 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 106.418541949, 2024-09-14T22:39:05.006 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074231631000000006, 2024-09-14T22:39:05.006 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:05.006 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:05.006 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:05.007 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:05.007 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:05.007 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:05.007 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104199 2024-09-14T22:39:05.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.007 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:05.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.007 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:05.007 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:39:05.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:05.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.007 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:05.007 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490347+0000", 2024-09-14T22:39:05.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:05.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:05.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490349+0000", 2024-09-14T22:39:05.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9429999999999999e-06 2024-09-14T22:39:05.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:05.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490355+0000", 2024-09-14T22:39:05.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1670000000000004e-06 2024-09-14T22:39:05.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:05.009 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490374+0000", 2024-09-14T22:39:05.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9613e-05 2024-09-14T22:39:05.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:05.009 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.490405+0000", 2024-09-14T22:39:05.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0936000000000003e-05 2024-09-14T22:39:05.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:05.009 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564451+0000", 2024-09-14T22:39:05.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.074045266999999998 2024-09-14T22:39:05.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.010 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:05.010 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564494+0000", 2024-09-14T22:39:05.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3133000000000002e-05 2024-09-14T22:39:05.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.010 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:05.010 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564578+0000", 2024-09-14T22:39:05.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4572000000000003e-05 2024-09-14T22:39:05.010 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:05.010 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:05.010 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:05.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.010 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4412.0:104200 3.e 3:71353f73:::rbd_data.113cdbec4ad.0000000000000004:18 [sparse-read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-14T22:39:05.011 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:39:05.011 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 106.416746721, 2024-09-14T22:39:05.011 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072582435000000001, 2024-09-14T22:39:05.011 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T22:39:05.011 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T22:39:05.011 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T22:39:05.011 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T22:39:05.011 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4412", 2024-09-14T22:39:05.011 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:05.011 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104200 2024-09-14T22:39:05.011 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.011 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T22:39:05.011 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.011 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T22:39:05.011 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:39:05.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:05.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T22:39:05.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492142+0000", 2024-09-14T22:39:05.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T22:39:05.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T22:39:05.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492144+0000", 2024-09-14T22:39:05.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3070000000000001e-06 2024-09-14T22:39:05.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T22:39:05.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492155+0000", 2024-09-14T22:39:05.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0573e-05 2024-09-14T22:39:05.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T22:39:05.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492185+0000", 2024-09-14T22:39:05.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.008e-05 2024-09-14T22:39:05.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T22:39:05.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.492205+0000", 2024-09-14T22:39:05.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9882999999999999e-05 2024-09-14T22:39:05.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T22:39:05.014 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564609+0000", 2024-09-14T22:39:05.014 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.072404181999999997 2024-09-14T22:39:05.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T22:39:05.014 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564667+0000", 2024-09-14T22:39:05.014 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8102e-05 2024-09-14T22:39:05.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T22:39:05.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T22:39:05.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T22:39:05.014 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T22:37:18.564724+0000", 2024-09-14T22:39:05.014 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7308000000000001e-05 2024-09-14T22:39:05.014 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:05.014 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:05.015 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:05.015 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T22:39:05.015 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T22:39:05.015 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:39:05.016 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:39:05.143 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:39:05.143 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:39:05.143 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-14T22:39:05.143 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:39:05.158 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:39:05.257 INFO:tasks.thrashosds.thrasher:reviving osd 2024-09-14T22:39:05.257 INFO:tasks.thrashosds.thrasher:Reviving osd 4 2024-09-14T22:39:05.257 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-14T22:39:05.257 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-14T22:39:05.260 INFO:tasks.ceph.osd.4:Started 2024-09-14T22:39:05.260 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:39:05.283 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:39:05.284 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-14T22:39:05.284 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-14T22:39:05.284 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-14T22:39:05.284 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:39:05.298 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:39:05.381 INFO:teuthology.orchestra.run.smithi176.stderr:no valid command found; 10 closest matches: 2024-09-14T22:39:05.381 INFO:teuthology.orchestra.run.smithi176.stderr:0 2024-09-14T22:39:05.381 INFO:teuthology.orchestra.run.smithi176.stderr:1 2024-09-14T22:39:05.382 INFO:teuthology.orchestra.run.smithi176.stderr:2 2024-09-14T22:39:05.382 INFO:teuthology.orchestra.run.smithi176.stderr:abort 2024-09-14T22:39:05.382 INFO:teuthology.orchestra.run.smithi176.stderr:assert 2024-09-14T22:39:05.382 INFO:teuthology.orchestra.run.smithi176.stderr:config diff 2024-09-14T22:39:05.382 INFO:teuthology.orchestra.run.smithi176.stderr:config diff get 2024-09-14T22:39:05.382 INFO:teuthology.orchestra.run.smithi176.stderr:config get 2024-09-14T22:39:05.382 INFO:teuthology.orchestra.run.smithi176.stderr:config help [] 2024-09-14T22:39:05.382 INFO:teuthology.orchestra.run.smithi176.stderr:config set ... 2024-09-14T22:39:05.382 INFO:teuthology.orchestra.run.smithi176.stderr:admin_socket: invalid command 2024-09-14T22:39:05.384 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T22:39:05.384 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-4, ['dump_ops_in_flight'] 2024-09-14T22:39:05.420 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:39:05.420 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-14T22:39:05.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-14T22:39:05.420 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-14T22:39:05.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.420 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144987 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3424256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:05.420 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549297+0000", 2024-09-14T22:39:05.420 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 72.866826810999996, 2024-09-14T22:39:05.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.079931556000000001, 2024-09-14T22:39:05.420 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:05.421 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:05.421 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:05.421 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:05.421 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:05.421 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:05.421 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144987 2024-09-14T22:39:05.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.421 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:05.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:05.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549297+0000", 2024-09-14T22:39:05.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:05.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549297+0000", 2024-09-14T22:39:05.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:05.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549298+0000", 2024-09-14T22:39:05.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2699999999999999e-07 2024-09-14T22:39:05.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:05.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549301+0000", 2024-09-14T22:39:05.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2959999999999999e-06 2024-09-14T22:39:05.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:05.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549314+0000", 2024-09-14T22:39:05.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2588999999999999e-05 2024-09-14T22:39:05.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:05.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549323+0000", 2024-09-14T22:39:05.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.927e-06 2024-09-14T22:39:05.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:05.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629038+0000", 2024-09-14T22:39:05.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.079714220000000002 2024-09-14T22:39:05.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:05.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629116+0000", 2024-09-14T22:39:05.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7966000000000005e-05 2024-09-14T22:39:05.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:05.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629228+0000", 2024-09-14T22:39:05.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112831 2024-09-14T22:39:05.424 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.424 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:05.424 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.425 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144988 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3325952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:05.425 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549335+0000", 2024-09-14T22:39:05.425 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 72.866788585999998, 2024-09-14T22:39:05.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080179116999999994, 2024-09-14T22:39:05.425 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:05.425 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:05.425 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:05.425 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:05.425 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:05.425 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:05.425 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144988 2024-09-14T22:39:05.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.425 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:05.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:05.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549335+0000", 2024-09-14T22:39:05.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:05.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549335+0000", 2024-09-14T22:39:05.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:05.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549336+0000", 2024-09-14T22:39:05.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7999999999999995e-07 2024-09-14T22:39:05.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:05.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549339+0000", 2024-09-14T22:39:05.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.836e-06 2024-09-14T22:39:05.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:05.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549352+0000", 2024-09-14T22:39:05.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3069e-05 2024-09-14T22:39:05.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:05.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549359+0000", 2024-09-14T22:39:05.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.357e-06 2024-09-14T22:39:05.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:05.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629288+0000", 2024-09-14T22:39:05.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.079928492000000004 2024-09-14T22:39:05.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:05.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629412+0000", 2024-09-14T22:39:05.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012413699999999999 2024-09-14T22:39:05.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:05.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629514+0000", 2024-09-14T22:39:05.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000102546 2024-09-14T22:39:05.429 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.429 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:05.429 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.429 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144989 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3522560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:05.429 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:39:05.429 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 72.866754177999994, 2024-09-14T22:39:05.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080393122999999997, 2024-09-14T22:39:05.429 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:05.429 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:05.429 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:05.430 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:05.430 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:05.430 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:05.430 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144989 2024-09-14T22:39:05.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.430 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:05.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:05.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:39:05.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:05.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:39:05.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:05.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549370+0000", 2024-09-14T22:39:05.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3600000000000003e-07 2024-09-14T22:39:05.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:05.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549373+0000", 2024-09-14T22:39:05.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8169999999999999e-06 2024-09-14T22:39:05.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:05.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549385+0000", 2024-09-14T22:39:05.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1803e-05 2024-09-14T22:39:05.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:05.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549394+0000", 2024-09-14T22:39:05.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4290000000000007e-06 2024-09-14T22:39:05.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:05.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629576+0000", 2024-09-14T22:39:05.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080181551000000004 2024-09-14T22:39:05.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:05.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629654+0000", 2024-09-14T22:39:05.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7633999999999996e-05 2024-09-14T22:39:05.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:05.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629763+0000", 2024-09-14T22:39:05.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109153 2024-09-14T22:39:05.433 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.433 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:05.433 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.433 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144990 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:05.433 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:39:05.434 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 72.866711154000001, 2024-09-14T22:39:05.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080614496999999993, 2024-09-14T22:39:05.434 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:05.434 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:05.434 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:05.434 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:05.434 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:05.434 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:05.434 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144990 2024-09-14T22:39:05.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.434 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:05.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:05.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:39:05.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:05.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:39:05.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:05.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549413+0000", 2024-09-14T22:39:05.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0999999999999997e-07 2024-09-14T22:39:05.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:05.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549419+0000", 2024-09-14T22:39:05.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5339999999999999e-06 2024-09-14T22:39:05.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:05.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549439+0000", 2024-09-14T22:39:05.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0035e-05 2024-09-14T22:39:05.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:05.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549449+0000", 2024-09-14T22:39:05.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0429e-05 2024-09-14T22:39:05.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:05.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629815+0000", 2024-09-14T22:39:05.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080365467999999995 2024-09-14T22:39:05.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:05.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.629922+0000", 2024-09-14T22:39:05.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106914 2024-09-14T22:39:05.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:05.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630027+0000", 2024-09-14T22:39:05.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105207 2024-09-14T22:39:05.438 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.438 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:05.438 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.438 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144991 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 2998272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:05.438 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549463+0000", 2024-09-14T22:39:05.438 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 72.866660850000002, 2024-09-14T22:39:05.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080847637, 2024-09-14T22:39:05.438 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:05.438 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:05.438 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:05.438 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:05.438 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:05.439 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:05.439 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144991 2024-09-14T22:39:05.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.439 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:05.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:05.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549463+0000", 2024-09-14T22:39:05.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:05.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549463+0000", 2024-09-14T22:39:05.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:05.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549464+0000", 2024-09-14T22:39:05.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9999999999999997e-07 2024-09-14T22:39:05.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:05.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549467+0000", 2024-09-14T22:39:05.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0800000000000002e-06 2024-09-14T22:39:05.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:05.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549478+0000", 2024-09-14T22:39:05.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1813e-05 2024-09-14T22:39:05.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:05.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549488+0000", 2024-09-14T22:39:05.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7030000000000001e-06 2024-09-14T22:39:05.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:05.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630090+0000", 2024-09-14T22:39:05.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080601962999999999 2024-09-14T22:39:05.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:05.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630187+0000", 2024-09-14T22:39:05.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6552000000000007e-05 2024-09-14T22:39:05.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:05.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630311+0000", 2024-09-14T22:39:05.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000123826 2024-09-14T22:39:05.442 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.442 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:05.442 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.442 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144992 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3620864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:05.442 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:39:05.443 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 72.866625041999995, 2024-09-14T22:39:05.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081097875999999999, 2024-09-14T22:39:05.443 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:05.443 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:05.443 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:05.443 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:05.443 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:05.443 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:05.443 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144992 2024-09-14T22:39:05.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.443 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:05.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:05.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:39:05.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:05.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:39:05.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:05.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549499+0000", 2024-09-14T22:39:05.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5400000000000001e-07 2024-09-14T22:39:05.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:05.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549502+0000", 2024-09-14T22:39:05.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7889999999999999e-06 2024-09-14T22:39:05.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:05.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549513+0000", 2024-09-14T22:39:05.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1204e-05 2024-09-14T22:39:05.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:05.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549523+0000", 2024-09-14T22:39:05.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5990000000000008e-06 2024-09-14T22:39:05.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:05.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630362+0000", 2024-09-14T22:39:05.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.080839272000000004 2024-09-14T22:39:05.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:05.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630469+0000", 2024-09-14T22:39:05.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107285 2024-09-14T22:39:05.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:05.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630597+0000", 2024-09-14T22:39:05.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012707300000000001 2024-09-14T22:39:05.446 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.446 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:05.447 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.447 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144993 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:05.447 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:39:05.447 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 72.866591966000001, 2024-09-14T22:39:05.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081338411999999999, 2024-09-14T22:39:05.447 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:05.447 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:05.447 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:05.447 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:05.447 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:05.447 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:05.448 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144993 2024-09-14T22:39:05.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.448 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:05.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:05.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:39:05.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:05.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:39:05.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:05.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549532+0000", 2024-09-14T22:39:05.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5300000000000004e-07 2024-09-14T22:39:05.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:05.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549544+0000", 2024-09-14T22:39:05.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1912e-05 2024-09-14T22:39:05.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:05.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549557+0000", 2024-09-14T22:39:05.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2413e-05 2024-09-14T22:39:05.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.450 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:05.450 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549567+0000", 2024-09-14T22:39:05.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8660000000000001e-06 2024-09-14T22:39:05.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.450 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:05.450 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630651+0000", 2024-09-14T22:39:05.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081084229999999993 2024-09-14T22:39:05.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.450 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:05.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630729+0000", 2024-09-14T22:39:05.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8265999999999999e-05 2024-09-14T22:39:05.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:05.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630870+0000", 2024-09-14T22:39:05.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014107200000000001 2024-09-14T22:39:05.451 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.451 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:05.451 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.451 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144994 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3293184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:05.451 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:39:05.452 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 72.866544998999998, 2024-09-14T22:39:05.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081547958000000004, 2024-09-14T22:39:05.452 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:05.452 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:05.452 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:05.452 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:05.452 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:05.452 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:05.452 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144994 2024-09-14T22:39:05.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.452 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:05.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:05.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:39:05.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:05.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:39:05.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:05.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549579+0000", 2024-09-14T22:39:05.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6599999999999996e-07 2024-09-14T22:39:05.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:05.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549582+0000", 2024-09-14T22:39:05.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.004e-06 2024-09-14T22:39:05.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:05.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549594+0000", 2024-09-14T22:39:05.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1459e-05 2024-09-14T22:39:05.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:05.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549603+0000", 2024-09-14T22:39:05.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6129999999999993e-06 2024-09-14T22:39:05.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:05.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.630932+0000", 2024-09-14T22:39:05.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081328558999999995 2024-09-14T22:39:05.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:05.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631027+0000", 2024-09-14T22:39:05.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4742999999999997e-05 2024-09-14T22:39:05.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:05.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631127+0000", 2024-09-14T22:39:05.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9914000000000006e-05 2024-09-14T22:39:05.455 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.456 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:05.456 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.456 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144995 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3391488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:05.456 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549615+0000", 2024-09-14T22:39:05.456 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 72.866508861, 2024-09-14T22:39:05.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081734251999999993, 2024-09-14T22:39:05.456 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:05.456 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:05.456 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:05.456 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:05.456 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:05.456 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:05.457 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144995 2024-09-14T22:39:05.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.457 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:05.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:05.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549615+0000", 2024-09-14T22:39:05.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:05.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549615+0000", 2024-09-14T22:39:05.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:05.458 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549616+0000", 2024-09-14T22:39:05.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9299999999999997e-07 2024-09-14T22:39:05.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:05.458 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549618+0000", 2024-09-14T22:39:05.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7269999999999999e-06 2024-09-14T22:39:05.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:05.458 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549630+0000", 2024-09-14T22:39:05.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1313e-05 2024-09-14T22:39:05.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.459 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:05.459 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549639+0000", 2024-09-14T22:39:05.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4159999999999993e-06 2024-09-14T22:39:05.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.459 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:05.459 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631183+0000", 2024-09-14T22:39:05.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081543633000000004 2024-09-14T22:39:05.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.459 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:05.459 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631252+0000", 2024-09-14T22:39:05.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9567999999999995e-05 2024-09-14T22:39:05.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.460 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:05.460 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631349+0000", 2024-09-14T22:39:05.460 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6801999999999999e-05 2024-09-14T22:39:05.460 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.460 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:05.460 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.460 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144996 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3194880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:05.460 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:39:05.460 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 72.866473182999997, 2024-09-14T22:39:05.460 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081947213000000005, 2024-09-14T22:39:05.460 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:05.460 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:05.460 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:05.461 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:05.461 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:05.461 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:05.461 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144996 2024-09-14T22:39:05.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.461 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:05.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:05.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:39:05.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:05.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:39:05.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:05.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549651+0000", 2024-09-14T22:39:05.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3e-07 2024-09-14T22:39:05.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:05.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549654+0000", 2024-09-14T22:39:05.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6900000000000001e-06 2024-09-14T22:39:05.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:05.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549665+0000", 2024-09-14T22:39:05.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1392999999999999e-05 2024-09-14T22:39:05.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.463 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:05.463 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549675+0000", 2024-09-14T22:39:05.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5459999999999997e-06 2024-09-14T22:39:05.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.463 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:05.463 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631401+0000", 2024-09-14T22:39:05.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081726036000000002 2024-09-14T22:39:05.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.463 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:05.463 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631492+0000", 2024-09-14T22:39:05.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0886999999999999e-05 2024-09-14T22:39:05.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.464 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:05.464 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631598+0000", 2024-09-14T22:39:05.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106031 2024-09-14T22:39:05.464 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.464 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:05.464 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.464 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144997 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3489792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:05.464 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549692+0000", 2024-09-14T22:39:05.464 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 72.866431519000002, 2024-09-14T22:39:05.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082149925999999998, 2024-09-14T22:39:05.465 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:05.465 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:05.465 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:05.465 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:05.465 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:05.465 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:05.465 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144997 2024-09-14T22:39:05.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.465 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:05.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:05.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549692+0000", 2024-09-14T22:39:05.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:05.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549692+0000", 2024-09-14T22:39:05.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:05.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549693+0000", 2024-09-14T22:39:05.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.540000000000001e-07 2024-09-14T22:39:05.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:05.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549698+0000", 2024-09-14T22:39:05.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1460000000000004e-06 2024-09-14T22:39:05.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:05.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549715+0000", 2024-09-14T22:39:05.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6983e-05 2024-09-14T22:39:05.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:05.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549728+0000", 2024-09-14T22:39:05.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2326e-05 2024-09-14T22:39:05.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:05.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631649+0000", 2024-09-14T22:39:05.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.081921830000000001 2024-09-14T22:39:05.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:05.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631726+0000", 2024-09-14T22:39:05.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6759000000000001e-05 2024-09-14T22:39:05.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:05.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631842+0000", 2024-09-14T22:39:05.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011592800000000001 2024-09-14T22:39:05.469 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.469 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:05.469 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.469 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144998 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 2965504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:05.469 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549741+0000", 2024-09-14T22:39:05.469 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 72.866382227000003, 2024-09-14T22:39:05.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082394832000000001, 2024-09-14T22:39:05.469 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:05.469 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:05.470 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:05.470 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:05.470 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:05.470 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:05.470 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144998 2024-09-14T22:39:05.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.470 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:05.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:05.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549741+0000", 2024-09-14T22:39:05.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:05.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549741+0000", 2024-09-14T22:39:05.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:05.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549742+0000", 2024-09-14T22:39:05.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.06e-07 2024-09-14T22:39:05.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:05.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549757+0000", 2024-09-14T22:39:05.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4839e-05 2024-09-14T22:39:05.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:05.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549770+0000", 2024-09-14T22:39:05.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2879e-05 2024-09-14T22:39:05.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:05.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549780+0000", 2024-09-14T22:39:05.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0304e-05 2024-09-14T22:39:05.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:05.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631904+0000", 2024-09-14T22:39:05.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082124168999999997 2024-09-14T22:39:05.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:05.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.631995+0000", 2024-09-14T22:39:05.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0993000000000001e-05 2024-09-14T22:39:05.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:05.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632136+0000", 2024-09-14T22:39:05.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014094199999999999 2024-09-14T22:39:05.473 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.473 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:05.473 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.473 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:144999 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3588096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:05.474 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:39:05.474 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 72.866332017000005, 2024-09-14T22:39:05.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082593742999999997, 2024-09-14T22:39:05.474 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:05.474 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:05.474 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:05.474 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:05.474 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:05.474 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:05.474 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 144999 2024-09-14T22:39:05.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.474 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:05.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:05.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:39:05.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:05.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:39:05.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:05.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549792+0000", 2024-09-14T22:39:05.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1999999999999999e-07 2024-09-14T22:39:05.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:05.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549795+0000", 2024-09-14T22:39:05.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9569999999999998e-06 2024-09-14T22:39:05.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:05.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549807+0000", 2024-09-14T22:39:05.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1691999999999999e-05 2024-09-14T22:39:05.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:05.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549817+0000", 2024-09-14T22:39:05.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5100000000000004e-06 2024-09-14T22:39:05.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:05.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632190+0000", 2024-09-14T22:39:05.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082373374999999999 2024-09-14T22:39:05.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:05.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632264+0000", 2024-09-14T22:39:05.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3972000000000003e-05 2024-09-14T22:39:05.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:05.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632385+0000", 2024-09-14T22:39:05.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121517 2024-09-14T22:39:05.478 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.478 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:05.478 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.478 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145000 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3063808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:05.478 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549828+0000", 2024-09-14T22:39:05.478 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 72.866295738999995, 2024-09-14T22:39:05.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082860235000000004, 2024-09-14T22:39:05.478 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:05.478 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:05.478 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:05.478 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:05.479 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:05.479 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:05.479 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145000 2024-09-14T22:39:05.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.479 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:05.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:05.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549828+0000", 2024-09-14T22:39:05.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:05.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549828+0000", 2024-09-14T22:39:05.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:05.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549829+0000", 2024-09-14T22:39:05.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3300000000000002e-07 2024-09-14T22:39:05.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:05.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549831+0000", 2024-09-14T22:39:05.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6740000000000001e-06 2024-09-14T22:39:05.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:05.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549842+0000", 2024-09-14T22:39:05.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0316000000000001e-05 2024-09-14T22:39:05.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:05.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549851+0000", 2024-09-14T22:39:05.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7229999999999999e-06 2024-09-14T22:39:05.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:05.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632448+0000", 2024-09-14T22:39:05.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082596648999999994 2024-09-14T22:39:05.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:05.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632568+0000", 2024-09-14T22:39:05.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119592 2024-09-14T22:39:05.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:05.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632688+0000", 2024-09-14T22:39:05.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120648 2024-09-14T22:39:05.482 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.482 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:05.482 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.482 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145001 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3162112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:05.482 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549863+0000", 2024-09-14T22:39:05.483 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 72.866260510999993, 2024-09-14T22:39:05.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083105534999999994, 2024-09-14T22:39:05.483 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:05.483 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:05.483 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:05.483 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:05.483 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:05.483 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:05.483 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145001 2024-09-14T22:39:05.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.483 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:05.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:05.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549863+0000", 2024-09-14T22:39:05.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.484 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.484 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:05.484 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549863+0000", 2024-09-14T22:39:05.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.484 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.484 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:05.484 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549864+0000", 2024-09-14T22:39:05.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4300000000000003e-07 2024-09-14T22:39:05.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.484 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.484 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:05.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549867+0000", 2024-09-14T22:39:05.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.757e-06 2024-09-14T22:39:05.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:05.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549878+0000", 2024-09-14T22:39:05.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1345999999999999e-05 2024-09-14T22:39:05.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:05.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549888+0000", 2024-09-14T22:39:05.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6460000000000005e-06 2024-09-14T22:39:05.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:05.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632754+0000", 2024-09-14T22:39:05.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.082866258999999998 2024-09-14T22:39:05.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:05.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632841+0000", 2024-09-14T22:39:05.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6971000000000005e-05 2024-09-14T22:39:05.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:05.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.632969+0000", 2024-09-14T22:39:05.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012791300000000001 2024-09-14T22:39:05.486 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.487 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:05.487 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.487 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145002 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3260416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:05.487 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549933+0000", 2024-09-14T22:39:05.487 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 72.866191008000001, 2024-09-14T22:39:05.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083291027000000004, 2024-09-14T22:39:05.487 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:05.487 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:05.487 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:05.487 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:05.487 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:05.487 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:05.488 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145002 2024-09-14T22:39:05.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.488 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:05.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.488 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:05.488 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549933+0000", 2024-09-14T22:39:05.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.488 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:05.488 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549933+0000", 2024-09-14T22:39:05.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:05.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549934+0000", 2024-09-14T22:39:05.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8700000000000004e-07 2024-09-14T22:39:05.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:05.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549938+0000", 2024-09-14T22:39:05.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5029999999999999e-06 2024-09-14T22:39:05.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:05.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549952+0000", 2024-09-14T22:39:05.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4382999999999999e-05 2024-09-14T22:39:05.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:05.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549963+0000", 2024-09-14T22:39:05.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0665999999999999e-05 2024-09-14T22:39:05.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:05.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633025+0000", 2024-09-14T22:39:05.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083061727000000002 2024-09-14T22:39:05.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:05.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633117+0000", 2024-09-14T22:39:05.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1812999999999997e-05 2024-09-14T22:39:05.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:05.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633224+0000", 2024-09-14T22:39:05.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107048 2024-09-14T22:39:05.491 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.491 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:05.491 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.491 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145003 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3457024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:05.491 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.549985+0000", 2024-09-14T22:39:05.491 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 72.866138590000006, 2024-09-14T22:39:05.492 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083465871999999997, 2024-09-14T22:39:05.492 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:05.492 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:05.492 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:05.492 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:05.492 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:05.492 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:05.492 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145003 2024-09-14T22:39:05.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.492 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:05.492 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.492 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:05.492 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549985+0000", 2024-09-14T22:39:05.493 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.493 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:05.493 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549985+0000", 2024-09-14T22:39:05.493 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.493 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:05.493 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549986+0000", 2024-09-14T22:39:05.493 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7299999999999995e-07 2024-09-14T22:39:05.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.493 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:05.494 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549989+0000", 2024-09-14T22:39:05.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8169999999999999e-06 2024-09-14T22:39:05.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.494 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:05.494 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.549998+0000", 2024-09-14T22:39:05.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4290000000000007e-06 2024-09-14T22:39:05.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.494 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:05.494 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550009+0000", 2024-09-14T22:39:05.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0633e-05 2024-09-14T22:39:05.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:05.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633275+0000", 2024-09-14T22:39:05.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083266397000000006 2024-09-14T22:39:05.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:05.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633353+0000", 2024-09-14T22:39:05.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8129999999999999e-05 2024-09-14T22:39:05.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:05.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633451+0000", 2024-09-14T22:39:05.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7793000000000004e-05 2024-09-14T22:39:05.495 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.496 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:05.496 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.496 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145004 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3358720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:05.496 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.550029+0000", 2024-09-14T22:39:05.496 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 72.866095018999999, 2024-09-14T22:39:05.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083662089999999995, 2024-09-14T22:39:05.496 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:05.496 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:05.496 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:05.496 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:05.496 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:05.497 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:05.497 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145004 2024-09-14T22:39:05.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.497 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:05.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:05.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550029+0000", 2024-09-14T22:39:05.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:05.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550029+0000", 2024-09-14T22:39:05.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.498 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:05.498 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550030+0000", 2024-09-14T22:39:05.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.33e-07 2024-09-14T22:39:05.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.498 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:05.498 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550035+0000", 2024-09-14T22:39:05.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3430000000000003e-06 2024-09-14T22:39:05.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.498 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:05.498 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550053+0000", 2024-09-14T22:39:05.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8485e-05 2024-09-14T22:39:05.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:05.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550064+0000", 2024-09-14T22:39:05.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0264e-05 2024-09-14T22:39:05.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:05.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633502+0000", 2024-09-14T22:39:05.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083438498999999999 2024-09-14T22:39:05.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:05.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633592+0000", 2024-09-14T22:39:05.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.022e-05 2024-09-14T22:39:05.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:05.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633691+0000", 2024-09-14T22:39:05.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8345999999999997e-05 2024-09-14T22:39:05.500 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.500 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:05.500 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.500 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145005 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3555328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:05.500 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.550077+0000", 2024-09-14T22:39:05.500 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 72.866046558999997, 2024-09-14T22:39:05.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083854340999999999, 2024-09-14T22:39:05.500 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:05.501 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:05.501 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:05.501 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:05.501 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:05.501 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:05.501 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145005 2024-09-14T22:39:05.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.501 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:05.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:05.501 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550077+0000", 2024-09-14T22:39:05.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:05.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550077+0000", 2024-09-14T22:39:05.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.502 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:05.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550078+0000", 2024-09-14T22:39:05.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8700000000000005e-07 2024-09-14T22:39:05.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.502 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:05.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550101+0000", 2024-09-14T22:39:05.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3632e-05 2024-09-14T22:39:05.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:05.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550115+0000", 2024-09-14T22:39:05.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3893000000000001e-05 2024-09-14T22:39:05.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:05.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550126+0000", 2024-09-14T22:39:05.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0426e-05 2024-09-14T22:39:05.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:05.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633742+0000", 2024-09-14T22:39:05.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083616633999999995 2024-09-14T22:39:05.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.504 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:05.504 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633816+0000", 2024-09-14T22:39:05.504 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3690999999999996e-05 2024-09-14T22:39:05.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.504 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:05.504 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633932+0000", 2024-09-14T22:39:05.504 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011537800000000001 2024-09-14T22:39:05.504 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.504 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:05.504 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.505 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4412.0:145006 3.1d 3:b8e318d1:::rbd_data.113ce98dded2.0000000000000001:24 [read 3653632~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-09-14T22:39:05.505 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-14T22:37:52.550236+0000", 2024-09-14T22:39:05.505 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 72.865888053000006, 2024-09-14T22:39:05.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083924027999999998, 2024-09-14T22:39:05.505 INFO:teuthology.orchestra.run.smithi001.stdout: "continuous": false, 2024-09-14T22:39:05.505 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-14T22:39:05.505 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-14T22:39:05.505 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-14T22:39:05.505 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4412", 2024-09-14T22:39:05.505 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4141996392", 2024-09-14T22:39:05.505 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 145006 2024-09-14T22:39:05.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.505 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-14T22:39:05.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-14T22:39:05.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550236+0000", 2024-09-14T22:39:05.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-14T22:39:05.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550236+0000", 2024-09-14T22:39:05.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-14T22:39:05.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-14T22:39:05.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550237+0000", 2024-09-14T22:39:05.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2929999999999999e-06 2024-09-14T22:39:05.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-14T22:39:05.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550243+0000", 2024-09-14T22:39:05.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0390000000000001e-06 2024-09-14T22:39:05.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-14T22:39:05.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550256+0000", 2024-09-14T22:39:05.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3393e-05 2024-09-14T22:39:05.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-14T22:39:05.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.550271+0000", 2024-09-14T22:39:05.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4166e-05 2024-09-14T22:39:05.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-14T22:39:05.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.633974+0000", 2024-09-14T22:39:05.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.083703825999999995 2024-09-14T22:39:05.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-14T22:39:05.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.634047+0000", 2024-09-14T22:39:05.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2935999999999994e-05 2024-09-14T22:39:05.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-14T22:39:05.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-14T22:39:05.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-14T22:39:05.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-14T22:37:52.634160+0000", 2024-09-14T22:39:05.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011237499999999999 2024-09-14T22:39:05.509 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.509 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:05.509 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.509 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-14T22:39:05.509 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-14T22:39:05.509 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:39:05.792 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2024-09-14T22:39:05.816 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T22:39:06.349 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:39:06.342+0000 7f128cca17c0 -1 Falling back to public interface 2024-09-14T22:39:06.809 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2024-09-14T22:39:10.385 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:39:10.397 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:39:10.390+0000 7f128cca17c0 -1 osd.4 158 log_to_monitors true 2024-09-14T22:39:10.499 INFO:teuthology.orchestra.run.smithi176.stderr:no valid command found; 10 closest matches: 2024-09-14T22:39:10.499 INFO:teuthology.orchestra.run.smithi176.stderr:0 2024-09-14T22:39:10.499 INFO:teuthology.orchestra.run.smithi176.stderr:1 2024-09-14T22:39:10.499 INFO:teuthology.orchestra.run.smithi176.stderr:2 2024-09-14T22:39:10.499 INFO:teuthology.orchestra.run.smithi176.stderr:abort 2024-09-14T22:39:10.499 INFO:teuthology.orchestra.run.smithi176.stderr:assert 2024-09-14T22:39:10.499 INFO:teuthology.orchestra.run.smithi176.stderr:bluefs debug_inject_read_zeros 2024-09-14T22:39:10.499 INFO:teuthology.orchestra.run.smithi176.stderr:bluefs files list 2024-09-14T22:39:10.500 INFO:teuthology.orchestra.run.smithi176.stderr:bluefs stats 2024-09-14T22:39:10.500 INFO:teuthology.orchestra.run.smithi176.stderr:bluestore allocator dump block 2024-09-14T22:39:10.500 INFO:teuthology.orchestra.run.smithi176.stderr:bluestore allocator fragmentation block 2024-09-14T22:39:10.500 INFO:teuthology.orchestra.run.smithi176.stderr:admin_socket: invalid command 2024-09-14T22:39:10.502 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T22:39:10.502 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-4, ['dump_ops_in_flight'] 2024-09-14T22:39:11.021 INFO:tasks.ceph.osd.4.smithi176.stderr:2024-09-14T22:39:11.014+0000 7f1280a7f640 -1 osd.4 173 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T22:39:15.504 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:39:15.636 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T22:39:15.636 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T22:39:15.636 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T22:39:15.637 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T22:39:15.651 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-14T22:39:15.802 INFO:teuthology.orchestra.run.smithi001.stdout:mon.a: {} 2024-09-14T22:39:15.802 INFO:teuthology.orchestra.run.smithi001.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T22:39:15.815 INFO:teuthology.orchestra.run.smithi001.stdout:mon.b: {} 2024-09-14T22:39:15.816 INFO:teuthology.orchestra.run.smithi001.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T22:39:15.825 INFO:teuthology.orchestra.run.smithi001.stdout:mon.c: {} 2024-09-14T22:39:15.825 INFO:teuthology.orchestra.run.smithi001.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T22:39:15.843 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-14T22:39:17.015 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.0 to 1 (10000) 2024-09-14T22:39:17.040 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-14T22:39:18.022 INFO:teuthology.orchestra.run.smithi001.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-14T22:39:18.049 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-14T22:39:19.020 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.1 to 1 (10000) 2024-09-14T22:39:19.046 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-14T22:39:19.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-14T22:39:19.380+0000 7ff352d9d640 0 --2- 172.21.15.1:0/546377205 >> [v2:172.21.15.1:3301/0,v1:172.21.15.1:6790/0] conn(0x7ff35413d220 0x7ff3541403b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-14T22:39:20.030 INFO:teuthology.orchestra.run.smithi001.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-14T22:39:20.062 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-14T22:39:20.740 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.3 to 1 (10000) 2024-09-14T22:39:20.769 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-14T22:39:22.041 INFO:teuthology.orchestra.run.smithi001.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-14T22:39:22.067 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-14T22:39:23.051 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.5 to 1 (10000) 2024-09-14T22:39:23.081 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-14T22:39:24.068 INFO:teuthology.orchestra.run.smithi001.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-14T22:39:24.092 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-14T22:39:25.072 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.6 to 1 (10000) 2024-09-14T22:39:25.100 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-14T22:39:26.087 INFO:teuthology.orchestra.run.smithi001.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-14T22:39:26.111 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-14T22:39:27.091 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.7 to 1 (10000) 2024-09-14T22:39:27.118 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-14T22:39:28.108 INFO:teuthology.orchestra.run.smithi001.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-14T22:39:28.134 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-14T22:39:29.112 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.2 to 1 (10000) 2024-09-14T22:39:29.141 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-14T22:39:30.127 INFO:teuthology.orchestra.run.smithi001.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-14T22:39:30.151 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-14T22:39:31.137 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.4 to 1 (10000) 2024-09-14T22:39:31.162 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-14T22:39:32.146 INFO:teuthology.orchestra.run.smithi001.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-14T22:39:32.173 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-14T22:39:32.173 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-14T22:39:32.174 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-14T22:39:32.174 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-14T22:39:32.174 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-14T22:39:32.174 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T22:39:32.471 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:39:32.471 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":191,"fsid":"f07f407a-b17d-42c4-bf73-92dd80114a4f","created":"2024-09-14T22:35:45.983055+0000","modified":"2024-09-14T22:39:32.137370+0000","last_up_change":"2024-09-14T22:39:11.976048+0000","last_in_change":"2024-09-14T22:35:48.283160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T22:35:58.938784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T22:36:04.304242+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-14T22:36:14.374249+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":44,"snap_epoch":170,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9831fcc5-9040-4a80-99d4-838225e65139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1702929600},{"type":"v1","addr":"172.21.15.1:6825","nonce":1702929600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1702929600},{"type":"v1","addr":"172.21.15.1:6827","nonce":1702929600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":1702929600},{"type":"v1","addr":"172.21.15.1:6831","nonce":1702929600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":1702929600},{"type":"v1","addr":"172.21.15.1:6829","nonce":1702929600}]},"public_addr":"172.21.15.1:6825/1702929600","cluster_addr":"172.21.15.1:6827/1702929600","heartbeat_back_addr":"172.21.15.1:6831/1702929600","heartbeat_front_addr":"172.21.15.1:6829/1702929600","state":["exists","up"]},{"osd":1,"uuid":"7f79abe8-115d-46b2-8677-72fcb0c8ab7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":719990653},{"type":"v1","addr":"172.21.15.1:6809","nonce":719990653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":719990653},{"type":"v1","addr":"172.21.15.1:6811","nonce":719990653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":719990653},{"type":"v1","addr":"172.21.15.1:6815","nonce":719990653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":719990653},{"type":"v1","addr":"172.21.15.1:6813","nonce":719990653}]},"public_addr":"172.21.15.1:6809/719990653","cluster_addr":"172.21.15.1:6811/719990653","heartbeat_back_addr":"172.21.15.1:6815/719990653","heartbeat_front_addr":"172.21.15.1:6813/719990653","state":["exists","up"]},{"osd":2,"uuid":"1f5768b2-4db4-4b2e-8d5e-5206f8bde28d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":30,"up_from":66,"up_thru":159,"down_at":31,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1006730601},{"type":"v1","addr":"172.21.15.1:6817","nonce":1006730601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1006730601},{"type":"v1","addr":"172.21.15.1:6819","nonce":1006730601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1006730601},{"type":"v1","addr":"172.21.15.1:6823","nonce":1006730601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1006730601},{"type":"v1","addr":"172.21.15.1:6821","nonce":1006730601}]},"public_addr":"172.21.15.1:6817/1006730601","cluster_addr":"172.21.15.1:6819/1006730601","heartbeat_back_addr":"172.21.15.1:6823/1006730601","heartbeat_front_addr":"172.21.15.1:6821/1006730601","state":["exists","up"]},{"osd":3,"uuid":"8103f3c7-7899-4fc3-bb02-dd2273120138","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":750403671},{"type":"v1","addr":"172.21.15.1:6801","nonce":750403671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":750403671},{"type":"v1","addr":"172.21.15.1:6803","nonce":750403671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":750403671},{"type":"v1","addr":"172.21.15.1:6807","nonce":750403671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":750403671},{"type":"v1","addr":"172.21.15.1:6805","nonce":750403671}]},"public_addr":"172.21.15.1:6801/750403671","cluster_addr":"172.21.15.1:6803/750403671","heartbeat_back_addr":"172.21.15.1:6807/750403671","heartbeat_front_addr":"172.21.15.1:6805/750403671","state":["exists","up"]},{"osd":4,"uuid":"23cce5cb-726e-44c2-8edb-c2567a89dfbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":158,"up_from":174,"up_thru":174,"down_at":159,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":3208688125},{"type":"v1","addr":"172.21.15.176:6817","nonce":3208688125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":3208688125},{"type":"v1","addr":"172.21.15.176:6819","nonce":3208688125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":3208688125},{"type":"v1","addr":"172.21.15.176:6823","nonce":3208688125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":3208688125},{"type":"v1","addr":"172.21.15.176:6821","nonce":3208688125}]},"public_addr":"172.21.15.176:6817/3208688125","cluster_addr":"172.21.15.176:6819/3208688125","heartbeat_back_addr":"172.21.15.176:6823/3208688125","heartbeat_front_addr":"172.21.15.176:6821/3208688125","state":["exists","up"]},{"osd":5,"uuid":"eb2f187b-d2f1-49ab-ad5e-ea6dc441a28e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6805","nonce":1945902326},{"type":"v1","addr":"172.21.15.176:6808","nonce":1945902326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":1945902326},{"type":"v1","addr":"172.21.15.176:6811","nonce":1945902326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":1945902326},{"type":"v1","addr":"172.21.15.176:6815","nonce":1945902326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":1945902326},{"type":"v1","addr":"172.21.15.176:6813","nonce":1945902326}]},"public_addr":"172.21.15.176:6808/1945902326","cluster_addr":"172.21.15.176:6811/1945902326","heartbeat_back_addr":"172.21.15.176:6815/1945902326","heartbeat_front_addr":"172.21.15.176:6813/1945902326","state":["exists","up"]},{"osd":6,"uuid":"d371aed8-2ad2-49a5-8ece-4b7fd37b4822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":4116085586},{"type":"v1","addr":"172.21.15.176:6801","nonce":4116085586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":4116085586},{"type":"v1","addr":"172.21.15.176:6803","nonce":4116085586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6807","nonce":4116085586},{"type":"v1","addr":"172.21.15.176:6809","nonce":4116085586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":4116085586},{"type":"v1","addr":"172.21.15.176:6806","nonce":4116085586}]},"public_addr":"172.21.15.176:6801/4116085586","cluster_addr":"172.21.15.176:6803/4116085586","heartbeat_back_addr":"172.21.15.176:6809/4116085586","heartbeat_front_addr":"172.21.15.176:6806/4116085586","state":["exists","up"]},{"osd":7,"uuid":"a012c819-8d49-4310-a840-c2752d9314ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":877760244},{"type":"v1","addr":"172.21.15.176:6825","nonce":877760244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":877760244},{"type":"v1","addr":"172.21.15.176:6827","nonce":877760244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":877760244},{"type":"v1","addr":"172.21.15.176:6831","nonce":877760244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":877760244},{"type":"v1","addr":"172.21.15.176:6829","nonce":877760244}]},"public_addr":"172.21.15.176:6825/877760244","cluster_addr":"172.21.15.176:6827/877760244","heartbeat_back_addr":"172.21.15.176:6831/877760244","heartbeat_front_addr":"172.21.15.176:6829/877760244","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:55.511688+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.742414+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.871355+0000","dead_epoch":31},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.504553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.589640+0000","dead_epoch":159},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.518845+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.731632+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.952634+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.0","osds":[4,2]},{"pgid":"3.1d","osds":[1,2]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T22:39:32.490 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-14T22:39:32.490 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T22:39:32.800 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:39:32.800 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":191,"fsid":"f07f407a-b17d-42c4-bf73-92dd80114a4f","created":"2024-09-14T22:35:45.983055+0000","modified":"2024-09-14T22:39:32.137370+0000","last_up_change":"2024-09-14T22:39:11.976048+0000","last_in_change":"2024-09-14T22:35:48.283160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T22:35:58.938784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T22:36:04.304242+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-14T22:36:14.374249+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":44,"snap_epoch":170,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9831fcc5-9040-4a80-99d4-838225e65139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1702929600},{"type":"v1","addr":"172.21.15.1:6825","nonce":1702929600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1702929600},{"type":"v1","addr":"172.21.15.1:6827","nonce":1702929600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":1702929600},{"type":"v1","addr":"172.21.15.1:6831","nonce":1702929600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":1702929600},{"type":"v1","addr":"172.21.15.1:6829","nonce":1702929600}]},"public_addr":"172.21.15.1:6825/1702929600","cluster_addr":"172.21.15.1:6827/1702929600","heartbeat_back_addr":"172.21.15.1:6831/1702929600","heartbeat_front_addr":"172.21.15.1:6829/1702929600","state":["exists","up"]},{"osd":1,"uuid":"7f79abe8-115d-46b2-8677-72fcb0c8ab7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":719990653},{"type":"v1","addr":"172.21.15.1:6809","nonce":719990653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":719990653},{"type":"v1","addr":"172.21.15.1:6811","nonce":719990653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":719990653},{"type":"v1","addr":"172.21.15.1:6815","nonce":719990653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":719990653},{"type":"v1","addr":"172.21.15.1:6813","nonce":719990653}]},"public_addr":"172.21.15.1:6809/719990653","cluster_addr":"172.21.15.1:6811/719990653","heartbeat_back_addr":"172.21.15.1:6815/719990653","heartbeat_front_addr":"172.21.15.1:6813/719990653","state":["exists","up"]},{"osd":2,"uuid":"1f5768b2-4db4-4b2e-8d5e-5206f8bde28d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":30,"up_from":66,"up_thru":159,"down_at":31,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1006730601},{"type":"v1","addr":"172.21.15.1:6817","nonce":1006730601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1006730601},{"type":"v1","addr":"172.21.15.1:6819","nonce":1006730601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1006730601},{"type":"v1","addr":"172.21.15.1:6823","nonce":1006730601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1006730601},{"type":"v1","addr":"172.21.15.1:6821","nonce":1006730601}]},"public_addr":"172.21.15.1:6817/1006730601","cluster_addr":"172.21.15.1:6819/1006730601","heartbeat_back_addr":"172.21.15.1:6823/1006730601","heartbeat_front_addr":"172.21.15.1:6821/1006730601","state":["exists","up"]},{"osd":3,"uuid":"8103f3c7-7899-4fc3-bb02-dd2273120138","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":750403671},{"type":"v1","addr":"172.21.15.1:6801","nonce":750403671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":750403671},{"type":"v1","addr":"172.21.15.1:6803","nonce":750403671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":750403671},{"type":"v1","addr":"172.21.15.1:6807","nonce":750403671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":750403671},{"type":"v1","addr":"172.21.15.1:6805","nonce":750403671}]},"public_addr":"172.21.15.1:6801/750403671","cluster_addr":"172.21.15.1:6803/750403671","heartbeat_back_addr":"172.21.15.1:6807/750403671","heartbeat_front_addr":"172.21.15.1:6805/750403671","state":["exists","up"]},{"osd":4,"uuid":"23cce5cb-726e-44c2-8edb-c2567a89dfbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":158,"up_from":174,"up_thru":174,"down_at":159,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":3208688125},{"type":"v1","addr":"172.21.15.176:6817","nonce":3208688125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":3208688125},{"type":"v1","addr":"172.21.15.176:6819","nonce":3208688125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":3208688125},{"type":"v1","addr":"172.21.15.176:6823","nonce":3208688125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":3208688125},{"type":"v1","addr":"172.21.15.176:6821","nonce":3208688125}]},"public_addr":"172.21.15.176:6817/3208688125","cluster_addr":"172.21.15.176:6819/3208688125","heartbeat_back_addr":"172.21.15.176:6823/3208688125","heartbeat_front_addr":"172.21.15.176:6821/3208688125","state":["exists","up"]},{"osd":5,"uuid":"eb2f187b-d2f1-49ab-ad5e-ea6dc441a28e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6805","nonce":1945902326},{"type":"v1","addr":"172.21.15.176:6808","nonce":1945902326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":1945902326},{"type":"v1","addr":"172.21.15.176:6811","nonce":1945902326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":1945902326},{"type":"v1","addr":"172.21.15.176:6815","nonce":1945902326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":1945902326},{"type":"v1","addr":"172.21.15.176:6813","nonce":1945902326}]},"public_addr":"172.21.15.176:6808/1945902326","cluster_addr":"172.21.15.176:6811/1945902326","heartbeat_back_addr":"172.21.15.176:6815/1945902326","heartbeat_front_addr":"172.21.15.176:6813/1945902326","state":["exists","up"]},{"osd":6,"uuid":"d371aed8-2ad2-49a5-8ece-4b7fd37b4822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":4116085586},{"type":"v1","addr":"172.21.15.176:6801","nonce":4116085586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":4116085586},{"type":"v1","addr":"172.21.15.176:6803","nonce":4116085586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6807","nonce":4116085586},{"type":"v1","addr":"172.21.15.176:6809","nonce":4116085586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":4116085586},{"type":"v1","addr":"172.21.15.176:6806","nonce":4116085586}]},"public_addr":"172.21.15.176:6801/4116085586","cluster_addr":"172.21.15.176:6803/4116085586","heartbeat_back_addr":"172.21.15.176:6809/4116085586","heartbeat_front_addr":"172.21.15.176:6806/4116085586","state":["exists","up"]},{"osd":7,"uuid":"a012c819-8d49-4310-a840-c2752d9314ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":877760244},{"type":"v1","addr":"172.21.15.176:6825","nonce":877760244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":877760244},{"type":"v1","addr":"172.21.15.176:6827","nonce":877760244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":877760244},{"type":"v1","addr":"172.21.15.176:6831","nonce":877760244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":877760244},{"type":"v1","addr":"172.21.15.176:6829","nonce":877760244}]},"public_addr":"172.21.15.176:6825/877760244","cluster_addr":"172.21.15.176:6827/877760244","heartbeat_back_addr":"172.21.15.176:6831/877760244","heartbeat_front_addr":"172.21.15.176:6829/877760244","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:55.511688+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.742414+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.871355+0000","dead_epoch":31},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.504553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.589640+0000","dead_epoch":159},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.518845+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.731632+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.952634+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.0","osds":[4,2]},{"pgid":"3.1d","osds":[1,2]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T22:39:32.820 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-14T22:39:32.821 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-14T22:39:32.821 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-14T22:39:32.821 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-14T22:39:32.822 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-14T22:39:32.822 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-14T22:39:32.823 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-14T22:39:32.823 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-14T22:39:33.090 INFO:teuthology.orchestra.run.smithi001.stdout:60129542191 2024-09-14T22:39:33.090 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T22:39:33.096 INFO:teuthology.orchestra.run.smithi001.stdout:60129542192 2024-09-14T22:39:33.096 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T22:39:33.097 INFO:teuthology.orchestra.run.smithi001.stdout:55834574896 2024-09-14T22:39:33.097 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T22:39:33.098 INFO:teuthology.orchestra.run.smithi001.stdout:283467841566 2024-09-14T22:39:33.099 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T22:39:33.100 INFO:teuthology.orchestra.run.smithi001.stdout:60129542192 2024-09-14T22:39:33.100 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T22:39:33.101 INFO:teuthology.orchestra.run.smithi001.stdout:747324309510 2024-09-14T22:39:33.102 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T22:39:33.105 INFO:teuthology.orchestra.run.smithi001.stdout:60129542192 2024-09-14T22:39:33.105 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-14T22:39:33.107 INFO:teuthology.orchestra.run.smithi001.stdout:60129542191 2024-09-14T22:39:33.107 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-14T22:39:33.610 INFO:teuthology.orchestra.run.smithi001.stdout:60129542191 2024-09-14T22:39:33.625 INFO:teuthology.orchestra.run.smithi001.stdout:60129542189 2024-09-14T22:39:33.630 INFO:teuthology.orchestra.run.smithi001.stdout:283467841565 2024-09-14T22:39:33.637 INFO:teuthology.orchestra.run.smithi001.stdout:60129542190 2024-09-14T22:39:33.638 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542192 got 60129542191 for osd.5 2024-09-14T22:39:33.643 INFO:teuthology.orchestra.run.smithi001.stdout:747324309509 2024-09-14T22:39:33.646 INFO:teuthology.orchestra.run.smithi001.stdout:55834574895 2024-09-14T22:39:33.658 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542191 got 60129542189 for osd.0 2024-09-14T22:39:33.661 INFO:tasks.ceph.ceph_manager.ceph:need seq 283467841566 got 283467841565 for osd.2 2024-09-14T22:39:33.669 INFO:teuthology.orchestra.run.smithi001.stdout:60129542191 2024-09-14T22:39:33.674 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542192 got 60129542190 for osd.3 2024-09-14T22:39:33.678 INFO:tasks.ceph.ceph_manager.ceph:need seq 747324309510 got 747324309509 for osd.4 2024-09-14T22:39:33.681 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574896 got 55834574895 for osd.1 2024-09-14T22:39:33.688 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542192 got 60129542191 for osd.6 2024-09-14T22:39:33.694 INFO:teuthology.orchestra.run.smithi001.stdout:60129542189 2024-09-14T22:39:33.711 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542191 got 60129542189 for osd.7 2024-09-14T22:39:34.639 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T22:39:34.658 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T22:39:34.661 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T22:39:34.675 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T22:39:34.679 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T22:39:34.682 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T22:39:34.690 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-14T22:39:34.712 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-14T22:39:35.177 INFO:teuthology.orchestra.run.smithi001.stdout:60129542192 2024-09-14T22:39:35.190 INFO:teuthology.orchestra.run.smithi001.stdout:283467841566 2024-09-14T22:39:35.202 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542192 got 60129542192 for osd.5 2024-09-14T22:39:35.202 DEBUG:teuthology.parallel:result is None 2024-09-14T22:39:35.218 INFO:tasks.ceph.ceph_manager.ceph:need seq 283467841566 got 283467841566 for osd.2 2024-09-14T22:39:35.218 DEBUG:teuthology.parallel:result is None 2024-09-14T22:39:35.220 INFO:teuthology.orchestra.run.smithi001.stdout:55834574896 2024-09-14T22:39:35.221 INFO:teuthology.orchestra.run.smithi001.stdout:60129542191 2024-09-14T22:39:35.234 INFO:teuthology.orchestra.run.smithi001.stdout:747324309510 2024-09-14T22:39:35.251 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574896 got 55834574896 for osd.1 2024-09-14T22:39:35.252 INFO:teuthology.orchestra.run.smithi001.stdout:60129542192 2024-09-14T22:39:35.252 DEBUG:teuthology.parallel:result is None 2024-09-14T22:39:35.252 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542191 got 60129542191 for osd.0 2024-09-14T22:39:35.252 DEBUG:teuthology.parallel:result is None 2024-09-14T22:39:35.255 INFO:tasks.ceph.ceph_manager.ceph:need seq 747324309510 got 747324309510 for osd.4 2024-09-14T22:39:35.255 DEBUG:teuthology.parallel:result is None 2024-09-14T22:39:35.259 INFO:teuthology.orchestra.run.smithi001.stdout:60129542192 2024-09-14T22:39:35.269 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542192 got 60129542192 for osd.6 2024-09-14T22:39:35.270 DEBUG:teuthology.parallel:result is None 2024-09-14T22:39:35.276 INFO:teuthology.orchestra.run.smithi001.stdout:60129542191 2024-09-14T22:39:35.285 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542192 got 60129542192 for osd.3 2024-09-14T22:39:35.285 DEBUG:teuthology.parallel:result is None 2024-09-14T22:39:35.295 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542191 got 60129542191 for osd.7 2024-09-14T22:39:35.296 DEBUG:teuthology.parallel:result is None 2024-09-14T22:39:35.296 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-14T22:39:35.296 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:39:35.630 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:39:35.631 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-14T22:39:35.647 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-14T22:39:34.919415+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":897,"num_object_clones":0,"num_object_copies":1794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":897,"num_whiteouts":889,"num_read":206802,"num_read_kb":829088,"num_write":25375,"num_write_kb":384255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3336,"num_bytes_recovered":173602381,"num_keys_recovered":342,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21693,"ondisk_log_size":21693,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":389312,"kb_used_data":18064,"kb_used_omap":50,"kb_used_meta":371021,"kb_avail":754585408,"statfs":{"total":773094113280,"available":772695457792,"internally_reserved":0,"allocated":18497536,"data_stored":10029432,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":51845,"internal_metadata":379925883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.807219"},"pg_stats":[{"pgid":"3.1f","version":"170'491","reported_seq":5322,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306474+0000","last_change":"2024-09-14T22:39:02.783678+0000","last_active":"2024-09-14T22:39:32.306474+0000","last_peered":"2024-09-14T22:39:32.306474+0000","last_clean":"2024-09-14T22:39:32.306474+0000","last_became_active":"2024-09-14T22:37:25.718906+0000","last_became_peered":"2024-09-14T22:37:25.718906+0000","last_unstale":"2024-09-14T22:39:32.306474+0000","last_undegraded":"2024-09-14T22:39:32.306474+0000","last_fullsized":"2024-09-14T22:39:32.306474+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:16:07.317648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023103960000000002,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":5019,"num_read_kb":16516,"num_write":439,"num_write_kb":7121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":3277632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"170'1231","reported_seq":7362,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.472426+0000","last_change":"2024-09-14T22:39:02.771231+0000","last_active":"2024-09-14T22:39:31.472426+0000","last_peered":"2024-09-14T22:39:31.472426+0000","last_clean":"2024-09-14T22:39:31.472426+0000","last_became_active":"2024-09-14T22:38:08.741149+0000","last_became_peered":"2024-09-14T22:38:08.741149+0000","last_unstale":"2024-09-14T22:39:31.472426+0000","last_undegraded":"2024-09-14T22:39:31.472426+0000","last_fullsized":"2024-09-14T22:39:31.472426+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1231,"log_dups_size":0,"ondisk_log_size":1231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:32:08.194278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024840370000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":11835,"num_read_kb":34663,"num_write":1722,"num_write_kb":10763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":7412288,"num_keys_recovered":90,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"168'724","reported_seq":10563,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149517+0000","last_change":"2024-09-14T22:39:20.753911+0000","last_active":"2024-09-14T22:39:32.149517+0000","last_peered":"2024-09-14T22:39:32.149517+0000","last_clean":"2024-09-14T22:39:32.149517+0000","last_became_active":"2024-09-14T22:39:20.751505+0000","last_became_peered":"2024-09-14T22:39:20.751505+0000","last_unstale":"2024-09-14T22:39:32.149517+0000","last_undegraded":"2024-09-14T22:39:32.149517+0000","last_fullsized":"2024-09-14T22:39:32.149517+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:17:01.639166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0083529999999999993,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":10932,"num_read_kb":63225,"num_write":868,"num_write_kb":13703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":8030815,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"175'718","reported_seq":6436,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298203+0000","last_change":"2024-09-14T22:39:25.087327+0000","last_active":"2024-09-14T22:39:32.298203+0000","last_peered":"2024-09-14T22:39:32.298203+0000","last_clean":"2024-09-14T22:39:32.298203+0000","last_became_active":"2024-09-14T22:39:25.084122+0000","last_became_peered":"2024-09-14T22:39:25.084122+0000","last_unstale":"2024-09-14T22:39:32.298203+0000","last_undegraded":"2024-09-14T22:39:32.298203+0000","last_fullsized":"2024-09-14T22:39:32.298203+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:17:17.035425+0000","scrub_duration":0,"objects_trimmed":20,"snaptrim_duration":0.055023557000000001,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":34,"num_read":6435,"num_read_kb":26956,"num_write":884,"num_write_kb":8982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"175'602","reported_seq":5586,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306492+0000","last_change":"2024-09-14T22:39:13.115776+0000","last_active":"2024-09-14T22:39:32.306492+0000","last_peered":"2024-09-14T22:39:32.306492+0000","last_clean":"2024-09-14T22:39:32.306492+0000","last_became_active":"2024-09-14T22:39:13.025036+0000","last_became_peered":"2024-09-14T22:39:13.025036+0000","last_unstale":"2024-09-14T22:39:32.306492+0000","last_undegraded":"2024-09-14T22:39:32.306492+0000","last_fullsized":"2024-09-14T22:39:32.306492+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:23:31.541475+0000","scrub_duration":0,"objects_trimmed":14,"snaptrim_duration":0.061855755999999998,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":5663,"num_read_kb":32434,"num_write":648,"num_write_kb":13330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":7858104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"175'527","reported_seq":4225,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149467+0000","last_change":"2024-09-14T22:39:13.150950+0000","last_active":"2024-09-14T22:39:32.149467+0000","last_peered":"2024-09-14T22:39:32.149467+0000","last_clean":"2024-09-14T22:39:32.149467+0000","last_became_active":"2024-09-14T22:39:13.016132+0000","last_became_peered":"2024-09-14T22:39:13.016132+0000","last_unstale":"2024-09-14T22:39:32.149467+0000","last_undegraded":"2024-09-14T22:39:32.149467+0000","last_fullsized":"2024-09-14T22:39:32.149467+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:34:37.241298+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.069063225000000006,"stat_sum":{"num_bytes":19,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":33,"num_read":4551,"num_read_kb":13173,"num_write":474,"num_write_kb":7035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":236,"num_bytes_recovered":10036538,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"169'465","reported_seq":5658,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149161+0000","last_change":"2024-09-14T22:39:02.778427+0000","last_active":"2024-09-14T22:39:32.149161+0000","last_peered":"2024-09-14T22:39:32.149161+0000","last_clean":"2024-09-14T22:39:32.149161+0000","last_became_active":"2024-09-14T22:36:15.700990+0000","last_became_peered":"2024-09-14T22:36:15.700990+0000","last_unstale":"2024-09-14T22:39:32.149161+0000","last_undegraded":"2024-09-14T22:39:32.149161+0000","last_fullsized":"2024-09-14T22:39:32.149161+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:24:53.327956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013968780000000001,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":5273,"num_read_kb":15380,"num_write":425,"num_write_kb":5486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"170'511","reported_seq":4995,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.472029+0000","last_change":"2024-09-14T22:39:02.769340+0000","last_active":"2024-09-14T22:39:31.472029+0000","last_peered":"2024-09-14T22:39:31.472029+0000","last_clean":"2024-09-14T22:39:31.472029+0000","last_became_active":"2024-09-14T22:38:07.734308+0000","last_became_peered":"2024-09-14T22:38:07.734308+0000","last_unstale":"2024-09-14T22:39:31.472029+0000","last_undegraded":"2024-09-14T22:39:31.472029+0000","last_fullsized":"2024-09-14T22:39:31.472029+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:12:44.546869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013730859999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4941,"num_read_kb":35923,"num_write":636,"num_write_kb":27736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":20067293,"num_keys_recovered":21,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"167'467","reported_seq":4865,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306510+0000","last_change":"2024-09-14T22:39:02.784093+0000","last_active":"2024-09-14T22:39:32.306510+0000","last_peered":"2024-09-14T22:39:32.306510+0000","last_clean":"2024-09-14T22:39:32.306510+0000","last_became_active":"2024-09-14T22:36:15.715869+0000","last_became_peered":"2024-09-14T22:36:15.715869+0000","last_unstale":"2024-09-14T22:39:32.306510+0000","last_undegraded":"2024-09-14T22:39:32.306510+0000","last_fullsized":"2024-09-14T22:39:32.306510+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:57:02.149610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023931830000000001,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":4593,"num_read_kb":25142,"num_write":516,"num_write_kb":22003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"166'605","reported_seq":6669,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298379+0000","last_change":"2024-09-14T22:39:02.774376+0000","last_active":"2024-09-14T22:39:32.298379+0000","last_peered":"2024-09-14T22:39:32.298379+0000","last_clean":"2024-09-14T22:39:32.298379+0000","last_became_active":"2024-09-14T22:36:15.725078+0000","last_became_peered":"2024-09-14T22:36:15.725078+0000","last_unstale":"2024-09-14T22:39:32.298379+0000","last_undegraded":"2024-09-14T22:39:32.298379+0000","last_fullsized":"2024-09-14T22:39:32.298379+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:26:12.926170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035779840000000002,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":6675,"num_read_kb":51224,"num_write":729,"num_write_kb":18952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"167'467","reported_seq":4804,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776443+0000","last_change":"2024-09-14T22:39:02.867529+0000","last_active":"2024-09-14T22:39:31.776443+0000","last_peered":"2024-09-14T22:39:31.776443+0000","last_clean":"2024-09-14T22:39:31.776443+0000","last_became_active":"2024-09-14T22:36:15.717341+0000","last_became_peered":"2024-09-14T22:36:15.717341+0000","last_unstale":"2024-09-14T22:39:31.776443+0000","last_undegraded":"2024-09-14T22:39:31.776443+0000","last_fullsized":"2024-09-14T22:39:31.776443+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:29:47.296331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016388500000000001,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":5090,"num_read_kb":12789,"num_write":419,"num_write_kb":7375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"175'1202","reported_seq":9124,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149855+0000","last_change":"2024-09-14T22:39:13.160248+0000","last_active":"2024-09-14T22:39:32.149855+0000","last_peered":"2024-09-14T22:39:32.149855+0000","last_clean":"2024-09-14T22:39:32.149855+0000","last_became_active":"2024-09-14T22:39:13.026842+0000","last_became_peered":"2024-09-14T22:39:13.026842+0000","last_unstale":"2024-09-14T22:39:32.149855+0000","last_undegraded":"2024-09-14T22:39:32.149855+0000","last_fullsized":"2024-09-14T22:39:32.149855+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1202,"log_dups_size":0,"ondisk_log_size":1202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:06:37.741547+0000","scrub_duration":0,"objects_trimmed":14,"snaptrim_duration":0.055942110000000003,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":11247,"num_read_kb":29618,"num_write":1623,"num_write_kb":11950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":265,"num_bytes_recovered":17498216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":354,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.247655+0000","last_change":"2024-09-14T22:36:08.042860+0000","last_active":"2024-09-14T22:39:31.247655+0000","last_peered":"2024-09-14T22:39:31.247655+0000","last_clean":"2024-09-14T22:39:31.247655+0000","last_became_active":"2024-09-14T22:36:06.040961+0000","last_became_peered":"2024-09-14T22:36:06.040961+0000","last_unstale":"2024-09-14T22:39:31.247655+0000","last_undegraded":"2024-09-14T22:39:31.247655+0000","last_fullsized":"2024-09-14T22:39:31.247655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:03:48.547834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010780950000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"170'738","reported_seq":3855,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306469+0000","last_change":"2024-09-14T22:39:02.782844+0000","last_active":"2024-09-14T22:39:32.306469+0000","last_peered":"2024-09-14T22:39:32.306469+0000","last_clean":"2024-09-14T22:39:32.306469+0000","last_became_active":"2024-09-14T22:36:15.700982+0000","last_became_peered":"2024-09-14T22:36:15.700982+0000","last_unstale":"2024-09-14T22:39:32.306469+0000","last_undegraded":"2024-09-14T22:39:32.306469+0000","last_fullsized":"2024-09-14T22:39:32.306469+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:06:43.937383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001672663,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":4804,"num_read_kb":22802,"num_write":686,"num_write_kb":12175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":181,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149492+0000","last_change":"2024-09-14T22:39:20.885509+0000","last_active":"2024-09-14T22:39:32.149492+0000","last_peered":"2024-09-14T22:39:32.149492+0000","last_clean":"2024-09-14T22:39:32.149492+0000","last_became_active":"2024-09-14T22:39:20.885045+0000","last_became_peered":"2024-09-14T22:39:20.885045+0000","last_unstale":"2024-09-14T22:39:32.149492+0000","last_undegraded":"2024-09-14T22:39:32.149492+0000","last_fullsized":"2024-09-14T22:39:32.149492+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:37:23.145835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"170'537","reported_seq":5821,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.472368+0000","last_change":"2024-09-14T22:39:23.066067+0000","last_active":"2024-09-14T22:39:31.472368+0000","last_peered":"2024-09-14T22:39:31.472368+0000","last_clean":"2024-09-14T22:39:31.472368+0000","last_became_active":"2024-09-14T22:39:23.064710+0000","last_became_peered":"2024-09-14T22:39:23.064710+0000","last_unstale":"2024-09-14T22:39:31.472368+0000","last_undegraded":"2024-09-14T22:39:31.472368+0000","last_fullsized":"2024-09-14T22:39:31.472368+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:53:48.607465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044565630000000002,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":6110,"num_read_kb":28977,"num_write":556,"num_write_kb":8495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":356,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776747+0000","last_change":"2024-09-14T22:36:08.054606+0000","last_active":"2024-09-14T22:39:31.776747+0000","last_peered":"2024-09-14T22:39:31.776747+0000","last_clean":"2024-09-14T22:39:31.776747+0000","last_became_active":"2024-09-14T22:36:06.266908+0000","last_became_peered":"2024-09-14T22:36:06.266908+0000","last_unstale":"2024-09-14T22:39:31.776747+0000","last_undegraded":"2024-09-14T22:39:31.776747+0000","last_fullsized":"2024-09-14T22:39:31.776747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:36.738643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098249300000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"169'1480","reported_seq":5562,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149277+0000","last_change":"2024-09-14T22:39:20.753197+0000","last_active":"2024-09-14T22:39:32.149277+0000","last_peered":"2024-09-14T22:39:32.149277+0000","last_clean":"2024-09-14T22:39:32.149277+0000","last_became_active":"2024-09-14T22:39:20.751272+0000","last_became_peered":"2024-09-14T22:39:20.751272+0000","last_unstale":"2024-09-14T22:39:32.149277+0000","last_undegraded":"2024-09-14T22:39:32.149277+0000","last_fullsized":"2024-09-14T22:39:32.149277+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1480,"log_dups_size":0,"ondisk_log_size":1480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:15:46.970993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037027409999999998,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":9975,"num_read_kb":23940,"num_write":2463,"num_write_kb":17944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":4264139,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":88,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149438+0000","last_change":"2024-09-14T22:39:20.748672+0000","last_active":"2024-09-14T22:39:32.149438+0000","last_peered":"2024-09-14T22:39:32.149438+0000","last_clean":"2024-09-14T22:39:32.149438+0000","last_became_active":"2024-09-14T22:39:20.748164+0000","last_became_peered":"2024-09-14T22:39:20.748164+0000","last_unstale":"2024-09-14T22:39:32.149438+0000","last_undegraded":"2024-09-14T22:39:32.149438+0000","last_fullsized":"2024-09-14T22:39:32.149438+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:45:58.659226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"170'625","reported_seq":4878,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298433+0000","last_change":"2024-09-14T22:39:25.088610+0000","last_active":"2024-09-14T22:39:32.298433+0000","last_peered":"2024-09-14T22:39:32.298433+0000","last_clean":"2024-09-14T22:39:32.298433+0000","last_became_active":"2024-09-14T22:39:25.085794+0000","last_became_peered":"2024-09-14T22:39:25.085794+0000","last_unstale":"2024-09-14T22:39:32.298433+0000","last_undegraded":"2024-09-14T22:39:32.298433+0000","last_fullsized":"2024-09-14T22:39:32.298433+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:03:57.958236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002907598,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":5488,"num_read_kb":14305,"num_write":599,"num_write_kb":5812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":5797306,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"175'596","reported_seq":5470,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149769+0000","last_change":"2024-09-14T22:39:13.146266+0000","last_active":"2024-09-14T22:39:32.149769+0000","last_peered":"2024-09-14T22:39:32.149769+0000","last_clean":"2024-09-14T22:39:32.149769+0000","last_became_active":"2024-09-14T22:39:13.012755+0000","last_became_peered":"2024-09-14T22:39:13.012755+0000","last_unstale":"2024-09-14T22:39:32.149769+0000","last_undegraded":"2024-09-14T22:39:32.149769+0000","last_fullsized":"2024-09-14T22:39:32.149769+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:17:13.530413+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.084951515000000005,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":30,"num_read":4623,"num_read_kb":25789,"num_write":608,"num_write_kb":14115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":254,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298189+0000","last_change":"2024-09-14T22:39:25.086414+0000","last_active":"2024-09-14T22:39:32.298189+0000","last_peered":"2024-09-14T22:39:32.298189+0000","last_clean":"2024-09-14T22:39:32.298189+0000","last_became_active":"2024-09-14T22:39:25.085378+0000","last_became_peered":"2024-09-14T22:39:25.085378+0000","last_unstale":"2024-09-14T22:39:32.298189+0000","last_undegraded":"2024-09-14T22:39:32.298189+0000","last_fullsized":"2024-09-14T22:39:32.298189+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:01:23.251308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021617870000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"170'991","reported_seq":9285,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149171+0000","last_change":"2024-09-14T22:39:20.754533+0000","last_active":"2024-09-14T22:39:32.149171+0000","last_peered":"2024-09-14T22:39:32.149171+0000","last_clean":"2024-09-14T22:39:32.149171+0000","last_became_active":"2024-09-14T22:39:20.751993+0000","last_became_peered":"2024-09-14T22:39:20.751993+0000","last_unstale":"2024-09-14T22:39:32.149171+0000","last_undegraded":"2024-09-14T22:39:32.149171+0000","last_fullsized":"2024-09-14T22:39:32.149171+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":991,"log_dups_size":0,"ondisk_log_size":991,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:08.353422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030469329999999999,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":10327,"num_read_kb":29859,"num_write":1283,"num_write_kb":15017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":4851034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":291,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.178581+0000","last_change":"2024-09-14T22:38:05.724182+0000","last_active":"2024-09-14T22:39:31.178581+0000","last_peered":"2024-09-14T22:39:31.178581+0000","last_clean":"2024-09-14T22:39:31.178581+0000","last_became_active":"2024-09-14T22:38:05.723636+0000","last_became_peered":"2024-09-14T22:38:05.723636+0000","last_unstale":"2024-09-14T22:39:31.178581+0000","last_undegraded":"2024-09-14T22:39:31.178581+0000","last_fullsized":"2024-09-14T22:39:31.178581+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:14:09.687576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"175'449","reported_seq":5800,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306344+0000","last_change":"2024-09-14T22:39:13.119596+0000","last_active":"2024-09-14T22:39:32.306344+0000","last_peered":"2024-09-14T22:39:32.306344+0000","last_clean":"2024-09-14T22:39:32.306344+0000","last_became_active":"2024-09-14T22:39:13.025169+0000","last_became_peered":"2024-09-14T22:39:13.025169+0000","last_unstale":"2024-09-14T22:39:32.306344+0000","last_undegraded":"2024-09-14T22:39:32.306344+0000","last_fullsized":"2024-09-14T22:39:32.306344+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:01:44.220418+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.027619476,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":4593,"num_read_kb":23198,"num_write":420,"num_write_kb":7231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":3679238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":53,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149252+0000","last_change":"2024-09-14T22:39:13.009681+0000","last_active":"2024-09-14T22:39:32.149252+0000","last_peered":"2024-09-14T22:39:32.149252+0000","last_clean":"2024-09-14T22:39:32.149252+0000","last_became_active":"2024-09-14T22:39:13.008709+0000","last_became_peered":"2024-09-14T22:39:13.008709+0000","last_unstale":"2024-09-14T22:39:32.149252+0000","last_undegraded":"2024-09-14T22:39:32.149252+0000","last_fullsized":"2024-09-14T22:39:32.149252+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:14:31.162323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010817520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"169'1116","reported_seq":4770,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.472132+0000","last_change":"2024-09-14T22:39:23.066824+0000","last_active":"2024-09-14T22:39:31.472132+0000","last_peered":"2024-09-14T22:39:31.472132+0000","last_clean":"2024-09-14T22:39:31.472132+0000","last_became_active":"2024-09-14T22:39:23.064694+0000","last_became_peered":"2024-09-14T22:39:23.064694+0000","last_unstale":"2024-09-14T22:39:31.472132+0000","last_undegraded":"2024-09-14T22:39:31.472132+0000","last_fullsized":"2024-09-14T22:39:31.472132+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:32:09.848042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033884660000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":7515,"num_read_kb":16135,"num_write":1577,"num_write_kb":11855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":2936757,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":260,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298203+0000","last_change":"2024-09-14T22:39:25.082725+0000","last_active":"2024-09-14T22:39:32.298203+0000","last_peered":"2024-09-14T22:39:32.298203+0000","last_clean":"2024-09-14T22:39:32.298203+0000","last_became_active":"2024-09-14T22:39:25.081686+0000","last_became_peered":"2024-09-14T22:39:25.081686+0000","last_unstale":"2024-09-14T22:39:32.298203+0000","last_undegraded":"2024-09-14T22:39:32.298203+0000","last_fullsized":"2024-09-14T22:39:32.298203+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:35:39.626521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011401880000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":507,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776344+0000","last_change":"2024-09-14T22:36:00.402862+0000","last_active":"2024-09-14T22:39:31.776344+0000","last_peered":"2024-09-14T22:39:31.776344+0000","last_clean":"2024-09-14T22:39:31.776344+0000","last_became_active":"2024-09-14T22:36:00.400529+0000","last_became_peered":"2024-09-14T22:36:00.400529+0000","last_unstale":"2024-09-14T22:39:31.776344+0000","last_undegraded":"2024-09-14T22:39:31.776344+0000","last_fullsized":"2024-09-14T22:39:31.776344+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_clean_scrub_stamp":"2024-09-14T22:35:58.989025+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:07:12.951728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"169'676","reported_seq":7829,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.472125+0000","last_change":"2024-09-14T22:39:02.769315+0000","last_active":"2024-09-14T22:39:31.472125+0000","last_peered":"2024-09-14T22:39:31.472125+0000","last_clean":"2024-09-14T22:39:31.472125+0000","last_became_active":"2024-09-14T22:38:08.739090+0000","last_became_peered":"2024-09-14T22:38:08.739090+0000","last_unstale":"2024-09-14T22:39:31.472125+0000","last_undegraded":"2024-09-14T22:39:31.472125+0000","last_fullsized":"2024-09-14T22:39:31.472125+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:30:42.554797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012849739999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":8821,"num_read_kb":22101,"num_write":951,"num_write_kb":14470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":18690892,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"175'461","reported_seq":7414,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149505+0000","last_change":"2024-09-14T22:39:13.119552+0000","last_active":"2024-09-14T22:39:32.149505+0000","last_peered":"2024-09-14T22:39:32.149505+0000","last_clean":"2024-09-14T22:39:32.149505+0000","last_became_active":"2024-09-14T22:39:13.008690+0000","last_became_peered":"2024-09-14T22:39:13.008690+0000","last_unstale":"2024-09-14T22:39:32.149505+0000","last_undegraded":"2024-09-14T22:39:32.149505+0000","last_fullsized":"2024-09-14T22:39:32.149505+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:02:56.318281+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.064091727000000001,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":7056,"num_read_kb":18679,"num_write":421,"num_write_kb":7008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":16527933,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"175'803","reported_seq":4494,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.247652+0000","last_change":"2024-09-14T22:39:13.078772+0000","last_active":"2024-09-14T22:39:31.247652+0000","last_peered":"2024-09-14T22:39:31.247652+0000","last_clean":"2024-09-14T22:39:31.247652+0000","last_became_active":"2024-09-14T22:39:13.024372+0000","last_became_peered":"2024-09-14T22:39:13.024372+0000","last_unstale":"2024-09-14T22:39:31.247652+0000","last_undegraded":"2024-09-14T22:39:31.247652+0000","last_fullsized":"2024-09-14T22:39:31.247652+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:50.926795+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.037620476,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":28,"num_read":6415,"num_read_kb":26025,"num_write":756,"num_write_kb":14809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"170'583","reported_seq":2961,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.472227+0000","last_change":"2024-09-14T22:39:02.771485+0000","last_active":"2024-09-14T22:39:31.472227+0000","last_peered":"2024-09-14T22:39:31.472227+0000","last_clean":"2024-09-14T22:39:31.472227+0000","last_became_active":"2024-09-14T22:36:15.702885+0000","last_became_peered":"2024-09-14T22:36:15.702885+0000","last_unstale":"2024-09-14T22:39:31.472227+0000","last_undegraded":"2024-09-14T22:39:31.472227+0000","last_fullsized":"2024-09-14T22:39:31.472227+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:59:16.704705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031173400000000001,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":3570,"num_read_kb":11670,"num_write":536,"num_write_kb":6877,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"170'447","reported_seq":6923,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298274+0000","last_change":"2024-09-14T22:39:25.090361+0000","last_active":"2024-09-14T22:39:32.298274+0000","last_peered":"2024-09-14T22:39:32.298274+0000","last_clean":"2024-09-14T22:39:32.298274+0000","last_became_active":"2024-09-14T22:39:25.088012+0000","last_became_peered":"2024-09-14T22:39:25.088012+0000","last_unstale":"2024-09-14T22:39:32.298274+0000","last_undegraded":"2024-09-14T22:39:32.298274+0000","last_fullsized":"2024-09-14T22:39:32.298274+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:10:33.661519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018646120000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5808,"num_read_kb":33509,"num_write":486,"num_write_kb":17853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":23218688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"168'647","reported_seq":4232,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776747+0000","last_change":"2024-09-14T22:39:02.868251+0000","last_active":"2024-09-14T22:39:31.776747+0000","last_peered":"2024-09-14T22:39:31.776747+0000","last_clean":"2024-09-14T22:39:31.776747+0000","last_became_active":"2024-09-14T22:36:15.723205+0000","last_became_peered":"2024-09-14T22:36:15.723205+0000","last_unstale":"2024-09-14T22:39:31.776747+0000","last_undegraded":"2024-09-14T22:39:31.776747+0000","last_fullsized":"2024-09-14T22:39:31.776747+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:13:11.920909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001862636,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":4348,"num_read_kb":16212,"num_write":775,"num_write_kb":5956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"175'549","reported_seq":2917,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776796+0000","last_change":"2024-09-14T22:39:13.093929+0000","last_active":"2024-09-14T22:39:31.776796+0000","last_peered":"2024-09-14T22:39:31.776796+0000","last_clean":"2024-09-14T22:39:31.776796+0000","last_became_active":"2024-09-14T22:39:13.025887+0000","last_became_peered":"2024-09-14T22:39:13.025887+0000","last_unstale":"2024-09-14T22:39:31.776796+0000","last_undegraded":"2024-09-14T22:39:31.776796+0000","last_fullsized":"2024-09-14T22:39:31.776796+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:31:08.644424+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.030321287999999998,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":2666,"num_read_kb":8490,"num_write":594,"num_write_kb":3830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":4944418,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"175'499","reported_seq":8501,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776826+0000","last_change":"2024-09-14T22:39:13.145388+0000","last_active":"2024-09-14T22:39:31.776826+0000","last_peered":"2024-09-14T22:39:31.776826+0000","last_clean":"2024-09-14T22:39:31.776826+0000","last_became_active":"2024-09-14T22:39:13.030456+0000","last_became_peered":"2024-09-14T22:39:13.030456+0000","last_unstale":"2024-09-14T22:39:31.776826+0000","last_undegraded":"2024-09-14T22:39:31.776826+0000","last_fullsized":"2024-09-14T22:39:31.776826+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:10:30.405497+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.017930723999999999,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":8487,"num_read_kb":23501,"num_write":537,"num_write_kb":9535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"168'493","reported_seq":4108,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.247573+0000","last_change":"2024-09-14T22:39:02.866134+0000","last_active":"2024-09-14T22:39:31.247573+0000","last_peered":"2024-09-14T22:39:31.247573+0000","last_clean":"2024-09-14T22:39:31.247573+0000","last_became_active":"2024-09-14T22:37:26.717088+0000","last_became_peered":"2024-09-14T22:37:26.717088+0000","last_unstale":"2024-09-14T22:39:31.247573+0000","last_undegraded":"2024-09-14T22:39:31.247573+0000","last_fullsized":"2024-09-14T22:39:31.247573+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:03:51.158429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024508580000000002,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":3408,"num_read_kb":29082,"num_write":535,"num_write_kb":14817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"175'685","reported_seq":2731,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776343+0000","last_change":"2024-09-14T22:39:13.123794+0000","last_active":"2024-09-14T22:39:31.776343+0000","last_peered":"2024-09-14T22:39:31.776343+0000","last_clean":"2024-09-14T22:39:31.776343+0000","last_became_active":"2024-09-14T22:39:13.029544+0000","last_became_peered":"2024-09-14T22:39:13.029544+0000","last_unstale":"2024-09-14T22:39:31.776343+0000","last_undegraded":"2024-09-14T22:39:31.776343+0000","last_fullsized":"2024-09-14T22:39:31.776343+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:47:09.608401+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.020111626000000001,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":3515,"num_read_kb":11729,"num_write":870,"num_write_kb":10937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"168'589","reported_seq":8731,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306353+0000","last_change":"2024-09-14T22:39:02.782999+0000","last_active":"2024-09-14T22:39:32.306353+0000","last_peered":"2024-09-14T22:39:32.306353+0000","last_clean":"2024-09-14T22:39:32.306353+0000","last_became_active":"2024-09-14T22:36:15.716696+0000","last_became_peered":"2024-09-14T22:36:15.716696+0000","last_unstale":"2024-09-14T22:39:32.306353+0000","last_undegraded":"2024-09-14T22:39:32.306353+0000","last_fullsized":"2024-09-14T22:39:32.306353+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:14:34.437495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019004219999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":9756,"num_read_kb":50618,"num_write":568,"num_write_kb":13171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"175'599","reported_seq":7681,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776429+0000","last_change":"2024-09-14T22:39:13.167257+0000","last_active":"2024-09-14T22:39:31.776429+0000","last_peered":"2024-09-14T22:39:31.776429+0000","last_clean":"2024-09-14T22:39:31.776429+0000","last_became_active":"2024-09-14T22:39:13.030502+0000","last_became_peered":"2024-09-14T22:39:13.030502+0000","last_unstale":"2024-09-14T22:39:31.776429+0000","last_undegraded":"2024-09-14T22:39:31.776429+0000","last_fullsized":"2024-09-14T22:39:31.776429+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:35:32.951904+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.023146856,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":7147,"num_read_kb":35324,"num_write":610,"num_write_kb":15494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":14511090,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":893,"num_object_clones":0,"num_object_copies":1786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":893,"num_whiteouts":889,"num_read":206686,"num_read_kb":828988,"num_write":25214,"num_write_kb":381837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3336,"num_bytes_recovered":173602381,"num_keys_recovered":342,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22322,"internal_metadata":0},"log_size":21573,"ondisk_log_size":21573,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542191,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59848,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311992,"statfs":{"total":96636764160,"available":96575479808,"internally_reserved":0,"allocated":2347008,"data_stored":1693510,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542192,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45904,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325936,"statfs":{"total":96636764160,"available":96589758464,"internally_reserved":0,"allocated":2289664,"data_stored":1103142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542192,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55652,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316188,"statfs":{"total":96636764160,"available":96579776512,"internally_reserved":0,"allocated":2306048,"data_stored":1109032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":174,"seq":747324309510,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":2252,"kb_used_omap":34,"kb_used_meta":29277,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":2306048,"data_stored":1109032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35258,"internal_metadata":29980230},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542192,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51472,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320368,"statfs":{"total":96636764160,"available":96584056832,"internally_reserved":0,"allocated":2289664,"data_stored":1103142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":66,"seq":283467841566,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43220,"kb_used_data":2236,"kb_used_omap":6,"kb_used_meta":40953,"kb_avail":94328620,"statfs":{"total":96636764160,"available":96592506880,"internally_reserved":0,"allocated":2289664,"data_stored":1103142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7025,"internal_metadata":41936015},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59816,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312024,"statfs":{"total":96636764160,"available":96575512576,"internally_reserved":0,"allocated":2310144,"data_stored":1109051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542191,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41816,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330024,"statfs":{"total":96636764160,"available":96593944576,"internally_reserved":0,"allocated":2359296,"data_stored":1699381,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4238,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18084,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:39:35.649 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:39:35.947 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:39:35.948 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-14T22:39:35.964 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-14T22:39:34.919415+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":897,"num_object_clones":0,"num_object_copies":1794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":897,"num_whiteouts":889,"num_read":206802,"num_read_kb":829088,"num_write":25375,"num_write_kb":384255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3336,"num_bytes_recovered":173602381,"num_keys_recovered":342,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21693,"ondisk_log_size":21693,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":389312,"kb_used_data":18064,"kb_used_omap":50,"kb_used_meta":371021,"kb_avail":754585408,"statfs":{"total":773094113280,"available":772695457792,"internally_reserved":0,"allocated":18497536,"data_stored":10029432,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":51845,"internal_metadata":379925883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.807219"},"pg_stats":[{"pgid":"3.1f","version":"170'491","reported_seq":5322,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306474+0000","last_change":"2024-09-14T22:39:02.783678+0000","last_active":"2024-09-14T22:39:32.306474+0000","last_peered":"2024-09-14T22:39:32.306474+0000","last_clean":"2024-09-14T22:39:32.306474+0000","last_became_active":"2024-09-14T22:37:25.718906+0000","last_became_peered":"2024-09-14T22:37:25.718906+0000","last_unstale":"2024-09-14T22:39:32.306474+0000","last_undegraded":"2024-09-14T22:39:32.306474+0000","last_fullsized":"2024-09-14T22:39:32.306474+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:16:07.317648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023103960000000002,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":5019,"num_read_kb":16516,"num_write":439,"num_write_kb":7121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":3277632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"170'1231","reported_seq":7362,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.472426+0000","last_change":"2024-09-14T22:39:02.771231+0000","last_active":"2024-09-14T22:39:31.472426+0000","last_peered":"2024-09-14T22:39:31.472426+0000","last_clean":"2024-09-14T22:39:31.472426+0000","last_became_active":"2024-09-14T22:38:08.741149+0000","last_became_peered":"2024-09-14T22:38:08.741149+0000","last_unstale":"2024-09-14T22:39:31.472426+0000","last_undegraded":"2024-09-14T22:39:31.472426+0000","last_fullsized":"2024-09-14T22:39:31.472426+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1231,"log_dups_size":0,"ondisk_log_size":1231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:32:08.194278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024840370000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":11835,"num_read_kb":34663,"num_write":1722,"num_write_kb":10763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":7412288,"num_keys_recovered":90,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"168'724","reported_seq":10563,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149517+0000","last_change":"2024-09-14T22:39:20.753911+0000","last_active":"2024-09-14T22:39:32.149517+0000","last_peered":"2024-09-14T22:39:32.149517+0000","last_clean":"2024-09-14T22:39:32.149517+0000","last_became_active":"2024-09-14T22:39:20.751505+0000","last_became_peered":"2024-09-14T22:39:20.751505+0000","last_unstale":"2024-09-14T22:39:32.149517+0000","last_undegraded":"2024-09-14T22:39:32.149517+0000","last_fullsized":"2024-09-14T22:39:32.149517+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:17:01.639166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0083529999999999993,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":10932,"num_read_kb":63225,"num_write":868,"num_write_kb":13703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":8030815,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"175'718","reported_seq":6436,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298203+0000","last_change":"2024-09-14T22:39:25.087327+0000","last_active":"2024-09-14T22:39:32.298203+0000","last_peered":"2024-09-14T22:39:32.298203+0000","last_clean":"2024-09-14T22:39:32.298203+0000","last_became_active":"2024-09-14T22:39:25.084122+0000","last_became_peered":"2024-09-14T22:39:25.084122+0000","last_unstale":"2024-09-14T22:39:32.298203+0000","last_undegraded":"2024-09-14T22:39:32.298203+0000","last_fullsized":"2024-09-14T22:39:32.298203+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:17:17.035425+0000","scrub_duration":0,"objects_trimmed":20,"snaptrim_duration":0.055023557000000001,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":34,"num_read":6435,"num_read_kb":26956,"num_write":884,"num_write_kb":8982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"175'602","reported_seq":5586,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306492+0000","last_change":"2024-09-14T22:39:13.115776+0000","last_active":"2024-09-14T22:39:32.306492+0000","last_peered":"2024-09-14T22:39:32.306492+0000","last_clean":"2024-09-14T22:39:32.306492+0000","last_became_active":"2024-09-14T22:39:13.025036+0000","last_became_peered":"2024-09-14T22:39:13.025036+0000","last_unstale":"2024-09-14T22:39:32.306492+0000","last_undegraded":"2024-09-14T22:39:32.306492+0000","last_fullsized":"2024-09-14T22:39:32.306492+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:23:31.541475+0000","scrub_duration":0,"objects_trimmed":14,"snaptrim_duration":0.061855755999999998,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":5663,"num_read_kb":32434,"num_write":648,"num_write_kb":13330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":7858104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"175'527","reported_seq":4225,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149467+0000","last_change":"2024-09-14T22:39:13.150950+0000","last_active":"2024-09-14T22:39:32.149467+0000","last_peered":"2024-09-14T22:39:32.149467+0000","last_clean":"2024-09-14T22:39:32.149467+0000","last_became_active":"2024-09-14T22:39:13.016132+0000","last_became_peered":"2024-09-14T22:39:13.016132+0000","last_unstale":"2024-09-14T22:39:32.149467+0000","last_undegraded":"2024-09-14T22:39:32.149467+0000","last_fullsized":"2024-09-14T22:39:32.149467+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:34:37.241298+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.069063225000000006,"stat_sum":{"num_bytes":19,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":33,"num_read":4551,"num_read_kb":13173,"num_write":474,"num_write_kb":7035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":236,"num_bytes_recovered":10036538,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"169'465","reported_seq":5658,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149161+0000","last_change":"2024-09-14T22:39:02.778427+0000","last_active":"2024-09-14T22:39:32.149161+0000","last_peered":"2024-09-14T22:39:32.149161+0000","last_clean":"2024-09-14T22:39:32.149161+0000","last_became_active":"2024-09-14T22:36:15.700990+0000","last_became_peered":"2024-09-14T22:36:15.700990+0000","last_unstale":"2024-09-14T22:39:32.149161+0000","last_undegraded":"2024-09-14T22:39:32.149161+0000","last_fullsized":"2024-09-14T22:39:32.149161+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:24:53.327956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013968780000000001,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":5273,"num_read_kb":15380,"num_write":425,"num_write_kb":5486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"170'511","reported_seq":4995,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.472029+0000","last_change":"2024-09-14T22:39:02.769340+0000","last_active":"2024-09-14T22:39:31.472029+0000","last_peered":"2024-09-14T22:39:31.472029+0000","last_clean":"2024-09-14T22:39:31.472029+0000","last_became_active":"2024-09-14T22:38:07.734308+0000","last_became_peered":"2024-09-14T22:38:07.734308+0000","last_unstale":"2024-09-14T22:39:31.472029+0000","last_undegraded":"2024-09-14T22:39:31.472029+0000","last_fullsized":"2024-09-14T22:39:31.472029+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:12:44.546869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013730859999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4941,"num_read_kb":35923,"num_write":636,"num_write_kb":27736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":20067293,"num_keys_recovered":21,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"167'467","reported_seq":4865,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306510+0000","last_change":"2024-09-14T22:39:02.784093+0000","last_active":"2024-09-14T22:39:32.306510+0000","last_peered":"2024-09-14T22:39:32.306510+0000","last_clean":"2024-09-14T22:39:32.306510+0000","last_became_active":"2024-09-14T22:36:15.715869+0000","last_became_peered":"2024-09-14T22:36:15.715869+0000","last_unstale":"2024-09-14T22:39:32.306510+0000","last_undegraded":"2024-09-14T22:39:32.306510+0000","last_fullsized":"2024-09-14T22:39:32.306510+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:57:02.149610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023931830000000001,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":4593,"num_read_kb":25142,"num_write":516,"num_write_kb":22003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"166'605","reported_seq":6669,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298379+0000","last_change":"2024-09-14T22:39:02.774376+0000","last_active":"2024-09-14T22:39:32.298379+0000","last_peered":"2024-09-14T22:39:32.298379+0000","last_clean":"2024-09-14T22:39:32.298379+0000","last_became_active":"2024-09-14T22:36:15.725078+0000","last_became_peered":"2024-09-14T22:36:15.725078+0000","last_unstale":"2024-09-14T22:39:32.298379+0000","last_undegraded":"2024-09-14T22:39:32.298379+0000","last_fullsized":"2024-09-14T22:39:32.298379+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:26:12.926170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035779840000000002,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":6675,"num_read_kb":51224,"num_write":729,"num_write_kb":18952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"167'467","reported_seq":4804,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776443+0000","last_change":"2024-09-14T22:39:02.867529+0000","last_active":"2024-09-14T22:39:31.776443+0000","last_peered":"2024-09-14T22:39:31.776443+0000","last_clean":"2024-09-14T22:39:31.776443+0000","last_became_active":"2024-09-14T22:36:15.717341+0000","last_became_peered":"2024-09-14T22:36:15.717341+0000","last_unstale":"2024-09-14T22:39:31.776443+0000","last_undegraded":"2024-09-14T22:39:31.776443+0000","last_fullsized":"2024-09-14T22:39:31.776443+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:29:47.296331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016388500000000001,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":5090,"num_read_kb":12789,"num_write":419,"num_write_kb":7375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"175'1202","reported_seq":9124,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149855+0000","last_change":"2024-09-14T22:39:13.160248+0000","last_active":"2024-09-14T22:39:32.149855+0000","last_peered":"2024-09-14T22:39:32.149855+0000","last_clean":"2024-09-14T22:39:32.149855+0000","last_became_active":"2024-09-14T22:39:13.026842+0000","last_became_peered":"2024-09-14T22:39:13.026842+0000","last_unstale":"2024-09-14T22:39:32.149855+0000","last_undegraded":"2024-09-14T22:39:32.149855+0000","last_fullsized":"2024-09-14T22:39:32.149855+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1202,"log_dups_size":0,"ondisk_log_size":1202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:06:37.741547+0000","scrub_duration":0,"objects_trimmed":14,"snaptrim_duration":0.055942110000000003,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":11247,"num_read_kb":29618,"num_write":1623,"num_write_kb":11950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":265,"num_bytes_recovered":17498216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":354,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.247655+0000","last_change":"2024-09-14T22:36:08.042860+0000","last_active":"2024-09-14T22:39:31.247655+0000","last_peered":"2024-09-14T22:39:31.247655+0000","last_clean":"2024-09-14T22:39:31.247655+0000","last_became_active":"2024-09-14T22:36:06.040961+0000","last_became_peered":"2024-09-14T22:36:06.040961+0000","last_unstale":"2024-09-14T22:39:31.247655+0000","last_undegraded":"2024-09-14T22:39:31.247655+0000","last_fullsized":"2024-09-14T22:39:31.247655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:03:48.547834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010780950000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"170'738","reported_seq":3855,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306469+0000","last_change":"2024-09-14T22:39:02.782844+0000","last_active":"2024-09-14T22:39:32.306469+0000","last_peered":"2024-09-14T22:39:32.306469+0000","last_clean":"2024-09-14T22:39:32.306469+0000","last_became_active":"2024-09-14T22:36:15.700982+0000","last_became_peered":"2024-09-14T22:36:15.700982+0000","last_unstale":"2024-09-14T22:39:32.306469+0000","last_undegraded":"2024-09-14T22:39:32.306469+0000","last_fullsized":"2024-09-14T22:39:32.306469+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:06:43.937383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001672663,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":4804,"num_read_kb":22802,"num_write":686,"num_write_kb":12175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":181,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149492+0000","last_change":"2024-09-14T22:39:20.885509+0000","last_active":"2024-09-14T22:39:32.149492+0000","last_peered":"2024-09-14T22:39:32.149492+0000","last_clean":"2024-09-14T22:39:32.149492+0000","last_became_active":"2024-09-14T22:39:20.885045+0000","last_became_peered":"2024-09-14T22:39:20.885045+0000","last_unstale":"2024-09-14T22:39:32.149492+0000","last_undegraded":"2024-09-14T22:39:32.149492+0000","last_fullsized":"2024-09-14T22:39:32.149492+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:37:23.145835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"170'537","reported_seq":5821,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.472368+0000","last_change":"2024-09-14T22:39:23.066067+0000","last_active":"2024-09-14T22:39:31.472368+0000","last_peered":"2024-09-14T22:39:31.472368+0000","last_clean":"2024-09-14T22:39:31.472368+0000","last_became_active":"2024-09-14T22:39:23.064710+0000","last_became_peered":"2024-09-14T22:39:23.064710+0000","last_unstale":"2024-09-14T22:39:31.472368+0000","last_undegraded":"2024-09-14T22:39:31.472368+0000","last_fullsized":"2024-09-14T22:39:31.472368+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:53:48.607465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044565630000000002,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":6110,"num_read_kb":28977,"num_write":556,"num_write_kb":8495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":356,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776747+0000","last_change":"2024-09-14T22:36:08.054606+0000","last_active":"2024-09-14T22:39:31.776747+0000","last_peered":"2024-09-14T22:39:31.776747+0000","last_clean":"2024-09-14T22:39:31.776747+0000","last_became_active":"2024-09-14T22:36:06.266908+0000","last_became_peered":"2024-09-14T22:36:06.266908+0000","last_unstale":"2024-09-14T22:39:31.776747+0000","last_undegraded":"2024-09-14T22:39:31.776747+0000","last_fullsized":"2024-09-14T22:39:31.776747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:36.738643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098249300000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"169'1480","reported_seq":5562,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149277+0000","last_change":"2024-09-14T22:39:20.753197+0000","last_active":"2024-09-14T22:39:32.149277+0000","last_peered":"2024-09-14T22:39:32.149277+0000","last_clean":"2024-09-14T22:39:32.149277+0000","last_became_active":"2024-09-14T22:39:20.751272+0000","last_became_peered":"2024-09-14T22:39:20.751272+0000","last_unstale":"2024-09-14T22:39:32.149277+0000","last_undegraded":"2024-09-14T22:39:32.149277+0000","last_fullsized":"2024-09-14T22:39:32.149277+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1480,"log_dups_size":0,"ondisk_log_size":1480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:15:46.970993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037027409999999998,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":9975,"num_read_kb":23940,"num_write":2463,"num_write_kb":17944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":4264139,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":88,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149438+0000","last_change":"2024-09-14T22:39:20.748672+0000","last_active":"2024-09-14T22:39:32.149438+0000","last_peered":"2024-09-14T22:39:32.149438+0000","last_clean":"2024-09-14T22:39:32.149438+0000","last_became_active":"2024-09-14T22:39:20.748164+0000","last_became_peered":"2024-09-14T22:39:20.748164+0000","last_unstale":"2024-09-14T22:39:32.149438+0000","last_undegraded":"2024-09-14T22:39:32.149438+0000","last_fullsized":"2024-09-14T22:39:32.149438+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:45:58.659226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"170'625","reported_seq":4878,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298433+0000","last_change":"2024-09-14T22:39:25.088610+0000","last_active":"2024-09-14T22:39:32.298433+0000","last_peered":"2024-09-14T22:39:32.298433+0000","last_clean":"2024-09-14T22:39:32.298433+0000","last_became_active":"2024-09-14T22:39:25.085794+0000","last_became_peered":"2024-09-14T22:39:25.085794+0000","last_unstale":"2024-09-14T22:39:32.298433+0000","last_undegraded":"2024-09-14T22:39:32.298433+0000","last_fullsized":"2024-09-14T22:39:32.298433+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:03:57.958236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002907598,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":5488,"num_read_kb":14305,"num_write":599,"num_write_kb":5812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":5797306,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"175'596","reported_seq":5470,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149769+0000","last_change":"2024-09-14T22:39:13.146266+0000","last_active":"2024-09-14T22:39:32.149769+0000","last_peered":"2024-09-14T22:39:32.149769+0000","last_clean":"2024-09-14T22:39:32.149769+0000","last_became_active":"2024-09-14T22:39:13.012755+0000","last_became_peered":"2024-09-14T22:39:13.012755+0000","last_unstale":"2024-09-14T22:39:32.149769+0000","last_undegraded":"2024-09-14T22:39:32.149769+0000","last_fullsized":"2024-09-14T22:39:32.149769+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:17:13.530413+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.084951515000000005,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":30,"num_read":4623,"num_read_kb":25789,"num_write":608,"num_write_kb":14115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":254,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298189+0000","last_change":"2024-09-14T22:39:25.086414+0000","last_active":"2024-09-14T22:39:32.298189+0000","last_peered":"2024-09-14T22:39:32.298189+0000","last_clean":"2024-09-14T22:39:32.298189+0000","last_became_active":"2024-09-14T22:39:25.085378+0000","last_became_peered":"2024-09-14T22:39:25.085378+0000","last_unstale":"2024-09-14T22:39:32.298189+0000","last_undegraded":"2024-09-14T22:39:32.298189+0000","last_fullsized":"2024-09-14T22:39:32.298189+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:01:23.251308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021617870000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"170'991","reported_seq":9285,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149171+0000","last_change":"2024-09-14T22:39:20.754533+0000","last_active":"2024-09-14T22:39:32.149171+0000","last_peered":"2024-09-14T22:39:32.149171+0000","last_clean":"2024-09-14T22:39:32.149171+0000","last_became_active":"2024-09-14T22:39:20.751993+0000","last_became_peered":"2024-09-14T22:39:20.751993+0000","last_unstale":"2024-09-14T22:39:32.149171+0000","last_undegraded":"2024-09-14T22:39:32.149171+0000","last_fullsized":"2024-09-14T22:39:32.149171+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":991,"log_dups_size":0,"ondisk_log_size":991,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:08.353422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030469329999999999,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":10327,"num_read_kb":29859,"num_write":1283,"num_write_kb":15017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":4851034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":291,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.178581+0000","last_change":"2024-09-14T22:38:05.724182+0000","last_active":"2024-09-14T22:39:31.178581+0000","last_peered":"2024-09-14T22:39:31.178581+0000","last_clean":"2024-09-14T22:39:31.178581+0000","last_became_active":"2024-09-14T22:38:05.723636+0000","last_became_peered":"2024-09-14T22:38:05.723636+0000","last_unstale":"2024-09-14T22:39:31.178581+0000","last_undegraded":"2024-09-14T22:39:31.178581+0000","last_fullsized":"2024-09-14T22:39:31.178581+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:14:09.687576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"175'449","reported_seq":5800,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306344+0000","last_change":"2024-09-14T22:39:13.119596+0000","last_active":"2024-09-14T22:39:32.306344+0000","last_peered":"2024-09-14T22:39:32.306344+0000","last_clean":"2024-09-14T22:39:32.306344+0000","last_became_active":"2024-09-14T22:39:13.025169+0000","last_became_peered":"2024-09-14T22:39:13.025169+0000","last_unstale":"2024-09-14T22:39:32.306344+0000","last_undegraded":"2024-09-14T22:39:32.306344+0000","last_fullsized":"2024-09-14T22:39:32.306344+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:01:44.220418+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.027619476,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":4593,"num_read_kb":23198,"num_write":420,"num_write_kb":7231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":3679238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":53,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149252+0000","last_change":"2024-09-14T22:39:13.009681+0000","last_active":"2024-09-14T22:39:32.149252+0000","last_peered":"2024-09-14T22:39:32.149252+0000","last_clean":"2024-09-14T22:39:32.149252+0000","last_became_active":"2024-09-14T22:39:13.008709+0000","last_became_peered":"2024-09-14T22:39:13.008709+0000","last_unstale":"2024-09-14T22:39:32.149252+0000","last_undegraded":"2024-09-14T22:39:32.149252+0000","last_fullsized":"2024-09-14T22:39:32.149252+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:14:31.162323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010817520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"169'1116","reported_seq":4770,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.472132+0000","last_change":"2024-09-14T22:39:23.066824+0000","last_active":"2024-09-14T22:39:31.472132+0000","last_peered":"2024-09-14T22:39:31.472132+0000","last_clean":"2024-09-14T22:39:31.472132+0000","last_became_active":"2024-09-14T22:39:23.064694+0000","last_became_peered":"2024-09-14T22:39:23.064694+0000","last_unstale":"2024-09-14T22:39:31.472132+0000","last_undegraded":"2024-09-14T22:39:31.472132+0000","last_fullsized":"2024-09-14T22:39:31.472132+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:32:09.848042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033884660000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":7515,"num_read_kb":16135,"num_write":1577,"num_write_kb":11855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":2936757,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":260,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298203+0000","last_change":"2024-09-14T22:39:25.082725+0000","last_active":"2024-09-14T22:39:32.298203+0000","last_peered":"2024-09-14T22:39:32.298203+0000","last_clean":"2024-09-14T22:39:32.298203+0000","last_became_active":"2024-09-14T22:39:25.081686+0000","last_became_peered":"2024-09-14T22:39:25.081686+0000","last_unstale":"2024-09-14T22:39:32.298203+0000","last_undegraded":"2024-09-14T22:39:32.298203+0000","last_fullsized":"2024-09-14T22:39:32.298203+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:35:39.626521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011401880000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":507,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776344+0000","last_change":"2024-09-14T22:36:00.402862+0000","last_active":"2024-09-14T22:39:31.776344+0000","last_peered":"2024-09-14T22:39:31.776344+0000","last_clean":"2024-09-14T22:39:31.776344+0000","last_became_active":"2024-09-14T22:36:00.400529+0000","last_became_peered":"2024-09-14T22:36:00.400529+0000","last_unstale":"2024-09-14T22:39:31.776344+0000","last_undegraded":"2024-09-14T22:39:31.776344+0000","last_fullsized":"2024-09-14T22:39:31.776344+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_clean_scrub_stamp":"2024-09-14T22:35:58.989025+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:07:12.951728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"169'676","reported_seq":7829,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.472125+0000","last_change":"2024-09-14T22:39:02.769315+0000","last_active":"2024-09-14T22:39:31.472125+0000","last_peered":"2024-09-14T22:39:31.472125+0000","last_clean":"2024-09-14T22:39:31.472125+0000","last_became_active":"2024-09-14T22:38:08.739090+0000","last_became_peered":"2024-09-14T22:38:08.739090+0000","last_unstale":"2024-09-14T22:39:31.472125+0000","last_undegraded":"2024-09-14T22:39:31.472125+0000","last_fullsized":"2024-09-14T22:39:31.472125+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:30:42.554797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012849739999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":8821,"num_read_kb":22101,"num_write":951,"num_write_kb":14470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":18690892,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"175'461","reported_seq":7414,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149505+0000","last_change":"2024-09-14T22:39:13.119552+0000","last_active":"2024-09-14T22:39:32.149505+0000","last_peered":"2024-09-14T22:39:32.149505+0000","last_clean":"2024-09-14T22:39:32.149505+0000","last_became_active":"2024-09-14T22:39:13.008690+0000","last_became_peered":"2024-09-14T22:39:13.008690+0000","last_unstale":"2024-09-14T22:39:32.149505+0000","last_undegraded":"2024-09-14T22:39:32.149505+0000","last_fullsized":"2024-09-14T22:39:32.149505+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:02:56.318281+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.064091727000000001,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":7056,"num_read_kb":18679,"num_write":421,"num_write_kb":7008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":16527933,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"175'803","reported_seq":4494,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.247652+0000","last_change":"2024-09-14T22:39:13.078772+0000","last_active":"2024-09-14T22:39:31.247652+0000","last_peered":"2024-09-14T22:39:31.247652+0000","last_clean":"2024-09-14T22:39:31.247652+0000","last_became_active":"2024-09-14T22:39:13.024372+0000","last_became_peered":"2024-09-14T22:39:13.024372+0000","last_unstale":"2024-09-14T22:39:31.247652+0000","last_undegraded":"2024-09-14T22:39:31.247652+0000","last_fullsized":"2024-09-14T22:39:31.247652+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:50.926795+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.037620476,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":28,"num_read":6415,"num_read_kb":26025,"num_write":756,"num_write_kb":14809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"170'583","reported_seq":2961,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.472227+0000","last_change":"2024-09-14T22:39:02.771485+0000","last_active":"2024-09-14T22:39:31.472227+0000","last_peered":"2024-09-14T22:39:31.472227+0000","last_clean":"2024-09-14T22:39:31.472227+0000","last_became_active":"2024-09-14T22:36:15.702885+0000","last_became_peered":"2024-09-14T22:36:15.702885+0000","last_unstale":"2024-09-14T22:39:31.472227+0000","last_undegraded":"2024-09-14T22:39:31.472227+0000","last_fullsized":"2024-09-14T22:39:31.472227+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:59:16.704705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031173400000000001,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":3570,"num_read_kb":11670,"num_write":536,"num_write_kb":6877,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"170'447","reported_seq":6923,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298274+0000","last_change":"2024-09-14T22:39:25.090361+0000","last_active":"2024-09-14T22:39:32.298274+0000","last_peered":"2024-09-14T22:39:32.298274+0000","last_clean":"2024-09-14T22:39:32.298274+0000","last_became_active":"2024-09-14T22:39:25.088012+0000","last_became_peered":"2024-09-14T22:39:25.088012+0000","last_unstale":"2024-09-14T22:39:32.298274+0000","last_undegraded":"2024-09-14T22:39:32.298274+0000","last_fullsized":"2024-09-14T22:39:32.298274+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:10:33.661519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018646120000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5808,"num_read_kb":33509,"num_write":486,"num_write_kb":17853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":23218688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"168'647","reported_seq":4232,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776747+0000","last_change":"2024-09-14T22:39:02.868251+0000","last_active":"2024-09-14T22:39:31.776747+0000","last_peered":"2024-09-14T22:39:31.776747+0000","last_clean":"2024-09-14T22:39:31.776747+0000","last_became_active":"2024-09-14T22:36:15.723205+0000","last_became_peered":"2024-09-14T22:36:15.723205+0000","last_unstale":"2024-09-14T22:39:31.776747+0000","last_undegraded":"2024-09-14T22:39:31.776747+0000","last_fullsized":"2024-09-14T22:39:31.776747+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:13:11.920909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001862636,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":4348,"num_read_kb":16212,"num_write":775,"num_write_kb":5956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"175'549","reported_seq":2917,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776796+0000","last_change":"2024-09-14T22:39:13.093929+0000","last_active":"2024-09-14T22:39:31.776796+0000","last_peered":"2024-09-14T22:39:31.776796+0000","last_clean":"2024-09-14T22:39:31.776796+0000","last_became_active":"2024-09-14T22:39:13.025887+0000","last_became_peered":"2024-09-14T22:39:13.025887+0000","last_unstale":"2024-09-14T22:39:31.776796+0000","last_undegraded":"2024-09-14T22:39:31.776796+0000","last_fullsized":"2024-09-14T22:39:31.776796+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:31:08.644424+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.030321287999999998,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":2666,"num_read_kb":8490,"num_write":594,"num_write_kb":3830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":4944418,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"175'499","reported_seq":8501,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776826+0000","last_change":"2024-09-14T22:39:13.145388+0000","last_active":"2024-09-14T22:39:31.776826+0000","last_peered":"2024-09-14T22:39:31.776826+0000","last_clean":"2024-09-14T22:39:31.776826+0000","last_became_active":"2024-09-14T22:39:13.030456+0000","last_became_peered":"2024-09-14T22:39:13.030456+0000","last_unstale":"2024-09-14T22:39:31.776826+0000","last_undegraded":"2024-09-14T22:39:31.776826+0000","last_fullsized":"2024-09-14T22:39:31.776826+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:10:30.405497+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.017930723999999999,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":8487,"num_read_kb":23501,"num_write":537,"num_write_kb":9535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"168'493","reported_seq":4108,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.247573+0000","last_change":"2024-09-14T22:39:02.866134+0000","last_active":"2024-09-14T22:39:31.247573+0000","last_peered":"2024-09-14T22:39:31.247573+0000","last_clean":"2024-09-14T22:39:31.247573+0000","last_became_active":"2024-09-14T22:37:26.717088+0000","last_became_peered":"2024-09-14T22:37:26.717088+0000","last_unstale":"2024-09-14T22:39:31.247573+0000","last_undegraded":"2024-09-14T22:39:31.247573+0000","last_fullsized":"2024-09-14T22:39:31.247573+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:03:51.158429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024508580000000002,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":3408,"num_read_kb":29082,"num_write":535,"num_write_kb":14817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"175'685","reported_seq":2731,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776343+0000","last_change":"2024-09-14T22:39:13.123794+0000","last_active":"2024-09-14T22:39:31.776343+0000","last_peered":"2024-09-14T22:39:31.776343+0000","last_clean":"2024-09-14T22:39:31.776343+0000","last_became_active":"2024-09-14T22:39:13.029544+0000","last_became_peered":"2024-09-14T22:39:13.029544+0000","last_unstale":"2024-09-14T22:39:31.776343+0000","last_undegraded":"2024-09-14T22:39:31.776343+0000","last_fullsized":"2024-09-14T22:39:31.776343+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:47:09.608401+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.020111626000000001,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":3515,"num_read_kb":11729,"num_write":870,"num_write_kb":10937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"168'589","reported_seq":8731,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306353+0000","last_change":"2024-09-14T22:39:02.782999+0000","last_active":"2024-09-14T22:39:32.306353+0000","last_peered":"2024-09-14T22:39:32.306353+0000","last_clean":"2024-09-14T22:39:32.306353+0000","last_became_active":"2024-09-14T22:36:15.716696+0000","last_became_peered":"2024-09-14T22:36:15.716696+0000","last_unstale":"2024-09-14T22:39:32.306353+0000","last_undegraded":"2024-09-14T22:39:32.306353+0000","last_fullsized":"2024-09-14T22:39:32.306353+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:14:34.437495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019004219999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":9756,"num_read_kb":50618,"num_write":568,"num_write_kb":13171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"175'599","reported_seq":7681,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776429+0000","last_change":"2024-09-14T22:39:13.167257+0000","last_active":"2024-09-14T22:39:31.776429+0000","last_peered":"2024-09-14T22:39:31.776429+0000","last_clean":"2024-09-14T22:39:31.776429+0000","last_became_active":"2024-09-14T22:39:13.030502+0000","last_became_peered":"2024-09-14T22:39:13.030502+0000","last_unstale":"2024-09-14T22:39:31.776429+0000","last_undegraded":"2024-09-14T22:39:31.776429+0000","last_fullsized":"2024-09-14T22:39:31.776429+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:35:32.951904+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.023146856,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":7147,"num_read_kb":35324,"num_write":610,"num_write_kb":15494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":14511090,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":893,"num_object_clones":0,"num_object_copies":1786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":893,"num_whiteouts":889,"num_read":206686,"num_read_kb":828988,"num_write":25214,"num_write_kb":381837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3336,"num_bytes_recovered":173602381,"num_keys_recovered":342,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22322,"internal_metadata":0},"log_size":21573,"ondisk_log_size":21573,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542191,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59848,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311992,"statfs":{"total":96636764160,"available":96575479808,"internally_reserved":0,"allocated":2347008,"data_stored":1693510,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542192,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45904,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325936,"statfs":{"total":96636764160,"available":96589758464,"internally_reserved":0,"allocated":2289664,"data_stored":1103142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542192,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55652,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316188,"statfs":{"total":96636764160,"available":96579776512,"internally_reserved":0,"allocated":2306048,"data_stored":1109032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":174,"seq":747324309510,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":2252,"kb_used_omap":34,"kb_used_meta":29277,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":2306048,"data_stored":1109032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35258,"internal_metadata":29980230},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542192,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51472,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320368,"statfs":{"total":96636764160,"available":96584056832,"internally_reserved":0,"allocated":2289664,"data_stored":1103142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":66,"seq":283467841566,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43220,"kb_used_data":2236,"kb_used_omap":6,"kb_used_meta":40953,"kb_avail":94328620,"statfs":{"total":96636764160,"available":96592506880,"internally_reserved":0,"allocated":2289664,"data_stored":1103142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7025,"internal_metadata":41936015},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59816,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312024,"statfs":{"total":96636764160,"available":96575512576,"internally_reserved":0,"allocated":2310144,"data_stored":1109051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542191,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41816,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330024,"statfs":{"total":96636764160,"available":96593944576,"internally_reserved":0,"allocated":2359296,"data_stored":1699381,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4238,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18084,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:39:35.966 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-14T22:39:35.966 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-14T22:39:36.279 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-14T22:39:36.288 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-14T22:39:36.289 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:39:36.586 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:39:36.587 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-14T22:39:36.603 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-14T22:39:34.919415+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":897,"num_object_clones":0,"num_object_copies":1794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":897,"num_whiteouts":889,"num_read":206802,"num_read_kb":829088,"num_write":25375,"num_write_kb":384255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3336,"num_bytes_recovered":173602381,"num_keys_recovered":342,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21693,"ondisk_log_size":21693,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":389312,"kb_used_data":18064,"kb_used_omap":50,"kb_used_meta":371021,"kb_avail":754585408,"statfs":{"total":773094113280,"available":772695457792,"internally_reserved":0,"allocated":18497536,"data_stored":10029432,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":51845,"internal_metadata":379925883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.807219"},"pg_stats":[{"pgid":"3.1f","version":"170'491","reported_seq":5322,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306474+0000","last_change":"2024-09-14T22:39:02.783678+0000","last_active":"2024-09-14T22:39:32.306474+0000","last_peered":"2024-09-14T22:39:32.306474+0000","last_clean":"2024-09-14T22:39:32.306474+0000","last_became_active":"2024-09-14T22:37:25.718906+0000","last_became_peered":"2024-09-14T22:37:25.718906+0000","last_unstale":"2024-09-14T22:39:32.306474+0000","last_undegraded":"2024-09-14T22:39:32.306474+0000","last_fullsized":"2024-09-14T22:39:32.306474+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:16:07.317648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023103960000000002,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":5019,"num_read_kb":16516,"num_write":439,"num_write_kb":7121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":3277632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"170'1231","reported_seq":7362,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.472426+0000","last_change":"2024-09-14T22:39:02.771231+0000","last_active":"2024-09-14T22:39:31.472426+0000","last_peered":"2024-09-14T22:39:31.472426+0000","last_clean":"2024-09-14T22:39:31.472426+0000","last_became_active":"2024-09-14T22:38:08.741149+0000","last_became_peered":"2024-09-14T22:38:08.741149+0000","last_unstale":"2024-09-14T22:39:31.472426+0000","last_undegraded":"2024-09-14T22:39:31.472426+0000","last_fullsized":"2024-09-14T22:39:31.472426+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1231,"log_dups_size":0,"ondisk_log_size":1231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:32:08.194278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024840370000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":11835,"num_read_kb":34663,"num_write":1722,"num_write_kb":10763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":7412288,"num_keys_recovered":90,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"168'724","reported_seq":10563,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149517+0000","last_change":"2024-09-14T22:39:20.753911+0000","last_active":"2024-09-14T22:39:32.149517+0000","last_peered":"2024-09-14T22:39:32.149517+0000","last_clean":"2024-09-14T22:39:32.149517+0000","last_became_active":"2024-09-14T22:39:20.751505+0000","last_became_peered":"2024-09-14T22:39:20.751505+0000","last_unstale":"2024-09-14T22:39:32.149517+0000","last_undegraded":"2024-09-14T22:39:32.149517+0000","last_fullsized":"2024-09-14T22:39:32.149517+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:17:01.639166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0083529999999999993,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":10932,"num_read_kb":63225,"num_write":868,"num_write_kb":13703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":8030815,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"175'718","reported_seq":6436,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298203+0000","last_change":"2024-09-14T22:39:25.087327+0000","last_active":"2024-09-14T22:39:32.298203+0000","last_peered":"2024-09-14T22:39:32.298203+0000","last_clean":"2024-09-14T22:39:32.298203+0000","last_became_active":"2024-09-14T22:39:25.084122+0000","last_became_peered":"2024-09-14T22:39:25.084122+0000","last_unstale":"2024-09-14T22:39:32.298203+0000","last_undegraded":"2024-09-14T22:39:32.298203+0000","last_fullsized":"2024-09-14T22:39:32.298203+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:17:17.035425+0000","scrub_duration":0,"objects_trimmed":20,"snaptrim_duration":0.055023557000000001,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":34,"num_read":6435,"num_read_kb":26956,"num_write":884,"num_write_kb":8982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"175'602","reported_seq":5586,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306492+0000","last_change":"2024-09-14T22:39:13.115776+0000","last_active":"2024-09-14T22:39:32.306492+0000","last_peered":"2024-09-14T22:39:32.306492+0000","last_clean":"2024-09-14T22:39:32.306492+0000","last_became_active":"2024-09-14T22:39:13.025036+0000","last_became_peered":"2024-09-14T22:39:13.025036+0000","last_unstale":"2024-09-14T22:39:32.306492+0000","last_undegraded":"2024-09-14T22:39:32.306492+0000","last_fullsized":"2024-09-14T22:39:32.306492+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:23:31.541475+0000","scrub_duration":0,"objects_trimmed":14,"snaptrim_duration":0.061855755999999998,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":5663,"num_read_kb":32434,"num_write":648,"num_write_kb":13330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":7858104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"175'527","reported_seq":4225,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149467+0000","last_change":"2024-09-14T22:39:13.150950+0000","last_active":"2024-09-14T22:39:32.149467+0000","last_peered":"2024-09-14T22:39:32.149467+0000","last_clean":"2024-09-14T22:39:32.149467+0000","last_became_active":"2024-09-14T22:39:13.016132+0000","last_became_peered":"2024-09-14T22:39:13.016132+0000","last_unstale":"2024-09-14T22:39:32.149467+0000","last_undegraded":"2024-09-14T22:39:32.149467+0000","last_fullsized":"2024-09-14T22:39:32.149467+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:34:37.241298+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.069063225000000006,"stat_sum":{"num_bytes":19,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":33,"num_read":4551,"num_read_kb":13173,"num_write":474,"num_write_kb":7035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":236,"num_bytes_recovered":10036538,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"169'465","reported_seq":5658,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149161+0000","last_change":"2024-09-14T22:39:02.778427+0000","last_active":"2024-09-14T22:39:32.149161+0000","last_peered":"2024-09-14T22:39:32.149161+0000","last_clean":"2024-09-14T22:39:32.149161+0000","last_became_active":"2024-09-14T22:36:15.700990+0000","last_became_peered":"2024-09-14T22:36:15.700990+0000","last_unstale":"2024-09-14T22:39:32.149161+0000","last_undegraded":"2024-09-14T22:39:32.149161+0000","last_fullsized":"2024-09-14T22:39:32.149161+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:24:53.327956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013968780000000001,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":5273,"num_read_kb":15380,"num_write":425,"num_write_kb":5486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"170'511","reported_seq":4995,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.472029+0000","last_change":"2024-09-14T22:39:02.769340+0000","last_active":"2024-09-14T22:39:31.472029+0000","last_peered":"2024-09-14T22:39:31.472029+0000","last_clean":"2024-09-14T22:39:31.472029+0000","last_became_active":"2024-09-14T22:38:07.734308+0000","last_became_peered":"2024-09-14T22:38:07.734308+0000","last_unstale":"2024-09-14T22:39:31.472029+0000","last_undegraded":"2024-09-14T22:39:31.472029+0000","last_fullsized":"2024-09-14T22:39:31.472029+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:12:44.546869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013730859999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4941,"num_read_kb":35923,"num_write":636,"num_write_kb":27736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":20067293,"num_keys_recovered":21,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"167'467","reported_seq":4865,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306510+0000","last_change":"2024-09-14T22:39:02.784093+0000","last_active":"2024-09-14T22:39:32.306510+0000","last_peered":"2024-09-14T22:39:32.306510+0000","last_clean":"2024-09-14T22:39:32.306510+0000","last_became_active":"2024-09-14T22:36:15.715869+0000","last_became_peered":"2024-09-14T22:36:15.715869+0000","last_unstale":"2024-09-14T22:39:32.306510+0000","last_undegraded":"2024-09-14T22:39:32.306510+0000","last_fullsized":"2024-09-14T22:39:32.306510+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:57:02.149610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023931830000000001,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":4593,"num_read_kb":25142,"num_write":516,"num_write_kb":22003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"166'605","reported_seq":6669,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298379+0000","last_change":"2024-09-14T22:39:02.774376+0000","last_active":"2024-09-14T22:39:32.298379+0000","last_peered":"2024-09-14T22:39:32.298379+0000","last_clean":"2024-09-14T22:39:32.298379+0000","last_became_active":"2024-09-14T22:36:15.725078+0000","last_became_peered":"2024-09-14T22:36:15.725078+0000","last_unstale":"2024-09-14T22:39:32.298379+0000","last_undegraded":"2024-09-14T22:39:32.298379+0000","last_fullsized":"2024-09-14T22:39:32.298379+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:26:12.926170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035779840000000002,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":6675,"num_read_kb":51224,"num_write":729,"num_write_kb":18952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"167'467","reported_seq":4804,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776443+0000","last_change":"2024-09-14T22:39:02.867529+0000","last_active":"2024-09-14T22:39:31.776443+0000","last_peered":"2024-09-14T22:39:31.776443+0000","last_clean":"2024-09-14T22:39:31.776443+0000","last_became_active":"2024-09-14T22:36:15.717341+0000","last_became_peered":"2024-09-14T22:36:15.717341+0000","last_unstale":"2024-09-14T22:39:31.776443+0000","last_undegraded":"2024-09-14T22:39:31.776443+0000","last_fullsized":"2024-09-14T22:39:31.776443+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:29:47.296331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016388500000000001,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":5090,"num_read_kb":12789,"num_write":419,"num_write_kb":7375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"175'1202","reported_seq":9124,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149855+0000","last_change":"2024-09-14T22:39:13.160248+0000","last_active":"2024-09-14T22:39:32.149855+0000","last_peered":"2024-09-14T22:39:32.149855+0000","last_clean":"2024-09-14T22:39:32.149855+0000","last_became_active":"2024-09-14T22:39:13.026842+0000","last_became_peered":"2024-09-14T22:39:13.026842+0000","last_unstale":"2024-09-14T22:39:32.149855+0000","last_undegraded":"2024-09-14T22:39:32.149855+0000","last_fullsized":"2024-09-14T22:39:32.149855+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1202,"log_dups_size":0,"ondisk_log_size":1202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:06:37.741547+0000","scrub_duration":0,"objects_trimmed":14,"snaptrim_duration":0.055942110000000003,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":11247,"num_read_kb":29618,"num_write":1623,"num_write_kb":11950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":265,"num_bytes_recovered":17498216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":354,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.247655+0000","last_change":"2024-09-14T22:36:08.042860+0000","last_active":"2024-09-14T22:39:31.247655+0000","last_peered":"2024-09-14T22:39:31.247655+0000","last_clean":"2024-09-14T22:39:31.247655+0000","last_became_active":"2024-09-14T22:36:06.040961+0000","last_became_peered":"2024-09-14T22:36:06.040961+0000","last_unstale":"2024-09-14T22:39:31.247655+0000","last_undegraded":"2024-09-14T22:39:31.247655+0000","last_fullsized":"2024-09-14T22:39:31.247655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:03:48.547834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010780950000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"170'738","reported_seq":3855,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306469+0000","last_change":"2024-09-14T22:39:02.782844+0000","last_active":"2024-09-14T22:39:32.306469+0000","last_peered":"2024-09-14T22:39:32.306469+0000","last_clean":"2024-09-14T22:39:32.306469+0000","last_became_active":"2024-09-14T22:36:15.700982+0000","last_became_peered":"2024-09-14T22:36:15.700982+0000","last_unstale":"2024-09-14T22:39:32.306469+0000","last_undegraded":"2024-09-14T22:39:32.306469+0000","last_fullsized":"2024-09-14T22:39:32.306469+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:06:43.937383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001672663,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":4804,"num_read_kb":22802,"num_write":686,"num_write_kb":12175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":181,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149492+0000","last_change":"2024-09-14T22:39:20.885509+0000","last_active":"2024-09-14T22:39:32.149492+0000","last_peered":"2024-09-14T22:39:32.149492+0000","last_clean":"2024-09-14T22:39:32.149492+0000","last_became_active":"2024-09-14T22:39:20.885045+0000","last_became_peered":"2024-09-14T22:39:20.885045+0000","last_unstale":"2024-09-14T22:39:32.149492+0000","last_undegraded":"2024-09-14T22:39:32.149492+0000","last_fullsized":"2024-09-14T22:39:32.149492+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:37:23.145835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"170'537","reported_seq":5821,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.472368+0000","last_change":"2024-09-14T22:39:23.066067+0000","last_active":"2024-09-14T22:39:31.472368+0000","last_peered":"2024-09-14T22:39:31.472368+0000","last_clean":"2024-09-14T22:39:31.472368+0000","last_became_active":"2024-09-14T22:39:23.064710+0000","last_became_peered":"2024-09-14T22:39:23.064710+0000","last_unstale":"2024-09-14T22:39:31.472368+0000","last_undegraded":"2024-09-14T22:39:31.472368+0000","last_fullsized":"2024-09-14T22:39:31.472368+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:53:48.607465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044565630000000002,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":6110,"num_read_kb":28977,"num_write":556,"num_write_kb":8495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":356,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776747+0000","last_change":"2024-09-14T22:36:08.054606+0000","last_active":"2024-09-14T22:39:31.776747+0000","last_peered":"2024-09-14T22:39:31.776747+0000","last_clean":"2024-09-14T22:39:31.776747+0000","last_became_active":"2024-09-14T22:36:06.266908+0000","last_became_peered":"2024-09-14T22:36:06.266908+0000","last_unstale":"2024-09-14T22:39:31.776747+0000","last_undegraded":"2024-09-14T22:39:31.776747+0000","last_fullsized":"2024-09-14T22:39:31.776747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:36.738643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098249300000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"169'1480","reported_seq":5562,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149277+0000","last_change":"2024-09-14T22:39:20.753197+0000","last_active":"2024-09-14T22:39:32.149277+0000","last_peered":"2024-09-14T22:39:32.149277+0000","last_clean":"2024-09-14T22:39:32.149277+0000","last_became_active":"2024-09-14T22:39:20.751272+0000","last_became_peered":"2024-09-14T22:39:20.751272+0000","last_unstale":"2024-09-14T22:39:32.149277+0000","last_undegraded":"2024-09-14T22:39:32.149277+0000","last_fullsized":"2024-09-14T22:39:32.149277+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1480,"log_dups_size":0,"ondisk_log_size":1480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:15:46.970993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037027409999999998,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":9975,"num_read_kb":23940,"num_write":2463,"num_write_kb":17944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":4264139,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":88,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149438+0000","last_change":"2024-09-14T22:39:20.748672+0000","last_active":"2024-09-14T22:39:32.149438+0000","last_peered":"2024-09-14T22:39:32.149438+0000","last_clean":"2024-09-14T22:39:32.149438+0000","last_became_active":"2024-09-14T22:39:20.748164+0000","last_became_peered":"2024-09-14T22:39:20.748164+0000","last_unstale":"2024-09-14T22:39:32.149438+0000","last_undegraded":"2024-09-14T22:39:32.149438+0000","last_fullsized":"2024-09-14T22:39:32.149438+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:45:58.659226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"170'625","reported_seq":4878,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298433+0000","last_change":"2024-09-14T22:39:25.088610+0000","last_active":"2024-09-14T22:39:32.298433+0000","last_peered":"2024-09-14T22:39:32.298433+0000","last_clean":"2024-09-14T22:39:32.298433+0000","last_became_active":"2024-09-14T22:39:25.085794+0000","last_became_peered":"2024-09-14T22:39:25.085794+0000","last_unstale":"2024-09-14T22:39:32.298433+0000","last_undegraded":"2024-09-14T22:39:32.298433+0000","last_fullsized":"2024-09-14T22:39:32.298433+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:03:57.958236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002907598,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":5488,"num_read_kb":14305,"num_write":599,"num_write_kb":5812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":5797306,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"175'596","reported_seq":5470,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149769+0000","last_change":"2024-09-14T22:39:13.146266+0000","last_active":"2024-09-14T22:39:32.149769+0000","last_peered":"2024-09-14T22:39:32.149769+0000","last_clean":"2024-09-14T22:39:32.149769+0000","last_became_active":"2024-09-14T22:39:13.012755+0000","last_became_peered":"2024-09-14T22:39:13.012755+0000","last_unstale":"2024-09-14T22:39:32.149769+0000","last_undegraded":"2024-09-14T22:39:32.149769+0000","last_fullsized":"2024-09-14T22:39:32.149769+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:17:13.530413+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.084951515000000005,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":30,"num_read":4623,"num_read_kb":25789,"num_write":608,"num_write_kb":14115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":254,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298189+0000","last_change":"2024-09-14T22:39:25.086414+0000","last_active":"2024-09-14T22:39:32.298189+0000","last_peered":"2024-09-14T22:39:32.298189+0000","last_clean":"2024-09-14T22:39:32.298189+0000","last_became_active":"2024-09-14T22:39:25.085378+0000","last_became_peered":"2024-09-14T22:39:25.085378+0000","last_unstale":"2024-09-14T22:39:32.298189+0000","last_undegraded":"2024-09-14T22:39:32.298189+0000","last_fullsized":"2024-09-14T22:39:32.298189+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:01:23.251308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021617870000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"170'991","reported_seq":9285,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149171+0000","last_change":"2024-09-14T22:39:20.754533+0000","last_active":"2024-09-14T22:39:32.149171+0000","last_peered":"2024-09-14T22:39:32.149171+0000","last_clean":"2024-09-14T22:39:32.149171+0000","last_became_active":"2024-09-14T22:39:20.751993+0000","last_became_peered":"2024-09-14T22:39:20.751993+0000","last_unstale":"2024-09-14T22:39:32.149171+0000","last_undegraded":"2024-09-14T22:39:32.149171+0000","last_fullsized":"2024-09-14T22:39:32.149171+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":991,"log_dups_size":0,"ondisk_log_size":991,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:08.353422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030469329999999999,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":10327,"num_read_kb":29859,"num_write":1283,"num_write_kb":15017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":4851034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":291,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.178581+0000","last_change":"2024-09-14T22:38:05.724182+0000","last_active":"2024-09-14T22:39:31.178581+0000","last_peered":"2024-09-14T22:39:31.178581+0000","last_clean":"2024-09-14T22:39:31.178581+0000","last_became_active":"2024-09-14T22:38:05.723636+0000","last_became_peered":"2024-09-14T22:38:05.723636+0000","last_unstale":"2024-09-14T22:39:31.178581+0000","last_undegraded":"2024-09-14T22:39:31.178581+0000","last_fullsized":"2024-09-14T22:39:31.178581+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:14:09.687576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"175'449","reported_seq":5800,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306344+0000","last_change":"2024-09-14T22:39:13.119596+0000","last_active":"2024-09-14T22:39:32.306344+0000","last_peered":"2024-09-14T22:39:32.306344+0000","last_clean":"2024-09-14T22:39:32.306344+0000","last_became_active":"2024-09-14T22:39:13.025169+0000","last_became_peered":"2024-09-14T22:39:13.025169+0000","last_unstale":"2024-09-14T22:39:32.306344+0000","last_undegraded":"2024-09-14T22:39:32.306344+0000","last_fullsized":"2024-09-14T22:39:32.306344+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:01:44.220418+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.027619476,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":4593,"num_read_kb":23198,"num_write":420,"num_write_kb":7231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":3679238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":53,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149252+0000","last_change":"2024-09-14T22:39:13.009681+0000","last_active":"2024-09-14T22:39:32.149252+0000","last_peered":"2024-09-14T22:39:32.149252+0000","last_clean":"2024-09-14T22:39:32.149252+0000","last_became_active":"2024-09-14T22:39:13.008709+0000","last_became_peered":"2024-09-14T22:39:13.008709+0000","last_unstale":"2024-09-14T22:39:32.149252+0000","last_undegraded":"2024-09-14T22:39:32.149252+0000","last_fullsized":"2024-09-14T22:39:32.149252+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:14:31.162323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010817520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"169'1116","reported_seq":4770,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.472132+0000","last_change":"2024-09-14T22:39:23.066824+0000","last_active":"2024-09-14T22:39:31.472132+0000","last_peered":"2024-09-14T22:39:31.472132+0000","last_clean":"2024-09-14T22:39:31.472132+0000","last_became_active":"2024-09-14T22:39:23.064694+0000","last_became_peered":"2024-09-14T22:39:23.064694+0000","last_unstale":"2024-09-14T22:39:31.472132+0000","last_undegraded":"2024-09-14T22:39:31.472132+0000","last_fullsized":"2024-09-14T22:39:31.472132+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:32:09.848042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033884660000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":7515,"num_read_kb":16135,"num_write":1577,"num_write_kb":11855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":2936757,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":260,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298203+0000","last_change":"2024-09-14T22:39:25.082725+0000","last_active":"2024-09-14T22:39:32.298203+0000","last_peered":"2024-09-14T22:39:32.298203+0000","last_clean":"2024-09-14T22:39:32.298203+0000","last_became_active":"2024-09-14T22:39:25.081686+0000","last_became_peered":"2024-09-14T22:39:25.081686+0000","last_unstale":"2024-09-14T22:39:32.298203+0000","last_undegraded":"2024-09-14T22:39:32.298203+0000","last_fullsized":"2024-09-14T22:39:32.298203+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:35:39.626521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011401880000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":507,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776344+0000","last_change":"2024-09-14T22:36:00.402862+0000","last_active":"2024-09-14T22:39:31.776344+0000","last_peered":"2024-09-14T22:39:31.776344+0000","last_clean":"2024-09-14T22:39:31.776344+0000","last_became_active":"2024-09-14T22:36:00.400529+0000","last_became_peered":"2024-09-14T22:36:00.400529+0000","last_unstale":"2024-09-14T22:39:31.776344+0000","last_undegraded":"2024-09-14T22:39:31.776344+0000","last_fullsized":"2024-09-14T22:39:31.776344+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_clean_scrub_stamp":"2024-09-14T22:35:58.989025+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:07:12.951728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"169'676","reported_seq":7829,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.472125+0000","last_change":"2024-09-14T22:39:02.769315+0000","last_active":"2024-09-14T22:39:31.472125+0000","last_peered":"2024-09-14T22:39:31.472125+0000","last_clean":"2024-09-14T22:39:31.472125+0000","last_became_active":"2024-09-14T22:38:08.739090+0000","last_became_peered":"2024-09-14T22:38:08.739090+0000","last_unstale":"2024-09-14T22:39:31.472125+0000","last_undegraded":"2024-09-14T22:39:31.472125+0000","last_fullsized":"2024-09-14T22:39:31.472125+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:30:42.554797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012849739999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":8821,"num_read_kb":22101,"num_write":951,"num_write_kb":14470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":18690892,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"175'461","reported_seq":7414,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149505+0000","last_change":"2024-09-14T22:39:13.119552+0000","last_active":"2024-09-14T22:39:32.149505+0000","last_peered":"2024-09-14T22:39:32.149505+0000","last_clean":"2024-09-14T22:39:32.149505+0000","last_became_active":"2024-09-14T22:39:13.008690+0000","last_became_peered":"2024-09-14T22:39:13.008690+0000","last_unstale":"2024-09-14T22:39:32.149505+0000","last_undegraded":"2024-09-14T22:39:32.149505+0000","last_fullsized":"2024-09-14T22:39:32.149505+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:02:56.318281+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.064091727000000001,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":7056,"num_read_kb":18679,"num_write":421,"num_write_kb":7008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":16527933,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"175'803","reported_seq":4494,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.247652+0000","last_change":"2024-09-14T22:39:13.078772+0000","last_active":"2024-09-14T22:39:31.247652+0000","last_peered":"2024-09-14T22:39:31.247652+0000","last_clean":"2024-09-14T22:39:31.247652+0000","last_became_active":"2024-09-14T22:39:13.024372+0000","last_became_peered":"2024-09-14T22:39:13.024372+0000","last_unstale":"2024-09-14T22:39:31.247652+0000","last_undegraded":"2024-09-14T22:39:31.247652+0000","last_fullsized":"2024-09-14T22:39:31.247652+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:50.926795+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.037620476,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":28,"num_read":6415,"num_read_kb":26025,"num_write":756,"num_write_kb":14809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"170'583","reported_seq":2961,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.472227+0000","last_change":"2024-09-14T22:39:02.771485+0000","last_active":"2024-09-14T22:39:31.472227+0000","last_peered":"2024-09-14T22:39:31.472227+0000","last_clean":"2024-09-14T22:39:31.472227+0000","last_became_active":"2024-09-14T22:36:15.702885+0000","last_became_peered":"2024-09-14T22:36:15.702885+0000","last_unstale":"2024-09-14T22:39:31.472227+0000","last_undegraded":"2024-09-14T22:39:31.472227+0000","last_fullsized":"2024-09-14T22:39:31.472227+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:59:16.704705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031173400000000001,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":3570,"num_read_kb":11670,"num_write":536,"num_write_kb":6877,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"170'447","reported_seq":6923,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298274+0000","last_change":"2024-09-14T22:39:25.090361+0000","last_active":"2024-09-14T22:39:32.298274+0000","last_peered":"2024-09-14T22:39:32.298274+0000","last_clean":"2024-09-14T22:39:32.298274+0000","last_became_active":"2024-09-14T22:39:25.088012+0000","last_became_peered":"2024-09-14T22:39:25.088012+0000","last_unstale":"2024-09-14T22:39:32.298274+0000","last_undegraded":"2024-09-14T22:39:32.298274+0000","last_fullsized":"2024-09-14T22:39:32.298274+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:10:33.661519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018646120000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5808,"num_read_kb":33509,"num_write":486,"num_write_kb":17853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":23218688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"168'647","reported_seq":4232,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776747+0000","last_change":"2024-09-14T22:39:02.868251+0000","last_active":"2024-09-14T22:39:31.776747+0000","last_peered":"2024-09-14T22:39:31.776747+0000","last_clean":"2024-09-14T22:39:31.776747+0000","last_became_active":"2024-09-14T22:36:15.723205+0000","last_became_peered":"2024-09-14T22:36:15.723205+0000","last_unstale":"2024-09-14T22:39:31.776747+0000","last_undegraded":"2024-09-14T22:39:31.776747+0000","last_fullsized":"2024-09-14T22:39:31.776747+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:13:11.920909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001862636,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":4348,"num_read_kb":16212,"num_write":775,"num_write_kb":5956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"175'549","reported_seq":2917,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776796+0000","last_change":"2024-09-14T22:39:13.093929+0000","last_active":"2024-09-14T22:39:31.776796+0000","last_peered":"2024-09-14T22:39:31.776796+0000","last_clean":"2024-09-14T22:39:31.776796+0000","last_became_active":"2024-09-14T22:39:13.025887+0000","last_became_peered":"2024-09-14T22:39:13.025887+0000","last_unstale":"2024-09-14T22:39:31.776796+0000","last_undegraded":"2024-09-14T22:39:31.776796+0000","last_fullsized":"2024-09-14T22:39:31.776796+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:31:08.644424+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.030321287999999998,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":2666,"num_read_kb":8490,"num_write":594,"num_write_kb":3830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":4944418,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"175'499","reported_seq":8501,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776826+0000","last_change":"2024-09-14T22:39:13.145388+0000","last_active":"2024-09-14T22:39:31.776826+0000","last_peered":"2024-09-14T22:39:31.776826+0000","last_clean":"2024-09-14T22:39:31.776826+0000","last_became_active":"2024-09-14T22:39:13.030456+0000","last_became_peered":"2024-09-14T22:39:13.030456+0000","last_unstale":"2024-09-14T22:39:31.776826+0000","last_undegraded":"2024-09-14T22:39:31.776826+0000","last_fullsized":"2024-09-14T22:39:31.776826+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:10:30.405497+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.017930723999999999,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":8487,"num_read_kb":23501,"num_write":537,"num_write_kb":9535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"168'493","reported_seq":4108,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.247573+0000","last_change":"2024-09-14T22:39:02.866134+0000","last_active":"2024-09-14T22:39:31.247573+0000","last_peered":"2024-09-14T22:39:31.247573+0000","last_clean":"2024-09-14T22:39:31.247573+0000","last_became_active":"2024-09-14T22:37:26.717088+0000","last_became_peered":"2024-09-14T22:37:26.717088+0000","last_unstale":"2024-09-14T22:39:31.247573+0000","last_undegraded":"2024-09-14T22:39:31.247573+0000","last_fullsized":"2024-09-14T22:39:31.247573+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:03:51.158429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024508580000000002,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":3408,"num_read_kb":29082,"num_write":535,"num_write_kb":14817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"175'685","reported_seq":2731,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776343+0000","last_change":"2024-09-14T22:39:13.123794+0000","last_active":"2024-09-14T22:39:31.776343+0000","last_peered":"2024-09-14T22:39:31.776343+0000","last_clean":"2024-09-14T22:39:31.776343+0000","last_became_active":"2024-09-14T22:39:13.029544+0000","last_became_peered":"2024-09-14T22:39:13.029544+0000","last_unstale":"2024-09-14T22:39:31.776343+0000","last_undegraded":"2024-09-14T22:39:31.776343+0000","last_fullsized":"2024-09-14T22:39:31.776343+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:47:09.608401+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.020111626000000001,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":3515,"num_read_kb":11729,"num_write":870,"num_write_kb":10937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"168'589","reported_seq":8731,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306353+0000","last_change":"2024-09-14T22:39:02.782999+0000","last_active":"2024-09-14T22:39:32.306353+0000","last_peered":"2024-09-14T22:39:32.306353+0000","last_clean":"2024-09-14T22:39:32.306353+0000","last_became_active":"2024-09-14T22:36:15.716696+0000","last_became_peered":"2024-09-14T22:36:15.716696+0000","last_unstale":"2024-09-14T22:39:32.306353+0000","last_undegraded":"2024-09-14T22:39:32.306353+0000","last_fullsized":"2024-09-14T22:39:32.306353+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:14:34.437495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019004219999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":9756,"num_read_kb":50618,"num_write":568,"num_write_kb":13171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"175'599","reported_seq":7681,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776429+0000","last_change":"2024-09-14T22:39:13.167257+0000","last_active":"2024-09-14T22:39:31.776429+0000","last_peered":"2024-09-14T22:39:31.776429+0000","last_clean":"2024-09-14T22:39:31.776429+0000","last_became_active":"2024-09-14T22:39:13.030502+0000","last_became_peered":"2024-09-14T22:39:13.030502+0000","last_unstale":"2024-09-14T22:39:31.776429+0000","last_undegraded":"2024-09-14T22:39:31.776429+0000","last_fullsized":"2024-09-14T22:39:31.776429+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:35:32.951904+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.023146856,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":7147,"num_read_kb":35324,"num_write":610,"num_write_kb":15494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":14511090,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":893,"num_object_clones":0,"num_object_copies":1786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":893,"num_whiteouts":889,"num_read":206686,"num_read_kb":828988,"num_write":25214,"num_write_kb":381837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3336,"num_bytes_recovered":173602381,"num_keys_recovered":342,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22322,"internal_metadata":0},"log_size":21573,"ondisk_log_size":21573,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542191,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59848,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311992,"statfs":{"total":96636764160,"available":96575479808,"internally_reserved":0,"allocated":2347008,"data_stored":1693510,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542192,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45904,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325936,"statfs":{"total":96636764160,"available":96589758464,"internally_reserved":0,"allocated":2289664,"data_stored":1103142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542192,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55652,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316188,"statfs":{"total":96636764160,"available":96579776512,"internally_reserved":0,"allocated":2306048,"data_stored":1109032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":174,"seq":747324309510,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":2252,"kb_used_omap":34,"kb_used_meta":29277,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":2306048,"data_stored":1109032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35258,"internal_metadata":29980230},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542192,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51472,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320368,"statfs":{"total":96636764160,"available":96584056832,"internally_reserved":0,"allocated":2289664,"data_stored":1103142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":66,"seq":283467841566,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43220,"kb_used_data":2236,"kb_used_omap":6,"kb_used_meta":40953,"kb_avail":94328620,"statfs":{"total":96636764160,"available":96592506880,"internally_reserved":0,"allocated":2289664,"data_stored":1103142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7025,"internal_metadata":41936015},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59816,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312024,"statfs":{"total":96636764160,"available":96575512576,"internally_reserved":0,"allocated":2310144,"data_stored":1109051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542191,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41816,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330024,"statfs":{"total":96636764160,"available":96593944576,"internally_reserved":0,"allocated":2359296,"data_stored":1699381,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4238,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18084,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:39:36.605 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:39:36.893 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:39:36.894 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-14T22:39:36.910 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-14T22:39:34.919415+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":897,"num_object_clones":0,"num_object_copies":1794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":897,"num_whiteouts":889,"num_read":206802,"num_read_kb":829088,"num_write":25375,"num_write_kb":384255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3336,"num_bytes_recovered":173602381,"num_keys_recovered":342,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21693,"ondisk_log_size":21693,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":389312,"kb_used_data":18064,"kb_used_omap":50,"kb_used_meta":371021,"kb_avail":754585408,"statfs":{"total":773094113280,"available":772695457792,"internally_reserved":0,"allocated":18497536,"data_stored":10029432,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":51845,"internal_metadata":379925883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.807219"},"pg_stats":[{"pgid":"3.1f","version":"170'491","reported_seq":5322,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306474+0000","last_change":"2024-09-14T22:39:02.783678+0000","last_active":"2024-09-14T22:39:32.306474+0000","last_peered":"2024-09-14T22:39:32.306474+0000","last_clean":"2024-09-14T22:39:32.306474+0000","last_became_active":"2024-09-14T22:37:25.718906+0000","last_became_peered":"2024-09-14T22:37:25.718906+0000","last_unstale":"2024-09-14T22:39:32.306474+0000","last_undegraded":"2024-09-14T22:39:32.306474+0000","last_fullsized":"2024-09-14T22:39:32.306474+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:16:07.317648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023103960000000002,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":5019,"num_read_kb":16516,"num_write":439,"num_write_kb":7121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":3277632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"170'1231","reported_seq":7362,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.472426+0000","last_change":"2024-09-14T22:39:02.771231+0000","last_active":"2024-09-14T22:39:31.472426+0000","last_peered":"2024-09-14T22:39:31.472426+0000","last_clean":"2024-09-14T22:39:31.472426+0000","last_became_active":"2024-09-14T22:38:08.741149+0000","last_became_peered":"2024-09-14T22:38:08.741149+0000","last_unstale":"2024-09-14T22:39:31.472426+0000","last_undegraded":"2024-09-14T22:39:31.472426+0000","last_fullsized":"2024-09-14T22:39:31.472426+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1231,"log_dups_size":0,"ondisk_log_size":1231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:32:08.194278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024840370000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":11835,"num_read_kb":34663,"num_write":1722,"num_write_kb":10763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":7412288,"num_keys_recovered":90,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"168'724","reported_seq":10563,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149517+0000","last_change":"2024-09-14T22:39:20.753911+0000","last_active":"2024-09-14T22:39:32.149517+0000","last_peered":"2024-09-14T22:39:32.149517+0000","last_clean":"2024-09-14T22:39:32.149517+0000","last_became_active":"2024-09-14T22:39:20.751505+0000","last_became_peered":"2024-09-14T22:39:20.751505+0000","last_unstale":"2024-09-14T22:39:32.149517+0000","last_undegraded":"2024-09-14T22:39:32.149517+0000","last_fullsized":"2024-09-14T22:39:32.149517+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:17:01.639166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0083529999999999993,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":10932,"num_read_kb":63225,"num_write":868,"num_write_kb":13703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":8030815,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"175'718","reported_seq":6436,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298203+0000","last_change":"2024-09-14T22:39:25.087327+0000","last_active":"2024-09-14T22:39:32.298203+0000","last_peered":"2024-09-14T22:39:32.298203+0000","last_clean":"2024-09-14T22:39:32.298203+0000","last_became_active":"2024-09-14T22:39:25.084122+0000","last_became_peered":"2024-09-14T22:39:25.084122+0000","last_unstale":"2024-09-14T22:39:32.298203+0000","last_undegraded":"2024-09-14T22:39:32.298203+0000","last_fullsized":"2024-09-14T22:39:32.298203+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:17:17.035425+0000","scrub_duration":0,"objects_trimmed":20,"snaptrim_duration":0.055023557000000001,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":34,"num_read":6435,"num_read_kb":26956,"num_write":884,"num_write_kb":8982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"175'602","reported_seq":5586,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306492+0000","last_change":"2024-09-14T22:39:13.115776+0000","last_active":"2024-09-14T22:39:32.306492+0000","last_peered":"2024-09-14T22:39:32.306492+0000","last_clean":"2024-09-14T22:39:32.306492+0000","last_became_active":"2024-09-14T22:39:13.025036+0000","last_became_peered":"2024-09-14T22:39:13.025036+0000","last_unstale":"2024-09-14T22:39:32.306492+0000","last_undegraded":"2024-09-14T22:39:32.306492+0000","last_fullsized":"2024-09-14T22:39:32.306492+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:23:31.541475+0000","scrub_duration":0,"objects_trimmed":14,"snaptrim_duration":0.061855755999999998,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":5663,"num_read_kb":32434,"num_write":648,"num_write_kb":13330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":7858104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"175'527","reported_seq":4225,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149467+0000","last_change":"2024-09-14T22:39:13.150950+0000","last_active":"2024-09-14T22:39:32.149467+0000","last_peered":"2024-09-14T22:39:32.149467+0000","last_clean":"2024-09-14T22:39:32.149467+0000","last_became_active":"2024-09-14T22:39:13.016132+0000","last_became_peered":"2024-09-14T22:39:13.016132+0000","last_unstale":"2024-09-14T22:39:32.149467+0000","last_undegraded":"2024-09-14T22:39:32.149467+0000","last_fullsized":"2024-09-14T22:39:32.149467+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:34:37.241298+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.069063225000000006,"stat_sum":{"num_bytes":19,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":33,"num_read":4551,"num_read_kb":13173,"num_write":474,"num_write_kb":7035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":236,"num_bytes_recovered":10036538,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"169'465","reported_seq":5658,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149161+0000","last_change":"2024-09-14T22:39:02.778427+0000","last_active":"2024-09-14T22:39:32.149161+0000","last_peered":"2024-09-14T22:39:32.149161+0000","last_clean":"2024-09-14T22:39:32.149161+0000","last_became_active":"2024-09-14T22:36:15.700990+0000","last_became_peered":"2024-09-14T22:36:15.700990+0000","last_unstale":"2024-09-14T22:39:32.149161+0000","last_undegraded":"2024-09-14T22:39:32.149161+0000","last_fullsized":"2024-09-14T22:39:32.149161+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:24:53.327956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013968780000000001,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":5273,"num_read_kb":15380,"num_write":425,"num_write_kb":5486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"170'511","reported_seq":4995,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.472029+0000","last_change":"2024-09-14T22:39:02.769340+0000","last_active":"2024-09-14T22:39:31.472029+0000","last_peered":"2024-09-14T22:39:31.472029+0000","last_clean":"2024-09-14T22:39:31.472029+0000","last_became_active":"2024-09-14T22:38:07.734308+0000","last_became_peered":"2024-09-14T22:38:07.734308+0000","last_unstale":"2024-09-14T22:39:31.472029+0000","last_undegraded":"2024-09-14T22:39:31.472029+0000","last_fullsized":"2024-09-14T22:39:31.472029+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:12:44.546869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013730859999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4941,"num_read_kb":35923,"num_write":636,"num_write_kb":27736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":20067293,"num_keys_recovered":21,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"167'467","reported_seq":4865,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306510+0000","last_change":"2024-09-14T22:39:02.784093+0000","last_active":"2024-09-14T22:39:32.306510+0000","last_peered":"2024-09-14T22:39:32.306510+0000","last_clean":"2024-09-14T22:39:32.306510+0000","last_became_active":"2024-09-14T22:36:15.715869+0000","last_became_peered":"2024-09-14T22:36:15.715869+0000","last_unstale":"2024-09-14T22:39:32.306510+0000","last_undegraded":"2024-09-14T22:39:32.306510+0000","last_fullsized":"2024-09-14T22:39:32.306510+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:57:02.149610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023931830000000001,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":4593,"num_read_kb":25142,"num_write":516,"num_write_kb":22003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"166'605","reported_seq":6669,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298379+0000","last_change":"2024-09-14T22:39:02.774376+0000","last_active":"2024-09-14T22:39:32.298379+0000","last_peered":"2024-09-14T22:39:32.298379+0000","last_clean":"2024-09-14T22:39:32.298379+0000","last_became_active":"2024-09-14T22:36:15.725078+0000","last_became_peered":"2024-09-14T22:36:15.725078+0000","last_unstale":"2024-09-14T22:39:32.298379+0000","last_undegraded":"2024-09-14T22:39:32.298379+0000","last_fullsized":"2024-09-14T22:39:32.298379+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:26:12.926170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035779840000000002,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":6675,"num_read_kb":51224,"num_write":729,"num_write_kb":18952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"167'467","reported_seq":4804,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776443+0000","last_change":"2024-09-14T22:39:02.867529+0000","last_active":"2024-09-14T22:39:31.776443+0000","last_peered":"2024-09-14T22:39:31.776443+0000","last_clean":"2024-09-14T22:39:31.776443+0000","last_became_active":"2024-09-14T22:36:15.717341+0000","last_became_peered":"2024-09-14T22:36:15.717341+0000","last_unstale":"2024-09-14T22:39:31.776443+0000","last_undegraded":"2024-09-14T22:39:31.776443+0000","last_fullsized":"2024-09-14T22:39:31.776443+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:29:47.296331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016388500000000001,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":5090,"num_read_kb":12789,"num_write":419,"num_write_kb":7375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"175'1202","reported_seq":9124,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149855+0000","last_change":"2024-09-14T22:39:13.160248+0000","last_active":"2024-09-14T22:39:32.149855+0000","last_peered":"2024-09-14T22:39:32.149855+0000","last_clean":"2024-09-14T22:39:32.149855+0000","last_became_active":"2024-09-14T22:39:13.026842+0000","last_became_peered":"2024-09-14T22:39:13.026842+0000","last_unstale":"2024-09-14T22:39:32.149855+0000","last_undegraded":"2024-09-14T22:39:32.149855+0000","last_fullsized":"2024-09-14T22:39:32.149855+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1202,"log_dups_size":0,"ondisk_log_size":1202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:06:37.741547+0000","scrub_duration":0,"objects_trimmed":14,"snaptrim_duration":0.055942110000000003,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":11247,"num_read_kb":29618,"num_write":1623,"num_write_kb":11950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":265,"num_bytes_recovered":17498216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":354,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.247655+0000","last_change":"2024-09-14T22:36:08.042860+0000","last_active":"2024-09-14T22:39:31.247655+0000","last_peered":"2024-09-14T22:39:31.247655+0000","last_clean":"2024-09-14T22:39:31.247655+0000","last_became_active":"2024-09-14T22:36:06.040961+0000","last_became_peered":"2024-09-14T22:36:06.040961+0000","last_unstale":"2024-09-14T22:39:31.247655+0000","last_undegraded":"2024-09-14T22:39:31.247655+0000","last_fullsized":"2024-09-14T22:39:31.247655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:03:48.547834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010780950000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"170'738","reported_seq":3855,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306469+0000","last_change":"2024-09-14T22:39:02.782844+0000","last_active":"2024-09-14T22:39:32.306469+0000","last_peered":"2024-09-14T22:39:32.306469+0000","last_clean":"2024-09-14T22:39:32.306469+0000","last_became_active":"2024-09-14T22:36:15.700982+0000","last_became_peered":"2024-09-14T22:36:15.700982+0000","last_unstale":"2024-09-14T22:39:32.306469+0000","last_undegraded":"2024-09-14T22:39:32.306469+0000","last_fullsized":"2024-09-14T22:39:32.306469+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:06:43.937383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001672663,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":4804,"num_read_kb":22802,"num_write":686,"num_write_kb":12175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":181,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149492+0000","last_change":"2024-09-14T22:39:20.885509+0000","last_active":"2024-09-14T22:39:32.149492+0000","last_peered":"2024-09-14T22:39:32.149492+0000","last_clean":"2024-09-14T22:39:32.149492+0000","last_became_active":"2024-09-14T22:39:20.885045+0000","last_became_peered":"2024-09-14T22:39:20.885045+0000","last_unstale":"2024-09-14T22:39:32.149492+0000","last_undegraded":"2024-09-14T22:39:32.149492+0000","last_fullsized":"2024-09-14T22:39:32.149492+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:37:23.145835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"170'537","reported_seq":5821,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.472368+0000","last_change":"2024-09-14T22:39:23.066067+0000","last_active":"2024-09-14T22:39:31.472368+0000","last_peered":"2024-09-14T22:39:31.472368+0000","last_clean":"2024-09-14T22:39:31.472368+0000","last_became_active":"2024-09-14T22:39:23.064710+0000","last_became_peered":"2024-09-14T22:39:23.064710+0000","last_unstale":"2024-09-14T22:39:31.472368+0000","last_undegraded":"2024-09-14T22:39:31.472368+0000","last_fullsized":"2024-09-14T22:39:31.472368+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:53:48.607465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044565630000000002,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":6110,"num_read_kb":28977,"num_write":556,"num_write_kb":8495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":356,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776747+0000","last_change":"2024-09-14T22:36:08.054606+0000","last_active":"2024-09-14T22:39:31.776747+0000","last_peered":"2024-09-14T22:39:31.776747+0000","last_clean":"2024-09-14T22:39:31.776747+0000","last_became_active":"2024-09-14T22:36:06.266908+0000","last_became_peered":"2024-09-14T22:36:06.266908+0000","last_unstale":"2024-09-14T22:39:31.776747+0000","last_undegraded":"2024-09-14T22:39:31.776747+0000","last_fullsized":"2024-09-14T22:39:31.776747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:36.738643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098249300000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"169'1480","reported_seq":5562,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149277+0000","last_change":"2024-09-14T22:39:20.753197+0000","last_active":"2024-09-14T22:39:32.149277+0000","last_peered":"2024-09-14T22:39:32.149277+0000","last_clean":"2024-09-14T22:39:32.149277+0000","last_became_active":"2024-09-14T22:39:20.751272+0000","last_became_peered":"2024-09-14T22:39:20.751272+0000","last_unstale":"2024-09-14T22:39:32.149277+0000","last_undegraded":"2024-09-14T22:39:32.149277+0000","last_fullsized":"2024-09-14T22:39:32.149277+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1480,"log_dups_size":0,"ondisk_log_size":1480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:15:46.970993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037027409999999998,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":9975,"num_read_kb":23940,"num_write":2463,"num_write_kb":17944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":4264139,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":88,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149438+0000","last_change":"2024-09-14T22:39:20.748672+0000","last_active":"2024-09-14T22:39:32.149438+0000","last_peered":"2024-09-14T22:39:32.149438+0000","last_clean":"2024-09-14T22:39:32.149438+0000","last_became_active":"2024-09-14T22:39:20.748164+0000","last_became_peered":"2024-09-14T22:39:20.748164+0000","last_unstale":"2024-09-14T22:39:32.149438+0000","last_undegraded":"2024-09-14T22:39:32.149438+0000","last_fullsized":"2024-09-14T22:39:32.149438+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:45:58.659226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"170'625","reported_seq":4878,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298433+0000","last_change":"2024-09-14T22:39:25.088610+0000","last_active":"2024-09-14T22:39:32.298433+0000","last_peered":"2024-09-14T22:39:32.298433+0000","last_clean":"2024-09-14T22:39:32.298433+0000","last_became_active":"2024-09-14T22:39:25.085794+0000","last_became_peered":"2024-09-14T22:39:25.085794+0000","last_unstale":"2024-09-14T22:39:32.298433+0000","last_undegraded":"2024-09-14T22:39:32.298433+0000","last_fullsized":"2024-09-14T22:39:32.298433+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:03:57.958236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002907598,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":5488,"num_read_kb":14305,"num_write":599,"num_write_kb":5812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":5797306,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"175'596","reported_seq":5470,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149769+0000","last_change":"2024-09-14T22:39:13.146266+0000","last_active":"2024-09-14T22:39:32.149769+0000","last_peered":"2024-09-14T22:39:32.149769+0000","last_clean":"2024-09-14T22:39:32.149769+0000","last_became_active":"2024-09-14T22:39:13.012755+0000","last_became_peered":"2024-09-14T22:39:13.012755+0000","last_unstale":"2024-09-14T22:39:32.149769+0000","last_undegraded":"2024-09-14T22:39:32.149769+0000","last_fullsized":"2024-09-14T22:39:32.149769+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:17:13.530413+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.084951515000000005,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":30,"num_read":4623,"num_read_kb":25789,"num_write":608,"num_write_kb":14115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":254,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298189+0000","last_change":"2024-09-14T22:39:25.086414+0000","last_active":"2024-09-14T22:39:32.298189+0000","last_peered":"2024-09-14T22:39:32.298189+0000","last_clean":"2024-09-14T22:39:32.298189+0000","last_became_active":"2024-09-14T22:39:25.085378+0000","last_became_peered":"2024-09-14T22:39:25.085378+0000","last_unstale":"2024-09-14T22:39:32.298189+0000","last_undegraded":"2024-09-14T22:39:32.298189+0000","last_fullsized":"2024-09-14T22:39:32.298189+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:01:23.251308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021617870000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"170'991","reported_seq":9285,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149171+0000","last_change":"2024-09-14T22:39:20.754533+0000","last_active":"2024-09-14T22:39:32.149171+0000","last_peered":"2024-09-14T22:39:32.149171+0000","last_clean":"2024-09-14T22:39:32.149171+0000","last_became_active":"2024-09-14T22:39:20.751993+0000","last_became_peered":"2024-09-14T22:39:20.751993+0000","last_unstale":"2024-09-14T22:39:32.149171+0000","last_undegraded":"2024-09-14T22:39:32.149171+0000","last_fullsized":"2024-09-14T22:39:32.149171+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":991,"log_dups_size":0,"ondisk_log_size":991,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:08.353422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030469329999999999,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":10327,"num_read_kb":29859,"num_write":1283,"num_write_kb":15017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":4851034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":291,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.178581+0000","last_change":"2024-09-14T22:38:05.724182+0000","last_active":"2024-09-14T22:39:31.178581+0000","last_peered":"2024-09-14T22:39:31.178581+0000","last_clean":"2024-09-14T22:39:31.178581+0000","last_became_active":"2024-09-14T22:38:05.723636+0000","last_became_peered":"2024-09-14T22:38:05.723636+0000","last_unstale":"2024-09-14T22:39:31.178581+0000","last_undegraded":"2024-09-14T22:39:31.178581+0000","last_fullsized":"2024-09-14T22:39:31.178581+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:14:09.687576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"175'449","reported_seq":5800,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306344+0000","last_change":"2024-09-14T22:39:13.119596+0000","last_active":"2024-09-14T22:39:32.306344+0000","last_peered":"2024-09-14T22:39:32.306344+0000","last_clean":"2024-09-14T22:39:32.306344+0000","last_became_active":"2024-09-14T22:39:13.025169+0000","last_became_peered":"2024-09-14T22:39:13.025169+0000","last_unstale":"2024-09-14T22:39:32.306344+0000","last_undegraded":"2024-09-14T22:39:32.306344+0000","last_fullsized":"2024-09-14T22:39:32.306344+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:01:44.220418+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.027619476,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":4593,"num_read_kb":23198,"num_write":420,"num_write_kb":7231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":3679238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":53,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149252+0000","last_change":"2024-09-14T22:39:13.009681+0000","last_active":"2024-09-14T22:39:32.149252+0000","last_peered":"2024-09-14T22:39:32.149252+0000","last_clean":"2024-09-14T22:39:32.149252+0000","last_became_active":"2024-09-14T22:39:13.008709+0000","last_became_peered":"2024-09-14T22:39:13.008709+0000","last_unstale":"2024-09-14T22:39:32.149252+0000","last_undegraded":"2024-09-14T22:39:32.149252+0000","last_fullsized":"2024-09-14T22:39:32.149252+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:14:31.162323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010817520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"169'1116","reported_seq":4770,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.472132+0000","last_change":"2024-09-14T22:39:23.066824+0000","last_active":"2024-09-14T22:39:31.472132+0000","last_peered":"2024-09-14T22:39:31.472132+0000","last_clean":"2024-09-14T22:39:31.472132+0000","last_became_active":"2024-09-14T22:39:23.064694+0000","last_became_peered":"2024-09-14T22:39:23.064694+0000","last_unstale":"2024-09-14T22:39:31.472132+0000","last_undegraded":"2024-09-14T22:39:31.472132+0000","last_fullsized":"2024-09-14T22:39:31.472132+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:32:09.848042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033884660000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":7515,"num_read_kb":16135,"num_write":1577,"num_write_kb":11855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":2936757,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":260,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298203+0000","last_change":"2024-09-14T22:39:25.082725+0000","last_active":"2024-09-14T22:39:32.298203+0000","last_peered":"2024-09-14T22:39:32.298203+0000","last_clean":"2024-09-14T22:39:32.298203+0000","last_became_active":"2024-09-14T22:39:25.081686+0000","last_became_peered":"2024-09-14T22:39:25.081686+0000","last_unstale":"2024-09-14T22:39:32.298203+0000","last_undegraded":"2024-09-14T22:39:32.298203+0000","last_fullsized":"2024-09-14T22:39:32.298203+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:35:39.626521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011401880000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":507,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776344+0000","last_change":"2024-09-14T22:36:00.402862+0000","last_active":"2024-09-14T22:39:31.776344+0000","last_peered":"2024-09-14T22:39:31.776344+0000","last_clean":"2024-09-14T22:39:31.776344+0000","last_became_active":"2024-09-14T22:36:00.400529+0000","last_became_peered":"2024-09-14T22:36:00.400529+0000","last_unstale":"2024-09-14T22:39:31.776344+0000","last_undegraded":"2024-09-14T22:39:31.776344+0000","last_fullsized":"2024-09-14T22:39:31.776344+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_clean_scrub_stamp":"2024-09-14T22:35:58.989025+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:07:12.951728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"169'676","reported_seq":7829,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.472125+0000","last_change":"2024-09-14T22:39:02.769315+0000","last_active":"2024-09-14T22:39:31.472125+0000","last_peered":"2024-09-14T22:39:31.472125+0000","last_clean":"2024-09-14T22:39:31.472125+0000","last_became_active":"2024-09-14T22:38:08.739090+0000","last_became_peered":"2024-09-14T22:38:08.739090+0000","last_unstale":"2024-09-14T22:39:31.472125+0000","last_undegraded":"2024-09-14T22:39:31.472125+0000","last_fullsized":"2024-09-14T22:39:31.472125+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:30:42.554797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012849739999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":8821,"num_read_kb":22101,"num_write":951,"num_write_kb":14470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":18690892,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"175'461","reported_seq":7414,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149505+0000","last_change":"2024-09-14T22:39:13.119552+0000","last_active":"2024-09-14T22:39:32.149505+0000","last_peered":"2024-09-14T22:39:32.149505+0000","last_clean":"2024-09-14T22:39:32.149505+0000","last_became_active":"2024-09-14T22:39:13.008690+0000","last_became_peered":"2024-09-14T22:39:13.008690+0000","last_unstale":"2024-09-14T22:39:32.149505+0000","last_undegraded":"2024-09-14T22:39:32.149505+0000","last_fullsized":"2024-09-14T22:39:32.149505+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:02:56.318281+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.064091727000000001,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":7056,"num_read_kb":18679,"num_write":421,"num_write_kb":7008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":16527933,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"175'803","reported_seq":4494,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.247652+0000","last_change":"2024-09-14T22:39:13.078772+0000","last_active":"2024-09-14T22:39:31.247652+0000","last_peered":"2024-09-14T22:39:31.247652+0000","last_clean":"2024-09-14T22:39:31.247652+0000","last_became_active":"2024-09-14T22:39:13.024372+0000","last_became_peered":"2024-09-14T22:39:13.024372+0000","last_unstale":"2024-09-14T22:39:31.247652+0000","last_undegraded":"2024-09-14T22:39:31.247652+0000","last_fullsized":"2024-09-14T22:39:31.247652+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:50.926795+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.037620476,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":28,"num_read":6415,"num_read_kb":26025,"num_write":756,"num_write_kb":14809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"170'583","reported_seq":2961,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.472227+0000","last_change":"2024-09-14T22:39:02.771485+0000","last_active":"2024-09-14T22:39:31.472227+0000","last_peered":"2024-09-14T22:39:31.472227+0000","last_clean":"2024-09-14T22:39:31.472227+0000","last_became_active":"2024-09-14T22:36:15.702885+0000","last_became_peered":"2024-09-14T22:36:15.702885+0000","last_unstale":"2024-09-14T22:39:31.472227+0000","last_undegraded":"2024-09-14T22:39:31.472227+0000","last_fullsized":"2024-09-14T22:39:31.472227+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:59:16.704705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031173400000000001,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":3570,"num_read_kb":11670,"num_write":536,"num_write_kb":6877,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"170'447","reported_seq":6923,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298274+0000","last_change":"2024-09-14T22:39:25.090361+0000","last_active":"2024-09-14T22:39:32.298274+0000","last_peered":"2024-09-14T22:39:32.298274+0000","last_clean":"2024-09-14T22:39:32.298274+0000","last_became_active":"2024-09-14T22:39:25.088012+0000","last_became_peered":"2024-09-14T22:39:25.088012+0000","last_unstale":"2024-09-14T22:39:32.298274+0000","last_undegraded":"2024-09-14T22:39:32.298274+0000","last_fullsized":"2024-09-14T22:39:32.298274+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:10:33.661519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018646120000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5808,"num_read_kb":33509,"num_write":486,"num_write_kb":17853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":23218688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"168'647","reported_seq":4232,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776747+0000","last_change":"2024-09-14T22:39:02.868251+0000","last_active":"2024-09-14T22:39:31.776747+0000","last_peered":"2024-09-14T22:39:31.776747+0000","last_clean":"2024-09-14T22:39:31.776747+0000","last_became_active":"2024-09-14T22:36:15.723205+0000","last_became_peered":"2024-09-14T22:36:15.723205+0000","last_unstale":"2024-09-14T22:39:31.776747+0000","last_undegraded":"2024-09-14T22:39:31.776747+0000","last_fullsized":"2024-09-14T22:39:31.776747+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:13:11.920909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001862636,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":4348,"num_read_kb":16212,"num_write":775,"num_write_kb":5956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"175'549","reported_seq":2917,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776796+0000","last_change":"2024-09-14T22:39:13.093929+0000","last_active":"2024-09-14T22:39:31.776796+0000","last_peered":"2024-09-14T22:39:31.776796+0000","last_clean":"2024-09-14T22:39:31.776796+0000","last_became_active":"2024-09-14T22:39:13.025887+0000","last_became_peered":"2024-09-14T22:39:13.025887+0000","last_unstale":"2024-09-14T22:39:31.776796+0000","last_undegraded":"2024-09-14T22:39:31.776796+0000","last_fullsized":"2024-09-14T22:39:31.776796+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:31:08.644424+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.030321287999999998,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":2666,"num_read_kb":8490,"num_write":594,"num_write_kb":3830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":4944418,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"175'499","reported_seq":8501,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776826+0000","last_change":"2024-09-14T22:39:13.145388+0000","last_active":"2024-09-14T22:39:31.776826+0000","last_peered":"2024-09-14T22:39:31.776826+0000","last_clean":"2024-09-14T22:39:31.776826+0000","last_became_active":"2024-09-14T22:39:13.030456+0000","last_became_peered":"2024-09-14T22:39:13.030456+0000","last_unstale":"2024-09-14T22:39:31.776826+0000","last_undegraded":"2024-09-14T22:39:31.776826+0000","last_fullsized":"2024-09-14T22:39:31.776826+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:10:30.405497+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.017930723999999999,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":8487,"num_read_kb":23501,"num_write":537,"num_write_kb":9535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"168'493","reported_seq":4108,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.247573+0000","last_change":"2024-09-14T22:39:02.866134+0000","last_active":"2024-09-14T22:39:31.247573+0000","last_peered":"2024-09-14T22:39:31.247573+0000","last_clean":"2024-09-14T22:39:31.247573+0000","last_became_active":"2024-09-14T22:37:26.717088+0000","last_became_peered":"2024-09-14T22:37:26.717088+0000","last_unstale":"2024-09-14T22:39:31.247573+0000","last_undegraded":"2024-09-14T22:39:31.247573+0000","last_fullsized":"2024-09-14T22:39:31.247573+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:03:51.158429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024508580000000002,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":3408,"num_read_kb":29082,"num_write":535,"num_write_kb":14817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"175'685","reported_seq":2731,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776343+0000","last_change":"2024-09-14T22:39:13.123794+0000","last_active":"2024-09-14T22:39:31.776343+0000","last_peered":"2024-09-14T22:39:31.776343+0000","last_clean":"2024-09-14T22:39:31.776343+0000","last_became_active":"2024-09-14T22:39:13.029544+0000","last_became_peered":"2024-09-14T22:39:13.029544+0000","last_unstale":"2024-09-14T22:39:31.776343+0000","last_undegraded":"2024-09-14T22:39:31.776343+0000","last_fullsized":"2024-09-14T22:39:31.776343+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:47:09.608401+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.020111626000000001,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":3515,"num_read_kb":11729,"num_write":870,"num_write_kb":10937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"168'589","reported_seq":8731,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306353+0000","last_change":"2024-09-14T22:39:02.782999+0000","last_active":"2024-09-14T22:39:32.306353+0000","last_peered":"2024-09-14T22:39:32.306353+0000","last_clean":"2024-09-14T22:39:32.306353+0000","last_became_active":"2024-09-14T22:36:15.716696+0000","last_became_peered":"2024-09-14T22:36:15.716696+0000","last_unstale":"2024-09-14T22:39:32.306353+0000","last_undegraded":"2024-09-14T22:39:32.306353+0000","last_fullsized":"2024-09-14T22:39:32.306353+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:14:34.437495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019004219999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":9756,"num_read_kb":50618,"num_write":568,"num_write_kb":13171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"175'599","reported_seq":7681,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776429+0000","last_change":"2024-09-14T22:39:13.167257+0000","last_active":"2024-09-14T22:39:31.776429+0000","last_peered":"2024-09-14T22:39:31.776429+0000","last_clean":"2024-09-14T22:39:31.776429+0000","last_became_active":"2024-09-14T22:39:13.030502+0000","last_became_peered":"2024-09-14T22:39:13.030502+0000","last_unstale":"2024-09-14T22:39:31.776429+0000","last_undegraded":"2024-09-14T22:39:31.776429+0000","last_fullsized":"2024-09-14T22:39:31.776429+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:35:32.951904+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.023146856,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":7147,"num_read_kb":35324,"num_write":610,"num_write_kb":15494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":14511090,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":893,"num_object_clones":0,"num_object_copies":1786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":893,"num_whiteouts":889,"num_read":206686,"num_read_kb":828988,"num_write":25214,"num_write_kb":381837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3336,"num_bytes_recovered":173602381,"num_keys_recovered":342,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22322,"internal_metadata":0},"log_size":21573,"ondisk_log_size":21573,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542191,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59848,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311992,"statfs":{"total":96636764160,"available":96575479808,"internally_reserved":0,"allocated":2347008,"data_stored":1693510,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542192,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45904,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325936,"statfs":{"total":96636764160,"available":96589758464,"internally_reserved":0,"allocated":2289664,"data_stored":1103142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542192,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55652,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316188,"statfs":{"total":96636764160,"available":96579776512,"internally_reserved":0,"allocated":2306048,"data_stored":1109032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":174,"seq":747324309510,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":2252,"kb_used_omap":34,"kb_used_meta":29277,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":2306048,"data_stored":1109032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35258,"internal_metadata":29980230},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542192,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51472,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320368,"statfs":{"total":96636764160,"available":96584056832,"internally_reserved":0,"allocated":2289664,"data_stored":1103142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":66,"seq":283467841566,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43220,"kb_used_data":2236,"kb_used_omap":6,"kb_used_meta":40953,"kb_avail":94328620,"statfs":{"total":96636764160,"available":96592506880,"internally_reserved":0,"allocated":2289664,"data_stored":1103142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7025,"internal_metadata":41936015},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59816,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312024,"statfs":{"total":96636764160,"available":96575512576,"internally_reserved":0,"allocated":2310144,"data_stored":1109051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542191,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41816,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330024,"statfs":{"total":96636764160,"available":96593944576,"internally_reserved":0,"allocated":2359296,"data_stored":1699381,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4238,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18084,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:39:36.911 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-14T22:39:36.912 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:39:37.198 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:39:37.198 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-14T22:39:37.215 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-14T22:39:36.920151+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":897,"num_object_clones":0,"num_object_copies":1794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":897,"num_whiteouts":889,"num_read":206802,"num_read_kb":829088,"num_write":25375,"num_write_kb":384255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3336,"num_bytes_recovered":173602381,"num_keys_recovered":342,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21693,"ondisk_log_size":21693,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":389320,"kb_used_data":18076,"kb_used_omap":50,"kb_used_meta":371021,"kb_avail":754585400,"statfs":{"total":773094113280,"available":772695449600,"internally_reserved":0,"allocated":18509824,"data_stored":10035303,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":51845,"internal_metadata":379925883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.798182"},"pg_stats":[{"pgid":"3.1f","version":"170'491","reported_seq":5322,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306474+0000","last_change":"2024-09-14T22:39:02.783678+0000","last_active":"2024-09-14T22:39:32.306474+0000","last_peered":"2024-09-14T22:39:32.306474+0000","last_clean":"2024-09-14T22:39:32.306474+0000","last_became_active":"2024-09-14T22:37:25.718906+0000","last_became_peered":"2024-09-14T22:37:25.718906+0000","last_unstale":"2024-09-14T22:39:32.306474+0000","last_undegraded":"2024-09-14T22:39:32.306474+0000","last_fullsized":"2024-09-14T22:39:32.306474+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:16:07.317648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023103960000000002,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":5019,"num_read_kb":16516,"num_write":439,"num_write_kb":7121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":3277632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"170'1231","reported_seq":7362,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.472426+0000","last_change":"2024-09-14T22:39:02.771231+0000","last_active":"2024-09-14T22:39:31.472426+0000","last_peered":"2024-09-14T22:39:31.472426+0000","last_clean":"2024-09-14T22:39:31.472426+0000","last_became_active":"2024-09-14T22:38:08.741149+0000","last_became_peered":"2024-09-14T22:38:08.741149+0000","last_unstale":"2024-09-14T22:39:31.472426+0000","last_undegraded":"2024-09-14T22:39:31.472426+0000","last_fullsized":"2024-09-14T22:39:31.472426+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1231,"log_dups_size":0,"ondisk_log_size":1231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:32:08.194278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024840370000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":11835,"num_read_kb":34663,"num_write":1722,"num_write_kb":10763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":7412288,"num_keys_recovered":90,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"168'724","reported_seq":10563,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149517+0000","last_change":"2024-09-14T22:39:20.753911+0000","last_active":"2024-09-14T22:39:32.149517+0000","last_peered":"2024-09-14T22:39:32.149517+0000","last_clean":"2024-09-14T22:39:32.149517+0000","last_became_active":"2024-09-14T22:39:20.751505+0000","last_became_peered":"2024-09-14T22:39:20.751505+0000","last_unstale":"2024-09-14T22:39:32.149517+0000","last_undegraded":"2024-09-14T22:39:32.149517+0000","last_fullsized":"2024-09-14T22:39:32.149517+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:17:01.639166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0083529999999999993,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":10932,"num_read_kb":63225,"num_write":868,"num_write_kb":13703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":8030815,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"175'718","reported_seq":6436,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298203+0000","last_change":"2024-09-14T22:39:25.087327+0000","last_active":"2024-09-14T22:39:32.298203+0000","last_peered":"2024-09-14T22:39:32.298203+0000","last_clean":"2024-09-14T22:39:32.298203+0000","last_became_active":"2024-09-14T22:39:25.084122+0000","last_became_peered":"2024-09-14T22:39:25.084122+0000","last_unstale":"2024-09-14T22:39:32.298203+0000","last_undegraded":"2024-09-14T22:39:32.298203+0000","last_fullsized":"2024-09-14T22:39:32.298203+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:17:17.035425+0000","scrub_duration":0,"objects_trimmed":20,"snaptrim_duration":0.055023557000000001,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":34,"num_read":6435,"num_read_kb":26956,"num_write":884,"num_write_kb":8982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"175'602","reported_seq":5586,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306492+0000","last_change":"2024-09-14T22:39:13.115776+0000","last_active":"2024-09-14T22:39:32.306492+0000","last_peered":"2024-09-14T22:39:32.306492+0000","last_clean":"2024-09-14T22:39:32.306492+0000","last_became_active":"2024-09-14T22:39:13.025036+0000","last_became_peered":"2024-09-14T22:39:13.025036+0000","last_unstale":"2024-09-14T22:39:32.306492+0000","last_undegraded":"2024-09-14T22:39:32.306492+0000","last_fullsized":"2024-09-14T22:39:32.306492+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:23:31.541475+0000","scrub_duration":0,"objects_trimmed":14,"snaptrim_duration":0.061855755999999998,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":5663,"num_read_kb":32434,"num_write":648,"num_write_kb":13330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":7858104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"175'527","reported_seq":4225,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149467+0000","last_change":"2024-09-14T22:39:13.150950+0000","last_active":"2024-09-14T22:39:32.149467+0000","last_peered":"2024-09-14T22:39:32.149467+0000","last_clean":"2024-09-14T22:39:32.149467+0000","last_became_active":"2024-09-14T22:39:13.016132+0000","last_became_peered":"2024-09-14T22:39:13.016132+0000","last_unstale":"2024-09-14T22:39:32.149467+0000","last_undegraded":"2024-09-14T22:39:32.149467+0000","last_fullsized":"2024-09-14T22:39:32.149467+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:34:37.241298+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.069063225000000006,"stat_sum":{"num_bytes":19,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":33,"num_read":4551,"num_read_kb":13173,"num_write":474,"num_write_kb":7035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":236,"num_bytes_recovered":10036538,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"169'465","reported_seq":5658,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149161+0000","last_change":"2024-09-14T22:39:02.778427+0000","last_active":"2024-09-14T22:39:32.149161+0000","last_peered":"2024-09-14T22:39:32.149161+0000","last_clean":"2024-09-14T22:39:32.149161+0000","last_became_active":"2024-09-14T22:36:15.700990+0000","last_became_peered":"2024-09-14T22:36:15.700990+0000","last_unstale":"2024-09-14T22:39:32.149161+0000","last_undegraded":"2024-09-14T22:39:32.149161+0000","last_fullsized":"2024-09-14T22:39:32.149161+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:24:53.327956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013968780000000001,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":5273,"num_read_kb":15380,"num_write":425,"num_write_kb":5486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"170'511","reported_seq":4995,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.472029+0000","last_change":"2024-09-14T22:39:02.769340+0000","last_active":"2024-09-14T22:39:31.472029+0000","last_peered":"2024-09-14T22:39:31.472029+0000","last_clean":"2024-09-14T22:39:31.472029+0000","last_became_active":"2024-09-14T22:38:07.734308+0000","last_became_peered":"2024-09-14T22:38:07.734308+0000","last_unstale":"2024-09-14T22:39:31.472029+0000","last_undegraded":"2024-09-14T22:39:31.472029+0000","last_fullsized":"2024-09-14T22:39:31.472029+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:12:44.546869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013730859999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4941,"num_read_kb":35923,"num_write":636,"num_write_kb":27736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":20067293,"num_keys_recovered":21,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"167'467","reported_seq":4865,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306510+0000","last_change":"2024-09-14T22:39:02.784093+0000","last_active":"2024-09-14T22:39:32.306510+0000","last_peered":"2024-09-14T22:39:32.306510+0000","last_clean":"2024-09-14T22:39:32.306510+0000","last_became_active":"2024-09-14T22:36:15.715869+0000","last_became_peered":"2024-09-14T22:36:15.715869+0000","last_unstale":"2024-09-14T22:39:32.306510+0000","last_undegraded":"2024-09-14T22:39:32.306510+0000","last_fullsized":"2024-09-14T22:39:32.306510+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:57:02.149610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023931830000000001,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":4593,"num_read_kb":25142,"num_write":516,"num_write_kb":22003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"166'605","reported_seq":6669,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298379+0000","last_change":"2024-09-14T22:39:02.774376+0000","last_active":"2024-09-14T22:39:32.298379+0000","last_peered":"2024-09-14T22:39:32.298379+0000","last_clean":"2024-09-14T22:39:32.298379+0000","last_became_active":"2024-09-14T22:36:15.725078+0000","last_became_peered":"2024-09-14T22:36:15.725078+0000","last_unstale":"2024-09-14T22:39:32.298379+0000","last_undegraded":"2024-09-14T22:39:32.298379+0000","last_fullsized":"2024-09-14T22:39:32.298379+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:26:12.926170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035779840000000002,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":6675,"num_read_kb":51224,"num_write":729,"num_write_kb":18952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"167'467","reported_seq":4804,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776443+0000","last_change":"2024-09-14T22:39:02.867529+0000","last_active":"2024-09-14T22:39:31.776443+0000","last_peered":"2024-09-14T22:39:31.776443+0000","last_clean":"2024-09-14T22:39:31.776443+0000","last_became_active":"2024-09-14T22:36:15.717341+0000","last_became_peered":"2024-09-14T22:36:15.717341+0000","last_unstale":"2024-09-14T22:39:31.776443+0000","last_undegraded":"2024-09-14T22:39:31.776443+0000","last_fullsized":"2024-09-14T22:39:31.776443+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:29:47.296331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016388500000000001,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":5090,"num_read_kb":12789,"num_write":419,"num_write_kb":7375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"175'1202","reported_seq":9124,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149855+0000","last_change":"2024-09-14T22:39:13.160248+0000","last_active":"2024-09-14T22:39:32.149855+0000","last_peered":"2024-09-14T22:39:32.149855+0000","last_clean":"2024-09-14T22:39:32.149855+0000","last_became_active":"2024-09-14T22:39:13.026842+0000","last_became_peered":"2024-09-14T22:39:13.026842+0000","last_unstale":"2024-09-14T22:39:32.149855+0000","last_undegraded":"2024-09-14T22:39:32.149855+0000","last_fullsized":"2024-09-14T22:39:32.149855+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1202,"log_dups_size":0,"ondisk_log_size":1202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:06:37.741547+0000","scrub_duration":0,"objects_trimmed":14,"snaptrim_duration":0.055942110000000003,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":11247,"num_read_kb":29618,"num_write":1623,"num_write_kb":11950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":265,"num_bytes_recovered":17498216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":354,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.247655+0000","last_change":"2024-09-14T22:36:08.042860+0000","last_active":"2024-09-14T22:39:31.247655+0000","last_peered":"2024-09-14T22:39:31.247655+0000","last_clean":"2024-09-14T22:39:31.247655+0000","last_became_active":"2024-09-14T22:36:06.040961+0000","last_became_peered":"2024-09-14T22:36:06.040961+0000","last_unstale":"2024-09-14T22:39:31.247655+0000","last_undegraded":"2024-09-14T22:39:31.247655+0000","last_fullsized":"2024-09-14T22:39:31.247655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:03:48.547834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010780950000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"170'738","reported_seq":3855,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306469+0000","last_change":"2024-09-14T22:39:02.782844+0000","last_active":"2024-09-14T22:39:32.306469+0000","last_peered":"2024-09-14T22:39:32.306469+0000","last_clean":"2024-09-14T22:39:32.306469+0000","last_became_active":"2024-09-14T22:36:15.700982+0000","last_became_peered":"2024-09-14T22:36:15.700982+0000","last_unstale":"2024-09-14T22:39:32.306469+0000","last_undegraded":"2024-09-14T22:39:32.306469+0000","last_fullsized":"2024-09-14T22:39:32.306469+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:06:43.937383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001672663,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":4804,"num_read_kb":22802,"num_write":686,"num_write_kb":12175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":181,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149492+0000","last_change":"2024-09-14T22:39:20.885509+0000","last_active":"2024-09-14T22:39:32.149492+0000","last_peered":"2024-09-14T22:39:32.149492+0000","last_clean":"2024-09-14T22:39:32.149492+0000","last_became_active":"2024-09-14T22:39:20.885045+0000","last_became_peered":"2024-09-14T22:39:20.885045+0000","last_unstale":"2024-09-14T22:39:32.149492+0000","last_undegraded":"2024-09-14T22:39:32.149492+0000","last_fullsized":"2024-09-14T22:39:32.149492+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:37:23.145835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"170'537","reported_seq":5821,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.472368+0000","last_change":"2024-09-14T22:39:23.066067+0000","last_active":"2024-09-14T22:39:31.472368+0000","last_peered":"2024-09-14T22:39:31.472368+0000","last_clean":"2024-09-14T22:39:31.472368+0000","last_became_active":"2024-09-14T22:39:23.064710+0000","last_became_peered":"2024-09-14T22:39:23.064710+0000","last_unstale":"2024-09-14T22:39:31.472368+0000","last_undegraded":"2024-09-14T22:39:31.472368+0000","last_fullsized":"2024-09-14T22:39:31.472368+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:53:48.607465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044565630000000002,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":6110,"num_read_kb":28977,"num_write":556,"num_write_kb":8495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":356,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776747+0000","last_change":"2024-09-14T22:36:08.054606+0000","last_active":"2024-09-14T22:39:31.776747+0000","last_peered":"2024-09-14T22:39:31.776747+0000","last_clean":"2024-09-14T22:39:31.776747+0000","last_became_active":"2024-09-14T22:36:06.266908+0000","last_became_peered":"2024-09-14T22:36:06.266908+0000","last_unstale":"2024-09-14T22:39:31.776747+0000","last_undegraded":"2024-09-14T22:39:31.776747+0000","last_fullsized":"2024-09-14T22:39:31.776747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:36.738643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098249300000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"169'1480","reported_seq":5562,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149277+0000","last_change":"2024-09-14T22:39:20.753197+0000","last_active":"2024-09-14T22:39:32.149277+0000","last_peered":"2024-09-14T22:39:32.149277+0000","last_clean":"2024-09-14T22:39:32.149277+0000","last_became_active":"2024-09-14T22:39:20.751272+0000","last_became_peered":"2024-09-14T22:39:20.751272+0000","last_unstale":"2024-09-14T22:39:32.149277+0000","last_undegraded":"2024-09-14T22:39:32.149277+0000","last_fullsized":"2024-09-14T22:39:32.149277+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1480,"log_dups_size":0,"ondisk_log_size":1480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:15:46.970993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037027409999999998,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":9975,"num_read_kb":23940,"num_write":2463,"num_write_kb":17944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":4264139,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":88,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149438+0000","last_change":"2024-09-14T22:39:20.748672+0000","last_active":"2024-09-14T22:39:32.149438+0000","last_peered":"2024-09-14T22:39:32.149438+0000","last_clean":"2024-09-14T22:39:32.149438+0000","last_became_active":"2024-09-14T22:39:20.748164+0000","last_became_peered":"2024-09-14T22:39:20.748164+0000","last_unstale":"2024-09-14T22:39:32.149438+0000","last_undegraded":"2024-09-14T22:39:32.149438+0000","last_fullsized":"2024-09-14T22:39:32.149438+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:45:58.659226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"170'625","reported_seq":4878,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298433+0000","last_change":"2024-09-14T22:39:25.088610+0000","last_active":"2024-09-14T22:39:32.298433+0000","last_peered":"2024-09-14T22:39:32.298433+0000","last_clean":"2024-09-14T22:39:32.298433+0000","last_became_active":"2024-09-14T22:39:25.085794+0000","last_became_peered":"2024-09-14T22:39:25.085794+0000","last_unstale":"2024-09-14T22:39:32.298433+0000","last_undegraded":"2024-09-14T22:39:32.298433+0000","last_fullsized":"2024-09-14T22:39:32.298433+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:03:57.958236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002907598,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":5488,"num_read_kb":14305,"num_write":599,"num_write_kb":5812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":5797306,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"175'596","reported_seq":5470,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149769+0000","last_change":"2024-09-14T22:39:13.146266+0000","last_active":"2024-09-14T22:39:32.149769+0000","last_peered":"2024-09-14T22:39:32.149769+0000","last_clean":"2024-09-14T22:39:32.149769+0000","last_became_active":"2024-09-14T22:39:13.012755+0000","last_became_peered":"2024-09-14T22:39:13.012755+0000","last_unstale":"2024-09-14T22:39:32.149769+0000","last_undegraded":"2024-09-14T22:39:32.149769+0000","last_fullsized":"2024-09-14T22:39:32.149769+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:17:13.530413+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.084951515000000005,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":30,"num_read":4623,"num_read_kb":25789,"num_write":608,"num_write_kb":14115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":254,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298189+0000","last_change":"2024-09-14T22:39:25.086414+0000","last_active":"2024-09-14T22:39:32.298189+0000","last_peered":"2024-09-14T22:39:32.298189+0000","last_clean":"2024-09-14T22:39:32.298189+0000","last_became_active":"2024-09-14T22:39:25.085378+0000","last_became_peered":"2024-09-14T22:39:25.085378+0000","last_unstale":"2024-09-14T22:39:32.298189+0000","last_undegraded":"2024-09-14T22:39:32.298189+0000","last_fullsized":"2024-09-14T22:39:32.298189+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:01:23.251308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021617870000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"170'991","reported_seq":9285,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149171+0000","last_change":"2024-09-14T22:39:20.754533+0000","last_active":"2024-09-14T22:39:32.149171+0000","last_peered":"2024-09-14T22:39:32.149171+0000","last_clean":"2024-09-14T22:39:32.149171+0000","last_became_active":"2024-09-14T22:39:20.751993+0000","last_became_peered":"2024-09-14T22:39:20.751993+0000","last_unstale":"2024-09-14T22:39:32.149171+0000","last_undegraded":"2024-09-14T22:39:32.149171+0000","last_fullsized":"2024-09-14T22:39:32.149171+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":991,"log_dups_size":0,"ondisk_log_size":991,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:08.353422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030469329999999999,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":10327,"num_read_kb":29859,"num_write":1283,"num_write_kb":15017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":4851034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":293,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:33.347822+0000","last_change":"2024-09-14T22:38:05.724182+0000","last_active":"2024-09-14T22:39:33.347822+0000","last_peered":"2024-09-14T22:39:33.347822+0000","last_clean":"2024-09-14T22:39:33.347822+0000","last_became_active":"2024-09-14T22:38:05.723636+0000","last_became_peered":"2024-09-14T22:38:05.723636+0000","last_unstale":"2024-09-14T22:39:33.347822+0000","last_undegraded":"2024-09-14T22:39:33.347822+0000","last_fullsized":"2024-09-14T22:39:33.347822+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:14:09.687576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"175'449","reported_seq":5800,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306344+0000","last_change":"2024-09-14T22:39:13.119596+0000","last_active":"2024-09-14T22:39:32.306344+0000","last_peered":"2024-09-14T22:39:32.306344+0000","last_clean":"2024-09-14T22:39:32.306344+0000","last_became_active":"2024-09-14T22:39:13.025169+0000","last_became_peered":"2024-09-14T22:39:13.025169+0000","last_unstale":"2024-09-14T22:39:32.306344+0000","last_undegraded":"2024-09-14T22:39:32.306344+0000","last_fullsized":"2024-09-14T22:39:32.306344+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:01:44.220418+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.027619476,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":4593,"num_read_kb":23198,"num_write":420,"num_write_kb":7231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":3679238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":53,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149252+0000","last_change":"2024-09-14T22:39:13.009681+0000","last_active":"2024-09-14T22:39:32.149252+0000","last_peered":"2024-09-14T22:39:32.149252+0000","last_clean":"2024-09-14T22:39:32.149252+0000","last_became_active":"2024-09-14T22:39:13.008709+0000","last_became_peered":"2024-09-14T22:39:13.008709+0000","last_unstale":"2024-09-14T22:39:32.149252+0000","last_undegraded":"2024-09-14T22:39:32.149252+0000","last_fullsized":"2024-09-14T22:39:32.149252+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:14:31.162323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010817520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"169'1116","reported_seq":4770,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.472132+0000","last_change":"2024-09-14T22:39:23.066824+0000","last_active":"2024-09-14T22:39:31.472132+0000","last_peered":"2024-09-14T22:39:31.472132+0000","last_clean":"2024-09-14T22:39:31.472132+0000","last_became_active":"2024-09-14T22:39:23.064694+0000","last_became_peered":"2024-09-14T22:39:23.064694+0000","last_unstale":"2024-09-14T22:39:31.472132+0000","last_undegraded":"2024-09-14T22:39:31.472132+0000","last_fullsized":"2024-09-14T22:39:31.472132+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:32:09.848042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033884660000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":7515,"num_read_kb":16135,"num_write":1577,"num_write_kb":11855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":2936757,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":260,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298203+0000","last_change":"2024-09-14T22:39:25.082725+0000","last_active":"2024-09-14T22:39:32.298203+0000","last_peered":"2024-09-14T22:39:32.298203+0000","last_clean":"2024-09-14T22:39:32.298203+0000","last_became_active":"2024-09-14T22:39:25.081686+0000","last_became_peered":"2024-09-14T22:39:25.081686+0000","last_unstale":"2024-09-14T22:39:32.298203+0000","last_undegraded":"2024-09-14T22:39:32.298203+0000","last_fullsized":"2024-09-14T22:39:32.298203+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:35:39.626521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011401880000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":507,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776344+0000","last_change":"2024-09-14T22:36:00.402862+0000","last_active":"2024-09-14T22:39:31.776344+0000","last_peered":"2024-09-14T22:39:31.776344+0000","last_clean":"2024-09-14T22:39:31.776344+0000","last_became_active":"2024-09-14T22:36:00.400529+0000","last_became_peered":"2024-09-14T22:36:00.400529+0000","last_unstale":"2024-09-14T22:39:31.776344+0000","last_undegraded":"2024-09-14T22:39:31.776344+0000","last_fullsized":"2024-09-14T22:39:31.776344+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_clean_scrub_stamp":"2024-09-14T22:35:58.989025+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:07:12.951728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"169'676","reported_seq":7829,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.472125+0000","last_change":"2024-09-14T22:39:02.769315+0000","last_active":"2024-09-14T22:39:31.472125+0000","last_peered":"2024-09-14T22:39:31.472125+0000","last_clean":"2024-09-14T22:39:31.472125+0000","last_became_active":"2024-09-14T22:38:08.739090+0000","last_became_peered":"2024-09-14T22:38:08.739090+0000","last_unstale":"2024-09-14T22:39:31.472125+0000","last_undegraded":"2024-09-14T22:39:31.472125+0000","last_fullsized":"2024-09-14T22:39:31.472125+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:30:42.554797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012849739999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":8821,"num_read_kb":22101,"num_write":951,"num_write_kb":14470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":18690892,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"175'461","reported_seq":7414,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149505+0000","last_change":"2024-09-14T22:39:13.119552+0000","last_active":"2024-09-14T22:39:32.149505+0000","last_peered":"2024-09-14T22:39:32.149505+0000","last_clean":"2024-09-14T22:39:32.149505+0000","last_became_active":"2024-09-14T22:39:13.008690+0000","last_became_peered":"2024-09-14T22:39:13.008690+0000","last_unstale":"2024-09-14T22:39:32.149505+0000","last_undegraded":"2024-09-14T22:39:32.149505+0000","last_fullsized":"2024-09-14T22:39:32.149505+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:02:56.318281+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.064091727000000001,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":7056,"num_read_kb":18679,"num_write":421,"num_write_kb":7008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":16527933,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"175'803","reported_seq":4494,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.247652+0000","last_change":"2024-09-14T22:39:13.078772+0000","last_active":"2024-09-14T22:39:31.247652+0000","last_peered":"2024-09-14T22:39:31.247652+0000","last_clean":"2024-09-14T22:39:31.247652+0000","last_became_active":"2024-09-14T22:39:13.024372+0000","last_became_peered":"2024-09-14T22:39:13.024372+0000","last_unstale":"2024-09-14T22:39:31.247652+0000","last_undegraded":"2024-09-14T22:39:31.247652+0000","last_fullsized":"2024-09-14T22:39:31.247652+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:50.926795+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.037620476,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":28,"num_read":6415,"num_read_kb":26025,"num_write":756,"num_write_kb":14809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"170'583","reported_seq":2961,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.472227+0000","last_change":"2024-09-14T22:39:02.771485+0000","last_active":"2024-09-14T22:39:31.472227+0000","last_peered":"2024-09-14T22:39:31.472227+0000","last_clean":"2024-09-14T22:39:31.472227+0000","last_became_active":"2024-09-14T22:36:15.702885+0000","last_became_peered":"2024-09-14T22:36:15.702885+0000","last_unstale":"2024-09-14T22:39:31.472227+0000","last_undegraded":"2024-09-14T22:39:31.472227+0000","last_fullsized":"2024-09-14T22:39:31.472227+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:59:16.704705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031173400000000001,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":3570,"num_read_kb":11670,"num_write":536,"num_write_kb":6877,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"170'447","reported_seq":6923,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298274+0000","last_change":"2024-09-14T22:39:25.090361+0000","last_active":"2024-09-14T22:39:32.298274+0000","last_peered":"2024-09-14T22:39:32.298274+0000","last_clean":"2024-09-14T22:39:32.298274+0000","last_became_active":"2024-09-14T22:39:25.088012+0000","last_became_peered":"2024-09-14T22:39:25.088012+0000","last_unstale":"2024-09-14T22:39:32.298274+0000","last_undegraded":"2024-09-14T22:39:32.298274+0000","last_fullsized":"2024-09-14T22:39:32.298274+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:10:33.661519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018646120000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5808,"num_read_kb":33509,"num_write":486,"num_write_kb":17853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":23218688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"168'647","reported_seq":4232,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776747+0000","last_change":"2024-09-14T22:39:02.868251+0000","last_active":"2024-09-14T22:39:31.776747+0000","last_peered":"2024-09-14T22:39:31.776747+0000","last_clean":"2024-09-14T22:39:31.776747+0000","last_became_active":"2024-09-14T22:36:15.723205+0000","last_became_peered":"2024-09-14T22:36:15.723205+0000","last_unstale":"2024-09-14T22:39:31.776747+0000","last_undegraded":"2024-09-14T22:39:31.776747+0000","last_fullsized":"2024-09-14T22:39:31.776747+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:13:11.920909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001862636,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":4348,"num_read_kb":16212,"num_write":775,"num_write_kb":5956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"175'549","reported_seq":2917,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776796+0000","last_change":"2024-09-14T22:39:13.093929+0000","last_active":"2024-09-14T22:39:31.776796+0000","last_peered":"2024-09-14T22:39:31.776796+0000","last_clean":"2024-09-14T22:39:31.776796+0000","last_became_active":"2024-09-14T22:39:13.025887+0000","last_became_peered":"2024-09-14T22:39:13.025887+0000","last_unstale":"2024-09-14T22:39:31.776796+0000","last_undegraded":"2024-09-14T22:39:31.776796+0000","last_fullsized":"2024-09-14T22:39:31.776796+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:31:08.644424+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.030321287999999998,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":2666,"num_read_kb":8490,"num_write":594,"num_write_kb":3830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":4944418,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"175'499","reported_seq":8501,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776826+0000","last_change":"2024-09-14T22:39:13.145388+0000","last_active":"2024-09-14T22:39:31.776826+0000","last_peered":"2024-09-14T22:39:31.776826+0000","last_clean":"2024-09-14T22:39:31.776826+0000","last_became_active":"2024-09-14T22:39:13.030456+0000","last_became_peered":"2024-09-14T22:39:13.030456+0000","last_unstale":"2024-09-14T22:39:31.776826+0000","last_undegraded":"2024-09-14T22:39:31.776826+0000","last_fullsized":"2024-09-14T22:39:31.776826+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:10:30.405497+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.017930723999999999,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":8487,"num_read_kb":23501,"num_write":537,"num_write_kb":9535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"168'493","reported_seq":4108,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.247573+0000","last_change":"2024-09-14T22:39:02.866134+0000","last_active":"2024-09-14T22:39:31.247573+0000","last_peered":"2024-09-14T22:39:31.247573+0000","last_clean":"2024-09-14T22:39:31.247573+0000","last_became_active":"2024-09-14T22:37:26.717088+0000","last_became_peered":"2024-09-14T22:37:26.717088+0000","last_unstale":"2024-09-14T22:39:31.247573+0000","last_undegraded":"2024-09-14T22:39:31.247573+0000","last_fullsized":"2024-09-14T22:39:31.247573+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:03:51.158429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024508580000000002,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":3408,"num_read_kb":29082,"num_write":535,"num_write_kb":14817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"175'685","reported_seq":2731,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776343+0000","last_change":"2024-09-14T22:39:13.123794+0000","last_active":"2024-09-14T22:39:31.776343+0000","last_peered":"2024-09-14T22:39:31.776343+0000","last_clean":"2024-09-14T22:39:31.776343+0000","last_became_active":"2024-09-14T22:39:13.029544+0000","last_became_peered":"2024-09-14T22:39:13.029544+0000","last_unstale":"2024-09-14T22:39:31.776343+0000","last_undegraded":"2024-09-14T22:39:31.776343+0000","last_fullsized":"2024-09-14T22:39:31.776343+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:47:09.608401+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.020111626000000001,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":3515,"num_read_kb":11729,"num_write":870,"num_write_kb":10937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"168'589","reported_seq":8731,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306353+0000","last_change":"2024-09-14T22:39:02.782999+0000","last_active":"2024-09-14T22:39:32.306353+0000","last_peered":"2024-09-14T22:39:32.306353+0000","last_clean":"2024-09-14T22:39:32.306353+0000","last_became_active":"2024-09-14T22:36:15.716696+0000","last_became_peered":"2024-09-14T22:36:15.716696+0000","last_unstale":"2024-09-14T22:39:32.306353+0000","last_undegraded":"2024-09-14T22:39:32.306353+0000","last_fullsized":"2024-09-14T22:39:32.306353+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:14:34.437495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019004219999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":9756,"num_read_kb":50618,"num_write":568,"num_write_kb":13171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"175'599","reported_seq":7681,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-14T22:39:31.776429+0000","last_change":"2024-09-14T22:39:13.167257+0000","last_active":"2024-09-14T22:39:31.776429+0000","last_peered":"2024-09-14T22:39:31.776429+0000","last_clean":"2024-09-14T22:39:31.776429+0000","last_became_active":"2024-09-14T22:39:13.030502+0000","last_became_peered":"2024-09-14T22:39:13.030502+0000","last_unstale":"2024-09-14T22:39:31.776429+0000","last_undegraded":"2024-09-14T22:39:31.776429+0000","last_fullsized":"2024-09-14T22:39:31.776429+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:35:32.951904+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.023146856,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":7147,"num_read_kb":35324,"num_write":610,"num_write_kb":15494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":14511090,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":893,"num_object_clones":0,"num_object_copies":1786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":893,"num_whiteouts":889,"num_read":206686,"num_read_kb":828988,"num_write":25214,"num_write_kb":381837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3336,"num_bytes_recovered":173602381,"num_keys_recovered":342,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22322,"internal_metadata":0},"log_size":21573,"ondisk_log_size":21573,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542191,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59848,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311992,"statfs":{"total":96636764160,"available":96575479808,"internally_reserved":0,"allocated":2347008,"data_stored":1693510,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542192,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45904,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325936,"statfs":{"total":96636764160,"available":96589758464,"internally_reserved":0,"allocated":2289664,"data_stored":1103142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542192,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55652,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316188,"statfs":{"total":96636764160,"available":96579776512,"internally_reserved":0,"allocated":2306048,"data_stored":1109032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":174,"seq":747324309511,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":2252,"kb_used_omap":34,"kb_used_meta":29277,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":2306048,"data_stored":1109032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35258,"internal_metadata":29980230},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542192,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51472,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320368,"statfs":{"total":96636764160,"available":96584056832,"internally_reserved":0,"allocated":2289664,"data_stored":1103142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":66,"seq":283467841567,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43228,"kb_used_data":2248,"kb_used_omap":6,"kb_used_meta":40953,"kb_avail":94328612,"statfs":{"total":96636764160,"available":96592498688,"internally_reserved":0,"allocated":2301952,"data_stored":1109013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7025,"internal_metadata":41936015},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59816,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312024,"statfs":{"total":96636764160,"available":96575512576,"internally_reserved":0,"allocated":2310144,"data_stored":1109051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542191,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41816,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330024,"statfs":{"total":96636764160,"available":96593944576,"internally_reserved":0,"allocated":2359296,"data_stored":1699381,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4238,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18084,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:39:37.217 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T22:39:37.509 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:39:37.509 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":191,"fsid":"f07f407a-b17d-42c4-bf73-92dd80114a4f","created":"2024-09-14T22:35:45.983055+0000","modified":"2024-09-14T22:39:32.137370+0000","last_up_change":"2024-09-14T22:39:11.976048+0000","last_in_change":"2024-09-14T22:35:48.283160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T22:35:58.938784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T22:36:04.304242+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-14T22:36:14.374249+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":44,"snap_epoch":170,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9831fcc5-9040-4a80-99d4-838225e65139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1702929600},{"type":"v1","addr":"172.21.15.1:6825","nonce":1702929600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1702929600},{"type":"v1","addr":"172.21.15.1:6827","nonce":1702929600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":1702929600},{"type":"v1","addr":"172.21.15.1:6831","nonce":1702929600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":1702929600},{"type":"v1","addr":"172.21.15.1:6829","nonce":1702929600}]},"public_addr":"172.21.15.1:6825/1702929600","cluster_addr":"172.21.15.1:6827/1702929600","heartbeat_back_addr":"172.21.15.1:6831/1702929600","heartbeat_front_addr":"172.21.15.1:6829/1702929600","state":["exists","up"]},{"osd":1,"uuid":"7f79abe8-115d-46b2-8677-72fcb0c8ab7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":719990653},{"type":"v1","addr":"172.21.15.1:6809","nonce":719990653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":719990653},{"type":"v1","addr":"172.21.15.1:6811","nonce":719990653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":719990653},{"type":"v1","addr":"172.21.15.1:6815","nonce":719990653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":719990653},{"type":"v1","addr":"172.21.15.1:6813","nonce":719990653}]},"public_addr":"172.21.15.1:6809/719990653","cluster_addr":"172.21.15.1:6811/719990653","heartbeat_back_addr":"172.21.15.1:6815/719990653","heartbeat_front_addr":"172.21.15.1:6813/719990653","state":["exists","up"]},{"osd":2,"uuid":"1f5768b2-4db4-4b2e-8d5e-5206f8bde28d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":30,"up_from":66,"up_thru":159,"down_at":31,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1006730601},{"type":"v1","addr":"172.21.15.1:6817","nonce":1006730601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1006730601},{"type":"v1","addr":"172.21.15.1:6819","nonce":1006730601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1006730601},{"type":"v1","addr":"172.21.15.1:6823","nonce":1006730601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1006730601},{"type":"v1","addr":"172.21.15.1:6821","nonce":1006730601}]},"public_addr":"172.21.15.1:6817/1006730601","cluster_addr":"172.21.15.1:6819/1006730601","heartbeat_back_addr":"172.21.15.1:6823/1006730601","heartbeat_front_addr":"172.21.15.1:6821/1006730601","state":["exists","up"]},{"osd":3,"uuid":"8103f3c7-7899-4fc3-bb02-dd2273120138","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":750403671},{"type":"v1","addr":"172.21.15.1:6801","nonce":750403671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":750403671},{"type":"v1","addr":"172.21.15.1:6803","nonce":750403671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":750403671},{"type":"v1","addr":"172.21.15.1:6807","nonce":750403671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":750403671},{"type":"v1","addr":"172.21.15.1:6805","nonce":750403671}]},"public_addr":"172.21.15.1:6801/750403671","cluster_addr":"172.21.15.1:6803/750403671","heartbeat_back_addr":"172.21.15.1:6807/750403671","heartbeat_front_addr":"172.21.15.1:6805/750403671","state":["exists","up"]},{"osd":4,"uuid":"23cce5cb-726e-44c2-8edb-c2567a89dfbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":158,"up_from":174,"up_thru":174,"down_at":159,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":3208688125},{"type":"v1","addr":"172.21.15.176:6817","nonce":3208688125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":3208688125},{"type":"v1","addr":"172.21.15.176:6819","nonce":3208688125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":3208688125},{"type":"v1","addr":"172.21.15.176:6823","nonce":3208688125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":3208688125},{"type":"v1","addr":"172.21.15.176:6821","nonce":3208688125}]},"public_addr":"172.21.15.176:6817/3208688125","cluster_addr":"172.21.15.176:6819/3208688125","heartbeat_back_addr":"172.21.15.176:6823/3208688125","heartbeat_front_addr":"172.21.15.176:6821/3208688125","state":["exists","up"]},{"osd":5,"uuid":"eb2f187b-d2f1-49ab-ad5e-ea6dc441a28e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6805","nonce":1945902326},{"type":"v1","addr":"172.21.15.176:6808","nonce":1945902326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":1945902326},{"type":"v1","addr":"172.21.15.176:6811","nonce":1945902326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":1945902326},{"type":"v1","addr":"172.21.15.176:6815","nonce":1945902326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":1945902326},{"type":"v1","addr":"172.21.15.176:6813","nonce":1945902326}]},"public_addr":"172.21.15.176:6808/1945902326","cluster_addr":"172.21.15.176:6811/1945902326","heartbeat_back_addr":"172.21.15.176:6815/1945902326","heartbeat_front_addr":"172.21.15.176:6813/1945902326","state":["exists","up"]},{"osd":6,"uuid":"d371aed8-2ad2-49a5-8ece-4b7fd37b4822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":4116085586},{"type":"v1","addr":"172.21.15.176:6801","nonce":4116085586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":4116085586},{"type":"v1","addr":"172.21.15.176:6803","nonce":4116085586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6807","nonce":4116085586},{"type":"v1","addr":"172.21.15.176:6809","nonce":4116085586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":4116085586},{"type":"v1","addr":"172.21.15.176:6806","nonce":4116085586}]},"public_addr":"172.21.15.176:6801/4116085586","cluster_addr":"172.21.15.176:6803/4116085586","heartbeat_back_addr":"172.21.15.176:6809/4116085586","heartbeat_front_addr":"172.21.15.176:6806/4116085586","state":["exists","up"]},{"osd":7,"uuid":"a012c819-8d49-4310-a840-c2752d9314ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":877760244},{"type":"v1","addr":"172.21.15.176:6825","nonce":877760244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":877760244},{"type":"v1","addr":"172.21.15.176:6827","nonce":877760244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":877760244},{"type":"v1","addr":"172.21.15.176:6831","nonce":877760244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":877760244},{"type":"v1","addr":"172.21.15.176:6829","nonce":877760244}]},"public_addr":"172.21.15.176:6825/877760244","cluster_addr":"172.21.15.176:6827/877760244","heartbeat_back_addr":"172.21.15.176:6831/877760244","heartbeat_front_addr":"172.21.15.176:6829/877760244","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:55.511688+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.742414+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.871355+0000","dead_epoch":31},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.504553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.589640+0000","dead_epoch":159},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.518845+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.731632+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:35:54.952634+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.0","osds":[4,2]},{"pgid":"3.1d","osds":[1,2]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T22:39:38.527 INFO:tasks.ceph:Scrubbing osd.0 2024-09-14T22:39:38.528 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-14T22:39:38.694 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:39:38.694 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T22:39:38.694 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:39:38.710 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-14T22:39:39.006 INFO:teuthology.orchestra.run.smithi001.stderr:instructed osd(s) 0 to deep-scrub 2024-09-14T22:39:39.025 INFO:tasks.ceph:Scrubbing osd.1 2024-09-14T22:39:39.026 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-14T22:39:39.180 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:39:39.180 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T22:39:39.180 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:39:39.196 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-14T22:39:39.494 INFO:teuthology.orchestra.run.smithi001.stderr:instructed osd(s) 1 to deep-scrub 2024-09-14T22:39:39.514 INFO:tasks.ceph:Scrubbing osd.2 2024-09-14T22:39:39.514 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-14T22:39:39.668 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:39:39.668 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T22:39:39.668 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:39:39.685 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-14T22:39:39.986 INFO:teuthology.orchestra.run.smithi001.stderr:instructed osd(s) 2 to deep-scrub 2024-09-14T22:39:40.005 INFO:tasks.ceph:Scrubbing osd.3 2024-09-14T22:39:40.006 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-14T22:39:40.171 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:39:40.172 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T22:39:40.172 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:39:40.192 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-14T22:39:40.482 INFO:teuthology.orchestra.run.smithi001.stderr:instructed osd(s) 3 to deep-scrub 2024-09-14T22:39:40.502 INFO:tasks.ceph:Scrubbing osd.4 2024-09-14T22:39:40.502 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-14T22:39:40.662 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:39:40.662 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-14T22:39:40.662 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:39:40.678 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-14T22:39:40.965 INFO:teuthology.orchestra.run.smithi001.stderr:instructed osd(s) 4 to deep-scrub 2024-09-14T22:39:40.983 INFO:tasks.ceph:Scrubbing osd.5 2024-09-14T22:39:40.983 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-14T22:39:41.138 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:39:41.138 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T22:39:41.138 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:39:41.155 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-14T22:39:41.440 INFO:teuthology.orchestra.run.smithi001.stderr:instructed osd(s) 5 to deep-scrub 2024-09-14T22:39:41.458 INFO:tasks.ceph:Scrubbing osd.6 2024-09-14T22:39:41.458 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-14T22:39:41.608 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:39:41.608 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T22:39:41.608 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:39:41.625 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-14T22:39:41.909 INFO:teuthology.orchestra.run.smithi001.stderr:instructed osd(s) 6 to deep-scrub 2024-09-14T22:39:41.929 INFO:tasks.ceph:Scrubbing osd.7 2024-09-14T22:39:41.929 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-14T22:39:42.088 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-14T22:39:42.088 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T22:39:42.088 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-14T22:39:42.105 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-14T22:39:42.398 INFO:teuthology.orchestra.run.smithi001.stderr:instructed osd(s) 7 to deep-scrub 2024-09-14T22:39:42.417 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:39:42.731 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:39:42.732 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-14T22:39:42.749 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2024-09-14T22:39:40.921383+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":897,"num_object_clones":0,"num_object_copies":1794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":897,"num_whiteouts":889,"num_read":206802,"num_read_kb":829088,"num_write":25375,"num_write_kb":384255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3336,"num_bytes_recovered":173602381,"num_keys_recovered":342,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21693,"ondisk_log_size":21693,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":389364,"kb_used_data":18112,"kb_used_omap":50,"kb_used_meta":371021,"kb_avail":754585356,"statfs":{"total":773094113280,"available":772695404544,"internally_reserved":0,"allocated":18546688,"data_stored":10052916,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":51845,"internal_metadata":379925883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.790203"},"pg_stats":[{"pgid":"3.1f","version":"170'491","reported_seq":5322,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306474+0000","last_change":"2024-09-14T22:39:02.783678+0000","last_active":"2024-09-14T22:39:32.306474+0000","last_peered":"2024-09-14T22:39:32.306474+0000","last_clean":"2024-09-14T22:39:32.306474+0000","last_became_active":"2024-09-14T22:37:25.718906+0000","last_became_peered":"2024-09-14T22:37:25.718906+0000","last_unstale":"2024-09-14T22:39:32.306474+0000","last_undegraded":"2024-09-14T22:39:32.306474+0000","last_fullsized":"2024-09-14T22:39:32.306474+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:16:07.317648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023103960000000002,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":5019,"num_read_kb":16516,"num_write":439,"num_write_kb":7121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":3277632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"170'1231","reported_seq":7364,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:33.710575+0000","last_change":"2024-09-14T22:39:02.771231+0000","last_active":"2024-09-14T22:39:33.710575+0000","last_peered":"2024-09-14T22:39:33.710575+0000","last_clean":"2024-09-14T22:39:33.710575+0000","last_became_active":"2024-09-14T22:38:08.741149+0000","last_became_peered":"2024-09-14T22:38:08.741149+0000","last_unstale":"2024-09-14T22:39:33.710575+0000","last_undegraded":"2024-09-14T22:39:33.710575+0000","last_fullsized":"2024-09-14T22:39:33.710575+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1231,"log_dups_size":0,"ondisk_log_size":1231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:32:08.194278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024840370000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":11835,"num_read_kb":34663,"num_write":1722,"num_write_kb":10763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":7412288,"num_keys_recovered":90,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"168'724","reported_seq":10563,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149517+0000","last_change":"2024-09-14T22:39:20.753911+0000","last_active":"2024-09-14T22:39:32.149517+0000","last_peered":"2024-09-14T22:39:32.149517+0000","last_clean":"2024-09-14T22:39:32.149517+0000","last_became_active":"2024-09-14T22:39:20.751505+0000","last_became_peered":"2024-09-14T22:39:20.751505+0000","last_unstale":"2024-09-14T22:39:32.149517+0000","last_undegraded":"2024-09-14T22:39:32.149517+0000","last_fullsized":"2024-09-14T22:39:32.149517+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:17:01.639166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0083529999999999993,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":10932,"num_read_kb":63225,"num_write":868,"num_write_kb":13703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":8030815,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"175'718","reported_seq":6436,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298203+0000","last_change":"2024-09-14T22:39:25.087327+0000","last_active":"2024-09-14T22:39:32.298203+0000","last_peered":"2024-09-14T22:39:32.298203+0000","last_clean":"2024-09-14T22:39:32.298203+0000","last_became_active":"2024-09-14T22:39:25.084122+0000","last_became_peered":"2024-09-14T22:39:25.084122+0000","last_unstale":"2024-09-14T22:39:32.298203+0000","last_undegraded":"2024-09-14T22:39:32.298203+0000","last_fullsized":"2024-09-14T22:39:32.298203+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:17:17.035425+0000","scrub_duration":0,"objects_trimmed":20,"snaptrim_duration":0.055023557000000001,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":34,"num_read":6435,"num_read_kb":26956,"num_write":884,"num_write_kb":8982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"175'602","reported_seq":5586,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306492+0000","last_change":"2024-09-14T22:39:13.115776+0000","last_active":"2024-09-14T22:39:32.306492+0000","last_peered":"2024-09-14T22:39:32.306492+0000","last_clean":"2024-09-14T22:39:32.306492+0000","last_became_active":"2024-09-14T22:39:13.025036+0000","last_became_peered":"2024-09-14T22:39:13.025036+0000","last_unstale":"2024-09-14T22:39:32.306492+0000","last_undegraded":"2024-09-14T22:39:32.306492+0000","last_fullsized":"2024-09-14T22:39:32.306492+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:23:31.541475+0000","scrub_duration":0,"objects_trimmed":14,"snaptrim_duration":0.061855755999999998,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":5663,"num_read_kb":32434,"num_write":648,"num_write_kb":13330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":7858104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"175'527","reported_seq":4225,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149467+0000","last_change":"2024-09-14T22:39:13.150950+0000","last_active":"2024-09-14T22:39:32.149467+0000","last_peered":"2024-09-14T22:39:32.149467+0000","last_clean":"2024-09-14T22:39:32.149467+0000","last_became_active":"2024-09-14T22:39:13.016132+0000","last_became_peered":"2024-09-14T22:39:13.016132+0000","last_unstale":"2024-09-14T22:39:32.149467+0000","last_undegraded":"2024-09-14T22:39:32.149467+0000","last_fullsized":"2024-09-14T22:39:32.149467+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:34:37.241298+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.069063225000000006,"stat_sum":{"num_bytes":19,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":33,"num_read":4551,"num_read_kb":13173,"num_write":474,"num_write_kb":7035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":236,"num_bytes_recovered":10036538,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"169'465","reported_seq":5658,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149161+0000","last_change":"2024-09-14T22:39:02.778427+0000","last_active":"2024-09-14T22:39:32.149161+0000","last_peered":"2024-09-14T22:39:32.149161+0000","last_clean":"2024-09-14T22:39:32.149161+0000","last_became_active":"2024-09-14T22:36:15.700990+0000","last_became_peered":"2024-09-14T22:36:15.700990+0000","last_unstale":"2024-09-14T22:39:32.149161+0000","last_undegraded":"2024-09-14T22:39:32.149161+0000","last_fullsized":"2024-09-14T22:39:32.149161+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:24:53.327956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013968780000000001,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":5273,"num_read_kb":15380,"num_write":425,"num_write_kb":5486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"170'511","reported_seq":4997,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:33.710469+0000","last_change":"2024-09-14T22:39:02.769340+0000","last_active":"2024-09-14T22:39:33.710469+0000","last_peered":"2024-09-14T22:39:33.710469+0000","last_clean":"2024-09-14T22:39:33.710469+0000","last_became_active":"2024-09-14T22:38:07.734308+0000","last_became_peered":"2024-09-14T22:38:07.734308+0000","last_unstale":"2024-09-14T22:39:33.710469+0000","last_undegraded":"2024-09-14T22:39:33.710469+0000","last_fullsized":"2024-09-14T22:39:33.710469+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:12:44.546869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013730859999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4941,"num_read_kb":35923,"num_write":636,"num_write_kb":27736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":20067293,"num_keys_recovered":21,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"167'467","reported_seq":4865,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306510+0000","last_change":"2024-09-14T22:39:02.784093+0000","last_active":"2024-09-14T22:39:32.306510+0000","last_peered":"2024-09-14T22:39:32.306510+0000","last_clean":"2024-09-14T22:39:32.306510+0000","last_became_active":"2024-09-14T22:36:15.715869+0000","last_became_peered":"2024-09-14T22:36:15.715869+0000","last_unstale":"2024-09-14T22:39:32.306510+0000","last_undegraded":"2024-09-14T22:39:32.306510+0000","last_fullsized":"2024-09-14T22:39:32.306510+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:57:02.149610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023931830000000001,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":4593,"num_read_kb":25142,"num_write":516,"num_write_kb":22003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"166'605","reported_seq":6669,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298379+0000","last_change":"2024-09-14T22:39:02.774376+0000","last_active":"2024-09-14T22:39:32.298379+0000","last_peered":"2024-09-14T22:39:32.298379+0000","last_clean":"2024-09-14T22:39:32.298379+0000","last_became_active":"2024-09-14T22:36:15.725078+0000","last_became_peered":"2024-09-14T22:36:15.725078+0000","last_unstale":"2024-09-14T22:39:32.298379+0000","last_undegraded":"2024-09-14T22:39:32.298379+0000","last_fullsized":"2024-09-14T22:39:32.298379+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:26:12.926170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035779840000000002,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":6675,"num_read_kb":51224,"num_write":729,"num_write_kb":18952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"167'467","reported_seq":4806,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:34.405937+0000","last_change":"2024-09-14T22:39:02.867529+0000","last_active":"2024-09-14T22:39:34.405937+0000","last_peered":"2024-09-14T22:39:34.405937+0000","last_clean":"2024-09-14T22:39:34.405937+0000","last_became_active":"2024-09-14T22:36:15.717341+0000","last_became_peered":"2024-09-14T22:36:15.717341+0000","last_unstale":"2024-09-14T22:39:34.405937+0000","last_undegraded":"2024-09-14T22:39:34.405937+0000","last_fullsized":"2024-09-14T22:39:34.405937+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:29:47.296331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016388500000000001,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":5090,"num_read_kb":12789,"num_write":419,"num_write_kb":7375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"175'1202","reported_seq":9124,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149855+0000","last_change":"2024-09-14T22:39:13.160248+0000","last_active":"2024-09-14T22:39:32.149855+0000","last_peered":"2024-09-14T22:39:32.149855+0000","last_clean":"2024-09-14T22:39:32.149855+0000","last_became_active":"2024-09-14T22:39:13.026842+0000","last_became_peered":"2024-09-14T22:39:13.026842+0000","last_unstale":"2024-09-14T22:39:32.149855+0000","last_undegraded":"2024-09-14T22:39:32.149855+0000","last_fullsized":"2024-09-14T22:39:32.149855+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1202,"log_dups_size":0,"ondisk_log_size":1202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:06:37.741547+0000","scrub_duration":0,"objects_trimmed":14,"snaptrim_duration":0.055942110000000003,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":11247,"num_read_kb":29618,"num_write":1623,"num_write_kb":11950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":265,"num_bytes_recovered":17498216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":356,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:34.405256+0000","last_change":"2024-09-14T22:36:08.042860+0000","last_active":"2024-09-14T22:39:34.405256+0000","last_peered":"2024-09-14T22:39:34.405256+0000","last_clean":"2024-09-14T22:39:34.405256+0000","last_became_active":"2024-09-14T22:36:06.040961+0000","last_became_peered":"2024-09-14T22:36:06.040961+0000","last_unstale":"2024-09-14T22:39:34.405256+0000","last_undegraded":"2024-09-14T22:39:34.405256+0000","last_fullsized":"2024-09-14T22:39:34.405256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:03:48.547834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010780950000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"170'738","reported_seq":3855,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306469+0000","last_change":"2024-09-14T22:39:02.782844+0000","last_active":"2024-09-14T22:39:32.306469+0000","last_peered":"2024-09-14T22:39:32.306469+0000","last_clean":"2024-09-14T22:39:32.306469+0000","last_became_active":"2024-09-14T22:36:15.700982+0000","last_became_peered":"2024-09-14T22:36:15.700982+0000","last_unstale":"2024-09-14T22:39:32.306469+0000","last_undegraded":"2024-09-14T22:39:32.306469+0000","last_fullsized":"2024-09-14T22:39:32.306469+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:06:43.937383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001672663,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":4804,"num_read_kb":22802,"num_write":686,"num_write_kb":12175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":181,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149492+0000","last_change":"2024-09-14T22:39:20.885509+0000","last_active":"2024-09-14T22:39:32.149492+0000","last_peered":"2024-09-14T22:39:32.149492+0000","last_clean":"2024-09-14T22:39:32.149492+0000","last_became_active":"2024-09-14T22:39:20.885045+0000","last_became_peered":"2024-09-14T22:39:20.885045+0000","last_unstale":"2024-09-14T22:39:32.149492+0000","last_undegraded":"2024-09-14T22:39:32.149492+0000","last_fullsized":"2024-09-14T22:39:32.149492+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:37:23.145835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"170'537","reported_seq":5823,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:33.710605+0000","last_change":"2024-09-14T22:39:23.066067+0000","last_active":"2024-09-14T22:39:33.710605+0000","last_peered":"2024-09-14T22:39:33.710605+0000","last_clean":"2024-09-14T22:39:33.710605+0000","last_became_active":"2024-09-14T22:39:23.064710+0000","last_became_peered":"2024-09-14T22:39:23.064710+0000","last_unstale":"2024-09-14T22:39:33.710605+0000","last_undegraded":"2024-09-14T22:39:33.710605+0000","last_fullsized":"2024-09-14T22:39:33.710605+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:53:48.607465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044565630000000002,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":6110,"num_read_kb":28977,"num_write":556,"num_write_kb":8495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":358,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:34.405647+0000","last_change":"2024-09-14T22:36:08.054606+0000","last_active":"2024-09-14T22:39:34.405647+0000","last_peered":"2024-09-14T22:39:34.405647+0000","last_clean":"2024-09-14T22:39:34.405647+0000","last_became_active":"2024-09-14T22:36:06.266908+0000","last_became_peered":"2024-09-14T22:36:06.266908+0000","last_unstale":"2024-09-14T22:39:34.405647+0000","last_undegraded":"2024-09-14T22:39:34.405647+0000","last_fullsized":"2024-09-14T22:39:34.405647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:36.738643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098249300000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"169'1480","reported_seq":5562,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149277+0000","last_change":"2024-09-14T22:39:20.753197+0000","last_active":"2024-09-14T22:39:32.149277+0000","last_peered":"2024-09-14T22:39:32.149277+0000","last_clean":"2024-09-14T22:39:32.149277+0000","last_became_active":"2024-09-14T22:39:20.751272+0000","last_became_peered":"2024-09-14T22:39:20.751272+0000","last_unstale":"2024-09-14T22:39:32.149277+0000","last_undegraded":"2024-09-14T22:39:32.149277+0000","last_fullsized":"2024-09-14T22:39:32.149277+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1480,"log_dups_size":0,"ondisk_log_size":1480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:15:46.970993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037027409999999998,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":9975,"num_read_kb":23940,"num_write":2463,"num_write_kb":17944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":4264139,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":88,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149438+0000","last_change":"2024-09-14T22:39:20.748672+0000","last_active":"2024-09-14T22:39:32.149438+0000","last_peered":"2024-09-14T22:39:32.149438+0000","last_clean":"2024-09-14T22:39:32.149438+0000","last_became_active":"2024-09-14T22:39:20.748164+0000","last_became_peered":"2024-09-14T22:39:20.748164+0000","last_unstale":"2024-09-14T22:39:32.149438+0000","last_undegraded":"2024-09-14T22:39:32.149438+0000","last_fullsized":"2024-09-14T22:39:32.149438+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:45:58.659226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"170'625","reported_seq":4878,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298433+0000","last_change":"2024-09-14T22:39:25.088610+0000","last_active":"2024-09-14T22:39:32.298433+0000","last_peered":"2024-09-14T22:39:32.298433+0000","last_clean":"2024-09-14T22:39:32.298433+0000","last_became_active":"2024-09-14T22:39:25.085794+0000","last_became_peered":"2024-09-14T22:39:25.085794+0000","last_unstale":"2024-09-14T22:39:32.298433+0000","last_undegraded":"2024-09-14T22:39:32.298433+0000","last_fullsized":"2024-09-14T22:39:32.298433+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:03:57.958236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002907598,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":5488,"num_read_kb":14305,"num_write":599,"num_write_kb":5812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":5797306,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"175'596","reported_seq":5470,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149769+0000","last_change":"2024-09-14T22:39:13.146266+0000","last_active":"2024-09-14T22:39:32.149769+0000","last_peered":"2024-09-14T22:39:32.149769+0000","last_clean":"2024-09-14T22:39:32.149769+0000","last_became_active":"2024-09-14T22:39:13.012755+0000","last_became_peered":"2024-09-14T22:39:13.012755+0000","last_unstale":"2024-09-14T22:39:32.149769+0000","last_undegraded":"2024-09-14T22:39:32.149769+0000","last_fullsized":"2024-09-14T22:39:32.149769+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:17:13.530413+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.084951515000000005,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":30,"num_read":4623,"num_read_kb":25789,"num_write":608,"num_write_kb":14115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":254,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298189+0000","last_change":"2024-09-14T22:39:25.086414+0000","last_active":"2024-09-14T22:39:32.298189+0000","last_peered":"2024-09-14T22:39:32.298189+0000","last_clean":"2024-09-14T22:39:32.298189+0000","last_became_active":"2024-09-14T22:39:25.085378+0000","last_became_peered":"2024-09-14T22:39:25.085378+0000","last_unstale":"2024-09-14T22:39:32.298189+0000","last_undegraded":"2024-09-14T22:39:32.298189+0000","last_fullsized":"2024-09-14T22:39:32.298189+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:01:23.251308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021617870000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"170'991","reported_seq":9285,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149171+0000","last_change":"2024-09-14T22:39:20.754533+0000","last_active":"2024-09-14T22:39:32.149171+0000","last_peered":"2024-09-14T22:39:32.149171+0000","last_clean":"2024-09-14T22:39:32.149171+0000","last_became_active":"2024-09-14T22:39:20.751993+0000","last_became_peered":"2024-09-14T22:39:20.751993+0000","last_unstale":"2024-09-14T22:39:32.149171+0000","last_undegraded":"2024-09-14T22:39:32.149171+0000","last_fullsized":"2024-09-14T22:39:32.149171+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":991,"log_dups_size":0,"ondisk_log_size":991,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:08.353422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030469329999999999,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":10327,"num_read_kb":29859,"num_write":1283,"num_write_kb":15017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":4851034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":293,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:33.347822+0000","last_change":"2024-09-14T22:38:05.724182+0000","last_active":"2024-09-14T22:39:33.347822+0000","last_peered":"2024-09-14T22:39:33.347822+0000","last_clean":"2024-09-14T22:39:33.347822+0000","last_became_active":"2024-09-14T22:38:05.723636+0000","last_became_peered":"2024-09-14T22:38:05.723636+0000","last_unstale":"2024-09-14T22:39:33.347822+0000","last_undegraded":"2024-09-14T22:39:33.347822+0000","last_fullsized":"2024-09-14T22:39:33.347822+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:14:09.687576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"175'449","reported_seq":5800,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306344+0000","last_change":"2024-09-14T22:39:13.119596+0000","last_active":"2024-09-14T22:39:32.306344+0000","last_peered":"2024-09-14T22:39:32.306344+0000","last_clean":"2024-09-14T22:39:32.306344+0000","last_became_active":"2024-09-14T22:39:13.025169+0000","last_became_peered":"2024-09-14T22:39:13.025169+0000","last_unstale":"2024-09-14T22:39:32.306344+0000","last_undegraded":"2024-09-14T22:39:32.306344+0000","last_fullsized":"2024-09-14T22:39:32.306344+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:01:44.220418+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.027619476,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":4593,"num_read_kb":23198,"num_write":420,"num_write_kb":7231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":3679238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":53,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149252+0000","last_change":"2024-09-14T22:39:13.009681+0000","last_active":"2024-09-14T22:39:32.149252+0000","last_peered":"2024-09-14T22:39:32.149252+0000","last_clean":"2024-09-14T22:39:32.149252+0000","last_became_active":"2024-09-14T22:39:13.008709+0000","last_became_peered":"2024-09-14T22:39:13.008709+0000","last_unstale":"2024-09-14T22:39:32.149252+0000","last_undegraded":"2024-09-14T22:39:32.149252+0000","last_fullsized":"2024-09-14T22:39:32.149252+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:14:31.162323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010817520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"169'1116","reported_seq":4772,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:33.710528+0000","last_change":"2024-09-14T22:39:23.066824+0000","last_active":"2024-09-14T22:39:33.710528+0000","last_peered":"2024-09-14T22:39:33.710528+0000","last_clean":"2024-09-14T22:39:33.710528+0000","last_became_active":"2024-09-14T22:39:23.064694+0000","last_became_peered":"2024-09-14T22:39:23.064694+0000","last_unstale":"2024-09-14T22:39:33.710528+0000","last_undegraded":"2024-09-14T22:39:33.710528+0000","last_fullsized":"2024-09-14T22:39:33.710528+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:32:09.848042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033884660000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":7515,"num_read_kb":16135,"num_write":1577,"num_write_kb":11855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":2936757,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":260,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298203+0000","last_change":"2024-09-14T22:39:25.082725+0000","last_active":"2024-09-14T22:39:32.298203+0000","last_peered":"2024-09-14T22:39:32.298203+0000","last_clean":"2024-09-14T22:39:32.298203+0000","last_became_active":"2024-09-14T22:39:25.081686+0000","last_became_peered":"2024-09-14T22:39:25.081686+0000","last_unstale":"2024-09-14T22:39:32.298203+0000","last_undegraded":"2024-09-14T22:39:32.298203+0000","last_fullsized":"2024-09-14T22:39:32.298203+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:05.010064+0000","last_clean_scrub_stamp":"2024-09-14T22:36:05.010064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:35:39.626521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011401880000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":509,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:34.405803+0000","last_change":"2024-09-14T22:36:00.402862+0000","last_active":"2024-09-14T22:39:34.405803+0000","last_peered":"2024-09-14T22:39:34.405803+0000","last_clean":"2024-09-14T22:39:34.405803+0000","last_became_active":"2024-09-14T22:36:00.400529+0000","last_became_peered":"2024-09-14T22:36:00.400529+0000","last_unstale":"2024-09-14T22:39:34.405803+0000","last_undegraded":"2024-09-14T22:39:34.405803+0000","last_fullsized":"2024-09-14T22:39:34.405803+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:35:58.989025+0000","last_clean_scrub_stamp":"2024-09-14T22:35:58.989025+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:07:12.951728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"169'676","reported_seq":7831,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:33.710478+0000","last_change":"2024-09-14T22:39:02.769315+0000","last_active":"2024-09-14T22:39:33.710478+0000","last_peered":"2024-09-14T22:39:33.710478+0000","last_clean":"2024-09-14T22:39:33.710478+0000","last_became_active":"2024-09-14T22:38:08.739090+0000","last_became_peered":"2024-09-14T22:38:08.739090+0000","last_unstale":"2024-09-14T22:39:33.710478+0000","last_undegraded":"2024-09-14T22:39:33.710478+0000","last_fullsized":"2024-09-14T22:39:33.710478+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:30:42.554797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012849739999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":8821,"num_read_kb":22101,"num_write":951,"num_write_kb":14470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":18690892,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"175'461","reported_seq":7414,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.149505+0000","last_change":"2024-09-14T22:39:13.119552+0000","last_active":"2024-09-14T22:39:32.149505+0000","last_peered":"2024-09-14T22:39:32.149505+0000","last_clean":"2024-09-14T22:39:32.149505+0000","last_became_active":"2024-09-14T22:39:13.008690+0000","last_became_peered":"2024-09-14T22:39:13.008690+0000","last_unstale":"2024-09-14T22:39:32.149505+0000","last_undegraded":"2024-09-14T22:39:32.149505+0000","last_fullsized":"2024-09-14T22:39:32.149505+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:02:56.318281+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.064091727000000001,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":7056,"num_read_kb":18679,"num_write":421,"num_write_kb":7008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":16527933,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"175'803","reported_seq":4496,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:34.405127+0000","last_change":"2024-09-14T22:39:13.078772+0000","last_active":"2024-09-14T22:39:34.405127+0000","last_peered":"2024-09-14T22:39:34.405127+0000","last_clean":"2024-09-14T22:39:34.405127+0000","last_became_active":"2024-09-14T22:39:13.024372+0000","last_became_peered":"2024-09-14T22:39:13.024372+0000","last_unstale":"2024-09-14T22:39:34.405127+0000","last_undegraded":"2024-09-14T22:39:34.405127+0000","last_fullsized":"2024-09-14T22:39:34.405127+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:50.926795+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.037620476,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":28,"num_read":6415,"num_read_kb":26025,"num_write":756,"num_write_kb":14809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"170'583","reported_seq":2963,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:33.710590+0000","last_change":"2024-09-14T22:39:02.771485+0000","last_active":"2024-09-14T22:39:33.710590+0000","last_peered":"2024-09-14T22:39:33.710590+0000","last_clean":"2024-09-14T22:39:33.710590+0000","last_became_active":"2024-09-14T22:36:15.702885+0000","last_became_peered":"2024-09-14T22:36:15.702885+0000","last_unstale":"2024-09-14T22:39:33.710590+0000","last_undegraded":"2024-09-14T22:39:33.710590+0000","last_fullsized":"2024-09-14T22:39:33.710590+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:59:16.704705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031173400000000001,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":3570,"num_read_kb":11670,"num_write":536,"num_write_kb":6877,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"170'447","reported_seq":6923,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.298274+0000","last_change":"2024-09-14T22:39:25.090361+0000","last_active":"2024-09-14T22:39:32.298274+0000","last_peered":"2024-09-14T22:39:32.298274+0000","last_clean":"2024-09-14T22:39:32.298274+0000","last_became_active":"2024-09-14T22:39:25.088012+0000","last_became_peered":"2024-09-14T22:39:25.088012+0000","last_unstale":"2024-09-14T22:39:32.298274+0000","last_undegraded":"2024-09-14T22:39:32.298274+0000","last_fullsized":"2024-09-14T22:39:32.298274+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:10:33.661519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018646120000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5808,"num_read_kb":33509,"num_write":486,"num_write_kb":17853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":23218688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"168'647","reported_seq":4234,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:34.406103+0000","last_change":"2024-09-14T22:39:02.868251+0000","last_active":"2024-09-14T22:39:34.406103+0000","last_peered":"2024-09-14T22:39:34.406103+0000","last_clean":"2024-09-14T22:39:34.406103+0000","last_became_active":"2024-09-14T22:36:15.723205+0000","last_became_peered":"2024-09-14T22:36:15.723205+0000","last_unstale":"2024-09-14T22:39:34.406103+0000","last_undegraded":"2024-09-14T22:39:34.406103+0000","last_fullsized":"2024-09-14T22:39:34.406103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:13:11.920909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001862636,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":4348,"num_read_kb":16212,"num_write":775,"num_write_kb":5956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"175'549","reported_seq":2919,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:34.406367+0000","last_change":"2024-09-14T22:39:13.093929+0000","last_active":"2024-09-14T22:39:34.406367+0000","last_peered":"2024-09-14T22:39:34.406367+0000","last_clean":"2024-09-14T22:39:34.406367+0000","last_became_active":"2024-09-14T22:39:13.025887+0000","last_became_peered":"2024-09-14T22:39:13.025887+0000","last_unstale":"2024-09-14T22:39:34.406367+0000","last_undegraded":"2024-09-14T22:39:34.406367+0000","last_fullsized":"2024-09-14T22:39:34.406367+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:31:08.644424+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.030321287999999998,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":2666,"num_read_kb":8490,"num_write":594,"num_write_kb":3830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":4944418,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"175'499","reported_seq":8503,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:34.406218+0000","last_change":"2024-09-14T22:39:13.145388+0000","last_active":"2024-09-14T22:39:34.406218+0000","last_peered":"2024-09-14T22:39:34.406218+0000","last_clean":"2024-09-14T22:39:34.406218+0000","last_became_active":"2024-09-14T22:39:13.030456+0000","last_became_peered":"2024-09-14T22:39:13.030456+0000","last_unstale":"2024-09-14T22:39:34.406218+0000","last_undegraded":"2024-09-14T22:39:34.406218+0000","last_fullsized":"2024-09-14T22:39:34.406218+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:10:30.405497+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.017930723999999999,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":8487,"num_read_kb":23501,"num_write":537,"num_write_kb":9535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"168'493","reported_seq":4110,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:34.405104+0000","last_change":"2024-09-14T22:39:02.866134+0000","last_active":"2024-09-14T22:39:34.405104+0000","last_peered":"2024-09-14T22:39:34.405104+0000","last_clean":"2024-09-14T22:39:34.405104+0000","last_became_active":"2024-09-14T22:37:26.717088+0000","last_became_peered":"2024-09-14T22:37:26.717088+0000","last_unstale":"2024-09-14T22:39:34.405104+0000","last_undegraded":"2024-09-14T22:39:34.405104+0000","last_fullsized":"2024-09-14T22:39:34.405104+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:03:51.158429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024508580000000002,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":3408,"num_read_kb":29082,"num_write":535,"num_write_kb":14817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"175'685","reported_seq":2733,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:34.405818+0000","last_change":"2024-09-14T22:39:13.123794+0000","last_active":"2024-09-14T22:39:34.405818+0000","last_peered":"2024-09-14T22:39:34.405818+0000","last_clean":"2024-09-14T22:39:34.405818+0000","last_became_active":"2024-09-14T22:39:13.029544+0000","last_became_peered":"2024-09-14T22:39:13.029544+0000","last_unstale":"2024-09-14T22:39:34.405818+0000","last_undegraded":"2024-09-14T22:39:34.405818+0000","last_fullsized":"2024-09-14T22:39:34.405818+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:47:09.608401+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.020111626000000001,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":3515,"num_read_kb":11729,"num_write":870,"num_write_kb":10937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"168'589","reported_seq":8731,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:32.306353+0000","last_change":"2024-09-14T22:39:02.782999+0000","last_active":"2024-09-14T22:39:32.306353+0000","last_peered":"2024-09-14T22:39:32.306353+0000","last_clean":"2024-09-14T22:39:32.306353+0000","last_became_active":"2024-09-14T22:36:15.716696+0000","last_became_peered":"2024-09-14T22:36:15.716696+0000","last_unstale":"2024-09-14T22:39:32.306353+0000","last_undegraded":"2024-09-14T22:39:32.306353+0000","last_fullsized":"2024-09-14T22:39:32.306353+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:14:34.437495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019004219999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":9756,"num_read_kb":50618,"num_write":568,"num_write_kb":13171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"175'599","reported_seq":7683,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:34.405840+0000","last_change":"2024-09-14T22:39:13.167257+0000","last_active":"2024-09-14T22:39:34.405840+0000","last_peered":"2024-09-14T22:39:34.405840+0000","last_clean":"2024-09-14T22:39:34.405840+0000","last_became_active":"2024-09-14T22:39:13.030502+0000","last_became_peered":"2024-09-14T22:39:13.030502+0000","last_unstale":"2024-09-14T22:39:34.405840+0000","last_undegraded":"2024-09-14T22:39:34.405840+0000","last_fullsized":"2024-09-14T22:39:34.405840+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:36:15.070743+0000","last_clean_scrub_stamp":"2024-09-14T22:36:15.070743+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:35:32.951904+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.023146856,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":7147,"num_read_kb":35324,"num_write":610,"num_write_kb":15494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":14511090,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":893,"num_object_clones":0,"num_object_copies":1786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":893,"num_whiteouts":889,"num_read":206686,"num_read_kb":828988,"num_write":25214,"num_write_kb":381837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3336,"num_bytes_recovered":173602381,"num_keys_recovered":342,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22322,"internal_metadata":0},"log_size":21573,"ondisk_log_size":21573,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542192,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59864,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311976,"statfs":{"total":96636764160,"available":96575463424,"internally_reserved":0,"allocated":2359296,"data_stored":1699381,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542193,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45916,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325924,"statfs":{"total":96636764160,"available":96589746176,"internally_reserved":0,"allocated":2301952,"data_stored":1109013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542193,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55652,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316188,"statfs":{"total":96636764160,"available":96579776512,"internally_reserved":0,"allocated":2306048,"data_stored":1109032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":174,"seq":747324309511,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":2252,"kb_used_omap":34,"kb_used_meta":29277,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":2306048,"data_stored":1109032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35258,"internal_metadata":29980230},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542193,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51488,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320352,"statfs":{"total":96636764160,"available":96584040448,"internally_reserved":0,"allocated":2301952,"data_stored":1109013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":66,"seq":283467841567,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43228,"kb_used_data":2248,"kb_used_omap":6,"kb_used_meta":40953,"kb_avail":94328612,"statfs":{"total":96636764160,"available":96592498688,"internally_reserved":0,"allocated":2301952,"data_stored":1109013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7025,"internal_metadata":41936015},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59816,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312024,"statfs":{"total":96636764160,"available":96575512576,"internally_reserved":0,"allocated":2310144,"data_stored":1109051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542192,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41816,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330024,"statfs":{"total":96636764160,"available":96593944576,"internally_reserved":0,"allocated":2359296,"data_stored":1699381,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4238,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18084,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:39:42.753 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-14T22:36:15.070743+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=15, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.753 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-14T22:36:15.070743+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=15, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.753 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-14T22:36:15.070743+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=15, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.754 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-14T22:36:15.070743+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=15, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.754 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-14T22:36:15.070743+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=15, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.754 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-14T22:36:15.070743+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=15, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.754 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-14T22:36:15.070743+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=15, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.754 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-14T22:36:15.070743+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=15, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.754 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-14T22:36:15.070743+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=15, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.754 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-14T22:36:15.070743+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=15, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.754 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-14T22:36:15.070743+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=15, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.755 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-14T22:36:15.070743+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=15, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.755 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-14T22:36:05.010064+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.755 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-14T22:36:15.070743+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=15, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.755 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-14T22:36:05.010064+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.755 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-14T22:36:15.070743+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=15, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.755 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-14T22:36:05.010064+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.755 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-14T22:36:15.070743+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=15, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.755 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-14T22:36:05.010064+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.755 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-14T22:36:15.070743+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=15, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.756 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-14T22:36:15.070743+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=15, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.756 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-14T22:36:05.010064+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.756 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-14T22:36:15.070743+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=15, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.756 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-14T22:36:05.010064+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.756 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-14T22:36:15.070743+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=15, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.756 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-14T22:36:05.010064+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.756 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-14T22:36:15.070743+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=15, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.756 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-14T22:36:05.010064+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.756 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-14T22:35:58.989025+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=35, tm_sec=58, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.756 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-14T22:36:15.070743+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=15, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.757 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-14T22:36:15.070743+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=15, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.757 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-14T22:36:15.070743+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=15, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.757 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-14T22:36:15.070743+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=15, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.757 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-14T22:36:15.070743+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=15, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.757 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-14T22:36:15.070743+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=15, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.757 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-14T22:36:15.070743+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=15, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.757 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-14T22:36:15.070743+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=15, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.757 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-14T22:36:15.070743+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=15, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.757 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-14T22:36:15.070743+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=15, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.758 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-14T22:36:15.070743+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=15, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.758 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-14T22:36:15.070743+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=36, tm_sec=15, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=39, tm_sec=37, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:39:42.758 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-14T22:40:02.759 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:40:03.079 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-14T22:40:03.080 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-14T22:40:03.096 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-09-14T22:40:02.928948+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":897,"num_object_clones":0,"num_object_copies":1794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":897,"num_whiteouts":889,"num_read":206802,"num_read_kb":829088,"num_write":25375,"num_write_kb":384255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3336,"num_bytes_recovered":173602381,"num_keys_recovered":342,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22590,"ondisk_log_size":22590,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":372672,"kb_used_data":18112,"kb_used_omap":50,"kb_used_meta":354381,"kb_avail":754602048,"statfs":{"total":773094113280,"available":772712497152,"internally_reserved":0,"allocated":18546688,"data_stored":10052916,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":51845,"internal_metadata":362886523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004278"},"pg_stats":[{"pgid":"3.1f","version":"191'520","reported_seq":5361,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:42.557563+0000","last_change":"2024-09-14T22:39:42.557624+0000","last_active":"2024-09-14T22:39:42.557563+0000","last_peered":"2024-09-14T22:39:42.557563+0000","last_clean":"2024-09-14T22:39:42.557563+0000","last_became_active":"2024-09-14T22:37:25.718906+0000","last_became_peered":"2024-09-14T22:37:25.718906+0000","last_unstale":"2024-09-14T22:39:42.557563+0000","last_undegraded":"2024-09-14T22:39:42.557563+0000","last_fullsized":"2024-09-14T22:39:42.557563+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'520","last_scrub_stamp":"2024-09-14T22:39:42.557469+0000","last_deep_scrub":"191'520","last_deep_scrub_stamp":"2024-09-14T22:39:42.557469+0000","last_clean_scrub_stamp":"2024-09-14T22:39:42.557469+0000","objects_scrubbed":29,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:15:32.547029+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0.0023103960000000002,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":5019,"num_read_kb":16516,"num_write":439,"num_write_kb":7121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":3277632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"191'1251","reported_seq":7393,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:43.581402+0000","last_change":"2024-09-14T22:39:43.581503+0000","last_active":"2024-09-14T22:39:43.581402+0000","last_peered":"2024-09-14T22:39:43.581402+0000","last_clean":"2024-09-14T22:39:43.581402+0000","last_became_active":"2024-09-14T22:38:08.741149+0000","last_became_peered":"2024-09-14T22:38:08.741149+0000","last_unstale":"2024-09-14T22:39:43.581402+0000","last_undegraded":"2024-09-14T22:39:43.581402+0000","last_fullsized":"2024-09-14T22:39:43.581402+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'1251","last_scrub_stamp":"2024-09-14T22:39:43.581220+0000","last_deep_scrub":"191'1251","last_deep_scrub_stamp":"2024-09-14T22:39:43.581220+0000","last_clean_scrub_stamp":"2024-09-14T22:39:43.581220+0000","objects_scrubbed":20,"log_size":1251,"log_dups_size":0,"ondisk_log_size":1251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:18:54.152453+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.0024840370000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":11835,"num_read_kb":34663,"num_write":1722,"num_write_kb":10763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":7412288,"num_keys_recovered":90,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"191'753","reported_seq":10602,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:43.260396+0000","last_change":"2024-09-14T22:39:43.260572+0000","last_active":"2024-09-14T22:39:43.260396+0000","last_peered":"2024-09-14T22:39:43.260396+0000","last_clean":"2024-09-14T22:39:43.260396+0000","last_became_active":"2024-09-14T22:39:20.751505+0000","last_became_peered":"2024-09-14T22:39:20.751505+0000","last_unstale":"2024-09-14T22:39:43.260396+0000","last_undegraded":"2024-09-14T22:39:43.260396+0000","last_fullsized":"2024-09-14T22:39:43.260396+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'753","last_scrub_stamp":"2024-09-14T22:39:43.260177+0000","last_deep_scrub":"191'753","last_deep_scrub_stamp":"2024-09-14T22:39:43.260177+0000","last_clean_scrub_stamp":"2024-09-14T22:39:43.260177+0000","objects_scrubbed":29,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:00:51.530682+0000","scrub_duration":59,"objects_trimmed":0,"snaptrim_duration":0.0083529999999999993,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":10932,"num_read_kb":63225,"num_write":868,"num_write_kb":13703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":8030815,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"191'753","reported_seq":6481,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:44.463518+0000","last_change":"2024-09-14T22:39:44.463666+0000","last_active":"2024-09-14T22:39:44.463518+0000","last_peered":"2024-09-14T22:39:44.463518+0000","last_clean":"2024-09-14T22:39:44.463518+0000","last_became_active":"2024-09-14T22:39:25.084122+0000","last_became_peered":"2024-09-14T22:39:25.084122+0000","last_unstale":"2024-09-14T22:39:44.463518+0000","last_undegraded":"2024-09-14T22:39:44.463518+0000","last_fullsized":"2024-09-14T22:39:44.463518+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'753","last_scrub_stamp":"2024-09-14T22:39:44.463254+0000","last_deep_scrub":"191'753","last_deep_scrub_stamp":"2024-09-14T22:39:44.463254+0000","last_clean_scrub_stamp":"2024-09-14T22:39:44.463254+0000","objects_scrubbed":35,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:12:17.797374+0000","scrub_duration":67,"objects_trimmed":20,"snaptrim_duration":0.055023557000000001,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":34,"num_read":6435,"num_read_kb":26956,"num_write":884,"num_write_kb":8982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"191'627","reported_seq":5621,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:44.497732+0000","last_change":"2024-09-14T22:39:44.497872+0000","last_active":"2024-09-14T22:39:44.497732+0000","last_peered":"2024-09-14T22:39:44.497732+0000","last_clean":"2024-09-14T22:39:44.497732+0000","last_became_active":"2024-09-14T22:39:13.025036+0000","last_became_peered":"2024-09-14T22:39:13.025036+0000","last_unstale":"2024-09-14T22:39:44.497732+0000","last_undegraded":"2024-09-14T22:39:44.497732+0000","last_fullsized":"2024-09-14T22:39:44.497732+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'627","last_scrub_stamp":"2024-09-14T22:39:44.497498+0000","last_deep_scrub":"191'627","last_deep_scrub_stamp":"2024-09-14T22:39:44.497498+0000","last_clean_scrub_stamp":"2024-09-14T22:39:44.497498+0000","objects_scrubbed":25,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:19:05.008031+0000","scrub_duration":51,"objects_trimmed":14,"snaptrim_duration":0.061855755999999998,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":5663,"num_read_kb":32434,"num_write":648,"num_write_kb":13330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":7858104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"191'561","reported_seq":4269,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:44.514677+0000","last_change":"2024-09-14T22:39:44.514787+0000","last_active":"2024-09-14T22:39:44.514677+0000","last_peered":"2024-09-14T22:39:44.514677+0000","last_clean":"2024-09-14T22:39:44.514677+0000","last_became_active":"2024-09-14T22:39:13.016132+0000","last_became_peered":"2024-09-14T22:39:13.016132+0000","last_unstale":"2024-09-14T22:39:44.514677+0000","last_undegraded":"2024-09-14T22:39:44.514677+0000","last_fullsized":"2024-09-14T22:39:44.514677+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'561","last_scrub_stamp":"2024-09-14T22:39:44.514494+0000","last_deep_scrub":"191'561","last_deep_scrub_stamp":"2024-09-14T22:39:44.514494+0000","last_clean_scrub_stamp":"2024-09-14T22:39:44.514494+0000","objects_scrubbed":34,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:10:55.240522+0000","scrub_duration":63,"objects_trimmed":10,"snaptrim_duration":0.069063225000000006,"stat_sum":{"num_bytes":19,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":33,"num_read":4551,"num_read_kb":13173,"num_write":474,"num_write_kb":7035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":236,"num_bytes_recovered":10036538,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"191'497","reported_seq":5700,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:40.180633+0000","last_change":"2024-09-14T22:39:40.180676+0000","last_active":"2024-09-14T22:39:40.180633+0000","last_peered":"2024-09-14T22:39:40.180633+0000","last_clean":"2024-09-14T22:39:40.180633+0000","last_became_active":"2024-09-14T22:36:15.700990+0000","last_became_peered":"2024-09-14T22:36:15.700990+0000","last_unstale":"2024-09-14T22:39:40.180633+0000","last_undegraded":"2024-09-14T22:39:40.180633+0000","last_fullsized":"2024-09-14T22:39:40.180633+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'497","last_scrub_stamp":"2024-09-14T22:39:40.180544+0000","last_deep_scrub":"191'497","last_deep_scrub_stamp":"2024-09-14T22:39:40.180544+0000","last_clean_scrub_stamp":"2024-09-14T22:39:40.180544+0000","objects_scrubbed":32,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:08:36.414663+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.0013968780000000001,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":5273,"num_read_kb":15380,"num_write":425,"num_write_kb":5486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"191'532","reported_seq":5027,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:41.610602+0000","last_change":"2024-09-14T22:39:41.610659+0000","last_active":"2024-09-14T22:39:41.610602+0000","last_peered":"2024-09-14T22:39:41.610602+0000","last_clean":"2024-09-14T22:39:41.610602+0000","last_became_active":"2024-09-14T22:38:07.734308+0000","last_became_peered":"2024-09-14T22:38:07.734308+0000","last_unstale":"2024-09-14T22:39:41.610602+0000","last_undegraded":"2024-09-14T22:39:41.610602+0000","last_fullsized":"2024-09-14T22:39:41.610602+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'532","last_scrub_stamp":"2024-09-14T22:39:41.610519+0000","last_deep_scrub":"191'532","last_deep_scrub_stamp":"2024-09-14T22:39:41.610519+0000","last_clean_scrub_stamp":"2024-09-14T22:39:41.610519+0000","objects_scrubbed":21,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:37:26.334973+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0013730859999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4941,"num_read_kb":35923,"num_write":636,"num_write_kb":27736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":20067293,"num_keys_recovered":21,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"191'496","reported_seq":4904,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:40.555618+0000","last_change":"2024-09-14T22:39:40.555705+0000","last_active":"2024-09-14T22:39:40.555618+0000","last_peered":"2024-09-14T22:39:40.555618+0000","last_clean":"2024-09-14T22:39:40.555618+0000","last_became_active":"2024-09-14T22:36:15.715869+0000","last_became_peered":"2024-09-14T22:36:15.715869+0000","last_unstale":"2024-09-14T22:39:40.555618+0000","last_undegraded":"2024-09-14T22:39:40.555618+0000","last_fullsized":"2024-09-14T22:39:40.555618+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'496","last_scrub_stamp":"2024-09-14T22:39:40.555487+0000","last_deep_scrub":"191'496","last_deep_scrub_stamp":"2024-09-14T22:39:40.555487+0000","last_clean_scrub_stamp":"2024-09-14T22:39:40.555487+0000","objects_scrubbed":29,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:38:34.404488+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0.0023931830000000001,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":4593,"num_read_kb":25142,"num_write":516,"num_write_kb":22003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"191'638","reported_seq":6712,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:41.596288+0000","last_change":"2024-09-14T22:39:41.596373+0000","last_active":"2024-09-14T22:39:41.596288+0000","last_peered":"2024-09-14T22:39:41.596288+0000","last_clean":"2024-09-14T22:39:41.596288+0000","last_became_active":"2024-09-14T22:36:15.725078+0000","last_became_peered":"2024-09-14T22:36:15.725078+0000","last_unstale":"2024-09-14T22:39:41.596288+0000","last_undegraded":"2024-09-14T22:39:41.596288+0000","last_fullsized":"2024-09-14T22:39:41.596288+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'638","last_scrub_stamp":"2024-09-14T22:39:41.596094+0000","last_deep_scrub":"191'638","last_deep_scrub_stamp":"2024-09-14T22:39:41.596094+0000","last_clean_scrub_stamp":"2024-09-14T22:39:41.596094+0000","objects_scrubbed":33,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:51:04.063234+0000","scrub_duration":71,"objects_trimmed":0,"snaptrim_duration":0.0035779840000000002,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":6675,"num_read_kb":51224,"num_write":729,"num_write_kb":18952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"191'497","reported_seq":4846,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:45.057839+0000","last_change":"2024-09-14T22:39:45.057990+0000","last_active":"2024-09-14T22:39:45.057839+0000","last_peered":"2024-09-14T22:39:45.057839+0000","last_clean":"2024-09-14T22:39:45.057839+0000","last_became_active":"2024-09-14T22:36:15.717341+0000","last_became_peered":"2024-09-14T22:36:15.717341+0000","last_unstale":"2024-09-14T22:39:45.057839+0000","last_undegraded":"2024-09-14T22:39:45.057839+0000","last_fullsized":"2024-09-14T22:39:45.057839+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'497","last_scrub_stamp":"2024-09-14T22:39:45.057605+0000","last_deep_scrub":"191'497","last_deep_scrub_stamp":"2024-09-14T22:39:45.057605+0000","last_clean_scrub_stamp":"2024-09-14T22:39:45.057605+0000","objects_scrubbed":30,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:54:56.227579+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0.0016388500000000001,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":5090,"num_read_kb":12789,"num_write":419,"num_write_kb":7375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"191'1232","reported_seq":9164,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:45.495446+0000","last_change":"2024-09-14T22:39:45.495543+0000","last_active":"2024-09-14T22:39:45.495446+0000","last_peered":"2024-09-14T22:39:45.495446+0000","last_clean":"2024-09-14T22:39:45.495446+0000","last_became_active":"2024-09-14T22:39:13.026842+0000","last_became_peered":"2024-09-14T22:39:13.026842+0000","last_unstale":"2024-09-14T22:39:45.495446+0000","last_undegraded":"2024-09-14T22:39:45.495446+0000","last_fullsized":"2024-09-14T22:39:45.495446+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'1232","last_scrub_stamp":"2024-09-14T22:39:45.495262+0000","last_deep_scrub":"191'1232","last_deep_scrub_stamp":"2024-09-14T22:39:45.495262+0000","last_clean_scrub_stamp":"2024-09-14T22:39:45.495262+0000","objects_scrubbed":30,"log_size":1232,"log_dups_size":0,"ondisk_log_size":1232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:43:26.757724+0000","scrub_duration":75,"objects_trimmed":14,"snaptrim_duration":0.055942110000000003,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":11247,"num_read_kb":29618,"num_write":1623,"num_write_kb":11950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":265,"num_bytes_recovered":17498216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":365,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:41.913536+0000","last_change":"2024-09-14T22:39:41.913589+0000","last_active":"2024-09-14T22:39:41.913536+0000","last_peered":"2024-09-14T22:39:41.913536+0000","last_clean":"2024-09-14T22:39:41.913536+0000","last_became_active":"2024-09-14T22:36:06.040961+0000","last_became_peered":"2024-09-14T22:36:06.040961+0000","last_unstale":"2024-09-14T22:39:41.913536+0000","last_undegraded":"2024-09-14T22:39:41.913536+0000","last_fullsized":"2024-09-14T22:39:41.913536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:39:41.913337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:39:41.913337+0000","last_clean_scrub_stamp":"2024-09-14T22:39:41.913337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:16:12.460795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010780950000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"191'767","reported_seq":3894,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:39.536894+0000","last_change":"2024-09-14T22:39:39.536924+0000","last_active":"2024-09-14T22:39:39.536894+0000","last_peered":"2024-09-14T22:39:39.536894+0000","last_clean":"2024-09-14T22:39:39.536894+0000","last_became_active":"2024-09-14T22:36:15.700982+0000","last_became_peered":"2024-09-14T22:36:15.700982+0000","last_unstale":"2024-09-14T22:39:39.536894+0000","last_undegraded":"2024-09-14T22:39:39.536894+0000","last_fullsized":"2024-09-14T22:39:39.536894+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'767","last_scrub_stamp":"2024-09-14T22:39:39.536810+0000","last_deep_scrub":"191'767","last_deep_scrub_stamp":"2024-09-14T22:39:39.536810+0000","last_clean_scrub_stamp":"2024-09-14T22:39:39.536810+0000","objects_scrubbed":29,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:21:48.650778+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.001672663,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":4804,"num_read_kb":22802,"num_write":686,"num_write_kb":12175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":190,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:45.190772+0000","last_change":"2024-09-14T22:39:45.190829+0000","last_active":"2024-09-14T22:39:45.190772+0000","last_peered":"2024-09-14T22:39:45.190772+0000","last_clean":"2024-09-14T22:39:45.190772+0000","last_became_active":"2024-09-14T22:39:20.885045+0000","last_became_peered":"2024-09-14T22:39:20.885045+0000","last_unstale":"2024-09-14T22:39:45.190772+0000","last_undegraded":"2024-09-14T22:39:45.190772+0000","last_fullsized":"2024-09-14T22:39:45.190772+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:39:45.190578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:39:45.190578+0000","last_clean_scrub_stamp":"2024-09-14T22:39:45.190578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:37:09.362933+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"191'563","reported_seq":5859,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:44.554700+0000","last_change":"2024-09-14T22:39:44.554850+0000","last_active":"2024-09-14T22:39:44.554700+0000","last_peered":"2024-09-14T22:39:44.554700+0000","last_clean":"2024-09-14T22:39:44.554700+0000","last_became_active":"2024-09-14T22:39:23.064710+0000","last_became_peered":"2024-09-14T22:39:23.064710+0000","last_unstale":"2024-09-14T22:39:44.554700+0000","last_undegraded":"2024-09-14T22:39:44.554700+0000","last_fullsized":"2024-09-14T22:39:44.554700+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'563","last_scrub_stamp":"2024-09-14T22:39:44.554452+0000","last_deep_scrub":"191'563","last_deep_scrub_stamp":"2024-09-14T22:39:44.554452+0000","last_clean_scrub_stamp":"2024-09-14T22:39:44.554452+0000","objects_scrubbed":26,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:42:27.582436+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0.0044565630000000002,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":6110,"num_read_kb":28977,"num_write":556,"num_write_kb":8495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":367,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:44.023337+0000","last_change":"2024-09-14T22:39:44.023388+0000","last_active":"2024-09-14T22:39:44.023337+0000","last_peered":"2024-09-14T22:39:44.023337+0000","last_clean":"2024-09-14T22:39:44.023337+0000","last_became_active":"2024-09-14T22:36:06.266908+0000","last_became_peered":"2024-09-14T22:36:06.266908+0000","last_unstale":"2024-09-14T22:39:44.023337+0000","last_undegraded":"2024-09-14T22:39:44.023337+0000","last_fullsized":"2024-09-14T22:39:44.023337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:39:44.023137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:39:44.023137+0000","last_clean_scrub_stamp":"2024-09-14T22:39:44.023137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:59:12.776770+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00098249300000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"191'1505","reported_seq":5597,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:42.215087+0000","last_change":"2024-09-14T22:39:42.215110+0000","last_active":"2024-09-14T22:39:42.215087+0000","last_peered":"2024-09-14T22:39:42.215087+0000","last_clean":"2024-09-14T22:39:42.215087+0000","last_became_active":"2024-09-14T22:39:20.751272+0000","last_became_peered":"2024-09-14T22:39:20.751272+0000","last_unstale":"2024-09-14T22:39:42.215087+0000","last_undegraded":"2024-09-14T22:39:42.215087+0000","last_fullsized":"2024-09-14T22:39:42.215087+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'1505","last_scrub_stamp":"2024-09-14T22:39:42.215016+0000","last_deep_scrub":"191'1505","last_deep_scrub_stamp":"2024-09-14T22:39:42.215016+0000","last_clean_scrub_stamp":"2024-09-14T22:39:42.215016+0000","objects_scrubbed":25,"log_size":1505,"log_dups_size":0,"ondisk_log_size":1505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:35:31.784528+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.0037027409999999998,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":9975,"num_read_kb":23940,"num_write":2463,"num_write_kb":17944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":4264139,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":97,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:41.152186+0000","last_change":"2024-09-14T22:39:41.152206+0000","last_active":"2024-09-14T22:39:41.152186+0000","last_peered":"2024-09-14T22:39:41.152186+0000","last_clean":"2024-09-14T22:39:41.152186+0000","last_became_active":"2024-09-14T22:39:20.748164+0000","last_became_peered":"2024-09-14T22:39:20.748164+0000","last_unstale":"2024-09-14T22:39:41.152186+0000","last_undegraded":"2024-09-14T22:39:41.152186+0000","last_fullsized":"2024-09-14T22:39:41.152186+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:39:41.152106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:39:41.152106+0000","last_clean_scrub_stamp":"2024-09-14T22:39:41.152106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:12:08.301579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"191'649","reported_seq":4911,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:45.474834+0000","last_change":"2024-09-14T22:39:45.474990+0000","last_active":"2024-09-14T22:39:45.474834+0000","last_peered":"2024-09-14T22:39:45.474834+0000","last_clean":"2024-09-14T22:39:45.474834+0000","last_became_active":"2024-09-14T22:39:25.085794+0000","last_became_peered":"2024-09-14T22:39:25.085794+0000","last_unstale":"2024-09-14T22:39:45.474834+0000","last_undegraded":"2024-09-14T22:39:45.474834+0000","last_fullsized":"2024-09-14T22:39:45.474834+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'649","last_scrub_stamp":"2024-09-14T22:39:45.474615+0000","last_deep_scrub":"191'649","last_deep_scrub_stamp":"2024-09-14T22:39:45.474615+0000","last_clean_scrub_stamp":"2024-09-14T22:39:45.474615+0000","objects_scrubbed":24,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:41:01.331384+0000","scrub_duration":55,"objects_trimmed":0,"snaptrim_duration":0.002907598,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":5488,"num_read_kb":14305,"num_write":599,"num_write_kb":5812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":5797306,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"191'627","reported_seq":5511,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:43.497325+0000","last_change":"2024-09-14T22:39:43.497401+0000","last_active":"2024-09-14T22:39:43.497325+0000","last_peered":"2024-09-14T22:39:43.497325+0000","last_clean":"2024-09-14T22:39:43.497325+0000","last_became_active":"2024-09-14T22:39:13.012755+0000","last_became_peered":"2024-09-14T22:39:13.012755+0000","last_unstale":"2024-09-14T22:39:43.497325+0000","last_undegraded":"2024-09-14T22:39:43.497325+0000","last_fullsized":"2024-09-14T22:39:43.497325+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'627","last_scrub_stamp":"2024-09-14T22:39:43.497117+0000","last_deep_scrub":"191'627","last_deep_scrub_stamp":"2024-09-14T22:39:43.497117+0000","last_clean_scrub_stamp":"2024-09-14T22:39:43.497117+0000","objects_scrubbed":31,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:23:53.852574+0000","scrub_duration":55,"objects_trimmed":8,"snaptrim_duration":0.084951515000000005,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":30,"num_read":4623,"num_read_kb":25789,"num_write":608,"num_write_kb":14115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"191'3","reported_seq":264,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:43.444520+0000","last_change":"2024-09-14T22:39:43.444587+0000","last_active":"2024-09-14T22:39:43.444520+0000","last_peered":"2024-09-14T22:39:43.444520+0000","last_clean":"2024-09-14T22:39:43.444520+0000","last_became_active":"2024-09-14T22:39:25.085378+0000","last_became_peered":"2024-09-14T22:39:25.085378+0000","last_unstale":"2024-09-14T22:39:43.444520+0000","last_undegraded":"2024-09-14T22:39:43.444520+0000","last_fullsized":"2024-09-14T22:39:43.444520+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'3","last_scrub_stamp":"2024-09-14T22:39:43.444327+0000","last_deep_scrub":"191'3","last_deep_scrub_stamp":"2024-09-14T22:39:43.444327+0000","last_clean_scrub_stamp":"2024-09-14T22:39:43.444327+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:31:58.518990+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0021617870000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"191'1016","reported_seq":9320,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:44.288333+0000","last_change":"2024-09-14T22:39:44.288398+0000","last_active":"2024-09-14T22:39:44.288333+0000","last_peered":"2024-09-14T22:39:44.288333+0000","last_clean":"2024-09-14T22:39:44.288333+0000","last_became_active":"2024-09-14T22:39:20.751993+0000","last_became_peered":"2024-09-14T22:39:20.751993+0000","last_unstale":"2024-09-14T22:39:44.288333+0000","last_undegraded":"2024-09-14T22:39:44.288333+0000","last_fullsized":"2024-09-14T22:39:44.288333+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'1016","last_scrub_stamp":"2024-09-14T22:39:44.288143+0000","last_deep_scrub":"191'1016","last_deep_scrub_stamp":"2024-09-14T22:39:44.288143+0000","last_clean_scrub_stamp":"2024-09-14T22:39:44.288143+0000","objects_scrubbed":25,"log_size":1016,"log_dups_size":0,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:24:21.920595+0000","scrub_duration":55,"objects_trimmed":0,"snaptrim_duration":0.0030469329999999999,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":10327,"num_read_kb":29859,"num_write":1283,"num_write_kb":15017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":4851034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":302,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:40.288322+0000","last_change":"2024-09-14T22:39:40.288343+0000","last_active":"2024-09-14T22:39:40.288322+0000","last_peered":"2024-09-14T22:39:40.288322+0000","last_clean":"2024-09-14T22:39:40.288322+0000","last_became_active":"2024-09-14T22:38:05.723636+0000","last_became_peered":"2024-09-14T22:38:05.723636+0000","last_unstale":"2024-09-14T22:39:40.288322+0000","last_undegraded":"2024-09-14T22:39:40.288322+0000","last_fullsized":"2024-09-14T22:39:40.288322+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:39:40.288237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:39:40.288237+0000","last_clean_scrub_stamp":"2024-09-14T22:39:40.288237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:22.477455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"191'473","reported_seq":5833,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:43.522548+0000","last_change":"2024-09-14T22:39:43.522660+0000","last_active":"2024-09-14T22:39:43.522548+0000","last_peered":"2024-09-14T22:39:43.522548+0000","last_clean":"2024-09-14T22:39:43.522548+0000","last_became_active":"2024-09-14T22:39:13.025169+0000","last_became_peered":"2024-09-14T22:39:13.025169+0000","last_unstale":"2024-09-14T22:39:43.522548+0000","last_undegraded":"2024-09-14T22:39:43.522548+0000","last_fullsized":"2024-09-14T22:39:43.522548+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'473","last_scrub_stamp":"2024-09-14T22:39:43.522355+0000","last_deep_scrub":"191'473","last_deep_scrub_stamp":"2024-09-14T22:39:43.522355+0000","last_clean_scrub_stamp":"2024-09-14T22:39:43.522355+0000","objects_scrubbed":24,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:13:42.387867+0000","scrub_duration":39,"objects_trimmed":4,"snaptrim_duration":0.027619476,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":4593,"num_read_kb":23198,"num_write":420,"num_write_kb":7231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":3679238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":62,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:41.463888+0000","last_change":"2024-09-14T22:39:41.463945+0000","last_active":"2024-09-14T22:39:41.463888+0000","last_peered":"2024-09-14T22:39:41.463888+0000","last_clean":"2024-09-14T22:39:41.463888+0000","last_became_active":"2024-09-14T22:39:13.008709+0000","last_became_peered":"2024-09-14T22:39:13.008709+0000","last_unstale":"2024-09-14T22:39:41.463888+0000","last_undegraded":"2024-09-14T22:39:41.463888+0000","last_fullsized":"2024-09-14T22:39:41.463888+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:39:41.463681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:39:41.463681+0000","last_clean_scrub_stamp":"2024-09-14T22:39:41.463681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:41:58.216835+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0010817520000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"191'1140","reported_seq":4805,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:45.502325+0000","last_change":"2024-09-14T22:39:45.502515+0000","last_active":"2024-09-14T22:39:45.502325+0000","last_peered":"2024-09-14T22:39:45.502325+0000","last_clean":"2024-09-14T22:39:45.502325+0000","last_became_active":"2024-09-14T22:39:23.064694+0000","last_became_peered":"2024-09-14T22:39:23.064694+0000","last_unstale":"2024-09-14T22:39:45.502325+0000","last_undegraded":"2024-09-14T22:39:45.502325+0000","last_fullsized":"2024-09-14T22:39:45.502325+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'1140","last_scrub_stamp":"2024-09-14T22:39:45.502070+0000","last_deep_scrub":"191'1140","last_deep_scrub_stamp":"2024-09-14T22:39:45.502070+0000","last_clean_scrub_stamp":"2024-09-14T22:39:45.502070+0000","objects_scrubbed":24,"log_size":1140,"log_dups_size":0,"ondisk_log_size":1140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:31:13.590653+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0.0033884660000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":7515,"num_read_kb":16135,"num_write":1577,"num_write_kb":11855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":2936757,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"191'2","reported_seq":270,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:42.485354+0000","last_change":"2024-09-14T22:39:42.485518+0000","last_active":"2024-09-14T22:39:42.485354+0000","last_peered":"2024-09-14T22:39:42.485354+0000","last_clean":"2024-09-14T22:39:42.485354+0000","last_became_active":"2024-09-14T22:39:25.081686+0000","last_became_peered":"2024-09-14T22:39:25.081686+0000","last_unstale":"2024-09-14T22:39:42.485354+0000","last_undegraded":"2024-09-14T22:39:42.485354+0000","last_fullsized":"2024-09-14T22:39:42.485354+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'2","last_scrub_stamp":"2024-09-14T22:39:42.485031+0000","last_deep_scrub":"191'2","last_deep_scrub_stamp":"2024-09-14T22:39:42.485031+0000","last_clean_scrub_stamp":"2024-09-14T22:39:42.485031+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:43:22.797370+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0011401880000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"191'119","reported_seq":520,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:43.046027+0000","last_change":"2024-09-14T22:39:43.046131+0000","last_active":"2024-09-14T22:39:43.046027+0000","last_peered":"2024-09-14T22:39:43.046027+0000","last_clean":"2024-09-14T22:39:43.046027+0000","last_became_active":"2024-09-14T22:36:00.400529+0000","last_became_peered":"2024-09-14T22:36:00.400529+0000","last_unstale":"2024-09-14T22:39:43.046027+0000","last_undegraded":"2024-09-14T22:39:43.046027+0000","last_fullsized":"2024-09-14T22:39:43.046027+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'119","last_scrub_stamp":"2024-09-14T22:39:43.045844+0000","last_deep_scrub":"191'119","last_deep_scrub_stamp":"2024-09-14T22:39:43.045844+0000","last_clean_scrub_stamp":"2024-09-14T22:39:43.045844+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:07:05.536039+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"191'694","reported_seq":7858,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:42.599322+0000","last_change":"2024-09-14T22:39:42.599384+0000","last_active":"2024-09-14T22:39:42.599322+0000","last_peered":"2024-09-14T22:39:42.599322+0000","last_clean":"2024-09-14T22:39:42.599322+0000","last_became_active":"2024-09-14T22:38:08.739090+0000","last_became_peered":"2024-09-14T22:38:08.739090+0000","last_unstale":"2024-09-14T22:39:42.599322+0000","last_undegraded":"2024-09-14T22:39:42.599322+0000","last_fullsized":"2024-09-14T22:39:42.599322+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'694","last_scrub_stamp":"2024-09-14T22:39:42.599209+0000","last_deep_scrub":"191'694","last_deep_scrub_stamp":"2024-09-14T22:39:42.599209+0000","last_clean_scrub_stamp":"2024-09-14T22:39:42.599209+0000","objects_scrubbed":18,"log_size":694,"log_dups_size":0,"ondisk_log_size":694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:20:29.069778+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0012849739999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":8821,"num_read_kb":22101,"num_write":951,"num_write_kb":14470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":18690892,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"191'493","reported_seq":7456,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:42.503956+0000","last_change":"2024-09-14T22:39:42.504027+0000","last_active":"2024-09-14T22:39:42.503956+0000","last_peered":"2024-09-14T22:39:42.503956+0000","last_clean":"2024-09-14T22:39:42.503956+0000","last_became_active":"2024-09-14T22:39:13.008690+0000","last_became_peered":"2024-09-14T22:39:13.008690+0000","last_unstale":"2024-09-14T22:39:42.503956+0000","last_undegraded":"2024-09-14T22:39:42.503956+0000","last_fullsized":"2024-09-14T22:39:42.503956+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'493","last_scrub_stamp":"2024-09-14T22:39:42.503766+0000","last_deep_scrub":"191'493","last_deep_scrub_stamp":"2024-09-14T22:39:42.503766+0000","last_clean_scrub_stamp":"2024-09-14T22:39:42.503766+0000","objects_scrubbed":32,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:59:06.335476+0000","scrub_duration":51,"objects_trimmed":10,"snaptrim_duration":0.064091727000000001,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":7056,"num_read_kb":18679,"num_write":421,"num_write_kb":7008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":16527933,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"191'832","reported_seq":4535,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:43.971020+0000","last_change":"2024-09-14T22:39:43.971085+0000","last_active":"2024-09-14T22:39:43.971020+0000","last_peered":"2024-09-14T22:39:43.971020+0000","last_clean":"2024-09-14T22:39:43.971020+0000","last_became_active":"2024-09-14T22:39:13.024372+0000","last_became_peered":"2024-09-14T22:39:13.024372+0000","last_unstale":"2024-09-14T22:39:43.971020+0000","last_undegraded":"2024-09-14T22:39:43.971020+0000","last_fullsized":"2024-09-14T22:39:43.971020+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'832","last_scrub_stamp":"2024-09-14T22:39:43.970846+0000","last_deep_scrub":"191'832","last_deep_scrub_stamp":"2024-09-14T22:39:43.970846+0000","last_clean_scrub_stamp":"2024-09-14T22:39:43.970846+0000","objects_scrubbed":29,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:34:02.893757+0000","scrub_duration":59,"objects_trimmed":8,"snaptrim_duration":0.037620476,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":28,"num_read":6415,"num_read_kb":26025,"num_write":756,"num_write_kb":14809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"191'610","reported_seq":3000,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:40.649195+0000","last_change":"2024-09-14T22:39:40.649224+0000","last_active":"2024-09-14T22:39:40.649195+0000","last_peered":"2024-09-14T22:39:40.649195+0000","last_clean":"2024-09-14T22:39:40.649195+0000","last_became_active":"2024-09-14T22:36:15.702885+0000","last_became_peered":"2024-09-14T22:36:15.702885+0000","last_unstale":"2024-09-14T22:39:40.649195+0000","last_undegraded":"2024-09-14T22:39:40.649195+0000","last_fullsized":"2024-09-14T22:39:40.649195+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'610","last_scrub_stamp":"2024-09-14T22:39:40.649118+0000","last_deep_scrub":"191'610","last_deep_scrub_stamp":"2024-09-14T22:39:40.649118+0000","last_clean_scrub_stamp":"2024-09-14T22:39:40.649118+0000","objects_scrubbed":27,"log_size":610,"log_dups_size":0,"ondisk_log_size":610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:43:15.142922+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0031173400000000001,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":3570,"num_read_kb":11670,"num_write":536,"num_write_kb":6877,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"191'470","reported_seq":6955,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:46.489694+0000","last_change":"2024-09-14T22:39:46.489786+0000","last_active":"2024-09-14T22:39:46.489694+0000","last_peered":"2024-09-14T22:39:46.489694+0000","last_clean":"2024-09-14T22:39:46.489694+0000","last_became_active":"2024-09-14T22:39:25.088012+0000","last_became_peered":"2024-09-14T22:39:25.088012+0000","last_unstale":"2024-09-14T22:39:46.489694+0000","last_undegraded":"2024-09-14T22:39:46.489694+0000","last_fullsized":"2024-09-14T22:39:46.489694+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'470","last_scrub_stamp":"2024-09-14T22:39:46.489511+0000","last_deep_scrub":"191'470","last_deep_scrub_stamp":"2024-09-14T22:39:46.489511+0000","last_clean_scrub_stamp":"2024-09-14T22:39:46.489511+0000","objects_scrubbed":23,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:41:00.576323+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0.0018646120000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5808,"num_read_kb":33509,"num_write":486,"num_write_kb":17853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":23218688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"191'671","reported_seq":4267,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:46.040476+0000","last_change":"2024-09-14T22:39:46.040592+0000","last_active":"2024-09-14T22:39:46.040476+0000","last_peered":"2024-09-14T22:39:46.040476+0000","last_clean":"2024-09-14T22:39:46.040476+0000","last_became_active":"2024-09-14T22:36:15.723205+0000","last_became_peered":"2024-09-14T22:36:15.723205+0000","last_unstale":"2024-09-14T22:39:46.040476+0000","last_undegraded":"2024-09-14T22:39:46.040476+0000","last_fullsized":"2024-09-14T22:39:46.040476+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'671","last_scrub_stamp":"2024-09-14T22:39:46.040265+0000","last_deep_scrub":"191'671","last_deep_scrub_stamp":"2024-09-14T22:39:46.040265+0000","last_clean_scrub_stamp":"2024-09-14T22:39:46.040265+0000","objects_scrubbed":24,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:10:28.509465+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0.001862636,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":4348,"num_read_kb":16212,"num_write":775,"num_write_kb":5956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"191'582","reported_seq":2962,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:47.019516+0000","last_change":"2024-09-14T22:39:47.019600+0000","last_active":"2024-09-14T22:39:47.019516+0000","last_peered":"2024-09-14T22:39:47.019516+0000","last_clean":"2024-09-14T22:39:47.019516+0000","last_became_active":"2024-09-14T22:39:13.025887+0000","last_became_peered":"2024-09-14T22:39:13.025887+0000","last_unstale":"2024-09-14T22:39:47.019516+0000","last_undegraded":"2024-09-14T22:39:47.019516+0000","last_fullsized":"2024-09-14T22:39:47.019516+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'582","last_scrub_stamp":"2024-09-14T22:39:47.019321+0000","last_deep_scrub":"191'582","last_deep_scrub_stamp":"2024-09-14T22:39:47.019321+0000","last_clean_scrub_stamp":"2024-09-14T22:39:47.019321+0000","objects_scrubbed":33,"log_size":582,"log_dups_size":0,"ondisk_log_size":582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:03:19.707042+0000","scrub_duration":63,"objects_trimmed":10,"snaptrim_duration":0.030321287999999998,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":2666,"num_read_kb":8490,"num_write":594,"num_write_kb":3830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":4944418,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"191'526","reported_seq":8540,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:49.026288+0000","last_change":"2024-09-14T22:39:49.026357+0000","last_active":"2024-09-14T22:39:49.026288+0000","last_peered":"2024-09-14T22:39:49.026288+0000","last_clean":"2024-09-14T22:39:49.026288+0000","last_became_active":"2024-09-14T22:39:13.030456+0000","last_became_peered":"2024-09-14T22:39:13.030456+0000","last_unstale":"2024-09-14T22:39:49.026288+0000","last_undegraded":"2024-09-14T22:39:49.026288+0000","last_fullsized":"2024-09-14T22:39:49.026288+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'526","last_scrub_stamp":"2024-09-14T22:39:49.026071+0000","last_deep_scrub":"191'526","last_deep_scrub_stamp":"2024-09-14T22:39:49.026071+0000","last_clean_scrub_stamp":"2024-09-14T22:39:49.026071+0000","objects_scrubbed":27,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:38:53.344947+0000","scrub_duration":43,"objects_trimmed":8,"snaptrim_duration":0.017930723999999999,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":8487,"num_read_kb":23501,"num_write":537,"num_write_kb":9535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"191'521","reported_seq":4148,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:42.987402+0000","last_change":"2024-09-14T22:39:42.987477+0000","last_active":"2024-09-14T22:39:42.987402+0000","last_peered":"2024-09-14T22:39:42.987402+0000","last_clean":"2024-09-14T22:39:42.987402+0000","last_became_active":"2024-09-14T22:37:26.717088+0000","last_became_peered":"2024-09-14T22:37:26.717088+0000","last_unstale":"2024-09-14T22:39:42.987402+0000","last_undegraded":"2024-09-14T22:39:42.987402+0000","last_fullsized":"2024-09-14T22:39:42.987402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'521","last_scrub_stamp":"2024-09-14T22:39:42.987200+0000","last_deep_scrub":"191'521","last_deep_scrub_stamp":"2024-09-14T22:39:42.987200+0000","last_clean_scrub_stamp":"2024-09-14T22:39:42.987200+0000","objects_scrubbed":28,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:52:21.030337+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0.0024508580000000002,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":3408,"num_read_kb":29082,"num_write":535,"num_write_kb":14817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"191'719","reported_seq":2777,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:48.065343+0000","last_change":"2024-09-14T22:39:48.065465+0000","last_active":"2024-09-14T22:39:48.065343+0000","last_peered":"2024-09-14T22:39:48.065343+0000","last_clean":"2024-09-14T22:39:48.065343+0000","last_became_active":"2024-09-14T22:39:13.029544+0000","last_became_peered":"2024-09-14T22:39:13.029544+0000","last_unstale":"2024-09-14T22:39:48.065343+0000","last_undegraded":"2024-09-14T22:39:48.065343+0000","last_fullsized":"2024-09-14T22:39:48.065343+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'719","last_scrub_stamp":"2024-09-14T22:39:48.065084+0000","last_deep_scrub":"191'719","last_deep_scrub_stamp":"2024-09-14T22:39:48.065084+0000","last_clean_scrub_stamp":"2024-09-14T22:39:48.065084+0000","objects_scrubbed":34,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:12:12.173897+0000","scrub_duration":59,"objects_trimmed":8,"snaptrim_duration":0.020111626000000001,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":3515,"num_read_kb":11729,"num_write":870,"num_write_kb":10937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"191'619","reported_seq":8771,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:41.604682+0000","last_change":"2024-09-14T22:39:41.604741+0000","last_active":"2024-09-14T22:39:41.604682+0000","last_peered":"2024-09-14T22:39:41.604682+0000","last_clean":"2024-09-14T22:39:41.604682+0000","last_became_active":"2024-09-14T22:36:15.716696+0000","last_became_peered":"2024-09-14T22:36:15.716696+0000","last_unstale":"2024-09-14T22:39:41.604682+0000","last_undegraded":"2024-09-14T22:39:41.604682+0000","last_fullsized":"2024-09-14T22:39:41.604682+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'619","last_scrub_stamp":"2024-09-14T22:39:41.604560+0000","last_deep_scrub":"191'619","last_deep_scrub_stamp":"2024-09-14T22:39:41.604560+0000","last_clean_scrub_stamp":"2024-09-14T22:39:41.604560+0000","objects_scrubbed":30,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:29:40.347132+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0.0019004219999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":9756,"num_read_kb":50618,"num_write":568,"num_write_kb":13171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"191'632","reported_seq":7726,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-14T22:39:50.003537+0000","last_change":"2024-09-14T22:39:50.003644+0000","last_active":"2024-09-14T22:39:50.003537+0000","last_peered":"2024-09-14T22:39:50.003537+0000","last_clean":"2024-09-14T22:39:50.003537+0000","last_became_active":"2024-09-14T22:39:13.030502+0000","last_became_peered":"2024-09-14T22:39:13.030502+0000","last_unstale":"2024-09-14T22:39:50.003537+0000","last_undegraded":"2024-09-14T22:39:50.003537+0000","last_fullsized":"2024-09-14T22:39:50.003537+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'632","last_scrub_stamp":"2024-09-14T22:39:50.003324+0000","last_deep_scrub":"191'632","last_deep_scrub_stamp":"2024-09-14T22:39:50.003324+0000","last_clean_scrub_stamp":"2024-09-14T22:39:50.003324+0000","objects_scrubbed":33,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:08:16.096949+0000","scrub_duration":59,"objects_trimmed":12,"snaptrim_duration":0.023146856,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":7147,"num_read_kb":35324,"num_write":610,"num_write_kb":15494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":14511090,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":893,"num_object_clones":0,"num_object_copies":1786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":893,"num_whiteouts":889,"num_read":206686,"num_read_kb":828988,"num_write":25214,"num_write_kb":381837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3336,"num_bytes_recovered":173602381,"num_keys_recovered":342,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22322,"internal_metadata":0},"log_size":22466,"ondisk_log_size":22466,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11618,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542196,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59856,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311984,"statfs":{"total":96636764160,"available":96575471616,"internally_reserved":0,"allocated":2359296,"data_stored":1699381,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542198,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29272,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342568,"statfs":{"total":96636764160,"available":96606789632,"internally_reserved":0,"allocated":2301952,"data_stored":1109013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542198,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55644,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316196,"statfs":{"total":96636764160,"available":96579784704,"internally_reserved":0,"allocated":2306048,"data_stored":1109032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":174,"seq":747324309516,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":2252,"kb_used_omap":34,"kb_used_meta":29277,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":2306048,"data_stored":1109032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35258,"internal_metadata":29980230},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542198,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51480,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320360,"statfs":{"total":96636764160,"available":96584048640,"internally_reserved":0,"allocated":2301952,"data_stored":1109013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":66,"seq":283467841572,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43224,"kb_used_data":2248,"kb_used_omap":6,"kb_used_meta":40953,"kb_avail":94328616,"statfs":{"total":96636764160,"available":96592502784,"internally_reserved":0,"allocated":2301952,"data_stored":1109013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7025,"internal_metadata":41936015},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59808,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312032,"statfs":{"total":96636764160,"available":96575520768,"internally_reserved":0,"allocated":2310144,"data_stored":1109051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542197,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41808,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330032,"statfs":{"total":96636764160,"available":96593952768,"internally_reserved":0,"allocated":2359296,"data_stored":1699381,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5809,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4238,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18084,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:40:03.100 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-14T22:40:03.414 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-14T22:40:03.414 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-14T22:40:03.415 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-14T22:40:03.415 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T22:40:03.498 INFO:tasks.ceph.osd.0:Stopped 2024-09-14T22:40:03.498 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-14T22:40:03.498 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T22:40:09.222 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-14T22:40:09.600 INFO:tasks.ceph.osd.1:Stopped 2024-09-14T22:40:09.600 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-14T22:40:09.600 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T22:40:09.679 INFO:tasks.ceph.osd.2:Stopped 2024-09-14T22:40:09.679 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-14T22:40:09.679 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T22:40:09.752 INFO:tasks.ceph.osd.3:Stopped 2024-09-14T22:40:09.752 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-14T22:40:09.752 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T22:40:09.836 INFO:tasks.ceph.osd.4:Stopped 2024-09-14T22:40:09.836 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-14T22:40:09.837 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T22:40:09.937 INFO:tasks.ceph.osd.5:Stopped 2024-09-14T22:40:09.938 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-14T22:40:09.938 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T22:40:10.014 INFO:tasks.ceph.osd.6:Stopped 2024-09-14T22:40:10.015 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-14T22:40:10.015 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T22:40:10.108 INFO:tasks.ceph.osd.7:Stopped 2024-09-14T22:40:10.108 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-14T22:40:10.108 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-14T22:40:10.108 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T22:40:10.136 INFO:tasks.ceph.mgr.y:Stopped 2024-09-14T22:40:10.136 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-14T22:40:10.137 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T22:40:10.167 INFO:tasks.ceph.mgr.x:Stopped 2024-09-14T22:40:10.167 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-14T22:40:10.167 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-14T22:40:10.167 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T22:40:10.199 INFO:tasks.ceph.mon.a:Stopped 2024-09-14T22:40:10.199 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-14T22:40:10.199 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T22:40:10.257 INFO:tasks.ceph.mon.c:Stopped 2024-09-14T22:40:10.258 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-14T22:40:10.258 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T22:40:10.275 INFO:tasks.ceph.mon.b:Stopped 2024-09-14T22:40:10.275 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-14T22:40:10.275 DEBUG:teuthology.orchestra.run.smithi001:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-14T22:40:10.317 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi001.front.sepia.ceph.com 2024-09-14T22:40:10.318 DEBUG:teuthology.orchestra.run.smithi001:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-14T22:40:10.613 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi001.front.sepia.ceph.com 2024-09-14T22:40:10.613 DEBUG:teuthology.orchestra.run.smithi001:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-14T22:40:10.838 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi001.front.sepia.ceph.com 2024-09-14T22:40:10.838 DEBUG:teuthology.orchestra.run.smithi001:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-14T22:40:11.110 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi001.front.sepia.ceph.com 2024-09-14T22:40:11.110 DEBUG:teuthology.orchestra.run.smithi001:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-14T22:40:11.279 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi176.front.sepia.ceph.com 2024-09-14T22:40:11.279 DEBUG:teuthology.orchestra.run.smithi176:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-14T22:40:11.534 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi176.front.sepia.ceph.com 2024-09-14T22:40:11.535 DEBUG:teuthology.orchestra.run.smithi176:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-14T22:40:11.732 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi176.front.sepia.ceph.com 2024-09-14T22:40:11.733 DEBUG:teuthology.orchestra.run.smithi176:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-14T22:40:11.881 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi176.front.sepia.ceph.com 2024-09-14T22:40:11.881 DEBUG:teuthology.orchestra.run.smithi176:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-14T22:40:12.038 INFO:tasks.ceph:Archiving mon data... 2024-09-14T22:40:12.039 DEBUG:teuthology.misc:Transferring archived files from smithi001:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904719/data/mon.a.tgz 2024-09-14T22:40:12.039 DEBUG:teuthology.orchestra.run.smithi001:> mktemp 2024-09-14T22:40:12.045 INFO:teuthology.orchestra.run.smithi001.stdout:/tmp/tmp.vWjP0nymhR 2024-09-14T22:40:12.045 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.vWjP0nymhR 2024-09-14T22:40:12.195 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 0666 /tmp/tmp.vWjP0nymhR 2024-09-14T22:40:12.259 DEBUG:teuthology.orchestra.remote:smithi001:/tmp/tmp.vWjP0nymhR is 444KB 2024-09-14T22:40:12.321 DEBUG:teuthology.orchestra.run.smithi001:> rm -fr /tmp/tmp.vWjP0nymhR 2024-09-14T22:40:12.326 DEBUG:teuthology.misc:Transferring archived files from smithi001:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904719/data/mon.c.tgz 2024-09-14T22:40:12.326 DEBUG:teuthology.orchestra.run.smithi001:> mktemp 2024-09-14T22:40:12.375 INFO:teuthology.orchestra.run.smithi001.stdout:/tmp/tmp.lE6wV9pefe 2024-09-14T22:40:12.375 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.lE6wV9pefe 2024-09-14T22:40:12.565 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 0666 /tmp/tmp.lE6wV9pefe 2024-09-14T22:40:12.631 DEBUG:teuthology.orchestra.remote:smithi001:/tmp/tmp.lE6wV9pefe is 597KB 2024-09-14T22:40:12.693 DEBUG:teuthology.orchestra.run.smithi001:> rm -fr /tmp/tmp.lE6wV9pefe 2024-09-14T22:40:12.698 DEBUG:teuthology.misc:Transferring archived files from smithi176:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904719/data/mon.b.tgz 2024-09-14T22:40:12.698 DEBUG:teuthology.orchestra.run.smithi176:> mktemp 2024-09-14T22:40:12.704 INFO:teuthology.orchestra.run.smithi176.stdout:/tmp/tmp.Nhd3gq6VA1 2024-09-14T22:40:12.704 DEBUG:teuthology.orchestra.run.smithi176:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.Nhd3gq6VA1 2024-09-14T22:40:12.893 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod 0666 /tmp/tmp.Nhd3gq6VA1 2024-09-14T22:40:12.955 DEBUG:teuthology.orchestra.remote:smithi176:/tmp/tmp.Nhd3gq6VA1 is 597KB 2024-09-14T22:40:13.017 DEBUG:teuthology.orchestra.run.smithi176:> rm -fr /tmp/tmp.Nhd3gq6VA1 2024-09-14T22:40:13.022 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-14T22:40:13.022 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-14T22:40:13.025 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-14T22:40:13.085 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-14T22:40:13.127 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-14T22:40:13.128 INFO:tasks.ceph:Archiving crash dumps... 2024-09-14T22:40:13.129 DEBUG:teuthology.misc:Transferring archived files from smithi001:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904719/remote/smithi001/crash 2024-09-14T22:40:13.130 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-14T22:40:13.144 DEBUG:teuthology.misc:Transferring archived files from smithi176:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904719/remote/smithi176/crash 2024-09-14T22:40:13.146 DEBUG:teuthology.orchestra.run.smithi176:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-14T22:40:13.183 INFO:tasks.ceph:Compressing logs... 2024-09-14T22:40:13.183 DEBUG:teuthology.orchestra.run.smithi001:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-14T22:40:13.192 DEBUG:teuthology.orchestra.run.smithi176:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-14T22:40:13.213 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35536.log 2024-09-14T22:40:13.214 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31129.log 2024-09-14T22:40:13.214 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.35536.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35536.log.gz 2024-09-14T22:40:13.214 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30761.log 2024-09-14T22:40:13.215 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.31129.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31129.log.gz 2024-09-14T22:40:13.215 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34770.log 2024-09-14T22:40:13.215 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.30761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34962.log 2024-09-14T22:40:13.215 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30761.log.gz 2024-09-14T22:40:13.217 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.34770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41673.log 2024-09-14T22:40:13.218 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34770.log.gz 2024-09-14T22:40:13.218 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.34962.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34962.log.gz 2024-09-14T22:40:13.218 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38633.log 2024-09-14T22:40:13.218 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41673.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41673.log.gz 2024-09-14T22:40:13.218 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38097.log 2024-09-14T22:40:13.218 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.38633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44198.log 2024-09-14T22:40:13.218 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38633.log.gz 2024-09-14T22:40:13.218 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.38097.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38097.log.gz 2024-09-14T22:40:13.218 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30321.log 2024-09-14T22:40:13.218 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44198.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44198.log.gz 2024-09-14T22:40:13.218 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-14T22:40:13.222 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.30321.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30321.log.gz 2024-09-14T22:40:13.222 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41086.log 2024-09-14T22:40:13.222 INFO:teuthology.orchestra.run.smithi001.stderr:gzip/var/log/ceph/ceph-mgr.y.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38461.log 2024-09-14T22:40:13.222 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35498.log 2024-09-14T22:40:13.222 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41086.log.gz 2024-09-14T22:40:13.222 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.38461.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38461.log.gz 2024-09-14T22:40:13.222 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34871.log 2024-09-14T22:40:13.222 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.35498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35813.log 2024-09-14T22:40:13.222 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35498.log.gz 2024-09-14T22:40:13.223 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.34871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40677.log 2024-09-14T22:40:13.223 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34871.log.gz 2024-09-14T22:40:13.223 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.35813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39800.log 2024-09-14T22:40:13.223 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35813.log.gz 2024-09-14T22:40:13.223 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30059.log 2024-09-14T22:40:13.223 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40677.log.gz 2024-09-14T22:40:13.223 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.39800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42924.log 2024-09-14T22:40:13.223 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39800.log.gz 2024-09-14T22:40:13.223 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.30059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38519.log 2024-09-14T22:40:13.223 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30059.log.gz 2024-09-14T22:40:13.227 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41736.log 2024-09-14T22:40:13.227 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42924.log.gz 2024-09-14T22:40:13.227 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.38519.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38519.log.gz 2024-09-14T22:40:13.227 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35770.log 2024-09-14T22:40:13.227 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41736.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41736.log.gz 2024-09-14T22:40:13.227 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42716.log 2024-09-14T22:40:13.227 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.35770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43592.log 2024-09-14T22:40:13.227 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35770.log.gz 2024-09-14T22:40:13.227 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42716.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42716.log.gz 2024-09-14T22:40:13.228 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37369.log 2024-09-14T22:40:13.228 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43592.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-14T22:40:13.228 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43592.log.gz 2024-09-14T22:40:13.228 INFO:teuthology.orchestra.run.smithi001.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-14T22:40:13.228 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.37369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37465.log 2024-09-14T22:40:13.228 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37369.log.gz 2024-09-14T22:40:13.228 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42037.log 2024-09-14T22:40:13.228 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.37465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35382.log 2024-09-14T22:40:13.228 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37465.log.gz 2024-09-14T22:40:13.228 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42232.log 2024-09-14T22:40:13.228 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42037.log.gz 2024-09-14T22:40:13.228 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.35382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30111.log 2024-09-14T22:40:13.228 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35382.log.gz 2024-09-14T22:40:13.241 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42232.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42232.log.gz 2024-09-14T22:40:13.241 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39258.log 2024-09-14T22:40:13.241 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.30111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42950.log 2024-09-14T22:40:13.241 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30111.log.gz 2024-09-14T22:40:13.242 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.39258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40747.log 2024-09-14T22:40:13.242 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39258.log.gz 2024-09-14T22:40:13.242 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42794.log 2024-09-14T22:40:13.242 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42950.log.gz 2024-09-14T22:40:13.242 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42595.log 2024-09-14T22:40:13.242 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40747.log.gz 2024-09-14T22:40:13.242 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40313.log 2024-09-14T22:40:13.242 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42794.log.gz 2024-09-14T22:40:13.242 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42041.log 2024-09-14T22:40:13.242 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42595.log.gz 2024-09-14T22:40:13.242 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38165.log 2024-09-14T22:40:13.242 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40313.log.gz 2024-09-14T22:40:13.242 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.42041.log: /var/log/ceph/ceph-client.admin.30787.log 2024-09-14T22:40:13.243 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42041.log.gz 2024-09-14T22:40:13.243 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.38165.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.38165.log.gz -5 2024-09-14T22:40:13.243 INFO:teuthology.orchestra.run.smithi001.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42490.log 2024-09-14T22:40:13.243 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.30787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31267.log 2024-09-14T22:40:13.243 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30787.log.gz 2024-09-14T22:40:13.243 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43590.log 2024-09-14T22:40:13.243 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42490.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42490.log.gz 2024-09-14T22:40:13.243 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.31267.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31267.log.gz 2024-09-14T22:40:13.243 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43367.log 2024-09-14T22:40:13.243 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43886.log 2024-09-14T22:40:13.244 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43590.log.gz 2024-09-14T22:40:13.244 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43167.log/var/log/ceph/ceph-client.admin.43367.log: 2024-09-14T22:40:13.244 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43367.log.gz 2024-09-14T22:40:13.244 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38864.log 0.0% 2024-09-14T22:40:13.244 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43886.log.gz 2024-09-14T22:40:13.244 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30508.log 2024-09-14T22:40:13.244 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43167.log.gz 2024-09-14T22:40:13.244 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.38864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39397.log 2024-09-14T22:40:13.244 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38864.log.gz 2024-09-14T22:40:13.244 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.30508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30303.log 2024-09-14T22:40:13.244 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30508.log.gz 2024-09-14T22:40:13.244 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.39397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38783.log 2024-09-14T22:40:13.245 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39397.log.gz 2024-09-14T22:40:13.245 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5/var/log/ceph/ceph-client.admin.30303.log: --verbose -- /var/log/ceph/ceph-client.admin.31348.log 2024-09-14T22:40:13.245 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30303.log.gz 2024-09-14T22:40:13.245 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.38783.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.38816.log -- replaced with /var/log/ceph/ceph-client.admin.38783.log.gz 2024-09-14T22:40:13.245 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-09-14T22:40:13.245 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.31348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42690.log 2024-09-14T22:40:13.245 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31348.log.gz 2024-09-14T22:40:13.245 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.38816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38303.log 2024-09-14T22:40:13.245 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38816.log.gz 2024-09-14T22:40:13.245 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42690.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42690.log.gz 2024-09-14T22:40:13.245 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40887.log 2024-09-14T22:40:13.245 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.38303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30813.log 2024-09-14T22:40:13.246 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38303.log.gz 2024-09-14T22:40:13.246 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44146.log 2024-09-14T22:40:13.246 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40887.log.gz 2024-09-14T22:40:13.246 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.30813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35472.log 2024-09-14T22:40:13.246 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30813.log.gz 2024-09-14T22:40:13.246 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39631.log 2024-09-14T22:40:13.246 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44146.log.gz 2024-09-14T22:40:13.246 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.35472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31088.log 2024-09-14T22:40:13.246 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35472.log.gz 2024-09-14T22:40:13.246 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.39631.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-14T22:40:13.246 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39631.log.gz 2024-09-14T22:40:13.246 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.31088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43181.log 2024-09-14T22:40:13.246 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31088.log.gz 2024-09-14T22:40:13.246 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose/var/log/ceph/ceph-osd.2.log: -- /var/log/ceph/ceph-client.admin.31173.log 2024-09-14T22:40:13.247 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39579.log 2024-09-14T22:40:13.247 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43181.log.gz 2024-09-14T22:40:13.263 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.33444.log 2024-09-14T22:40:13.263 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26309.log 2024-09-14T22:40:13.263 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/objectstore_tool.33444.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-14T22:40:13.263 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.26309.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26309.log.gz 2024-09-14T22:40:13.263 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-14T22:40:13.264 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26333.log 2024-09-14T22:40:13.264 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26453.log 2024-09-14T22:40:13.264 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.26333.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26333.log.gz 2024-09-14T22:40:13.264 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-14T22:40:13.264 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-14T22:40:13.264 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.26453.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26453.log.gz 2024-09-14T22:40:13.264 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-14T22:40:13.264 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26477.log 2024-09-14T22:40:13.264 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-14T22:40:13.264 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.26477.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.34224.log 2024-09-14T22:40:13.264 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26477.log.gz 2024-09-14T22:40:13.264 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26381.log 2024-09-14T22:40:13.265 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/objectstore_tool.34224.log: 85.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-14T22:40:13.265 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26405.log 2024-09-14T22:40:13.265 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.26381.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26381.log.gz 2024-09-14T22:40:13.265 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.32696.log 2024-09-14T22:40:13.265 INFO:teuthology.orchestra.run.smithi176.stderr: 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-14T22:40:13.265 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.26405.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26405.log.gz 2024-09-14T22:40:13.265 INFO:teuthology.orchestra.run.smithi176.stderr: 95.1% -- replaced with /var/log/ceph/objectstore_tool.33444.log.gz 2024-09-14T22:40:13.265 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-14T22:40:13.265 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/objectstore_tool.32696.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-14T22:40:13.265 INFO:teuthology.orchestra.run.smithi176.stderr:gzip/var/log/ceph/ceph-mon.b.log: -5 --verbose -- /var/log/ceph/objectstore_tool.31920.log 2024-09-14T22:40:13.268 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-osd.4.log: 95.0% -- replaced with /var/log/ceph/objectstore_tool.34224.log.gz 2024-09-14T22:40:13.268 INFO:teuthology.orchestra.run.smithi176.stderr: 95.3% -- replaced with /var/log/ceph/objectstore_tool.32696.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26429.log 2024-09-14T22:40:13.268 INFO:teuthology.orchestra.run.smithi176.stderr: 2024-09-14T22:40:13.275 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/objectstore_tool.31920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26357.log 2024-09-14T22:40:13.275 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.26429.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26429.log.gz 2024-09-14T22:40:13.275 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.26357.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26357.log.gz 2024-09-14T22:40:13.275 INFO:teuthology.orchestra.run.smithi176.stderr: 95.1% -- replaced with /var/log/ceph/objectstore_tool.31920.log.gz 2024-09-14T22:40:13.288 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.31173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30293.log 2024-09-14T22:40:13.288 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31173.log.gz 2024-09-14T22:40:13.288 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.39579.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39579.log.gz 2024-09-14T22:40:13.288 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30310.log 2024-09-14T22:40:13.288 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37058.log 2024-09-14T22:40:13.289 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.30293.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30293.log.gz 2024-09-14T22:40:13.289 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.30310.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30310.log.gz 2024-09-14T22:40:13.289 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42872.log 2024-09-14T22:40:13.289 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38248.log 2024-09-14T22:40:13.289 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.37058.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37058.log.gz 2024-09-14T22:40:13.289 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42872.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42872.log.gz 2024-09-14T22:40:13.289 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34679.log 2024-09-14T22:40:13.289 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40253.log 2024-09-14T22:40:13.289 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.38248.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38248.log.gz 2024-09-14T22:40:13.289 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.34679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39707.log 2024-09-14T22:40:13.289 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34679.log.gz 2024-09-14T22:40:13.289 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30935.log 2024-09-14T22:40:13.289 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40253.log.gz 2024-09-14T22:40:13.290 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.39707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30103.log 2024-09-14T22:40:13.290 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39707.log.gz 2024-09-14T22:40:13.290 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.30935.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30935.log.gz 2024-09-14T22:40:13.290 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38055.log 2024-09-14T22:40:13.290 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.30103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43054.log 2024-09-14T22:40:13.290 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30103.log.gz 2024-09-14T22:40:13.290 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.38055.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.38055.log.gz 2024-09-14T22:40:13.290 INFO:teuthology.orchestra.run.smithi001.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.40540.log 2024-09-14T22:40:13.290 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43054.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-14T22:40:13.290 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43054.log.gz 2024-09-14T22:40:13.290 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40540.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40540.log.gz 2024-09-14T22:40:13.290 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30113.log 2024-09-14T22:40:13.290 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30107.log 2024-09-14T22:40:13.291 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.30113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30095.log 2024-09-14T22:40:13.291 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30113.log.gz 2024-09-14T22:40:13.291 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.30107.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30107.log.gz 2024-09-14T22:40:13.291 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33483.log 2024-09-14T22:40:13.291 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34936.log 2024-09-14T22:40:13.291 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.30095.log: 0.0%/var/log/ceph/ceph-client.admin.33483.log: -- replaced with /var/log/ceph/ceph-client.admin.30095.log.gz 2024-09-14T22:40:13.291 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33483.log.gz 2024-09-14T22:40:13.291 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40601.log 2024-09-14T22:40:13.291 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40361.log 2024-09-14T22:40:13.291 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.34936.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34936.log.gz 2024-09-14T22:40:13.291 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35910.log 2024-09-14T22:40:13.291 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40601.log.gz 2024-09-14T22:40:13.291 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40361.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.43912.log 2024-09-14T22:40:13.292 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40361.log.gz 2024-09-14T22:40:13.292 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44380.log 2024-09-14T22:40:13.292 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.35910.log: /var/log/ceph/ceph-client.admin.43912.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35910.log.gz 2024-09-14T22:40:13.292 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43912.log.gz 2024-09-14T22:40:13.292 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37401.log 2024-09-14T22:40:13.292 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38742.log 2024-09-14T22:40:13.292 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44380.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44380.log.gz 2024-09-14T22:40:13.292 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.37401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26439.log 2024-09-14T22:40:13.292 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37401.log.gz 2024-09-14T22:40:13.292 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.38742.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38742.log.gz 2024-09-14T22:40:13.292 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41364.log 2024-09-14T22:40:13.292 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37554.log 2024-09-14T22:40:13.292 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.26439.log: /var/log/ceph/ceph-client.admin.41364.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26439.log.gz 2024-09-14T22:40:13.293 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41364.log.gz 2024-09-14T22:40:13.293 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38030.log 2024-09-14T22:40:13.293 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43028.log 2024-09-14T22:40:13.293 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.37554.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37554.log.gz 2024-09-14T22:40:13.293 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.38030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37013.log 2024-09-14T22:40:13.293 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38030.log.gz 2024-09-14T22:40:13.293 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43028.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43028.log.gz 2024-09-14T22:40:13.293 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38602.log 2024-09-14T22:40:13.293 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41851.log 2024-09-14T22:40:13.293 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.37013.log: /var/log/ceph/ceph-client.admin.38602.log: 27.8% -- replaced with /var/log/ceph/ceph-client.admin.37013.log.gz 2024-09-14T22:40:13.293 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38602.log.gz 2024-09-14T22:40:13.293 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31398.log 2024-09-14T22:40:13.293 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43938.log 2024-09-14T22:40:13.294 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41851.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41851.log.gz 2024-09-14T22:40:13.294 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.31398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41397.log 2024-09-14T22:40:13.294 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31398.log.gz 2024-09-14T22:40:13.294 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43938.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43938.log.gz 2024-09-14T22:40:13.294 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.32289.log 2024-09-14T22:40:13.294 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29987.log 2024-09-14T22:40:13.294 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41397.log: /var/log/ceph/objectstore_tool.32289.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41397.log.gz 2024-09-14T22:40:13.294 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30941.log 2024-09-14T22:40:13.294 INFO:teuthology.orchestra.run.smithi001.stderr:gzip/var/log/ceph/ceph-client.admin.29987.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43574.log 2024-09-14T22:40:13.294 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29987.log.gz 2024-09-14T22:40:13.294 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.30941.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30941.log.gz 2024-09-14T22:40:13.294 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39304.log 2024-09-14T22:40:13.294 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30501.log 2024-09-14T22:40:13.294 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43574.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43574.log.gz 2024-09-14T22:40:13.295 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.39304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40918.log 2024-09-14T22:40:13.295 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39304.log.gz 2024-09-14T22:40:13.295 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.30501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41128.log 2024-09-14T22:40:13.295 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30501.log.gz 2024-09-14T22:40:13.295 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40722.log 2024-09-14T22:40:13.295 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40918.log.gz 2024-09-14T22:40:13.295 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44016.log 2024-09-14T22:40:13.295 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41128.log.gz 2024-09-14T22:40:13.295 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39921.log 2024-09-14T22:40:13.295 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40722.log.gz 2024-09-14T22:40:13.295 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42304.log 2024-09-14T22:40:13.295 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44016.log.gz 2024-09-14T22:40:13.295 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35421.log 2024-09-14T22:40:13.295 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.39921.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39921.log.gz 2024-09-14T22:40:13.296 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42304.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42304.log.gz 2024-09-14T22:40:13.296 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30523.log 2024-09-14T22:40:13.296 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.35421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38941.log 2024-09-14T22:40:13.296 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35421.log.gz 2024-09-14T22:40:13.296 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.30523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43782.log 2024-09-14T22:40:13.296 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30523.log.gz 2024-09-14T22:40:13.296 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.38941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42425.log 2024-09-14T22:40:13.296 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38941.log.gz 2024-09-14T22:40:13.296 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40911.log 2024-09-14T22:40:13.296 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43782.log.gz 2024-09-14T22:40:13.296 INFO:teuthology.orchestra.run.smithi001.stderr: 95.4% -- replaced with /var/log/ceph/objectstore_tool.32289.log.gz 2024-09-14T22:40:13.296 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31490.log 2024-09-14T22:40:13.296 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42425.log.gz 2024-09-14T22:40:13.296 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5/var/log/ceph/ceph-client.admin.40911.log: --verbose -- /var/log/ceph/ceph-client.admin.37769.log 2024-09-14T22:40:13.297 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40911.log.gz 2024-09-14T22:40:13.297 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.31490.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31490.log.gz 2024-09-14T22:40:13.297 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43392.log 2024-09-14T22:40:13.297 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30033.log 2024-09-14T22:40:13.297 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.37769.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37769.log.gz 2024-09-14T22:40:13.297 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30839.log 0.0% 2024-09-14T22:40:13.297 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43392.log.gz 2024-09-14T22:40:13.297 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.30033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34795.log 2024-09-14T22:40:13.297 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30033.log.gz 2024-09-14T22:40:13.297 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.30839.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30839.log.gz 2024-09-14T22:40:13.297 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30313.log 2024-09-14T22:40:13.297 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35600.log 2024-09-14T22:40:13.297 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.34795.log: /var/log/ceph/ceph-client.admin.30313.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34795.log.gz 2024-09-14T22:40:13.298 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40200.log 2024-09-14T22:40:13.298 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30313.log.gz 2024-09-14T22:40:13.298 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.35600.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-14T22:40:13.298 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35600.log.gz 2024-09-14T22:40:13.298 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38979.log 2024-09-14T22:40:13.298 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40200.log.gz 2024-09-14T22:40:13.298 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38556.log 2024-09-14T22:40:13.298 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40279.log 2024-09-14T22:40:13.298 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.38979.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38979.log.gz 2024-09-14T22:40:13.298 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.38556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31455.log 2024-09-14T22:40:13.298 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38556.log.gz 2024-09-14T22:40:13.298 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40279.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40279.log.gz 2024-09-14T22:40:13.298 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39876.log 2024-09-14T22:40:13.299 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30320.log 2024-09-14T22:40:13.299 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.31455.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31455.log.gz 2024-09-14T22:40:13.299 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.39876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37679.log 2024-09-14T22:40:13.299 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39876.log.gz 2024-09-14T22:40:13.299 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.30320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35063.log 0.0% 2024-09-14T22:40:13.299 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30320.log.gz 2024-09-14T22:40:13.299 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41335.log 2024-09-14T22:40:13.299 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.37679.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37679.log.gz 2024-09-14T22:40:13.299 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.35063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26193.log 2024-09-14T22:40:13.299 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35063.log.gz 2024-09-14T22:40:13.299 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41335.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41335.log.gzgzip 2024-09-14T22:40:13.299 INFO:teuthology.orchestra.run.smithi001.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.34895.log 2024-09-14T22:40:13.299 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.26193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41580.log 2024-09-14T22:40:13.299 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26193.log.gz 2024-09-14T22:40:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.34895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35029.log 2024-09-14T22:40:13.300 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34895.log.gz 2024-09-14T22:40:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41580.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.41580.log.gz -- 2024-09-14T22:40:13.300 INFO:teuthology.orchestra.run.smithi001.stderr: /var/log/ceph/ceph-client.admin.40100.log 2024-09-14T22:40:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.35029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35966.log 2024-09-14T22:40:13.300 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35029.log.gz 2024-09-14T22:40:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30101.log 2024-09-14T22:40:13.300 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40100.log.gz 2024-09-14T22:40:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.35966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43184.log 2024-09-14T22:40:13.300 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35966.log.gz 2024-09-14T22:40:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.30101.log: -- /var/log/ceph/ceph-client.admin.43601.log 2024-09-14T22:40:13.300 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30101.log.gz 2024-09-14T22:40:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42820.log 2024-09-14T22:40:13.301 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43184.log.gz 2024-09-14T22:40:13.301 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41954.log 2024-09-14T22:40:13.301 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43601.log.gz 2024-09-14T22:40:13.301 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43593.log 2024-09-14T22:40:13.301 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42820.log.gz 2024-09-14T22:40:13.301 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41954.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-14T22:40:13.301 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41954.log.gz 2024-09-14T22:40:13.301 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40597.log 2024-09-14T22:40:13.301 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43593.log.gz 2024-09-14T22:40:13.301 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44250.log 2024-09-14T22:40:13.301 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.40597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44224.log 2024-09-14T22:40:13.301 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40597.log.gz 2024-09-14T22:40:13.301 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30112.log 2024-09-14T22:40:13.302 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44250.log.gz 2024-09-14T22:40:13.302 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44224.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35281.log 2024-09-14T22:40:13.302 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44224.log.gz 2024-09-14T22:40:13.302 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.30112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40468.log 2024-09-14T22:40:13.302 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30112.log.gz 2024-09-14T22:40:13.302 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.35281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44094.log 2024-09-14T22:40:13.302 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35281.log.gz 2024-09-14T22:40:13.302 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37744.log 2024-09-14T22:40:13.302 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40468.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40468.log.gz 2024-09-14T22:40:13.302 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36520.log 2024-09-14T22:40:13.302 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44094.log: /var/log/ceph/ceph-client.admin.37744.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44094.log.gz 2024-09-14T22:40:13.302 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37744.log.gz 2024-09-14T22:40:13.302 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44431.log 2024-09-14T22:40:13.303 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.36520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42976.log 2024-09-14T22:40:13.303 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36520.log.gz 2024-09-14T22:40:13.303 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41124.log 2024-09-14T22:40:13.303 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44431.log.gz 2024-09-14T22:40:13.303 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39447.log 2024-09-14T22:40:13.303 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42976.log: /var/log/ceph/ceph-client.admin.41124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39254.log 2024-09-14T22:40:13.303 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42976.log.gz 2024-09-14T22:40:13.303 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41124.log.gz 2024-09-14T22:40:13.303 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.39447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35445.log 2024-09-14T22:40:13.303 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39447.log.gz 2024-09-14T22:40:13.303 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30108.log 2024-09-14T22:40:13.303 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.39254.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39254.log.gz 2024-09-14T22:40:13.303 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.35445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43106.log 2024-09-14T22:40:13.304 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35445.log.gz 2024-09-14T22:40:13.304 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41487.log 2024-09-14T22:40:13.304 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.30108.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30108.log.gz 2024-09-14T22:40:13.304 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38210.log 2024-09-14T22:40:13.304 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43106.log.gz 2024-09-14T22:40:13.304 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41487.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41487.log.gz 2024-09-14T22:40:13.304 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43964.log 2024-09-14T22:40:13.304 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.38210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41034.log 2024-09-14T22:40:13.304 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38210.log.gz 2024-09-14T22:40:13.304 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38341.log 2024-09-14T22:40:13.304 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43964.log.gz 2024-09-14T22:40:13.304 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41034.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41034.log.gz 2024-09-14T22:40:13.304 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30525.log 2024-09-14T22:40:13.326 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.38341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32484.log 2024-09-14T22:40:13.326 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38341.log.gz 2024-09-14T22:40:13.326 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.30525.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.30525.log.gz 2024-09-14T22:40:13.326 INFO:teuthology.orchestra.run.smithi001.stderr: /var/log/ceph/ceph-client.admin.42742.log 2024-09-14T22:40:13.326 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.32484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30735.log 2024-09-14T22:40:13.326 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32484.log.gz 2024-09-14T22:40:13.326 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42742.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42742.log.gz 2024-09-14T22:40:13.326 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40354.log 2024-09-14T22:40:13.327 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.30735.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.33074.log 2024-09-14T22:40:13.327 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30735.log.gz 2024-09-14T22:40:13.327 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35102.log 2024-09-14T22:40:13.327 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40354.log.gz 2024-09-14T22:40:13.327 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37837.log 2024-09-14T22:40:13.327 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/objectstore_tool.33074.log: /var/log/ceph/ceph-client.admin.35102.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35102.log.gz 2024-09-14T22:40:13.327 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35158.log 2024-09-14T22:40:13.327 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.37837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39020.log 2024-09-14T22:40:13.327 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37837.log.gz 2024-09-14T22:40:13.327 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.35158.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35158.log.gz 2024-09-14T22:40:13.327 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37588.log 2024-09-14T22:40:13.327 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38812.log 2024-09-14T22:40:13.327 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.39020.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39020.log.gz 2024-09-14T22:40:13.327 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.37588.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41643.log 2024-09-14T22:40:13.328 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37588.log.gz 2024-09-14T22:40:13.328 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.38812.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38812.log.gz 2024-09-14T22:40:13.328 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35625.log 2024-09-14T22:40:13.328 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36524.log 2024-09-14T22:40:13.328 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41643.log: 95.4% 0.0% -- replaced with /var/log/ceph/objectstore_tool.33074.log.gz -- replaced with /var/log/ceph/ceph-client.admin.41643.log.gz 2024-09-14T22:40:13.328 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-09-14T22:40:13.328 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.35625.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35625.log.gz 2024-09-14T22:40:13.328 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40069.log 2024-09-14T22:40:13.328 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30299.log 2024-09-14T22:40:13.328 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.36524.log: /var/log/ceph/ceph-client.admin.40069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37469.log 2024-09-14T22:40:13.328 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36524.log.gz 2024-09-14T22:40:13.328 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40069.log.gz 2024-09-14T22:40:13.328 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.30299.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30299.log.gz 2024-09-14T22:40:13.328 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37120.log 2024-09-14T22:40:13.329 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37161.log 2024-09-14T22:40:13.329 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.37469.log: /var/log/ceph/ceph-client.admin.37120.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-14T22:40:13.329 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37469.log.gz 2024-09-14T22:40:13.329 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37120.log.gz/var/log/ceph/ceph-client.admin.37161.log: 2024-09-14T22:40:13.329 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.37161.log.gz -5 2024-09-14T22:40:13.329 INFO:teuthology.orchestra.run.smithi001.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43179.log 2024-09-14T22:40:13.329 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30007.log 2024-09-14T22:40:13.329 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.43179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38714.log 2024-09-14T22:40:13.329 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43179.log.gz 2024-09-14T22:40:13.329 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.30007.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30007.log.gz 2024-09-14T22:40:13.329 INFO:teuthology.orchestra.run.smithi001.stderr: 92.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-14T22:40:13.329 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39212.log 2024-09-14T22:40:13.329 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.38714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42330.log 2024-09-14T22:40:13.330 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38714.log.gz 2024-09-14T22:40:13.330 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.39212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37887.log 2024-09-14T22:40:13.330 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39212.log.gz 2024-09-14T22:40:13.330 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44407.log 2024-09-14T22:40:13.330 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42330.log: /var/log/ceph/ceph-client.admin.37887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38101.log 2024-09-14T22:40:13.330 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42330.log.gz 2024-09-14T22:40:13.330 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37887.log.gz/var/log/ceph/ceph-client.admin.44407.log: 2024-09-14T22:40:13.330 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44407.log.gz 2024-09-14T22:40:13.330 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38429.log 2024-09-14T22:40:13.330 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40463.log 2024-09-14T22:40:13.330 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.38101.log: /var/log/ceph/ceph-client.admin.38429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36075.log 2024-09-14T22:40:13.330 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38101.log.gz 2024-09-14T22:40:13.330 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38429.log.gz 2024-09-14T22:40:13.330 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40463.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40463.log.gz 2024-09-14T22:40:13.331 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30524.log 2024-09-14T22:40:13.331 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38392.log 2024-09-14T22:40:13.331 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.36075.log: /var/log/ceph/ceph-client.admin.30524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37343.log 2024-09-14T22:40:13.331 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36075.log.gz 2024-09-14T22:40:13.331 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30524.log.gz 2024-09-14T22:40:13.331 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.38392.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.38392.log.gz 2024-09-14T22:40:13.331 INFO:teuthology.orchestra.run.smithi001.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.31231.log 2024-09-14T22:40:13.331 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37629.log 2024-09-14T22:40:13.331 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.37343.log: /var/log/ceph/ceph-client.admin.31231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43382.log 2024-09-14T22:40:13.331 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31231.log.gz 2024-09-14T22:40:13.331 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37343.log.gz 2024-09-14T22:40:13.331 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.37629.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37629.log.gz 2024-09-14T22:40:13.331 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30895.log 2024-09-14T22:40:13.331 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35186.log 2024-09-14T22:40:13.332 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43382.log: /var/log/ceph/ceph-client.admin.30895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43132.log 2024-09-14T22:40:13.332 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30895.log.gz 2024-09-14T22:40:13.332 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43382.log.gz 2024-09-14T22:40:13.332 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.35186.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35186.log.gz 2024-09-14T22:40:13.332 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-14T22:40:13.332 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39928.log 2024-09-14T22:40:13.332 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43132.log: /var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37199.log 2024-09-14T22:40:13.332 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43132.log.gz 2024-09-14T22:40:13.333 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.39928.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41268.log 2024-09-14T22:40:13.333 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39928.log.gz 2024-09-14T22:40:13.333 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.37199.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37199.log.gz 2024-09-14T22:40:13.333 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43186.log 2024-09-14T22:40:13.333 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44042.log 2024-09-14T22:40:13.333 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41268.log.gz 2024-09-14T22:40:13.334 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43186.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43186.log.gz 2024-09-14T22:40:13.334 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30865.log 2024-09-14T22:40:13.334 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43594.log 2024-09-14T22:40:13.334 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44042.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44042.log.gz 2024-09-14T22:40:13.334 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.30865.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30865.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43602.log 2024-09-14T22:40:13.334 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-09-14T22:40:13.335 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43594.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43594.log.gz 2024-09-14T22:40:13.335 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42659.log 2024-09-14T22:40:13.335 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31299.log 2024-09-14T22:40:13.335 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43602.log: /var/log/ceph/ceph-client.admin.42659.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43602.log.gz 2024-09-14T22:40:13.335 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42659.log.gz 2024-09-14T22:40:13.335 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39488.log 2024-09-14T22:40:13.335 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40977.log 2024-09-14T22:40:13.336 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.31299.log: /var/log/ceph/ceph-client.admin.39488.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31299.log.gz 2024-09-14T22:40:13.336 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39488.log.gz 2024-09-14T22:40:13.336 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30709.log 2024-09-14T22:40:13.336 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43080.log 2024-09-14T22:40:13.336 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40977.log: /var/log/ceph/ceph-client.admin.30709.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40977.log.gz 2024-09-14T22:40:13.336 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30709.log.gz 2024-09-14T22:40:13.336 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41798.log 2024-09-14T22:40:13.336 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44328.log 2024-09-14T22:40:13.337 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43080.log: /var/log/ceph/ceph-client.admin.41798.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43080.log.gz 2024-09-14T22:40:13.337 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41798.log.gz 2024-09-14T22:40:13.337 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41297.log 2024-09-14T22:40:13.337 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39183.log 2024-09-14T22:40:13.337 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44328.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44328.log.gz 2024-09-14T22:40:13.337 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41297.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41297.log.gzgzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-14T22:40:13.338 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-09-14T22:40:13.338 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.39183.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39183.log.gz 2024-09-14T22:40:13.338 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41877.log 2024-09-14T22:40:13.338 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42396.log 2024-09-14T22:40:13.338 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.41877.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41877.log.gz 2024-09-14T22:40:13.353 INFO:teuthology.orchestra.run.smithi176.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-14T22:40:13.354 INFO:teuthology.orchestra.run.smithi001.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-14T22:40:13.355 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44302.log 2024-09-14T22:40:13.355 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43587.log 2024-09-14T22:40:13.355 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42396.log: /var/log/ceph/ceph-client.admin.44302.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42396.log.gz 2024-09-14T22:40:13.355 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44302.log.gz 2024-09-14T22:40:13.355 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39133.log 2024-09-14T22:40:13.355 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43860.log 2024-09-14T22:40:13.355 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43587.log: /var/log/ceph/ceph-client.admin.39133.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43587.log.gz 2024-09-14T22:40:13.355 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39133.log.gz 2024-09-14T22:40:13.355 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42537.log 2024-09-14T22:40:13.355 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39993.log 2024-09-14T22:40:13.356 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43860.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43860.log.gz 2024-09-14T22:40:13.356 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42537.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42537.log.gzgzip 2024-09-14T22:40:13.356 INFO:teuthology.orchestra.run.smithi001.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.35128.log 2024-09-14T22:40:13.356 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.39993.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39993.log.gz 2024-09-14T22:40:13.356 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.31459.log 2024-09-14T22:40:13.356 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39842.log 2024-09-14T22:40:13.356 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.35128.log: /var/log/ceph/objectstore_tool.31459.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35128.log.gz 2024-09-14T22:40:13.356 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29875.log 2024-09-14T22:40:13.356 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.39842.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39842.log.gz 2024-09-14T22:40:13.356 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42846.log 2024-09-14T22:40:13.356 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.29875.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29875.log.gz 2024-09-14T22:40:13.356 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37437.log 2024-09-14T22:40:13.356 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42846.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42846.log.gz 2024-09-14T22:40:13.357 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43834.log 2024-09-14T22:40:13.357 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.37437.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37437.log.gz 2024-09-14T22:40:13.357 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37891.log 2024-09-14T22:40:13.357 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43834.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43834.log.gz 2024-09-14T22:40:13.357 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43381.log 2024-09-14T22:40:13.357 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.37891.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37891.log.gz 2024-09-14T22:40:13.357 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35307.log 2024-09-14T22:40:13.357 INFO:teuthology.orchestra.run.smithi001.stderr: 95.2% -- replaced with /var/log/ceph/objectstore_tool.31459.log.gz 2024-09-14T22:40:13.357 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43381.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43381.log.gz 2024-09-14T22:40:13.357 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41008.log 2024-09-14T22:40:13.357 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39681.log 2024-09-14T22:40:13.357 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.35307.log: /var/log/ceph/ceph-client.admin.41008.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35307.log.gz 2024-09-14T22:40:13.358 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41008.log.gz 2024-09-14T22:40:13.358 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40045.log 2024-09-14T22:40:13.358 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34259.log 2024-09-14T22:40:13.358 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.39681.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39681.log.gz 2024-09-14T22:40:13.358 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40045.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40045.log.gz 2024-09-14T22:40:13.358 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30509.log 2024-09-14T22:40:13.358 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36049.log 2024-09-14T22:40:13.358 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.34259.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34259.log.gz 2024-09-14T22:40:13.358 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.30509.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30509.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42768.log 2024-09-14T22:40:13.358 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-09-14T22:40:13.358 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.36049.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36049.log.gz 2024-09-14T22:40:13.358 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38004.log 2024-09-14T22:40:13.358 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37939.log 2024-09-14T22:40:13.359 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42768.log: 27.5% -- replaced with /var/log/ceph/ceph-client.admin.42768.log.gz 2024-09-14T22:40:13.359 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.38004.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38004.log.gz 2024-09-14T22:40:13.359 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29937.log 2024-09-14T22:40:13.359 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.37939.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37939.log.gz 2024-09-14T22:40:13.359 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39714.log 2024-09-14T22:40:13.359 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30531.log 2024-09-14T22:40:13.359 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.29937.log: /var/log/ceph/ceph-client.admin.39714.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29937.log.gz 2024-09-14T22:40:13.359 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39714.log.gz 2024-09-14T22:40:13.359 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39532.log 2024-09-14T22:40:13.359 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41552.log 2024-09-14T22:40:13.359 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.30531.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30531.log.gz 2024-09-14T22:40:13.359 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.39532.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39532.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37783.log 2024-09-14T22:40:13.360 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-09-14T22:40:13.360 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41552.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41552.log.gz 2024-09-14T22:40:13.360 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43002.log 2024-09-14T22:40:13.360 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35220.log 2024-09-14T22:40:13.360 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.37783.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37783.log.gz/var/log/ceph/ceph-client.admin.43002.log: 2024-09-14T22:40:13.360 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43002.log.gz 2024-09-14T22:40:13.360 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42898.log 2024-09-14T22:40:13.360 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39512.log 2024-09-14T22:40:13.360 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.35220.log: /var/log/ceph/ceph-client.admin.42898.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35220.log.gz 2024-09-14T22:40:13.360 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42898.log.gz 2024-09-14T22:40:13.360 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43374.log 2024-09-14T22:40:13.360 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39069.log 2024-09-14T22:40:13.360 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.39512.log: /var/log/ceph/ceph-client.admin.43374.log: 27.4% -- replaced with /var/log/ceph/ceph-client.admin.39512.log.gz 2024-09-14T22:40:13.361 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43374.log.gz 2024-09-14T22:40:13.361 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.33875.log 2024-09-14T22:40:13.361 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44354.log 2024-09-14T22:40:13.361 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.39069.log: /var/log/ceph/objectstore_tool.33875.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39069.log.gz 2024-09-14T22:40:13.361 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.30928.log 2024-09-14T22:40:13.366 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44354.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44354.log.gz 2024-09-14T22:40:13.366 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41461.log 2024-09-14T22:40:13.366 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.0.30928.log: 95.3% -- replaced with /var/log/ceph/ceph-client.0.30928.log.gz 2024-09-14T22:40:13.366 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34727.log 2024-09-14T22:40:13.366 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41461.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41461.log.gz 2024-09-14T22:40:13.366 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39715.log 2024-09-14T22:40:13.367 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.34727.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34727.log.gz 2024-09-14T22:40:13.367 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35689.log 2024-09-14T22:40:13.367 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.39715.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39715.log.gz 2024-09-14T22:40:13.367 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30522.log 2024-09-14T22:40:13.367 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.35689.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35689.log.gz 2024-09-14T22:40:13.367 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35697.log 2024-09-14T22:40:13.367 INFO:teuthology.orchestra.run.smithi001.stderr: 95.1% -- replaced with /var/log/ceph/objectstore_tool.33875.log.gz 2024-09-14T22:40:13.367 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.30522.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30522.log.gz 2024-09-14T22:40:13.367 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40146.log 2024-09-14T22:40:13.367 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41994.log 2024-09-14T22:40:13.367 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.35697.log: /var/log/ceph/ceph-client.admin.40146.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40146.log.gz -- replaced with /var/log/ceph/ceph-client.admin.35697.log.gz 2024-09-14T22:40:13.367 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-09-14T22:40:13.367 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43384.log 2024-09-14T22:40:13.367 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40019.log 2024-09-14T22:40:13.367 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41994.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41994.log.gz 2024-09-14T22:40:13.368 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43384.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43384.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35871.log 2024-09-14T22:40:13.368 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-09-14T22:40:13.368 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40019.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40019.log.gz 2024-09-14T22:40:13.368 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43394.log 2024-09-14T22:40:13.368 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44120.log 2024-09-14T22:40:13.368 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.35871.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35871.log.gz 2024-09-14T22:40:13.372 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43394.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43394.log.gz 2024-09-14T22:40:13.372 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.21399.log 2024-09-14T22:40:13.372 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41672.log 2024-09-14T22:40:13.372 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44120.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44120.log.gz 2024-09-14T22:40:13.372 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph.tmp-client.admin.21399.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.21399.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42191.log 2024-09-14T22:40:13.372 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-09-14T22:40:13.372 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41672.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41672.log.gz 2024-09-14T22:40:13.372 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26340.log 2024-09-14T22:40:13.372 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43169.log 2024-09-14T22:40:13.372 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42191.log: /var/log/ceph/ceph-client.admin.26340.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42191.log.gz 2024-09-14T22:40:13.373 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26340.log.gz 2024-09-14T22:40:13.373 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30309.log 2024-09-14T22:40:13.373 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38367.log 2024-09-14T22:40:13.373 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43169.log: /var/log/ceph/ceph-client.admin.30309.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43169.log.gz 2024-09-14T22:40:13.373 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30309.log.gz 2024-09-14T22:40:13.373 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41227.log 2024-09-14T22:40:13.373 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40305.log 2024-09-14T22:40:13.373 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.38367.log: /var/log/ceph/ceph-client.admin.41227.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38367.log.gz 2024-09-14T22:40:13.373 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41227.log.gz 2024-09-14T22:40:13.373 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37276.log 2024-09-14T22:40:13.376 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44068.log 2024-09-14T22:40:13.376 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40305.log: /var/log/ceph/ceph-client.admin.37276.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40305.log.gz 2024-09-14T22:40:13.376 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37276.log.gz 2024-09-14T22:40:13.376 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43990.log 2024-09-14T22:40:13.376 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37301.log 2024-09-14T22:40:13.376 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44068.log: /var/log/ceph/ceph-client.admin.43990.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44068.log.gz 2024-09-14T22:40:13.376 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43990.log.gz 2024-09-14T22:40:13.376 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43808.log 2024-09-14T22:40:13.377 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44172.log 2024-09-14T22:40:13.377 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.37301.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37301.log.gz 2024-09-14T22:40:13.377 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43808.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43180.log 2024-09-14T22:40:13.377 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43808.log.gz 2024-09-14T22:40:13.377 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44172.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44172.log.gz 2024-09-14T22:40:13.377 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42152.log 2024-09-14T22:40:13.379 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39340.log 2024-09-14T22:40:13.379 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43180.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43180.log.gz 2024-09-14T22:40:13.379 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42152.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42152.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43386.log 2024-09-14T22:40:13.379 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-09-14T22:40:13.380 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.39340.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39340.log.gz 2024-09-14T22:40:13.380 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29962.log 2024-09-14T22:40:13.380 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43183.log 2024-09-14T22:40:13.380 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43386.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43386.log.gz 2024-09-14T22:40:13.380 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.29962.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29962.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42633.log 2024-09-14T22:40:13.380 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-09-14T22:40:13.380 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43183.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43183.log.gz 2024-09-14T22:40:13.382 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41060.log 2024-09-14T22:40:13.382 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44276.log 2024-09-14T22:40:13.382 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42633.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42633.log.gz 2024-09-14T22:40:13.382 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41060.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41060.log.gzgzip 2024-09-14T22:40:13.382 INFO:teuthology.orchestra.run.smithi001.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.38290.log 2024-09-14T22:40:13.382 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44276.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44276.log.gz 2024-09-14T22:40:13.382 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40837.log 2024-09-14T22:40:13.383 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40833.log 2024-09-14T22:40:13.384 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.38290.log: /var/log/ceph/ceph-client.admin.40837.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38290.log.gz 2024-09-14T22:40:13.384 INFO:teuthology.orchestra.run.smithi001.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40837.log.gz 2024-09-14T22:40:13.384 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30934.log 2024-09-14T22:40:13.384 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39044.log 2024-09-14T22:40:13.384 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40833.log: /var/log/ceph/ceph-client.admin.30934.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30934.log.gz -- replaced with /var/log/ceph/ceph-client.admin.40833.log.gz 2024-09-14T22:40:13.384 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-09-14T22:40:13.384 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.39044.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39044.log.gz 2024-09-14T22:40:13.433 INFO:teuthology.orchestra.run.smithi176.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-14T22:40:13.501 INFO:teuthology.orchestra.run.smithi001.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-14T22:40:13.622 INFO:teuthology.orchestra.run.smithi001.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-14T22:40:14.224 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-14T22:40:14.433 INFO:teuthology.orchestra.run.smithi001.stderr: 94.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-14T22:40:14.593 INFO:teuthology.orchestra.run.smithi176.stderr: 95.2% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-14T22:40:15.754 INFO:teuthology.orchestra.run.smithi001.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-14T22:40:16.030 INFO:teuthology.orchestra.run.smithi001.stderr: 95.5% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-14T22:40:16.415 INFO:teuthology.orchestra.run.smithi176.stderr: 95.3% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-14T22:40:16.687 INFO:teuthology.orchestra.run.smithi001.stderr: 95.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-14T22:40:16.688 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-09-14T22:40:16.688 INFO:teuthology.orchestra.run.smithi001.stderr:real 0m3.494s 2024-09-14T22:40:16.688 INFO:teuthology.orchestra.run.smithi001.stderr:user 0m10.366s 2024-09-14T22:40:16.688 INFO:teuthology.orchestra.run.smithi001.stderr:sys 0m0.467s 2024-09-14T22:40:16.693 INFO:teuthology.orchestra.run.smithi176.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-14T22:40:16.977 INFO:teuthology.orchestra.run.smithi176.stderr: 95.5% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-14T22:40:16.979 INFO:teuthology.orchestra.run.smithi176.stderr: 2024-09-14T22:40:16.979 INFO:teuthology.orchestra.run.smithi176.stderr:real 0m3.747s 2024-09-14T22:40:16.979 INFO:teuthology.orchestra.run.smithi176.stderr:user 0m11.578s 2024-09-14T22:40:16.979 INFO:teuthology.orchestra.run.smithi176.stderr:sys 0m0.438s 2024-09-14T22:40:16.980 INFO:tasks.ceph:Archiving logs... 2024-09-14T22:40:16.980 DEBUG:teuthology.misc:Transferring archived files from smithi001:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904719/remote/smithi001/log 2024-09-14T22:40:16.981 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-14T22:40:18.176 DEBUG:teuthology.misc:Transferring archived files from smithi176:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904719/remote/smithi176/log 2024-09-14T22:40:18.177 DEBUG:teuthology.orchestra.run.smithi176:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-14T22:40:19.353 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-14T22:40:19.361 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-14T22:40:19.361 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-14T22:40:19.364 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-14T22:40:19.444 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-14T22:40:19.453 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-14T22:40:19.453 DEBUG:teuthology.orchestra.run.smithi001:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T22:40:19.455 DEBUG:teuthology.orchestra.run.smithi176:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T22:40:19.468 INFO:teuthology.orchestra.run.smithi176.stdout: remote refid st t when poll reach delay offset jitter 2024-09-14T22:40:19.468 INFO:teuthology.orchestra.run.smithi176.stdout:============================================================================== 2024-09-14T22:40:19.468 INFO:teuthology.orchestra.run.smithi176.stdout:+hv01.front.sepi 67.205.162.81 3 u 21 64 177 0.090 -0.553 0.389 2024-09-14T22:40:19.468 INFO:teuthology.orchestra.run.smithi176.stdout:*hv02.front.sepi 63.231.80.2 3 u 26 64 177 0.073 -0.160 0.161 2024-09-14T22:40:19.468 INFO:teuthology.orchestra.run.smithi176.stdout:+hv03.front.sepi 74.6.168.72 3 u 21 64 177 0.099 +2.693 0.170 2024-09-14T22:40:19.468 INFO:teuthology.orchestra.run.smithi176.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T22:40:19.642 INFO:teuthology.orchestra.run.smithi001.stdout: remote refid st t when poll reach delay offset jitter 2024-09-14T22:40:19.642 INFO:teuthology.orchestra.run.smithi001.stdout:============================================================================== 2024-09-14T22:40:19.642 INFO:teuthology.orchestra.run.smithi001.stdout:+hv01.front.sepi 67.205.162.81 3 u 27 64 177 0.074 -1.327 0.902 2024-09-14T22:40:19.642 INFO:teuthology.orchestra.run.smithi001.stdout:*hv02.front.sepi 63.231.80.2 3 u 29 64 177 0.139 -0.342 1.225 2024-09-14T22:40:19.642 INFO:teuthology.orchestra.run.smithi001.stdout:+hv03.front.sepi 74.6.168.72 3 u 25 64 177 0.078 +0.038 2.035 2024-09-14T22:40:19.642 INFO:teuthology.orchestra.run.smithi001.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T22:40:19.643 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-14T22:40:19.652 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-14T22:40:19.652 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-14T22:40:19.660 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-14T22:40:19.669 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-14T22:40:19.679 INFO:teuthology.task.internal:Duration was 883.335364 seconds 2024-09-14T22:40:19.679 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-14T22:40:19.687 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-14T22:40:19.688 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-14T22:40:19.690 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-14T22:40:19.729 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-14T22:40:19.729 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi001.front.sepia.ceph.com 2024-09-14T22:40:19.729 DEBUG:teuthology.orchestra.run.smithi001:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-14T22:40:19.780 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi176.front.sepia.ceph.com 2024-09-14T22:40:19.780 DEBUG:teuthology.orchestra.run.smithi176:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-14T22:40:19.790 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-14T22:40:19.790 DEBUG:teuthology.orchestra.run.smithi001:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-14T22:40:19.824 DEBUG:teuthology.orchestra.run.smithi176:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-14T22:40:19.862 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-14T22:40:19.862 DEBUG:teuthology.orchestra.run.smithi001:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-14T22:40:19.896 DEBUG:teuthology.orchestra.run.smithi176:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-14T22:40:19.977 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-14T22:40:19.986 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-14T22:40:19.986 DEBUG:teuthology.orchestra.run.smithi001:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-14T22:40:19.998 DEBUG:teuthology.orchestra.run.smithi176:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-14T22:40:20.009 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-14T22:40:20.018 DEBUG:teuthology.orchestra.run.smithi001:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-14T22:40:20.040 DEBUG:teuthology.orchestra.run.smithi176:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-14T22:40:20.048 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern = core 2024-09-14T22:40:20.061 INFO:teuthology.orchestra.run.smithi176.stdout:kernel.core_pattern = core 2024-09-14T22:40:20.077 DEBUG:teuthology.orchestra.run.smithi001:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-14T22:40:20.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T22:40:20.114 DEBUG:teuthology.orchestra.run.smithi176:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-14T22:40:20.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T22:40:20.123 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-14T22:40:20.132 INFO:teuthology.task.internal:Transferring archived files... 2024-09-14T22:40:20.133 DEBUG:teuthology.misc:Transferring archived files from smithi001:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904719/remote/smithi001 2024-09-14T22:40:20.133 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-14T22:40:20.184 DEBUG:teuthology.misc:Transferring archived files from smithi176:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904719/remote/smithi176 2024-09-14T22:40:20.185 DEBUG:teuthology.orchestra.run.smithi176:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-14T22:40:20.206 INFO:teuthology.task.internal:Removing archive directory... 2024-09-14T22:40:20.206 DEBUG:teuthology.orchestra.run.smithi001:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-14T22:40:20.220 DEBUG:teuthology.orchestra.run.smithi176:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-14T22:40:20.251 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-14T22:40:20.262 INFO:teuthology.task.internal:Not uploading archives. 2024-09-14T22:40:20.262 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-14T22:40:20.272 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-14T22:40:20.272 DEBUG:teuthology.orchestra.run.smithi001:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-14T22:40:20.274 DEBUG:teuthology.orchestra.run.smithi176:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-14T22:40:20.278 INFO:teuthology.orchestra.run.smithi001.stdout: 658616 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 14 22:40 /home/ubuntu/cephtest 2024-09-14T22:40:20.299 INFO:teuthology.orchestra.run.smithi176.stdout: 658634 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 14 22:40 /home/ubuntu/cephtest 2024-09-14T22:40:20.301 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-14T22:40:20.310 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-14T22:40:20.363 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writearound} duration: 883.3353638648987 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-14T22:40:20.363 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-14T22:40:20.441 INFO:teuthology.run:pass